Bản quyền thuộc về QRS-website: qr-solutions.com.

vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 1
 Quality Resources & Solutions

Tài liệu tóm tắtôn thi chứng

chỉ quốc tế ISTQB

Giảng viên: Tạ Thị Thinh

Email: [email protected]

SDT: 0986775464

Skype: ta.thinh0204

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 2
CHAPTER 0: ISTQB introduction 5
1. ISTQB Foundation Level Examination 5
1.1 ISTQB Course structure (Cấu trúc ISTQB) 5
1.2 Who is ISTQB (ISTQB là gì)? 6
1.3 Exams in ISTQB (Bài thi trong ISTQB cập nhật phiên bản 2018) 6
2. How to study ISTQB and prepare for exam 6
2.1 How to read and use ISTQB books 6
2.2 Foundation Level examination – Bài thi ở mức cơ sở 6
2.3 Plan for examination -Lập kế hoạch cho kỳ thi 7
Chapter 1: Fundamentals of Testing 8
1.1 What is Testing? - Testing là gì 8
1.1.1 Typical Objectives of Testing 8
1.1.2 Testing and Debugging 8
1.2 Why is Testing Necessary? 8
1.2.1 Testing’s Contributions to Success 8
1.2.2 Quality Assurance and Testing 9
1.2.3 Errors, Defects, and Failures 11
1.2.4 Defects, Root Causes and Effects 11
1.3 Seven Principles of Testing – 7 nguyên lý cơ bản của testing 12
1.4 Test Process 12
1.4.1 Test Process in Context 12
1.4.2 Test Activities and Tasks 13
1.4.4 Traceability between the Test Basis and Test Work Products 16
1.5 The Psychology of Testing 16
1.5.1 Human Psychology and Testing 16
1.5.2 Tester’s and Developer’s Mindsets 17
Chapter 2: Testing Throughout the Software Development Lifecycle 18
2.1 Software Development Lifecycle Models 18
2.1.2 Software Development Lifecycle Models in Context 20

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 3
2.2 Test Levels (K2) 20
2.3 Test Types 25
2.3.1 Functional Testing 25
2.3.2 Non-functional Testing 25
2.3.3 White-box Testing 25
2.3.4 Change-related Testing 26
2.3.5 Test Types and Test Levels 27
2.4 Maintenance Testing 27
2.4.1 Triggers for Maintenance 28
2.4.2 Impact Analysis for Maintenance 28
CHAPTER 3 STATIC TESTING 29
3.1 Static Testing Basics 29
3.1.1 Work Products that Can Be Examined by Static Testing 29
3.1.2 Benefits of Static Testing 29
3.1.3 Differences between Static and Dynamic Testing 29
3.2 Review Process 30
3.2.1 Work Product Review Process 30
3.2.2 Roles and responsibilities in a formal review 30
3.2.3 Review Types 31
3.2.4 Applying Review Techniques 32
3.2.5 Success Factors for Reviews 34
CHAPTER 4: Test Design Techniques 35
4.1 Categories of Test Techniques 35
4.1.1 Choosing Test Techniques 35
4.1.2 Categories of Test Techniques and Their Characteristics 35
4.2 Black-box Test Techniques 36
4.2.1 Equivalence partitioning (EP) - Phân vùng tương đương (EP) 36
4.2.2 Boundary value analysis (BVA)- Phân tích giá trị biên (BVA) 36
4.2.3 Decision tables - bảng quyết định 37

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 4
4.2.4 State transition testing - Test chuyển đổi trạng thái 37
4.3 While Box test design 37
4.3.1 Statement coverage 38
4.3.2 Decision coverage (Branch coverage) 38
4.3.3 Path coverage 38
4.4 Experience_base techniques (K2) 39
4.4.1 Error Guessing 39
4.4.2 Exploratory Testing 40
4.4.3 Checklist-based Testing 40
Chapter 5: Test Management 41
5.1 Test Organization 41
5.1.2 Tasks of a Test Manager and Tester 41
5.2 Test Planning and Estimation 43
5.2.1 Purpose and Content of a Test Plan 43
5.2.2 Test Strategy and Test Approach 43
5.2.3 Entry Criteria and Exit Criteria (Definition of Ready and Definition of Done) 43
5.2.4 Test Execution Schedule - Lịch trình thực hiện 44
5.2.5 Factors Influencing the Test Effort 44
5.3 Test Monitoring and Control 45
5.3.1 Metrics Used in Testing 45
5.3.2 Purposes, Contents, and Audiences for Test Reports 46
5.4 Configuration Management 46
5.5 Risks and Testing 47
5.5.1 Definition of Risk 47
5.5.2 Product and Project Risks 47
5.5.3 Risk-based Testing and Product Quality 47
5.6 Defect Management 48
Chapter 6: Tool Support for Testing 49
6.1 Test Tool Considerations 49

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 5
6.1.1 Test Tool Classification 49
6.1.2 Benefits and Risks of Test Automation 50
6.1.3 Special Considerations for Test Execution and Test Management Tools 50
6.2 Effective Use of Tools 51
6.2.1 Main Principles for Tool Selection 51
6.2.2 Pilot Projects for Introducing a Tool into an Organization 51
6.2.3 Success Factors for Tools 51

CHAPTER 0: ISTQB introduction

Nội dung chính:
1. ISTQB Foundation Level Examination
- Giới thiệu về bài thi ISTQB
- Who is ISTQB -ISTQB là gì?
- Course structure -Exams in ISTQB
- About Foundation level Examination -Về các mức độ kiến thức trong bài thi
2. How to study ISTQB and prepare for exam -
Plan for examination- Lập kế hoạch cho kỳ thi
- How to read and use ISTQB books- Cách đọc và sử dụng các cuốn sách ISTQB bằng tiếng Anh
- Tips for doing exercises-Các chỉ dẫn, kinh nghiệm để làm bài tập

1. ISTQB Foundation Level Examination

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 6
1.1 ISTQB Course structure (Cấu trúc ISTQB)

- There are currently three levels of certification Có 3 mức độ của chứng chỉ o The Foundation
Level certification (one module) - CTFL (Certified Tester Foundation level
Chứng chỉ ở mức cơ bản (1 mảng) o Advanced
Level certifications (three modules)
Chứng chỉ ở cao hơn (3 mảng: quản lý, kỹ thuật test và
automation) o An Expert levels is currently being developed
Chứng chỉ ở mức chuyên gia đang được xây dựng
- For all three levels, international working groups develop and maintain internationally uniform
curricula and exams - Đối với tất cả 3 mức chứng chỉ, thì đều thống nhất tài liệu giảng và bài thi
trên toàn thế giới.
1.2 Who is ISTQB (ISTQB là gì)?
- ISTQB là 1 tổ chức phi lợi nhuận có trách nhiệm định nghĩa ra các hướng dẫn như cấu trúc bài thi,
quy ước, khái niệm, chứng chỉ,..
- Nhóm làm việc trong tổ chức ISTQB có trách nhiệm phát triển và duy trì các giáo trình và xây dựng
bài thi chung trên toàn thế giới.

1.3 Exams in ISTQB (Bài thi trong ISTQB cập nhật phiên bản 2018)
2. How to study ISTQB and prepare for exam
Cách nghiên cứu và chuẩn bị cho kỳ thi lấy chứng chỉ ISTQB
2.1 How to read and use ISTQB books
Cách đọc và sử dụng các cuốn sách ISTQB bằng tiếng Anh

Sách dùng để thi chính thức ISTQB gồm có:

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 7
 1. ISTQB FOUNDATIONS LEVEL
Giới thiệu và giải thích 1 cách đầy đủ về các khái niệm và hoạt động của testing. Gồm có 207 trang.
2. ISTQB FOUNDATION LEVEL SYLLABUS
Tóm tắt các khái niệm chính về testing, những key word cần nhớ của quyển 1
3. ISTQB GLOSSARY OF TESTING TERMS
Từ điển các khái niệm về testing
Ngoài ra các bộ đề thi, các ví dụ về mẫu bài thi có rất nhiều trên mạng

Mục tiêu nghiên cứu các tài liệu ISTQB:

- K1 Remember – Ghi nhớ

- K2 Understand – Hiểu biết
- K3 Apply – Áp dụng

2.2 Foundation Level examination – Bài thi ở mức cơ sở

1. Cấu trúc bài thi
- Thi trắc nghiệm multi choices với 40 câu hỏi

- Mỗi câu trả lời đúng là 1 điểm, tối đa 40 điểm

- Trả lời đúng 65%, tức là 26 câu thì mới PASS

- Thời gian làm bài là 60 phút cho thí sinh nói tiếng Anh là tiếng mẹ đẻ, thí sinh ở vùng khác là 75 phút

2. Phân bổ câu hỏi của đề thi

Điểm khác biệt so với đề thi cũ là chỉ dùng 3 khái niệm K1- Remember, K2- Understand, K3-
Analysis và số câu hỏi thuộc K2 tăng lên nhiều hơn so với đề cũ
- K1 – 8 câu hỏi

- K2 – 24 câu hỏi

- K3 – 8 câu hỏi 3. Phân bổ theo chương

Bài thi vẫn dựa chủ yếu vào sách Syllabus 2018 để ra đề thi và phân bố theo chương như sau
Chương Tổng số câu Phân bổ theo độ khó
Chương 1- Fundamental of testing 8 câu K1 = 2 K2 = 6 K3 = 0
Chương 2- Test throughout lifecycles 5 câu K1 = 1 K2 = 4 K3 = 0
Chương 3- Static testing 5 câu K1 = 1 K2 = 3 K3 = 1
Chương 4- Test Design Techniques 11 câu K1 = 1 K2 = 5 K3 = 5
Chương 5- Test management 9 câu K1 = 2 K2 = 5 K3 = 2
Chương 6- Test tools 2 câu K1 = 1 K2 = 1 K3 = 0

2.3 Plan for examination -Lập kế hoạch cho kỳ thi

- Chuẩn bị thi trong vòng 1 tháng đến 1.5 tháng

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 8
 - Mỗi tuần tối thiểu phải đọc hết 1 chương và làm bài tập

Tips for doing exercises -Các chỉ dẫn, kinh nghiệm để làm bài tập

- Bạn phải nắm chắc kiến thức, thậm chí có thể phải thuộc lý thuyết trong sách
- Bạn nên tạo bản tóm tắt bằng tiếng Anh ngắn gọn các kiến thức này để việc ghi nhớ dễ dàng hơn
- Bạn cần tăng cường đọc đi đọc lại sách syllabus nhiều lần để tăng khả năng đọc hiểu tiếng Anh của mình
- Cần có sự phân bố thời gian hợp lý cho từng phần để ôn tập tốt hơn. Ví dụ:
o Chương 4 có nhiều bài tập để làm, nhưng đi thi cũng chỉ có 3 câu bài tập còn lại là lý thuyết. Vì
thế việc luyện nhiều bài tập không giúp được gì nhiều
o Chương 5 là chương khó hiểu nhất trong các nội dung và thường mất nhiều điểm ở đây. Cuối
cùng, bạn hãy chuẩn bị tốt cho kỳ thi, khi đã lên mục tiêu thì hãy tập trung học trong thời gian
ngắn, thường việc ôn thi chỉ mất từ 1-2 tháng là được. Việc kéo dài 6 tháng ôn thi cũng không
giúp được gì nhiều lắm.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 9
Chapter 1: Fundamentals of Testing
1.1 What is Testing? - Testing là gì
What is testing?
- Test activities exist before and after test execution.
- Both dynamic testing and static testing

1.1.1 Typical Objectives of Testing

- To evaluate work products such as requirements, user stories, design, and code
- To verify whether all specified requirements have been fulfilled
- To validate whether the test object is complete and works as the users and other stakeholders expect
- To build confidence in the level of quality of the test object
- To prevent defects
- To find failures and defects
- To provide sufficient information to stakeholders to allow them to make informed decisions
- To reduce the level of risk of inadequate software quality
- To comply with contractual, legal, or regulatory requirements or standards

Different viewpoints in testing take different objectives into account

- During component testing, one objective may be to find as many failures as possible so that the
underlying defects are identified and fixed early. Another objective may be to increase code coverage of
the component tests.
- During acceptance testing, one objective may be to confirm that the system works as expected and
satisfies requirements. Another objective of this testing may be to give information to stakeholders about
the risk of releasing the system at a given time.

1.1.2 Testing and Debugging

Testing and debugging are different.

- Executing tests can show failures that are caused by defects in the software.
- Debugging is the development activity that finds, analyzes, and fixes such defects.
- Subsequent confirmation testing checks whether the fixes resolved the defects.

1.2 Why is Testing Necessary?

- Rigorous testing of systems and documentation can help to reduce the risk of problems occurring
during operation
- When defects are detected, and subsequently fixed, this contributes to the quality of the components
or systems
- Software testing may also be required to meet contractual or legal requirements, or industryspecific
standards.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 10
1.2.1 Testing’s Contributions to Success
Testing contributes to overall software development and maintenance success. Example:
Phase (Giai đoạn) Contributes (Đóng góp)

Testers involved in requirements reviews or Reduces the risk of incorrect or untestable

user story refinement could detect defects functionality being developed.
Testers work closely with system designers Reduce the risk of fundamental design defects
while the system is being designed can and enable tests to be identified at an early
increase understanding and how to test it stage.
Testers work closely with developers while the Reduce the risk of defects within the code and
code can increase understanding and how to the tests.
test it
Testers verify and validate the software prior Increases the likelihood that the software meets
to release can detect failures stakeholder needs and satisfies requirements.

1.2.2 Quality Assurance and Testing

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 11
Quality assurance and testing are not the same, but they are related

- Quality management includes all activities that direct and control an organization with regard to
quality, includes both quality assurance and quality control
- Quality assurance is typically focused on adherence to proper processes, in order to provide
confidence that the appropriate levels of quality will be achieved. Quality assurance contributes to
defect prevention (example: the use of root cause analysis to detect and remove the causes of
defects, retrospective meetings to improve processes)
- Quality control involves various activities, including test activities, that support the achievement
of appropriate levels of quality. Test activities are part of the overall software development or
maintenance process
- Since quality assurance is concerned with the proper execution of the entire process, quality
assurance supports proper testing

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 12
1.2.3 Errors, Defects, and Failures

Errors may occur for many reasons, such as:

- Time pressure Human fallibility

- Inexperienced or insufficiently skilled project participants
- Miscommunication between project participants, including miscommunication about requirements
and design
- Complexity of the code, design, architecture, the underlying problem to be solved, and/or the
technologies used
- Misunderstandings about intra-system and inter-system interfaces, especially when such
intrasystem and inter-system interactions are large in number
- New, unfamiliar technologies
In addition to failures caused due to defects in the code, failures can also be caused by environmental
conditions
Not all unexpected test results are failures:

- False positives may occur due to errors in the way tests were executed, or due to defects in the test
data, the test environment, or other testware, or for other reasons. The inverse situation can also
occur, where similar errors or defects lead to false negatives.
- False negatives are tests that do not detect defects that they should have detected; false positives
are reported as defects, but aren’t actually defects.
1.2.4 Defects, Root Causes and Effects
The root causes of defects are the earliest actions or conditions that contributed to creating the defects -
Identify the root causes of failure can

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 13
 - Reduce the occurrence of similar defects in the future
- Lead to process improvements that prevent a significant number of future defects from being
introduced
1.3 Seven Principles of Testing – 7 nguyên lý cơ bản của testing
Testing shows presence of defects

- Testing can show that defects are present, but cannot prove that there are no defects.
- Testing reduces the probability of undiscovered defects remaining in a software but, even if no
defects are found, it is not a proof of correctness
Exhaustive testing is impossible – Complete test

- Test everything (all combination of inputs and preconditions) is not feasible

- Instead of exhaustive testing, risk analysis and priorities should be used to focus testing efforts
Early testing

- Testing activities should start as early as possible in the software or software development life
cycle and should focus on defined objectives
- Perform the test design and review activities early can find defects early on when they are cheap
to find and fix.
Defect clustering: defect density

- A small numbers of modules usually contains most of the defects discovered during pre-release
testing, or is responsible for most of the operational failures
- Rule 80/20: Module core often contains 80% defects Pesticide paradox

- If the same tests are repeated over and over again, no new defects can be found.
- To overcome this pesticide paradox, test cases need to be regularly reviewed and revised; new and
different tests need to be written to exercise different parts of the software to find potentially more
defects
Testing is context dependent
- Testing is done differently in different context
- Ex: Safety-critical software is tested differently from an ecommerce site Absence of error
fallacy

- All specified requirements and fixing all defects found could still produce a system that is difficult
to use, that does not fulfill the users’ needs and expectations, or that is inferior compared to other
competing systems

1.4 Test Process

1.4.1 Test Process in Context
Contextual factors that influence the test process for an organization, include, but are not limited to:

- Software development lifecycle model and project methodologies being used

o Test levels and test types being considered
o Product and project risks
o Business domain
- Operational constraints, including but not limited to
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 14
 o Budgets and resources

o Timescales
o Complexity
o Contractual and regulatory requirements
- Organizational policies and practices
- Required internal and external standards

1.4.2 Test Activities and Tasks

Test planing

Test monitoring and

control

Test analysis

Test design

Test implement

Test execution

Test completion

In Agile development involves small iterations of software design, build, and test that happen on a
continuous basis, supported by on-going planning. So test activities are also happening on an iterative,
continuous basis within this development approach.
In sequential development, the stepped logical sequence of activities will involve overlap, combination,
concurrency, or omission, so tailoring these main activities within the context of the system and the
project is usually required.
Test activities Objectives Work products
Test planning - Define the objectives of testing and the One or more test plans includes:
approach for meeting test objectives (e.g., - information about the test basis
specifying suitable test techniques and - to which the other test work
tasks, and formulating a test schedule for products will be related via
meeting a deadline). traceability information
- May be revisited based on feedback from - exit criteria (or definition of done)
monitoring and control activities which will be used during test
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 15
 Xác định mục tiêu của thử nghiệm và cách tiếp monitoring and control
cận để đáp ứng các mục tiêu kiểm tra (ví dụ: Một hoặc nhiều kế hoạch thử nghiệm
chỉ định các kỹ thuật và nhiệm vụ kiểm tra phù bao gồm: thông tin về cơ sở kiểm tra
hợp và xây dựng lịch kiểm tra để đáp ứng thời mà các sản phẩm công việc thử nghiệm
hạn). Có thể được xem xét lại dựa trên phản khác sẽ được liên kết thông qua thông
hồi từ các hoạt động giám sát và kiểm soát tin truy xuất nguồn gốc
tiêu chí thoát (hoặc định nghĩa của
done) sẽ được sử dụng trong quá trình
giám sát và kiểm soát thử nghiệm

Test - Involves the on-going comparison of test progress reports (produced on

monitoring and actual progress against the test plan using an ongoing and/or a regular basis)
control any test monitoring metrics defined in the test summary reports (produced at
test plan various completion milestones)
- Involves taking actions necessary to meet báo cáo tiến độ thử nghiệm (được
the objectives of the test plan tạo ra liên tục và / hoặc thường
- Are supported by the evaluation of exit
xuyên) báo cáo tóm tắt kiểm tra
criteria for test execution as part of a
(được tạo ra ở các mốc hoàn thành
khác nhau)

- Liên quan đến việc so sánh liên tục tiến độ

thực tế so với kế hoạch thử nghiệm bằng
cách sử dụng bất kỳ số liệu giám sát thử
nghiệm nào được xác định trong kế hoạch
thử nghiệm
- Liên quan đến việc thực hiện các hành động
cần thiết để đáp ứng các mục tiêu của kế
hoạch thử nghiệm
- Được hỗ trợ bởi việc đánh giá các tiêu chí
thoát để thực hiện thử nghiệm như một
phần của mức thử nghiệm nhất định có thể
bao gồm:
Kiểm tra kết quả thử nghiệm và nhật ký theo
tiêu chí phạm vi được chỉ định
+ Đánh giá mức chất lượng của thành phần
hoặc hệ thống dựa trên kết quả kiểm tra và
nhật ký
+ Xác định xem có cần kiểm tra thêm không
- Cung cấp báo cáo tiến độ thử nghiệm, bao
gồm cả những sai lệch so với kế hoạch và
thông tin để hỗ trợ bất kỳ quyết định dừng
thử nghiệm nào

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 16
Test analysis Analyzing the test basis appropriate to the defined and prioritized test conditions
test level being considered: bi-directionally traceable test charters
- Requirement specifications reporting of defects in the test basis
- Design and implementation Xác định và ưu tiên điều kiện thử
information nghiệm điều lệ thử nghiệm truy xuất
- The implementation of the nguồn gốc hai chiều báo cáo các khiếm
component or system khuyết trong cơ sở kiểm tra
- Risk analysis reports
Evaluating the test basis and test items to
identify defects of various types, such as:
- Ambiguities

- Omissions

- Inconsistencies
- Inaccuracies
- Contradictions
- Superfluous statements
Identifying features and sets of features to be
tested
- Defining and prioritizing test
conditions (functional,
nonfunctional, and structural
characteristics, other business
and technical factors, and
levels of risks)
- Capturing bi-directional
traceability between each
element of the test basis and
the associated test conditions
Phân tích cơ sở kiểm tra phù hợp với mức độ
thử nghiệm đang được xem xét: Thông số kỹ
thuật yêu cầu Thông tin thiết kế và thực hiện
Việc thực hiện thành phần hoặc hệ thống Báo
cáo phân tích rủi ro Đánh giá cơ sở thử
nghiệm và các mục thử nghiệm để xác định
các khiếm khuyết của các loại khác nhau,
chẳng hạn như: Mơ hồ Thiếu sót Mâu thuẫn
Không chính xác Mâu thuẫn Những tuyên bố
thừa thãi Xác định các tính năng và bộ tính
năng cần kiểm tra Xác định và ưu tiên các
điều kiện thử nghiệm (đặc điểm chức năng,
không chức năng và cấu trúc, các yếu tố kinh
doanh và kỹ thuật khác và mức độ rủi ro)
Nắm bắt truy xuất nguồn gốc hai chiều giữa
từng phần tử của cơ sở thử nghiệm và các
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 17
 điều kiện thử nghiệm liên quan
Test design Designing and prioritizing test cases and sets High-level test cases, without concrete
of test cases values for input data and expected
results
Identifying necessary test data to support test Bi-directionally traceable to the test
conditions and test cases condition(s) it covers.
Test data, the design of the test
Designing the test environment and environment, and the identification
identifying any required infrastructure and of infrastructure and tools, though

tools the extent to which these results

are documented varies
Capturing bi-directional traceability between significantly.
the test basis, test conditions, test cases, and Test conditions defined in test analysis
test procedures may be further refined in test design
Thiết kế và ưu tiên các trường hợp kiểm thử Các trường hợp thử nghiệm cấp cao,
và tập hợp các trường hợp kiểm thử không có giá trị cụ thể cho dữ liệu đầu
vào và kết quả mong đợi
Xác định dữ liệu thử nghiệm cần thiết để hỗ Có thể theo dõi hai chiều theo (các)
trợ các điều kiện thử nghiệm và trường hợp điều kiện thử nghiệm mà nó đề cập.
thử nghiệm Dữ liệu thử nghiệm, thiết kế môi
trường thử nghiệm và xác định cơ sở
Thiết kế môi trường thử nghiệm và xác định hạ tầng và công cụ, mặc dù mức độ mà
bất kỳ cơ sở hạ tầng và công cụ cần thiết nào những kết quả này
được ghi nhận khác nhau đáng kể.
Nắm bắt khả năng truy nguyên hai chiều giữa Các điều kiện thử nghiệm được xác
cơ sở thử nghiệm, điều kiện thử nghiệm, định trong phân tích thử nghiệm có thể
trường hợp thử nghiệm và quy trình thử được tinh chỉnh thêm trong thiết kế thử
nghiệm nghiệm
Test Developing and prioritizing test Test procedures and the sequencing of
implementation procedures, and, potentially, creating those test procedures
automated test scripts Test suites
Creating test suites from the test A test execution schedule
procedures and (if any) automated test The test data serve to assign concrete
scripts values to the inputs and expected
Arranging the test suites within a test results of test cases
execution schedule in a way that results in The concrete expected results which
efficient test execution are associated with concrete test data
Building the test environment and verifying are identified by using a test oracle.
that everything needed has been set up Quy trình kiểm tra và trình tự các thủ
correctly tục kiểm tra đó Bộ thử nghiệm Lịch
Preparing test data and ensuring it is properly thực hiện thử nghiệm Dữ liệu thử
loaded in the test environment Verifying and
nghiệm phục vụ để gán các giá trị cụ
updating bi-directional traceability between
thể cho các đầu vào và kết quả dự kiến
the test basis, test conditions, test cases, test
procedures, and test suites của các trường hợp thử nghiệm Các kết
Phát triển và ưu tiên quy trình thử nghiệm, quả dự kiến cụ thể có liên quan đến dữ
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 18
 và, có khả năng, tạo ra các kịch bản thử liệu thử nghiệm cụ thể được xác định
nghiệm tự động bằng cách sử dụng một oracle thử
Tạo bộ thử nghiệm từ rocedures thử nghiệm nghiệm.
và (nếu có) kịch bản thử nghiệm tự động Sắp
xếp các bộ thử nghiệm trong lịch trình thực
hiện thử nghiệm theo cách dẫn đến thực hiện
thử nghiệm hiệu quả Xây dựng môi trường
thử nghiệm và xác minh rằng mọi thứ cần
thiết đã được thiết lập chính xác Chuẩn bị dữ
liệu kiểm tra và đảm bảo nó được nạp đúng
cách trong môi trường thử nghiệm Xác minh
và cập nhật truy xuất nguồn gốc hai chiều
giữa cơ sở thử nghiệm, điều kiện thử nghiệm,
trường hợp thử nghiệm, quy trình thử nghiệm
và bộ bài kiểm tra
Test execution Recording the IDs and versions of the test Documentation of the status of
item(s) or test object, test tool(s), and individual test cases or test
testware Executing tests either manually or procedures (e.g., ready to run, pass,
by using test execution tools fail, blocked, deliberately skipped,
Comparing actual results with expected etc.)
results
Analyzing anomalies to establish their likely Defect reports
causes Documentation about which test
item(s), test object(s), test tools, and
Reporting defects based on the testware were involved in the testing
failures observed Tài liệu về tình trạng của các trường
Logging the outcome of test execution hợp thử nghiệm cá nhân hoặc thủ tục
(e.g., pass, fail, blocked) kiểm tra (ví dụ: sẵn sàng chạy, vượt
Repeating test activities either as a qua, thất bại, bị chặn, cố tình bỏ qua,
result of action taken for an anomaly, v.v.) Báo cáo khiếm khuyết Tài liệu về
or as part of the planned testing (các) mục thử nghiệm, (các) đối tượng
thử nghiệm, công cụ kiểm tra và phần
Verifying and updating bi-directional
mềm thử nghiệm có liên quan đến thử
traceability between the test basis, test
nghiệm
conditions, test cases, test procedures, and
test results.
Ghi lại ID và phiên bản của (các) mục thử
nghiệm hoặc đối tượng thử nghiệm, công cụ
kiểm tra và phần mềm kiểm tra Thực thi các
bài kiểm tra theo cách thủ công hoặc bằng
cách sử dụng các công cụ thực thi thử nghiệm
So sánh kết quả thực tế với kết quả mong đợi
Phân tích sự bất thường để xác định nguyên
nhân có thể của chúng Báo cáo lỗi dựa trên
những thất bại được quan sát Ghi lại kết quả
thực hiện bài kiểm tra (ví dụ: vượt qua, thất

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 19
 bại, bị chặn) Lặp lại các hoạt động thử
nghiệm hoặc là kết quả của hành động được
thực hiện cho một sự bất thường, hoặc là một
phần của thử nghiệm theo kế hoạch Xác minh,
cập nhật truy xuất nguồn gốc hai chiều giữa
cơ sở thử nghiệm, điều kiện xét nghiệm,
trường hợp xét nghiệm, quy trình xét nghiệm
và kết quả xét nghiệm.
Test Checking whether all defect reports are Test summary reports
completion closed, entering change requests or Action items for improvement of
product backlog items for any defects subsequent projects or iterations
that remain unresolved at the end of test Change requests or product backlog
execution items
Finalized testware.
Creating a test summary report to be Báo cáo tóm tắt kiểm tra Các mục
communicated to stakeholders hành động để cải thiện các dự án tiếp
theo hoặc lặp lại Thay đổi yêu cầu
Finalizing and archiving the test hoặc mục tồn đọng sản phẩm Phần
environment, the test data, the test mềm kiểm tra đã hoàn thiện.
infrastructure, and other testware for later
reuse

Handing over the testware to the

maintenance teams, other project teams,
and/or other stakeholders
who could benefit from its use

Analyzing lessons learned from the

completed test activities to determine changes
needed for future iterations, releases, and
projects

Using the information gathered to improve

test process maturity
Kiểm tra xem tất cả các báo cáo lỗi có bị
đóng hay không, nhập yêu cầu thay đổi hoặc
mặt hàng tồn đọng sản phẩm cho bất kỳ lỗi
nào vẫn chưa được giải quyết khi kết thúc thử
nghiệm Thực hiện Tạo báo cáo tóm tắt thử
nghiệm thông báo cho các bên liên quan Hoàn
thiện và lưu trữ môi trường thử nghiệm, dữ
liệu thử nghiệm, cơ sở hạ tầng thử nghiệm và
các phần mềm thử nghiệm khác để tái sử dụng
sau này Bàn giao phần mềm thử nghiệm cho
các nhóm bảo trì, các nhóm dự án khác và /
hoặc các bên liên quan khác Ai có thể hưởng
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 20
 lợi từ việc sử dụng nó Phân tích các bài học
rút ra từ các hoạt động thử nghiệm đã hoàn
thành để xác định những thay đổi cần thiết
cho các lần lặp lại, phát hành và dự án trong
tương lai Sử dụng thông tin thu thập được để
cải thiện sự trưởng thành của quá trình kiểm
tra
1.4.4 Traceability between the Test Basis and Test Work Products (Truy xuất nguồn gốc giữa cơ sở
thử nghiệm và sản phẩm làm việc thử nghiệm)
in order to implement effective test monitoring and control, it is important to establish and maintain
traceability throughout the test process between each element of the test basis and the various test
work products associated with that element Good traceability supports:
để thực hiện giám sát và kiểm soát thử nghiệm hiệu quả, điều quan trọng là phải thiết lập và duy trì
truy xuất nguồn gốc trong suốt quá trình thử nghiệm giữa từng yếu tố của cơ sở thử nghiệm và các sản
phẩm công việc thử nghiệm khác nhau liên quan đến yếu tố đó Truy xuất nguồn gốc tốt hỗ trợ:

-Analyzing the impact of changes Phân tích tác động của những thay đổi
-Making testing auditable Làm cho kiểm thử có thể kiểm tra
-Meeting IT governance criteria Đáp ứng các tiêu chí quản trị CNTT
-Improving the understandability of test progress reports and test summary reports to include the
status of elements of the test basis (e.g., requirements that passed their tests, requirements that
failed their tests, and requirements that have pending tests) Cải thiện tính hiểu biết của các báo
cáo tiến độ thử nghiệm và báo cáo tóm tắt thử nghiệm để bao gồm tình trạng của các yếu tố của
cơ sở thử nghiệm (ví dụ: các yêu cầu đã vượt qua các bài kiểm tra của họ, các yêu cầu thất bại
trong bài kiểm tra và các yêu cầu đang chờ kiểm tra)
-Relating the technical aspects of testing to stakeholders in terms that they can understand Liên
quan đến các khía cạnh kỹ thuật của thử nghiệm cho các bên liên quan về mặt mà họ có thể hiểu

-Providing information to assess product quality, process capability, and project progress against
business goals Cung cấp thông tin để đánh giá chất lượng sản phẩm, khả năng quy trình và tiến độ
dự án so với mục tiêu kinh doanh
1.5 The Psychology of Testing Tâm lý của kiểm tra
1.5.1 Human Psychology and Testing Tâm lý học và thử nghiệm của con người
Identifying defects may be perceived as criticism of the product and of its author. An element of human
psychology called confirmation bias can make it difficult to accept information that disagrees with
currently held beliefs.
Xác định các khiếm khuyết có thể được coi là những lời chỉ trích về sản phẩm và của tác giả của nó. Một
yếu tố tâm lý con người được gọi là thiên vị xác nhận có thể gây khó khăn cho việc chấp nhận thông tin
không đồng ý với niềm tin hiện tại.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 21
Some people may perceive testing as a destructive activity, even though it contributes greatly to project
progress and product quality
Một số người có thể coi thử nghiệm là một hoạt động phá hoại, mặc dù nó đóng góp rất nhiều vào tiến độ
dự án và chất lượng sản phẩm.

To reduce these perceptions: Để giảm bớt những nhận thức này

- Information about defects and failures should be communicated in a constructive way
Thông tin về những khiếm khuyết và thất bại cần được truyền đạt một cách xây dựng

- Good interpersonal skills to be able to communicate effectively about defects, failures, test
results, test progress, and risks, and to build positive relationships with colleagues - Good
communicate: Kỹ năng giao tiếp tốt để có thể giao tiếp hiệu quả về các khiếm khuyết, thất bại,
kết quả kiểm tra, tiến độ kiểm tra và rủi ro, và xây dựng mối quan hệ tích cực với đồng
nghiệp- Giao tiếp tốt:
o Start with collaboration rather than battles. Remind everyone of the common goal
of better quality systems. Bắt đầu với sự hợp tác hơn là chiến đấu. Nhắc nhở mọi
người về mục tiêu chung của các hệ thống chất lượng tốt hơn.
o Emphasize the benefits of testing. Nhấn mạnh lợi ích của thử nghiệm
o Communicate test results and other findings in a neutral, fact-focused way without
criticizing the person who created the defective item. Truyền đạt kết quả kiểm tra và
các phát hiện khác một cách trung lập, tập trung vào thực tế mà không chỉ trích người
tạo ra mặt hàng bị lỗi.

o Try to understand how the other person feels and the reasons they may react
negatively to the information. Cố gắng hiểu cảm giác của người khác và lý do họ
có thể phản ứng tiêu cực với thông tin
o Confirm that the other person has understood what has been said and vice versa.
Xác nhận rằng người kia đã hiểu những gì đã được nói và ngược lại.

- Clearly defining the right set of test objectives has important psychological implications
Xác định rõ ràng bộ mục tiêu kiểm tra phù hợp có ý nghĩa tâm lý quan trọng

1.5.2 Tester’s and Developer’s Mindsets Tư duy của tester và nhà phát triển

Developers and testers often think differently Các nhà phát triển và người thử nghiệm thường nghĩ khác
nhau
Developer Tester
objective design and build a product Thiết kế và xây verifying and validating the product,
dựng một sản phẩm finding defects prior to release
xác minh và xác nhận sản phẩm, tìm ra
lỗi trước khi phát hành

mindset more interested in designing and building curiosity, professional pessimism, a

Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 22
 solutions than in contemplating what might critical eye, attention to detail, and a
be wrong with those solutions Quan tâm motivation for good and positive
nhiều hơn đến việc thiết kế và xây dựng các communications and relationships
giải pháp hơn là suy ngẫm về những gì có tò mò, bi quan chuyên nghiệp, con mắt
thể sai với các giải pháp đó phê phán, chú ý đến từng chi tiết và
difficult to find mistakes in their own work động lực cho giao tiếp và mối quan hệ
Khó tìm ra sai lầm trong công việc của minh tốt và tích cực

developers should be able to test their own

code Các nhà phát triển sẽ có thể kiểm tra
mã của riêng họ

Some of the test activities done by independent testers: Một số hoạt động thử nghiệm được thực hiện bởi
những người thử nghiệm độc lập:

- to increases defect detection effectiveness, which is particularly important for large, complex, or
safety-critical systems. để tăng hiệu quả phát hiện khiếm khuyết, điều này đặc biệt quan trọng đối
với các hệ thống lớn, phức tạp hoặc quan trọng về an toàn.

- they have different cognitive biases from the authors họ có những thành kiến nhận thức khác nhau
từ các tác giả
Chapter 2: Testing Throughout the Software Development Lifecycle Kiểm thử
trong suốt vòng đời phát triển phần mềm

2.1 Software Development Lifecycle Models Mô hình vòng đời phát triển phần mềm
In any software development lifecycle model, there are several characteristics of good testing: Trong bất
kỳ mô hình vòng đời phát triển phần mềm nào, có một số đặc điểm của thử nghiệm tốt:
- For every development activity, there is a corresponding test activity Đối với mỗi hoạt động phát
triển, có một hoạt động thử nghiệm tương ứng
- Each test level has test objectives specific to that level Mỗi cấp độ thử nghiệm có mục tiêu kiểm
tra cụ thể cho cấp độ đó
- Test analysis and design for a given test level begin during the corresponding development
activity Phân tích thử nghiệm và thiết kế cho một mức độ thử nghiệm nhất định bắt đầu trong hoạt
động phát triển tương ứng
- Testers participate in discussions to define and refine requirements and design, and are involved
in reviewing work products (e.g., requirements, design, user stories, etc.) as soon as drafts are
available Người thử nghiệm tham gia vào các cuộc thảo luận để xác định và tinh chỉnh các yêu
cầu và thiết kế, và tham gia vào việc xem xét các sản phẩm làm việc (ví dụ: yêu cầu, thiết kế, câu
chuyện người dùng, v.v.) ngay khi có bản nháp.
-
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 23
In any model, test activities should start in the early stages of the lifecycle, adhering to the testing
principle of early testing. There are two common models: Trong bất kỳ mô hình nào, các hoạt động thử
nghiệm nên bắt đầu trong giai đoạn đầu của vòng đời, tuân thủ nguyên tắc thử nghiệm thử nghiệm sớm.
Có hai mô hình phổ biến
- Sequential development models Mô hình phát triển tuần tự
- Iterative and incremental development models Mô hình phát triển lặp đi lặp lại và gia tăng
-

1. Sequential development models

In the Waterfall model, the development activities (e.g., requirements analysis, design, coding, testing)
are completed one after another. In this model, test activities only occur after all other development
activities have been completed. Trong mô hình Thác nước, các hoạt động phát triển (ví dụ: phân tích yêu
cầu, thiết kế, mã hóa, thử nghiệm) được hoàn thành lần lượt. Trong mô hình này, các hoạt động thử
nghiệm chỉ xảy ra sau khi tất cả các hoạt động phát triển khác đã được hoàn thành.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 24
The V-model integrates the test process throughout the development process, implementing the principle
of early testing. Further, the V-model includes test levels associated with each corresponding
development phase, which further supports early testing. In this model, the execution of tests associated
with each test level proceeds sequentially, but in some cases overlapping occurs
Sequential development models deliver software that contains the complete set of features, but typically
require months or years for delivery to stakeholders and users.

Mô hình V tích hợp quá trình thử nghiệm trong suốt quá trình phát triển, thực hiện nguyên tắc thử
nghiệm sớm. Hơn nữa, mô hình V bao gồm các mức thử nghiệm liên quan đến từng giai đoạn phát triển
tương ứng, hỗ trợ thử nghiệm sớm hơn nữa. Trong mô hình này, việc thực hiện các thử nghiệm liên quan
đến từng cấp độ thử nghiệm tiến hành tuần tự, nhưng trong một số trường hợp chồng chéo xảy ra. Các
mô hình phát triển tuần tự cung cấp phần mềm chứa bộ tính năng hoàn chỉnh, nhưng thường yêu cầu
hàng tháng hoặc năm để phân phối cho các bên liên quan và người dùng.

2. Iterative and incremental development models

Examples of Incremental development:

- Rational Unified Process: long iterations (e.g., two to three months), and the feature increments
are correspondingly large, such as two or three groups of related features Quy trình thống nhất
hợp lý: lặp lại dài (ví dụ: hai đến ba tháng) và gia số tính năng tương ứng lớn, chẳng hạn như hai
hoặc ba nhóm các tính năng liên quan
-
- Scrum: short iterations (e.g., hours, days, or a few weeks), and the feature increments are
correspondingly small, such as a few enhancements and/or two or three new features Scrum: lặp
lại ngắn (ví dụ: giờ, ngày hoặc vài tuần) và các tính năng tăng tương ứng nhỏ, chẳng hạn như một
vài cải tiến và / hoặc hai hoặc ba tính năng mới

- Kanban: Implemented with or without fixed-length iterations, which can deliver either a single
enhancement or feature upon completion, or can group features together to release at once
Kanban: Được thực hiện có hoặc không có lặp lại độ dài cố định, có thể cung cấp một cải tiến
hoặc tính năng duy nhất sau khi hoàn thành hoặc có thể nhóm các tính năng lại với nhau để phát
hành cùng một lúc
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 25
 -
- Spiral (or prototyping): Involves creating experimental increments, some of which may be heavily
re-worked or even abandoned in subsequent development work
- Xoắn ốc (hoặc tạo mẫu): Liên quan đến việc tạo ra các gia tăng thử nghiệm, một số trong đó có
thể được làm lại nhiều hoặc thậm chí bị bỏ rơi trong công việc phát triển tiếp theo
Incremental development has some characteristics: Phát triển gia tăng có một số đặc điểm:

- Establishing requirements, designing, building, and testing a system in pieces, which means that
the software’s features grow incrementally. Thiết lập các yêu cầu, thiết kế, xây dựng và thử
nghiệm một hệ thống theo từng mảnh, có nghĩa là các tính năng của phần mềm phát triển dần dần.
-
- Groups of features are specified, designed, built, and tested together in a series of cycles, often of
a fixed duration. Các nhóm tính năng được chỉ định, thiết kế, xây dựng và thử nghiệm cùng nhau
trong một loạt các chu kỳ, thường có thời lượng cố định.
-
- Involve changes to features developed in earlier iterations, along with changes in project scope.
Liên quan đến những thay đổi đối với các tính năng được phát triển trong các lần lặp trước đó,
cùng với những thay đổi trong phạm vi dự án.
-
- Each feature is tested at several test levels as it moves towards delivery. Mỗi tính năng được thử
nghiệm ở một số cấp độ thử nghiệm khi nó di chuyển theo hướng phân phối
-
- In some cases, teams use continuous delivery or continuous deployment, both of which involve
significant automation of multiple test levels as part of their delivery pipelines. Trong một số
trường hợp, các nhóm sử dụng giao hàng liên tục hoặc triển khai liên tục, cả hai đều liên quan đến
tự động hóa đáng kể nhiều cấp độ thử nghiệm như một phần của đường ống phân phối của họ.
-
- Release to end-users on a feature-by-feature basis, on an iteration- by-iteration basis, or in a more
traditional major-release fashion. Phát hành cho người dùng cuối trên cơ sở tính năng theo tính
năng, trên cơ sở lặp lại hoặc theo cách phát hành lớn truyền thống hơn.
-
- Regression testing is increasingly important as the system grows. Kiểm tra hồi quy ngày càng
quan trọng khi hệ thống phát triển.

2.1.2 Software Development Lifecycle Models in Context

Software development lifecycle models must be selected and adapted to the context of project and
product characteristics: Các mô hình vòng đời phát triển phần mềm phải được lựa chọn và điều chỉnh cho
phù hợp với bối cảnh đặc điểm của dự án và sản phẩm:

- the project goal Muctiêu dự án

- the type of product being developed loại sản phẩm đang được phát triển
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 26
 - business priorities (e.g., time-to-market) ưu tiên kinh doanh (ví dụ: thời gian ra thị trường)
- identified product and project risks xác định rủi ro sản phẩm và dự án

Depending on the context of the project, it may be necessary to combine or reorganize test levels and/or
test activities
Software development lifecycle models themselves may be combined. For example, a V-model may be
used for the development and testing of the backend systems and their integrations, while an Agile
development model may be used to develop and test the front-end user interface (UI) and functionality.
Prototyping may be used early in a project, with an incremental development model adopted once the
experimental phase is complete.

2.2 Test Levels (K2)

The test levels:
- Component testing
- Integration testing
- System testing
- Acceptance testing

Test levels are characterized by the following attributes:

- Specific objectives
- Test basis, referenced to derive test cases
- Test object (i.e., what is being tested)
- Typical defects and failures
- Specific approaches and responsibilities
- A suitable test environment is required
Common objectives of test levels:
- Reducing risk
- Verifying whether the functional and non-functional behaviors
- Building confidence that changes have not broken existing components, systems
- Finding defects
- Preventing defects from escaping to higher test levels
Item Component testing Integration Testing System Testing

Objective Focuses on components focuses on interactions focuses on the behavior

that are separately testable and interfaces between and capabilities of a
components or systems whole system or product

Special Done in isolation from the two different levels of Incompleted or

rest of the system, require integration testing: undocumented specification
mock objects, service
Component integration
virtualization, harnesses,
testing
stubs, and drivers. System integration testing
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 27
Test types Functionality (e.g., Functional and nonfunctional Functional and
correctness of nonfunctional and
calculations) Non- data quality
functional characteristic
characteristics (e.g.,
searching for memory
leaks)
Structural properties
(e.g., decision testing).

Test basis Detailed design Software and system design System and software
Code requirement specifications
Sequence diagrams (functional and nonfunctional)
Data model
Component specifications Interface and communication Risk analysis reports
Test objects protocol specifications
Use cases
Use cases
Epics and user stories
Architecture at component or
system level Models of system behavior
Workflows State diagrams
External interface definitions System and user manuals

Typical test Components, units or Subsystems Applications

objects modules Databases
Code and data structures Infrastructure Hardware/software systems

Classes Interfaces Operating systems

Database modules APIs
System under test (SUT)
Microservices
System configuration and
configuration data
Environment Development Specific environment Correspond to the production
environment with environment
framework, debug tool,...

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 28
Typical defects Incorrect functionality Inconsistent message Incorrect calculations
and failures (e.g., not as described in structures between systems
design specifications) Incorrect or unexpected
Incorrect data, missing system functional or
data, or incorrect data nonfunctional behavior
Data flow problems
encoding
Incorrect code and logic Incorrect control and/or data
Incorrect sequencing or flows within the system
Defects are typically timing of interface calls Failure to properly and
fixed as soon as they are Interface mismatch completely carry out end-
found toend functional tasks
Failures in communication
between components Failure of the system to work
properly in the production
Unhandled or improperly environment(s)
handled communication
failures between Failure of the system to work
components as described in system and
user manuals
Incorrect assumptions
about the meaning, units,
or boundaries of the data
being passed between
component
Failure to comply with
mandatory security
regulations

Responsibilities Developer Developer and tester Independent testers

Approach Test-first approach The greater the scope of The most appropriate t
integration, the more difficultechniques for the aspect(s) of
Test driven development becomes to isolate defects to the system to be tested
(TDD) specific component or
system
Big-bang integration
Incremental integration

2.2.4 Acceptance Testing Objectives:

- Establishing confidence in the quality of the system as a whole
- Validating that the system is complete and will work as expected
- Verifying that functional and non-functional behaviors of the system are as specified

Common forms of acceptance testing:

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 29
 1. User acceptance testing (UAT)
The acceptance testing of the system by users is typically focused on validating the fitness for use of the
system by intended users in a real or simulated operational environment. The main objective is building
confidence that the users can use the system to meet their needs, fulfill requirements, and perform
business processes with minimum difficulty, cost, and risk.

2. Operational acceptance testing (OAT)

The acceptance testing of the system by operations or systems administration staff is usually performed
in a (simulated) production environment. The tests focus on operational aspects, and may include:

- Testing of backup and restore

- Installing, uninstalling and upgrading
- Disaster recovery
- User management
- Maintenance tasks
- Data load and migration tasks
- Checks for security vulnerabilities
- Performance testing
The main objective of operational acceptance testing is building confidence that the operators or system
administrators can keep the system working properly for the users in the operational environment, even
under exceptional or difficult conditions

3. Contractual and regulatory acceptance testing

Contractual acceptance testing is performed against a contract’s acceptance criteria for producing
customdeveloped software. Acceptance criteria should be defined when the parties agree to the contract.
Regulatory acceptance testing is performed against any regulations that must be adhered to, such as
government, legal, or safety regulations
Contractual acceptance testing and regulatory acceptance testing are often performed by users or by
independent testers

4. Alpha and beta testing

Alpha and beta testing are typically used by developers of commercial off-the-shelf (COTS) software
who want to get feedback from potential or existing users, customers, and/or operators before the
software product is put on the market. Alpha testing is performed at the developing organization’s site,
not by the development team, but by potential or existing customers, and/or operators or an independent
test team.
Beta testing is performed by potential or existing customers, and/or operators at their own locations. Beta
testing may come after alpha testing, or may occur without any preceding alpha testing having occurred.
Test basis
- Business processes
- User or business requirements
- Regulations, legal contracts and standards
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 30
 - Use cases
- System requirements
- System or user documentation
- Installation procedures - Risk analysis reports
In addition, as a test basis for deriving test cases for operational acceptance testing, one or more of the
following work products can be used:

- Backup and restore procedures

- Disaster recovery procedures
- Non-functional requirements
- Operations documentation
- Deployment and installation instructions
- Performance targets
- Database packages
- Security standards or regulations Typical test objects

- System under test

- System configuration and configuration data
- Business processes for a fully integrated system
- Recovery systems and hot sites (for business continuity and disaster recovery testing) -
Operational and maintenance processes
- Forms Reports
- Existing and converted production data Typical defects and failures

- System workflows do not meet business or user requirements - Business rules are not
implemented correctly
- System does not satisfy contractual or regulatory requirements
- Non-functional failures such as security vulnerabilities, inadequate performance efficiency under
high loads, or improper operation on a supported platform
Specific approaches and responsibilities
Acceptance testing may also occur at other times, for example:
In a sequential development lifecycle

- Acceptance testing of a COTS software product may occur when it is installed or integrated -
Acceptance testing of a new functional enhancement may occur before system testing
In iterative development
- Acceptance testing during and at the end of each iteration, it focused on verifying a new feature
against its acceptance criteria and those focused on validating that a new feature satisfies the
users’ needs
- Alpha tests and beta tests may occur, either at the end of each iteration, after the completion of
each iteration, or after a series of iterations

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 31
 - User acceptance tests, operational acceptance tests, regulatory acceptance tests, and contractual
acceptance tests also may occur, either at the close of each iteration, after the completion of each
iteration, or after a series of iterations
2.3 Test Types
- Evaluating functional quality characteristics, such as completeness, correctness, and
appropriateness
- Evaluating non-functional quality characteristics, such as reliability, performance efficiency,
security, compatibility, and usability
- Evaluating whether the structure or architecture of the component or system is correct, complete,
and as specified
- Evaluating the effects of changes, such as confirming that defects have been fixed (confirmation
testing) and looking for unintended changes in behavior resulting from software or environment
changes (regression testing)
- ISO standard (ISO/IEC 25010)

2.3.1 Functional Testing

Functional testing of a system involves tests that evaluate functions that the system should perform.
Functional requirements may be described in work products such as business requirements specifications,
epics, user stories, use cases, or functional specifications, or they may be undocumented. The functions
are “what” the system should do
2.3.2 Non-functional Testing
Non-functional testing of a system evaluates characteristics of systems and software such as
usability, performance efficiency or security

2.3.3 White-box Testing

White-box testing derives tests based on the system’s internal structure or implementation
Structural coverage is the extent to which some type of structural element has been exercised by tests,
and is expressed as a percentage of the type of element being covered.
2.3.4 Change-related Testing
- Confirmation testing: confirm whether the original defect has been successfully fixed.
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 32
 - Regression testing: a change made in one part of the code, may accidentally affect the behavior of
other parts of the code. Such unintended side-effects are called regressions. Regression testing
involves running tests to detect such unintended side-effects. Changes may include changes to the
environment, such as a new version of an operating system or database management system.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 33
2.3.5 Test Types and Test Levels
Examples of functional, non-functional, white-box, and change-related tests will be given across all test
levels, for a banking application
Levels Functional Non-functional tests White-box tests Change-related
tests tests
Component How a Performance tests are Achieve complete Automated
test component designed to evaluate statement and regression tests are
should the number of CPU decision coverage built for each
calculate cycles required to for all components component and
compound perform a complex that perform included within the
interest total interest
financial continuous
calculation
calculations integration
framework
Component How account Security tests are Exercise how each Confirm fixes to
integration information designed for buffer screen in the interface-related
test captured at the overflow browser interface defects as the fixes
user interface is vulnerabilities due to passes data to the are checked into the
passed to the data passed from the next screen and to code repository.
business logic. user interface to the the business logic
business logic
System test How account Portability tests are Cover sequences All tests for a
holders can designed to check of web pages that given workflow are
apply for a line whether the presentation can occur during a re-executed if any
of credit on layer works on all credit line screen on that
their checking supported browsers and application. workflow changes.
accounts. mobile devices.

System How the Reliability tests are Exercise all possible Tests of the
integration system uses an designed to evaluate inquiry types sent to application
test external system robustness if the credit score interacting with the
microservice to the credit score microservice. credit scoring
check an microservice fails to microservice are
account respond reexecuted daily as
holder’s credit part of continuous
score. deployment of that
microservice.
Acceptance How the Usability tests are Cover all supported All previously-failed
test banker handles designed to evaluate financial data file tests are re-executed
approving or the accessibility of the structures and value after a defect found
declining a banker’s credit ranges for bank- in acceptance testing
credit processing interface tobank transfers. is fixed.
application. for people with
disabilities

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 34
2.4 Maintenance Testing
Maintenance testing focuses on testing the changes to the system, as well as testing unchanged parts that
might have been affected by the changes. Maintenance can involve planned releases and unplanned
releases (hot fixes).
A maintenance release may require maintenance testing at multiple test levels, using various test types,
based on its scope. The scope of maintenance testing depends on:
- The degree of risk of the change, for example, the degree to which the changed area of software
communicates with other components or systems
- The size of the existing system
- The size of the change

2.4.1 Triggers for Maintenance

We can classify the triggers for maintenance as follows:
- Modification, such as planned enhancements (e.g., release-based), corrective and emergency
changes, changes of the operational environment (such as planned operating system or database
upgrades), upgrades of COTS software, and patches for defects and vulnerabilities
- Migration, such as from one platform to another, which can require operational tests of the new
environment as well as of the changed software, or tests of data conversion when data from another
application will be migrated into the system being maintained
- Retirement, such as when an application reaches the end of its life

2.4.2 Impact Analysis for Maintenance

Impact analysis evaluates the changes that were made for a maintenance release to identify the intended
consequences as well as expected and possible side effects of a change, and to identify the areas in the
system that will be affected by the change.
Impact analysis can also help to identify the impact of a change on existing tests. The side effects and
affected areas in the system need to be tested for regressions, possibly after updating any existing tests
affected by the change.
Impact analysis may be done before a change is made, to help decide if the change should be made, based
on the potential consequences in other areas of the system.
Impact analysis can be difficult if:
- Specifications (e.g., business requirements, user stories, architecture) are out of date or missing
- Test cases are not documented or are out of date
- Bi-directional traceability between tests and the test basis has not been maintained - Tool support is
weak or non-existent
- The people involved do not have domain and/or system knowledge
- Insufficient attention has been paid to the software's maintainability during development
CHAPTER 3 STATIC TESTING
3.1 Static Testing Basics
Static testing relies on the manual examination of work products (i.e., reviews) or tool-driven evaluation
of the code or other work products (i.e., static analysis). Both types of static testing assess the code or
other work product being tested without actually executing the code or work product being tested
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 35
3.1.1 Work Products that Can Be Examined by Static Testing
Almost any work product can be examined using static testing (reviews and/or static analysis)
Reviews can be applied to any work product that the participants know how to read and understand.
Static analysis can be applied efficiently to any work product with a formal structure (typically code or
models) for which an appropriate static analysis tool exists.
Static analysis can even be applied with tools that evaluate work products written in natural language
such as requirements (e.g., checking for spelling, grammar, and readability).
3.1.2 Benefits of Static Testing
- Detecting and correcting defects more efficiently, and prior to dynamic test execution
- Identifying defects which are not easily found by dynamic testing
- Preventing defects in design or coding by uncovering inconsistencies, ambiguities, contradictions,
omissions, inaccuracies, and redundancies in requirements
- Increasing development productivity (e.g., due to improved design, more maintainable code)
- Reducing development cost and time
- Reducing testing cost and time
- Reducing total cost of quality over the software’s lifetime, due to fewer failures later in the
lifecycle or after delivery into operation
- Improving communication between team members in the course of participating in reviews
3.1.3 Differences between Static and Dynamic Testing
Compare with dynamic testing, static testing can:
- Provide an assessment of the quality of the work products
- Identify defects as early as possible
- Finds defects in work products directly rather than identifying failures caused by defects when the
software is run.
- Find the defect with much less effort
- Used to improve the consistency and internal quality of work products
Typical defects that are easier and cheaper to find and fix through static testing include:
- Requirement defects (e.g., inconsistencies, ambiguities, contradictions, omissions, inaccuracies,
and redundancies)
- Design defects (e.g., inefficient algorithms or database structures, high coupling, low cohesion)
- Coding defects (e.g., variables with undefined values, variables that are declared but never used,
unreachable code, duplicate code)
- Deviations from standards (e.g., lack of adherence to coding standards)
- Incorrect interface specifications (e.g., different units of measurement used by the calling system
than by the called system)

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 36
 -
-
Security vulnerabilities (e.g., susceptibility to buffer overflows)
Gaps or inaccuracies in test basis traceability or coverage (e.g., missing tests for an acceptance
criterion)
- Maintainability defects (e.g., improper modularization, poor reusability of components, code that
is difficult to analyze and modify without introducing new defects)
3.2 Review Process
3.2.1 Work Product Review Process
Planning
- Defining the scope, which includes the purpose of the review, what documents or parts of
documents to review, and the quality characteristics to be evaluated
- Estimating effort and timeframe
- Identifying review characteristics such as the review type with roles, activities, and checklists
- Selecting the people to participate in the review and allocating roles
- Defining the entry and exit criteria for more formal review types (e.g., inspections) -
Checking that entry criteria are met (for more formal review types) Initiate review
- Distributing the work product (physically or by electronic means) and other material, such as issue
log forms, checklists, and related work products
- Explaining the scope, objectives, process, roles, and work products to the participants
- Answering any questions that participants may have about the review Individual review (i.e.,

individual preparation)

- Reviewing all or part of the work product

- Noting potential defects, recommendations, and questions Issue communication and analysis

- Communicating identified potential defects (e.g., in a review meeting)

- Analyzing potential defects, assigning ownership and status to them
- Evaluating and documenting quality characteristics
- Evaluating the review findings against the exit criteria to make a review decision (reject; major
changes needed; accept, possibly with minor changes) Fixing and reporting
- Creating defect reports for those findings that require changes
- Fixing defects found (typically done by the author) in the work product reviewed
- Communicating defects to the appropriate person or team (when found in a work product related
to the work product reviewed)
- Recording updated status of defects (in formal reviews), potentially including the agreement of
the comment originator
- Gathering metrics (for more formal review types)
- Checking that exit criteria are met (for more formal review types)
- Accepting the work product when the exit criteria are reached

3.2.2 Roles and responsibilities in a formal review

Author

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 37
 -
-
Creates the work product under review
Fixes defects in the work product under review (if necessary)
Management
- Is responsible for review planning
- Decides on the execution of reviews
- Assigns staff, budget, and time
- Monitors ongoing cost-effectiveness
- Executes control decisions in the event of inadequate outcomes Facilitator (often called

moderator)

- Ensures effective running of review meetings (when held) Mediates, if necessary, between
the various points of view
- Is often the person upon whom the success of the review depends? Review leader

- Takes overall responsibility for the review

- Decides who will be involved and organizes when and where it will take place Reviewers

- May be subject matter experts, persons working on the project, stakeholders with an
interest in the work product, and/or individuals with specific technical or business backgrounds
- Identify potential defects in the work product under review
- May represent different perspectives (e.g., tester, programmer, user, operator, business
analyst, usability expert, etc.) Scribe (or recorder)
- Collates potential defects found during the individual review activity
- Records new potential defects, open points, and decisions from the review meeting (when
held)

3.2.3 Review Types

Informal review (e.g., buddy check, pairing, pair review)
- Main purpose: detecting potential defects
- Possible additional purposes: generating new ideas or solutions, quickly solving minor problems
- Not based on a formal (documented) process
- May not involve a review meeting
- May be performed by a colleague of the author (buddy check) or by more people
- Results may be documented
- Varies in usefulness depending on the reviewers
- Use of checklists is optional
- Very commonly used in Agile development Walkthrough

- Main purposes: find defects, improve the software product, consider alternative implementations,
evaluate conformance to standards and specifications
- Possible additional purposes: exchanging ideas about techniques or style variations, training of
participants, achieving consensus
Individual preparation before the review meeting is optional
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 38
 -
-
Review meeting is typically led by the author of the work product
- Scribe is mandatory
- Use of checklists is optional
- May take the form of scenarios, dry runs, or simulations
- Potential defect logs and review reports may be produced
- May vary in practice from quite informal to very formal Technical review

- Main purposes: gaining consensus, detecting potential defects

- Possible further purposes: evaluating quality and building confidence in the work product,
generating new ideas, motivating and enabling authors to improve future work products,
considering alternative implementations
- Reviewers should be technical peers of the author, and technical experts in the same or other
disciplines
- Individual preparation before the review meeting is required
- Review meeting is optional, ideally led by a trained facilitator (typically not the author)
- Scribe is mandatory, ideally not the author
- Use of checklists is optional
- Potential defect logs and review reports are typically produced Inspection

- Main purposes: detecting potential defects, evaluating quality and building confidence in the work
product, preventing future similar defects through author learning and root cause analysis
- Possible further purposes: motivating and enabling authors to improve future work products and
the software development process, achieving consensus
- Follows a defined process with formal documented outputs, based on rules and checklists
- Uses clearly defined roles
- Individual preparation before the review meeting is required
- Reviewers are either peers of the author or experts in other disciplines that are relevant to the
work product
- Specified entry and exit criteria are used
- Scribe is mandatory
- Review meeting is led by a trained facilitator (not the author)
- Author cannot act as the review leader, reader, or scribe
- Potential defect logs and review report are produced
- Metrics are collected and used to improve the entire software development process, including the
inspection process
3.2.4 Applying Review Techniques
Ad hoc
Reviewers are provided with little or no guidance on how this task should be performed.
Reviewers often read the work product sequentially, identifying and documenting issues as they
encounter them.
Ad hoc reviewing is a commonly used technique needing little preparation.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 39
 -
-
This technique is highly dependent on reviewer skills and may lead to many duplicate issues being
reported by different reviewers.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 40
Checklist-based
A checklist-based review is a systematic technique, whereby the reviewers detect issues based on
checklists that are distributed at review initiation (e.g., by the facilitator).
A review checklist consists of a set of questions based on potential defects, which may be derived from
experience.
Checklists should be specific to the type of work product under review and should be maintained regularly
to cover issue types missed in previous reviews.
The main advantage of the checklist-based technique is a systematic coverage of typical defect types.
Care should be taken not to simply follow the checklist in individual reviewing, but also to look for
defects outside the checklist.
Scenarios and dry runs
In a scenario-based review, reviewers are provided with structured guidelines on how to read through the
work product.

A scenario-based approach supports reviewers in performing “dry runs” on the work product based on
expected usage of the work product (if the work product is documented in a suitable format such as use
cases).
These scenarios provide reviewers with better guidelines on how to identify specific defect types than
simple checklist entries.
As with checklist-based reviews, in order not to miss other defect types (e.g., missing features),
reviewers should not be constrained to the documented scenarios.
Role-based
A role-based review is a technique in which the reviewers evaluate the work product from the
perspective of individual stakeholder roles.
Typical roles include specific end user types (experienced, inexperienced, senior, child, etc.), and
specific roles in the organization (user administrator, system administrator, performance tester, etc.).
Perspective-based
In perspective-based reading, similar to a role-based review, reviewers take on different stakeholder
viewpoints in individual reviewing.
Typical stakeholder viewpoints include end user, marketing, designer, tester, or operations.
Using different stakeholder viewpoints leads to more depth in individual reviewing with less duplication
of issues across reviewers.
In addition, perspective-based reading also requires the reviewers to attempt to use the work product
under review to generate the product they would derive from it. For example, a tester would attempt to
generate draft acceptance tests if performing a perspective-based reading on a requirements
specification to see if all the necessary information was included.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 41
Further, in perspective-based reading, checklists are expected to be used.
3.2.5 Success Factors for Reviews
- Each review has clear objectives, defined during review planning, and used as measurable exit criteria
- Review types are applied which are suitable to achieve the objectives and are appropriate to the type
and level of software work products and participants
- Any review techniques used, such as checklist-based or role-based reviewing, are suitable for
effective defect identification in the work product to be reviewed
- Any checklists used address the main risks and are up to date
- Large documents are written and reviewed in small chunks, so that quality control is exercised by
providing authors early and frequent feedback on defects
- Participants have adequate time to prepare
- Reviews are scheduled with adequate notice
- Management supports the review process (e.g., by incorporating adequate time for review activities in
project schedules)
People-related success factors for reviews include:
- The right people are involved to meet the review objectives, for example, people with different skill
sets or perspectives, who may use the document as a work input
- Testers are seen as valued reviewers who contribute to the review and learn about the work product,
which enables them to prepare more effective tests, and to prepare those tests earlier
- Participants dedicate adequate time and attention to detail
- Reviews are conducted on small chunks, so that reviewers do not lose concentration during individual
review and/or the review meeting (when held)
- Defects found are acknowledged, appreciated, and handled objectively
- The meeting is well-managed, so that participants consider it a valuable use of their time
- The review is conducted in an atmosphere of trust; the outcome will not be used for the evaluation of
the participants
- Participants avoid body language and behaviors that might indicate boredom, exasperation, or
hostility to other participants
- Adequate training is provided, especially for more formal review types such as inspections A culture
of learning and process improvement is promoted

CHAPTER 4: Test Design Techniques

4.1 Categories of Test Techniques
4.1.1 Choosing Test Techniques
Depends on a number of factors:
- Type of component or system
- Component or system complexity
- Regulatory standards
- Customer or contractual requirements
- Risk levels

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 42
 - Risk types
- Test objectives
- Available documentation
- Tester knowledge and skills
- Available tools
- Time and budget
- Software development lifecycle model
- Expected use of the software
- Previous experience with using the test techniques on the component or system to be tested
- The types of defects expected in the component or system

4.1.2 Categories of Test Techniques and Their Characteristics

Common characteristics of specification-based test design techniques include - Đặc điểm chung của kỹ
thuật thiết kế test hướng đặc tả:
- Test conditions, test cases, and test data are derived from a test basis that may include software
requirements, specifications, use cases, and user stories
- Test cases may be used to detect gaps between the requirements and the implementation of the
requirements, as well as deviations from the requirements
- Coverage is measured based on the items tested in the test basis and the technique applied to the test
basis

Common characteristics of structure-based test design techniques include - Đặc điểm chung của kỹ
thuật thiết kế test hướng cấu trúc:
- Test conditions, test cases, and test data are derived from a test basis that may include code, software
architecture, detailed design, or any other source of information regarding the structure of the software
- Coverage is measured based on the items tested within a selected structure (e.g., the code or interfaces)

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 43
 - Specifications are often used as an additional source of information to determine the expected outcome
of test cases

Common characteristics of experience-based test design techniques include - Đặc điểm chung của kỹ
thuật thiết kế test hướng kinh nghiệm:
- Test conditions, test cases, and test data are derived from a test basis that may include knowledge and
experience of testers, developers, users and other stakeholders
- These techniques can be helpful in identifying tests that were not easily identified by other more
systematic techniques. Depending on the tester’s approach and experience, these techniques may
achieve widely varying degrees of coverage and effectiveness.
- Coverage can be difficult to assess and may not be measurable with these techniques

4.2 Black-box Test Techniques

4.2.1 Equivalence partitioning (EP) - Phân vùng tương đương (EP)
- Divide (partition) the inputs, outputs, etc. into areas which are expected to exhibit similar behavior
(equivalent) so they are likely to be processed in the same way.
- Assumption: if one value works, all will work
- One from each partition better than all from one
- Tests can be designed to cover all valid and invalid partitions. Equivalence partitioning is applicable at all
levels of testing.

4.2.2 Boundary value analysis (BVA)- Phân tích giá trị biên (BVA)
- Behavior at the edge of each equivalence partition is more likely to be incorrect than behavior within
the partition, so boundaries are an area where testing is likely to yield defects
- Two-point boundary: The maximum and minimum values of a partition are its boundary values -
Three boundary values: the values before, at, and just over the boundary - Tests can be designed
to cover both valid and invalid boundary values.
- Boundary value analysis can be applied at all test levels. It is relatively easy to apply and its defect-
finding capability is high.
- Detailed specifications are helpful in determining the interesting boundaries.

4.2.3 Decision tables - bảng quyết định

- Explore combinations of inputs, situations or events
- It is very easy to overlook specific combinations of input
- Start by expressing the input conditions of interest so that they are TRUE or FALSE
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 44
4.2.4 State transition testing - Test chuyển đổi trạng thái
- Example: State diagram for PIN entry

Switch:
0- Switch: single transition
1- Switch: couple transitions
2- Switch: triple transitions

States table – Bảng trạng thái

4.3 While Box test design

- The basic coverage measure is:

4.3.1 Statement coverage

- Percentage of executable statements exercised by a test suite o Formula =
number of statements exercised / Total number of statements Example of
statement coverage
Read (a) If a>6 then A=b endif

We need only 1 test case a=7 to have 100% statement coverage

Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 45
4.3.2 Decision coverage (Branch coverage)
- Percentage of decision outcomes exercised by a test suite

4.3.3 Path coverage

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 46
Summary while box test

4.4 Experience_base techniques (K2)

4.4.1 Error Guessing
Error guessing is a technique used to anticipate the occurrence of mistakes, defects, and failures, based
on the tester’s knowledge, including

- How the application has worked in the past

- What types of mistakes the developers tend to make
- Failures that have occurred in other applications
- Experience about defect and failure data
- Common knowledge about why software fails

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 47
Error guessing technique is to create a list of possible mistakes, defects, and failures, and design tests that
will expose those failures and the defects that caused them
4.4.2 Exploratory Testing
In exploratory testing, informal tests are designed, executed, logged, and evaluated dynamically during
test execution. The test results are used to learn more about the component or system, and to create tests
for the areas that may need more testing
Exploratory testing is sometimes conducted using session-based testing to structure the activity:

- Tester uses a test charter containing test objectives within a defined time-box to guide the testing.
- Tester may use test session sheets to document the steps followed and the discoveries made.
Exploratory testing is most useful when there are few or inadequate specifications or significant
time pressure on testing. Exploratory testing is also useful to complement other more formal testing
techniques.

4.4.3 Checklist-based Testing

Checklists can be built based on experience, knowledge about what is important for the user, or an understanding
of why and how software fails.
Checklists can be created to support various test types, including functional and non-functional testing.
In the absence of detailed test cases, checklist-based testing can provide guidelines and a degree of
consistency.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 48
Chapter 5: Test Management
5.1 Test Organization

Testers external
to the
Testers from organization
the business
Test team or organization
group
Developers or
testers within
None the
developers test development
their own code teams

Benefits of test independence include

- Recognize different kinds of failures
- Verify, challenge, or disprove assumptions
Drawbacks of test independence include
- Isolation from the development team, leading to a lack of collaboration, delays in providing feedback to the
development team, or an adversarial relationship with the development team
- Developers may lose a sense of responsibility for quality
- Independent testers may be seen as a bottleneck or blamed for delays in release
- Independent testers may lack some important information (e.g., about the test object)

5.1.2 Tasks of a Test Manager and Tester

Testing phase Test Manager tasks Tester tasks
Planning - Plan the test activities - Review and contribute to test plans
- Write and update the test plan(s) - Create the detailed test execution schedule
- Coordinate the test plan(s) with project
managers, product owners, and others
- Share testing perspectives with other
project activities, such as integration
planning

Analysis and - Initiate the analysis, design, - Analyze, review, and assess requirements
design implementation, and execution of tests for testability (i.e., the test basis)
Implement and - Support the selection and - Identify and document test conditions,
execution implementation of tools to support the capture traceability
test process - Design, set up, and verify test
- Support setting up the defect environment(s)
management system
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 49
 - Set up adequate configuration - Design and implement test cases and test
management of testware procedures
- Prepare and acquire test data
- Decide about the implementation of - Execute tests, evaluate the results
test environment(s) - Automate tests as needed
- Evaluate non-functional characteristics
- Review tests developed by others

Monitoring - Monitor test progress and results, and - Use appropriate tools to facilitate the test
and control check the status of exit criteria process
- Create test progress reports and test
summary reports
- Adapt planning based on test results
and progress
- Take any actions necessary for test
control

Organization - Develop or review a test policy and -

task test strategy
- Introduce suitable metrics for
measuring test progress and evaluating
the quality of the testing and the
product
- Promote and advocate the testers, the
test team
- Develop the skills and careers of
testers

5.2 Test Planning and Estimation

5.2.1 Purpose and Content of a Test Plan
- Determining the scope, objectives, and risks of testing
- Defining the overall approach of testing
- Integrating and coordinating the test activities into the software lifecycle activities
- Making decisions about what to test, the people and other resources
- Scheduling of test analysis, design, implementation, execution, and evaluation activities
- Selecting metrics for test monitoring and control
- Budgeting for the test activities
- Determining the level of detail and structure for test documentation

5.2.2 Test Strategy and Test Approach

Analytical: analysis of some factor (e.g., requirement or risk)

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 50
Model-Based: tests are designed based on some model of some required aspect of the product. Examples
of such models include business process models, state models, and reliability growth models
Methodical: use of some predefined set of tests or test conditions, such as a taxonomy of common or
likely types of failures, a list of important quality characteristics, or standards for mobile apps or web
pages
Process-compliant (or standard-compliant): based on external rules and standards, process
Directed (or consultative): base on the advice, guidance, or instructions of stakeholders, business
domain experts, or technology experts
Regression-averse: highly automation of regression tests, and standard test suites
Reactive: Tests are designed and implemented, and may immediately be executed in response to
knowledge gained from prior test results. Exploratory testing is a common technique employed in
reactive strategies

5.2.3 Entry Criteria and Exit Criteria (Definition of Ready and Definition of Done)

Typical entry criteria include:

- Availability of testable requirements, user stories, and/or models

- Availability of test items that have met the exit criteria for any prior test levels
- Availability of test environment
- Availability of necessary test tools
- Availability of test data and other necessary resources Typical exit criteria include:

- Planned tests have been executed

- A defined level of coverage
- The number of unresolved defects is within an agreed limit
- The number of estimated remaining defects is sufficiently low
- The evaluated levels of reliability, performance efficiency, usability, security, and other relevant quality
characteristics are sufficient
5.2.4 Test Execution Schedule - Lịch trình thực hiện
Once the various test cases and test procedures are produced and assembled into test suites
The test suites can be arranged in a test execution schedule that defines the order in which they are to
be run.
The test execution schedule should take into account such factors as prioritization, dependencies,
confirmation tests, regression tests, and the most efficient sequence for executing the tests.

5.2.5 Factors Influencing the Test Effort

Test effort estimation involves predicting the amount of test-related work that will be needed in order to
meet the objectives of the testing for a particular project, release, or iteration

Product characteristics
The risks associated with the product

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 51
 The quality of the test basis
The size of the product
The complexity of the product domain
The requirements for quality characteristics (e.g., security, reliability)
The required level of detail for test documentation
Requirements for legal and regulatory compliance
Development process characteristics
The stability and maturity of the organization
The development model in use
The test approach
The tools used
The test process
Time pressure
People characteristics
The skills and experience of the people involved, especially with similar projects and products
(e.g., domain knowledge)
Team cohesion and leadership
Test results
The number and severity of defects found
The amount of rework required
Two of the most commonly used techniques are:
1. The metrics-based technique: estimating the test effort based on metrics of former similar
projects, or based on typical values
Project Requirement Design Coding Testing Quality CM Risk
management development assurance buffer
Average 10% 15% 10% 25% 35% 2.5% 2.5% 10%
100 old
projects
Project ? ? ? 1250pd ?? ? ?
A

2. The expert-based technique: estimating the test effort based on the experience of the owners
of the testing tasks or by experts

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 52
WBS- Work Breakdown Structure

5.3 Test Monitoring and Control

Test monitoring is to gather information and provide feedback and visibility about test
activities, measure and assess test progress and measure whether the test exit criteria Test
control describes any guiding or corrective actions taken:

- Re-prioritizing tests when an identified risk occurs (e.g., software delivered late)
- Changing the test schedule due to availability or unavailability of a test environment or other resources
- Re-evaluating whether a test item meets an entry or exit criterion due to rework

5.3.1 Metrics Used in Testing

Metrics can be collected during and at the end of test activities in order to assess:

- Progress against the planned schedule and budget

- Current quality of the test object
- Adequacy of the test approach
- Effectiveness of the test activities with respect to the objectives
Common test metrics include:

-
- Percentage of planned work done in each test activities
- Test case execution (e.g., number of test cases run/not run, test cases passed/failed, and/or test conditions
passed/failed)
-
- Defect information (e.g., defect density, defects found and fixed, failure rate, and confirmation test
results)
-
- Test coverage of requirements, user stories, acceptance criteria, risks, or code
- Task completion, resource allocation and usage, and effort
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 53
 -
- Cost of testing, including the cost compared to the benefit of finding the next defect or the cost compared
to the benefit of running the next test

5.3.2 Purposes, Contents, and Audiences for Test Reports

The purpose of test reporting is to summarize and communicate test activity information, both during and
at the end of a test activity
Content common to test progress reports:

- The status of the test activities and progress against the test plan
- Factors impeding progress
- Testing planned for the next reporting period
- The quality of the test object
Typical test progress reports and test summary reports may include:

- Summary of testing performed

- Analysis the information on what occurred during a test period
- Deviations from plan
-
- Metrics of defects, test cases, test coverage, activity progress, and resource consumption.
- Residual risks
- Reusable test work products produced

5.4 Configuration Management

The purpose of configuration management is to establish and maintain the integrity of the component or
system, the testware, and their relationships to one another through the project and product lifecycle:

- All test items are uniquely identified, version controlled, tracked for changes, and related to each
other
- All items of testware are uniquely identified, version controlled, tracked for changes, related to
each other and related to versions of the test item(s) so that traceability can be maintained
throughout the test process
- All identified documents and software items are referenced unambiguously in test documentation

5.5 Risks and Testing

5.5.1 Definition of Risk
Risk involves the possibility of an event in the future which has negative consequences.
The level of risk is determined by the likelihood of the event and the impact (the harm) from that event.

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 54
5.5.2 Product and Project Risks
Project risk Product risk (quality risks)
A risk to the project’s capability to deliver A risk to quality of product
products: Scope, cost, Time
Related to management and control of the Directly related to the test object
(test) project: - Failure-prone software delivered
- Skill, training and staff shortages - The potential that the software/hardware could
- Personnel issues cause harm to an individual or company
- Political issues - Poor software characteristics (e.g., functionality,
- Technical issues reliability, usability and performance)
- Schedules - Poor data integrity and quality
- Budget - Software that does not perform its intended
functions
- User experience (UX) feedback might not meet
product expectations
- A loop control structure may be coded incorrectly
- A particular computation may be performed
incorrectly in some circumstances

5.5.3 Risk-based Testing and Product Quality

Risk is used to focus the effort required during testing. It is used to decide where and when to start testing
and to identify areas that need more attention.
Testing is used to reduce the probability of an adverse event occurring, or to reduce the impact of an
adverse event.
Testing is used as a risk mitigation activity, to provide feedback about identified risks, as well as
providing feedback on residual (unresolved) risks
In a risk-based approach, the results of product risk analysis are used to:

- Determine the test techniques to be employed

- Determine the particular levels and types of testing to be performed (e.g., security testing, accessibility
testing)
- Determine the extent of testing to be carried out
- Prioritize testing in an attempt to find the critical defects as early as possible
- Determine whether any activities in addition to testing could be employed to reduce risk (e.g., providing
training to inexperienced designers) Risk activities include:
- Analyze (and re-evaluate on a regular basis) what can go wrong (risks)
- Determine which risks are important to deal with (risk level: likelihood, impact)
- Implement actions to mitigate those risks
- Make contingency plans to deal with the risks should they become actual events
The testing may identify new risks, help to determine what risks should be mitigated, and lower
uncertainty about risks
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 55
5.6 Defect Management
Since one of the objectives of testing is to find defects, the discrepancies between actual and expected
outcomes need to be logged as incidents. An incident must be investigated and may turn out to be a
defect.
Incident reports have the following objectives:

- Provide developers and other parties with feedback about the problem to enable identification, isolation
and correction as necessary
- Provide test leaders a means of tracking the quality of the system under test and the progress of the
testing
- Provide ideas for test process improvement
A defect report filed during dynamic testing typically includes:

- An identifier
- A title and a short summary of the defect being reported
- Date of the defect report, issuing organization, and author
- Identification of the test item (configuration item being tested) and environment
- The development lifecycle phase(s) in which the defect was observed
- A description of the defect to enable reproduction and resolution, including logs, database dumps
screenshots, or recordings
-
- Expected and actual results
- Scope or degree of impact (severity) of the defect on the interests of stakeholder(s) Urgency/priority to fix
- State of the defect report (e.g., open, deferred, duplicate, waiting to be fixed, awaiting confirmation
testing, re-opened, closed)
-
- Conclusions, recommendations and approvals
- Global issues, such as other areas that may be affected by a change resulting from the defect
- Change history, such as the sequence of actions taken by project team members with respect to the defect
to isolate, repair, and confirm it as fixed
-
- References, including the test case that revealed the problem
Refer: ISO standard (ISO/IEC/IEEE 29119-3)

Chapter 6: Tool Support for Testing

6.1 Test Tool Considerations
6.1.1 Test Tool Classification
Test tools can have some purposes:
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 56
 - Improve the efficiency of test activities by automating repetitive tasks or tasks that require significant
resources when done manually (e.g., test execution, regression testing)
- Improve the efficiency of test activities by supporting manual test activities throughout the test process
- Improve the quality of test activities by allowing for more consistent testing and a higher level of
defect reproducibility
- Automate activities that cannot be executed manually (e.g., large scale performance testing)
- Increase reliability of testing (e.g., by automating large data comparisons or simulating behavior) Tool
categories

6.1.2 Benefits and Risks of Test Automation

Potential benefits of using tools to support test execution include:

- Reduction in repetitive manual work (e.g., running regression tests, environment set up/tear down tasks,
re-entering the same test data, and checking against coding standards), thus saving time
- Greater consistency and repeatability (e.g., test data is created in a coherent manner, tests are executed by
a tool in the same order with the same frequency, and tests are consistently derived from requirements)
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 57
 - More objective assessment (e.g., static measures, coverage)
- Easier access to information about testing (e.g., statistics and graphs about test progress, defect rates and
performance)
Potential risks of using tools to support testing include:

- Expectations for the tool may be unrealistic (including functionality and ease of use)
- The time, cost and effort for the initial introduction of a tool may be under-estimated (including training
and external expertise)
- The time and effort needed to achieve significant and continuing benefits from the tool may be under-
estimated (including the need for changes in the test process and continuous improvement in the way the
tool is used)
- The effort required to maintain the test assets generated by the tool may be under-estimated
- The tool may be relied on too much (seen as a replacement for test design or execution, or the use of
automated testing where manual testing would be better)
- Version control of test assets may be neglected
- Relationships and interoperability issues between critical tools may be neglected, such as requirements
management tools, configuration management tools, defect management tools and tools from multiple
vendors
- The tool vendor may go out of business, retire the tool, or sell the tool to a different vendor
- The vendor may provide a poor response for support, upgrades, and defect fixes
- An open source project may be suspended
- A new platform or technology may not be supported by the tool
- There may be no clear ownership of the tool (e.g., for mentoring, updates, etc.)

6.1.3 Special Considerations for Test Execution and Test Management Tools
Test execution tools
Data-driven scripting technique Keyword-driven scripting Model-Based testing (MBT)
technique
Data files store test input and data files store test input, a functional specification to be
expected results in table or expected results and keywords in captured in the form of a
spreadsheet table or spreadsheet model, such as an activity
diagram
support capture/playback tools writing script manually The MBT tool interprets the
model in order to create test
case specifications which can
then be saved in a test
management tool and/or
executed by a test execution
tool
Example keyword driven:

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 58
Test management tools
Test management tools often need to interface with other tools or spreadsheets for various reasons,
including:

- To produce useful information in a format that fits the needs of the organization
- To maintain consistent traceability to requirements in a requirements management tool
- To link with test object version information in the configuration management tool

6.2 Effective Use of Tools

6.2.1 Main Principles for Tool Selection
- Assessment of organizational maturity, strengths and weaknesses and identification of opportunities for an
improved test process supported by tools
- Understanding of the technologies used by the test object(s), in order to select a tool that is compatible with
that technology
- The build and continuous integration tools already in use within the organization, in order to ensure tool
compatibility and integration
- Evaluation of the tool against clear requirements and objective criteria
- Consideration of whether or not the tool is available for a free trial period (and for how long)
- Evaluation of the vendor (including training, support and commercial aspects) or support for noncommercial
(e.g., open source) tools
- Identification of internal requirements for coaching and mentoring in the use of the tool
- Evaluation of training needs, considering the testing (and test automation) skills of those who will be
working directly with the tool(s)
- Consideration of pros and cons of various licensing models (e.g., commercial or open source)
- Estimation of a cost-benefit ratio based on a concrete business case (if required)

6.2.2 Pilot Projects for Introducing a Tool into an Organization

- Gaining in-depth knowledge about the tool, understanding both its strengths and weaknesses
- Evaluate how the tool fits with existing processes and practices, and determine what would need to change
- Decide on standard ways of using, managing, storing and maintaining the tool and the test assets
- Assess whether the benefits will be achieved at reasonable cost
- Understanding the metrics that you wish the tool to collect and report, and configuring the tool to ensure
these metrics can be captured and reported

6.2.3 Success Factors for Tools

- Rolling out the tool to the rest of the organization incrementally
- Adapting and improving processes to fit with the use of the tool
- Providing training and coaching/mentoring for new users
Bản quyền thuộc về QRS-website: qr-solutions.com.vn
DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 59
- Defining usage guidelines
- Implementing a way to gather usage information from the actual use - Monitoring tool use and benefits
- Providing support for the test team for a given tool
- Gathering lessons learned from all teams

Bản quyền thuộc về QRS-website: qr-solutions.com.vn

DC: tầng 3, số 1, ngõ 36/7 Duy Tân Trang 60