Levels of Testing

Chapter - 5

1
Contents
• Component testing
• Integration Testing
• System Testing
• Acceptance Testing

2
1. Component testing
• Also known as unit, module and program testing,
• searches for defects in, and verifies the functioning of software (e.g.
modules, programs, objects, classes, etc.) that are separately testable.
• May be done in isolation from the rest of the system depending on the
context of the development life cycle and the system.
• May include testing of functionality and specific nonfunctional
characteristics such as resource-behavior (e.g. memory leaks),
performance or robustness testing, as well as structural testing (e.g.
decision coverage).
• Test cases are derived from work products such as the software
design or the data model.

3
1. Component testing …
• Typically, it occurs with access to the code being tested and
with the support of the development environment, such as a unit test
framework or debugging tool, and in practice usually involves the
programmer who wrote the code.
• Sometimes, depending on the applicable level of risk, component
testing is carried out by a different programmer thereby introducing
independence.
• Defects are typically fixed as soon as they are found, without
formally recording the incidents found.

4
1. Component testing …
• One approach in component testing, used in Extreme Programming
(XP), is to prepare and automate test cases before coding.
• This is called a test-first approach or test-driven development. This
approach is highly iterative and is based on cycles of developing test
cases, then building and integrating small pieces of code, and
executing the component tests until they pass.

5
2. Integration testing
• Tests interfaces between components, interactions to different parts
of a system such as an operating system, file system and hardware or
interfaces between systems.
• Note that integration testing should be differentiated from other
integration activities. Integration testing is often carried out by the
integrator, but preferably by a specific integration tester or test team.
• There may be more than one level of integration testing and it may be
carried out on test objects of varying size.

6
2. Integration testing …
• The greater the scope of integration, the more difficult it becomes to
isolate failures to a specific interface, which may lead to an increased risk.
• This leads to varying approaches to integration testing.
• One extreme is that all components or systems are integrated
simultaneously, after which everything is tested as a whole. This is called
'big-bang' integration testing. Big-bang testing has the advantage that
everything is finished before integration testing starts. There is
no need to simulate (as yet unfinished) parts. The major disadvantage is
that in general it is time-consuming and difficult to trace the cause of
failures with this late integration.

7
2. Integration testing …
• So big-bang integration may seem like a good idea when planning the
project, being optimistic and expecting to find no problems.
• Another extreme is that all programs are integrated one by one, and a
test is carried out after each step (incremental testing).
• The incremental approach has the advantage that
the defects are found early in a smaller assembly when it is relatively
easy to detect the cause. A disadvantage is that it can be time-
consuming since stubs and drivers have to be developed and used in
the test.

8
2. Integration testing …
• Within incremental integration testing a range of possibilities exist, partly
depending on the system architecture:
• Top-down: testing takes place from top to bottom, following the control
flow or architectural structure (e.g. starting from the GUI or main menu).
Components or systems are substituted by stubs.
• Bottom-up: testing takes place from the bottom of the control flow
upwards. Components or systems are substituted by drivers.
• Functional incremental: integration and testing takes place on the basis of
the functions or functionality, as documented in the functional
specification.

9
2. Integration testing ….
• The preferred integration sequence and the number of integration
steps
required depend on the location in the architecture of the high-risk
interfaces.
• Ideally testers
should understand the architecture and influence integration
planning.

10
3. System Testing
• System testing is concerned with the behavior of the whole
system/product as defined by the scope of a development project or
product
• System testing is most often the final test on behalf of development to
verify that the system to be delivered meets the specification and its
purpose may be to find as many defects as possible.
• System testing should investigate both functional and non-functional
requirements of the system.
• System testing requires a controlled test environment with regard to,
amongst other things, control of the software versions, testware and the
test data

11
3. System Testing …
• A system test is executed by the development organization in a
(properly controlled) environment.
• The test environment should correspond to the final target or
production environment as much as possible in order to minimize the
risk of environment specific failures not being found by testing.

12
4. Acceptance Testing
• When the development organization has performed its system test
and has corrected all or most defects, the system will be delivered to
the user or customer for acceptance testing.
• The acceptance test should answer questions such as: 'Can the
system be released?', 'What, if any, are the outstanding (business)
risks?' and 'Has development met their obligations?'
• Acceptance testing is most often the responsibility of the user or
customer, although other stakeholders may be involved as well. The
execution of the acceptance test requires a test environment that is
for most aspects, representative of the production environment ('as-if
production').

13
4. Acceptance Testing …
• The goal of acceptance testing is to establish confidence in the
system, part of the system or specific non-functional characteristics,
e.g. usability, of the system. Acceptance testing is most often focused
on a validation type of testing, whereby we are trying to determine
whether the system is fit for purpose.
• Finding defects should not be the main focus in acceptance testing.
Although it
assesses the system's readiness for deployment and use, it is not
necessarily the
final level of testing.

14
4. Acceptance Testing …
• Within the acceptance test for a business-supporting system, two main test
types can be distinguished; as a result of their special character, they are
usually prepared and executed separately.
• Within the acceptance test for a business-supporting system, two main test
types can be distinguished; as a result of their special character, they are
usually prepared and executed separately. The user acceptance test focuses
mainly on the functionality thereby validating the fitness-for-use of the
system by the business user, while the operational acceptance test (also
called production acceptance test) validates whether the system meets the
requirements for operation.

15
4. Acceptance Testing …
• The user acceptance test is performed by the
users and application managers. In terms of planning, the user
acceptance test usually links tightly to the system test and will, in
many cases, be organized partly overlapping in time. If the system to
be tested consists of a number of more or less independent
subsystems, the acceptance test for a subsystem that complies to the
exit criteria of the system test can start while another subsystem may
still be in the system test phase.

16
4. Acceptance Testing …
• In most organizations, system administration will perform the operational
acceptance test shortly before the system is released. The operational acceptance
test may include testing of backup/restore, disaster recovery, maintenance tasks
and periodic check of security vulnerabilities.
• Other types of acceptance testing that exist are contract acceptance testing
and compliance acceptance testing. Contract acceptance testing is performed
against a contract's acceptance criteria for producing custom-developed
software. Acceptance should be formally defined when the contract is agreed.
Compliance acceptance testing or regulation acceptance testing is performed
against the regulations which must be adhered to, such as governmental, legal
or safety regulations.

17
4. Acceptance Testing …
• If the system has been developed for the mass market, e.g.
commercial off the-shelf software (COTS), then testing it for individual
users or customers is not practical or even possible in some cases.
Feedback is needed from potential or existing users in their market
before the software product is put out for sale commercially.

18
4. Acceptance Testing …
• Very often this type of system undergoes two stages of acceptance
test. The first is called alpha testing. This test takes place at the
developer's site. A cross-section of potential users and members of
the developer's organization are invited to use the system.
Developers observe the users and note problems. Alpha testing may
also be carried out by an independent test team. Beta testing, or field
testing, sends the system to a cross-section of users who install it and
use it under real-world working conditions. The users send
records of incidents with the system to the development organization
where the defects are repaired.
•

19
4. Acceptance Testing …
• Note that organizations may use other terms, such as factory
acceptance
testing and site acceptance testing for systems that are tested before
and after
being moved to a customer's site.

20