Scribd
Upload
32 views

Basics of Software Testing

Automated testing helps improve the efficiency and coverage of testing software. Both manual and automated testing are important, as is using both black-box and white-box testing approaches. To evaluate the quality of a test suite, metrics like code coverage and mutation analysis are used to ensure the tests sufficiently cover the code and can detect bugs when the code is mutated. Testing specifications like pre-conditions and post-conditions can make tests more executable and check facets of the code without needing precise specifications.

Uploaded by

Brian Mwanzia
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
32 views

Basics of Software Testing

Automated testing helps improve the efficiency and coverage of testing software. Both manual and automated testing are important, as is using both black-box and white-box testing approaches. To evaluate the quality of a test suite, metrics like code coverage and mutation analysis are used to ensure the tests sufficiently cover the code and can detect bugs when the code is mutated. Testing specifications like pre-conditions and post-conditions can make tests more executable and check facets of the code without needing precise specifications.

Uploaded by

Brian Mwanzia
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

The Basics of Software Testing

10 August 2023 09:16

Relationship of testing to development, classifying testing methods, identifying specifications and


measuring testing quality.

The process:
- specifications are explicit.
- development and testing are independent (dev implements, tester specifies though dev may
still specify).
- resources are finite and priority is key when testing.
- specifications evolve thus tests must update.

Testing is consistency checking between implementation and specifications.


What about away to automate the process of writing or running tests? Automated testing.

Testing Approaches
Automated Fuzzer to run Feedback directed
commands on GUI random testing to GUI,
symbolic execution with
static analysis and code
monitoring
Manual Exercise different Looking at source code
GUI events to determine possible
rights
Black-box White-box

Testing cannot be fully manual or fully automated. Neither fully black-box or white-box.

Automated testing Manual testing


Quick bug report Efficient
Autogenerated test scripts Potentially better coverage
Auto-maintenance

Black-box White-box
Does not require code Efficient test suite
modification
Doesn't need to analyse Potentially better coverage
code - a details rabbit hole
Can be performed on any
code format

Example
- Monitoring network activity on blackbox to detect malicious network calls. Inspect source or
binary code on whitebox to see calls to malicious network calls.
Automated Testing can be hard for entire complex systems.

Specification mechanisms: Frame conditions


Frame conditions infer only based on condition set and no other information.
Pre-condition - hold before function executes. Done on input.
Post-condition - hold after function executes and whenever pre-condition also holds. Done on
output.
Factors:
- Most useful when executable
- Need not to be precise, can check certain facets only: May become more complex than code
Usage
Pre-conditions may include null checks and/or length of two related arrays.

Condition Function
B is non-null B != null
B is same length as A B.length == A.length
Elements of B are in sorted order for (int i = 0; i < B.length-1; i++) B[i] <= B[i+1];
Elements of B are a permutation of the /* count no. of occurrences of each number in each array and
elements of A: DIY compare these counts */

How Good is your Test Suite?


Too few tests may miss bugs. Too many tests are costly to run, bloat and redundantly hard to
maintain.

Approaches of Measuring Test Suite Quality


- Code coverage metrics - quantifies the extent to which a program's code is tested by a given
test suite. As a percentage of some aspect of the program. 100% coverage is rare in practice
due to some inaccessible code apart from safety critical apps.
○ Function coverage - functions called
○ Statement coverage - statements executed
○ Branch coverage - branches taken

- Mutation analysis/testing - assumes program is close to being correct or begin with thus tests
variations/mutants of the program i.e. Switching assignment or arithmetic operators. A good
suite should report failed tests in the mutants. Issue arises when mutants created are
equivalent to the original therefore no test will kill it. Inequivalence of programs is undecidable
and requires manual intervention to confirm that programs are not equivalent.

Testing is important.

You might also like