Chapter 13: Software testing - Testing is the process of exercising a program with the specific intent of finding errors

prior to delivery to the end user. What is it - Software is tested to uncover errors that were made as it was designed an constructed Why it is important - To detect errors at an early stage What is the product - Test specification documents the teams approaches to testing by defining a plan that describe over all strategy Testing Strategic characteristics 1. software team conduct effective formal technical review 2. testing begins at the components level and works the integration of the entire compute based system 3. Testing is conducted by developer or by independent team 4. testing and debugging are different activities but debugging must accommodated in any testing strategy 5. it must accommodate low level tests that are necessary to verify that a small source code segment has been correctly implemented Verification an validation - verification, it refers to a set of activities that ensure that software correctly implemented - validation, it refers to asset of activities that check if system has been according to customer requirements Organizing for software testing Software developer is always responsible for testing individual component of the system Independent test group(ITG) is to remove the inherent problem associated with letting the builder test the thing that has been build Software testing architecture It may appears as a spiral that starts as following from inner to outer 1. unit testing, in concentrates on testing source code for each unit in the system 2. code testing 3. integration testing 4. validation testing 5. system testing What are testing 1. performance constraints 2. information domain 3.function 4.behavior 5.

Strategic Issues for testing State testing objectives explicitly.

 Understand the users of the software and develop a profile for each user category. Develop a testing plan that emphasizes rapid cycle testing. Build robust software that is designed to test itself Use effective formal technical reviews as a filter prior to testing Conduct formal technical reviews to assess the test strategy and test cases themselves. Develop a continuous improvement approach for the testing process. Unit testing It focuses on verification effort on smallest unit of software design( components or modules) Unit testing consideration The tests that occur as a part of unit testing includes 1.interface, to ensure that information properly flow into and out of program unit under test 2.local data structure, to ensure integrity during all steps in algorithm execution 3.boundary conditions 4.independent paths, control structure paths are tested 5.error handling paths Unit testing procedure - component is not a stand alone program so driver and/or stub software must be developed for each unit test - driver not more than main program that accept test case data, passes data to component - stubs serve to replace modules that are subordinate to the components to be tested(module interface) - drivers and stubs represent over head. That is both are software must be written and not delivered with product Integration testing - Is a systematic techniques for constructing the software architecture while at the same conducting tests to uncover errors associated with interface - The objective it to take unit testing components and build a program structure that has been ducted by design - There is a tendency to test the program in a whole( non-technical integration), a set of error may encounter so correction them are difficult - Incremental integration , the program is constructed and tested in a small elements or increments Top down integration - its is an incremental integration, modules are integrated by moving downward through the control hierarchy, beginning with the main control module Top down Integration process steps 1. the main control module I used a test driver, and stubs are substituted for all components 2. depending on the integration approach selected( depth or breadth) stubs are replaced

3. tests are conducted as each component is integrated 4. on completion of each set of tests another stub is replaced with real component 5. regression test may be used to ensure that new errors have not been introduced Bottom up integration Steps 1. low level component are combined into clusters( or builds) 2. a driver is written to coordinate teats case input and output 3. the cluster is tested 4. drive are removed and cluster combined moving upward in the program structure Regression test - re-execution of some subset of tests that have already been conducted to ensure that changes not propagated unintended side effect - Regression classes 1.representative sample of tests that will exercises all software functions 2.additional tests that focus on functions that were affected by change 3.tests that focus on components that have been changed Smoking testing Smoking test includes 1. software components that have been translated into code are integrated into build. Build includes all data files, libraries, reusable modules 2. series of tests is designed to expose errors that will keep the build properly performing its functions 3. build is integrated with other builds and the result is smoke tested daily Smoking test advantages 1. integration risk is minimized, because smoke tests are conducted daily 2. quality of end product is improved 3. error diagnoses and correction are simplified 4. progress is easier to assess Strategic option - main disadvantage for top down is using stubs - main disadvantage for bottom up program dose not exist until added the last module - selecting strategy based on the characteristics of software and sometimes project schedule - sandwich testing, it combines top down with bottom up. At upper level ii uses top down and at lower level used bottom up Critical modules features 1. address several software requirements 2. has a high level of control 3. is complex or error prone 4. has defined performance requirements

Criteria that applied to all test phases 1. interface integrity, internal and external interfaces are tested 2. functional validity, tests designed to uncover errors are conducted 3. information content, tests are designed to uncover error associated with local or global data structure 4. performance - Test specification, it includes test plan, test procedure and so on , it considers as work product of the software process Object-Oriented Testing begins by evaluating the correctness and consistency of the OOA and OOD models testing strategy changes the concept of the unit broadens due to encapsulation integration focuses on classes and their execution across a thread or in the context of a usage scenario validation uses conventional black box methods two different approaches are used for integration testing 1. thread base testing, integrates the set of classes required to response to one input or event for the system 2. use base testing begins the construction of the system testing those classes that use very few server classes 3. cluster testingintegrates the set of classes required to demonstrate one collaboration

Validation testing - validation, it refers to asset of activities that check if system has been according to customer requirements - after each validation test two possible conditions exist 1. performance conforms to specification 2. deviation from specification is uncovered and deficiency list is created - Audits or configuration review, its a review to ensure all elements configuration have been properly developed - Alpha test is conducted at developer site by end users - Beta test is conducted at end user site System testing - is actually a series of different test whose primary purpose is to fully exercise the computer based system - Different type of testing are used some of them are 1. Validation testing: Focus is on software requirements 2. System testing: Focus is on system integration 3. Alpha/Beta testing: Focus is on customer usage

4. Recovery testing: forces the software to fail in a variety of ways and verifies that recovery is properly performed(MTTR is evaluated) 5. Security testing: verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration 6. Stress testing: executes a system in a manner that demands resources in abnormal quantity, frequency, or volume 7. Performance Testing: test the run-time performance of software within the context of an integrated system The Art of debugging - Debugging occurs as a consequence of successful testing. That is when a test case uncover an error, debugging is an action that results in the removal of the error Debugging process - it is not a testing but always occurs as a consequence of testing. Debugging begins with the execution of a test case - It attempts to match symptom with cause, thereby leading to error correction - Debugging has one of two outcomes 1. the cause will be found and corrected 2. the cause will not be found Why debugging difficult 1. the symptom and cause may be geographically remote 2. symptom may disappear when another error is corrected 3. symptom may actually be caused by no errors 4. symptom may actually be caused by human errors that is not easily traced 5. symptom may be a result of timing problem rather than processing problems 6. the symptom may be due to cause that are distributed across of tasks running on different processors Psychological consideration - There appears to some evidence that debugging process is an innate human trait. Some people are good at it and other are not Debugging strategic - main objective is to find and correct the cause of errors - debugging strategies are[ brute force, backtracking , and cause elimination] Brute force - most common and least efficient to isolate cause of error, which is applied when all else fail to discover - memory dump are taken, run time traces are invoked, and program is loaded with an output statement Backtracking - common methods used in small programs. Source code is traced until the cause is found Cause elimination

- it is manifested by induction and deduction and introduce the concept of binary partitioning - data related to the error are organized to isolate potential cause - a list of causes are is developed and tests are conducted to eliminate each of them Note : automated debugging and people factor are powerful factors in debugging process Correcting errors Three important questions should be answered when correcting errors 1. is correcting this produce new one 2. what next bug might be introduced by correcting this error( high coupling as an example) 3. what could we have done to prevent this bug in the first place( in current and future programs)