Introduction

Software Testing Computer programs are designed and developed by human beings and hence are prone to errors.
Unchecked Software, they can lead to a lot of problems, including social implications. Testing the software becomes an essential part of the software development lifecycle. Carrying out the testing activities for projects has to be practiced with proper planning and must be implemented correctly.

2013 QuEST Global, Inc. Confidential

Basic Questions on Testing

Why to test? Testing becomes absolutely essential to make sure the software works properly and does the work that it is meant to perform. Any working product which forms part of the software application has to be tested. Both data and programs must be tested. How often to test? When a program (source code) is modified or newly developed, it has to be tested. Who tests? Programmer, Tester and Customer.

2013 QuEST Global, Inc. Confidential

Definition: Software Testing

Software Testing is the process used to measure the quality of developed computer software. Software Testing means Finding an Errors in Software. Software Testing is stated as a process of Validating and Verifying that a Software program/application/product: meets the requirements that guided its design and development. works as expected. can be implemented with the same characteristics. satisfies the needs of stakeholders.

2013 QuEST Global, Inc. Confidential

TYPES OF TESTING

MANUAL TESTING Manual testing is the process of manually testing software for defects. It requires a tester to play the role of an end user. Testing that part of software testing that requires human input, analysis, or evaluation.

2013 QuEST Global, Inc. Confidential

TYPES OF TESTING

AUTOMATION TESTING Testing application with the help of tools. Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions . The Test Scripts are reusable, the process ensures time Saving, and mainly done when we have monotonous or repetitive task.

2013 QuEST Global, Inc. Confidential

Topics Covered

Software Problems and Objectives of Testing Quality Principles Software Development Life Cycle Software Life Cycle Models Testing Methodologies Software Testing Life Cycle Defect / Bug Life Cycle Test Automation Testing Certifications

2013 QuEST Global, Inc. Confidential

Most Common Software Problems And Objectives of Testing

Common Software Problems Incorrect Coding / Implementation of business rules. Confusing and misleading data. Incorrect File handling.

Objectives of Testing Executing a Program with the intent of finding an Error. To check if the System meets Requirements. A good test case is one that has a probability of finding an as yet undiscovered Error. A good test should neither be too Simple nor too Complex.

2013 QuEST Global, Inc. Confidential

Objectives of a Software Tester

To understand Application Requirements well. Finding bugs as early as possible and make they get fixed. Study the functionality in detail to find where the bugs are likely to occur. Create the Test cases in such a way that testing is done to uncover the hidden bugs. Ensuring that the Software is Usable and Reliable.

2013 QuEST Global, Inc. Confidential

QUALITY PRINCIPLES

Quality the most important factor affecting an Organizations Long-term performance. Quality the way to achieve improved productivity and competitiveness in an Organization. Quality saves. It does not cost. Quality is the solution to the problem, not a problem.

2013 QuEST Global, Inc. Confidential

Quality Assurance Vs. Quality Control

Quality Assurance (QA) A planned and systematic set of activities necessary to provide confidence that requirements are properly established, products and services conform to specified requirements. QA is the responsibility for entire team. Quality Control (QC) The process by which product quality is compared with applicable standards; and action taken when non-conformance is detected. QC is the responsibility for Testers.

2013 QuEST Global, Inc. Confidential

10

Verification and Validation

Verification process confirms that the Software meets its Technical Specifications. It involves Reviews and meeting to evaluate documents , plans, code requirements.

Validation process confirms that the Software meets the Business requirements. It involves actual testing and takes place after Verifications are completed. Verification and Validation process continue in a Cycle till the Software becomes defects free.

2013 QuEST Global, Inc. Confidential

11

Software Development Life Cycle (SDLC)

SDLC Software Development Life Cycle is step by step procedure to develop a software application. The Different phases of SDLC are : Requirement Collection Feasibility Study Design Coding Testing Installation Maintenance

2013 QuEST Global, Inc. Confidential

12

Different Stages in SDLC

Process of SDLC phases

REQUIREMENTS
Business Analyst (collects Requirement)

ANALYSIS
DESIGN

Finance Dept, HR, BA, Architect (Cost, Time, Resources, Project Doable or Not) H.L.D (Architect) L.L.D (Sr. Developer)

CODING TESTING IMPLEMENTATION MAINTENANCE

Developer

Test Engineer (Application is stable or not)

Installation Engineer

2013 QuEST Global, Inc. Confidential

13

Different Stages in SDLC

REQUIREMENTS: Requirements are set of functionalities and constraints that the end-user expects from the system. The requirements are gathered from the end-user by consultation . REQUIREMENT SPECIFICATION CRS [Customer requirement specification] SRS [Software requirement specification]

2013 QuEST Global, Inc. Confidential

14

Different Stages in SDLC

CRS This document will describe in detail , about what is expected out of the software product from the user's perspective. The wordings of this document will be in the same tone that of a user . SRS This document will describe in detail about what is expected out of the software product from the user's perspective. The wordings of this document will be in the same tone that of a developers which means software language.

2013 QuEST Global, Inc. Confidential

15

Different Stages in SDLC

ANALYSES: The requirements are analyzed for their validity and the possibility of incorporating the requirements in the system to be developed are studied. Finally, a Requirement Specification document is created which serves the purpose of guideline for the next phase of the model.

DESIGN: TWO TYPES

HLD: [High Level Design] Architecture of the project. LLD: [Low Level Design] Designing the each & every model of project in detail.

2013 QuEST Global, Inc. Confidential

16

Different Stages in SDLC

CODING: Converting the pseudo code into a programming language in the specified platform. TESTING: Verifying the functionalities of an project against requirements. INSTALLATION: Application has been installed at the clients place.

MAINTENANCE: It generally refers to warranty period given by the company to the client for the application.

2013 QuEST Global, Inc. Confidential

17

Software Life Cycle Models

WATERFALL MODEL SPIRAL MODEL PROTOTYPE MODEL V-PROCESS MODEL AGILE MODEL

2013 QuEST Global, Inc. Confidential

18

V V Model

CRS

Customer

SRS

Test Engineer

H.L.D

Test Engineer

L.L.D

Developer

Verification (Review Process)

Coding

Validation (Testing)

2013 QuEST Global, Inc. Confidential

19

V-Shaped Steps

Project and Requirements Planning: Allocate resources. Product Requirements and Specification Analysis: Complete specification of the software system. Architecture or High-Level Design: Defines how software functions fulfill the design. Detailed Design: Develop algorithms for each architectural component. Production, operation and maintenance: Provide for enhancement and corrections. System and acceptance testing: Check the entire software system in its environment. Integration and Testing: Check that modules interconnect correctly. Unit testing: Check that each module acts as expected. Coding: Transform algorithms into function specification.

2013 QuEST Global, Inc. Confidential

20

Testing Methodologies

Types Of Testing

White Box Testing (Unit Testing)

Black Box Testing (Test Engineer)

Grey Box Testing

Unit Testing: It is small program which is written by the developers that in tern test the main program.

2013 QuEST Global, Inc. Confidential

21

Testing Methodologies

White Box Testing It is the type of testing done by the developers where they test each and every line of code written by them before handing over the product to the test engineer. Black Box Testing It is type of testing done by the Test Engineer where they check if the application is working according to the requirements document.

White Box testing Generally it is done by the Test Engineer who has knowledge Of both coding as well as testing.

2013 QuEST Global, Inc. Confidential

22

Types of White Box Testing

Path Testing In this type developers test for the complete flow of the programs or path of the programs & total number of paths in the program . Conditional Testing In this type developers check if all the conditional statements written in the program are executed for both true as well as false conditions. Loop Testing (Memory and Response Time) In this type developers check if all the looping conditions written in the program are executed for the complete number of cycles with these types developers also test the memory and response time w. r. t code.

2013 QuEST Global, Inc. Confidential

23

How to do Testing and its Life cycle

Testing Objectives: Testing is a process of executing software with the intent of finding errors Good testing has a high probability of finding as-yetundiscovered errors

Successful testing discovers unknown errors

If did not find any errors, need to ask whether our testing approach is good Goal of testing: given some code, uncover as many errors are possible

2013 QuEST Global, Inc. Confidential

24

How to do Testing and its Life cycle

Major Testing Methods: Unit Testing Sanity Testing Functional Testing Integration Testing UAT Testing System (End to End ) Testing Retesting Regression Testing Globalization Testing

2013 QuEST Global, Inc. Confidential

25

Unit and Sanity Testing

Unit Testing Is a level of the Software Testing process where individual units/components of a Software/System are Tested. A unit is the smallest testable part of Software. It covers testing on a specific part of code perform by developers . Sanity Testing /Smoke Testing Testing basic functionality of an application Or Critical features of an application. Its very basic level Testing done by QA after fixing some bug.

2013 QuEST Global, Inc. Confidential

26

Black Box Testing

Test the functionality of the application against requirement specification. Types of black box testing Functional Testing Integration Testing System Testing

Requirement

Coding White Box Testing

Functional Testing

Integration Testing

System Testing

2013 QuEST Global, Inc. Confidential

27

Black Box Testing

Functional Testing Testing each and every component of an application of an application and each and every module independently is called as functional testing. For functional testing we require: Requirements, Application Points to remember while doing functional testing. Testing application with positive inputs first. Then we should do negative testing. Dont do Over testing /Exhaustive testing. Do not assume requirements by oneself, No assumptions. Under testing, if two components are same.

2013 QuEST Global, Inc. Confidential

28

Black Box Testing

Integration Testing Testing the data flow or integration from one module to another module is Integration Testing. Points to remember while doing Integration testing: Complete the functional testing on all the features first. As an T.E, one should have complete knowledge of the modules and data flow between modules. Do not do the Integration testing randomly, whereas pick each and every module and complete the data flow from that module to all other modules wherever the data flow is available.
2013 QuEST Global, Inc. Confidential

29

Black Box Testing

System Testing Testing the end to end flow of an application is called as system testing. In system testing the test environment is similar to product environment, similar w. r. t software, hardware, data. To begin our testing one should have Required documents. Application should be installed on to the test server. Test URL should be given.

2013 QuEST Global, Inc. Confidential

30

UAT and Exploratory Testing

User Acceptance Testing (UAT) The client will test it, in their place, in a near-real-time or simulated environment. AT is a type of testing done by engineers sitting in it division of customer place. Exploratory Testing Checking application functionalities without document, Explore the application. Test Design and Test Execution at the same time.

2013 QuEST Global, Inc. Confidential

31

Difference of Re test and Regression Testing

Re-test Testing only certain part of an application again and not considering how it will effect in the other part or in the whole application. Testing the bug again once the code is fixed to ensure that the particular functionality is working fine. Regression Testing Testing the application after a change in a module or part of the application for testing that is the code change will affect rest of the application. Testing some other test cases/functionality also to ensure that the above fix havent introduce any problem to existing code/functionality which was working fine earlier.
2013 QuEST Global, Inc. Confidential 32

Globalization Testing

Globalization Testing the applications for different languages is called as globalization testing. Globalization Testing is validating a specific Application or Product that can support multiple languages or that can be accessed by different Locales around the Globes.

Two major Components Internationalization & Localization. Internationalization (I18N) : Testing the application for multiple languages using property files. This type of testing is done for multiple languages using suffix and prefix. Localization (L10N) Testing the application as per country standard is called localization(L10N) testing e.g.. pin code, std code , currency.

2013 QuEST Global, Inc. Confidential

33

Software Testing Life Cycle (STLC)

Software Development and Software Testing Life cycles go parallel

2013 QuEST Global, Inc. Confidential

34

Test Plans

Test Plan - Derives future activities of Testing process. A Test Plan should be Overall coverage of Test to be conducted useful and Understandable by the people Inside and Outside the Test group. 1. 2. 3. 4. 5. 6. 7. Test Plan Consists of following attributes. Objective 8.Risk Scope 9.Mitigation/ Contingency Plan Test Approach 10.Enviornment(H/W, S/W) Test Methodology 11.Automation tools Roles & Responsibility 12.Effort Estimation Schedule 13.Entry & Exit Criteria Assumption 14.Test Deliverables

2013 QuEST Global, Inc. Confidential

35

Test Cases / Use cases

Test Case is a Collection of Test steps also call it as a Design Case.

Test Case Template should contains 3 parts : 1. Header 2. Body 3. Footer

Use Case can be a basis for deriving the tests for a System. UML (Unified Modeling Language)is the facilities for drawing the Use case diagrams. It is used during the Analysis phase of Project, they separate the System into Actors and Use Cases.
2013 QuEST Global, Inc. Confidential

36

Test Case Template

2013 QuEST Global, Inc. Confidential

37

Test Case Design Techniques

Involves designing the Test cases (Inputs/Outputs) used to test the System.

The goal of Test case design is to create a set of tests that are effective in Validation and Defect Testing.
Design Approaches

Error Guessing In this approach the test engineer keeps on guessing the values according to his understanding
Equivalence Partitioning If input is range of values then derive the test case for one valid and two invalid data. If the input is set of values then derive the test case for one valid and two invalid inputs.
2013 QuEST Global, Inc. Confidential 38

Testing Approaches

White-box testing
Control-flow-based testing
Data-flow-based testing

Black-box testing
Equivalence partitioning

2013 QuEST Global, Inc. Confidential

39

Control-flow-based Testing

From the source code, create a graph describing the flow of control Called the control flow graph The graph is created (extracted from the source code) manually or automatically. Design test cases to cover certain elements of this graph: Nodes, edges, paths

s:=0; d:=0;

1 2 3 4 5 6 7

while(x<y) {
x:=x+3; y:=y+2; if x + y < 100

s:=s+x+y
else d:=d+x-y }
8

2013 QuEST Global, Inc. Confidential

40

Elements of a control Flow Graph

Three kinds of nodes

Statement nodes: single-entry-single-exit sequence of statements Predicate(decision) nodes: conditions for branching Auxiliary nodes: (optional) for easier understanding (e.g. merge points for IF) Edges: possible flow of control
IF-THEN, IF-THEN-ELSE, SWITCH

2013 QuEST Global, Inc. Confidential

41

IF-THEN, IF-THEN-ELSE, SWITCH

If (c) then

If (c) then

switch (c) case 1: . case 2: join point

join point

else
join point

2013 QuEST Global, Inc. Confidential

42

EXAMPLE

switch(position) case CASHIER: if(empl_yrs > 5) bonus := 1; else bonus := 0.7; break; case MANAGER: bonus := 1.5; if (retiring_soon) bonus := 1.2 * bonus break; case . endswitch

2013 QuEST Global, Inc. Confidential

43

Mapping for Loops

While (c) { . }

Note: other loops (e.g. FOR,DO-WHILE, ) are mapped similarly

Mini-assignment: figure out how this is done

2013 QuEST Global, Inc. Confidential

44

Statement Coverage
Given the control flow graph, define a coverage target and write test cases to achieve it Traditional target: statement coverage Test cases that cover all nodes Code that has not been executed during testing is more likely to contain errors Often this is the low-probability code

2013 QuEST Global, Inc. Confidential

45

EXAMPLE

Same example as before: two test

cases Path 1-2-exit Path 1-2-3-4-5-7-8-2-3-4-5-7-8-2exit Problem: The false branch of 4 is never taken dont have 100% branch coverage

2
3 4

T 5

F 6

7 8
2013 QuEST Global, Inc. Confidential 46

Branch Coverage

Target: write test cases that cover all branches of predicate nodes
True and false branches of each IF The two branches corresponding to the condition of a loop All alternatives in a SWITCH In modern languages, branch coverage implies statement coverage

2013 QuEST Global, Inc. Confidential

47

Branch Coverage

Statement coverage does not imply branch coverage

Example: if (c) then s; By executing only with c=true, we will achieve statement coverage, but not branch coverage

Motivation: experience shows that many errors occur in decision making

Plus, it subsumes statement coverage

2013 QuEST Global, Inc. Confidential

48

EXAMPLE

Same example as before: two test cases Path 1-2-exit Path 1-2-3-4-5-7-8-2-3-4-5-7-82-exit

2
3 4

Problem: the false branch of 4 is never taken dont have 100% branch coverage

T 5

F 6

7 8
2013 QuEST Global, Inc. Confidential 49

Achieving Branch Coverage

Branch coverage: a necessary minimum Pick a set of start-to-end paths that cover all branches, and write test cases to execute these paths Basic strategy Add a new path that covers at least one edge that is not covered by the current paths

Sometimes the set of paths chosen with this strategy is called the basis set

2013 QuEST Global, Inc. Confidential

50

Data-flow-based Testing
Test connections between variable definitions (write) and variable uses (read) Variation of the control flow graph A node represents a single statement, not a single-entry-single-exit chain of statements Set DEF(n) contains variables that are defined at node n (i.e., they are written) Set USE(n): variables that are read

2013 QuEST Global, Inc. Confidential

51

EXAMPLE

Assume y is already initialized

1 s:=0 2 x:=0

3 4 5 6 7 9 10 8

3 while(x<y) { 4 x:=x+3; 5 y:=y+2; 6 If(x+y<10) 7 s:=s+x+y; else 8 s:=s+x-y; }

2013 QuEST Global, Inc. Confidential

52

Black-box Testing

Unlike white-box testing: dont use any knowledge about the internals of the code Test cases are designed based on specifications

Example: search for a value in an array

Post condition: return value is the index of some occurrence of the value, or -1 if the value does not occur in the array

2013 QuEST Global, Inc. Confidential

53

Equivalence Partitioning
Consider input/output domains and partition them into equivalence classes For different values from the same class, the software should behave equivalently Test values from each class Example for input range 2..5: less than 2, between 2 and 5 and greater than 5 Testing with values from different classes is more likely to find errors than testing with values from the same class

2013 QuEST Global, Inc. Confidential

54

Equivalence Classes
Examples Input x in range [a..b]: three classes x<a, a<=x<=b, b<x

Boolean: classes true and false

Some classes may represent invalid input Choosing test values

Choose a typical value in the middle of the class(es) that represent valid input
Choose values at the boundaries of classes E.g. for [a..b], use a-1, a, a+1, b-1, b, b+1

2013 QuEST Global, Inc. Confidential

55

Example

Spec says that the code accepts between 4 and 24 inputs; each is a 3digit integer One partition: number of inputs Classes x<4, 4<=x<=24, 24<x Chosen values: 3,4,5,14,23,24,25

Another partition: integer values

Classes x<100, 100<=x<=999, 999<x Chosen values: 99,100,101,500,998,999,1000

2013 QuEST Global, Inc. Confidential

56

Another Example

Similarly for the output: exercise boundary values Spec: the output is between 3 and 6 integers, each in the range 10002500 Try to design inputs that produce 3 outputs with value 1000 3 outputs with value 2500 6 outputs with value 1000 6 outputs with value 2500

2013 QuEST Global, Inc. Confidential

57

Example: Searching

Search for a value in an array. Return: index of some occurrence of the value, or -1 if the value does not occur One partition: size of the array. Programmer errors are often made for size 1: a separate equivalence class. Classes: empty array, array with one element, array with many elements. Another partition: location of the value, first element, last element, middle element, not found.

2013 QuEST Global, Inc. Confidential

58

Array empty [7]

Value 5 7

Output -1 0

[7]
[1,6,4,7,2] [1,6,4,7,2] [1,6,4,7,2] [1,6,4,7,2]
2013 QuEST Global, Inc. Confidential

2
1 4 2 3

-1
0 2 4 -1
59

Defect / Bug Life Cycle

What is Defect ? A Defect is a variance from a desired product attribute. Two Categories of Defects are Variance from Product Specification. Variance from Customer / User expectations. Why Defects occur ? Wrong Implementation Missing Implementation Extra Implementation

Defect / Bug Tracking Tools Bugzilla Rational Clear Quest Quality Center(QC) etc.

2013 QuEST Global, Inc. Confidential

60

Defect / Bug Classification

Based on Priority
Defect Priority Critical High Guidelines Must be fixed immediately. Serious effect on Testing. Must be fixed before testing has completed.

Medium Low

Fixed if possible before application implemented in production. Fixed if time available.

2013 QuEST Global, Inc. Confidential

61

Defect / Bug Classification

Based on Defect Severity

Defect Severity Show Stopper Major Guidelines Almost the entire test activities suspended due to fault. System un-usable The test Activities for a module is been suspended. Major module of system unstable. The test script is suspended but other testing process can be continued. Individual functions affected and individual users prevented from completing tasks. Minor fault or Cosmetic Error that has no impact on testing schedule. Users can continue working. Action All testing stopped until fixed All testing for this module is stopped A single test stopped until fixed

Minor

Trivial Error/Cosmetic

No testing stopped

2013 QuEST Global, Inc. Confidential

62

Defect Status
Defect Life cycle/ Bug Life cycle comprises of all the defect status changes it would under go once a new defect is logged and till the defect is closed or cancelled.

Defect Life Cycle

2013 QuEST Global, Inc. Confidential

63

Defects Status Explanation

NEW defect is one of ASSIGNED, DROPPED and DEFERRED. ASSIGNED/OPEN : Test /Development/Project lead studies the NEW defect and if it is found to be valid, it is assigned to a member of the Development Team. The assigned Developers responsibility is now to fix the defect and have it COMPLETED. DEFERRED : If a valid NEW or ASSIGNED defect is decided to be fixed in upcoming releases instead of the current release it is DEFERRED. This defect is ASSIGNED when the time comes. PROPPED/REJECTED : Test/Development/Project Leads approaches the NEW defect and if it is found to be invalid, it is DROPPED/REJECTED .
2013 QuEST Global, Inc. Confidential 64

Defects Status Explanation

DROPPED/REJECTED : Test/Development/Project Leads approaches the NEW defect and if it is found to be invalid, it is DROPPED/REJECTED . COMPLETED/FIXED/RESOLVED/TEST : Developer fixes the defect that is ASSIGNED to him or her. Now, the fixed defect needs to be verified by the Test Team and the Development Team assigns the defect back to the Test Team. A COMPLETED defect is either CLOSED, if fine , or REASSIGNED, if still not fine.

2013 QuEST Global, Inc. Confidential

65

Defects Status Explanation

If a Developer cannot fix a defect , some Organizations may offer the following statuses : Wont Fix/Cant Fix : The Developer will not or cannot fix the defect due to some reason Cant Reproduce : The Developers is unable to reproduce the defect. Need more Information : The Developer needs more information on the defect from the Tester.

REASSIGNED / REOPENED : If the Tester finds that the fixed defect is in fact not fixed or only partially fixed, it is reassigned to the Developer who fixed it. A REASSIGNED defect needs to be COMPLETED again. CLOSED/VERIFIED : If the Tester /Test Lead finds that the defect is indeed fixed and is no more of any concern, it is CLOSED/VERIFIED.

2013 QuEST Global, Inc. Confidential

66

Severity Vs. Priority

Severity Factor that shows how bad the defect is and the impact it has on the Product. Severity Levels :
Critical Major / High Average / Medium Minor / Low Cosmetic defects

Priority Based on input from Users regarding which defects are most important to them, and to be fixed first.

2013 QuEST Global, Inc. Confidential

67

Test Automation

Automation Is the process of writing a computer program to do Testing. Once tests have been automated, they cab be run quickly and repeatedly. Doing any task using a Tool without any Manual intervention . Automation increases Speed of Test Execution. Automation helps increase Test Coverage. Ex: AutoCAD. Various Automation tools available in market are Why Automation ? Faster Reduces Human efforts Reliable and Consistency Reusable

2013 QuEST Global, Inc. Confidential

68

Test Automation

Some Test Automation Tools Tool Name Produced By

HP Quick Test Professional

IBM Rational Functional Testes Silk Test Selenium Robot Framework Sikuli

HP Software Division
IBM Rational Borland Open Source Open Source Open Source

2013 QuEST Global, Inc. Confidential

69

Software Testing Certification

IEEE Standards (Institute Of Electrical and Electronics Engineers) designed an entire set of standards for Software and to be followed by the Testers. IEEE Standard for Software Quality Assurance Plan IEEE Standard for Software Unit Testing IEEE Standard for Software Management IEEE Standard for Software Verification and Validation IEEE Standard for Software Quality Metrics Methodology

2013 QuEST Global, Inc. Confidential

70

Other Standards

ISO International Organization for Standards Six Sigma Zero Defect Orientation NIST National Institute of Standards and Technology ISTQB International Software Testing Qualification Board

2013 QuEST Global, Inc. Confidential

71

Conclusion

Software process are the activities involved in producing and evolving a Software System. Testing can show the presence of faults in a System ; it cannot prove there are no remaining faults. General activities are specification, design ,implementation, Validation and evolution. Developers are responsible for Unit Testing , System Testing is the responsibility of a Testing Team. Test Automation reduces Testing costs by supporting the test process with range of Software Tools.

2013 QuEST Global, Inc. Confidential

72