Automated testing overview

Automated testing overview
February 2012

Summary
• Why do we need software testing
• Software testing definition
• Types of testing
• Why do we need automated testing
• Unit testing for simple objects
• Unit testing for objects that have
dependencies
• Automated testing for user interface

Why do we need software testing
• To reduce operating costs : software bugs are a
big cost for the business;
• To protect company reputation : software
quality is a key requirement of any new project;
• any developer will do some kind of testing
before delivering the software to the client;
• software testing is a core activity associated
with the development process;

Software testing definition
• Definition: Software testing is an investigation
conducted to provide stakeholders with
information about the quality of the product
or service under test;
• software testing verifies and validates that the
software deliverable:
– meets the business and technical requirements that
guided its design and development;
– works as expected;

Types of testing
• system testing : exercises the application end
to end. Based on specifications and executed
using tests scripts as part of a test plan.
• integration testing : targets parts of the
application e.g. the service layer;
• unit testing : tests individual code units : e.g.
classes.

Cost of software bugs
Cost to fix a defect
Time detected
Requirem
ents
Architectu
re
Constructi
on
System
test
After
release
Time
introd
uced
Requirement
s
1x 3x 5-10x 10x 10-100x
Architecture - 1x 10x 15x 25-100x
Construction - - 1x 10x 10-25x
• the cost of removing a software bug is linked to the development stage the
software bug is found;

Difficulties of software testing
• It is a lengthy process and can happen only after the
alpha stage of the application;
• Constant retesting : After each release we need to
test new features and retest existing ones - even if
the changes affect only a part of the application we
can never be sure that the rest of the application is
unaffected;
• Usually the full test plan is applied right at the end of
the delivery cycle : regression errors or unexpected
errors can be detected too close to final delivery -
they can affect the delivery schedule;

Solution: automated tests
• Automate the test plan right from the beginning of
the development cycle : the developer can now
exercise the application before is deployed to
testing;
• System testing will always have a manual part but
can be also comprised of automated tests that can
cover a lot of scenarios
• Use unit testing and integration testing extensively
as they can be automated with minimal costs;

Why do we need automated testing
• TIME - speed up the delivery schedule : the test
window is reduced and testers can concentrate more
on exploratory testing and test cases that cannot be
automated;
• COST - reduce the number of bugs : bugs can be
detected before a deployment is made and
regression issues are reduced significantly;
• SCOPE - reduce the amount of rework and make
scope changes cheap: the testers, business analysts
and developers will start from test scenarios first
that will increase the product knowledge right from
the start;

How to implement automated tests
• The application need to be testable;
• System testing : coded ui tests using an automation
framework or using record and replay tools;
• Integrations tests : inject data that is used to exercise
the application layers for specific scenarios;
• Unit testing : they are code only tests so they don’t
have an impact on external systems and their
dependencies

What is a unit test
• Can I run and get results from a unit test I wrote two
weeks or even months or years ago?
• Can any member of my team run and get the results
from unit tests I wrote two months ago?
• Can I run all the unit tests I’ve written in no more than
a few minutes?
• Can I run all the unit tests I’ve written at the push of a
button?
• Can I write a basic unit test in no more than a few
minutes?

Properties of a good unit test
• It should be automated and repeatable.
• It should be easy to implement.
• Once it’s written, it should remain for future
use.
• Anyone should be able to run it.
• It should run at the push of a button.
• It should run quickly.

Styles of writing unit tests
• write production code first then unit tests;
• test driven development(TDD): write the unit
tests first then production code;
• behaviour driven development(BDD): start
from user stories and decompose them in
scenarios then write code to validate the
scenarios

Unit testing naming conventions used
in examples
Object to be tested Object to create on the testing side
Project Create a test project named [ProjectUnderTest].Tests.
Class For each class, create at least one class with the name
[ClassName]Tests.
Method For each method, create at least one test method with
the following name:
[MethodName]_[StateUnderTest]_[ExpectedBehavior]
.

Example 1
SimpleParserCustomTests

Unit testing framework - MsTest
• Automates unit testing;
• Provides an intuitive easy to use interface;
• Generates output that can be used by the
continuous integration process;
• to write a unit test you just need to use
– two attributes [TestClass], [TestMethod];
– a static function call Assert that validates the test;

Unit testing classes that have
dependencies

Example 2
LegacyLogAnalyser
• Requirements:
– create a class that analyses a log file;
– if the log file is invalid call a webservice and log
the problem;
– if an error is encountered send an email to an
administrator;

Controlling dependencies
• identify dependencies;
• isolate them by introducing interfaces;
• this process is referred to as making the class
under test testable;
• during test replace interfaces with objects
that you can control(replace the dependency
with a fake);
• this process is referred to as mocking the
dependencies;

Example 2
LogAnalyser
LogAnalyserTests

Mocking framework – Rhino.Mocks
• a mocking framework facilitates the generation
of fakes;
• there are two main types of unit testing;
– state-based unit testing;
– interaction-based unit testing;
• there are two type of fakes to match the two
types of unit testing: stub and mock;
• Rhino.Mocks is the most widely used open
source framework;
• MockReposity.GenerateStub<T> and
MockReposity.GenerateMock<T> are the
methods used to generate fakes;

Example 2
LogAnalyserTestsWithRhinoMocks

Conclusion
• Defined the place of unit testing in the wider
context of software testing;
• Defined properties of a good unit test;
• Introduction to an unit testing framework;
• Discussed about the need to isolate and
control dependencies during testing;
• Introduction to a mocking framework.

Automated testing overview

Recommended

More Related Content

What's hot (20)

Viewers also liked (6)

Similar to Automated testing overview (20)

Recently uploaded (20)

Automated testing overview

Editor's Notes