Hybrid framework for test automation

Hybrid Framework for Test Automation
by: Vinay Srinivasan
email: vinay.srinivasan@techmahindra.com
srinivasan_vinay@yahoo.com
Phone: +91-9823104620

© Tech Mahindra Limited 4/23/2013

Table of Contents Page
1.INTRODUCTION.......................................................................3
2.WHY TEST AUTOMATION FRAMEWORK...............................4
..................................................................................................... 5
3.GOAL OF TEST AUTOMATION FRAMEWORKS ....................5
4.DIFFERENT TYPES OF TEST AUTOMATION FRAMEWORKS .............................................................6
4.1Modular Framework............................................................................................................................ 6
4.2Data Driven Framework...................................................................................................................... 6
4.3Keyword Driven Framework................................................................................................................ 7
4.4Hybrid Framework............................................................................................................................... 7
5.HYBRID FRAMEWORK ARCHITECTURE................................7
6.HYBRID FRAMEWORK IMPLEMENTATION............................8
6.1Initialization......................................................................................................................................... 9
6.2Driver and Controller........................................................................................................................... 9
6.3Data Table........................................................................................................................................ 10
6.4Generic Functions............................................................................................................................. 10
6.5Utility Functions................................................................................................................................. 11
6.6Test Case Functions......................................................................................................................... 11
6.7Logger Functions.............................................................................................................................. 12
6.8Timer Functions................................................................................................................................ 13
7.CONCLUSION.........................................................................14


1. INTRODUCTION
Test Automation Framework is set of assumptions, concepts and practices that provide
support for automated Software testing. Basing an automated testing effort on using only
a record and playback test cases has its own drawbacks. Running complex and powerful
tests is time consuming and expensive when using only a capture tool. Because these
tests are created ad hoc, their functionality can be difficult to track and reproduce, and
they can be costly to maintain. A better choice for automation testing team is to use a
test automation framework, defined as set of assumptions, concepts and practices that
constitute a work platform or support for automated testing. There are few types of
Framework which are suitable for Test Automation. Out of this Hybrid Framework is
found to be the most suitable for Test Automation as it employs most of the features of
other Frameworks and is easy to implement. In following sections we will look into some
of the Frameworks used in the IT Industry and discuss about how to implement Hybrid
Framework for Test Automation.


2. WHY TEST AUTOMATION FRAMEWORK
Although applications are relatively unique, the components that comprise them, in
general, are not. Thus, we should focus our automation framework to deal with the
common components that make up our unique applications. By doing this, we can
remove all application-specific context from our framework and reuse virtually everything
we develop for every application that comes through the automated test process. Nearly
all applications come with some form of menu system. They also have buttons to push,
boxes to check, lists to view, and so on. In a typical automation tool script there is,
generally, a very small number of component functions for each type of component.
These functions work with the component objects independent of the applications that
contain them.

Traditional, captured automation scripts are filled with thousands of calls to these
component functions. So the tools already exist to achieve application independence.
The problem is most of these scripts construct the function calls using application-
specific, hard coded values. This immediately reduces their effectiveness as application-
independent constructs. Furthermore, the functions by themselves are prone to failure
unless a very specific application state or synchronization exists at the time they are
executed. There is little error correction or prevention built-in to these functions.

To deal with this in traditional scripts we must place additional code before andor after
the command, or a set of commands, to insure the proper application state and
synchronization is maintained. We need to make sure our window has the current focus.
We need to make sure the component we want to select, or press, or edit exists and is in
the proper state. Only then can we perform the desired operation and separately verify
the result of our actions.

For maximum robustness, we would have to code these state and synchronization tests
for every component function call in our scripts. Realistically, we could never afford to do
this. It would make the scripts huge, nearly unreadable, and difficult to maintain. Yet,
where we forego this extra effort, we increase the possibility of script failure.

What we must do is develop a truly application-independent framework for these
component functions. This will allow us to implement that extra effort just once, and


execute it for every call to any component function. This framework should handle all the
details of insuring we have the correct window, verifying the element of interest is in the
proper state, doing something with that element, and logging the success or failure of the
entire activity.

We do this by using variables, and providing application-specific data to our application-
independent framework. In essence, we will provide our completed test designs as
executable input into our automation framework. Does this mean that we will never have
to develop application-specific test scripts? Of course not. However, if we can limit our
application-specific test scripts to some small percentage, while reusing the best features
of our automation framework, we will reap the rewards project after project.

3. GOAL OF TEST AUTOMATION FRAMEWORKS
One of our goals should be a highly modular and maintainable framework. Generally,
each module should be independent and separate from all the other modules. What
happens inside one is of no concern to the others.

With this modular black-box approach, the functionality available within each module can
be readily expanded without affecting any other part of the system. This makes code
maintenance much simpler. Additionally, the complexity of any one module will likely be
quite minimal.

However, modularity alone will not be enough to ensure a highly maintainable
framework. Like any good software project, our design must be fully documented and
published. Without adequate, published documentation it will be very difficult for anyone
to decipher what it is the framework is designed to do. Any hope of maintenance will not
last far beyond the departure of the original framework designers. Our test automation
efforts will eventually become another negative statistic.

To prevent this, we should define documentation standards and templates. Wherever
possible, module documentation should be developed "in-context". That is, directly in the
source code itself. Tools should be retained, or designed and developed, so that we can
automatically extract and publish the documentation. This will eliminate the task of
maintaining two separate sets of files: the source code, and its documentation. It will also
provide those doing the code maintenance quite a ready reference. Nearly everything
they need to know should exist right there in the code.

We must always remember: our ultimate goal is to simplify and perpetuate a successful
test automation framework. To put something in place that people will use and reuse for
as long as it is technically viable and productive.


4. DIFFERENT TYPES OF TEST AUTOMATION

FRAMEWORKS
There are four main types of test automation frameworks as listed below.

4.1 Modular Framework
A collection of small, independent scripts represents modules/sections/functions
of the application under test. The modules are then used in a hierarchical or
logical fashion to construct larger test realizing an actual test case features in
QTP to support Modularity Framework like Reusable Actions and Functional
Libraries

4.2 Data Driven Framework
Data-Driven Framework is a methodology used in Test automation where test
scripts are executed and verified based on the data values stored in one or more
central data sources or databases. These databases can range from data pools,
ODBC sources, CSV files, Excel files, DAO objects, ADO objects, etc. Data-
driven testing is the establishment of several interacting test scripts together with
their related data results in a framework used for the methodology. In this
framework, variables are used for both input values and output verification
values: navigation through the program, reading of the data sources, and logging
of test status and information are all coded in the test script. Thus, the logic
executed in the script is also dependant on the data values.


4.3 Keyword Driven Framework
Keyword-Driven Framework is also known as table-driven testing, action-word
testing methodology. It is a technique particularly well suited to automate testing.
The advantages for automated tests are the reusability and therefore ease of
maintenance of tests, which have been created at a high level of abstraction. The
data table records contain the keywords that describe the actions we want to
perform. They also provide any additional data needed as input to the application,
and where appropriate, the benchmark information we use to verify the state of
our components and the application in general.

4.4 Hybrid Framework
This is the most successful automation framework and it generally
accommodates both keywords driven testing as well as data driven scripts. This
allows data driven scripts to take advantage of the powerful libraries and utilities
that usually accompany a keyword driven architecture. The framework utilities
can make the data driven scripts more compact and less prone to failure than
they otherwise would have been. The utilities can also facilitate the gradual and
manageable conversion of existing scripts to keyword driven equivalents when
and where that appears desirable. On the other hand, the framework can use
scripts to perform some tasks that might be too difficult to re-implement in a pure
keyword driven approach, or where the keyword driven capabilities are not yet in
place.

5. HYBRID FRAMEWORK ARCHITECTURE


6. HYBRID FRAMEWORK IMPLEMENTATION
The Hybrid Framework is implemented using the following Functions:


6.1 Initialization
This function is used to initialize the initial start up variables like Application Name
or Executable Name, Application Url or its path, log file names and its path, Data
Table name and its path, Application Login credentials like User ID and
Password. This initialization is accomplished using values from the Test Data and
Environment Values.

Initialization Function Diagram

Begin of Initialization

Get Environment Values
Get Connection to the Initialization Table

Set Values for:
Application Name
Application Path
User ID
Password
Log File Name
Log File Path
Test Data File Name
Test Data File Path

Close Connection to the Initialization Table

End of Initialization

6.2 Driver and Controller
This Driver Script calls initialization function to initialize the input data parameters.
It then calls a controller function, which has a Keyword Routine. It reads the
Keyword and Data from the Data Table and then calls the corresponding
Functions and Subroutines along with the input from the data table. The controller
function handles this feature of calling the functions separately. Once the
functions are executed, the control is then transferred back to the Controller for
the execution of next set of records based on the keyword in the table.

Driver Script Diagram

Begin of Driver

Call Initialization

Call Controller
Controller Function Diagram
End of Driver


Begin of Controller

Get Connection to the Data Table
Begin of Keyword Routine
Get Keyword from Data Table
Start Timer for Test Run
Log Test Name and Test Start Time

Initialize Boolean State to True

Handle Test Case Functions

Collect Test Result

Write Test Result to Data Table

Stop the Timer for Test Run
Log the End of the Test and Test End Time
Save the changes made to the Data Table
End of Keyword Routine
Close Connection to the Data Table

End of Controller

6.3 Data Table
This can be an Excel based table or a Database based table. Here each record
or row contains a keyword; execution condition like yes or no, set of test data and
this represents each test case that needs to be executed. It also contains
separate columns for updating the test case and the test result like pass or fail.

Sample Data Table
S.NoKeywordTest Case NameTest Data1Test Data2…ExecuteTest
Result1Ky1TC1……………….……..YesPass2Ky1TC2……………….
……..NoNoRun3Ky3TC3……………….……..YesFail

6.4 Generic Functions
These functions are very specific to the client application and are generic in
nature. The advantage here is that we can use these functions for similar client
applications. These Generic Functions include built in check points,
synchronization and wait property. It also initializes a Boolean State condition
based on the state of the objects present in the application under test. This is
useful for error and exception handling. These functions also use Logging
functions.


Generic Function Diagram

Begin of Generic Function G1

Check Boolean State

Set the Object

Check if the Object Exist

Reset the Boolean State to True or False

Log Object Exist or Does Not Exist

End of Generic Function G1

6.5 Utility Functions
These functions are very specific to actions that need to be carried out frequently
at beginning of test case, during the test case execution and at the end of the test
case. It includes specific Page, Link, Tab Navigation, Login and Logout. These
functions also use Generic and Logging functions.

Utility Function Diagram

Begin of Utility Function U1

Call Generic Function for the Object (Say G1)

If Boolean State is True then Continue

Perform Utility Action on the Object

Log the Performed Utility Action

Else Log Utility Action could not be Performed and Exit

End of Utility Function U1

6.6 Test Case Functions
These functions are very specific to the test cases and they utilize the generic
and utility functions. These functions perform the test case execution along with
the data passed on to them with by the Driver/ Controller. They also call custom
validation functions, which are specific to the test case/ test step validation. After
completing the tasks these functions, return the Test Execution Status as return
value to the Driver/ Controller. Based on this Status the Test Result is also
classified and it then populated into the Data Table.


Test Case Function Diagram

Begin of Test Case Function TC1

Start Test Case Execution Timer

Log the Test Case Name and Test Case Execution Start Time

Call Utility Functions

Call to Test Case/ Logic Implementation Functions

Call to Custom Validation Functions

Stop Test Case Execution Timer

Log the Test Case Execution End Time and Time Elapsed

Return the Test Execution Status as Pass/ Fail

End of Test Case Function TC1

6.7 Logger Functions
Logging functions are specific to creating logs for each step of action performed,
verification and validation points and for logging Errors. The Logs are broadly
classified into Step Logs and Error Logs. All the test case activities along with the
respective time stamps which include date and time of the occurrence of the
events are appended to Step Logs. Only Errors are reported in append mode into
the Error logs along with the respective time stamps which also include date and
time of the occurrence of the events. In case if the error logs are growing then it
is an indication that some thing is going wrong in the test execution. As the logs
are logged for each step it gives a clear idea about what was happening on each
step. Also in the event of an error, it will be easy to locate the step where the
error occurred, the nature of the error and the date and time at which the error
occurred.

Logger Function Diagram

Begin of Logging Function LF1

Get the String of Data that has to be appended to the Log File

Get Handle to the Log File

Append the String of Data to the Log File

Save and Close the Log File and free the Handle

End of Logging Function LF1


6.8 Timer Functions
The Timer functions consist of functions for date and time. It provides the time
stamp which consist of start date and time at the start and end of each Test Run
(can be Batch/ Test Suite) as well as for the beginning and end of each individual
Test Case Execution. In addition, they provide a summary of Total Time Duration
for each Test Run as well as for each individual Test Case Execution.

GetTime Function Diagram

Begin of GetTime Function GT

Get Current Time

Return Time in Hrs:Min:Sec String format

End of GetTime Function GT

GetDateTime Function Diagram

Begin of GetDateTime Function GDT

Use GetTime Function

Get Current Date

Convert Date to DD-MM-YYYY String format

Join the Date and Time in String format and Return the value

End of GetDateTime Function GDT

GetTimeDuration Function Diagram

Begin of GetTimeDuration Function GTD

Get Start Time in Integer format

Get End Time in Integer format

Calculate the Difference between Start and End Time

Return the Time Difference in Hrs:Min:Sec String format

End of GetTimeDuration Function GTD


7. CONCLUSION
The Conclusion is that Hybrid Framework is rugged, easy to implement, easy to use,
easy to expand, easy to maintain and easy to perpetuate. It is separate from the Test
Design and it is also Technology and Platform independent. Once implemented it
improves the speed of automating the Test Cases with the help of reusable libraries and
provides better Cost Benefits/ ROI within a short span of time.


Hybrid framework for test automation

Recommended

More Related Content

What's hot (20)

Similar to Hybrid framework for test automation (20)

More from srivinayak (13)

Hybrid framework for test automation