Automation testing: how tools are important?

Automation testing: how tools are important?
Automation testing is a buzzword in today’s testing discussion. Whenever anyone met with
buddy from other testing organization there will be very common chat “how do you automate
your testing task; which tools you are using and so on”.
Today I am going to write on this discussion and how fruitful this automation; machine base
testing.
There are two types of testing tools available to implement; one from different vendors and
other from open source. I guess I have left one more type that is tools using different language
and for this type there is no barrier. As per the testing needs we can build the tools/script.
Testing is a sequence of actions and cannot be accurately executing by testing tools. There
needs to be intervention of human action.
Vendor tools: Scripts are created using record and even can write manually and these scripts
are always vendor defined structured e.g. C like, VB like and so on. There are so many vendor’s
namely Mercury now HP Mercury, Segue Inc., IBM. These tools are easy for the non-
programmers since scripts are creating by record and play back system.
These vendor’s tools are very expensive and all organization cannot afford of these specially the
load test tools.
I have read the article from Bret Pettichord “Hey Vendors, Give Us Real Scripting Languages”
and I was commenting: Vendors can easily make standard language base scripting and that will
be benefited both for vendor and tester. Only drawback is; they will not be able to earn money by
certification :)
Now we can use few tools for free: WATIR, WATIN, Selenium and these are really good for
scripting.

Still I will say these tools are useful and if we cannot afford of these we have choices :)
Open source tools: Watir, WatiN, Selenium, iMacro, JMeter and so on. These tools are very
popular in the automation testing world since it’s FREE. We have our choice for functional
regression test as well performance test. We are happy to use these tools and now we are not
depending on vendor’s tools.
There is drawback for having knowledge only on open source tools that is many big
organization wish to have test engineer with vendor’s certification and they will go for those
vendor base testing tools but fact is it’s not always. Personally I am happy with these open
source tools but sometime got weakness for vendor tools since I have started with those tools
in my CSTP courses. Another drawback of open source tools is having bug or limitation since its
open and there is no dedicated team to support it. We need to grow the community to help the
open source tools more mature.
Programming language: different programming languages used for specific test; now days this
is common practice to build different tools for testing purpose and other time creating scripts
for resolving the harness of vendor and open source tools.
How these tools are important: One of the most important point for automation tool is it can
be very useful for repetitive tasks. Data entry type job could be manageable with least effort
and easily. For this type of task we can use iMacro; this is very easy to generate the scripts and
change it as needed and run for the next time. It saves time while regression test since the
automation tools take care of the maximum scenario coverage as we have prepared earlier.
But if we like to do some complex functional job with automation test tools; we need an expert
in any tools. Because logics need to be implemented and writing the scripts from one to
another tool vary. So, first of all we need to select the proper tools for the targeted job done.
Changes in the code are another vital point for doing automated tests; how frequently the
codes and UI are changing. If there are frequent major changes then the test scripts need to be
work out for working with the changes.
I have used Watir for one of our big project; its fun. But problem comes e.g. I have used some
XPath since there was no ID or NAME to capture the steps. Later what happened, the HTML &
CSS are changed and now my XPath is not working. It’s simply horrible, my XPath is getting the
pages it’s supposed to not.
One thing I have noticed; if we go for automation testing then development time itself there
should be test suitable code so that it will be easier to do write test script. Otherwise we have
to face those problems and it’s easy to minimize if we have taken care earlier.
Things to consider for automation test:

Test plan & early start: If we plan to go for automation testing; first thing to considered is test
plan. We must include development team and overview of the importance of the automation
test and how it will be effective. The plan and the meeting should be as early as possible before
the projects start so that it could come up with the resolution as I have mentioned earlier. Late
start always makes problem since it’s always better something to change earlier stage of the
project. In the test plan itself we could come with tools selection, talents assignment, time
estimation and targeted plan for the automation task.
100% automation test: I am doing testing job last 6 years and 90% of my execution was manual
and planning, test case writing, project/task assigning. Still I believe that 100% coverage of
automation test is not possible better to say will not be a wise decision.
“Automated tests execute a sequence of actions without human intervention. This approach
helps eliminate human error, and provides faster results. Since most products require tests to be
run many times, automated testing generally leads to significant labor cost savings over time.
Typically a company will pass the break-even point for labor costs after just two or three runs of
an automated test.”
Manual testing, on the other hand, is a process that adapts easily to change and can cope with
complexity. Humans are able to detect hundreds of problem patterns, in a glance, an instantly
distinguish them from harmless anomalies. Humans may not even be aware of all the
evaluation that they are doing, but in a mere "sequence of actions" every evaluation must be
explicitly planned. Testing may seem like just a set of actions, but good testing is an interactive
cognitive process. That's why automation is best applied only to a narrow spectrum of testing,
not to the majority of the test process.
If you set out to automate all the necessary test execution, you'll probably spend a lot of money
and time creating relatively weak tests that ignore many interesting bugs, and find many
"problems" that turn out to be merely unanticipated correct behavior.
My opinion for automation test; it saves many things e.g. time, cost, effort and faster but it
cannot save MANUAL test.
References:
1. http://www.stickyminds.com/sitewide.asp?ObjectId=2326&Function=edetail&ObjectTy
pe=
2. http://www.stickyminds.com/sitewide.asp?ObjectId=2084&ObjectType=COL&Function=
edetail
3. http://www.io.com/~wazmo/papers/seven_steps.html

4. Test Automation Snake Oil V2.1 6/13/99:
http://www.satisfice.com/articles/test_automation_snake_oil.pdf
5. http://www.io.com/~wazmo/papers/seven_steps.html
6. Software Test Automation: A Real-World Problem, Cem Kaner, Ph.D., J.D.

Automation testing: how tools are important?

Recommended

More Related Content

What's hot (20)

Similar to Automation testing: how tools are important? (20)

Recently uploaded (20)

Automation testing: how tools are important?