ICIRHT-2017, 28" 29” August, 2017, ISBN: 978-93-5279-071-5 Analysis of automated tools & Productization of automation testing framework Kinnari N. Shah M.Tech in Computer Engineering Department U. V, Patel college of Engineering Prof. Prashant S. Modi ‘M.Tech Coordinator Information Technology U. V, Patel college of Engineering Kherva, Mehsana , Gujarat, India Mr. Nishil Patel Sr. Automation Test Architect, Kiwiga Services Pvt LTD. Abmedabad , Gujarat, India nishil [email protected] Kherva, Mehsana, Gujarat, India [email protected] [email protected] Abstract—Software testing is the emerging area in current generation. In the digital marketing approach all thing moves towards the computer software and applications. So software testing is more needed to test all these software and provide the quality without any error to the end users. Software testing has two approaches manual testing and automation testing. Automation testing main benefit is write once and execute many times. To perform automation testing there are number of tools available in the market, some are the open source and some are commercial. The main objective of this paper is to select the automation tool best as per requirement and make ‘the Framework which is more reusable and structure Dased to manage the seript. Tools can use with ‘multiple browser, multiple OS at parallel using selenium grid, Developed framework is combination of TestNG, selenium grid, continuous integration of Jenkins. Keywords—Software Testing, Automation, Manual; Automation tools comparison: Framework Structure, Graphical Interface GUI 1. Introduction Software testing is the heart of software engineering. It is used to find out number of errors bug in the particular software or application, It is also provide the high quality of assurance. End user can used it effectively without any intermediate issues. Basically this quality is achieved by following the software testing life eycle. Software testing is having two approach manual testing and Automation testing [2]. Automation testing is generally used to test the sofware by script, automatically without human interaction. Automation testing is more useful when the application is more complex, human cannot test application manually in short time duration Manual testing is also more costly and time consuming. Both are important in their own prospective but in one line we can say that manual testing is. more time consuming, more human efforts required and more costly. In the other side automation testing performance is virtuous. It can reduce human efforts and give more accurate results every time without miss any functionality Currently there are number of tools available in the market to perform automation testing [10] Software testing contains the number of testing type, Tester identify best type of testing as per the requirements. Testing can be apply on Web based application testing, Mobile testing, Desktop application testing, Cloud testing, Database testing, API testing etc, [7]. Web based automation testing ccan be possible with using the different tools and different language. List of tools and their ‘comparison we have explain in next topie. For more reusability and maintaining of the code structure as well for reduced the complexity of the code and make it moro users friendly we have developed one structure of framework [4]. Using. that structure itis more reliable to identify the code and make it more portable, We can also integrate that framework with continuous integration tools, And for the nontechnical person who having no ‘any idea about the eclipse IDE or any language, for them we also developed the GUI, using that they ccan check the items for run, and click on particular button, automatically the test cases will be written at run time and execution will be started using the continuous integration tool, Il, Related Tools Comparison Automation testing tools are available in market for different language, different requirements and for different browser support [1, 3, 6, 8-10]. Some are elaborated in below (table I), International Conference on Interdisciplinary Research in Humanitarian Technology (ICIRHT)ICIRHT-2017, 28" 29" August, 2017, ISBN. 978-93-5279-071-5 ‘TABLE |, COMPARISON OF AUTOMATION TOOL FEATURE | SELENIUM ore test | waTIR | SAHI | SIKULI | RANOREX | TELERIK s COMPLETE, FullForm | Selenium Quick Test | Test Complete | Web | SAU | SIKULE | RanoreXPath | Telenk Test Professional Arico studio Testing in Ruby License | OpenSource | Commercial | Commercial | open open | Open | Commercial | Commercial source | souree | source Price Fee $3,500 $2,000 Free Fee | Free | séao | $3499 Record ves ves ves | veswin | yes | ves ves ves, and Play suppor o cle WATIR Features Recorder Supported | AllBrowsess | te-Fiefox | Firefox Ee All | TEFirefo | Firefox | All Browser Browsers Chrome | OPERA | Firefox: | Browsers | OPERA. Chrome | OPERA: Chrome | Chrome Chrome Satu Supported | Windows-MAC- | Windows only | Windows? | Windows | Windows. | Window | Windows | Windows 0s, Tin andHigher | -MAC- | MAC” | s-MAC- | onlyexcept | Vistaand Linux Linws xP Tlgher Language | Ruby;javs-NET- | VBScript | vBSsrpt- CH | Ruby | Java script VB script | VB.net-ci python pip jve Ssript toased | Ruby" script, Report uTML HIM mm | uMix | Me ame | uM Generation ML yes YES. Nest More | YES yes | yes | Need More | YES Training ‘Training Data- nxcckcsy | Excatfiler | csvetxt | xmt- | csv-aPt | No | CSVeExou | Excel flee Driven text files SQL | Exec fs SOL | textiles XML-DB files XMLDB Programm! | Required Pariatly | Required | Pariety | Paniaty | No | Paniatly | Reguied ag Skil Variants + No No No No xo No IDERCWebdtive Grid iz classi Communit, More More Perialy | Wesk | Paraly | No | Average | Good support Easeoftise | Experience | Fasytotcam | Experience | No No | Fasyo | Experience | Experience Nesded Nesied | Experiene feam | Needed |” Needed International Conference on Interdisciplinary Research in Humanitarian Technology (ICIRHT) 121ICIRHT-2017, 28" 29" August, 2017, ISBN: 978-93-5279-071-5 Spectalty | More Etiientand | Efficient for | Support Data | Provenas | vsedbyall | GUI | Imagebased | Complex ‘User fiendly | Functional | driven testing, | Lightweig | developer | Testing | Detection | tax like Test objet driven’ | ht to fixing tool | deag-ndrop regresion test | and keyword | powerful _ pure Ul diven teaing | and easy. reproduc ‘tions alongwith” | “touse” | “bugs.Qe database ‘or Testing uncon testing and BA for Defining & Coding | MuchMore | icuttto | Less Trained | Easy | Avergze | NO | Ditticuttio | Less Person Experience | Easily Avatlable | Trained, Teal Trained Trained | Support of rained Person | Version Person Rgincer Desktop No Yes YES NO No ves ves No Applicatio Supported Web Based ves No ves ves ves | ves ves ves Applicatio supported Mobite No No No No No No ves ves Applicatio Supported No No No No No YES ves No To summarize this literature we can identify that overall Selenium tool is extremely well for automation. As it provides the more community support as well more portable and integrate with Gifferent tools. Also it is available in different language. And mainly it is open source. We can also integrate the selenium Grid with our Framework so achieve the goal of parallel testing with different browser and different OS as well run different test cases simultaneously on different systems, IIL. Proposed Approach ‘As there are number of approach available for framework that are listed below [7] ‘© Data driven approach ‘+ Keyword Driven approach Library driven approach Module driven approach Hybrid driven approach ‘+ Behavior driven approach, A. Structure of the Framework Framework is dividing into main three packages that are elaborated below (in fig. 1). This structure includes the Common class which is called the functionality of the Library driven approach. It includes the most common methods which can use by everyone. This contains the Test Data class which is connect with the excel sheet for feteh the data from the sheets. So this will cover the Data driven approach. Here in this structure we divide the code into three parts. Those are © Index © Index Page © Verification. International Conference on Interdisciplinary Research in Humanitarian Technology (ICIRHD) 122ICIRHT-2017, 28" -29"™ August, 2017, ISB 978-93-5279-071-5 Package: [Admin] Package [com.projectname] Package:[Init] J Package: Utilf J Vv Index Interface: TestStatus Class: customReporter s9-CustomReporter nN ple) bln ) Lee) Structure ofthe Framework So this will cover the functionality of the modular driven approach, This structure of the framework is combination of more than two approaches. So this is called the hybrid B. Component and Tools We developed this framework in Java language with the support of Core Java in Eclipse IDE. And Java servlet for the support of Graphical user interface. Integrated tools and technologies used for developed this approach are listed below. * Selenium WebDriver © Selenium Grid © TestNG Maven ‘© Jenkins Continuous Integration ‘* For mobile automation Android SDK and Appium, ©. Execution Flow Framework Execution flow is elaborated (in fig 2). Execution will be start from the POM.XML file, where it contains the different required dependency for the particular project. With this it contains the XML file path from which the Test case execution will be stored, So then after flow switched towards the “IndexSuitexml", it contains the main parameter for start the browser. And test cases for the execution of the project. Then after flow divert into the Index file, It includes only definition of the methods. Here index file contains only calling of ‘methods and calling of verification if needed, Index file extends the “Selenium Init", which is contains the pre and post configuration of the project. Basically, pre configuration contains the “Setup of the browser with all the parameter s like browser profiles its capability etc. then afler test case ‘executes, And in the post configuration it contains the teardown methods which is used for provide the lest case Status as well if any execution lailed then attached the logic for the make the screenshot for that point of failure. Then “IndexPage “contains the ‘main logic of the automation, And it is extends the “Abstract Page”, which is contains the concept of “Page factory” means run time get the fresh ‘clement for the execution of the page. So script will be International Conference on Interdisciplinary Research in Humanitarian Technology (ICIRHT)ICIRHT-2017, 28™ -29™ August, 2017, ISBN: 978-93-5279-071-5 Init re and Post Setup Coding Structure Reporter and Retry + ‘Analyser Common Function Fig, 2, Execotion Flow of Framework based projet ‘execute successfully even there may change the clement at run time, Index page can call the verification page if required. At last “Util” Package contains the Custom Report. ‘The concept of custom report is like that when any execution of the script completed, then after TestNG report will be generated. So the Custom Reporter will fetch the data from the TestNG , and. convert it into the company standard Report format.so whenever client want the final result in their Email Inbox, it is possible to attached it ‘automatically after the completion of the script. “Retry” Test is used when duc to any internal problem, or internet issue. Unfortunately script will get failed on that time using this class we can provide the number of try to make the script Passed. ‘automatically. IV, Experimental Result This section contains the experimental work, which we have tried on Ordinary system and using the framework based system, Here we take the three Gifferent seenario of the one project. Three scenarios build using the ordinary flow. And try to run it, We put the Start time and end time logic ‘before and after the script. So get the exact idea of how much time it took to ran the scenario, Same way we also developed the script using the developed structure Framework approach. And similarly run the Framework based script with using the same browser, same platforms and considering all the criteria as same. Consider the below (fg. 3). re Totetedox System | inteiodox System ° system tot | ene Sone [ealinary system [23s [3.78 34 o sreemmne Prramework system [2.72 [3.47 336 Fig 3 [Bxperimentl Rest and Time Chart International Conference on Interdisciplinary Research in Humanitarian Technology (ICIRHT) 124ICIRHT-2017, 28 .29™ August, 2017, ISBN: 978-93-5279-071-5 From above result we can conclude that the framework approach is more suitable. As it takes the less time compare to the Ordinary system. This time is measure in the “seconds” unit. We can get the idea that for running only three scenarios this much time has been saved. Now when we have the large project which contains the numbers of scripts, then we can save the more execution time. V. Conclusion and Future Work Here From the number of tools we can conclude that the Selenium tools is best for work as it is open source and comparatively to other tools it provide the more functionality And easily integrate with the other. Now from the framework structure, we can save more time and the space. And make the structure more reusable, reduced the complexity of code and more tester friendly. It is more portable for used with different OS, Different Browser. Every time no need of setup from starting point. Just give some parameters and the setup will be ready within the so less time, ‘And as we know the automation testing is reduced the human efforts, and give every time give the result with the accuracy. So we can use the automation testing using the selenium tool, and this structure framework to achieve the more goals with short time, For the future work we try to provide the security into this framework, so that make our script uniquely secure, and gets the more benefits of maintainability of the seript References [1H Kaur and G. Gupta, “Comparative Sty of Automated Testing ‘Tools: Selenium, Quick Test Profession Eng. Kes. Appl, vol. 3, 0.5, pp. 1739-1743, 2013, (} 1 Singh 8 U. Eriksson, White box Testing” vol. 4, no. 15,pp. 1-8, 2016 [3] A. Jain, M, Jain, and S, Dhanke, “A. Comparison of RANOREX and QTP Automated Testing Tools and their impact oa Software and TestComplets” J rik, 8, Features, P, Customers, L, B, Conta, ad “est design techniques explained # 1: Back box vs Testing.” Home, vol. 1.20 pp 8-12, 2014, TMG. D. Gojae Satish, Joshi Rabul, “Analysis and Design of Selenium Webdsiver Automation Testing Framework pd” 2nd Internation Symposium on Big Dsla and Cloud Computing (SBC), pp. 341-846, 2015. 5} Kuster, “www ulitatemcom,” 2016, (Online. Available: pew kus comblogsiat-an-awomatin-tes to [6] M. Monier and M. M. El-mahdy, “Evalustion of automat web sting tool," Int J. Comput Ap. Technol. Res, vol. 4, m0. 52 405-408, 2015 (7) K.Revath and P. V Janani, “SELENIUM TEST AUTOMATION FRAMEWORK IN ON-LINE BASED APPLICATION,” int J Ad. Res. Sc. Eng. LARSE, Val, No, Spec. sue (02), Febr 2015, vol, 8354, n0.4, pp. 55-68, 2015, [8] N, Gogna, “Study of Browser Based Automated Test Tools WAT and Selenium” I J. In ue. Techno vol 4,1. 4, 1 uo} 336, 2014 S. Duvivedi and M. Garima, “Compare of Open Source Tool using in Web Application Testing” vo. 3, 0,2, pp, 918-920, 2015, IN. Bhatea, “A Smuy on Various Software Automation Testing "Tool" Int. J Adv: Res. Comput Sc. Sov. Eng. ol. 5,80 6, PP. 1280-1282, 2018, International Conference on Interdisciplinary Research in Humanitarian Technology (ICIRHT) 125