SlideShare a Scribd company logo
@BagmarAnand
Streamlining
End-to-End
Test Automation
Anand Bagmar
Software Quality Evangelist
@BagmarAnand
Anand Bagmar
@BagmarAnand
About Me
Ground
Reality
Distributed teams
Hybrid working
Different network setup and speeds
Many teams
(>100)
Mac, Windows and Linux laptops
Different Software versions
Certificates, Policies and multiple VPNs
Inconsistent
Developer &
SDET experience
Test execution environment setup is tedious
System Tests (e2e & component UI): teswiz (Appium, Selenium, Applitools, JDK 17)
• Emulator/Simulator setup (Android SDK, xcode)
API & API Workflow tests: karate
Contract tests: Specmatic
Unit tests, Sonar code quality checks
Test automation
toolset
Many environments
Test data
Branch
Configuring appropriate test execution in build and release pipelines
Complex path to
production
ADO agents: Windows Server & Linux agents
Firewall restrictions download dependencies
Direct access prohibited to CI agents
Multiple node & JDK versions
Connectivity issues to application-under-test
No browsers/devices on CI agents
CI execution
@BagmarAnand
Path to Production
@BagmarAnand
Getting a simple automated
test to run consistently for
all Developers and Testers
and in CI (ADO) is painful!
Setup
Execution (full or specific tests)
@BagmarAnand
Challenges of
End-2-End
Test
Automation
Ensuring Test
Environment Consistency
Coordinated Test
Execution
Test setup & execution on
CI Agents
@BagmarAnand
Solutions
Implemented
@BagmarAnand
Consistent
Environment Setup
#1
@BagmarAnand
• Setup important applications on Mac
• https://gist.github.com/anandbagmar/92b9f92298b1e17fa32c3404ad115
871
• Script to setup Android SDK on Mac
• https://github.com/anandbagmar/AppiumJavaSample/blob/master/setup
AndroidSDK.sh
• Script to setup Android SDK on Linux
• https://github.com/anandbagmar/AppiumJavaSample/blob/master/setup
_linux.sh
Test Authoring Environment Setup
@BagmarAnand
• Node script to install all dependencies (for system tests):
• https://github.com/znsio/getting-started-with-
teswiz/blob/main/package.json
• npm install – and you are ready!
Test Execution Environment Setup
@BagmarAnand
Test Automation
Framework support
#2
@BagmarAnand
• Setup should be simple – Ex:
• git pull
• ./gradlew build
• No code change required for
• Running tests against any environment (local, dev, qa, staging, prod, etc.)
• Test data and environment configurations are separately maintained
• Running all or subset of tests
• Tests should run from command-line
Test Automation Framework Criteria
@BagmarAnand
Your Ultimate Open-Source Solution to
Automate Real-User Scenarios!
@BagmarAnand
Architecture
@BagmarAnand
Run tests
from CLI
Test Authoring
Execution Setup
1
2
2
3
4
5
6
6
Execution Reports
CI Tool
Feature coverage
@BagmarAnand 16
• Web browsers
• Mobile-web browsers
• Android apps
• iOS apps
• Windows desktop apps
• Electron apps
Platform support
@BagmarAnand
• Open source framework to automate real-user scenarios
• Multi-user
• Multi-device
• Multi-app
• Setup a HARD-GATE for your functional tests!
Unique capabilities of teswiz
@BagmarAnand
• Cloud device farm integrations
• Applitools AI for validations
• Comprehensive reports with trend analysis, feature
coverage, failure analysis using AI-ML
• CLI
• Configurable
Unique capabilities of teswiz
Defaults
Property
files
Environment
Variables
@BagmarAnand
CI Execution
@BagmarAnand
Node Setup
#3
@BagmarAnand
Use the right node version
@BagmarAnand
Use the right node version
@BagmarAnand
Script for
downloading artifacts
#4
@BagmarAnand
• For System Tests, the artifact (apk/app) could have been
generated from another pipeline
• This artifact needs to be available in local or cloud device
before tests can start execution
Script for downloading artifacts – Why?
@BagmarAnand
• Understand the CI tool APIs
• Script downloads the android/iOS artifact for:
• Specific branch
• Latest successful build, or a specific build number
Script for downloading artifacts
@BagmarAnand
• Script uploads the android/iOS artifact to your device farm
OR
• Teswiz can upload it automatically for you
Script for downloading artifacts – Bonus!
@BagmarAnand
Proxy handling
#5
@BagmarAnand
• Understand what dependencies in your framework need proxy
information. Ex:
• Gradle/maven
• Downloading newer version of browser drivers
• Any external connectivity
• Framework should be configurable to pass this at test execution
time. Ex:
• No proxy required from local laptop execution
• Proxy required when running tests from CI
Proxy Handling
@BagmarAnand
@BagmarAnand
updateGradlePropertiesForDevOps.sh
@BagmarAnand
Downloading
dependencies - Uber jar
#6
@BagmarAnand
• To reduce the number of dependencies to be downloaded,
teswiz is built as a uber jar.
• Specify only “teswiz” as a dependency in your test framework
Uber jar
@BagmarAnand
• Run as a java process
• ./gradlew run
Uber jar
@BagmarAnand
Run browser in
docker
#7
@BagmarAnand
• CI agents may not have browsers installed
• The installed browser may be an older version
Run browser in docker – Why?
@BagmarAnand
• Should allow support for any os/architecture
• Should allow choosing the browser (ex: firefox, chrome, etc.)
• Should allow starting the containers with specific project
names and dynamic ports to prevent conflict with multiple
test executions
• Support specifying proxy information
• Can be used on local laptops as well as in CI executions
Run browser in docker
@BagmarAnand
https://github.com/znsio/teswiz/blob/main/dockerContainers.sh
https://github.com/znsio/teswiz/blob/main/docker-compose-v3.yml
Template for running
tests in build pipeline
#8
@BagmarAnand
Path to Production
@BagmarAnand
• Create templates
• Reuse with appropriate configuration parameters
Running tests in Build Pipeline
@BagmarAnand
@BagmarAnand
Task Groups for running
tests in release pipeline
#9
@BagmarAnand
Path to Production
@BagmarAnand
• Create Task Groups
• Include in each relevant stage of Release pipeline
Running tests in Release Pipeline
@BagmarAnand
@BagmarAnand
@BagmarAnand
@BagmarAnand
@BagmarAnand
@BagmarAnand
@BagmarAnand
Hard Gate
Make your tests valuable!
#10
@BagmarAnand
@BagmarAnand
What is a Hard Gate! Why is it required?
https://github.com/znsio/teswiz/blob/main/docs/HardGate.md
• Automated tests should allow you to take decisions on
product quality
@BagmarAnand
What is a Hard Gate! Why is it required?
https://github.com/znsio/teswiz/blob/main/docs/HardGate.md
• For every test execution cycle:
• Passing tests are expected to pass
• Known Failing tests are supposed to fail, unless:
• The product (bug) is fixed, OR
• The test is fixed/updated
If either criteria is not met, the build should fail!
@BagmarAnand
Hard Gate - Make your tests valuable!
https://github.com/znsio/teswiz/blob/main/docs/HardGate.md
Build passes if Hard Gate criteria is met.
Build fails if
• one or more passing tests have failed, or,
• one or more failing tests have passed
@BagmarAnand
Functional (e2e) Automation as Hard Gate!
https://github.com/znsio/teswiz/blob/main/docs/HardGate.md
Applitools Visual AI
for validations
#11
@BagmarAnand
AI-powered Validations
@BagmarAnand
90% less code to write & maintain with infinite coverage.
WITH APPLITOOLS AI
Every Element Is Validated They Look & Work Perfect
Use Applitools Ultrafast Grid (UFG)
- Test is simpler – one call to Applitools
(eyes.checkWindow()) validates the full screen
- Run the test once
- Get results from all browsers automatically
- Less test data
- No additional load on the application environment
@BagmarAnand
• Works for all platforms
• Native & hybrid apps – android, iOS
• Web browsers
• Desktop applications
• Electron applications
• Seamless scaling using Applitools Ultrafast Grid
@BagmarAnand
Applitools Visual AI
Specify as many browsers with viewports and devices
as required for validation
You do not need to do cross-browser validation at the
end anymore!
@BagmarAnand
AI-powered Cross Browser Test Automation
reportportal as a
Central reporting server
#12
@BagmarAnand
@BagmarAnand
A central reporting server for your
organization
@BagmarAnand
Test Execution
Real-time status
• See progress of launches
currently in progress
• Can also see details of tests
that are currently running, till
the point of execution
@BagmarAnand
Test Execution Details – Device farm report link & Device logs
• The link to the device farm test execution
dashboard is available in the result
• teswiz attaches browser logs/device logs
automatically to the result in ReportPortal
@BagmarAnand
Test Execution Details – with screenshots
• The test result includes screenshots as
captured by the test
@BagmarAnand
Test Execution Details – Applitools Visual AI Validation Results
• The test result includes the status of
Applitools Visual AI validation
• Link to the Applitools dashboard is available in
the result
@BagmarAnand
Test Execution Trend Analysis
• Each test shows the trend of its execution –
giving an indication of (in)stability
@BagmarAnand
Test Results – Next Steps
• On investigation of the failed tests, mark the
failures with appropriate reasons (as
configured)
@BagmarAnand
Auto-analysis of failed tests
Analyse the failure reasons by Auto-Analyzer based on Machine Learning
@BagmarAnand
Auto Analysis of Test Failures
• Why waste time
marking the test
failed for the same
reason as last time?
• ReportPortal can do
this automatically
for you with the
Auto Analysis and
Pattern Analysis
feature
@BagmarAnand
Auto Analysis of Test Failures
@BagmarAnand
Test Result
Visualization
Configure simple and
understandable
reports
• Create as many dashboards as
relevant for the team
• Dashboards may be for different
persona/role, giving appropriate
information
@BagmarAnand
• Teswiz and karate test frameworks can automatically upload
test results to your reportportal server
• sendToReportPortal:
• https://github.com/znsio/sendToReportPortal/blob/main/importRes
ultsAndUpdateAttributes.sh
• Can upload junit test results generated by any type of tests to
reportportal with relevant test execution metadata
reportportal.io
@BagmarAnand
Challenges
• Ensuring Test
Environment Consistency
• Coordinated Test
Execution
• Test setup & execution on
CI Agents
Solutions
Consistent environment setup
Test Automation Framework support
Node setup
Script for downloading artifacts
Proxy Handling
Downloading dependencies - Uber Jar
Browsers in docker
Template for build pipelines
Task groups for release pipelines
Hard Gate
AI for validations
Central reporting server
Summary
@BagmarAnand
@BagmarAnand
Anand Bagmar
@BagmarAnand
Thank you
Ad

More Related Content

Similar to Streamlining End-to-End Testing Automation (20)

Test Strategy For Future Cloud Architecture
Test Strategy For Future Cloud ArchitectureTest Strategy For Future Cloud Architecture
Test Strategy For Future Cloud Architecture
MaheshShri1
 
Software Testing includes Performance testing with Load Runner and the JMeter
Software Testing includes Performance testing with Load Runner and the JMeter Software Testing includes Performance testing with Load Runner and the JMeter
Software Testing includes Performance testing with Load Runner and the JMeter
Hima Bindu Kosuru
 
Cloud-based performance testing
Cloud-based performance testingCloud-based performance testing
Cloud-based performance testing
abhinavm
 
Modern Web-site Development Pipeline
Modern Web-site Development PipelineModern Web-site Development Pipeline
Modern Web-site Development Pipeline
GlobalLogic Ukraine
 
DCAST Meetup - Washington, DC Feb 2016
DCAST Meetup - Washington, DC Feb 2016DCAST Meetup - Washington, DC Feb 2016
DCAST Meetup - Washington, DC Feb 2016
Justin Ison
 
QA Automation Solution
QA Automation SolutionQA Automation Solution
QA Automation Solution
DataArt
 
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdfNET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
Tamir Dresher
 
Docker Enterprise Workshop - Technical
Docker Enterprise Workshop - TechnicalDocker Enterprise Workshop - Technical
Docker Enterprise Workshop - Technical
Patrick Chanezon
 
Getting to Walk with DevOps
Getting to Walk with DevOpsGetting to Walk with DevOps
Getting to Walk with DevOps
Eklove Mohan
 
Integration Testing as Validation and Monitoring
 Integration Testing as Validation and Monitoring Integration Testing as Validation and Monitoring
Integration Testing as Validation and Monitoring
Melissa Benua
 
Windows azure overview for SharePoint Pros
Windows azure overview for SharePoint Pros Windows azure overview for SharePoint Pros
Windows azure overview for SharePoint Pros
Usama Wahab Khan Cloud, Data and AI
 
Eradicate Flaky Tests - AppiumConf 2021
Eradicate Flaky Tests - AppiumConf 2021Eradicate Flaky Tests - AppiumConf 2021
Eradicate Flaky Tests - AppiumConf 2021
Anand Bagmar
 
Habitat talk at CodeMonsters Sofia, Bulgaria Nov 27 2018
Habitat talk at CodeMonsters Sofia, Bulgaria Nov 27 2018Habitat talk at CodeMonsters Sofia, Bulgaria Nov 27 2018
Habitat talk at CodeMonsters Sofia, Bulgaria Nov 27 2018
Mandi Walls
 
Azure Integration DTAP Series, How to go from Development to Production – Par...
Azure Integration DTAP Series, How to go from Development to Production – Par...Azure Integration DTAP Series, How to go from Development to Production – Par...
Azure Integration DTAP Series, How to go from Development to Production – Par...
BizTalk360
 
Udvid din test portefølje med coded ui test og cloud load test
Udvid din test portefølje med coded ui test og cloud load testUdvid din test portefølje med coded ui test og cloud load test
Udvid din test portefølje med coded ui test og cloud load test
Peter Lindberg
 
Tech Days 2015: Model Based Development with QGen
Tech Days 2015: Model Based Development with QGenTech Days 2015: Model Based Development with QGen
Tech Days 2015: Model Based Development with QGen
AdaCore
 
Azure Web Apps
Azure Web AppsAzure Web Apps
Azure Web Apps
Gaurav Madaan
 
New ThousandEyes Product Features and Release Highlights: March 2024
New ThousandEyes Product Features and Release Highlights: March 2024New ThousandEyes Product Features and Release Highlights: March 2024
New ThousandEyes Product Features and Release Highlights: March 2024
ThousandEyes
 
Automated Low Level Requirements Testing for DO-178C
Automated Low Level Requirements Testing for DO-178CAutomated Low Level Requirements Testing for DO-178C
Automated Low Level Requirements Testing for DO-178C
QA Systems
 
OSGi Community Event 2010 - App Store for the Connected Home Services
OSGi Community Event 2010 - App Store for the Connected Home ServicesOSGi Community Event 2010 - App Store for the Connected Home Services
OSGi Community Event 2010 - App Store for the Connected Home Services
mfrancis
 
Test Strategy For Future Cloud Architecture
Test Strategy For Future Cloud ArchitectureTest Strategy For Future Cloud Architecture
Test Strategy For Future Cloud Architecture
MaheshShri1
 
Software Testing includes Performance testing with Load Runner and the JMeter
Software Testing includes Performance testing with Load Runner and the JMeter Software Testing includes Performance testing with Load Runner and the JMeter
Software Testing includes Performance testing with Load Runner and the JMeter
Hima Bindu Kosuru
 
Cloud-based performance testing
Cloud-based performance testingCloud-based performance testing
Cloud-based performance testing
abhinavm
 
Modern Web-site Development Pipeline
Modern Web-site Development PipelineModern Web-site Development Pipeline
Modern Web-site Development Pipeline
GlobalLogic Ukraine
 
DCAST Meetup - Washington, DC Feb 2016
DCAST Meetup - Washington, DC Feb 2016DCAST Meetup - Washington, DC Feb 2016
DCAST Meetup - Washington, DC Feb 2016
Justin Ison
 
QA Automation Solution
QA Automation SolutionQA Automation Solution
QA Automation Solution
DataArt
 
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdfNET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
Tamir Dresher
 
Docker Enterprise Workshop - Technical
Docker Enterprise Workshop - TechnicalDocker Enterprise Workshop - Technical
Docker Enterprise Workshop - Technical
Patrick Chanezon
 
Getting to Walk with DevOps
Getting to Walk with DevOpsGetting to Walk with DevOps
Getting to Walk with DevOps
Eklove Mohan
 
Integration Testing as Validation and Monitoring
 Integration Testing as Validation and Monitoring Integration Testing as Validation and Monitoring
Integration Testing as Validation and Monitoring
Melissa Benua
 
Eradicate Flaky Tests - AppiumConf 2021
Eradicate Flaky Tests - AppiumConf 2021Eradicate Flaky Tests - AppiumConf 2021
Eradicate Flaky Tests - AppiumConf 2021
Anand Bagmar
 
Habitat talk at CodeMonsters Sofia, Bulgaria Nov 27 2018
Habitat talk at CodeMonsters Sofia, Bulgaria Nov 27 2018Habitat talk at CodeMonsters Sofia, Bulgaria Nov 27 2018
Habitat talk at CodeMonsters Sofia, Bulgaria Nov 27 2018
Mandi Walls
 
Azure Integration DTAP Series, How to go from Development to Production – Par...
Azure Integration DTAP Series, How to go from Development to Production – Par...Azure Integration DTAP Series, How to go from Development to Production – Par...
Azure Integration DTAP Series, How to go from Development to Production – Par...
BizTalk360
 
Udvid din test portefølje med coded ui test og cloud load test
Udvid din test portefølje med coded ui test og cloud load testUdvid din test portefølje med coded ui test og cloud load test
Udvid din test portefølje med coded ui test og cloud load test
Peter Lindberg
 
Tech Days 2015: Model Based Development with QGen
Tech Days 2015: Model Based Development with QGenTech Days 2015: Model Based Development with QGen
Tech Days 2015: Model Based Development with QGen
AdaCore
 
New ThousandEyes Product Features and Release Highlights: March 2024
New ThousandEyes Product Features and Release Highlights: March 2024New ThousandEyes Product Features and Release Highlights: March 2024
New ThousandEyes Product Features and Release Highlights: March 2024
ThousandEyes
 
Automated Low Level Requirements Testing for DO-178C
Automated Low Level Requirements Testing for DO-178CAutomated Low Level Requirements Testing for DO-178C
Automated Low Level Requirements Testing for DO-178C
QA Systems
 
OSGi Community Event 2010 - App Store for the Connected Home Services
OSGi Community Event 2010 - App Store for the Connected Home ServicesOSGi Community Event 2010 - App Store for the Connected Home Services
OSGi Community Event 2010 - App Store for the Connected Home Services
mfrancis
 

More from Anand Bagmar (20)

The Path to Autonomous Testing - Anand Bagmar
The Path to Autonomous Testing - Anand BagmarThe Path to Autonomous Testing - Anand Bagmar
The Path to Autonomous Testing - Anand Bagmar
Anand Bagmar
 
My Journey With Appium @AppiumConf Sept 2024
My Journey With Appium @AppiumConf Sept 2024My Journey With Appium @AppiumConf Sept 2024
My Journey With Appium @AppiumConf Sept 2024
Anand Bagmar
 
Automating the real-user scenarios across multi-apps, and multi-devices
Automating the real-user scenarios across multi-apps, and multi-devicesAutomating the real-user scenarios across multi-apps, and multi-devices
Automating the real-user scenarios across multi-apps, and multi-devices
Anand Bagmar
 
Visual Validation - The missing tip of the automation pyramid @GoT2022
Visual Validation - The missing tip of the automation pyramid @GoT2022Visual Validation - The missing tip of the automation pyramid @GoT2022
Visual Validation - The missing tip of the automation pyramid @GoT2022
Anand Bagmar
 
Design Patterns in Automation
Design Patterns in AutomationDesign Patterns in Automation
Design Patterns in Automation
Anand Bagmar
 
Change Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Change Tyres In A Moving Car - Make Functional Test Automation Effective KeynoteChange Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Change Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Anand Bagmar
 
Rewrite vs Refactor (AgileIndia 2021)
Rewrite vs Refactor (AgileIndia 2021)Rewrite vs Refactor (AgileIndia 2021)
Rewrite vs Refactor (AgileIndia 2021)
Anand Bagmar
 
Next Generation Functional & Visual Testing powered by AI
Next Generation Functional & Visual Testing powered by AINext Generation Functional & Visual Testing powered by AI
Next Generation Functional & Visual Testing powered by AI
Anand Bagmar
 
The Best Test Automation Framework is...
The Best Test Automation Framework is...The Best Test Automation Framework is...
The Best Test Automation Framework is...
Anand Bagmar
 
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Anand Bagmar
 
Selenium Deep Dive
Selenium Deep DiveSelenium Deep Dive
Selenium Deep Dive
Anand Bagmar
 
Does your functional automation really add value?
Does your functional automation really add value?Does your functional automation really add value?
Does your functional automation really add value?
Anand Bagmar
 
Measuring Coverage From E2E Tests
Measuring Coverage From E2E TestsMeasuring Coverage From E2E Tests
Measuring Coverage From E2E Tests
Anand Bagmar
 
Getting started with Visual Testing using Applitools - @TPC, Feb2020
Getting started with Visual Testing using Applitools - @TPC, Feb2020Getting started with Visual Testing using Applitools - @TPC, Feb2020
Getting started with Visual Testing using Applitools - @TPC, Feb2020
Anand Bagmar
 
Visual validation - The missing tip of the automation pyramid @ QA Symposium
Visual validation - The missing tip of the automation pyramid @ QA SymposiumVisual validation - The missing tip of the automation pyramid @ QA Symposium
Visual validation - The missing tip of the automation pyramid @ QA Symposium
Anand Bagmar
 
Collaboration - A Taboo!
Collaboration - A Taboo!Collaboration - A Taboo!
Collaboration - A Taboo!
Anand Bagmar
 
Visual Validation - The Missing Tip of the Automation Pyramid
Visual Validation - The Missing Tip of the Automation PyramidVisual Validation - The Missing Tip of the Automation Pyramid
Visual Validation - The Missing Tip of the Automation Pyramid
Anand Bagmar
 
Measuring Consumer Quality - The Missing Feedback Loop
Measuring Consumer Quality - The Missing Feedback LoopMeasuring Consumer Quality - The Missing Feedback Loop
Measuring Consumer Quality - The Missing Feedback Loop
Anand Bagmar
 
What is Agile Testing? How does Automation help?
What is Agile Testing? How does Automation help?What is Agile Testing? How does Automation help?
What is Agile Testing? How does Automation help?
Anand Bagmar
 
The What, Why and How of Analytics Testing
The What, Why and How of Analytics TestingThe What, Why and How of Analytics Testing
The What, Why and How of Analytics Testing
Anand Bagmar
 
The Path to Autonomous Testing - Anand Bagmar
The Path to Autonomous Testing - Anand BagmarThe Path to Autonomous Testing - Anand Bagmar
The Path to Autonomous Testing - Anand Bagmar
Anand Bagmar
 
My Journey With Appium @AppiumConf Sept 2024
My Journey With Appium @AppiumConf Sept 2024My Journey With Appium @AppiumConf Sept 2024
My Journey With Appium @AppiumConf Sept 2024
Anand Bagmar
 
Automating the real-user scenarios across multi-apps, and multi-devices
Automating the real-user scenarios across multi-apps, and multi-devicesAutomating the real-user scenarios across multi-apps, and multi-devices
Automating the real-user scenarios across multi-apps, and multi-devices
Anand Bagmar
 
Visual Validation - The missing tip of the automation pyramid @GoT2022
Visual Validation - The missing tip of the automation pyramid @GoT2022Visual Validation - The missing tip of the automation pyramid @GoT2022
Visual Validation - The missing tip of the automation pyramid @GoT2022
Anand Bagmar
 
Design Patterns in Automation
Design Patterns in AutomationDesign Patterns in Automation
Design Patterns in Automation
Anand Bagmar
 
Change Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Change Tyres In A Moving Car - Make Functional Test Automation Effective KeynoteChange Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Change Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Anand Bagmar
 
Rewrite vs Refactor (AgileIndia 2021)
Rewrite vs Refactor (AgileIndia 2021)Rewrite vs Refactor (AgileIndia 2021)
Rewrite vs Refactor (AgileIndia 2021)
Anand Bagmar
 
Next Generation Functional & Visual Testing powered by AI
Next Generation Functional & Visual Testing powered by AINext Generation Functional & Visual Testing powered by AI
Next Generation Functional & Visual Testing powered by AI
Anand Bagmar
 
The Best Test Automation Framework is...
The Best Test Automation Framework is...The Best Test Automation Framework is...
The Best Test Automation Framework is...
Anand Bagmar
 
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Anand Bagmar
 
Selenium Deep Dive
Selenium Deep DiveSelenium Deep Dive
Selenium Deep Dive
Anand Bagmar
 
Does your functional automation really add value?
Does your functional automation really add value?Does your functional automation really add value?
Does your functional automation really add value?
Anand Bagmar
 
Measuring Coverage From E2E Tests
Measuring Coverage From E2E TestsMeasuring Coverage From E2E Tests
Measuring Coverage From E2E Tests
Anand Bagmar
 
Getting started with Visual Testing using Applitools - @TPC, Feb2020
Getting started with Visual Testing using Applitools - @TPC, Feb2020Getting started with Visual Testing using Applitools - @TPC, Feb2020
Getting started with Visual Testing using Applitools - @TPC, Feb2020
Anand Bagmar
 
Visual validation - The missing tip of the automation pyramid @ QA Symposium
Visual validation - The missing tip of the automation pyramid @ QA SymposiumVisual validation - The missing tip of the automation pyramid @ QA Symposium
Visual validation - The missing tip of the automation pyramid @ QA Symposium
Anand Bagmar
 
Collaboration - A Taboo!
Collaboration - A Taboo!Collaboration - A Taboo!
Collaboration - A Taboo!
Anand Bagmar
 
Visual Validation - The Missing Tip of the Automation Pyramid
Visual Validation - The Missing Tip of the Automation PyramidVisual Validation - The Missing Tip of the Automation Pyramid
Visual Validation - The Missing Tip of the Automation Pyramid
Anand Bagmar
 
Measuring Consumer Quality - The Missing Feedback Loop
Measuring Consumer Quality - The Missing Feedback LoopMeasuring Consumer Quality - The Missing Feedback Loop
Measuring Consumer Quality - The Missing Feedback Loop
Anand Bagmar
 
What is Agile Testing? How does Automation help?
What is Agile Testing? How does Automation help?What is Agile Testing? How does Automation help?
What is Agile Testing? How does Automation help?
Anand Bagmar
 
The What, Why and How of Analytics Testing
The What, Why and How of Analytics TestingThe What, Why and How of Analytics Testing
The What, Why and How of Analytics Testing
Anand Bagmar
 
Ad

Recently uploaded (20)

Expand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchangeExpand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchange
Fexle Services Pvt. Ltd.
 
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
AxisTechnolabs
 
Top 10 Data Cleansing Tools for 2025.pdf
Top 10 Data Cleansing Tools for 2025.pdfTop 10 Data Cleansing Tools for 2025.pdf
Top 10 Data Cleansing Tools for 2025.pdf
AffinityCore
 
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdfMicrosoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
TechSoup
 
How to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud PerformanceHow to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud Performance
ThousandEyes
 
Tools of the Trade: Linux and SQL - Google Certificate
Tools of the Trade: Linux and SQL - Google CertificateTools of the Trade: Linux and SQL - Google Certificate
Tools of the Trade: Linux and SQL - Google Certificate
VICTOR MAESTRE RAMIREZ
 
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRYLEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
NidaFarooq10
 
🌱 Green Grafana 🌱 Essentials_ Data, Visualizations and Plugins.pdf
🌱 Green Grafana 🌱 Essentials_ Data, Visualizations and Plugins.pdf🌱 Green Grafana 🌱 Essentials_ Data, Visualizations and Plugins.pdf
🌱 Green Grafana 🌱 Essentials_ Data, Visualizations and Plugins.pdf
Imma Valls Bernaus
 
What Do Contribution Guidelines Say About Software Testing? (MSR 2025)
What Do Contribution Guidelines Say About Software Testing? (MSR 2025)What Do Contribution Guidelines Say About Software Testing? (MSR 2025)
What Do Contribution Guidelines Say About Software Testing? (MSR 2025)
Andre Hora
 
Full Cracked Resolume Arena Latest Version
Full Cracked Resolume Arena Latest VersionFull Cracked Resolume Arena Latest Version
Full Cracked Resolume Arena Latest Version
jonesmichealj2
 
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
Scaling GraphRAG:  Efficient Knowledge Retrieval for Enterprise AIScaling GraphRAG:  Efficient Knowledge Retrieval for Enterprise AI
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
danshalev
 
Kubernetes_101_Zero_to_Platform_Engineer.pptx
Kubernetes_101_Zero_to_Platform_Engineer.pptxKubernetes_101_Zero_to_Platform_Engineer.pptx
Kubernetes_101_Zero_to_Platform_Engineer.pptx
CloudScouts
 
Apple Logic Pro X Crack FRESH Version 2025
Apple Logic Pro X Crack FRESH Version 2025Apple Logic Pro X Crack FRESH Version 2025
Apple Logic Pro X Crack FRESH Version 2025
fs4635986
 
Creating Automated Tests with AI - Cory House - Applitools.pdf
Creating Automated Tests with AI - Cory House - Applitools.pdfCreating Automated Tests with AI - Cory House - Applitools.pdf
Creating Automated Tests with AI - Cory House - Applitools.pdf
Applitools
 
Proactive Vulnerability Detection in Source Code Using Graph Neural Networks:...
Proactive Vulnerability Detection in Source Code Using Graph Neural Networks:...Proactive Vulnerability Detection in Source Code Using Graph Neural Networks:...
Proactive Vulnerability Detection in Source Code Using Graph Neural Networks:...
Ranjan Baisak
 
Automation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath CertificateAutomation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath Certificate
VICTOR MAESTRE RAMIREZ
 
Not So Common Memory Leaks in Java Webinar
Not So Common Memory Leaks in Java WebinarNot So Common Memory Leaks in Java Webinar
Not So Common Memory Leaks in Java Webinar
Tier1 app
 
Landscape of Requirements Engineering for/by AI through Literature Review
Landscape of Requirements Engineering for/by AI through Literature ReviewLandscape of Requirements Engineering for/by AI through Literature Review
Landscape of Requirements Engineering for/by AI through Literature Review
Hironori Washizaki
 
Odoo ERP for Education Management to Streamline Your Education Process
Odoo ERP for Education Management to Streamline Your Education ProcessOdoo ERP for Education Management to Streamline Your Education Process
Odoo ERP for Education Management to Streamline Your Education Process
iVenture Team LLP
 
Innovative Approaches to Software Dev no good at all
Innovative Approaches to Software Dev no good at allInnovative Approaches to Software Dev no good at all
Innovative Approaches to Software Dev no good at all
ayeshakanwal75
 
Expand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchangeExpand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchange
Fexle Services Pvt. Ltd.
 
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
AxisTechnolabs
 
Top 10 Data Cleansing Tools for 2025.pdf
Top 10 Data Cleansing Tools for 2025.pdfTop 10 Data Cleansing Tools for 2025.pdf
Top 10 Data Cleansing Tools for 2025.pdf
AffinityCore
 
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdfMicrosoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
TechSoup
 
How to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud PerformanceHow to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud Performance
ThousandEyes
 
Tools of the Trade: Linux and SQL - Google Certificate
Tools of the Trade: Linux and SQL - Google CertificateTools of the Trade: Linux and SQL - Google Certificate
Tools of the Trade: Linux and SQL - Google Certificate
VICTOR MAESTRE RAMIREZ
 
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRYLEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
NidaFarooq10
 
🌱 Green Grafana 🌱 Essentials_ Data, Visualizations and Plugins.pdf
🌱 Green Grafana 🌱 Essentials_ Data, Visualizations and Plugins.pdf🌱 Green Grafana 🌱 Essentials_ Data, Visualizations and Plugins.pdf
🌱 Green Grafana 🌱 Essentials_ Data, Visualizations and Plugins.pdf
Imma Valls Bernaus
 
What Do Contribution Guidelines Say About Software Testing? (MSR 2025)
What Do Contribution Guidelines Say About Software Testing? (MSR 2025)What Do Contribution Guidelines Say About Software Testing? (MSR 2025)
What Do Contribution Guidelines Say About Software Testing? (MSR 2025)
Andre Hora
 
Full Cracked Resolume Arena Latest Version
Full Cracked Resolume Arena Latest VersionFull Cracked Resolume Arena Latest Version
Full Cracked Resolume Arena Latest Version
jonesmichealj2
 
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
Scaling GraphRAG:  Efficient Knowledge Retrieval for Enterprise AIScaling GraphRAG:  Efficient Knowledge Retrieval for Enterprise AI
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
danshalev
 
Kubernetes_101_Zero_to_Platform_Engineer.pptx
Kubernetes_101_Zero_to_Platform_Engineer.pptxKubernetes_101_Zero_to_Platform_Engineer.pptx
Kubernetes_101_Zero_to_Platform_Engineer.pptx
CloudScouts
 
Apple Logic Pro X Crack FRESH Version 2025
Apple Logic Pro X Crack FRESH Version 2025Apple Logic Pro X Crack FRESH Version 2025
Apple Logic Pro X Crack FRESH Version 2025
fs4635986
 
Creating Automated Tests with AI - Cory House - Applitools.pdf
Creating Automated Tests with AI - Cory House - Applitools.pdfCreating Automated Tests with AI - Cory House - Applitools.pdf
Creating Automated Tests with AI - Cory House - Applitools.pdf
Applitools
 
Proactive Vulnerability Detection in Source Code Using Graph Neural Networks:...
Proactive Vulnerability Detection in Source Code Using Graph Neural Networks:...Proactive Vulnerability Detection in Source Code Using Graph Neural Networks:...
Proactive Vulnerability Detection in Source Code Using Graph Neural Networks:...
Ranjan Baisak
 
Automation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath CertificateAutomation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath Certificate
VICTOR MAESTRE RAMIREZ
 
Not So Common Memory Leaks in Java Webinar
Not So Common Memory Leaks in Java WebinarNot So Common Memory Leaks in Java Webinar
Not So Common Memory Leaks in Java Webinar
Tier1 app
 
Landscape of Requirements Engineering for/by AI through Literature Review
Landscape of Requirements Engineering for/by AI through Literature ReviewLandscape of Requirements Engineering for/by AI through Literature Review
Landscape of Requirements Engineering for/by AI through Literature Review
Hironori Washizaki
 
Odoo ERP for Education Management to Streamline Your Education Process
Odoo ERP for Education Management to Streamline Your Education ProcessOdoo ERP for Education Management to Streamline Your Education Process
Odoo ERP for Education Management to Streamline Your Education Process
iVenture Team LLP
 
Innovative Approaches to Software Dev no good at all
Innovative Approaches to Software Dev no good at allInnovative Approaches to Software Dev no good at all
Innovative Approaches to Software Dev no good at all
ayeshakanwal75
 
Ad

Streamlining End-to-End Testing Automation

  • 3. Ground Reality Distributed teams Hybrid working Different network setup and speeds Many teams (>100) Mac, Windows and Linux laptops Different Software versions Certificates, Policies and multiple VPNs Inconsistent Developer & SDET experience Test execution environment setup is tedious System Tests (e2e & component UI): teswiz (Appium, Selenium, Applitools, JDK 17) • Emulator/Simulator setup (Android SDK, xcode) API & API Workflow tests: karate Contract tests: Specmatic Unit tests, Sonar code quality checks Test automation toolset Many environments Test data Branch Configuring appropriate test execution in build and release pipelines Complex path to production ADO agents: Windows Server & Linux agents Firewall restrictions download dependencies Direct access prohibited to CI agents Multiple node & JDK versions Connectivity issues to application-under-test No browsers/devices on CI agents CI execution @BagmarAnand
  • 5. Getting a simple automated test to run consistently for all Developers and Testers and in CI (ADO) is painful! Setup Execution (full or specific tests) @BagmarAnand
  • 6. Challenges of End-2-End Test Automation Ensuring Test Environment Consistency Coordinated Test Execution Test setup & execution on CI Agents @BagmarAnand
  • 9. • Setup important applications on Mac • https://gist.github.com/anandbagmar/92b9f92298b1e17fa32c3404ad115 871 • Script to setup Android SDK on Mac • https://github.com/anandbagmar/AppiumJavaSample/blob/master/setup AndroidSDK.sh • Script to setup Android SDK on Linux • https://github.com/anandbagmar/AppiumJavaSample/blob/master/setup _linux.sh Test Authoring Environment Setup @BagmarAnand
  • 10. • Node script to install all dependencies (for system tests): • https://github.com/znsio/getting-started-with- teswiz/blob/main/package.json • npm install – and you are ready! Test Execution Environment Setup @BagmarAnand
  • 12. • Setup should be simple – Ex: • git pull • ./gradlew build • No code change required for • Running tests against any environment (local, dev, qa, staging, prod, etc.) • Test data and environment configurations are separately maintained • Running all or subset of tests • Tests should run from command-line Test Automation Framework Criteria @BagmarAnand
  • 13. Your Ultimate Open-Source Solution to Automate Real-User Scenarios! @BagmarAnand
  • 15. Run tests from CLI Test Authoring Execution Setup 1 2 2 3 4 5 6 6 Execution Reports CI Tool Feature coverage @BagmarAnand 16
  • 16. • Web browsers • Mobile-web browsers • Android apps • iOS apps • Windows desktop apps • Electron apps Platform support @BagmarAnand
  • 17. • Open source framework to automate real-user scenarios • Multi-user • Multi-device • Multi-app • Setup a HARD-GATE for your functional tests! Unique capabilities of teswiz @BagmarAnand
  • 18. • Cloud device farm integrations • Applitools AI for validations • Comprehensive reports with trend analysis, feature coverage, failure analysis using AI-ML • CLI • Configurable Unique capabilities of teswiz Defaults Property files Environment Variables @BagmarAnand
  • 21. Use the right node version @BagmarAnand
  • 22. Use the right node version @BagmarAnand
  • 24. • For System Tests, the artifact (apk/app) could have been generated from another pipeline • This artifact needs to be available in local or cloud device before tests can start execution Script for downloading artifacts – Why? @BagmarAnand
  • 25. • Understand the CI tool APIs • Script downloads the android/iOS artifact for: • Specific branch • Latest successful build, or a specific build number Script for downloading artifacts @BagmarAnand
  • 26. • Script uploads the android/iOS artifact to your device farm OR • Teswiz can upload it automatically for you Script for downloading artifacts – Bonus! @BagmarAnand
  • 28. • Understand what dependencies in your framework need proxy information. Ex: • Gradle/maven • Downloading newer version of browser drivers • Any external connectivity • Framework should be configurable to pass this at test execution time. Ex: • No proxy required from local laptop execution • Proxy required when running tests from CI Proxy Handling @BagmarAnand
  • 31. Downloading dependencies - Uber jar #6 @BagmarAnand
  • 32. • To reduce the number of dependencies to be downloaded, teswiz is built as a uber jar. • Specify only “teswiz” as a dependency in your test framework Uber jar @BagmarAnand
  • 33. • Run as a java process • ./gradlew run Uber jar @BagmarAnand
  • 35. • CI agents may not have browsers installed • The installed browser may be an older version Run browser in docker – Why? @BagmarAnand
  • 36. • Should allow support for any os/architecture • Should allow choosing the browser (ex: firefox, chrome, etc.) • Should allow starting the containers with specific project names and dynamic ports to prevent conflict with multiple test executions • Support specifying proxy information • Can be used on local laptops as well as in CI executions Run browser in docker @BagmarAnand https://github.com/znsio/teswiz/blob/main/dockerContainers.sh https://github.com/znsio/teswiz/blob/main/docker-compose-v3.yml
  • 37. Template for running tests in build pipeline #8 @BagmarAnand
  • 39. • Create templates • Reuse with appropriate configuration parameters Running tests in Build Pipeline @BagmarAnand
  • 41. Task Groups for running tests in release pipeline #9 @BagmarAnand
  • 43. • Create Task Groups • Include in each relevant stage of Release pipeline Running tests in Release Pipeline @BagmarAnand
  • 50. Hard Gate Make your tests valuable! #10 @BagmarAnand
  • 51. @BagmarAnand What is a Hard Gate! Why is it required? https://github.com/znsio/teswiz/blob/main/docs/HardGate.md • Automated tests should allow you to take decisions on product quality
  • 52. @BagmarAnand What is a Hard Gate! Why is it required? https://github.com/znsio/teswiz/blob/main/docs/HardGate.md • For every test execution cycle: • Passing tests are expected to pass • Known Failing tests are supposed to fail, unless: • The product (bug) is fixed, OR • The test is fixed/updated If either criteria is not met, the build should fail!
  • 53. @BagmarAnand Hard Gate - Make your tests valuable! https://github.com/znsio/teswiz/blob/main/docs/HardGate.md Build passes if Hard Gate criteria is met. Build fails if • one or more passing tests have failed, or, • one or more failing tests have passed
  • 54. @BagmarAnand Functional (e2e) Automation as Hard Gate! https://github.com/znsio/teswiz/blob/main/docs/HardGate.md
  • 55. Applitools Visual AI for validations #11 @BagmarAnand
  • 57. 90% less code to write & maintain with infinite coverage. WITH APPLITOOLS AI Every Element Is Validated They Look & Work Perfect Use Applitools Ultrafast Grid (UFG) - Test is simpler – one call to Applitools (eyes.checkWindow()) validates the full screen - Run the test once - Get results from all browsers automatically - Less test data - No additional load on the application environment @BagmarAnand
  • 58. • Works for all platforms • Native & hybrid apps – android, iOS • Web browsers • Desktop applications • Electron applications • Seamless scaling using Applitools Ultrafast Grid @BagmarAnand Applitools Visual AI
  • 59. Specify as many browsers with viewports and devices as required for validation You do not need to do cross-browser validation at the end anymore! @BagmarAnand AI-powered Cross Browser Test Automation
  • 60. reportportal as a Central reporting server #12 @BagmarAnand
  • 62. A central reporting server for your organization @BagmarAnand
  • 63. Test Execution Real-time status • See progress of launches currently in progress • Can also see details of tests that are currently running, till the point of execution @BagmarAnand
  • 64. Test Execution Details – Device farm report link & Device logs • The link to the device farm test execution dashboard is available in the result • teswiz attaches browser logs/device logs automatically to the result in ReportPortal @BagmarAnand
  • 65. Test Execution Details – with screenshots • The test result includes screenshots as captured by the test @BagmarAnand
  • 66. Test Execution Details – Applitools Visual AI Validation Results • The test result includes the status of Applitools Visual AI validation • Link to the Applitools dashboard is available in the result @BagmarAnand
  • 67. Test Execution Trend Analysis • Each test shows the trend of its execution – giving an indication of (in)stability @BagmarAnand
  • 68. Test Results – Next Steps • On investigation of the failed tests, mark the failures with appropriate reasons (as configured) @BagmarAnand
  • 69. Auto-analysis of failed tests Analyse the failure reasons by Auto-Analyzer based on Machine Learning @BagmarAnand
  • 70. Auto Analysis of Test Failures • Why waste time marking the test failed for the same reason as last time? • ReportPortal can do this automatically for you with the Auto Analysis and Pattern Analysis feature @BagmarAnand
  • 71. Auto Analysis of Test Failures @BagmarAnand
  • 72. Test Result Visualization Configure simple and understandable reports • Create as many dashboards as relevant for the team • Dashboards may be for different persona/role, giving appropriate information @BagmarAnand
  • 73. • Teswiz and karate test frameworks can automatically upload test results to your reportportal server • sendToReportPortal: • https://github.com/znsio/sendToReportPortal/blob/main/importRes ultsAndUpdateAttributes.sh • Can upload junit test results generated by any type of tests to reportportal with relevant test execution metadata reportportal.io @BagmarAnand
  • 74. Challenges • Ensuring Test Environment Consistency • Coordinated Test Execution • Test setup & execution on CI Agents Solutions Consistent environment setup Test Automation Framework support Node setup Script for downloading artifacts Proxy Handling Downloading dependencies - Uber Jar Browsers in docker Template for build pipelines Task groups for release pipelines Hard Gate AI for validations Central reporting server Summary @BagmarAnand