SlideShare a Scribd company logo
August 26, 2015
“Accelerate Software Delivery with Continuous Integration”
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA
© 2014 SOASTA. All rights reserved. 2
Agenda:
• SOASTA Introduction
• The need for “Continuous”
• The “Big 3”
• CI with mobile and performance testing
© 2014 SOASTA. All rights reserved. 3
100 Billion
User Experiences Tested
10 Million
Tests Performed
Actual CloudTest
view
© 2014 SOASTA. All rights reserved. 4
Agenda:
• SOASTA Introduction
• The need for “Continuous”
• The “Big 3”
• CI with mobile and performance testing
© 2014 SOASTA. All rights reserved. 5
© 2014 SOASTA. All rights reserved. 6
o Small Batch Sizes
© 2014 SOASTA. All rights reserved. 7
o Small Batch Sizes
o Comprehensive Version Control
© 2014 SOASTA. All rights reserved. 8
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
© 2014 SOASTA. All rights reserved. 9
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
© 2014 SOASTA. All rights reserved. 10
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
o Continuous Feedback
© 2014 SOASTA. All rights reserved. 11
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
o Continuous Feedback
o Emphasis on working builds
© 2014 SOASTA. All rights reserved. 12
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
o Continuous Feedback
o Emphasis on working builds
o Consistent environments
© 2014 SOASTA. All rights reserved. 13
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
o Continuous Feedback
o Emphasis on working builds
o Consistent environments
o Developer – Tester Collaboration
© 2014 SOASTA. All rights reserved. 14
Agenda:
• SOASTA Introduction
• The need for “Continuous”
• The “Big 3”
• CI with mobile and performance testing
© 2014 SOASTA. All rights reserved. 15
© 2014 SOASTA. All rights reserved. 16
o It’s not what’s within the silos
o It’s about the relationships
A typical silo’d organization lacks end-to-end integration
Performance Engineering & Continuous Integration-Full Lifecycle
Operational Configuration
Continuous Integration touchpoints
ADLC
QA/Test Pre-
Production
Production
QA/Test
Environment
Operational
Production
Environment
Application
Development
Environment
Development Test
Configuration management
Release management
Change management
Performance management
CI Focus Areas – The assembly line Hand-offs
Key Processes
© 2014 SOASTA. All rights reserved. 19
o Bill of Materials (BOM!)
o The “Big 3” => Change, Configuration & Release
• That is your software assembly line conveyor belt
© 2014 SOASTA. All rights reserved. 20
• We can only test as fast as the conveyor belt allows.
• If the conveyor belt is moving slow, testing is delayed.
• The Big 3 drives testing cycle.
o Why is CI critical?
• Speed
• New features, functions, etc. need to get to the customer, ASAP!
© 2014 SOASTA. All rights reserved. 21
o CI is a key component of Performance Engineering
o Performance Engineering spans the silos
BIG 3
Testing
Application
Development
Life Cycle
IT
Operations
Capacity
Management
Performance
Management
Performance
Engineering
© 2014 SOASTA. All rights reserved. 22
Does it work as
designed?
Does it perform
under extreme
circumstances?
Approve materials Real Drivers
What happens
in actual
conditions?
Assembly Line
© 2014 SOASTA. All rights reserved. 23
Functional validation
Performance,
reliability,
scalability tests
Real Users
Build
What happens
in actual
conditions?
Unit-tested code
© 2014 SOASTA. All rights reserved. 24
Agenda:
• SOASTA Introduction
• The need for “Continuous”
• The “Big 3”
• CI with mobile and performance testing
© 2014 SOASTA. All rights reserved. 25
© 2014 SOASTA. All rights reserved. 26
Jenkins runs any
unit tests as usual
© 2014 SOASTA. All rights reserved. 27
Run iterative load
tests via Jenkins
Spin up suitable, consistent
test environments in the
cloud or the lab
Jenkins runs any
unit tests as usual
© 2014 SOASTA. All rights reserved. 28
Run iterative load
tests via Jenkins
Mobile? Run functional
automation on real devices
for functional & performance
validation
Spin up suitable, consistent
test environments in the
cloud or the lab
Jenkins runs any
unit tests as usual
© 2014 SOASTA. All rights reserved. 29
Run iterative load
tests via Jenkins
Daily (or more frequent) updates
keep teams quality focused
Embedded failure
details & bottleneck
data isolate issues
Mobile? Run functional
automation on real devices
for functional & performance
validation
Spin up suitable, consistent
test environments in the
cloud or the lab
Jenkins runs any
unit tests as usual
© 2014 SOASTA. All rights reserved. 30
Run iterative load
tests via Jenkins
Daily (or more frequent) updates
keep teams quality focused
Embedded failure
details & bottleneck
data isolate issues
Manage to a
performance
baseline
Mobile? Run functional
automation on real devices
for functional & performance
validation
Spin up suitable, consistent
test environments in the
cloud or the lab
Jenkins runs any
unit tests as usual
© 2014 SOASTA. All rights reserved. 31
Use real user data to
iterate dev & test plans
Run iterative load
tests via Jenkins
Daily (or more frequent) updates
keep teams quality focused
Embedded failure
details & bottleneck
data isolate issues
Manage to a
performance
baseline
Mobile? Run functional
automation on real devices
for functional & performance
validation
Spin up suitable, consistent
test environments in the
cloud or the lab
Jenkins runs any
unit tests as usual
TechWell Accelerating Software Delivery with Continuous Integration feat. Dan Boutin
SOASTA – Branch Build from CloudTest
SOASTA – Test Results
SOASTA – Drill Down (Test Failures)
SOASTA – UI Testing Bundle Results
SOASTA – UI Testing Failure Drill down into
SOASTA CloudTest from Jenkins
SOASTA – Successful regression test results
SOASTA – Release Promotion
© 2014 SOASTA. All rights reserved. 40
o SOASTA’s Process Metrics
• 3 Main Products
• (2) Builds per day of each product (Dev branch + Customer branch)
• 8,000 tests executed per build
• For Mobile (TouchTest), Over 300 fully-automated tests each for iOS & Android
© 2014 SOASTA. All rights reserved. 41
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
© 2014 SOASTA. All rights reserved. 42
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
© 2014 SOASTA. All rights reserved. 43
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
© 2014 SOASTA. All rights reserved. 44
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
© 2014 SOASTA. All rights reserved. 45
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
© 2014 SOASTA. All rights reserved. 46CONFIDENTIAL – Not for Distribution
Revenue
Brand
Competitive advantage
© 2014 SOASTA. All rights reserved. 47
We will close with some Mobile Trivia
@DanBoutinSOASTA
© 2014 SOASTA. All rights reserved. 48
Questions?
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA
o Visit our Blog: http://goo.gl/TA2HKP
o Questions? @DanBoutinSOASTA
August 26, 2015
“Accelerate Software Delivery with Continuous Integration”
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA

More Related Content

What's hot (20)

PPT
SOASTA Webinar: Process Compression For Mobile App Dev 120612
SOASTA
 
PDF
What Retailers Know About Performance and User Experience
SOASTA
 
PPTX
Soasta | CloudBees webinar 11/12/2015
SOASTA
 
PPTX
Secrets to Realistic Load Testing
SOASTA
 
PPTX
Final tips holiday readiness 2015 for slide share
SOASTA
 
PPT
New Features in CloudTest & TouchTest
Jennifer Finney
 
PDF
How To Introduce Cloud Based Load Testing to Your Jenkins Continuous Delivery...
Jennifer Finney
 
PPTX
O'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major Events
SOASTA
 
PPTX
Integrating Cloud-based performance test in VSTS with SOASTA CloudTest
Jennifer Finney
 
PPTX
Webinar: Load Testing for Your Peak Season
SOASTA
 
PPT
Agile Load Testing In The Real World
SOASTA
 
PDF
Soasta New Features in CloudTest & TouchTest Webinar
Jennifer Finney
 
PPTX
Building a Performance A-Team
SOASTA
 
PPT
Continuous Load Testing with CloudTest and Jenkins
SOASTA
 
PPTX
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
SOASTA
 
PPTX
Performance Warrior Tales: Cloud Load Testing the Retail Giants
SOASTA
 
PPTX
Webinar: Are you ready for your peak season?
Jennifer Finney
 
PPTX
Get Ready for Changes To Load Testing
SOASTA
 
PDF
7 steps to pragmatic mobile testing
SOASTA
 
PPTX
Adopting Cloud Testing for Continuous Delivery, with the premier global provi...
SOASTA
 
SOASTA Webinar: Process Compression For Mobile App Dev 120612
SOASTA
 
What Retailers Know About Performance and User Experience
SOASTA
 
Soasta | CloudBees webinar 11/12/2015
SOASTA
 
Secrets to Realistic Load Testing
SOASTA
 
Final tips holiday readiness 2015 for slide share
SOASTA
 
New Features in CloudTest & TouchTest
Jennifer Finney
 
How To Introduce Cloud Based Load Testing to Your Jenkins Continuous Delivery...
Jennifer Finney
 
O'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major Events
SOASTA
 
Integrating Cloud-based performance test in VSTS with SOASTA CloudTest
Jennifer Finney
 
Webinar: Load Testing for Your Peak Season
SOASTA
 
Agile Load Testing In The Real World
SOASTA
 
Soasta New Features in CloudTest & TouchTest Webinar
Jennifer Finney
 
Building a Performance A-Team
SOASTA
 
Continuous Load Testing with CloudTest and Jenkins
SOASTA
 
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
SOASTA
 
Performance Warrior Tales: Cloud Load Testing the Retail Giants
SOASTA
 
Webinar: Are you ready for your peak season?
Jennifer Finney
 
Get Ready for Changes To Load Testing
SOASTA
 
7 steps to pragmatic mobile testing
SOASTA
 
Adopting Cloud Testing for Continuous Delivery, with the premier global provi...
SOASTA
 

Viewers also liked (12)

PDF
161109 jornada eficiència associació congost
Empresa i Emprenedoria Granollers
 
PPTX
Patricia and the nature
Mariana Morales
 
PPTX
PMISSC_Outreach Committee Overview rev 11 2015
PMI Silver Spring
 
PDF
PUBLISHED_House Committee on Natural Resources Interim Report 2013
Elizabeth Fazio
 
PDF
Rías Baixas | zonas ZEPA
Alberte Murado Lorenzo
 
DOCX
Mongo db installation procedure for win 7
Otto-von-guericke university magdeburg
 
PPTX
Our houses and school: Darío and Ángel
Mariana Morales
 
PDF
Webinar: New Features in CloudTest & TouchTest
SOASTA
 
PDF
Fase Pruebas
johannaAC
 
PDF
Proceso administracionpsl
johannaAC
 
PPTX
Puntos clave para la correcta elecceción de un Operador Logístico Internacion...
Instituto de Promoción Exterior de Castilla La Mancha
 
PPTX
Cómo utilizar slideshare
Lilianfabianna
 
161109 jornada eficiència associació congost
Empresa i Emprenedoria Granollers
 
Patricia and the nature
Mariana Morales
 
PMISSC_Outreach Committee Overview rev 11 2015
PMI Silver Spring
 
PUBLISHED_House Committee on Natural Resources Interim Report 2013
Elizabeth Fazio
 
Rías Baixas | zonas ZEPA
Alberte Murado Lorenzo
 
Mongo db installation procedure for win 7
Otto-von-guericke university magdeburg
 
Our houses and school: Darío and Ángel
Mariana Morales
 
Webinar: New Features in CloudTest & TouchTest
SOASTA
 
Fase Pruebas
johannaAC
 
Proceso administracionpsl
johannaAC
 
Puntos clave para la correcta elecceción de un Operador Logístico Internacion...
Instituto de Promoción Exterior de Castilla La Mancha
 
Cómo utilizar slideshare
Lilianfabianna
 
Ad

Similar to TechWell Accelerating Software Delivery with Continuous Integration feat. Dan Boutin (20)

PPTX
Nova event-s florida-dboutin
Dan Boutin
 
PDF
They don't call it Continuous Integration for nothing!
Dan Boutin
 
PPTX
Accelerate Web and Mobile Testing for Continuous Integration and Delivery
SOASTA
 
PPTX
Serena Webcast: Accelerating Application Delivery with Continuous Testing
Serena Software
 
PPT
Continuous Load Testing with CloudTest and Jenkins
SOASTA
 
PPTX
STPCon fall 2012: The Testing Renaissance Has Arrived
SOASTA
 
PPTX
Continuous Testing for Optimal Mobile Peroformance - STPCon Spring 2014
SOASTA
 
PPTX
Continuous Testing for Optimal Mobile Peroformance - STPCon Spring 2014
SOASTA
 
PPTX
Sd times-june-24-2015
Dan Boutin
 
PPTX
Techniques, Tips & Tools For Mobile App Testing
SOASTA
 
PPTX
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
SOASTA
 
PPTX
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
SOASTA
 
PDF
ConFoo: Moving web performance testing to the left
Tom Chavez
 
PPTX
The Mobile Testing Checklist
SOASTA
 
PDF
Continuous Delivery Testing @HiQ
Tomas Riha
 
PDF
DOD 2016 - Diogo Oliveira - The OutSystems R&D Continuous Delivery Journey
PROIDEA
 
PPTX
Role of Test Automation in Modern Software Delivery Pipelines
Kasun Kodagoda
 
PPTX
Agile Engineering Sparker GLASScon 2015
Stephen Ritchie
 
PPTX
Test Automation and Continuous Integration
TestCampRO
 
PDF
Large-Scale Agile Test Automation Strategies in Practice
TechWell
 
Nova event-s florida-dboutin
Dan Boutin
 
They don't call it Continuous Integration for nothing!
Dan Boutin
 
Accelerate Web and Mobile Testing for Continuous Integration and Delivery
SOASTA
 
Serena Webcast: Accelerating Application Delivery with Continuous Testing
Serena Software
 
Continuous Load Testing with CloudTest and Jenkins
SOASTA
 
STPCon fall 2012: The Testing Renaissance Has Arrived
SOASTA
 
Continuous Testing for Optimal Mobile Peroformance - STPCon Spring 2014
SOASTA
 
Continuous Testing for Optimal Mobile Peroformance - STPCon Spring 2014
SOASTA
 
Sd times-june-24-2015
Dan Boutin
 
Techniques, Tips & Tools For Mobile App Testing
SOASTA
 
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
SOASTA
 
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
SOASTA
 
ConFoo: Moving web performance testing to the left
Tom Chavez
 
The Mobile Testing Checklist
SOASTA
 
Continuous Delivery Testing @HiQ
Tomas Riha
 
DOD 2016 - Diogo Oliveira - The OutSystems R&D Continuous Delivery Journey
PROIDEA
 
Role of Test Automation in Modern Software Delivery Pipelines
Kasun Kodagoda
 
Agile Engineering Sparker GLASScon 2015
Stephen Ritchie
 
Test Automation and Continuous Integration
TestCampRO
 
Large-Scale Agile Test Automation Strategies in Practice
TechWell
 
Ad

More from SOASTA (20)

PPTX
DPM in Pictures
SOASTA
 
PPTX
Optimizing your marketing promotions to mazimize your revenue
SOASTA
 
PPTX
Using JMeter in CloudTest for Continuous Testing
SOASTA
 
PPTX
Velocity Booth Session - Better Together: RUM & Synthetic
SOASTA
 
PPTX
Velocity Booth Presentation - Which 3rd Party Resources Are Eating Your Profits?
SOASTA
 
PPTX
Velocity 2016 Speaking Session - Using Machine Learning to Determine Drivers ...
SOASTA
 
PPTX
Velocity 15 Minute Booth Session - Building a Performance Team - Dave Murphy
SOASTA
 
PPTX
Radial | SOASTA IR Webinar
SOASTA
 
PPTX
IRCE 2016 Speaking Session – The Small Things That Add Up: How to Find What D...
SOASTA
 
PDF
Ann Ruckstuhl eTail West
SOASTA
 
PPTX
Rewriting The Revenue Rules: From Mobile-First To Mobile-Only Mobile Shopping...
SOASTA
 
PDF
Forensic tools for in-depth performance investigations
SOASTA
 
PPTX
Metrics, metrics everywhere (but where the heck do you start?)
SOASTA
 
PPTX
Webinar - Success Factors Behind Successful Flash Sales
SOASTA
 
PPTX
Business Value of Performance - Ann Ruckstuhl CMO DOC
SOASTA
 
PPTX
eTail East 2015 - Ann Ruckstuhl Presentation
SOASTA
 
PPTX
State of the Union - Mobile Performance
SOASTA
 
PPTX
20,000 Hours in the Cloud - Top 5 Cloud Lessons Learned By Tom Lounibos, CEO ...
SOASTA
 
PPTX
The Four Hats of Load and Performance Testing with special guest Mentora
SOASTA
 
PPTX
EMEA Webinar - An Introduction to Real User Measurement
SOASTA
 
DPM in Pictures
SOASTA
 
Optimizing your marketing promotions to mazimize your revenue
SOASTA
 
Using JMeter in CloudTest for Continuous Testing
SOASTA
 
Velocity Booth Session - Better Together: RUM & Synthetic
SOASTA
 
Velocity Booth Presentation - Which 3rd Party Resources Are Eating Your Profits?
SOASTA
 
Velocity 2016 Speaking Session - Using Machine Learning to Determine Drivers ...
SOASTA
 
Velocity 15 Minute Booth Session - Building a Performance Team - Dave Murphy
SOASTA
 
Radial | SOASTA IR Webinar
SOASTA
 
IRCE 2016 Speaking Session – The Small Things That Add Up: How to Find What D...
SOASTA
 
Ann Ruckstuhl eTail West
SOASTA
 
Rewriting The Revenue Rules: From Mobile-First To Mobile-Only Mobile Shopping...
SOASTA
 
Forensic tools for in-depth performance investigations
SOASTA
 
Metrics, metrics everywhere (but where the heck do you start?)
SOASTA
 
Webinar - Success Factors Behind Successful Flash Sales
SOASTA
 
Business Value of Performance - Ann Ruckstuhl CMO DOC
SOASTA
 
eTail East 2015 - Ann Ruckstuhl Presentation
SOASTA
 
State of the Union - Mobile Performance
SOASTA
 
20,000 Hours in the Cloud - Top 5 Cloud Lessons Learned By Tom Lounibos, CEO ...
SOASTA
 
The Four Hats of Load and Performance Testing with special guest Mentora
SOASTA
 
EMEA Webinar - An Introduction to Real User Measurement
SOASTA
 

Recently uploaded (20)

PDF
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
PDF
LLMs.txt: Easily Control How AI Crawls Your Site
Keploy
 
PPTX
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
PDF
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
PDF
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
PDF
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
 
PDF
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
PDF
HubSpot Main Hub: A Unified Growth Platform
Jaswinder Singh
 
PDF
Python basic programing language for automation
DanialHabibi2
 
PPTX
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
PDF
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PDF
DevBcn - Building 10x Organizations Using Modern Productivity Metrics
Justin Reock
 
PDF
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
PDF
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
 
PDF
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
PDF
Timothy Rottach - Ramp up on AI Use Cases, from Vector Search to AI Agents wi...
AWS Chicago
 
PDF
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PDF
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
PDF
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
LLMs.txt: Easily Control How AI Crawls Your Site
Keploy
 
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
 
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
HubSpot Main Hub: A Unified Growth Platform
Jaswinder Singh
 
Python basic programing language for automation
DanialHabibi2
 
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
DevBcn - Building 10x Organizations Using Modern Productivity Metrics
Justin Reock
 
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
 
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
Timothy Rottach - Ramp up on AI Use Cases, from Vector Search to AI Agents wi...
AWS Chicago
 
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 

TechWell Accelerating Software Delivery with Continuous Integration feat. Dan Boutin

  • 1. August 26, 2015 “Accelerate Software Delivery with Continuous Integration” Dan Boutin – Senior Product Evangelist [email protected] Mobile (404) 304-9529 @DanBoutinSOASTA
  • 2. © 2014 SOASTA. All rights reserved. 2 Agenda: • SOASTA Introduction • The need for “Continuous” • The “Big 3” • CI with mobile and performance testing
  • 3. © 2014 SOASTA. All rights reserved. 3 100 Billion User Experiences Tested 10 Million Tests Performed Actual CloudTest view
  • 4. © 2014 SOASTA. All rights reserved. 4 Agenda: • SOASTA Introduction • The need for “Continuous” • The “Big 3” • CI with mobile and performance testing
  • 5. © 2014 SOASTA. All rights reserved. 5
  • 6. © 2014 SOASTA. All rights reserved. 6 o Small Batch Sizes
  • 7. © 2014 SOASTA. All rights reserved. 7 o Small Batch Sizes o Comprehensive Version Control
  • 8. © 2014 SOASTA. All rights reserved. 8 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies
  • 9. © 2014 SOASTA. All rights reserved. 9 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation
  • 10. © 2014 SOASTA. All rights reserved. 10 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback
  • 11. © 2014 SOASTA. All rights reserved. 11 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback o Emphasis on working builds
  • 12. © 2014 SOASTA. All rights reserved. 12 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback o Emphasis on working builds o Consistent environments
  • 13. © 2014 SOASTA. All rights reserved. 13 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback o Emphasis on working builds o Consistent environments o Developer – Tester Collaboration
  • 14. © 2014 SOASTA. All rights reserved. 14 Agenda: • SOASTA Introduction • The need for “Continuous” • The “Big 3” • CI with mobile and performance testing
  • 15. © 2014 SOASTA. All rights reserved. 15
  • 16. © 2014 SOASTA. All rights reserved. 16 o It’s not what’s within the silos o It’s about the relationships
  • 17. A typical silo’d organization lacks end-to-end integration Performance Engineering & Continuous Integration-Full Lifecycle Operational Configuration Continuous Integration touchpoints ADLC QA/Test Pre- Production Production QA/Test Environment Operational Production Environment Application Development Environment
  • 18. Development Test Configuration management Release management Change management Performance management CI Focus Areas – The assembly line Hand-offs Key Processes
  • 19. © 2014 SOASTA. All rights reserved. 19 o Bill of Materials (BOM!) o The “Big 3” => Change, Configuration & Release • That is your software assembly line conveyor belt
  • 20. © 2014 SOASTA. All rights reserved. 20 • We can only test as fast as the conveyor belt allows. • If the conveyor belt is moving slow, testing is delayed. • The Big 3 drives testing cycle. o Why is CI critical? • Speed • New features, functions, etc. need to get to the customer, ASAP!
  • 21. © 2014 SOASTA. All rights reserved. 21 o CI is a key component of Performance Engineering o Performance Engineering spans the silos BIG 3 Testing Application Development Life Cycle IT Operations Capacity Management Performance Management Performance Engineering
  • 22. © 2014 SOASTA. All rights reserved. 22 Does it work as designed? Does it perform under extreme circumstances? Approve materials Real Drivers What happens in actual conditions? Assembly Line
  • 23. © 2014 SOASTA. All rights reserved. 23 Functional validation Performance, reliability, scalability tests Real Users Build What happens in actual conditions? Unit-tested code
  • 24. © 2014 SOASTA. All rights reserved. 24 Agenda: • SOASTA Introduction • The need for “Continuous” • The “Big 3” • CI with mobile and performance testing
  • 25. © 2014 SOASTA. All rights reserved. 25
  • 26. © 2014 SOASTA. All rights reserved. 26 Jenkins runs any unit tests as usual
  • 27. © 2014 SOASTA. All rights reserved. 27 Run iterative load tests via Jenkins Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
  • 28. © 2014 SOASTA. All rights reserved. 28 Run iterative load tests via Jenkins Mobile? Run functional automation on real devices for functional & performance validation Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
  • 29. © 2014 SOASTA. All rights reserved. 29 Run iterative load tests via Jenkins Daily (or more frequent) updates keep teams quality focused Embedded failure details & bottleneck data isolate issues Mobile? Run functional automation on real devices for functional & performance validation Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
  • 30. © 2014 SOASTA. All rights reserved. 30 Run iterative load tests via Jenkins Daily (or more frequent) updates keep teams quality focused Embedded failure details & bottleneck data isolate issues Manage to a performance baseline Mobile? Run functional automation on real devices for functional & performance validation Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
  • 31. © 2014 SOASTA. All rights reserved. 31 Use real user data to iterate dev & test plans Run iterative load tests via Jenkins Daily (or more frequent) updates keep teams quality focused Embedded failure details & bottleneck data isolate issues Manage to a performance baseline Mobile? Run functional automation on real devices for functional & performance validation Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
  • 33. SOASTA – Branch Build from CloudTest
  • 34. SOASTA – Test Results
  • 35. SOASTA – Drill Down (Test Failures)
  • 36. SOASTA – UI Testing Bundle Results
  • 37. SOASTA – UI Testing Failure Drill down into SOASTA CloudTest from Jenkins
  • 38. SOASTA – Successful regression test results
  • 39. SOASTA – Release Promotion
  • 40. © 2014 SOASTA. All rights reserved. 40 o SOASTA’s Process Metrics • 3 Main Products • (2) Builds per day of each product (Dev branch + Customer branch) • 8,000 tests executed per build • For Mobile (TouchTest), Over 300 fully-automated tests each for iOS & Android
  • 41. © 2014 SOASTA. All rights reserved. 41 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
  • 42. © 2014 SOASTA. All rights reserved. 42 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
  • 43. © 2014 SOASTA. All rights reserved. 43 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
  • 44. © 2014 SOASTA. All rights reserved. 44 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
  • 45. © 2014 SOASTA. All rights reserved. 45 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
  • 46. © 2014 SOASTA. All rights reserved. 46CONFIDENTIAL – Not for Distribution Revenue Brand Competitive advantage
  • 47. © 2014 SOASTA. All rights reserved. 47 We will close with some Mobile Trivia @DanBoutinSOASTA
  • 48. © 2014 SOASTA. All rights reserved. 48 Questions? Dan Boutin – Senior Product Evangelist [email protected] Mobile (404) 304-9529 @DanBoutinSOASTA o Visit our Blog: http://goo.gl/TA2HKP o Questions? @DanBoutinSOASTA
  • 49. August 26, 2015 “Accelerate Software Delivery with Continuous Integration” Dan Boutin – Senior Product Evangelist [email protected] Mobile (404) 304-9529 @DanBoutinSOASTA