SlideShare a Scribd company logo
Continuous Testing
Meets the Classroom
bcjordan.com
brian@code.org
@bcjordan
Brian Jordan, software engineer at Code.org
Code.org
non-profit
expanding
participation in CS
Hour of Code
Continuous Testing Meets the Classroom at Code.org
Continuous Testing Meets the Classroom at Code.org
200 million tried
Hour of Code
now what?
full curricula,
district partnerships,
professional
development,
policy change
K-12 CS Curriculum
20k K-12 teachers trained
intro courses:
260k teachers
8mil students
Continuous Testing Meets the Classroom at Code.org
how do you even start
automated testing?
developing in
2013-2014
why test?
what is Code.org's
testing context?
the context
the context
the context
the context
the context
so how do we
test all that?
how we started
Selenium testing
how our tests are
organized and run
selenium-webdriver
cucumber
takeaways
Build a library of
steps
Build a library of
test levels
Build a library of
annotations
@no_mobile
@no_ie9
@skip
@db_access@as_student
who writes tests?
who/what
runs tests?
Continuous Testing Meets the Classroom at Code.org
what challenges
did we end up facing?
challenge:
multiple browsers
solution:
selenium-webdriver
Sauce Labs
Continuous Testing Meets the Classroom at Code.org
Continuous Testing Meets the Classroom at Code.org
Continuous Testing Meets the Classroom at Code.org
challenge:
testing local changes
solution:
chromedriver
sauce-connect
challenge:
interaction,
drag-and-drop
solution:
jQuery Simulate
$(selector).drag…
challenge:
handling
(intermittent)
failures
solution:
always fixing!
solution:
always fixing!
(try rerunning!)
what causes
intermittent failures?
Sauce Labs API
test_flakiness.rb
what do they look like
when running?
Speed Run!
(not) wasting time
speeding up
test runs
Continuous Testing Meets the Classroom at Code.org
2014:
1-2h of tests
parallelization
at runner level
2016:
20-30min of tests
justifying
tests
the bug collection
Continuous Testing Meets the Classroom at Code.org
Continuous Testing Meets the Classroom at Code.org
Firefox getBBox()
my favorite
bug ever
Continuous Testing Meets the Classroom at Code.org
Continuous Testing Meets the Classroom at Code.org
Continuous Testing Meets the Classroom at Code.org
Continuous Testing Meets the Classroom at Code.org
who
investigates
failures?
dev of the day
what's next?
per-PR tests
takeaways
start small
everybody
tests
periodically
invest in
speed-ups
Q&A
bcjordan.com
brian@code.org
@bcjordan
Brian Jordan, software engineer at Code.org
code.org/help
github.com/code-dot-org

More Related Content

What's hot (20)

PDF
Shift left-csun-sagar-barbhaya
SAGAR BARBHAYA
 
PPT
Automated Testing With Jasmine, PhantomJS and Jenkins
Work at Play
 
PPTX
Test Design + Environment Management: Scaling UI Automation for Agile
SmartBear
 
PDF
Getting Started with Selenium
Dave Haeffner
 
PPTX
ASP.NET & Unit Testing
Shahed Chowdhuri
 
PPTX
Manual, Visual, and Automated Testing For Web Apps
SmartBear
 
PPTX
Getting Started With Selenium
SmartBear
 
PPTX
Cypress test techniques cucumber bdd framework,tdd,api tests course
Narayanan Palani
 
PDF
Web Application Testing with Selenium
Sargis Sargsyan
 
PDF
DevQA: make your testers happier with Groovy, Spock and Geb (Greach 2014)
Alvaro Sanchez-Mariscal
 
PPTX
QA Automation testing online training
Divya Shree
 
PPTX
Selenium and Appium Training from Sauce Labs
Sauce Labs
 
PDF
How To Use Selenium Successfully
Dave Haeffner
 
PPTX
How to Get Started with Cypress
Applitools
 
PDF
Selenium Testing on Chrome - Google DevFest Armenia 2015
Sargis Sargsyan
 
PPTX
Three Developer Behaviors to Eliminate 85 Percent of Accessibility Defects
Sean Kelly
 
PDF
Selenium Best Practices with Jason Huggins
Sauce Labs
 
PDF
Testing Code.org's Interactive CS Curriculum
Brian Jordan
 
PDF
Automated testing in javascript
Michael Yagudaev
 
PPTX
Visual Studio 2010 Testing for Developers
Steve Lange
 
Shift left-csun-sagar-barbhaya
SAGAR BARBHAYA
 
Automated Testing With Jasmine, PhantomJS and Jenkins
Work at Play
 
Test Design + Environment Management: Scaling UI Automation for Agile
SmartBear
 
Getting Started with Selenium
Dave Haeffner
 
ASP.NET & Unit Testing
Shahed Chowdhuri
 
Manual, Visual, and Automated Testing For Web Apps
SmartBear
 
Getting Started With Selenium
SmartBear
 
Cypress test techniques cucumber bdd framework,tdd,api tests course
Narayanan Palani
 
Web Application Testing with Selenium
Sargis Sargsyan
 
DevQA: make your testers happier with Groovy, Spock and Geb (Greach 2014)
Alvaro Sanchez-Mariscal
 
QA Automation testing online training
Divya Shree
 
Selenium and Appium Training from Sauce Labs
Sauce Labs
 
How To Use Selenium Successfully
Dave Haeffner
 
How to Get Started with Cypress
Applitools
 
Selenium Testing on Chrome - Google DevFest Armenia 2015
Sargis Sargsyan
 
Three Developer Behaviors to Eliminate 85 Percent of Accessibility Defects
Sean Kelly
 
Selenium Best Practices with Jason Huggins
Sauce Labs
 
Testing Code.org's Interactive CS Curriculum
Brian Jordan
 
Automated testing in javascript
Michael Yagudaev
 
Visual Studio 2010 Testing for Developers
Steve Lange
 

Viewers also liked (20)

PPTX
Selenium
Bryan Mikaelian
 
PPTX
Web testing with Selenium
XBOSoft
 
PDF
Pivotal Failure - Lessons Learned from Lean Startup Machine DC
Dave Haeffner
 
PPT
Full Stack Testing Done Well
Dave Haeffner
 
PDF
KISS Automation.py
Iakiv Kramarenko
 
PPTX
Agile testing for mere mortals
Dave Haeffner
 
PDF
The Testable Web
Dave Haeffner
 
PDF
You do not need automation engineer - Sqa Days - 2015 - EN
Iakiv Kramarenko
 
PDF
Selenium Basics
Dave Haeffner
 
PDF
Web ui tests examples with selenide, nselene, selene & capybara
Iakiv Kramarenko
 
PPTX
Cross Platform Appium Tests: How To
GlobalLogic Ukraine
 
PDF
Three Simple Chords of Alternative PageObjects and Hardcore of LoadableCompon...
Iakiv Kramarenko
 
PDF
Polyglot automation - QA Fest - 2015
Iakiv Kramarenko
 
PDF
Bdd lessons-learned
Dave Haeffner
 
PPTX
Write Selenide in Python 15 min
Iakiv Kramarenko
 
PDF
Easy automation.py
Iakiv Kramarenko
 
PDF
Selenium Users Anonymous
Dave Haeffner
 
PDF
Selenide alternative in Python - Introducing Selene [SeleniumCamp 2016]
Iakiv Kramarenko
 
PPTX
Cucumber Crash Course
Dave Haeffner
 
PDF
Easy tests with Selenide and Easyb
Iakiv Kramarenko
 
Selenium
Bryan Mikaelian
 
Web testing with Selenium
XBOSoft
 
Pivotal Failure - Lessons Learned from Lean Startup Machine DC
Dave Haeffner
 
Full Stack Testing Done Well
Dave Haeffner
 
KISS Automation.py
Iakiv Kramarenko
 
Agile testing for mere mortals
Dave Haeffner
 
The Testable Web
Dave Haeffner
 
You do not need automation engineer - Sqa Days - 2015 - EN
Iakiv Kramarenko
 
Selenium Basics
Dave Haeffner
 
Web ui tests examples with selenide, nselene, selene & capybara
Iakiv Kramarenko
 
Cross Platform Appium Tests: How To
GlobalLogic Ukraine
 
Three Simple Chords of Alternative PageObjects and Hardcore of LoadableCompon...
Iakiv Kramarenko
 
Polyglot automation - QA Fest - 2015
Iakiv Kramarenko
 
Bdd lessons-learned
Dave Haeffner
 
Write Selenide in Python 15 min
Iakiv Kramarenko
 
Easy automation.py
Iakiv Kramarenko
 
Selenium Users Anonymous
Dave Haeffner
 
Selenide alternative in Python - Introducing Selene [SeleniumCamp 2016]
Iakiv Kramarenko
 
Cucumber Crash Course
Dave Haeffner
 
Easy tests with Selenide and Easyb
Iakiv Kramarenko
 
Ad

Similar to Continuous Testing Meets the Classroom at Code.org (20)

PDF
Tests for Every Branch Using CircleCI and Sauce Labs to Continuously Test CS ...
Sauce Labs
 
PPT
Acceptance Testing With Selenium
elliando dias
 
PDF
Automated Testing in Angular Slides
Jim Lynch
 
PDF
Continuous Testing With React Storybook & WebdriverIO
Josh Cypher
 
PPTX
Google, quality and you
nelinger
 
PDF
Software testing with examples in Angular (and AngularJS)
Paweł Żurowski
 
PPTX
Automated Testing
Speed FC
 
PDF
Continuous Testing in the Cloud
TechWell
 
KEY
Graceful Failure with Selenium and Continuous Integration
Chris B. France
 
PDF
How to apply AI to Testing
SAP SE
 
PDF
Exploratory Testing As Code Eurostar23
Brendan Connolly
 
PDF
Exploratory Testing As Code
Brendan Connolly
 
PDF
UPC Plone Testing Talk
Timo Stollenwerk
 
PDF
Test-Tutorial
tutorialsruby
 
PDF
Test-Tutorial
tutorialsruby
 
PPTX
Continuous Testing and New Tools for Automation - Presentation from StarWest ...
Sauce Labs
 
PPTX
Continuous testing
Oleksandr Metelytsia
 
PDF
Certified Professional - Continuous Testing (CP-CT) course information v1.05 ...
DevOps++ Alliance
 
PPTX
Write tests, please
Joan López de la Franca
 
PDF
Workshop: Functional testing made easy with PHPUnit & Selenium (phpCE Poland,...
Ondřej Machulda
 
Tests for Every Branch Using CircleCI and Sauce Labs to Continuously Test CS ...
Sauce Labs
 
Acceptance Testing With Selenium
elliando dias
 
Automated Testing in Angular Slides
Jim Lynch
 
Continuous Testing With React Storybook & WebdriverIO
Josh Cypher
 
Google, quality and you
nelinger
 
Software testing with examples in Angular (and AngularJS)
Paweł Żurowski
 
Automated Testing
Speed FC
 
Continuous Testing in the Cloud
TechWell
 
Graceful Failure with Selenium and Continuous Integration
Chris B. France
 
How to apply AI to Testing
SAP SE
 
Exploratory Testing As Code Eurostar23
Brendan Connolly
 
Exploratory Testing As Code
Brendan Connolly
 
UPC Plone Testing Talk
Timo Stollenwerk
 
Test-Tutorial
tutorialsruby
 
Test-Tutorial
tutorialsruby
 
Continuous Testing and New Tools for Automation - Presentation from StarWest ...
Sauce Labs
 
Continuous testing
Oleksandr Metelytsia
 
Certified Professional - Continuous Testing (CP-CT) course information v1.05 ...
DevOps++ Alliance
 
Write tests, please
Joan López de la Franca
 
Workshop: Functional testing made easy with PHPUnit & Selenium (phpCE Poland,...
Ondřej Machulda
 
Ad

More from Sauce Labs (20)

PDF
Simplify Salesforce Testing with AI-Driven Codeless Tools
Sauce Labs
 
PDF
Testing on Mobile Devices with Location Services
Sauce Labs
 
PDF
Your Framework for Success: introduction to JavaScript Testing at Scale
Sauce Labs
 
PDF
Automating Hybrid Applications with Appium
Sauce Labs
 
PDF
Quality at Speed: More API Testing, Less UI Testing
Sauce Labs
 
PPTX
Creating Digital Confidence with Test Automation
Sauce Labs
 
PDF
Just Enough (Automated) Testing
Sauce Labs
 
PDF
Using Axe to Add Accessibility Checks to Your Existing Selenium Tests
Sauce Labs
 
PDF
How Open Source Helps to Bring Back Product Obsession
Sauce Labs
 
PDF
Webinar: A Sneak Peek at Selenium 4 with Simon Stewart
Sauce Labs
 
PDF
[Deu] Test Automatisierung Mit Web Driver.io
Sauce Labs
 
PDF
Accelerating Innovation: Leveraging Open Source to Optimize Your Shift-Left I...
Sauce Labs
 
PDF
Accelerating Your Digital Agenda with Continuous Testing ft. Forrester
Sauce Labs
 
PDF
How to Measure Success in Continuous Testing
Sauce Labs
 
PDF
From Zero to 2.7 Million - How Verizon Media Embraced Open Source to Accelera...
Sauce Labs
 
PDF
5 Steps to Jump Start Your Test Automation
Sauce Labs
 
PDF
Sauce Labs Webinar: Rising Importance of Software Testing
Sauce Labs
 
PDF
BDD With Selenide by Hima Bindu Peteti
Sauce Labs
 
PDF
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
Sauce Labs
 
PDF
Continuous Delivery for "Mature" Codebases by Melisa Benua
Sauce Labs
 
Simplify Salesforce Testing with AI-Driven Codeless Tools
Sauce Labs
 
Testing on Mobile Devices with Location Services
Sauce Labs
 
Your Framework for Success: introduction to JavaScript Testing at Scale
Sauce Labs
 
Automating Hybrid Applications with Appium
Sauce Labs
 
Quality at Speed: More API Testing, Less UI Testing
Sauce Labs
 
Creating Digital Confidence with Test Automation
Sauce Labs
 
Just Enough (Automated) Testing
Sauce Labs
 
Using Axe to Add Accessibility Checks to Your Existing Selenium Tests
Sauce Labs
 
How Open Source Helps to Bring Back Product Obsession
Sauce Labs
 
Webinar: A Sneak Peek at Selenium 4 with Simon Stewart
Sauce Labs
 
[Deu] Test Automatisierung Mit Web Driver.io
Sauce Labs
 
Accelerating Innovation: Leveraging Open Source to Optimize Your Shift-Left I...
Sauce Labs
 
Accelerating Your Digital Agenda with Continuous Testing ft. Forrester
Sauce Labs
 
How to Measure Success in Continuous Testing
Sauce Labs
 
From Zero to 2.7 Million - How Verizon Media Embraced Open Source to Accelera...
Sauce Labs
 
5 Steps to Jump Start Your Test Automation
Sauce Labs
 
Sauce Labs Webinar: Rising Importance of Software Testing
Sauce Labs
 
BDD With Selenide by Hima Bindu Peteti
Sauce Labs
 
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
Sauce Labs
 
Continuous Delivery for "Mature" Codebases by Melisa Benua
Sauce Labs
 

Recently uploaded (20)

PDF
Zilliz Cloud Demo for performance and scale
Zilliz
 
PPTX
fatigue in aircraft structures-221113192308-0ad6dc8c.pptx
aviatecofficial
 
PDF
Pressure Measurement training for engineers and Technicians
AIESOLUTIONS
 
PDF
Reasons for the succes of MENARD PRESSUREMETER.pdf
majdiamz
 
PDF
Biomechanics of Gait: Engineering Solutions for Rehabilitation (www.kiu.ac.ug)
publication11
 
PPTX
artificial intelligence applications in Geomatics
NawrasShatnawi1
 
PPTX
Damage of stability of a ship and how its change .pptx
ehamadulhaque
 
PPTX
Lecture 1 Shell and Tube Heat exchanger-1.pptx
mailforillegalwork
 
PPTX
265587293-NFPA 101 Life safety code-PPT-1.pptx
chandermwason
 
PDF
Basic_Concepts_in_Clinical_Biochemistry_2018كيمياء_عملي.pdf
AdelLoin
 
PDF
Ethics and Trustworthy AI in Healthcare – Governing Sensitive Data, Profiling...
AlqualsaDIResearchGr
 
PPTX
Green Building & Energy Conservation ppt
Sagar Sarangi
 
PDF
GTU Civil Engineering All Semester Syllabus.pdf
Vimal Bhojani
 
PPTX
Solar Thermal Energy System Seminar.pptx
Gpc Purapuza
 
PDF
International Journal of Information Technology Convergence and services (IJI...
ijitcsjournal4
 
PDF
AI TECHNIQUES FOR IDENTIFYING ALTERATIONS IN THE HUMAN GUT MICROBIOME IN MULT...
vidyalalltv1
 
PPTX
Server Side Web Development Unit 1 of Nodejs.pptx
sneha852132
 
DOC
MRRS Strength and Durability of Concrete
CivilMythili
 
PPTX
DATA BASE MANAGEMENT AND RELATIONAL DATA
gomathisankariv2
 
PDF
Design Thinking basics for Engineers.pdf
CMR University
 
Zilliz Cloud Demo for performance and scale
Zilliz
 
fatigue in aircraft structures-221113192308-0ad6dc8c.pptx
aviatecofficial
 
Pressure Measurement training for engineers and Technicians
AIESOLUTIONS
 
Reasons for the succes of MENARD PRESSUREMETER.pdf
majdiamz
 
Biomechanics of Gait: Engineering Solutions for Rehabilitation (www.kiu.ac.ug)
publication11
 
artificial intelligence applications in Geomatics
NawrasShatnawi1
 
Damage of stability of a ship and how its change .pptx
ehamadulhaque
 
Lecture 1 Shell and Tube Heat exchanger-1.pptx
mailforillegalwork
 
265587293-NFPA 101 Life safety code-PPT-1.pptx
chandermwason
 
Basic_Concepts_in_Clinical_Biochemistry_2018كيمياء_عملي.pdf
AdelLoin
 
Ethics and Trustworthy AI in Healthcare – Governing Sensitive Data, Profiling...
AlqualsaDIResearchGr
 
Green Building & Energy Conservation ppt
Sagar Sarangi
 
GTU Civil Engineering All Semester Syllabus.pdf
Vimal Bhojani
 
Solar Thermal Energy System Seminar.pptx
Gpc Purapuza
 
International Journal of Information Technology Convergence and services (IJI...
ijitcsjournal4
 
AI TECHNIQUES FOR IDENTIFYING ALTERATIONS IN THE HUMAN GUT MICROBIOME IN MULT...
vidyalalltv1
 
Server Side Web Development Unit 1 of Nodejs.pptx
sneha852132
 
MRRS Strength and Durability of Concrete
CivilMythili
 
DATA BASE MANAGEMENT AND RELATIONAL DATA
gomathisankariv2
 
Design Thinking basics for Engineers.pdf
CMR University
 

Continuous Testing Meets the Classroom at Code.org