SlideShare a Scribd company logo
Introduction to Automated Testing

    by Lars Thorup, ZeaLake
         May 26th 2011
Who is Lars Thorup?




• Software developer

• Coach: Teaching TDD and
  automated testing

• Advisor: Assesses software
  projects and companies

• Founder and CEO of
  BestBrains and ZeaLake
Why are we here today?

• How do we do automated testing?
   – Write test programs
   – Run the tests automatically

• Why should we write tests?
   –   Enjoy more efficient and predictable course of development
   –   Find and fix bugs fast
   –   Prevent bugs from reappearing
   –   Improve the design of our software
Different kinds of automated tests

• Unit tests
   – Tests individual pieces of code and the interaction between code blocks

• System tests
   – Tests the entire system against the requirements

• Performance tests
   – Tests non functional requirements
Unit tests or system tests?

• Unit tests are efficient
   –   Fast to run (a few seconds)
   –   Robust and predictable
   –   Easy to write
   –   Is written together with the code it is testing

• System tests are thorough
   – Tests all layers together
   – Most efficient way to create a set of tests for existing code
Can we automate performance tests?

• Performance tests are brittle
   – Tip: create performance trend curves instead
So, how do we actually do this?

• IsNumeric
  – C#
  – Ruby

• TransferFunds
  – C++
How do we run the tests automatically?

• From our programming environment (IDE)
   – Command line: make test
   – Right click | Run Tests

• On every commit
   – Setup a build server
       • Hudson / Jenkins
       • TeamCity
   – Let the build server run all tests
   – Get build notifications
   – Keep the build green
       • Fixing a broken build has priority over any other development task
How can tests help improve our design?

• The software design needs to evolve over time
• A refactoring modifies the design without changing behavior
• Tests ensure that behavior is not accidentally changed

• Without tests, refactoring is scary
   – and with no refactoring, the design decays over time

• With tests, we have the courage to refactor
   – so we continually keep our design healthy
What is good design?

• One element of good design is loose dependencies
   – Use interfaces (for static languages)
   – Inject dependencies

• Avoid this



• Do this instead
Are we wasting valuable developer time writing tests?

• No
• The time spent writing tests is not taken from the time spent
  coding
   – It is taken from the time otherwise spent on manual testing and debugging
• The cost of a bug keeps increasing until we fix it

• Find bugs fast
   – Avoid losing customer confidence
   – Free QA to do exploratory testing
     so they find the hard-to-find bugs
   – Spend less time trying to figure out
     what is causing the bug and how to fix it

• Avoid spending time testing again
How do we get started?

• When we have a lot of existing code without tests
   – Create a set of system tests to get a safety net

• When we are writing new code
   – Write unit tests in conjunction with the new code

• Set up a standard test environment for our specific application
   – Test data
       • Automate the creation of standard testdata in a local database
   – External dependencies
       • Write stubs to use in the tests
What does a real-world project look like?

• wizerize.com
    – Web application: C# and JavaScript
    – 3 years of production
    – 2-4 developers
•   40% test code, 60% production code (in lines of code)
•   71% code coverage of unit tests
•   614 unit tests – run in 1 minute
•   54 system tests – run in 20 minutes
•   No functional errors seen by end users in production (yet)
Where can I read more?

• http://googletesting.blogspot.com/
• http://testdrivendeveloper.com/
• http://codesheriff.blogspot.com/
Which tools do we use?

Environment           Tool
C#                    NUnit
JavaScript            qUnit
HTML-based UI         WatiN, Selenium
C++                   CppUnit, googletest
Python                unittest
Ruby                  Test::Unit
C                     check, cunit
Java                  JUnit
...                   ...
Ad

Recommended

Extreme Programming - to the next-level
Extreme Programming - to the next-level
Lars Thorup
 
Automated Performance Testing
Automated Performance Testing
Lars Thorup
 
Fast end-to-end-tests
Fast end-to-end-tests
Rikke Veng Petersen
 
Put "fast" back in "fast feedback"
Put "fast" back in "fast feedback"
Lars Thorup
 
QA Strategies for Testing Legacy Web Apps
QA Strategies for Testing Legacy Web Apps
Rainforest QA
 
Using Crowdsourced Testing to Turbocharge your Development Team
Using Crowdsourced Testing to Turbocharge your Development Team
Rainforest QA
 
UPC Plone Testing Talk
UPC Plone Testing Talk
Timo Stollenwerk
 
Unit testing
Unit testing
Brian Hu
 
icebreakerwithdevops-150218112943-conversion-gate02
icebreakerwithdevops-150218112943-conversion-gate02
Manohar Kumar
 
Why You Should Start Using Docker
Why You Should Start Using Docker
Alexandru Bolboaca
 
Project management frameworks for software developing
Project management frameworks for software developing
Vicente Bolea
 
Develop 4 Developers
Develop 4 Developers
Christian Güdemann
 
QA Automation testing online training
QA Automation testing online training
Divya Shree
 
testmon for Python
testmon for Python
tib0r
 
Agile Software Development Techniques for Daily Use
Agile Software Development Techniques for Daily Use
Hristo Iliev
 
OpenNTF Essentials
OpenNTF Essentials
Christian Güdemann
 
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Binary Studio
 
Putting the pro in programmer
Putting the pro in programmer
Switch Systems Ltd
 
Игорь Бондаренко (Neklo) - Организация процесса разработки модулей для Magento
Игорь Бондаренко (Neklo) - Организация процесса разработки модулей для Magento
meet_magento
 
That worked before
That worked before
Christian Güdemann
 
The art of being an agile programmer
The art of being an agile programmer
Claudia Rosu
 
SeleniumCamp 2015 Andrii Soldatenko
SeleniumCamp 2015 Andrii Soldatenko
Andrii Soldatenko
 
Improve the deployment process step by step
Improve the deployment process step by step
Daniel Fahlke
 
Survival of the Continuist
Survival of the Continuist
Paul Blundell
 
User story workflow (eng)
User story workflow (eng)
Anatoliy Okhotnikov
 
Андрей Солдатенко. Разработка высокопроизводительныx функциональных тестов д...
Андрей Солдатенко. Разработка высокопроизводительныx функциональных тестов д...
Alina Dolgikh
 
Usg Web Tech Day 2016 - Continuous Integration, Deployment, and Delivery
Usg Web Tech Day 2016 - Continuous Integration, Deployment, and Delivery
Stephen Garrett
 
Selenium done right
Selenium done right
Tal Landa
 
Agile Testing Pasadena JUG Aug2009
Agile Testing Pasadena JUG Aug2009
Grig Gheorghiu
 
Test and Behaviour Driven Development (TDD/BDD)
Test and Behaviour Driven Development (TDD/BDD)
Lars Thorup
 

More Related Content

What's hot (20)

icebreakerwithdevops-150218112943-conversion-gate02
icebreakerwithdevops-150218112943-conversion-gate02
Manohar Kumar
 
Why You Should Start Using Docker
Why You Should Start Using Docker
Alexandru Bolboaca
 
Project management frameworks for software developing
Project management frameworks for software developing
Vicente Bolea
 
Develop 4 Developers
Develop 4 Developers
Christian Güdemann
 
QA Automation testing online training
QA Automation testing online training
Divya Shree
 
testmon for Python
testmon for Python
tib0r
 
Agile Software Development Techniques for Daily Use
Agile Software Development Techniques for Daily Use
Hristo Iliev
 
OpenNTF Essentials
OpenNTF Essentials
Christian Güdemann
 
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Binary Studio
 
Putting the pro in programmer
Putting the pro in programmer
Switch Systems Ltd
 
Игорь Бондаренко (Neklo) - Организация процесса разработки модулей для Magento
Игорь Бондаренко (Neklo) - Организация процесса разработки модулей для Magento
meet_magento
 
That worked before
That worked before
Christian Güdemann
 
The art of being an agile programmer
The art of being an agile programmer
Claudia Rosu
 
SeleniumCamp 2015 Andrii Soldatenko
SeleniumCamp 2015 Andrii Soldatenko
Andrii Soldatenko
 
Improve the deployment process step by step
Improve the deployment process step by step
Daniel Fahlke
 
Survival of the Continuist
Survival of the Continuist
Paul Blundell
 
User story workflow (eng)
User story workflow (eng)
Anatoliy Okhotnikov
 
Андрей Солдатенко. Разработка высокопроизводительныx функциональных тестов д...
Андрей Солдатенко. Разработка высокопроизводительныx функциональных тестов д...
Alina Dolgikh
 
Usg Web Tech Day 2016 - Continuous Integration, Deployment, and Delivery
Usg Web Tech Day 2016 - Continuous Integration, Deployment, and Delivery
Stephen Garrett
 
Selenium done right
Selenium done right
Tal Landa
 
icebreakerwithdevops-150218112943-conversion-gate02
icebreakerwithdevops-150218112943-conversion-gate02
Manohar Kumar
 
Why You Should Start Using Docker
Why You Should Start Using Docker
Alexandru Bolboaca
 
Project management frameworks for software developing
Project management frameworks for software developing
Vicente Bolea
 
QA Automation testing online training
QA Automation testing online training
Divya Shree
 
testmon for Python
testmon for Python
tib0r
 
Agile Software Development Techniques for Daily Use
Agile Software Development Techniques for Daily Use
Hristo Iliev
 
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Binary Studio
 
Игорь Бондаренко (Neklo) - Организация процесса разработки модулей для Magento
Игорь Бондаренко (Neklo) - Организация процесса разработки модулей для Magento
meet_magento
 
The art of being an agile programmer
The art of being an agile programmer
Claudia Rosu
 
SeleniumCamp 2015 Andrii Soldatenko
SeleniumCamp 2015 Andrii Soldatenko
Andrii Soldatenko
 
Improve the deployment process step by step
Improve the deployment process step by step
Daniel Fahlke
 
Survival of the Continuist
Survival of the Continuist
Paul Blundell
 
Андрей Солдатенко. Разработка высокопроизводительныx функциональных тестов д...
Андрей Солдатенко. Разработка высокопроизводительныx функциональных тестов д...
Alina Dolgikh
 
Usg Web Tech Day 2016 - Continuous Integration, Deployment, and Delivery
Usg Web Tech Day 2016 - Continuous Integration, Deployment, and Delivery
Stephen Garrett
 
Selenium done right
Selenium done right
Tal Landa
 

Similar to Introduction to Automated Testing (20)

Agile Testing Pasadena JUG Aug2009
Agile Testing Pasadena JUG Aug2009
Grig Gheorghiu
 
Test and Behaviour Driven Development (TDD/BDD)
Test and Behaviour Driven Development (TDD/BDD)
Lars Thorup
 
Automated testing overview
Automated testing overview
Alex Pop
 
Agile Acceptance testing with Fitnesse
Agile Acceptance testing with Fitnesse
ClareMcLennan
 
Automated testing vs manual testing
Automated testing vs manual testing
Neha Polke
 
Making the Unstable Stable - An Intro To Testing
Making the Unstable Stable - An Intro To Testing
Cameron Presley
 
High Performance Software Engineering Teams
High Performance Software Engineering Teams
Lars Thorup
 
Testing 101
Testing 101
Noam Barkai
 
Unit Testing and TDD 2017
Unit Testing and TDD 2017
Xavi Hidalgo
 
Engaging IV&V Testing Services for Agile Projects
Engaging IV&V Testing Services for Agile Projects
Ravi Kumar
 
Continuous Integration testing based on Selenium and Hudson
Continuous Integration testing based on Selenium and Hudson
Zbyszek Mockun
 
Open Source tools in Continuous Integration environment (case study for agil...
Open Source tools in Continuous Integration environment (case study for agil...
suwalki24.pl
 
DSR Testing (Part 1)
DSR Testing (Part 1)
Steve Upton
 
Automated testing-whitepaper
Automated testing-whitepaper
imdurgesh
 
TeDevelopment Testing in Software Engineering
TeDevelopment Testing in Software Engineering
Karthik Rohan
 
Testing Sap: Modern Methodology
Testing Sap: Modern Methodology
Ethan Jewett
 
Testing In Agile
Testing In Agile
Ari Tanninen
 
Software testing
Software testing
Nico Heidtke
 
39.Testing-Part3.ppt
39.Testing-Part3.ppt
SapnaRastogi3
 
Unit testing & TDD concepts with best practice guidelines.
Unit testing & TDD concepts with best practice guidelines.
Mohamed Taman
 
Agile Testing Pasadena JUG Aug2009
Agile Testing Pasadena JUG Aug2009
Grig Gheorghiu
 
Test and Behaviour Driven Development (TDD/BDD)
Test and Behaviour Driven Development (TDD/BDD)
Lars Thorup
 
Automated testing overview
Automated testing overview
Alex Pop
 
Agile Acceptance testing with Fitnesse
Agile Acceptance testing with Fitnesse
ClareMcLennan
 
Automated testing vs manual testing
Automated testing vs manual testing
Neha Polke
 
Making the Unstable Stable - An Intro To Testing
Making the Unstable Stable - An Intro To Testing
Cameron Presley
 
High Performance Software Engineering Teams
High Performance Software Engineering Teams
Lars Thorup
 
Unit Testing and TDD 2017
Unit Testing and TDD 2017
Xavi Hidalgo
 
Engaging IV&V Testing Services for Agile Projects
Engaging IV&V Testing Services for Agile Projects
Ravi Kumar
 
Continuous Integration testing based on Selenium and Hudson
Continuous Integration testing based on Selenium and Hudson
Zbyszek Mockun
 
Open Source tools in Continuous Integration environment (case study for agil...
Open Source tools in Continuous Integration environment (case study for agil...
suwalki24.pl
 
DSR Testing (Part 1)
DSR Testing (Part 1)
Steve Upton
 
Automated testing-whitepaper
Automated testing-whitepaper
imdurgesh
 
TeDevelopment Testing in Software Engineering
TeDevelopment Testing in Software Engineering
Karthik Rohan
 
Testing Sap: Modern Methodology
Testing Sap: Modern Methodology
Ethan Jewett
 
39.Testing-Part3.ppt
39.Testing-Part3.ppt
SapnaRastogi3
 
Unit testing & TDD concepts with best practice guidelines.
Unit testing & TDD concepts with best practice guidelines.
Mohamed Taman
 
Ad

More from Lars Thorup (14)

100 tests per second - 40 releases per week
100 tests per second - 40 releases per week
Lars Thorup
 
SQL or NoSQL - how to choose
SQL or NoSQL - how to choose
Lars Thorup
 
Super fast end-to-end-tests
Super fast end-to-end-tests
Lars Thorup
 
Advanced Javascript Unit Testing
Advanced Javascript Unit Testing
Lars Thorup
 
Unit testing legacy code
Unit testing legacy code
Lars Thorup
 
Advanced QUnit - Front-End JavaScript Unit Testing
Advanced QUnit - Front-End JavaScript Unit Testing
Lars Thorup
 
Database Schema Evolution
Database Schema Evolution
Lars Thorup
 
Advanced Jasmine - Front-End JavaScript Unit Testing
Advanced Jasmine - Front-End JavaScript Unit Testing
Lars Thorup
 
Javascript unit testing with QUnit and Sinon
Javascript unit testing with QUnit and Sinon
Lars Thorup
 
Continuous Integration for front-end JavaScript
Continuous Integration for front-end JavaScript
Lars Thorup
 
Agile Contracts
Agile Contracts
Lars Thorup
 
Elephant Carpaccio
Elephant Carpaccio
Lars Thorup
 
Automated Testing for Embedded Software in C or C++
Automated Testing for Embedded Software in C or C++
Lars Thorup
 
Unit Testing in JavaScript with MVC and QUnit
Unit Testing in JavaScript with MVC and QUnit
Lars Thorup
 
100 tests per second - 40 releases per week
100 tests per second - 40 releases per week
Lars Thorup
 
SQL or NoSQL - how to choose
SQL or NoSQL - how to choose
Lars Thorup
 
Super fast end-to-end-tests
Super fast end-to-end-tests
Lars Thorup
 
Advanced Javascript Unit Testing
Advanced Javascript Unit Testing
Lars Thorup
 
Unit testing legacy code
Unit testing legacy code
Lars Thorup
 
Advanced QUnit - Front-End JavaScript Unit Testing
Advanced QUnit - Front-End JavaScript Unit Testing
Lars Thorup
 
Database Schema Evolution
Database Schema Evolution
Lars Thorup
 
Advanced Jasmine - Front-End JavaScript Unit Testing
Advanced Jasmine - Front-End JavaScript Unit Testing
Lars Thorup
 
Javascript unit testing with QUnit and Sinon
Javascript unit testing with QUnit and Sinon
Lars Thorup
 
Continuous Integration for front-end JavaScript
Continuous Integration for front-end JavaScript
Lars Thorup
 
Elephant Carpaccio
Elephant Carpaccio
Lars Thorup
 
Automated Testing for Embedded Software in C or C++
Automated Testing for Embedded Software in C or C++
Lars Thorup
 
Unit Testing in JavaScript with MVC and QUnit
Unit Testing in JavaScript with MVC and QUnit
Lars Thorup
 
Ad

Recently uploaded (20)

OpenPOWER Foundation & Open-Source Core Innovations
OpenPOWER Foundation & Open-Source Core Innovations
IBM
 
"Database isolation: how we deal with hundreds of direct connections to the d...
"Database isolation: how we deal with hundreds of direct connections to the d...
Fwdays
 
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Priyanka Aash
 
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
Safe Software
 
You are not excused! How to avoid security blind spots on the way to production
You are not excused! How to avoid security blind spots on the way to production
Michele Leroux Bustamante
 
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Nilesh Gule
 
Securing Account Lifecycles in the Age of Deepfakes.pptx
Securing Account Lifecycles in the Age of Deepfakes.pptx
FIDO Alliance
 
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
Priyanka Aash
 
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
Fwdays
 
Cyber Defense Matrix Workshop - RSA Conference
Cyber Defense Matrix Workshop - RSA Conference
Priyanka Aash
 
Python Conference Singapore - 19 Jun 2025
Python Conference Singapore - 19 Jun 2025
ninefyi
 
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC
 
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my bill
stijn40
 
Securing AI - There Is No Try, Only Do!.pdf
Securing AI - There Is No Try, Only Do!.pdf
Priyanka Aash
 
Wenn alles versagt - IBM Tape schützt, was zählt! Und besonders mit dem neust...
Wenn alles versagt - IBM Tape schützt, was zählt! Und besonders mit dem neust...
Josef Weingand
 
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
 
Lessons Learned from Developing Secure AI Workflows.pdf
Lessons Learned from Developing Secure AI Workflows.pdf
Priyanka Aash
 
Smarter Aviation Data Management: Lessons from Swedavia Airports and Sweco
Smarter Aviation Data Management: Lessons from Swedavia Airports and Sweco
Safe Software
 
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdf
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdf
Priyanka Aash
 
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
Fwdays
 
OpenPOWER Foundation & Open-Source Core Innovations
OpenPOWER Foundation & Open-Source Core Innovations
IBM
 
"Database isolation: how we deal with hundreds of direct connections to the d...
"Database isolation: how we deal with hundreds of direct connections to the d...
Fwdays
 
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Priyanka Aash
 
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
Safe Software
 
You are not excused! How to avoid security blind spots on the way to production
You are not excused! How to avoid security blind spots on the way to production
Michele Leroux Bustamante
 
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Nilesh Gule
 
Securing Account Lifecycles in the Age of Deepfakes.pptx
Securing Account Lifecycles in the Age of Deepfakes.pptx
FIDO Alliance
 
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
Priyanka Aash
 
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
Fwdays
 
Cyber Defense Matrix Workshop - RSA Conference
Cyber Defense Matrix Workshop - RSA Conference
Priyanka Aash
 
Python Conference Singapore - 19 Jun 2025
Python Conference Singapore - 19 Jun 2025
ninefyi
 
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC
 
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my bill
stijn40
 
Securing AI - There Is No Try, Only Do!.pdf
Securing AI - There Is No Try, Only Do!.pdf
Priyanka Aash
 
Wenn alles versagt - IBM Tape schützt, was zählt! Und besonders mit dem neust...
Wenn alles versagt - IBM Tape schützt, was zählt! Und besonders mit dem neust...
Josef Weingand
 
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
 
Lessons Learned from Developing Secure AI Workflows.pdf
Lessons Learned from Developing Secure AI Workflows.pdf
Priyanka Aash
 
Smarter Aviation Data Management: Lessons from Swedavia Airports and Sweco
Smarter Aviation Data Management: Lessons from Swedavia Airports and Sweco
Safe Software
 
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdf
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdf
Priyanka Aash
 
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
Fwdays
 

Introduction to Automated Testing

  • 1. Introduction to Automated Testing by Lars Thorup, ZeaLake May 26th 2011
  • 2. Who is Lars Thorup? • Software developer • Coach: Teaching TDD and automated testing • Advisor: Assesses software projects and companies • Founder and CEO of BestBrains and ZeaLake
  • 3. Why are we here today? • How do we do automated testing? – Write test programs – Run the tests automatically • Why should we write tests? – Enjoy more efficient and predictable course of development – Find and fix bugs fast – Prevent bugs from reappearing – Improve the design of our software
  • 4. Different kinds of automated tests • Unit tests – Tests individual pieces of code and the interaction between code blocks • System tests – Tests the entire system against the requirements • Performance tests – Tests non functional requirements
  • 5. Unit tests or system tests? • Unit tests are efficient – Fast to run (a few seconds) – Robust and predictable – Easy to write – Is written together with the code it is testing • System tests are thorough – Tests all layers together – Most efficient way to create a set of tests for existing code
  • 6. Can we automate performance tests? • Performance tests are brittle – Tip: create performance trend curves instead
  • 7. So, how do we actually do this? • IsNumeric – C# – Ruby • TransferFunds – C++
  • 8. How do we run the tests automatically? • From our programming environment (IDE) – Command line: make test – Right click | Run Tests • On every commit – Setup a build server • Hudson / Jenkins • TeamCity – Let the build server run all tests – Get build notifications – Keep the build green • Fixing a broken build has priority over any other development task
  • 9. How can tests help improve our design? • The software design needs to evolve over time • A refactoring modifies the design without changing behavior • Tests ensure that behavior is not accidentally changed • Without tests, refactoring is scary – and with no refactoring, the design decays over time • With tests, we have the courage to refactor – so we continually keep our design healthy
  • 10. What is good design? • One element of good design is loose dependencies – Use interfaces (for static languages) – Inject dependencies • Avoid this • Do this instead
  • 11. Are we wasting valuable developer time writing tests? • No • The time spent writing tests is not taken from the time spent coding – It is taken from the time otherwise spent on manual testing and debugging • The cost of a bug keeps increasing until we fix it • Find bugs fast – Avoid losing customer confidence – Free QA to do exploratory testing so they find the hard-to-find bugs – Spend less time trying to figure out what is causing the bug and how to fix it • Avoid spending time testing again
  • 12. How do we get started? • When we have a lot of existing code without tests – Create a set of system tests to get a safety net • When we are writing new code – Write unit tests in conjunction with the new code • Set up a standard test environment for our specific application – Test data • Automate the creation of standard testdata in a local database – External dependencies • Write stubs to use in the tests
  • 13. What does a real-world project look like? • wizerize.com – Web application: C# and JavaScript – 3 years of production – 2-4 developers • 40% test code, 60% production code (in lines of code) • 71% code coverage of unit tests • 614 unit tests – run in 1 minute • 54 system tests – run in 20 minutes • No functional errors seen by end users in production (yet)
  • 14. Where can I read more? • http://googletesting.blogspot.com/ • http://testdrivendeveloper.com/ • http://codesheriff.blogspot.com/
  • 15. Which tools do we use? Environment Tool C# NUnit JavaScript qUnit HTML-based UI WatiN, Selenium C++ CppUnit, googletest Python unittest Ruby Test::Unit C check, cunit Java JUnit ... ...