Intro To Unit and integration Testing
- 1. Unit & Integration Testing Infusion Bootcamp 2017 Paul Churchward, FRM Team Lead
- 2. ABOUT ME • Team Lead based in Toronto • With Infusion 2 years • Background in finance (FRM, CSC) • Stack: Java, Ruby • Companies worked for
- 3. Developers code UNIT and INTEGRATION tests which run automatically to test their software. compares expected behaviour with actual behaviour Unit and integration tests are part of the project source Written by developer, not QA Tests are executed automatically as part of continuous integration or by local builds, not by QA Failures are reported (web dashboard/email/slack) and cause the build to abort Fix failures by changing the code (sometimes, the test needs to be changed) Fixing test failure raised by continuous integration is developer’s top priority b/c this holds up the team.
- 4. Verify logic works as intended Simulate how code behaves under different circumstances (e.g. negative testing) Ensure logic continues to works as designed after you’ve written Not alone! Unit and integration testing is the first line of defense against bugs, not QA. In a continuous delivery setup, unit & integration tests are your only line of defense, there is no QA. When code passes the automated tests, it is automatically deployed to prod. ISN’T TESTING QA’S JOB? WHY WRITE A TEST?
- 5. A unit test tests the smallest unit of a program produces expected behaviour. A unit test typically tests a single method / function Tests the method produces expected behaviour under different circumstances Typical unit test will invoke a single method and test it produces expected output / right behaviour
- 6. Method’s return value is compared against the expected output for the input fed to it Method invokes a service the expected number of times with the expected arguments Note, we wouldn’t test the service’s output here. It should have its own test. This is a key unit testing concept. Each test tests the bare minimum. Method does not invoke a service when passed certain arguments Method saves to database only when its call to Service-A returns true Note, we wouldn’t test database was updated, only that the database service in our code was called. This is a key unit testing concept. Don’t talk to external services. Examples of testing expected behaviour in a unit test:
- 7. An integration test tests components work together to produce expected behaviour. Scope goes beyond the innerworkings of a single method / function Tests collaboration between components, taking it further than a unit test For it to be an integration test, a “boundary” must be crossed -> More than one component must be tested. e.g. calling another class or calling an external service Purpose is to test behaviour of external services like databases and other APIs Number of unit tests outweigh integration tests -> Integration tests are much slower because they talk to file systems / external APIs / databases.
- 8. UNIT TEST EXAMPLE: (Let’s do this in pseudo code first) When checkUserExists() on the UserDAO is invoked, force it to return false When the addUser() method on UserService is invoked with the argument "FOO“ then: Assert checkUserExists() on the UserDAO is invoked only once with the parameter "FOO" Assert saveUser() on the UserDAO is invoked only once with the parameter "FOO" Notice, we’re not testing the state of the database, only that we attempted to update the database. By only testing the behaviour of the method, without testing anything outside the method we have created a unit test. If we went further and tested the state of the database, we would have an integration test. This does not invoke the methods on the UserDAO or call out to the DB, it simply checks it should be called. That would be out of scope for a unit test. This is achieved by using a mocking library (we’ll talk about that later) which blocks actually invoking designated parts of the code we don’t want to be run during the test.
- 9. INTEGRATION TEST EXAMPLE: (Let’s do this in pseudo code first) When the addUser() method on UserService is invoked with the argument "FOO“ then: Assert getUser("FOO") on UserDAO returns a valid user with the username "FOO" Notice, we’re testing the state of the database here. The test adds a user and then tests we get the expected user back from the database when we query for it through our DAO. We’re not preventing the code invoking any service (mocking). This is an integration test. Build expected report When the generateAndSave() method on ReportService is invoked, capture it’s report ID. Assert retrieveReport("S3") on ReportService returns a report matching the expected report. This test invokes the ReportService to generate a report and save it to Amazon S3. We test the code by retrieving the report from S3 and comparing against one we already have. It should match. We’re not just testing the report itself here, but that ReportService can read and write to Amazon S3 as expected. This makes it an integration test. ANOTHER EXAMPLE:
- 10. Consider this example … You write a report for another system to consume and the columns must be in alphabetical order. public void generateReport() { List<String> columns = columnFactory.getColumns(); FooBarReport report = reportService.generateReport(ReportType.FooBar); publishReportToS3(report); } Output: FooBar Report 2015-03-18 13:00 ================ A|B|C|D|E|F 2|6|2|4|2|6 1|3|5|9|11| == 2 rows I WISH I WROTE A (UNIT) TEST!
- 11. … 5 months later an accidental change is made to ColumnFactory causing it to spit out columns in reverse alphabetical order and is pushed to production. Your report now looks like this: Output: FooBar Report 2015-08-18 13:00 ================ F|E|D|C|B|A 6|2|4|2|6|2 |11|9|5|3|1 == 2 rows ... The system consuming this report fails to read it and blows up in production A unit test would have caught this right away.
- 12. WHAT TO UNIT TEST: Method return value matches expected value (or null) Method invokes a method of another classes the expected number of times. Example: if your method calls UserDAO.save(), test that save() is only called once Method doesn’t invoke method of another class. Example: if you method should only call UserDAO.save() if the user doesn’t already exist, test for an existing user and make sure UserDAO.save() is not called. Method throws a specific exception Example: if your method throws an CannotProcessException when a null userId , test that passes a null and expects CannotProcessException to be thrown. If it were to throw IOException instead (or no exception at all), that would be considered a failure.
- 13. MOCKING In a unit test, we do not care about testing things outside of our method: e.g: state of database changed by our method, state of file on file system, state of another class invoked by our method. We go a step further and prevent our method from changing the state or databases, file systems and other classes. We use mocking to do this. Mocking allows us to create fake (mock) instances of classes that we can manipulate to make our test easier. Mocking can render a class inert (calling its methods do nothing), or we can specify the behaviour we want to happen when we call a specific method Mocking is not used in an Integration test. That would defeat the purpose.
- 14. EXAMPLE: Let’s write a unit test for this method in UserService public void addUser(UserDAO dao, Configuration config, Connection conn, String userName) throws BadArgException { if(conn == null || config == null) { throw new ConfigurationException(); } dao.init(config, conn); if(dao.userExists(userName)) { throw new BadArgException() } dao.saveUser(userName); }
- 15. EXAMPLE: Components of the test: Provide non-null instances of Configuration, Connection, UserDAO and a username String Force UserDAO.checkUserExists() to return false -> Without talking to the database Verify UserDAO.saveUser() is invoked only once -> BUT don’t write to the database We will use Mockito (Java Mocking library) to “mock” UserDAO. This will: > Preventing its methods from actually executing their code > Allow us to force UserDAO.checkUserExists() to return false > Allow us to count the number of times UserService.addUser() will call UserDAO.saveUser() when we execute it in the test. We will use Mockito to mock Configuration and Connection so that we get non null instances but don’t have to create them (let’s assume it takes a fair bit of code to create these)
- 16. EXAMPLE: public class UserService { @Test public void addUser_withRegualrUser_callsSaveUserOnce(){ UserDao dao = mock(UserDao.class); Configuration config = mock(Configuration.class); Connection conn = mock(Connection.class); String user ="FOO"; UserService service = new UserService(); addUser(dao, config, conn, user); verify(dao.saveUser(user), 1); } }
- 17. Test Driven Development advocates writing tests BEFORE writing the code. This forces you to plan how the component will work before you start coding it.
- 18. BEWARE of tests that don’t test anything Not uncommon for people to write “tests” to invoke parts of the code outside the normal flow. These so called tests don’t do any testing! A “test” that simple invokes code but does not test it against expected behaviours is not a test! If a test does not compare expected behaviour with actual behaviour, it is not a test. These are surprisingly common in the real world.
- 19. Multiple testing frameworks to chose from in each language. Here are a few Java: Junit, TestNg C#: Nunit Ruby: Rspec, FactoryGirl Multi Language: Cucumber (behaviour driven testing) Tools