Testing distributed, complex web applications
2 likes2,993 views
As applications grow from single Rails applications to complex systems with multiple, interacting applications & web services, testing becomes more and more difficult. While we can test each application independently, we need to be able to test the full stack. This presentation shows methods, tools and tipps & tricks from testing such a complex application.
![OBD
• 1996: The OBD-II specification is made mandatory for
all cars sold in the United States.
• 2001: The European Union makes EOBD mandatory for
all gasoline (petrol) vehicles sold in the European Union,
starting in MY2001 (see European emission standards
Directive 98/69/EC [1]).
• 2008: All cars sold in the United States are required to
use the ISO 15765-4 [2] signaling standard (a variant of
the Controller Area Network (CAN) bus).
http://en.wikipedia.org/wiki/On-board_diagnostics
Freitag, 3. Juni 2011](https://image.slidesharecdn.com/distributedtestingrwc2011-110603070339-phpapp01/85/Testing-distributed-complex-web-applications-62-320.jpg)
Ad
More Related Content
Similar to Testing distributed, complex web applications (18)
More from Jens-Christian Fischer (9)
Ad
Recently uploaded (20)
Ad
Testing distributed, complex web applications
- 1. Testing of distributed applications Rails Way Con 2011 Jens-Christian Fischer with Michael Mahlberg & Keith Bingman [email protected] @jcfischer Freitag, 3. Juni 2011
- 2. le x m p co Testing of distributed applications Rails Way Con 2011 Jens-Christian Fischer with Michael Mahlberg & Keith Bingman [email protected] @jcfischer Freitag, 3. Juni 2011
- 3. Agenda • Background • What & Why & How of Testing • Discover the state of your apps • Make testable apps Freitag, 3. Juni 2011
- 4. why? Freitag, 3. Juni 2011
- 5. Freitag, 3. Juni 2011
- 6. we move money Freitag, 3. Juni 2011
- 7. What is behind? 1 Rails app 5 Sinatra apps 1 Ruby daemon 1 VOIP Server 1 Speech Synthesizer 1 SQL Database (Postgres) 1 NoSQL Server (Redis) (with Resque) Freitag, 3. Juni 2011
- 8. Architecture Freitag, 3. Juni 2011
- 9. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 Acapela Freitag, 3. Juni 2011
- 10. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 Acapela Freitag, 3. Juni 2011
- 11. Merchant Web Customer Web Customer Voice Widget Volcker FrontTrichet Freeswitch Duisenberg Keynes Greenspan Back Voice Redis Smith VAAS Database Postgres 9 Acapela Freitag, 3. Juni 2011
- 12. Trichet Volcker Duisenberg Greenspan Keynes Smith Freitag, 3. Juni 2011
- 13. Freitag, 3. Juni 2011
- 14. Testing Freitag, 3. Juni 2011
- 15. The old days • Simple web apps • Click through testing Freitag, 3. Juni 2011
- 16. Testi g n Royce, W. (1970). Managing the development of large software systems. Proceedings of IEEE Wescon. Freitag, 3. Juni 2011
- 17. Freitag, 3. Juni 2011
- 18. Freitag, 3. Juni 2011
- 19. Freitag, 3. Juni 2011
- 20. In Rails — everything is better Freitag, 3. Juni 2011
- 21. Automated Tests Unit Integration Frontend tests Freitag, 3. Juni 2011
- 22. Complexity • Modern applications consist of many pieces • App Servers, DB, NoSQL, Speech WebServices, Telephony, Synthesizers • HTTP, Line Protocols, JSON, XML, Corba, RMI Freitag, 3. Juni 2011
- 23. What to test Robert „UncleBob“ Martin: •Unit Tests •Component Tests •Integration Tests •System Test •Manual Tests http://stackoverflow.com/questions/2965483 Freitag, 3. Juni 2011
- 24. Unit Tests • Close to 100% code coverage • Test Driven Development • Written by Programmers Freitag, 3. Juni 2011
- 25. Component Tests • Test ~50% of the System • Cucumber / Selenium / Fitnesse • Written by QA & Analysts • Happy Path & some unhappy paths Freitag, 3. Juni 2011
- 26. Integration Tests • Cover ~20% of system • Test assembly of components • Cucumber / Selenium / Fitnesse • Written by Architects Freitag, 3. Juni 2011
- 27. System Tests • Cover 10% of the system • Test the whole system • Cucumber / Selenium / Fitnesse • Written by Architects Freitag, 3. Juni 2011
- 28. Manual Tests • Exploratory Manual Tests • Manual but not scripted • Human ingenuity & creativity Bach, J. (n.d.). Exploratory testing explained. Online: http://www. satisfice. com/articles/et-article. pdf. Freitag, 3. Juni 2011
- 29. Where do we test? Freitag, 3. Juni 2011
- 30. Testing the edges Freitag, 3. Juni 2011
- 31. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 Acapela Freitag, 3. Juni 2011
- 32. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 Acapela Freitag, 3. Juni 2011
- 33. How do we test? Freitag, 3. Juni 2011
- 34. http://blog.objectmentor.com/articles/2009/09/29/ ruining-your-test-automation-strategy Freitag, 3. Juni 2011
- 35. Tools of the trade Selenium Cucumber Steak capybara webkit zombie Freitag, 3. Juni 2011
- 36. only cover a minimal part the web frontend, but our apps are so much more Freitag, 3. Juni 2011
- 37. Decoupling Systems diverge APIs diverge Freitag, 3. Juni 2011
- 38. Web Services Many services outside our control Freitag, 3. Juni 2011
- 39. Freitag, 3. Juni 2011
- 40. Synchronizing • Tests require complex state across several systems • Tests involvebetween systems interactions complex Freitag, 3. Juni 2011
- 41. Questions Freitag, 3. Juni 2011
- 42. Questions to ask • Development • Testing • Production Freitag, 3. Juni 2011
- 43. Development Is the system I‘m building correct? Freitag, 3. Juni 2011
- 44. Test Do all the pieces of the system work together? Freitag, 3. Juni 2011
- 45. Production Are all parts of my system working? Freitag, 3. Juni 2011
- 46. Development • Test Driven Development only takes you so far • Test a single component • Unit tests in work, but don‘t components multiple „cross the chasm“ Freitag, 3. Juni 2011
- 47. Development • Setting up a complete system for development can be taxing • pow to the rescue http://pow.cx/ • Foreman https://github.com/ddollar/foreman Freitag, 3. Juni 2011
- 48. Testing • Integration tests need to stress complete application stack • Does your CI Server handle this? Freitag, 3. Juni 2011
- 49. Outside your app Freitag, 3. Juni 2011
- 50. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 Acapela Freitag, 3. Juni 2011
- 51. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 Acapela Freitag, 3. Juni 2011
- 52. Write Mock Services • If youitdon‘t control it, mock • „Rails Enterprise ready ™ “ Freitag, 3. Juni 2011
- 53. Mock Services • Allowsspecification against a „real“ you to code • Allows of said code against failure you to service • Gives you a stick to beat up the developers of said service („Rails-Enterprise ready“ ™ ) Freitag, 3. Juni 2011
- 54. Making it work Freitag, 3. Juni 2011
- 55. Version Control Repository setup One Big repository? One repository for each component? Tagging / Syncing Freitag, 3. Juni 2011
- 56. Development Servers Webrick doesn‘t scale but: it‘s „easy“ for Rack based apps pow by 37 Signals (Mac Only) Freitag, 3. Juni 2011
- 57. Configuration Configuring half a dozen applications Again: Synchronisation Inject configuration (Example Picky) http://florianhanke.com/picky/ Freitag, 3. Juni 2011
- 58. Production Is my system working correctly? Probes • DTrace (OSX, Solaris) • Monitoring (Scout, NewRelic) • Your own probes Freitag, 3. Juni 2011
- 59. Freitag, 3. Juni 2011
- 60. Freitag, 3. Juni 2011
- 61. Car industry If car manufacturers would build engines the way enterprise IT systems are built, you‘d need to remove the engine from the car and disassemble it completely to find a fault — @MMahlberg, 2011 Freitag, 3. Juni 2011
- 62. OBD • 1996: The OBD-II specification is made mandatory for all cars sold in the United States. • 2001: The European Union makes EOBD mandatory for all gasoline (petrol) vehicles sold in the European Union, starting in MY2001 (see European emission standards Directive 98/69/EC [1]). • 2008: All cars sold in the United States are required to use the ISO 15765-4 [2] signaling standard (a variant of the Controller Area Network (CAN) bus). http://en.wikipedia.org/wiki/On-board_diagnostics Freitag, 3. Juni 2011
- 63. Solved Problem Freitag, 3. Juni 2011
- 64. Freitag, 3. Juni 2011
- 66. Diagnosis Channel • Known from Car / Planes / Mainframes • Red lights on your dashboard • Complete diagnosis of system possible • Complete remote control possible Freitag, 3. Juni 2011
- 67. Rolls Royce • InFlight Diagnosis of engines remotely Freitag, 3. Juni 2011
- 68. Dashboard • Look into state of application • Sanity Checker Freitag, 3. Juni 2011
- 69. Dashboarding • Extract relevant data from your system • How many req / s • How many logins • Delta / Trends Freitag, 3. Juni 2011
- 70. SanityChecker • API stateallows you to extract the that of your app • DB there? • All neighbouring components reachable? Freitag, 3. Juni 2011
- 71. Chaos Monkey • Random failures • Better reliability Freitag, 3. Juni 2011
- 72. Build testable apps • Expose your internals (through APIs) • Use SOLID design • Single Responsibility Principle • Open Closed • Liskov Substitution Principle • Inversion of Control • Dependency Injection Freitag, 3. Juni 2011
- 73. Testable apps • Obey the abstraction layers • keep of abstractionat it‘s own level each method Freitag, 3. Juni 2011
- 74. Build probe-able apps • Measure APIs $ rails console production Freitag, 3. Juni 2011
- 75. Build controllable apps • Call functionality of the app • Create new user • Create transactions • Delete stuff • Do stuff • ... Freitag, 3. Juni 2011
- 76. Test user / Magic • In „Enterprise“ systems: • Test Company • Test User • Test Accounts • Are those in production system? • Do everything - or not quite? Freitag, 3. Juni 2011
- 77. Use production data in development / test Freitag, 3. Juni 2011
- 78. Wrapping up Freitag, 3. Juni 2011
- 79. What & Why & How of Testing Discover the state of your apps Make testable apps Freitag, 3. Juni 2011
- 80. Thank you! Freitag, 3. Juni 2011
- 81. Credits • http://www.flickr.com/photos/ myoldpostcards/4020087797/ • http://www.flickr.com/photos/ benncapon/390585495/ • iStockphoto Freitag, 3. Juni 2011
- 82. License http://creativecommons.org/licenses/by-nc-sa/2.5/ch/ Freitag, 3. Juni 2011