API Testing with Open Source Code and Cucumber
Download as PPTX, PDF2 likes2,181 views
The document discusses the shift-left approach in software testing, emphasizing early involvement of QA in the development lifecycle and the adoption of code-driven testing methods for various testing types such as functional, load, and security testing. It highlights tools and libraries for API testing, such as Rest-Assured and Cucumber, and elaborates on challenges associated with code-driven testing, including environment configuration and external dependencies. Additionally, it includes practical examples of using Cucumber for API testing, demonstrating how scenarios can be written and executed effectively.
![1515
Automated Build / Test executions
Cucumber “execution pipeline”
Cucumber
Runner
StepDefs
Component /
System under test
1. Parse
2. Invoke
.feature files
Ant/Maven/Gra
dle/CLI/etc [@tag]](https://image.slidesharecdn.com/sb-apitestingwithcodeandcucumber-170810201114/85/API-Testing-with-Open-Source-Code-and-Cucumber-15-320.jpg)
API Testing with Open Source Code and Cucumber
- 1. API testing with Open Source Code libraries and Cucumber Ole Lensmar – SmartBear
- 2. 22 Short about “Shift-Left” • Enterprise testing and QA is moving from late to early phases in the development life-cycle – QA -> Dev -> Design • Tests are being expressed as code instead of using visual tools (by both QA and Dev) – Re-use of existing Test assets / scripts for different types of tests • Code-driven approach not just for functional testing – Load/performance testing – Security testing – Mocking / Virtualization
- 3. 33 Testing at API layer vs Unit tests • API-layer testing = integration testing of implementing classes/components • Unit-testing frameworks often used to ”orchestrate” – Junit / TestNG / Failsafe etc. • Tests are managed together with implementing code - using traditional SCM tools • Tests are executed as part of existing builds – Test running instance of API vs implementation
- 4. 44 Code driven approach to API Testing • HTTP clients – Apache HTTPClient, OkHttp, Super-agent, etc. • Generated clients – Swagger Codegen, WSDL2Java, etc. • Special purpose libraries – REST-Assured (Java/Groovy) – Chakram, (JS) – ReadyApi4j (Java) • Cucumber / BDD - many platforms/languages
- 5. 55 Approach for comparison • Simple scenario with single and chained REST requests testing the SwaggerHub integration API • Framework specific highlights • All available on GitHub: https://github.com/olensmar/apidays-apitesting • All run with maven (“mvn test”) except javascript sample
- 6. 66 Apache HTTP-Client • Widely used java client library for HTTP • Exposes fluent API for building / executing requests and reading responses • Lacks built in support for JSON / XML parsing or assertions – Need to do this manually using Jackson/DOM/etc.
- 7. 77 Generated clients • Use code-generation tools to generate client from API description • Write tests using generated client – Automate client generation as part of build • Pros – API changes can break tests – Isolates from network layer – tests at functional level • Cons – Relies on correctness of client-generator – Isolates from network layer – tests at functional level
- 8. 88 REST-Assured • Fluent Java API for testing REST APIs • Support both JSON and XML content assertions – Uses standalone JSONPath and Xpath implementations • Built in support for multiple authentication types, XSRF, etc. • Very customizable at the network/http/marshalling layers • http://rest-assured.io/
- 9. 99 ReadyAPI4j • Fluent Java API – Leverages the SoapUI Testing engine – REST, SOAP, Assertions, orchestration, etc. • Internally compiles tests into “Test Recipes” (json) – Can be reused for performance/security testing in SoapUI • Execution either locally or remotely – Local execution for ease of use (open source) – Remote execution for access control, performance, reporting, etc. (commercial)
- 10. 1010 Maven/Java project ReadyApi4j execution model API Test /src/test/java ReadyApi4j API Local Engine ReadyAPI Testserver API(s) under test use use call API calls
- 11. 1111 Challenges with code-driven API testing • Running against different environments – Temporary environments (Docker can help) – Actual environments • Testing either with or without network – Network is more realistic – more work to automate • External dependencies may need to be mocked – 3rd party APIs, databases, etc. – For both functional and non-functional testing • Simulate errors, response times, invalid responses, etc.
- 12. Cucumber for API testing
- 13. 1313 Cucumber – “Executable Specifications” • Heavily associated with Behavior Driven Development • “Plain language” verbalization of requirements – POs, Business Analysts, Developers, Testers, etc. • For example Given a user has logged in with valid credentials When a deposit of 100 USD is made Then the account balance increases by 100 USD And the user gets a confirmation message
- 14. 1414 Scenarios/Features written in “Gherkin” • Given X • When Y • Then Z • More keywords: And, Feature, Scenario, Background, etc. • Packaged into “feature files” • Cucumber reads feature files and executes corresponding “StepDefs” or “GlueCode”
- 15. 1515 Automated Build / Test executions Cucumber “execution pipeline” Cucumber Runner StepDefs Component / System under test 1. Parse 2. Invoke .feature files Ant/Maven/Gra dle/CLI/etc [@tag]
- 16. 1616 Cucumber StepDefs • Code that is executed by the cucumber framework for each statement • Must be implemented manually to support the desired vocabulary • Cucumber Java example -> code!
- 17. 1717 Cucumber “styles” • Declarative style – Domain-specific vocabulary for describing the business requirements – Appeals to non-technical users – High overhead in creation and maintenance • Imperative style – Generic vocabulary for describing requirements – Appeals more to technical users – Lower maintenance – but “Misses the point” • Either requires implementation of StepDefs
- 18. 1818 Declarative API Testing Example Feature: SwaggerHub REST API Scenario: Default API Listing When a request to the API listing is made Then a list of APIs should be returned within 500ms Scenario: TestServer API Retrieval Given an owner named smartbear And an api named ready-api-testserver And a version named 1.0.0 When a request to the API listing is made Then an API definition should be returned within 500ms
- 19. 1919 Imperative API Testing Example Feature: Petstore API Scenario: Find pet by status Given the API running at http://petstore.swagger.io/v2 When a GET request to /pet/findByStatus is made And the status parameter is test And the Accepts header is application/json Then a 200 response is returned within 50ms
- 20. 2020 Imperative API Testing Example with Swagger Feature: Petstore API operationId gives path and method operation parameter type not neededa defined response
- 21. 2121 Managing Feature files and StepDefs • Keep feature files together with source code and/or in separate repository • Use tagging feature of Cucumber to selectively (re)run tests • Use datasets for parameterization – Inline or external • Bundle StepDefs in separate library – Automate execution
- 22. 2222 ReadyApi4j - Cucumber for APIs • Out-of-the box generic vocabulary for REST API Testing - integrated Swagger support • Extensible – built on Cucumber for Java – Open-source on GitHub – https://github.com/readyapi/testserver-cucumber • Packaged as library or Docker image – https://hub.docker.com/r/smartbear/cucumber4apis • Uses ReadyApi4j as execution engine (either local or remote)
- 23. 2323 That’s it! – Questions? • @olensmar • https://github.com/olensmar/apidays-apitesting • https://hc.apache.org • http://rest-assured.io/ • http://dareid.github.io/chakram/ • http://frisbyjs.com/ • https://github.com/SmartBear/readyapi4j • https://cucumber.io/ • https://github.com/readyapi/testserver-cucumber • https://hub.docker.com/r/smartbear/cucumber4apis