What can possibly go wrong if i dont e2 e test my packages?
Download as PPTX, PDF•0 likes•31 views
The document discusses the importance of end-to-end (E2E) testing for software packages, particularly in the context of Node.js publishing via Verdaccio. It highlights potential issues that may arise without E2E testing, such as incorrect entry points and missing dependencies, and presents examples of how other projects (like @angular-cli and @create-react-app) integrated E2E testing into their workflows using Verdaccio. Recommendations include rigorous code reviews, local registry publishing, smoke testing, and priority on user behavior to mitigate risks.
What can possibly go wrong if i dont e2 e test my packages?
- 1. What can possibly go wrong if I don’t E2E test my packages? Juan Picado Core Team - Verdaccio Node.js Meetup - Berlin, 2019
- 2. Senior Front-End Engineer at eBay @jotadeveloper @juanpicado ● Joined Verdaccio August 2016 ● I love Open Source ● I do Node.js just for fun ● I like write documentation
- 3. Objective ● Brief Introduction to Verdaccio ● Create awareness about possible mistakes on publishing packages ● How others are doing it? ● What measures we might take in order to avoid issues
- 4. How to install Verdaccio? A lightweight proxy and private registry for Node.js https://bit.ly/2KMkNW8
- 6. End to End Testing End-to-end testing is a technique used to test whether the entire application flow behaves as expected from start to finish https://ebaytech.berlin/100-percent-test-coverage-is-not-enough-3d733551bc3f
- 7. End to End Testing Tests that simulate real user scenarios can easily help to determine how a failing test would impact the user. https://ebaytech.berlin/100-percent-test-coverage-is-not-enough-3d733551bc3f
- 8. Let’s translate this to Node.js publishing scenario Provider Consumer
- 9. What can possibly go wrong?
- 10. Awareness
- 11. Entry point is incorrect
- 12. Entry point is incorrect
- 13. Missing dependencies ● Flat node_modules structure issue ○ Using a sub-dependency provided by a dependency
- 15. ● Dependencies removed from the registry ○ npmjs provides 72h window to remove pkgs (immutable) ○ private registries do not follow this rule (mutable) https://github.com/ljharb Missing dependencies Jordan Harband
- 16. Dependencies holded by npmjs for security reasons Missing dependencies
- 17. Packages just vanish from the public registry Missing dependencies
- 18. Dependencies using git:// or relative locations Missing dependencies
- 19. Other reasons .npmignore is misconfigured main entry point is included by default
- 20. Wrong compilation target (eg: @babel/env) Other reasons
- 21. Excess of confidence on dependency bots Other reasons
- 22. What other devs are doing?
- 23. @angular-cli 22k ⭐️ 1,317,000 weekly downloads
- 24. @angular-cli ● Command Line Tool (scaffolds apps and libraries) ● Monorepo based on @angular-devkit ○ Similar to lerna structure ● Use yarn workspaces ● Custom build setup (no Jest or similar)
- 25. @angular-cli
- 26. @angular-cli
- 29. Before was required mock dependencies as local tarballs @angular-cli
- 30. ● Special local build for E2E ● Using absolute paths for dependencies ○ No semver ○ No guarantee the local dependencies and their transitive dependencies were used ● Unable to test the ng update command ● Some tests always tested last publish Which issues angular cli was having? @angular-cli
- 31. How @angular-cli solved the issue @angular-cli
- 33. Configuration ● Custom package access ● $all (any user) ● No proxy on private pkgs @angular-cli
- 34. Verdaccio integration at @angular/cli 1. Spawn verdaccio 2. Custom build read all tests 3. Build local packages 4. Local Publish (+build) to Verdaccio (@angular/dev-kit) 5. Create a test project with ng CLI (fetching from verdaccio) 6. Run all tests in serie @angular-cli
- 36. Spawn Verdaccio with JavaScript @angular-cli
- 37. create-react-app 74k ⭐️ 1,117,000 weekly downloads
- 38. @create-react-app ● Command Line Tool (scaffolds React apps) ● Monorepo based on Lerna ● Use yarn workspaces ● Use bash instead Javascript for E2E
- 41. ● Prepare a workspace, copy, paste files ● npm link workaround Which issues create-react-app was having? @create-react-app
- 42. How create-react-app solved the issue @create-react-app
- 43. Using bash to instantiate a server@create-react-app
- 44. Configuration ● Default package access ● $all (any user) ● Uplinks advanced connections options for fetch @create-react-app
- 45. Verdaccio integration at create-react-app 1. Bootstrap verdaccio (bash script) 2. Build all packages 3. Publish on Verdaccio with Lerna 4. Run npx create-react-app test-app i. Build the app ii. Run smoke tests iii. Verify internal stuff and clean up @create-react-app
- 46. Who else?
- 47. ● CircleCI Integrated ● Use Verdaccio within a CircleCI Orbs (https://circleci.com/orbs/) 11k ⭐️ 1,041,000 weekly downloads https://aurelia.io/ https://circleci.com/gh/aurelia/aurelia/37487
- 48. 34k ⭐️ 11,041,000 weekly downloads https://babeljs.io/
- 49. 34k ⭐️ 11,041,000 weekly downloads https://babeljs.io/
- 50. 34k ⭐️ 11,041,000 weekly downloads https://babeljs.io/
- 51. 🤓
- 52. Measures
- 53. ● Intense Code Reviews (config files that affect packages) ● Publish to a local registry in each PR ● Perform a real installation ● Run smoke tests ● Ensure you are testing the user behaviour ● Rely real tooling (npm, npx, yarn, etc...)
- 54. What can possibly go wrong if I don’t E2E test my packages?
- 55. Ship confidence
Editor's Notes
- #31: Context about “Some tests always tested last publish”