PHP Symfony MicroServices Migration @MeeticTech
- 2. Who am I ? Joined Meetic in 2014 Interested by their volumes, functionnal richness and experience Etienne Broutin Software Architect
- 3. Started in 2001 • Active in 15 countries • Dating leader in Europe • Millions of Monthly Active Users • 150 people in IT teams
- 4. Back again
- 5. 2013 Reasons for refactoring plan Unmaintainable code Duplicates Untestable Need elders validation Unable to monitor Continue on failure policy Based on global log volume
- 6. 2013 Initial project Webservices Backoffices Mobile Web WAP Desktop Cronjobs … Exposition Layer Private webservices
- 7. Started So cute in the beginning !
- 8. But new issues Duplicated business logic Merge conflicts Longer tests (40min) Refactorings started to become more and more difficult
- 9. We were building a new monolith ! Only 15% of business features Only 10 developers, target 40
- 10. Micro-service pros • Simpler design phase • Manage refactoring impacts • Faster feedbacks by software factory • Faster deployments • Real ownership by developers, easier upgrades
- 11. Micro-service cons • Implementation of interfaces • Response time • CPU and network usage • Time to setup a new application • Can also become hard to understand
- 12. New stack Exposition Layer Event bus Consumers Micro-services BackOffices Webservices
- 13. 2 years later.... Micro-service design Dependencies Team Production
- 15. How to split ? Functionnal splitting Built around a concept Pictures Right
- 16. What size ? 25 micro-services 9 avg route count 7k avg NLOC 4 avg relationnal tables to understand a micro-service 1 to 2 days
- 17. Testability Build in 3minSimpler functionnal tests
- 18. Modelization POST /photos/{accountId} GET /photos/{accountId}/{photoId} PUT /photos/{accountId}/{photoId} DELETE /photos/{accountId}/{photoId} POST /boost/{boostId}/increment REST approach Commands
- 19. Data isolation Strong and difficult choice But • brings scalability • brings visibility • data consistency • enables caching • manage data migration
- 20. Limits of data isolation Build a denormalized database OLTP Reactive (event bus) Batch processing BI
- 21. How to deal with dependencies ?
- 22. Product team has a new idea...
- 23. In practice 2/3 of our micro-services do not depend on another micro-service took several refactorings for that
- 24. Defining perimeter is the key Messages Authentication nickname business model account status Photo Announce Moderation Photo + Moderation Announce + Moderation
- 25. Event bus to manage dependency Event bus Consumer Consumer Profile Mailing Optin
- 26. get photos Pattern 1 : Pull data - performance - reusability does have a photo itself ?can see photos ? + hides complexity for clients + fast change of business rules Exposition Layer Rights Photo
- 27. Pattern 2 : Push data from client + mutualize data fetching - harder to change if multiple clients send message Exposition Layer Rights Message conversation already started ? can start a new conversation ?
- 28. Pattern 3 : Store data Event bus read message Exposition Layer Message get visible messages Blacklist + performance + reusable - long to implement blacklist added hide messages
- 29. Other tips Batch calls GET /photos/{accountId}/14,15,21 Parallelize calls using Guzzle promise
- 30. Working together
- 31. Context Several independent agile teams No strong synchronization Continuous deployment 15 deployments per day
- 32. How do you manage compatibility ? No versioning • used internally • less maintenance • interface backward compatible
- 33. Automated environment Scripts with docker compose
- 34. Can I use any framework ? Any langage ? Any database ?
- 35. Common interface • REST + JSON • Naming conventions • Security • Common build • Common deployment chain
- 36. Don’t waste time : need a common basis Micro-services chassis • Security • Logging • HTTP Interface • DB & cache components
- 37. Share knowledge with non-tech teams Talkative names Visibility for project management profile will be updated soon changes on message are ready thanks
- 38. Documentation Inventory <<< Call graph generated from Elastic Search GET /profile/{accountId} Profile Exposition Layer Right Backoffice 4% 85% 11%
- 39. Ownership More initiatives More refactorings More upgrades More monitoring
- 40. Production
- 41. Hosting concerns It will overload the network ! It will take time to configure ! We will need much more servers !
- 42. Figures 1B calls / day 25ms response time 100 servers Who calls micro-services ? Exposition LayerEvent Bus Micro-services
- 43. Networking • Every Micro-services deployed on each server • Affinity for inter micro-service call Have not experienced need for : • Service isolation • Network constraints
- 44. 8 ms Symfony boostrap overhead limited by application size 17 ms 25ms Cost : 25%
- 45. Monitoring Alerting Elastic Search + Logstash + Kibana • Easy to locate errors • Performance analysis Monitoring route for every dependency GET /profile/{accountId} 11ms 100% POST /profile/ 42ms 99.992% ... P P
- 46. Design for failure Objective 99.999% Internally : analyse logs Externally : implement fallback behavior Exposition Layer 99.9% 98.3% 99.2% Reliability < 97.4%
- 47. Conclusion
- 48. Re-usability Re-used micro-services from acquisition fusion Used for several products
- 49. Huge opportunity for Data Best Database for each Scalability
- 50. Feedback on that choice For Meetic, this architecture was a good choice : • Symfony helps standardization • Can scale teams • Clear implementation of all features • Answer business needs
- 51. Any questions?