10 years of microservices at finn.no - why is that dragon still here (ndc oslo 2019)
0 likes•413 views
Over the past 10 years, FINN transitioned their monolithic Java application built on a Sybase database to a microservices architecture running on PostgreSQL, Kafka and Kubernetes. They broke the monolith into independent services but still struggled with dependencies on the legacy Sybase database. They learned it takes a long time to fully migrate and should focus on delivering value incrementally. While the transition is still ongoing, they have made major progress deploying new services and migrating data and are now 93% in Kubernetes with the goal of moving completely to the cloud.
10 years of microservices at finn.no - why is that dragon still here (ndc oslo 2019)
- 1. 1 10 years of microservices - and we still haven’t slain that dragon! NDC 2019-06-19
- 5. Henning Spjelkavik Senior software developer / Architect Board of Directors @spjelkavik FINN travel - kart.finn.no Kafka - Architecture - Tcpdump - SEO - Nodejs - Java - Smidig Nybygg - Eiendom - Torget - Business center - DDD - Business cases - Garbage collection - Linux - EWC - Jstack - Yourkit Mexico, Morocco, Spain, France, Sweden, Poland, Chile & Hungary
- 6. Quiz
- 7. 7 96%
- 8. 8 1182
- 10. 10 586
- 13. Data 2008 1 Sybase database (The Legacy DB) (+ data warehouse)
- 14. Data 2019 22 postgres clusters, 128 pg databases 1337 kafka topics (564 private, 532 public) 37.000 messages/s (avg 24h) 1 Sybase database (The Legacy DB) (+ data warehouse + data lake stuff)
- 15. Where are we today? What did we learn? What do we mean by "out of legacy"? 10 years - major events Conclusion Outline
- 16. 10 years of breaking up the monolith - and we still haven’t found what we’re looking for
- 17. Target 99.9% uptime / monthMay 2019: 99.99%. YTD 99.97% 1700M NOK revenue 2018
- 18. What did we learn?
- 19. “What problem are you trying to solve?”
- 20. “Not all of a large system will be well designed” - Eric Evans
- 21. Out of legacy? What do we mean?
- 25. Out of legacy? What do we mean?
- 26. Out of legacy - What (technical) problem does it solve? 1. Have to change the code and wait for deploy of the big monolith 2. Synchronous database call to legacy database 3. Synchronous call to a service, that calls the legacy database (Trans, New APIs, Ad-lookup-server) 4. Asynchronous dependency on data from the legacy database a. asking a replica/duplicate, or b. consuming a stream 5. Conforming to the model, possible to map both ways (Ad-input-server) 6. Same meta model a. e.g own idea on how we model an ad, but the service own their model b. possibly one way mapping 7. Separate ways - no relationship It may be poor business value in going all the way to level 7.
- 27. Timeline
- 28. 2004 FinnTech launched internationally The monolith called iAd Java & stored procedures in Sybase Presentation Business logic Data access Sybase
- 29. 2006 FINN kart (Maps) outside the monolith
- 30. 2009 First “modules” were separated out Presentation Business Logic - iAd Data access – stored procedures Login User Sybase SSO
- 31. Out of legacy - What (technical) problem does it solve? 1. Have to change the code and wait for deploy of the big monolith 2. Synchronous database call to legacy database 3. Synchronous call to a service, that calls the legacy database (Trans, New APIs, Ad-lookup-server) 4. Asynchronous dependency on data from the legacy database a. asking a replica/duplicate, or b. consuming a stream 5. Conforming to the model, possible to map both ways (Ad-input-server) 6. Same meta model a. e.g own idea on how we model an ad, but the service own their model b. possibly one way mapping 7. Separate ways - no relationship
- 32. 2012- Platform project; Later turned into new Order, Product, Billing and Ad-insertion Mobile web
- 33. 2014 Freeze of the legacy (iAd) - nothing new should be made here Officially renamed “microservices” Requirements / definition: https://confluence.schibsted.io/pages/viewpage.action?pageId=23581198
- 34. Sam Newman
- 35. What is a microservice? Requirements (2014) - Deployed independently of other services - Have a single owner (team) - Ownership of the data it needs (own database) => avoid integration through the database - Have an API - or listen/publish messages - Healthcheck - Monitoring - Proper API versioning - can be upgraded without requiring the clients to update - Don’t crash even if dependencies are down
- 36. Out of legacy - What (technical) problem does it solve? 1. Have to change the code and wait for deploy of the big monolith 2. Synchronous database call to legacy database 3. Synchronous call to a service, that calls the legacy database (Trans, New APIs, Ad-lookup-server) 4. Asynchronous dependency on data from the legacy database a. asking a replica/duplicate, or b. consuming a stream 5. Conforming to the model, possible to map both ways (Ad-input-server) 6. Same meta model a. e.g own idea on how we model an ad, but the service own their model b. possibly one way mapping 7. Separate ways - no relationship
- 37. The FINN “Death Star - 2015” - a Distributed Monolith? FINN consisted of more than 200+ highly coupled services (RPC), resulting in a “distributed monolith”
- 38. Udi Dahan, Eric Evans, Fred George
- 39. 2016 New Order, Product, Billing and Ad-insertion in place for all direct customers (not Partners) - integrating over Kafka 95% of the volume; only 30% of the revenue is running on new stacks Data is available on Kafka from Legacy Responsive web site (for end users) - outside iAd Out of Sybase project (cancelled after 6 months)
- 40. Out of legacy - What (technical) problem does it solve? 1. Have to change the code and wait for deploy of the big monolith 2. Synchronous database call to legacy database 3. Synchronous call to a service, that calls the legacy database (Trans, New APIs, Ad-lookup-server) 4. Asynchronous dependency on data from the legacy database a. asking a replica/duplicate, or b. consuming a stream 5. Conforming to the model, possible to map both ways (Ad-input-server) 6. Same meta model a. e.g own idea on how we model an ad, but the service own their model b. possibly one way mapping 7. Separate ways - no relationship
- 41. 2017 We want to move to the cloud Fiaas - our deployment platform built on kubernetes
- 42. 2018 New business center (“control panel”) Make it look good; fetch data where it’s needed Packages (for direct customers) GDPR Sybase Postgres Postgres Postgres Kafka
- 43. GDPR Take out and delete - simplified view Kafka Privacy broker (Saga per user request) User Recommendation Messaging Travel +++
- 45. Moving to the cloud The elephant in the data center: Sybase on Sparc/Solaris
- 46. “Spinning rust is dead. SANs are now a complete waste of time.” - Adrian Cockcroft, 2008
- 47. Sybase migrated off Sparc/Solaris to Linux Moving main services towards a public cloud; 93% of apps already in kubernetes 2019 Sybase Postgres Postgres Postgres Kafka
- 49. Learnings It did not go as fast as we dreamed. Have a roadmap, but meanwhile make it better to live with what exists. Encourage and enable teams to make services on the outside If every client (downstream) need to rewrite to realise benefits - when can you realistically be finished? Is it still worth the effort? (or at what speed) Good Devops/ “Accelerate”-metrics enables! (Lead Time, Deployment Frequency, Mean Time to Restore)
- 50. There wasn’t (yet) a strong business need of moving out of the legacy
- 51. Done differently Adjust expectations: complicated system; - accept where you are it takes a looong time to “rewrite everything”. Use lipstick to offer customer/user/business value today, accept some kludges Consider to separate parts of the database first (Sam Newman) Be aware of “this time we’re gonna do it right”-syndrome or NIH
- 52. We have a direction and a roadmap out of legacy
- 54. Henning Spjelkavik @spjelkavik Remember: “Not all of a large system will be well designed” - Eric Evans “Successful visionaries start from where they are, what they have, and what their customers have.” - Tom Gilb “What problem are you trying to solve?” Thank you!