Modernization patterns to refactor a legacy application into event driven microservices
3 likes996 views
The document outlines strategies for migrating legacy applications to event-driven microservices, emphasizing the benefits of modernization such as reduced time to market and improved team autonomy. Key patterns discussed include the strangler pattern for incremental migration, the outbox pattern for reliable inter-service communication, and the use of tools like Apache Kafka and Debezium for change data capture. It emphasizes the importance of planning and utilizing standardized tools to facilitate a low-risk transition to cloud-native architectures.
Modernization patterns to refactor a legacy application into event driven microservices
- 1. Modernization Patterns to Refactor a Legacy Application Into Event-driven Microservices 1 Bilgin Ibryam Product Manager @Red Hat @bibryam
- 2. 2 We build our computers the way we build our cities - over time, without a plan, on top of ruins. Ellen Ullman
- 3. About me 3 ▸ Product Manager at Red Hat ▸ Former Principal Architect ▸ Committer at Apache Camel ▸ Author ･ Camel Design Patterns ･ Kubernetes Patterns ▸ @ bibryam ▸ https://www.ofbizian.com
- 4. About this talk 4 Application migration/modernization Rs ▸ Rehosting - lift-and-shift ▸ Replatorming - lift, tinker, and shift ▸ Refactor / Re-architect ･ Debezium ･ Apache Kafka ･ Kubernetes
- 5. Starting point 5 Challenges with monolithic applications ▸ Frequent deployment is difficult ▸ Obstacle to scaling development ▸ Scaling the application can be difficult Expected modernization benefits ▸ Reduce time to market ▸ Greater team autonomy ▸ Improved automation ▸ Increased application performance
- 6. Target state 6 Cloud native microservices’ characteristics ▸ Independently deployable ▸ Modeled around a business domain ▸ Own their data ▸ Emit events (based on EDA) ▸ Built on open source technology such as Kubernetes, Apache Kafka, Debezium Measuring the results ▸ Lead time for change ▸ Deployment frequency ▸ Mean time to recovery
- 8. Strangler Pattern 8 High-level steps ▸ Identify functional boundary ▸ Migrate/reimplement functionality ▸ Synchronize data ▸ Reroute traffic to new service ▸ Decomposition old service Main benefits ▸ No big bang migration ▸ Minimal changes to the monolith ▸ Low risk with the ability to fall back
- 9. Functional boundary 9 Functional boundary considerations ▸ Aggregates, bounded context based on DDD ▸ Event Storming technique by A. Brandolini ▸ Data model coupling ▸ Module dependencies Where to start from? ▸ Easy wins ▸ Noticeable improvement ▸ A representative effort
- 10. Interception options 10 Prerequisites ▸ Ability to identify inbound calls ▸ Ability to direct calls to new service (and back) Routing options ▸ Client redirection ▸ Transparent HTTP proxy ▸ Shared protocol translation proxy ▸ Embedded/sidecar proxy
- 11. What is Apache Camel? 11 Camel is an integration toolkit that offers: ▸ Hundreds of connectors ▸ Tens of integration patterns ▸ Multiple data formats and languages ▸ Domain specific languages: XML, Java, YAML
- 12. Interception options 12 https://k8spatterns.io/ Prerequisites ▸ Ability to identify inbound calls ▸ Ability to direct calls to new service (and back) Routing options ▸ Client redirection ▸ Transparent HTTP proxy ▸ Shared protocol translation proxy ▸ Embedded/sidecar custom proxy
- 13. Kubernetes Pods 13 Deployment guarantees Lifecycle guarantees
- 15. Sidecar Pattern 15 https://k8spatterns.io/ Prerequisites ▸ Ability to identify inbound calls ▸ Ability to direct calls to new service (and back) Routing options ▸ Client redirection ▸ Transparent HTTP proxy ▸ Shared protocol translation proxy ▸ Embedded/sidecar custom proxy
- 16. Database strangulation 16 Database first Database and code together Code first
- 17. Data synchronization 17 Cons ▸ Polling delay, might miss updates ▸ Polling overhead ▸ Deletes not captured ▸ Not transparent to writing application Pros ▸ Simple Installation and configuration Cons ▸ Varying capabilities ▸ Limited integration capabilities ▸ Not portable ▸ Not easily testable ▸ Not transparent to source database Pros ▸ No additional application required Cons ▸ Requires specialized tools ▸ Requires database configuration Pros ▸ All changes (deletes) are captured ▸ No polling delay or overhead ▸ Transparent to writing application Triggers Log readers Queries/Polling
- 18. What is Debezium? 18 Debezium in a nutshell ▸ Open source Change Data Capture platform ▸ Snapshotting, filtering, transformations ▸ MySQL, PostgreSQL, MongoDB, SQL Server, Db2, Oracle, Vitess, Cassandra ▸ Use cases: data replication, cache updates, search index updates, audit logs, sync data between services
- 19. Strangler Pattern with Debezium 19 Strangler Pattern with Debezium ▸ Transparent to the writing application ▸ Initial bulk import through snapshotting ▸ Schema, table, column filtering ▸ Single Message Transformations as anti-corruption layer ▸ Schema Registry for schema validation and compatibility enforcement of data models Debezium with Apache Kafka ▸ Guaranteed ordering, message compaction ▸ Pull based - re-read from start, or new changes ▸ Kafka Connect - offset tracking, fail over
- 20. Release steps 20 Steps so far ▸ Identified a functional boundary ▸ Migrated it as a new service ▸ Created a new data model in a new database ▸ Automated and deployed everything ▸ Bulk imported data and kept it in-sync ▸ No traffic routed to the new service
- 21. Release steps 21 Next step ▸ Route READ traffic to the new service
- 22. Release steps 22 Next steps ▸ Route READ and WRITE traffic to the new service ▸ Two-way data synchronization in progress
- 23. Release steps 23 Next steps ▸ Stop WRITE traffic to legacy system ▸ Stop data synchronization from legacy system
- 24. Release steps 24 Final steps ▸ STOP READ traffic to the legacy application ▸ Stop data synchronization to legacy system ▸ Decommission migrated module
- 25. New Challenges
- 26. Modernization challenges 26 Migration challenges ▸ Low-risk changes with demonstrable progress ▸ Deliver enhancements and new “business value” ▸ Simultaneously delivering new services Distributed systems challenges ▸ Automation and operations at scale ▸ Dual-writes and long-running business transactions ▸ Analytical and reporting data needs
- 27. Operating event-driven services at scale 27 Microservices on Kubernetes ▸ Deployment/rollback ▸ Placement/scheduling ▸ Configuration management ▸ Resource/failure isolation ▸ Auto heal/upgrade (through operators) Event-driven applications on Kubernetes ▸ Strimzi - operating Apache Kafka cluster ▸ KEDA - event-driven workload autoscaling ▸ Debezium - change data capture platform ▸ Apicurio - Schema Registry and operators to run them
- 28. Outbox Pattern 28 Offers an approach for services to update their data store and notify other services in a reliable and eventually consistent manner. ▸ Addresses the dual-write problem ▸ Offers “read your own writes" semantics ▸ “at-least-once” semantics for consumers Reliable Data Exchange With the Outbox Pattern
- 29. Saga Pattern 29 Splits up long-running business transactions into a series of multiple local transactions. When implemented using the Outbox Pattern, it offers the benefits of orchestration and asynchronous communication. ▸ Orchestration based coordination ▸ Asynchronous communication with Apache Kafka Saga Orchestration for Microservices Using the Outbox Pattern
- 30. Summary 30 Modern software systems are like cities - they evolve over time, on top of legacy systems. Using proven patterns and standardized tools help create long-lasting systems that grow with your needs. ▸ Strangler Pattern - a low-risk application migration technique ▸ Outbox/Saga Patterns - reliable inter-service communication approach ▸ Debezium - a non-intrusive toolking for change data capture ▸ Kubernetes / Strimzi / Apache Kafka - de facto standards in their fields
- 31. Resources 31 ▸ Monolith To Microservices ▸ Building Event-Driven Microservices ▸ Designing Data-Intensive Applications ▸ Kubernetes Patterns ▸ Debezium.io ▸ Keda.sh ▸ Strimzi.io ▸ github.com/windup
- 32. 32 Red Hat OpenShift Streams for Apache Kafka a fully managed Apache Kafka service by Red Hat http://red.ht/TryKafka Try Apache Kafka in seconds
- 33. Thank you! 33 Bilgin Ibryam Product Manager @Red Hat @bibryam