Benefits of Stream Processing and Apache Kafka Use Cases
6 likes14,515 views
The document outlines the benefits and use cases of Apache Kafka in event-driven systems, highlighting its importance for real-time data processing across various industries. It discusses the advantages of using the Confluent platform with Apache Kafka for enhancing scalability, improving customer experience, and enabling effective data integration. Additionally, it presents various applications of Kafka in sectors such as banking, e-commerce, government, and online gaming.
Benefits of Stream Processing and Apache Kafka Use Cases
- 1. 1 Fundamentals for Apache Kafka® Benefits of Stream Processing and Apache Kafka Use Cases Mark Fei, Sr. Technical Trainer, Confluent
- 2. 2 Session Schedule ● Session 1: Benefits of Stream Processing and Apache Kafka Use Cases ● Session 2: Apache Kafka Architecture & Fundamentals Explained ● Session 3: How Apache Kafka Works ● Session 4: Integrating Apache Kafka into your Environment
- 3. 3 Learning Objectives After this module you will be able to: ● Identify the need for event-driven systems ● Name specific use cases where real-time data is critical ● Explain the advantages of using Apache Kafka and Confluent Platform as your Event Streaming platform
- 4. 4 Motivation The shift to Event-driven Systems has Already Begun …
- 5. 5 Motivation This leads us to … A single platform to connect everyone to every event Real-time streams of events All events stored for historical view
- 6. 6 Motivation Successful Digital Businesses are Inherently Event-driven Born cloud-native … Traditional ones that adapt … Social Networks Enabling Event Sharing Streaming Provider On-demand Digital Content Newspaper Provide a single Source of Truth Credit Card Payments Microservices Architecture
- 7. 7 Motivation Born cloud-native … Traditional ones that adapt … Ride Hailing Connecting Provider with Customer in real- time Connected Cars IoT – Real-time Traffic Routing
- 8. 8 Motivation Apache Kafka: The De- facto Standard for Real- Time Event Streaming • Global-scale • Real-time • Persistent Storage • Stream Processing
- 9. 9 Motivation Thousands of Companies Worldwide trust Kafka for their Journey towards “Event-driven” https://kafka.apache.org/powered-by
- 10. 10 Motivation Over 35% of Fortune 500 Companies Already Trust Kafka for Mission Critical Apps
- 11. 11 Real-time Fraud Detection ● Act in real-time ● Detect Fraud ● Minimize risk ● Improve customer experience
- 12. 12 Automotive The Future of the Automotive Industry is a Real Time Data Cluster
- 13. 13 Real-time e-Commerce Rewards Program ● Onboarding new merchants faster ● Increased speed at which mobile applications are delivered to customers ● Enabled a full 360 view of customers ● Enhanced performance and monitoring ● Projected savings of millions of dollars
- 14. 14 Customer 360 ● Improved data integration ● Increased up-sell and cross-sell opportunities ● Increased scalability and flexibility ● Saved costs
- 15. 15 Core Banking ● Empowered successful core banking platform relaunch ● Met HA and SLA needs ● Improved scalability ● Power AI for ChatBots
- 16. 16 Online Gaming Data Pipelining ● Increased reliability ● Accurate, real-time data ● Ability to process data at scale ● Faster ramp time
- 17. 17 Government ● Near real-time events and better data quality ● Increased efficiency ● Ability to change their organization ● Produce & store population data from several sources ● Reduce welfare crime through strengthened identity management ● Provide better privacy and meet GDPR requirements
- 18. 18 Financial Services Customer communications ● Enhanced the customer experience ● Enabled “One Bank” strategy Payments engine ● Improved fraud detection engine, saving millions of euros ● Grew topics in production by 600 percent
- 21. 21 The Maturity Model Kafka is a Good Starting Point, Confluent Completes the Journey
- 22. 22 Confluent Platform Build a Central Nervous System for your Modern Event-driven Enterprise
- 23. 23 Confluent Platform The Confluent Platform enables companies to respond accurately and in real-time to business events
- 24. 24 Confluent Platform Confluent is Enabling Event-driven Transformation across Industries
- 25. 25 Confluent Platform A Complete Set of Development, Operations, and Management Capabilities to run Kafka at Scale
- 27. 27 Confluent Platform Deployment Models
- 29. 29 Confluent Control Center Kafka is powerful … but has many parts
- 30. 30 Confluent Control Center Management and Monitoring for the Enterprise
- 31. 31 Confluent CLI ● Platforms: Linux, Unix-based ● License: Proprietary ● Packaging: Independent of CP ● Key Features: ○ RBAC management ○ Password protection ○ Subsumed confluent-cli commands for local
- 32. 32 Role Based Access Control ● Availability: CP 5.3 Utilizes a predefined set of Roles ● User who is assigned a role receives all privileges of that role ● Each user could belong to multiple roles ● Privileged user within a given scope can create/update other user’s roles ● Privileged user can update roles for existing users ● Across the platform (KSQL, Connect, Schema Registry, Connectors, etc.)
- 33. 33 Confluent Operator Confluent Operator for Kubernetes runs on any platform at any scale What: Deployment and management automation for Confluent Platform across environments For Whom: Organizations that have standardized on Kubernetes as the platform runtime
- 34. 36 Confluent – Feature Comparison Feature Confluent Platform (On Premise) Confluent Cloud (Fully Managed) Scale Unlimited throughput, unlimited retention Unlimited throughput, unlimited retention (CCE) Availability Self Managed 99.95% uptime SLA Durability Self Managed Multi-AZ with 3 availability zones (option) Updates 3-4 Annual Releases Weekly Fully Managed Updates Support 24x7 Gold and Platinum (option) 24x7 Gold SVPC peering (option) Pricing Node based pricing Usage Based Pricing Packaging/Environment Tarball, deb, rpm, zip, Docker images AWS, GCP, Azure
- 36. 38 Confluent Hub Discover and use trusted sources and sinks from Confluent, our partners, and the community.
- 37. 39 Confluent Hub GOAL: Connect Everything!
- 38. 40 Continue your Apache Kafka Education! ● Confluent Operations for Apache Kafka ● Confluent Developer Skills for Building Apache Kafka ● Confluent Stream Processing using Apache Kafka Streams and KSQL ● Confluent Advanced Skills for Optimizing Apache Kafka For more details, seehttp://confluent.io/training
- 39. 4141 Certifications Confluent Certified Developer for Apache Kafka (aligns to Confluent Developer Skills for Building Apache Kafka course) Confluent Certified Administrator for Apache Kafka (aligns to Confluent Operations Skills for Apache Kafka) What you Need to Know ○ Qualifications: 6-to-9 months hands-on experience ○ Duration: 90 mins ○ Availability: Live, online 24/7 ○ Cost: $150 ○ Register online: www.confluent.io/certification
- 41. 4343 cnfl.io/blog cnfl.io/slack Stay in touch! cnfl.io/kafka-training
- 42. 44 Thank you for attending! • Thank you for attending thesession! • Feedback to: [email protected]
- 43. 45 Copyright ©Confluent, Inc. 2014-2019. Privacy Policy | Terms &Conditions. Apache, Apache Kafka, Kafka and the Kafka logo are trademarks of the Apache Software Foundation