Apache Kafka Fundamentals for Architects, Admins and Developers
1 like835 views
This document summarizes a presentation about Apache Kafka. It introduces Apache Kafka as a modern, distributed platform for data streams made up of distributed, immutable, append-only commit logs. It describes Kafka's scalability similar to a filesystem and guarantees similar to a database, with the ability to rewind and replay data. The document discusses Kafka topics and partitions, partition leadership and replication, and provides resources for further information.
Apache Kafka Fundamentals for Architects, Admins and Developers
- 1. 1 Mid-Ohio Data Users Group Meetup Organizer: Derek Kane Host: Navigator Management Partners
- 2. 2 Apache Kafka® Fundamentals for Architects, Admins and Developers
- 3. 3 About Me ● Patrick Druley ● Senior Systems Engineer @ Confluent (1yr) ● Formerly at Oracle and Teradata ● Ohioan, currently in Medina Ohio ● Github: https://github.com/pdruley ● Twitter: @PatrickLovesAK
- 4. 4
- 5. 5 A Modern, Distributed Platform for Data Streams
- 6. 6 Apache Kafka® is made up of distributed, immutable, append-only commit logs
- 7. 7 Intelligent Clients and Naive Pipes Writers Kafka cluster Readers
- 8. 88 Scalability of a filesystem • hundreds of MB/s throughput • many TB per server • commodity hardware
- 9. 99 Guarantees of a Database • Strict ordering • Persistence
- 10. 1010 Rewind & Replay Reset to any point in the shared narrative
- 11. 1111 Distributed by design • Replication • Fault Tolerance • Partitioning • Elastic Scaling
- 13. 1313 Partition Leadership and Replication Broker 1 Topic1 partition1 Broker 2 Broker 3 Broker 4 Topic1 partition1 Topic1 partition1 Leader Follower Topic1 partition2 Topic1 partition2 Topic1 partition2 Topic1 partition3 Topic1 partition4 Topic1 partition3 Topic1 partition3 Topic1 partition4 Topic1 partition4
- 14. 1414 Partition Leadership and Replication - node failure Broker 1 Topic1 partition1 Broker 2 Broker 3 Broker 4 Topic1 partition1 Topic1 partition1 Leader Follower Topic1 partition2 Topic1 partition2 Topic1 partition2 Topic1 partition3 Topic1 partition4 Topic1 partition3 Topic1 partition3 Topic1 partition4 Topic1 partition4
- 16. 1616 Resources Free E-Books from Confluent! I Heart Logs: https://www.confluent.io/ebook/i-heart-logs-event-data-stream-processing-and-data-integration/ Kafka: The Definitive Guide: https://www.confluent.io/resources/kafka-the-definitive-guide/ Confluent Blog: https://www.confluent.io/blog Confluent Podcast: https://podcasts.apple.com/us/podcast/streaming-audio-a-confluent-podcast-about-apache-kafka/id1 401509765?mt=2 Thank You! patrick@confluent.io @PatrickLovesAK
- 17. 1717 Thank You!