How to use Standard SQL over Kafka: From the basics to advanced use cases | Frank McSherry, Materialize
0 likes421 views
The document discusses the integration of SQL with Kafka for streaming data, covering basic to advanced use cases. It outlines the historical context of SQL systems and how they can handle unbounded data, event-driven queries, and produce outputs. Additionally, it highlights features like focusing on recent events and the combination of streams and SQL to innovate data handling processes.
How to use Standard SQL over Kafka: From the basics to advanced use cases | Frank McSherry, Materialize
- 1. Frank McSherry, Chief Scientist How to use Standard SQL over Kafka From the basics to advanced use cases
- 2. You can write SQL and make streaming infrastructure
- 3. A brief history of SQL systems (spoiler: it ends up with streams)
- 7. OLTP OLAP Reads Writes Transactions Analytics Dashboards Monitoring (row-based) (columnar) different designs
- 9. OLTP OLVM Reads Writes Transactions Analytics Dashboards Monitoring (pull) (push) different designs
- 14. Stream connection 1/3: SQL systems can have streaming internals
- 16. Stream connection 2/3: SQL systems can consume streams as inputs
- 18. Stream connection 3/3: SQL systems can produce streams as outputs
- 21. Bonus Features 1/3: SQL over unbounded data
- 22. This could be unbounded This is all we need to store
- 23. Bonus Features 2/3: Focus on recent events.
- 24. Like now() but for "event time"
- 25. Bonus Features 3/3: Event-driven SQL queries
- 27. For each query in queries ... ... restrict by the query argument ... ... keep the top three regions.
- 28. You can write SQL and make streaming infrastructure
- 29. Streams and SQL combine as inputs flow to systems flow to outputs Streams and SQL lead to new idioms working with unbounded data, focusing on recent events, event-driven SQL queries, ... and more. (seriously)
- 30. https://materialize.com/kafka Head to for free shirts, cloud sign-up, and a guide to Kafka. - @frankmcsherry "best DB shirt released in 2020 (great design, soft material)" @andy_pavlo