Confluent Partner Tech Talk with BearingPoint
0 likes•142 views
This document discusses best practices for debugging client applications in Kafka streams. It begins by asking a question about debugging practices for producers, consumers, and Kafka streams applications. It then describes a Partner Technical Sales Enablement offering that includes live sessions and on-demand learning paths on topics like Confluent fundamentals and use cases. It outlines additional support for partners through technical workshops, coaching, and solution discovery sessions. The document concludes by stating the goal of Partner Tech Talks is to provide insights and inspiration through use case discussions.
Confluent Partner Tech Talk with BearingPoint
- 1. @yourtwitterhandle | developer.confluent.io What are the best practices to debug client applications (producers/consumers in general but also Kafka Streams applications)? Starting soon…
- 2. @yourtwitterhandle | developer.confluent.io What are the best practices to debug client applications (producers/consumers in general but also Kafka Streams applications)?
- 3. Our Partner Technical Sales Enablement offering Scheduled sessions On-demand Join us for these live sessions where our experts will guide you through sessions of different level and will be available to answer your questions. Some examples of sessions are below: ● Confluent 101: for new starters ● Workshops ● Path to production series Learn the basics with a guided experience, at your own pace with our learning paths on-demand. You will also find an always growing repository of more advanced presentations to dig-deeper. Some examples are below: ● Confluent 10 ● Confluent Use Cases ● Positioning Confluent Value ● Confluent Cloud Networking ● … and many more AskTheExpert / Workshops For selected partners, we’ll offer additional support to: ● Technical Sales workshop ● JIT coaching on spotlight opportunity ● Build CoE inside partners by getting people with similar interest together ● Solution discovery ● Tech Talk ● Q&A
- 4. @yourtwitterhandle | developer.confluent.io What are the best practices to debug client applications (producers/consumers in general but also Kafka Streams applications)?
- 5. @yourtwitterhandle | developer.confluent.io What are the best practices to debug client applications (producers/consumers in general but also Kafka Streams applications)?
- 6. Goal Partners Tech Talks are webinars where subject matter experts from a Partner talk about a specific use case or project. The goal of Tech Talks is to provide best practices and applications insights, along with inspiration, and help you stay up to date about innovations in confluent ecosystem.
- 7. Confluent Perspective on Data Mesh Concepts 7
- 8. What kind of Data Mesh do you want? Physical vs. Virtual 8 Physical data movement (data moves to query) Fully decentralized→Better autonomy Data Virtualization Layer Orders Shipments Customer s Virtualized data (query goes to data) Centralized query→Easier to implement low-diversity use cases Data Engineer App Developer Data Scientist Business Analyst
- 9. What kind of Data Mesh do you want? Both? 9 Physical data movement (data moves to query) Fully decentralized→Better autonomy Data Virtualization Layer Orders Shipments Customer s Virtualized data (query goes to data) Centralized query→Easier to implement low-diversity use cases Data Engineer App Developer Data Scientist Business Analyst
- 10. Easily build real-time data pipelines to your data warehouse, database, and data lake Customer’s cloud env Data stores (I.e. PostgreSQL, MongoDB Atlas, MySQL, DB2, MSSQL, Oracle DB) Application data (i.e. Salesforce, ServiceNow, Github, SAP, Zendesk) Log data & messaging systems (i.e. MQTT, Azure Service Bus, Azure Event Hubs, Tibco, Solace) Amazon Redshift Source connectors Optional: SMT ksqlDB Sink connectors Optional: SMT Confluent Cloud Kafka topics Data Warehouses Snowflake Google BigQuery Azure Synapse Analytics MongoDB Atlas Amazon DynamoDB Azure Cosmos DB Google BigTable Databases Databricks Delta Lake Amazon S3 Google Cloud Storage Azure Blob Storage Data Lakes / / ● 200+ connectors ● infinite storage ● unlimited replaying of events ● hybrid & multi-cloud ● real-time analysis using ksqlDB ● complete ● everywhere ● cloud-native
- 11. Data ownership by domain Data as a product Data governed wherever it is Data available everywhere, self serve 1 2 3 4 The Principles of a Data Mesh
- 12. Pillars of a Streaming Data Mesh (by Confluent) Data Ownership - Have a Confluent cluster dedicated to a domain - Clusters could be sized to appropriate scale - No monolithic Kafka cluster Data as a Product - Cleanses, secures, governs data with ksqlDB - Query data with ksqlDB. - Data optimized for reading. - ACLs & RBAC - infinite/tiered storage Self-serve data platform - Automated deployment and access process with REST endpoints, ksqlDB and Confluent CLI. - Cluster linking Federated computational governance - Confluent governance tools - Confluent security RBAC/ACLs
- 13. @yourtwitterhandle | developer.confluent.io What are the best practices to debug client applications (producers/consumers in general but also Kafka Streams applications)?
- 14. Speaker Michael Stockhammer Lead Data Mesh & Data Platform Gerald Tretter Lead center of excellence real time enterprise & event streaming
- 15. BearingPoint – Who we are
- 16. 5,253 employees 50 countries in which BearingPoint carries out projects 41 offices in 23 countries BearingPoint has a global reach.
- 17. We don't just offer you consulting services, we also work with you to keep an eye on your future. Capital & JVs Joint Ventures • Joint Venture with IFS (Arcwide) • Joint Venture with Six Products IP Products & Services • HR People Development Cloud • Compliance Services FOSS & SAM • HyperCube / Nitro / Optix • Emissions Calculator (LogEC) • ETM.next / DemandSens • Coding Platform / Application Services / Security Services • Salesforce / SAP / Azure Consulting Market Segments • Automotive, Industrial Equipment and Manufacturing • Banking & Capital Markets • Chemicals, Life Sciences & Resources • Communications, Media & Entertainment • Consumer Goods & Retail • Government & Public Sector • Insurance • Utilities, Postal & Transportation People & Strategy Custome r & Growth Financ e & Risk Operation s Technolog y Capital • M&A Advisory • Investments & Ventures • Standalone software portfolio
- 18. 18 Our Event Streaming ecosystem: Experts in all roles from Architects and Business Analysts to Developers and Operations Center of Excellence – independent event streaming consulting Overview of our Real-Time Event Streaming practice @ BearingPoint Our Event Streaming competencies 50+ Event Streaming professionals 10+ Architects working with Event Streaming 15+ Business Analysts working with Event Streaming 6+ years experience in building Event Streaming applications Technology Stack Industry-specific expertise in • Transport and Logistics • Postal • Utilities • Telecommunications • Banking • Insurance Partnerships, awards and contributions • Consulting and SI Partner for Confluent since 2018 (Premier Partner since 2020) • Contribution to Kafka Meetups in Vienna and Graz • Digital Skipper Assistant was finalist at the Digital Business Trends Award 2018 • Several contributions to conferences about event streaming • Contribution to Open Source tools in the event streaming ecosystem Flagship Projects Total Recall Total Recall is an event streaming system for log and metric data. Kafka is used to connect and process the event streams from different sources and persist the results in elastic search. Digital Skipper Assistant (DSA) Digital assistant for skippers that includes waterway maps, routing functionality, ETA calculation, display of current water levels as well as water level predictions. Real-Time Transport Monitoring Dynamic evaluation of progress of a transport along defined goals with the help of generated events (e.g. ETA, geofencing) to signal the next process steps. Transformation of Compliance Organization in a Bank Real-time notification of breaches in compliance checks. Increasing efficiency by reduction of organizational barriers, workload, data storage & applications. Developing a Far-sighted and future proof IT-Architecture which improves time2market and responsiveness to new requirements. Experience Real-Time Transport Monitoring Digital Skipper Assistant (DSA) Registered Consulting and SI Partner for Confluent Confluent Premium Partner Enable ERP host system decomposition (by data streams) 2020 2021 2016 2019 2017 2018 2022 Compliance Organization Real-Time Enterprise Service Portfolio
- 19. 19 The transformation to a real-time enterprise can only be achieved through a paradigm shift in our maturity dimensions REACTIVE Decisions are made on the basis of outdated information Past PROACTIVE Decisions can be made proactively at the right time Now 1. Technology becomes an 'enabler' As the technical core, event streaming becomes the central nervous system for an event-driven organization 2. Processes go digital Through automatic and proactive decisions, processes are redesigned and established 3. Data becomes a product Events are part of domain-oriented data products and create added value in various areas of the company Events become new insights in context Events provide information at the right time Data become a trigger as an event 4. Competencies enable transformation Comprehensive design leverages hidden potential of corporate events and the associated information gain 5. People & Culture New roles, tasks and acceptance form the basis for the transformation to a real-time enterprise Strategy & Organization defines the direction of the transformation Scaling of event thinking in the organization by anchoring the transformation in the strategy Realtime Enterprise Data People & Culture Strategy & Organization Competence Processes Technology
- 20. Pillars of Data Mesh (by Starburst and Confluent) Data Ownership - Role management in self service platform - Data is assigned to individual Domains Data as a Product - Data Product is assigned to individual domains - Data is accessed via SQL no matter which source Self-serve data platform - Platform as enabler for business to access data easily Federated computational governance - Work together beyond silos
- 21. WHY KYC so slow? Real-Time KYC & Compl. checks Compliance Data Products Event triggered recalculation • Know Your Customer processes are to slow • Data is not available in the right way What‘s the common denominator for business?
- 23. Show case
- 24. Call to Connect Contact us if > we love to collaborate Technical and Organisational Data Mesh Architecture consulting Software development
- 25. Call to Connect Contact us if > we love to collaborate
- 26. Q&A