Scribd
Upload
101 views

Apache Kafka-Flink Syllabus

The document outlines a 3-day Apache Kafka-Flink syllabus covering Kafka and Flink concepts and APIs. Day 1 introduces Kafka, including its architecture, components, and operations. Attendees will install and configure Kafka clusters. Day 2 discusses Kafka consumers and offsets, then covers Flink's streaming architecture and programming model. Attendees will write Flink programs. Day 3 continues exploring Flink transformations and operations, discussing sinks and joining streams. Hands-on labs reinforce the concepts through problem solving and integrating Kafka and Flink. The syllabus aims to provide attendees with skills in real-time streaming analytics using Kafka and Flink.

Uploaded by

arya awasthi
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
101 views

Apache Kafka-Flink Syllabus

The document outlines a 3-day Apache Kafka-Flink syllabus covering Kafka and Flink concepts and APIs. Day 1 introduces Kafka, including its architecture, components, and operations. Attendees will install and configure Kafka clusters. Day 2 discusses Kafka consumers and offsets, then covers Flink's streaming architecture and programming model. Attendees will write Flink programs. Day 3 continues exploring Flink transformations and operations, discussing sinks and joining streams. Hands-on labs reinforce the concepts through problem solving and integrating Kafka and Flink. The syllabus aims to provide attendees with skills in real-time streaming analytics using Kafka and Flink.

Uploaded by

arya awasthi
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 2

Apache Kafka-Flink Syllabus

Day 1
Introduction to Apache Kafka

• What is Kafka?
• Need for Kafka
• Real time Analytics with Kafka
• Core Concepts of Kafka
• Kafka Architecture
• Kafka Usecases

Kafka Cluster

• Understanding Kafka Cluster


• Installing and Configuring Kafka Cluster
• Kafka Producer
• Kafka Consumer
• Producer and Consumer in Action
• Reading Data from Kafka
• Message Delivery Semantics
• Replication
• Log Compaction

Lab:
Kafka Installation and Configuration
Single Node and MultiNode Setup
Configuring Zookeeper

Kafka Core Components and operations

• Type of Messaging System


• Kafka Data Model
• Topics
• Partitions
• Partition Distribution
• Producer
• Consumer
• Kafka Reliability
• Replication in Kafka
• Persistence in Kafka
• High availability and reliability using Replication
• ISR - In Sync Replication
• Partition reassignment - Increasing Replication or partitions
• Creating a topic
• Modifying a Topic
• Creating a Message
• Reading a Message
• Java Interface to Kafka
• Producer Side API
• Consumer Side API

Day 2
Discussion:
1. Establish the contrast when using kafka consumer vs third party consumer such as flink.
2. Emphasize the role of Offset in Kafka.

Lab:
• Implement Kafka Producer, Consumer using real time streaming data.
• Working with Kafka API to implement Kafka producer , consumer , consumer group and Kafka Admin APIs to work with kafka
topics and fetch zookeeper metadata
Flink's Stream Processing Architecture
What is stream processing?
Stream processing vs Batch processing
Flink Architecture walkthrough
Stream processing with Apache Flink

Working with Flink


Installing Flink
Setting up your Flink project with Maven
Data Representation and Programming Model
Example 1: Writing a Flink program discussion

Lab:
Flink Instalation and Configuration
Setting up your Flink project with Maven
Writing a Flink program

Flink Transofrmations and Operations


The Filter operation;
The Map operation;
The Flat Map operation;
Stateless and Stateful Transformations;
Creating a stream of Tuples;
The Reduce Operation;

Lab:
Hands on problem solving using Flink API and transformations listed above

Day 3
Flink Transofrmations and Operations Continue..

Windows Transformation ;
Count Windows;
State and Check pointing;
Checkpointing strategies;
Operations on Multiple Streams ;
Unions;
Joining Streams;
coGroup;
Iterate
Discussion:
3. Flink sink operations : map vs flatmap
4. Flink sink as Oracle/RDBMS and best practices.

Lab:
Hands on problem solving using Flink API and transformations listed above
Configuration and Integration of Flink with Kafka

Project:
Real time streaming analytics using Kafka and Flink

You might also like