Big data-hadoop-training-course-content-content
0 likes22 views
Best Big Data Hadoop Training in Chennai at Credo Systemz will help you learn and upgrade your knowledge in the Core components, Database concepts and Linux Operating system.
![Big Data Hadoop Course Content
Chapter 1: Introduction to Big Data-hadoop
➢ Overview of Hadoop Ecosystem
➢ Role of Hadoop in Big Data– Overview of other Big DataSystems
➢ Who is using Hadoop
➢ Hadoop integrations into Exiting Software Products
➢ Current Scenario in Hadoop Ecosystem
➢ Installation
➢ Configuration
➢ Use Cases of Hadoop (HealthCare, Retail,Telecom)
Chapter 2 : HDFS
➢ Concepts
➢ Architecture
➢ Data Flow (File Read , FileWrite)
➢ Fault Tolerance
➢ Shell Commands
➢ Data Flow Archives
➢ Coherency -Data Integrity
➢ Role of Secondary Name Node
Chapter 3 : Mapreduce
➢ Theory
➢ Data Flow (Map – Shuffle –Reduce)
➢ MapRed vs MapReduce APIs
➢ Programming [Mapper, Reducer, Combiner, Partitioner]
➢ Writables
➢ Input Format
➢ Output format
➢ Streaming API using python
➢ Inherent Failure Handling using Speculative Execution
➢ Magic of Shuffle Phase
➢ File Formats](https://image.slidesharecdn.com/big-data-hadoop-training-course-content-content-201211134200/85/Big-data-hadoop-training-course-content-content-3-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to Big data-hadoop-training-course-content-content (20)
Ad
More from Training Institute (10)
Ad
Recently uploaded (20)
Big data-hadoop-training-course-content-content
- 1. REAL TIME PROJECT: Click Stream Data Analytics Report Project ClickStream Data ClickStream data could be generated from any activity performed by the user over a web application. What could be the user activity over any website? For example, I am logging into Amazon, what are the activities I could perform? In a pattern, I may navigate through some pages; spend some time over certain pages and click on certain things. All these activities, including reaching that particular page or application, clicking, navigating from one page to another and spending time make a set of data. All these will be logged by a web application. This data is known as ClickStream Data. It has a high business value, specific to e- commerce applications and for those who want to understand their users’ behavior. More formally, ClickStream can be defined as data about the links that a user clicked, including the point of time when each one of them were clicked. E-commerce businesses mine and analyse ClickStream data on their own websites. Most of the E-commerce applications have their built-in system, which mines all this information. ClickStream Analytics Using the ClickStream data adds a lot of value to businesses, through which they can bring many customers or visitors. It helps them understand whether the application is right, and the application experience of users is good or bad, based on the navigation patterns that people take. They can also predict which page you are most likely to visit next and can-do Ad Targeting as well. With this, they can understand the needs of users and come up with better recommendations. Several other things are possible using the ClickStream Data. Project Scope In this project candidates are given with sample click stream data which is taken from a web application in a text file along with problem statements. ➢ Users information in MySQL database. ➢ Click stream data in text file generated from Web application.
- 2. Each candidate has to come up with high level system architecture design based upon the Hadoop eco systems covered during the course. Each candidate has to table the High-level system architecture along with designed eco systems and pros and cons will be discussed with all the other candidates. Finally, will choose the best possible optimal system design approach for implementation. Candidates are given instructions to create an oozie work flow with the respective Hadoop Eco systems finalized based on the discussion. Candidates has to submit the project for the given problem statement and this will be validated by the trainer individually before course completion. ECO System involved in click stream analytics Project ➢ HDFS ➢ Sqoop ➢ Pig ➢ Hive ➢ Oozie
- 3. Big Data Hadoop Course Content Chapter 1: Introduction to Big Data-hadoop ➢ Overview of Hadoop Ecosystem ➢ Role of Hadoop in Big Data– Overview of other Big DataSystems ➢ Who is using Hadoop ➢ Hadoop integrations into Exiting Software Products ➢ Current Scenario in Hadoop Ecosystem ➢ Installation ➢ Configuration ➢ Use Cases of Hadoop (HealthCare, Retail,Telecom) Chapter 2 : HDFS ➢ Concepts ➢ Architecture ➢ Data Flow (File Read , FileWrite) ➢ Fault Tolerance ➢ Shell Commands ➢ Data Flow Archives ➢ Coherency -Data Integrity ➢ Role of Secondary Name Node Chapter 3 : Mapreduce ➢ Theory ➢ Data Flow (Map – Shuffle –Reduce) ➢ MapRed vs MapReduce APIs ➢ Programming [Mapper, Reducer, Combiner, Partitioner] ➢ Writables ➢ Input Format ➢ Output format ➢ Streaming API using python ➢ Inherent Failure Handling using Speculative Execution ➢ Magic of Shuffle Phase ➢ File Formats
- 4. ➢ Sequence Files Chapter 4: Hbase ➢ Introduction to NoSQL ➢ CAP Theorem ➢ Classification of NoSQL ➢ Hbase and RDBMS ➢ HBASE and HDFS ➢ Architecture (Read Path, Write Path, Compactions,Splits) ➢ Installation ➢ Configuration ➢ Role of Zookeeper ➢ HBase Shell Introduction to Filters ➢ Row Key Design -What’s New in HBase HandsOn Chapter 5 : Hive ➢ Architecture ➢ Installation ➢ Configuration ➢ Hive vs RDBMS ➢ Tables ➢ DDL ➢ DML ➢ UDF ➢ Partitioning ➢ Bucketing ➢ Hive functions ➢ Date functions ➢ String functions ➢ Cast function Meta Store ➢ Joins ➢ Real-time HQL will be shared along with database migrationproject Chapter 6 : pig ➢ Architecture ➢ Installation ➢ Hive vs Pig ➢ Pig Latin Syntax ➢ Data Types ➢ Functions (Eval, Load/Store, String, Date Time) ➢ Joins ➢ UDFs- Performance ➢ Troubleshooting
- 5. ➢ Commonly Used Functions Chapter 7 : sqoop ➢ Architecture , Installation, Commands(Import , Hive-Import, EVal, Hbase Import, Import All tables, Export) ➢ Connectors to Existing DBs and DW Practicals ➢ SQOOP to import Real Time Weblogs from application to DB and try to export the sameto MySQL Chapter 8 : kafka ➢ Kafka introduction ➢ Data streaming Introduction ➢ Producer-consumer-topics ➢ Brokers ➢ Partitions ➢ Unix Streaming via kafka Practicals Kafka ➢ Producer and Subscribers setup and publish a topic from Producer to subscriber Chapter 9 : oozie ➢ Architecture ➢ Installation ➢ Workflow ➢ Coordinator ➢ Action (Map reduce, Hive, Pig,Sqoop) ➢ Introduction to Bundle ➢ Mail Notifications Chapter 10: Hadoop 2.0 and spark ➢ Limitations in Hadoop ➢ –HDFS Federation ➢ High Availability in HDFS ➢ HDFS Snapshots ➢ Other Improvements inHDFS2 ➢ Introduction to YARN akaMR2 ➢ Limitations in MR1 ➢ Architecture of YARN
- 6. ➢ Map Reduce Job Flow inYARN ➢ Introduction to Stinger Initiative andTez ➢ Back Ward Compatibility for Hadoop1.X ➢ Spark Fundamentals ➢ RDD- Sample Scala Program- SparkStreaming Practicals ➢ Difference between SPARK1.x and SPARK2.x ➢ PySpark program to create word count program in pyspark Chapter 11: Big Data Use cases ➢ Hadoop ➢ HDFS architecture and usage ➢ MapReduce Architecture and real time exercises ➢ Hadoop Eco systems ➢ Sqoop - mysql Db Migration ➢ Hive. -- Deep drive ➢ Pig - weblog parsing and ETL ➢ Oozie - Workflow scheduling ➢ Flume - weblogs ingestion ➢ No SQL ➢ HBase ➢ Apache Kafka ➢ Pentaho ETL tool integration & working with Hadoop eco system ➢ Apache SPARK ➢ Introduction and working with RDD. ➢ Multi node Setup Guidance ➢ Hadoop latest version Pros & cons discussion ➢ Ends with Introduction of Data science. Chapter 12: Real Time Project ➢ Getting applications web logs ➢ Getting user information from my sql via sqoop ➢ Getting extracted data from Pig script ➢ Creating Hive SQL Table for querying ➢ Creating Reports from Hive QL