Hadoop hive presentation

Apr 21, 2013Download as PPT, PDF3 likes5,661 views

Hadoop is an open-source framework for distributed storage and processing of large datasets across clusters of commodity hardware. It addresses problems with traditional systems like data growth, network/server failures, and high costs by allowing data to be stored in a distributed manner and processed in parallel. Hadoop has two main components - the Hadoop Distributed File System (HDFS) which provides high-throughput access to application data across servers, and the MapReduce programming model which processes large amounts of data in parallel by splitting work into map and reduce tasks.

Agenda
• Problems with traditional large-scale systems
• Requirements for new approaches
• What is Hadoop..?
• Why Hadoop?
• Overview of Hadoop
• HDFS
• Map Reduce
• Applications
• Conclusion

Problems with traditional large-scale systems

Data is being increased day-by-day

Issues with the network failure

Server failure

Loss of data

Cost is more.

Distributed computing need manual processing

Requirements for new approaches

Data should be stored in a distributed manner
and parallel processing.

High performance and less cost.

Should be scalable

Should be simple to access and process

Fault tolerance

What is Hadoop…?

Open Source Framework

Process large amount of data

Why Hadoop…?
• Accessible
• Scalable
• Robust
• Simple

Overview of Hadoop

It handles 3 types of data
Structured
Semi – structured
Unstructured

Analyses and process large amounts of data (Peta byte)

Compare with traditional DB’s
RDBMS
• Stores GB’s of data
• Supports batch process
and interactive process
• Allows Updation
• Schemas must me defined
• Only structured data
HADOOP
• Stores PB’s of data
• Only batch process
• Does not allow Updation, it
follows WORM
• Schemas not required
• Supports 3 types of data

Components

Hadoop can be divided into 2 parts
1. HDFS – Hadoop Distributed File System
2. MapReduce Programming model

Hadoop Distributed File System

It is a distributed file system

Runs on commodity hardware

Provides high throughput access to application data

suitable for applications that have large data sets.

It is designed to store a very large amount of data (Tera or peta
bytes).

Core Architectural Goal of HDFS

A HDFS instance may consist of thousands of server machines.

Detection of faults and quickly recovering from them in an
automated manner

MapReduce Programming Model

MapReduce works on divide and conquer rule on the data.

Schedules execution across a set of machines

Manages inter-process communication

The Reducer processes all output from all mappers and arrives
at final output

MapReduce Programming Model
– MAP
• Map() function that processes a key/value pair to
generate a set of intermediate key/value pairs
– REDUCE
• reduce() function that merges all intermediate values
associated with the same intermediate key.

REFERENCE
• HADOOP IN ACTION
- By CHUK LAM
• YOUTUBE
• WIKEPEDIA
• GOOGLE IMAGES

This document provides a summary of improvements made to Hive's performance through the use of Apache Tez and other optimizations. Some key points include: - Hive was improved to use Apache Tez as its execution engine instead of MapReduce, reducing latency for interactive queries and improving throughput for batch queries. - Statistics collection was optimized to gather column-level statistics from ORC file footers, speeding up statistics gathering. - The cost-based optimizer Optiq was added to Hive, allowing it to choose better execution plans. - Vectorized query processing, broadcast joins, dynamic partitioning, and other optimizations improved individual query performance by over 100x in some cases.

Apache hivepradipbajpai68

Apache Tez: Accelerating Hadoop Query Processing DataWorks Summit

Tez is the next generation Hadoop Query Processing framework written on top of YARN. Computation topologies in higher level languages like Pig/Hive can be naturally expressed in the new graph dataflow model exposed by Tez. Multi-stage queries can be expressed as a single Tez job resulting in lower latency for short queries and improved throughput for large scale queries. MapReduce has been the workhorse for Hadoop but its monolithic structure had made innovation slower. YARN separates resource management from application logic and thus enables the creation of Tez, a more flexible and generic new framework for data processing for the benefit of the entire Hadoop query ecosystem.

Apache Tez - A New Chapter in Hadoop Data ProcessingDataWorks Summit

Apache Tez is a framework for accelerating Hadoop query processing. It is based on expressing a computation as a dataflow graph and executing it in a highly customizable way. Tez is built on top of YARN and provides benefits like better performance, predictability, and utilization of cluster resources compared to traditional MapReduce. It allows applications to focus on business logic rather than Hadoop internals.

Apache hive introductionMahmood Reza Esmaili Zand

Apache Hive is a data warehouse software built on top of Hadoop that allows users to query data stored in various databases and file systems using an SQL-like interface. It provides a way to summarize, query, and analyze large datasets stored in Hadoop distributed file system (HDFS). Hive gives SQL capabilities to analyze data without needing MapReduce programming. Users can build a data warehouse by creating Hive tables, loading data files into HDFS, and then querying and analyzing the data using HiveQL, which Hive then converts into MapReduce jobs.

Hadoop Hive Tutorial | Hive Fundamentals | Hive ArchitectureSkillspeed

This Hadoop Hive Tutorial will unravel the complete Introduction to Hive, Hive Architecture, Hive Commands, Hive Fundamentals & HiveQL. In addition to this, even fundamental concepts of BIG Data & Hadoop are extensively covered. At the end, you'll have a strong knowledge regarding Hadoop Hive Basics. PPT Agenda ✓ Introduction to BIG Data & Hadoop ✓ What is Hive? ✓ Hive Data Flows ✓ Hive Programming ---------- What is Apache Hive? Apache Hive is a data warehousing infrastructure built over Hadoop which is targeted towards SQL programmers. Hive permits SQL programmers to directly enter the Hadoop ecosystem without any pre-requisites in Java or other programming languages. HiveQL is similar to SQL, it is utilized to process Hadoop & MapReduce operations by managing & querying data. ---------- Hive has the following 5 Components: 1. Driver 2. Compiler 3. Shell 4. Metastore 5. Execution Engine ---------- Applications of Hive 1. Data Mining 2. Document Indexing 3. Business Intelligence 4. Predictive Modelling 5. Hypothesis Testing ---------- Skillspeed is a live e-learning company focusing on high-technology courses. We provide live instructor led training in BIG Data & Hadoop featuring Realtime Projects, 24/7 Lifetime Support & 100% Placement Assistance. Email: [email protected] Website: https://www.skillspeed.com

Introduction to Apache Hive(Big Data, Final Seminar)Takrim Ul Islam Laskar

This document provides an introduction and overview of Apache Hive. It discusses how Hive originated at Facebook to manage large amounts of data stored in Oracle databases. It then defines what Hive is, how it works by compiling SQL queries into MapReduce jobs, and its architecture. Key components of Hive like its data model, metastore, and commands for creating tables and loading data are summarized.

Apache Spark ArchitectureAlexey Grishchenko

YARN Ready: Integrating to YARN with Tez Hortonworks

Hive: Loading DataBenjamin Leonhardi

Introduction to Storm Chandler Huang

Storm is a distributed and fault-tolerant realtime computation system. It was created at BackType/Twitter to analyze tweets, links, and users on Twitter in realtime. Storm provides scalability, reliability, and ease of programming. It uses components like Zookeeper, ØMQ, and Thrift. A Storm topology defines the flow of data between spouts that read data and bolts that process data. Storm guarantees processing of all data through its reliability APIs and guarantees no data loss even during failures.

Introduction to Apache SparkRahul Jain

Hadoop Architecture | HDFS Architecture | Hadoop Architecture Tutorial | HDFS...Simplilearn

This presentation about Hadoop architecture will help you understand the architecture of Apache Hadoop in detail. In this video, you will learn what is Hadoop, components of Hadoop, what is HDFS, HDFS architecture, Hadoop MapReduce, Hadoop MapReduce example, Hadoop YARN and finally, a demo on MapReduce. Apache Hadoop offers a versatile, adaptable and reliable distributed computing big data framework for a group of systems with capacity limit and local computing power. After watching this video, you will also understand the Hadoop Distributed File System and its features along with the practical implementation. Below are the topics covered in this Hadoop Architecture presentation: 1. What is Hadoop? 2. Components of Hadoop 3. What is HDFS? 4. HDFS Architecture 5. Hadoop MapReduce 6. Hadoop MapReduce Example 7. Hadoop YARN 8. Demo on MapReduce What are the course objectives? This course will enable you to: 1. Understand the different components of Hadoop ecosystem such as Hadoop 2.7, Yarn, MapReduce, Pig, Hive, Impala, HBase, Sqoop, Flume, and Apache Spark 2. Understand Hadoop Distributed File System (HDFS) and YARN as well as their architecture, and learn how to work with them for storage and resource management 3. Understand MapReduce and its characteristics, and assimilate some advanced MapReduce concepts 4. Get an overview of Sqoop and Flume and describe how to ingest data using them 5. Create database and tables in Hive and Impala, understand HBase, and use Hive and Impala for partitioning 6. Understand different types of file formats, Avro Schema, using Arvo with Hive, and Sqoop and Schema evolution 7. Understand Flume, Flume architecture, sources, flume sinks, channels, and flume configurations 8. Understand HBase, its architecture, data storage, and working with HBase. You will also understand the difference between HBase and RDBMS 9. Gain a working knowledge of Pig and its components 10. Do functional programming in Spark 11. Understand resilient distribution datasets (RDD) in detail 12. Implement and build Spark applications 13. Gain an in-depth understanding of parallel processing in Spark and Spark RDD optimization techniques 14. Understand the common use-cases of Spark and the various interactive algorithms 15. Learn Spark SQL, creating, transforming, and querying Data frames Who should take up this Big Data and Hadoop Certification Training Course? Big Data career opportunities are on the rise, and Hadoop is quickly becoming a must-know technology for the following professionals: 1. Software Developers and Architects 2. Analytics Professionals 3. Senior IT professionals 4. Testing and Mainframe professionals 5. Data Management Professionals 6. Business Intelligence Professionals 7. Project Managers 8. Aspiring Data Scientists Learn more at https://www.simplilearn.com/big-data-and-analytics/big-data-and-hadoop-training

HBase.pptxSadhik7

HBase is a distributed column-oriented database built on top of Hadoop that provides quick random access to large amounts of structured data. It leverages the fault tolerance of HDFS and allows for real-time read/write access to data stored in HDFS. HBase sits above HDFS and provides APIs for reading and writing data randomly. It is a scalable, schema-less database modeled after Google's Bigtable.

Apache Hive TutorialSandeep Patil

Hadoop Overview & Architecture EMC

This document provides an overview of Hadoop architecture. It discusses how Hadoop uses MapReduce and HDFS to process and store large datasets reliably across commodity hardware. MapReduce allows distributed processing of data through mapping and reducing functions. HDFS provides a distributed file system that stores data reliably in blocks across nodes. The document outlines components like the NameNode, DataNodes and how Hadoop handles failures transparently at scale.

What is new in Apache Hive 3.0?DataWorks Summit

Apache Hive is a rapidly evolving project which continues to enjoy great adoption in the big data ecosystem. As Hive continues to grow its support for analytics, reporting, and interactive query, the community is hard at work in improving it along with many different dimensions and use cases. This talk will provide an overview of the latest and greatest features and optimizations which have landed in the project over the last year. Materialized views, the extension of ACID semantics to non-ORC data, and workload management are some noteworthy new features. We will discuss optimizations which provide major performance gains, including significantly improved performance for ACID tables. The talk will also provide a glimpse of what is expected to come in the near future.

Apache Spark - Basics of RDD | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab

Big Data with Hadoop & Spark Training: http://bit.ly/2L4rPmM This CloudxLab Basics of RDD tutorial helps you to understand Basics of RDD in detail. Below are the topics covered in this tutorial: 1) What is RDD - Resilient Distributed Datasets 2) Creating RDD in Scala 3) RDD Operations - Transformations & Actions 4) RDD Transformations - map() & filter() 5) RDD Actions - take() & saveAsTextFile() 6) Lazy Evaluation & Instant Evaluation 7) Lineage Graph 8) flatMap and Union 9) Scala Transformations - Union 10) Scala Actions - saveAsTextFile(), collect(), take() and count() 11) More Actions - reduce() 12) Can We Use reduce() for Computing Average? 13) Solving Problems with Spark 14) Compute Average and Standard Deviation with Spark 15) Pick Random Samples From a Dataset using Spark

Memory Management in Apache SparkDatabricks

Memory management is at the heart of any data-intensive system. Spark, in particular, must arbitrate memory allocation between two main use cases: buffering intermediate data for processing (execution) and caching user data (storage). This talk will take a deep dive through the memory management designs adopted in Spark since its inception and discuss their performance and usability implications for the end user.

Spark architectureGauravBiswas9

The document discusses Apache Spark, an open source cluster computing framework for real-time data processing. It notes that Spark is up to 100 times faster than Hadoop for in-memory processing and 10 times faster on disk. The main feature of Spark is its in-memory cluster computing capability, which increases processing speeds. Spark runs on a driver-executor model and uses resilient distributed datasets and directed acyclic graphs to process data in parallel across a cluster.

Hive presentationHitesh Agrawal

This document provides an introduction to Apache Hive, including what Hive is, its key features and architecture. Hive is a data warehouse infrastructure tool used to process structured data in Hadoop. It allows users to query and analyze large datasets using SQL-like queries. Hive resides on top of Hadoop and uses MapReduce to process queries internally. It includes a metastore to store metadata, query compiler and execution engine to process queries, and can operate on data stored in HDFS or HBase.

Apache Spark Introductionsudhakara st

Spark is an open source cluster computing framework for large-scale data processing. It provides high-level APIs and runs on Hadoop clusters. Spark components include Spark Core for execution, Spark SQL for SQL queries, Spark Streaming for real-time data, and MLlib for machine learning. The core abstraction in Spark is the resilient distributed dataset (RDD), which allows data to be partitioned across nodes for parallel processing. A word count example demonstrates how to use transformations like flatMap and reduceByKey to count word frequencies from an input file in Spark.

Hive(ppt)Abhinav Tyagi

Hive is a data warehouse infrastructure tool that allows users to query and analyze large datasets stored in Hadoop. It uses a SQL-like language called HiveQL to process structured data stored in HDFS. Hive stores metadata about the schema in a database and processes data into HDFS. It provides a familiar interface for querying large datasets using SQL-like queries and scales easily to large datasets.

Big data Analytics HadoopMishika Bharadwaj

Programming in Spark using PySpark Mostafa

This session covers how to work with PySpark interface to develop Spark applications. From loading, ingesting, and applying transformation on the data. The session covers how to work with different data sources of data, apply transformation, python best practices in developing Spark Apps. The demo covers integrating Apache Spark apps, In memory processing capabilities, working with notebooks, and integrating analytics tools into Spark Applications.

Session 14 - HiveAnandMHadoop

Hadoop MapReduce FundamentalsLynn Langit

This document provides an overview of the Hadoop MapReduce Fundamentals course. It discusses what Hadoop is, why it is used, common business problems it can address, and companies that use Hadoop. It also outlines the core parts of Hadoop distributions and the Hadoop ecosystem. Additionally, it covers common MapReduce concepts like HDFS, the MapReduce programming model, and Hadoop distributions. The document includes several code examples and screenshots related to Hadoop and MapReduce.

Introduction to HiveQLkristinferrier

Big Data and Cloud ComputingFarzad Nozarian

This document discusses cloud and big data technologies. It provides an overview of Hadoop and its ecosystem, which includes components like HDFS, MapReduce, HBase, Zookeeper, Pig and Hive. It also describes how data is stored in HDFS and HBase, and how MapReduce can be used for parallel processing across large datasets. Finally, it gives examples of using MapReduce to implement algorithms for word counting, building inverted indexes and performing joins.

2. hadoop fundamentalsLokesh Ramaswamy

Hadoop is an open source framework for distributed storage and processing of large datasets across clusters of commodity hardware. It uses Google's MapReduce programming model and Google File System for reliability. The Hadoop architecture includes a distributed file system (HDFS) that stores data across clusters and a job scheduling and resource management framework (YARN) that allows distributed processing of large datasets in parallel. Key components include the NameNode, DataNodes, ResourceManager and NodeManagers. Hadoop provides reliability through replication of data blocks and automatic recovery from failures.

More Related Content

What's hot (20)

YARN Ready: Integrating to YARN with Tez Hortonworks

Hive: Loading DataBenjamin Leonhardi

Introduction to Storm Chandler Huang

Introduction to Apache SparkRahul Jain

Hadoop Architecture | HDFS Architecture | Hadoop Architecture Tutorial | HDFS...Simplilearn

HBase.pptxSadhik7

Apache Hive TutorialSandeep Patil

Hadoop Overview & Architecture EMC

What is new in Apache Hive 3.0?DataWorks Summit

Apache Spark - Basics of RDD | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab

Memory Management in Apache SparkDatabricks

Spark architectureGauravBiswas9

Hive presentationHitesh Agrawal

Apache Spark Introductionsudhakara st

Hive(ppt)Abhinav Tyagi

Big data Analytics HadoopMishika Bharadwaj

Programming in Spark using PySpark Mostafa

Session 14 - HiveAnandMHadoop

Hadoop MapReduce FundamentalsLynn Langit

Introduction to HiveQLkristinferrier

YARN Ready: Integrating to YARN with Tez Hortonworks

Hive: Loading DataBenjamin Leonhardi

Introduction to Storm Chandler Huang

Introduction to Apache SparkRahul Jain

Hadoop Architecture | HDFS Architecture | Hadoop Architecture Tutorial | HDFS...Simplilearn

HBase.pptxSadhik7

Apache Hive TutorialSandeep Patil

Hadoop Overview & Architecture EMC

What is new in Apache Hive 3.0?DataWorks Summit

Apache Spark - Basics of RDD | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab

Memory Management in Apache SparkDatabricks

Spark architectureGauravBiswas9

Hive presentationHitesh Agrawal

Apache Spark Introductionsudhakara st

Hive(ppt)Abhinav Tyagi

Big data Analytics HadoopMishika Bharadwaj

Programming in Spark using PySpark Mostafa

Session 14 - HiveAnandMHadoop

Hadoop MapReduce FundamentalsLynn Langit

Introduction to HiveQLkristinferrier

Similar to Hadoop hive presentation (20)

Big Data and Cloud ComputingFarzad Nozarian

2. hadoop fundamentalsLokesh Ramaswamy

AnjuAnju Shekhawat

The document provides an overview of Apache Hadoop and how it addresses challenges related to big data. It discusses how Hadoop uses HDFS to distribute and store large datasets across clusters of commodity servers and uses MapReduce as a programming model to process and analyze the data in parallel. The core components of Hadoop - HDFS for storage and MapReduce for processing - allow it to efficiently handle large volumes and varieties of data across distributed systems in a fault-tolerant manner. Major companies have adopted Hadoop to derive insights from their big data.

Hadoopchandinisanz

Hadoop is an open-source software framework for distributed storage and processing of large datasets across clusters of computers. It has four main modules - Hadoop Common, HDFS, YARN and MapReduce. HDFS provides a distributed file system that stores data reliably across commodity hardware. MapReduce is a programming model used to process large amounts of data in parallel. Hadoop architecture uses a master-slave model, with a NameNode master and DataNode slaves. It provides fault tolerance, high throughput access to application data and scales to thousands of machines.

Big data and hadoopRoushan Sinha

Big data and Hadoop are frameworks for processing and storing large datasets. Hadoop uses HDFS for distributed storage and MapReduce for distributed processing. HDFS stores large files across multiple machines for redundancy and parallel access. MapReduce divides jobs into map and reduce tasks that run in parallel across a cluster. Hadoop provides scalable and fault-tolerant solutions to problems like processing terabytes of data from jet engines or scaling to Google's data processing needs.

Mapreduce Hadop.pptxBangladesh University of Professionals

Cppt Hadoopchunkypandey12

This document discusses Hadoop and its core components HDFS and MapReduce. It provides an overview of how Hadoop addresses the challenges of big data by allowing distributed processing of large datasets across clusters of computers. Key points include: Hadoop uses HDFS for distributed storage and MapReduce for distributed processing; HDFS works on a master-slave model with a Namenode and Datanodes; MapReduce utilizes a map and reduce programming model to parallelize tasks. Fault tolerance is built into Hadoop to prevent single points of failure.

Cpptchunkypandey12

This document provides an overview of Hadoop and how it addresses the challenges of big data. It discusses how Hadoop uses a distributed file system (HDFS) and MapReduce programming model to allow processing of large datasets across clusters of computers. Key aspects summarized include how HDFS works using namenodes and datanodes, how MapReduce leverages mappers and reducers to parallelize processing, and how Hadoop provides fault tolerance.

Cpptchunkypandey12

Managing Big data with HadoopNalini Mehta

The data management industry has matured over the last three decades, primarily based on relational database management system(RDBMS) technology. Since the amount of data collected, and analyzed in enterprises has increased several folds in volume, variety and velocityof generation and consumption, organisations have started struggling with architectural limitations of traditional RDBMS architecture. As a result a new class of systems had to be designed and implemented, giving rise to the new phenomenon of “Big Data”. In this paper we will trace the origin of new class of system called Hadoop to handle Big data.

Big data Hadoop Ayyappan Paramesh

The document provides an overview of big data and Hadoop fundamentals. It discusses what big data is, the characteristics of big data, and how it differs from traditional data processing approaches. It then describes the key components of Hadoop including HDFS for distributed storage, MapReduce for distributed processing, and YARN for resource management. HDFS architecture and features are explained in more detail. MapReduce tasks, stages, and an example word count job are also covered. The document concludes with a discussion of Hive, including its use as a data warehouse infrastructure on Hadoop and its query language HiveQL.

Introduccion a Hadoop / Introduction to HadoopGERARDO BARBERENA

Hadoop is a framework for distributed storage and processing of large datasets across clusters of computers. It addresses problems like hardware failure and combining data after analysis. The core components are HDFS for distributed storage and MapReduce for distributed processing. HDFS stores data as blocks across nodes and handles replication for reliability. The Namenode manages the file system namespace and metadata, while Datanodes store and retrieve blocks. Hadoop supports reliable analysis of large datasets in a distributed manner through its scalable architecture.

Big data and hadoopPrashanth Yennampelli

This document discusses big data and the Apache Hadoop framework. It defines big data as large, complex datasets that are difficult to process using traditional tools. Hadoop is an open-source framework for distributed storage and processing of big data across commodity hardware. It has two main components - the Hadoop Distributed File System (HDFS) for storage, and MapReduce for processing. HDFS stores data across clusters of machines with redundancy, while MapReduce splits tasks across processors and handles shuffling and sorting of data. Hadoop allows cost-effective processing of large, diverse datasets and has become a standard for big data.

Big data analysis using hadoop clusterFurqan Haider

Hadoopavnishagr

This document discusses Hadoop, an open-source software framework for distributed storage and processing of large datasets across clusters of computers. It describes how Hadoop uses HDFS for scalable, fault-tolerant storage and MapReduce for parallel processing. The core components of Hadoop - HDFS and MapReduce - allow for distributed processing of large datasets across commodity hardware, providing capabilities for scalability, cost-effectiveness, and efficient distributed computing.

M. Florence Dayana - Hadoop Foundation for Analytics.pptxDr.Florence Dayana

Hadoop TechnologyAtul Kushwaha

This document provides an overview of Big Data and Hadoop. It defines Big Data as large volumes of structured, semi-structured, and unstructured data that is too large to process using traditional databases and software. It provides examples of the large amounts of data generated daily by organizations. Hadoop is presented as a framework for distributed storage and processing of large datasets across clusters of commodity hardware. Key components of Hadoop including HDFS for distributed storage and fault tolerance, and MapReduce for distributed processing, are described at a high level. Common use cases for Hadoop by large companies are also mentioned.

MOD-2 presentation on engineering studentsrishavkumar1402

- Hadoop Distributed File System (HDFS) is a distributed file system that stores large datasets across clusters of computers. It divides files into blocks and stores the blocks across nodes, replicating them for fault tolerance. - HDFS is designed for distributed storage and processing of very large datasets. It allows applications to work with data in parallel on large clusters of commodity hardware.

HadoopKasam Sharif

hadoopswatic018

Hadoop is a framework for distributed storage and processing of large datasets across clusters of commodity hardware. It uses HDFS for fault-tolerant storage and MapReduce as a programming model for distributed computing. HDFS stores data across clusters of machines and replicates it for reliability. MapReduce allows processing of large datasets in parallel by splitting work into independent tasks. Hadoop provides reliable and scalable storage and analysis of very large amounts of data.

Big Data and Cloud ComputingFarzad Nozarian

2. hadoop fundamentalsLokesh Ramaswamy

AnjuAnju Shekhawat

Hadoopchandinisanz

Big data and hadoopRoushan Sinha

Mapreduce Hadop.pptxBangladesh University of Professionals

Cppt Hadoopchunkypandey12

Cpptchunkypandey12

Managing Big data with HadoopNalini Mehta

Big data Hadoop Ayyappan Paramesh

Introduccion a Hadoop / Introduction to HadoopGERARDO BARBERENA

Big data and hadoopPrashanth Yennampelli

Big data analysis using hadoop clusterFurqan Haider

Hadoopavnishagr

M. Florence Dayana - Hadoop Foundation for Analytics.pptxDr.Florence Dayana

Hadoop TechnologyAtul Kushwaha

MOD-2 presentation on engineering studentsrishavkumar1402

HadoopKasam Sharif

hadoopswatic018

Recently uploaded (20)

Presentation of the MIPLM subject matter expert Erdem KayaMIPLM

Understanding P–N Junction Semiconductors: A Beginner’s GuideGS Virdi

Dive into the fundamentals of P–N junctions, the heart of every diode and semiconductor device. In this concise presentation, Dr. G.S. Virdi (Former Chief Scientist, CSIR-CEERI Pilani) covers: What Is a P–N Junction? Learn how P-type and N-type materials join to create a diode. Depletion Region & Biasing: See how forward and reverse bias shape the voltage–current behavior. V–I Characteristics: Understand the curve that defines diode operation. Real-World Uses: Discover common applications in rectifiers, signal clipping, and more. Ideal for electronics students, hobbyists, and engineers seeking a clear, practical introduction to P–N junction semiconductors.

Multi-currency in odoo accounting and Update exchange rates automatically in ...Celine George

How to manage Multiple Warehouses for multiple floors in odoo point of saleCeline George

Quality Contril Analysis of Containers.pdfDr. Bindiya Chauhan

Unit 6_Introduction_Phishing_Password Cracking.pdfKanchanPatil34

To study the nervous system of insect.pptxArshad Shaikh

The *nervous system of insects* is a complex network of nerve cells (neurons) and supporting cells that process and transmit information. Here's an overview: Structure 1. *Brain*: The insect brain is a complex structure that processes sensory information, controls behavior, and integrates information. 2. *Ventral nerve cord*: A chain of ganglia (nerve clusters) that runs along the insect's body, controlling movement and sensory processing. 3. *Peripheral nervous system*: Nerves that connect the central nervous system to sensory organs and muscles. Functions 1. *Sensory processing*: Insects can detect and respond to various stimuli, such as light, sound, touch, taste, and smell. 2. *Motor control*: The nervous system controls movement, including walking, flying, and feeding. 3. *Behavioral responThe *nervous system of insects* is a complex network of nerve cells (neurons) and supporting cells that process and transmit information. Here's an overview: Structure 1. *Brain*: The insect brain is a complex structure that processes sensory information, controls behavior, and integrates information. 2. *Ventral nerve cord*: A chain of ganglia (nerve clusters) that runs along the insect's body, controlling movement and sensory processing. 3. *Peripheral nervous system*: Nerves that connect the central nervous system to sensory organs and muscles. Functions 1. *Sensory processing*: Insects can detect and respond to various stimuli, such as light, sound, touch, taste, and smell. 2. *Motor control*: The nervous system controls movement, including walking, flying, and feeding. 3. *Behavioral responses*: Insects can exhibit complex behaviors, such as mating, foraging, and social interactions. Characteristics 1. *Decentralized*: Insect nervous systems have some autonomy in different body parts. 2. *Specialized*: Different parts of the nervous system are specialized for specific functions. 3. *Efficient*: Insect nervous systems are highly efficient, allowing for rapid processing and response to stimuli. The insect nervous system is a remarkable example of evolutionary adaptation, enabling insects to thrive in diverse environments. The insect nervous system is a remarkable example of evolutionary adaptation, enabling insects to thrive

Sinhala_Male_Names.pdf Sinhala_Male_Namekeshanf79

Niamh Lucey, Mary Dunne. Health Sciences Libraries Group (LAI). Lighting the ...Library Association of Ireland

How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...Celine George

Analytic accounts are used to track and manage financial transactions related to specific projects, departments, or business units. They provide detailed insights into costs and revenues at a granular level, independent of the main accounting system. This helps to better understand profitability, performance, and resource allocation, making it easier to make informed financial decisions and strategic planning.

SCI BIZ TECH QUIZ (OPEN) PRELIMS XTASY 2025.pptxRonisha Das

Exploring-Substances-Acidic-Basic-and-Neutral.pdfSandeep Swamy

Exploring Substances: Acidic, Basic, and Neutral Welcome to the fascinating world of acids and bases! Join siblings Ashwin and Keerthi as they explore the colorful world of substances at their school's National Science Day fair. Their adventure begins with a mysterious white paper that reveals hidden messages when sprayed with a special liquid. In this presentation, we'll discover how different substances can be classified as acidic, basic, or neutral. We'll explore natural indicators like litmus, red rose extract, and turmeric that help us identify these substances through color changes. We'll also learn about neutralization reactions and their applications in our daily lives. by sandeep swamy

Biophysics Chapter 3 Methods of Studying Macromolecules.pdfPKLI-Institute of Nursing and Allied Health Sciences Lahore , Pakistan.

This chapter provides an in-depth overview of the viscosity of macromolecules, an essential concept in biophysics and medical sciences, especially in understanding fluid behavior like blood flow in the human body. Key concepts covered include: ✅ Definition and Types of Viscosity: Dynamic vs. Kinematic viscosity, cohesion, and adhesion. ⚙️ Methods of Measuring Viscosity: Rotary Viscometer Vibrational Viscometer Falling Object Method Capillary Viscometer 🌡️ Factors Affecting Viscosity: Temperature, composition, flow rate. 🩺 Clinical Relevance: Impact of blood viscosity in cardiovascular health. 🌊 Fluid Dynamics: Laminar vs. turbulent flow, Reynolds number. 🔬 Extension Techniques: Chromatography (adsorption, partition, TLC, etc.) Electrophoresis (protein/DNA separation) Sedimentation and Centrifugation methods.

YSPH VMOC Special Report - Measles Outbreak Southwest US 5-3-2025.pptxYale School of Public Health - The Virtual Medical Operations Center (VMOC)

A measles outbreak originating in West Texas has been linked to confirmed cases in New Mexico, with additional cases reported in Oklahoma and Kansas. The current case count is 817 from Texas, New Mexico, Oklahoma, and Kansas. 97 individuals have required hospitalization, and 3 deaths, 2 children in Texas and one adult in New Mexico. These fatalities mark the first measles-related deaths in the United States since 2015 and the first pediatric measles death since 2003. The YSPH Virtual Medical Operations Center Briefs (VMOC) were created as a service-learning project by faculty and graduate students at the Yale School of Public Health in response to the 2010 Haiti Earthquake. Each year, the VMOC Briefs are produced by students enrolled in Environmental Health Science Course 581 - Public Health Emergencies: Disaster Planning and Response. These briefs compile diverse information sources – including status reports, maps, news articles, and web content– into a single, easily digestible document that can be widely shared and used interactively. Key features of this report include: - Comprehensive Overview: Provides situation updates, maps, relevant news, and web resources. - Accessibility: Designed for easy reading, wide distribution, and interactive use. - Collaboration: The “unlocked" format enables other responders to share, copy, and adapt seamlessly. The students learn by doing, quickly discovering how and where to find critical information and presenting it in an easily understood manner. CURRENT CASE COUNT: 817 (As of 05/3/2025) • Texas: 688 (+20)(62% of these cases are in Gaines County). • New Mexico: 67 (+1 )(92.4% of the cases are from Eddy County) • Oklahoma: 16 (+1) • Kansas: 46 (32% of the cases are from Gray County) HOSPITALIZATIONS: 97 (+2) • Texas: 89 (+2) - This is 13.02% of all TX cases. • New Mexico: 7 - This is 10.6% of all NM cases. • Kansas: 1 - This is 2.7% of all KS cases. DEATHS: 3 • Texas: 2 – This is 0.31% of all cases • New Mexico: 1 – This is 1.54% of all cases US NATIONAL CASE COUNT: 967 (Confirmed and suspected): INTERNATIONAL SPREAD (As of 4/2/2025) • Mexico – 865 (+58) ‒Chihuahua, Mexico: 844 (+58) cases, 3 hospitalizations, 1 fatality • Canada: 1531 (+270) (This reflects Ontario's Outbreak, which began 11/24) ‒Ontario, Canada – 1243 (+223) cases, 84 hospitalizations. • Europe: 6,814

LDMMIA Reiki Master Spring 2025 Mini UpdatesLDM Mia eStudios

UNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACYDR.PRISCILLA MARY J

GDGLSPGCOER - Git and GitHub Workshop.pptxazeenhodekar

One Hot encoding a revolution in Machine learningmomer9505

Anti-Depressants pharmacology 1slide.pptxMayuri Chavan

Marie Boran Special Collections Librarian Hardiman Library, University of Gal...Library Association of Ireland

Presentation of the MIPLM subject matter expert Erdem KayaMIPLM

Understanding P–N Junction Semiconductors: A Beginner’s GuideGS Virdi

Multi-currency in odoo accounting and Update exchange rates automatically in ...Celine George

How to manage Multiple Warehouses for multiple floors in odoo point of saleCeline George

Quality Contril Analysis of Containers.pdfDr. Bindiya Chauhan

Unit 6_Introduction_Phishing_Password Cracking.pdfKanchanPatil34

To study the nervous system of insect.pptxArshad Shaikh

Sinhala_Male_Names.pdf Sinhala_Male_Namekeshanf79

Niamh Lucey, Mary Dunne. Health Sciences Libraries Group (LAI). Lighting the ...Library Association of Ireland

How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...Celine George

SCI BIZ TECH QUIZ (OPEN) PRELIMS XTASY 2025.pptxRonisha Das

Exploring-Substances-Acidic-Basic-and-Neutral.pdfSandeep Swamy

Biophysics Chapter 3 Methods of Studying Macromolecules.pdfPKLI-Institute of Nursing and Allied Health Sciences Lahore , Pakistan.

YSPH VMOC Special Report - Measles Outbreak Southwest US 5-3-2025.pptxYale School of Public Health - The Virtual Medical Operations Center (VMOC)

LDMMIA Reiki Master Spring 2025 Mini UpdatesLDM Mia eStudios

UNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACYDR.PRISCILLA MARY J

GDGLSPGCOER - Git and GitHub Workshop.pptxazeenhodekar

One Hot encoding a revolution in Machine learningmomer9505

Anti-Depressants pharmacology 1slide.pptxMayuri Chavan

Marie Boran Special Collections Librarian Hardiman Library, University of Gal...Library Association of Ireland

Hadoop hive presentation

1. Hadoop

2. Agenda • Problems with traditional large-scale systems • Requirements for new approaches • What is Hadoop..? • Why Hadoop? • Overview of Hadoop • HDFS • Map Reduce • Applications • Conclusion

4. Problems with traditional large-scale systems  Data is being increased day-by-day  Issues with the network failure  Server failure  Loss of data  Cost is more.  Distributed computing need manual processing

5. Requirements for new approaches  Data should be stored in a distributed manner and parallel processing.  High performance and less cost.  Should be scalable  Should be simple to access and process  Fault tolerance

6. What is Hadoop…?  Open Source Framework  Process large amount of data

8. Why Hadoop…? • Accessible • Scalable • Robust • Simple

9. Overview of Hadoop  It handles 3 types of data Structured Semi – structured Unstructured  Analyses and process large amounts of data (Peta byte)

10. Compare with traditional DB’s RDBMS • Stores GB’s of data • Supports batch process and interactive process • Allows Updation • Schemas must me defined • Only structured data HADOOP • Stores PB’s of data • Only batch process • Does not allow Updation, it follows WORM • Schemas not required • Supports 3 types of data

13. Components  Hadoop can be divided into 2 parts 1. HDFS – Hadoop Distributed File System 2. MapReduce Programming model

14. Hadoop Distributed File System  It is a distributed file system  Runs on commodity hardware  Provides high throughput access to application data  suitable for applications that have large data sets.  It is designed to store a very large amount of data (Tera or peta bytes).

16. Core Architectural Goal of HDFS  A HDFS instance may consist of thousands of server machines.  Detection of faults and quickly recovering from them in an automated manner

17. MapReduce Programming Model  MapReduce works on divide and conquer rule on the data.  Schedules execution across a set of machines  Manages inter-process communication  The Reducer processes all output from all mappers and arrives at final output

18. MapReduce Programming Model – MAP • Map() function that processes a key/value pair to generate a set of intermediate key/value pairs – REDUCE • reduce() function that merges all intermediate values associated with the same intermediate key.

20. Applications

25. REFERENCE • HADOOP IN ACTION - By CHUK LAM • YOUTUBE • WIKEPEDIA • GOOGLE IMAGES

26. Conclusion

Hadoop hive presentation

Recommended

More Related Content

What's hot (20)

Similar to Hadoop hive presentation (20)

Recently uploaded (20)

Hadoop hive presentation