Big Data simplified

Nov 10, 20161 like1,668 views

This document provides an overview of big data concepts including what big data is, how it is used, and common tools involved. It defines big data as a cluster of technologies like Hadoop, HDFS, and HCatalog used for fetching, processing, and visualizing large datasets. MapReduce and Hadoop clusters are described as common processing techniques. Example use cases mentioned include business intelligence. Resources for getting started with tools like Hortonworks, CloudEra, and examples of MapReduce jobs are also provided.

BIG DATA
simplified!
Pravin Hanchinal
pravinhanchinal.com

Big Data
Big Data is a
cluster of many technologies and tools
that are used in various scenarios.
(Hadoop + HDFS+ Hcatalog+Flume+PowerView)
(HortonWorks + PowerView)

What you can do in Big Data?
Fetching
Processing
Visualizing

How Big is Big Data?
Byte of data: one grain of rice
Kilobyte: cup of rice
Megabyte: 8 bags of rice
Gigabyte: 3 container of lorries
Terabyte: 2 container ships
Petabyte: covers Mumbai
Exabyte: covers India
Zettabyte: fills Indian Ocean

MapReduce
•MapReduce is a processing technique and a program
model for distributed computing based on java.
•The MapReduce algorithm contains two important tasks,
namely Map and Reduce.

What you can do on Big Data?
Get Started with this:
CloudEra
HortonWorks

Why Hadoop?
-> Hadoop modeling and development: MapReduce, Pig, Mahout
-> Hadoop storage and data management: HDFS, HBase, Cassandra
-> Hadoop data warehousing, summarization and query: Hive, Sqoop
-> Hadoop data collection, aggregation and analysis: Chukwa, Flume
-> Hadoop metadata, table and schema management: HCatalog
-> Hadoop cluster management, job scheduling and workflow:
ZooKeeper, Oozie and Ambari
-> Hadoop Data serialization: Avro

Got questions?
Text/WhatsApp on 974-086-1099

Resources
http://pravinhanchinal.com/what-is-for-what-hadoop-tools
https://blog.cloudera.com/blog/2014/01/how-to-create-a-simple-hadoop-cluster-wit
h-virtualbox/
http://pingax.com/install-apache-hadoop-ubuntu-cluster-setup/
https://de.slideshare.net/EdurekaIN/ha-webinar-48976388

Resources
https://ayende.com/blog/4435/map-reduce-a-visual-explanation
MultiNode on Amazon: https://dzone.com/articles/how-set-multi-node-hadoop
https://ayende.com/blog/4435/map-reduce-a-visual-explanation
Run Sample MapReduce Examples:
MapReduce examples:
http://www.informit.com/articles/article.aspx?p=2190194&seqNum=3
https://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-pig/

The document discusses big data and its applications. It defines big data as large and complex data sets that are difficult to process using traditional data management tools. It outlines the three V's of big data - volume, variety, and velocity. Various types of structured, semi-structured, and unstructured data are described. Examples are given of how big data is used in various industries like automotive, finance, manufacturing, policing, and utilities to improve products, detect fraud, perform simulations, track suspects, and monitor assets. Popular big data software like Hadoop and MongoDB are also mentioned.

BIG DATADr. Shashank Shetty

Big data abstractnandhiniarumugam619

Big Data Hadoop Training by Easylearning GuruKCC Software Ltd. & Easylearning.guru

Big data Analytics HadoopMishika Bharadwaj

Core concepts and Key technologies - Big Data AnalyticsKaniska Mandal

Big data analytics has evolved beyond batch processing with Hadoop to extract intelligence from data streams in real time. New technologies preserve data locality, allow real-time processing and streaming, support complex analytics functions, provide rich data models and queries, optimize data flow and queries, and leverage CPU caches and distributed memory for speed. Frameworks like Spark and Shark improve on MapReduce with in-memory computation and dynamic resource allocation.

Introduction to BIg Data and HadoopAmir Shaikh

Hadoop is an open-source framework for distributed storage and processing of large datasets across clusters of commodity hardware. It addresses limitations in traditional RDBMS for big data by allowing scaling to large clusters of commodity servers, high fault tolerance, and distributed processing. The core components of Hadoop are HDFS for distributed storage and MapReduce for distributed processing. Hadoop has an ecosystem of additional tools like Pig, Hive, HBase and more. Major companies use Hadoop to process and gain insights from massive amounts of structured and unstructured data.

Hadoop and BigData - July 2016Ranjith Sekar

This document provides an overview of Hadoop and Big Data. It begins with introducing key concepts like structured, semi-structured, and unstructured data. It then discusses the growth of data and need for Big Data solutions. The core components of Hadoop like HDFS and MapReduce are explained at a high level. The document also covers Hadoop architecture, installation, and developing a basic MapReduce program.

BigData Analytics with Hadoop and BIRTAmrit Chhetri

This document provides an agenda for a Big Data summer training session presented by Amrit Chhetri. The agenda includes modules on Big Data analytics with Apache Hadoop, installing Apache Hadoop on Ubuntu, using HBase, advanced Python techniques, and performing ETL with tools like Sqoop and Talend. Amrit introduces himself and his background before delving into the topics to be covered in the training.

Big data, map reduce and beyonddatasalt

Intro to Big Data HadoopApache Apex

Introduction to Bigdata and HADOOP vinoth kumar

- Hadoop is a framework for managing and processing big data distributed across clusters of computers. It allows for parallel processing of large datasets. - Big data comes from various sources like customer behavior, machine data from sensors, etc. It is used by companies to better understand customers and target ads. - Hadoop uses a master-slave architecture with a NameNode master and DataNode slaves. Files are divided into blocks and replicated across DataNodes for reliability. The NameNode tracks where data blocks are stored.

Big Data: An OverviewC. Scyphers

Big data refers to datasets that are too large to be managed by traditional database tools. It is characterized by volume, velocity, and variety. Hadoop is an open-source software framework that allows distributed processing of large datasets across clusters of computers. It works by distributing storage across nodes as blocks and distributing computation via a MapReduce programming paradigm where nodes process data in parallel. Common uses of big data include analyzing social media, sensor data, and using machine learning on large datasets.

Hadoop: An Industry PerspectiveCloudera, Inc.

This document provides an overview of Hadoop and how it can be used for data consolidation, schema flexibility, and query flexibility compared to a relational database. It describes the key components of Hadoop including HDFS for storage and MapReduce for distributed processing. Examples of industry use cases are also presented, showing how Hadoop enables affordable long-term storage and scalable processing of large amounts of structured and unstructured data.

Big data Big AnalyticsAjay Ohri

This document provides an agenda for a presentation on big data and big data analytics using R. The presentation introduces the presenter and has sections on defining big data, discussing tools for storing and analyzing big data in R like HDFS and MongoDB, and presenting case studies analyzing social network and customer data using R and Hadoop. The presentation also covers challenges of big data analytics, existing case studies using tools like SAP Hana and Revolution Analytics, and concerns around privacy with large-scale data analysis.

THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012Gigaom

Hadoop and big dataYukti Kaura

The document provides an overview of Hadoop and the Hadoop ecosystem. It discusses the history of Hadoop, how big data is defined in terms of volume, velocity, variety and veracity. It then explains what Hadoop is, the core components of HDFS and MapReduce, how Hadoop is used for distributed processing of large datasets, and how Hadoop compares to traditional RDBMS. The document also outlines other tools in the Hadoop ecosystem like Pig, Hive, HBase and gives a brief demo.

Big data technologies and Hadoop infrastructureRoman Nikitchenko

Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...Mahantesh Angadi

This document provides an introduction to big data and the installation of a single-node Apache Hadoop cluster. It defines key terms like big data, Hadoop, and MapReduce. It discusses traditional approaches to handling big data like storage area networks and their limitations. It then introduces Hadoop as an open-source framework for storing and processing vast amounts of data in a distributed fashion using the Hadoop Distributed File System (HDFS) and MapReduce programming model. The document outlines Hadoop's architecture and components, provides an example of how MapReduce works, and discusses advantages and limitations of the Hadoop framework.

Big Data Course - BigData HUBAhmed Salman

This document provides an introduction to big data and Hadoop. It discusses what big data is, characteristics of big data like volume, velocity and variety. It then introduces Hadoop as a framework for storing and analyzing big data, describing its main components like HDFS and MapReduce. The document outlines a typical big data workflow and gives examples of big data use cases. It also provides an overview of setting up Hadoop on a single node, including installing Java, configuring SSH, downloading and extracting Hadoop files, editing configuration files, formatting the namenode, starting Hadoop daemons and testing the installation.

Big Data Analytics 2014Stratebi

This document discusses open source tools for big data analytics. It introduces Hadoop, HDFS, MapReduce, HBase, and Hive as common tools for working with large and diverse datasets. It provides overviews of what each tool is used for, its architecture and components. Examples are given around processing log and word count data using these tools. The document also discusses using Pentaho Kettle for ETL and business intelligence projects with big data.

Introduction to Big Data and HadoopFebiyan Rachman

Big Data & Hadoop IntroductionJayant Mukherjee

Introduction of Big data, NoSQL & HadoopSavvycom Savvycom

Big data introduction, Hadoop in detailsMahmoud Yassin

WhatisbigdataandwhylearnhadoopEdureka!

This document is a presentation on big data and Hadoop. It introduces big data, how it is growing exponentially, and the challenges of storing and analyzing unstructured data. It discusses how Sears moved to Hadoop to gain insights from all of its customer data. The presentation explains why Hadoop is in high demand, as it allows distributed processing of large datasets across commodity hardware. It provides an overview of the Hadoop ecosystem including HDFS, MapReduce, Hive, HBase and more. Finally, it discusses job opportunities and salaries in big data which are high and growing significantly.

Big data pptNasrin Hussain

This document provides an overview of big data. It defines big data as large volumes of diverse data that are growing rapidly and require new techniques to capture, store, distribute, manage, and analyze. The key characteristics of big data are volume, velocity, and variety. Common sources of big data include sensors, mobile devices, social media, and business transactions. Tools like Hadoop and MapReduce are used to store and process big data across distributed systems. Applications of big data include smarter healthcare, traffic control, and personalized marketing. The future of big data is promising with the market expected to grow substantially in the coming years.

What is Big Data?Bernard Marr

More Related Content

What's hot (20)

Introduction to BIg Data and HadoopAmir Shaikh

Hadoop and BigData - July 2016Ranjith Sekar

BigData Analytics with Hadoop and BIRTAmrit Chhetri

Big data, map reduce and beyonddatasalt

Intro to Big Data HadoopApache Apex

Introduction to Bigdata and HADOOP vinoth kumar

Big Data: An OverviewC. Scyphers

Hadoop: An Industry PerspectiveCloudera, Inc.

Big data Big AnalyticsAjay Ohri

THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012Gigaom

Hadoop and big dataYukti Kaura

Big data technologies and Hadoop infrastructureRoman Nikitchenko

Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...Mahantesh Angadi

Big Data Course - BigData HUBAhmed Salman

Big Data Analytics 2014Stratebi

Introduction to Big Data and HadoopFebiyan Rachman

Big Data & Hadoop IntroductionJayant Mukherjee

Introduction of Big data, NoSQL & HadoopSavvycom Savvycom

Big data introduction, Hadoop in detailsMahmoud Yassin

WhatisbigdataandwhylearnhadoopEdureka!

Introduction to BIg Data and HadoopAmir Shaikh

Hadoop and BigData - July 2016Ranjith Sekar

BigData Analytics with Hadoop and BIRTAmrit Chhetri

Big data, map reduce and beyonddatasalt

Intro to Big Data HadoopApache Apex

Introduction to Bigdata and HADOOP vinoth kumar

Big Data: An OverviewC. Scyphers

Hadoop: An Industry PerspectiveCloudera, Inc.

Big data Big AnalyticsAjay Ohri

THE 3V's OF BIG DATA: VARIETY, VELOCITY, AND VOLUME from Structure:Data 2012Gigaom

Hadoop and big dataYukti Kaura

Big data technologies and Hadoop infrastructureRoman Nikitchenko

Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...Mahantesh Angadi

Big Data Course - BigData HUBAhmed Salman

Big Data Analytics 2014Stratebi

Introduction to Big Data and HadoopFebiyan Rachman

Big Data & Hadoop IntroductionJayant Mukherjee

Introduction of Big data, NoSQL & HadoopSavvycom Savvycom

Big data introduction, Hadoop in detailsMahmoud Yassin

WhatisbigdataandwhylearnhadoopEdureka!

Viewers also liked (20)

Big data pptNasrin Hussain

What is Big Data?Bernard Marr

Big data pptThirunavukkarasu Ps

Big data refers to the massive amounts of unstructured data that are growing exponentially. Hadoop is an open-source framework that allows processing and storing large data sets across clusters of commodity hardware. It provides reliability and scalability through its distributed file system HDFS and MapReduce programming model. The Hadoop ecosystem includes components like Hive, Pig, HBase, Flume, Oozie, and Mahout that provide SQL-like queries, data flows, NoSQL capabilities, data ingestion, workflows, and machine learning. Microsoft integrates Hadoop with its BI and analytics tools to enable insights from diverse data sources.

What is big data?David Wellman

Simple Tactics Superb Performance by INSPIRE-groupsPraveen Hanchinal

Cloud computing projects by inspire-groups (Pravin Hanchinal)Praveen Hanchinal

This document discusses cloud computing projects and provides an overview of cloud development. It summarizes that cloud development allows coding from anywhere at any time without heavy infrastructure needs. It also outlines various cloud IDEs and platforms that can be used to deploy apps, including Cloud9 and Codenvy. The document encourages experimenting with cloud technologies and collaborating with INSPIRE-groups to learn more.

Virtualization-the Cloud Enabler by INSPIRE-groupsPraveen Hanchinal

Vedic Sciences and ComputersPraveen Hanchinal

Entrepreneurship by INSPIRE-groups (Pravin Hanchinal)Praveen Hanchinal

This document provides an overview of entrepreneurship and the journey of INSPIRE groups. It encourages taking risks and failing fast when starting a business. It emphasizes the importance of culture, having a result-oriented work environment, thinking extremely, embracing innovation and cloud technologies. It highlights how unlimited reading, TED talks, inspiring people and failures helped shape the company. It invites the audience to collaborate with INSPIRE groups by tossing ideas or joining hands with them.

Cloud development and career pathPraveen Hanchinal

Big data Analytics hands-on sessionsPraveen Hanchinal

This document provides an overview of Hadoop and how to set up single node and multi-node Hadoop clusters. It discusses tools used with Hadoop like Flume, Sqoop and MapReduce. It also mentions resource management with YARN and certifications. Setup instructions and prerequisites are provided for virtual machine based clusters on Ubuntu or Windows. Links to additional resources on Hadoop tools, tutorials and sample programs are included.

Big data pptIDBI Bank Ltd.

Big data is large amounts of unstructured data that require new techniques and tools to analyze. Key drivers of big data growth are increased storage capacity, processing power, and data availability. Big data analytics can uncover hidden patterns to provide competitive advantages and better business decisions. Applications include healthcare, homeland security, finance, manufacturing, and retail. The global big data market is expected to grow significantly, with India's market projected to reach $1 billion by 2015. This growth will increase demand for data scientists and analysts to support big data solutions and technologies like Hadoop and NoSQL databases.

Cloud APIs and Cloud FrameworksPraveen Hanchinal

Virtualization, the cloud enablerPraveen Hanchinal

How to give final year project presentation?Praveen Hanchinal

Big Idea For Big DataDexlab Analytics

Big DataNGDATA

Big Data Analytics with HadoopPhilippe Julio

Big data presentation on Crystal Ball Event PredictionSujan Thapa

Apache Drill (ver. 0.1, check ver. 0.2)Camuel Gilyadov

This document outlines the history and design proposals of Apache Drill from the OpenDremel team. It describes OpenDremel starting in 2010 with an initial implementation based on the Dremel paper. Over time, the design was found to be naive and was restarted with a new architecture called Dazo inspired by BigQuery. The document proposes several design tenets for Apache Drill including supporting multi-tenancy, being flexible and customizable, being efficient through the use of ZeroVM for sandboxing, and having a suggested architecture with a browser frontend and multi-tenant backend.

Big data pptNasrin Hussain

What is Big Data?Bernard Marr

Big data pptThirunavukkarasu Ps

What is big data?David Wellman

Simple Tactics Superb Performance by INSPIRE-groupsPraveen Hanchinal

Cloud computing projects by inspire-groups (Pravin Hanchinal)Praveen Hanchinal

Virtualization-the Cloud Enabler by INSPIRE-groupsPraveen Hanchinal

Vedic Sciences and ComputersPraveen Hanchinal

Entrepreneurship by INSPIRE-groups (Pravin Hanchinal)Praveen Hanchinal

Cloud development and career pathPraveen Hanchinal

Big data Analytics hands-on sessionsPraveen Hanchinal

Big data pptIDBI Bank Ltd.

Cloud APIs and Cloud FrameworksPraveen Hanchinal

Virtualization, the cloud enablerPraveen Hanchinal

How to give final year project presentation?Praveen Hanchinal

Big Idea For Big DataDexlab Analytics

Big DataNGDATA

Big Data Analytics with HadoopPhilippe Julio

Big data presentation on Crystal Ball Event PredictionSujan Thapa

Apache Drill (ver. 0.1, check ver. 0.2)Camuel Gilyadov

Similar to Big Data simplified (20)

BDtrainingJithin S L

This document provides an overview of big data and Hadoop. It discusses the origin of big data and Hadoop, defines what big data is, and covers the dimensions and sources of big data. The document then defines Hadoop, explains why organizations use it, and discusses its key characteristics. It also outlines the Hadoop ecosystem and components, including HDFS architecture and MapReduce processing. The document aims to help readers understand an overview of Hadoop and its business uses.

Intro to BigData , Hadoop and MapreduceKrishna Sangeeth KS

This document provides an overview of big data and Hadoop. It discusses the scale of big data, noting that Facebook handles 180PB per year and Twitter handles 1.2 million tweets per second. It also covers the volume, variety, and velocity challenges of big data. Hadoop and MapReduce are introduced as the leading solutions for distributed storage and processing of big data using a scale-out architecture. Key ideas of Hadoop include storing large data across multiple machines in HDFS and processing that data in parallel using MapReduce jobs.

Introduction-to-Big-Data-and-Hadoop.pptxPratimakumari213460

This document provides an introduction to big data and Hadoop. It defines big data as large, complex datasets that are difficult to manage and analyze using traditional methods. Hadoop is an open-source software framework used to store and process big data across distributed systems. It includes components like HDFS for scalable storage, MapReduce for parallel processing, Hive for data summarization, and Pig for creating MapReduce programs. The document discusses how Hadoop offers advantages like scalability, ease of use, cost-effectiveness and flexibility for big data processing. It provides examples of Hadoop's real-world use in healthcare, finance, retail and social media. The future of big data and Hadoop is also examined.

Big Data - Hadoop and MapReduce - Aditya GargAgile Testing Alliance

This document discusses how big data and tools like Hadoop can help testing teams. It defines big data as large, complex datasets that are difficult to process using traditional tools. Hadoop uses HDFS for storage and MapReduce for processing. A example is given of using MapReduce to count words in a file. The document then describes a problem of testing a stock exchange adapter where the daily data was 150GB. It proposes using Hadoop to standardize and compare output files to test the adapter's performance. Finally, it lists other ways testing teams could use big data tools like for repeated tests, customer feedback analysis, and adding intelligence to test data.

Hadoop 2.0-developmentKnowledgehut

Companies around the world today find it increasingly difficult to organize and manage large volumes of data. Hadoop has emerged as the most efficient data platform for companies working with big data, and is an integral part of storing, handling and retrieving enormous amounts of data in a variety of applications. Hadoop helps to run deep analytics which cannot be effectively handled by a database engine. Big enterprises around the world have found Hadoop to be a game changer in their Big Data management, and as more companies embrace this powerful technology the demand for Hadoop Developers is also growing. By learning how to harness the power of Hadoop 2.0 to manipulate, analyse and perform computations on Big Data, you will be paving the way for an enriching and financially rewarding career as an expert Hadoop developer.

Analysing of big data using map reducePaladion Networks

Introduction to Big Data and HadoopGreyCampus

Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...Renato Bonomini

The document discusses capacity planning and performance tuning for Hadoop big data systems. It begins with an agenda that covers why capacity planners need to prepare for Hadoop, an overview of the Hadoop ecosystem, capacity planning and performance tuning of Hadoop, getting started, and the importance of measurement. The document then discusses various components of the Hadoop ecosystem and provides guidance on analyzing different types of workloads and components.

Overview of bigdataAbinaya B

Big data refers to large volumes of data that are difficult to process using traditional data processing applications. The volume of data generated every day is massive, including 2.5 quintillion bytes of data created daily and over 40,000 Google searches per second. Big data comes from a wide variety of sources, including social media, websites, sensors, and other devices. It has five key characteristics known as the 5Vs: volume, velocity, variety, veracity and value. Technologies like Hadoop, NoSQL, Hive and Sqoop help store, analyze and transfer both structured and unstructured big data across clusters of computers. Big data has many applications in industries such as government, healthcare, transportation and more to gain insights and improve

Big dataDeddy Setyadi

This document discusses big data, where it comes from, and how it is processed and analyzed. It notes that everything we do online now leaves a digital trace as data. This "big data" includes huge volumes of structured, semi-structured, and unstructured data from various sources like social media, sensors, and the internet of things. Traditional computing cannot handle such large datasets, so technologies like MapReduce, Hadoop, HDFS, and NoSQL databases were developed to distribute the work across clusters of machines and process the data in parallel.

BIG Data & Hadoop Applications in Social MediaSkillspeed

This document discusses how major social media networks like Facebook, Twitter, LinkedIn, Pinterest, and Instagram utilize big data and Hadoop technologies. It provides examples of how each network uses Hadoop for tasks like storing user data, performing analytics, and generating personalized recommendations at massive scales as their user bases and data volumes grow enormously. The document also briefly outlines SkillSpeed's Hadoop training course, which covers topics like HDFS, MapReduce, Pig, Hive, HBase and more to prepare students for jobs working with big data.

re:Introduce Big Data and Hadoop Eco-system.Shakir Ali

This document provides an overview of big data and the Hadoop ecosystem. It defines big data as large and complex datasets that are difficult to process using traditional data management tools. Characteristics of big data include volume, variety, velocity and veracity. The document discusses challenges of managing big data and how Hadoop provides solutions through its distributed architecture. It also summarizes some prominent Apache projects in the Hadoop ecosystem like Pig, Hive, Spark and Hbase.

re:Introduce Big Data and Hadoop Eco-system.Shakir Ali

This document provides an overview of big data and the Hadoop ecosystem. It defines big data as large and complex datasets that are difficult to process using traditional data management tools. Characteristics of big data include volume, variety, velocity and veracity. The document discusses challenges of managing big data and how Hadoop provides solutions through its distributed architecture. Finally, it summarizes key components of the Hadoop ecosystem including HDFS, MapReduce, YARN and various related Apache projects like Pig, Hive, Spark and HBase.

1. what is hadoop part 1wintersnow181189

Hadoop is an open source framework that allows for the distributed processing of large data sets across clusters of commodity hardware. It was designed to scale from terabytes to petabytes of data and to handle both structured and unstructured data. Hadoop uses a programming model called MapReduce that partitions work across nodes in a cluster. It is not a replacement for a relational database as it is designed for batch processing large volumes of data rather than transactional workloads or business intelligence queries. Big data refers to the large and growing volumes of structured, semi-structured and unstructured data that are beyond the ability of traditional databases to capture, manage, and process. Examples of big data sources include social media, sensors, and internet activity,

Hadoop : The Pile of Big DataEdureka!

The document is from Edureka about their Big Data and Hadoop course. It provides an overview of Hadoop including what it is, its key characteristics of being reliable, economical, scalable and flexible. It describes the core Hadoop components of HDFS for storage and YARN for processing. It also discusses when to use Hadoop such as for large datasets that are diverse, growing, and not time critical. Examples are provided for processing different data types like images, XML files, and CSVs. Contact information is given for the Big Data and Hadoop course.

How Do I Learn Big Databigdatabeginner

The document provides an overview of learning big data, including what big data is, the Hadoop ecosystem, common big data job roles and their salaries, skills needed, and steps for getting started. Big data refers to the large and complex data sets that are difficult to analyze using traditional systems. Hadoop is a framework for storing and processing big data, with components like HDFS for storage and MapReduce for processing. The document recommends learning Hadoop fundamentals, then building sample data pipelines using tools like Hive and Pig to start working with big data.

How Do I Learn Big Databigdatabeginner

Big datafactscomputersoftware

Big datarevathireddyb

Big data refers to large amounts of data from various sources that is analyzed to solve problems. It is characterized by volume, velocity, and variety. Hadoop is an open source framework used to store and process big data across clusters of computers. Key components of Hadoop include HDFS for storage, MapReduce for processing, and HIVE for querying. Other tools like Pig and HBase provide additional functionality. Together these tools provide a scalable infrastructure to handle the volume, speed, and complexity of big data.

Big datarevathireddyb

BDtrainingJithin S L

Intro to BigData , Hadoop and MapreduceKrishna Sangeeth KS

Introduction-to-Big-Data-and-Hadoop.pptxPratimakumari213460

Big Data - Hadoop and MapReduce - Aditya GargAgile Testing Alliance

Hadoop 2.0-developmentKnowledgehut

Analysing of big data using map reducePaladion Networks

Introduction to Big Data and HadoopGreyCampus

Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...Renato Bonomini

Overview of bigdataAbinaya B

Big dataDeddy Setyadi

BIG Data & Hadoop Applications in Social MediaSkillspeed

re:Introduce Big Data and Hadoop Eco-system.Shakir Ali

1. what is hadoop part 1wintersnow181189

Hadoop : The Pile of Big DataEdureka!

How Do I Learn Big Databigdatabeginner

Big datafactscomputersoftware

Big datarevathireddyb

More from Praveen Hanchinal (12)

Artificial Intelligence (AI): Applications in Life Science | Davangere Univer...Praveen Hanchinal

TensorFlow based Machine Learning VTU 2019 by pravin hanchinalPraveen Hanchinal

This document provides an overview of artificial intelligence and machine learning. It discusses what AI is, the differences between AI, machine learning, and deep learning. It also demonstrates some AI applications like self-driving cars, discusses the importance of AI, and reviews the types of machine learning. The document concludes by envisioning future applications of AI and discussing some challenges.

Internet of things | Research Directions in Green IoT and Case StudiesPraveen Hanchinal

Artificial Intelligence and Machine Learning by Praveen HanchinalPraveen Hanchinal

Invited as Resource Person on workshop named "Advanced Machine Learning with Python" held at SDMCET, Dharwad. The whole talk was structured into two sessions.One for Artificial Intelligence and another for Machine Learning. The Presentation touched upon techniques and classifications involved in Machine Learning, WorkFlow for building an AI model. Finally, participants explored how to work on TensorFlow using katacoda platform.

Economy and Big Data | Praveen HanchinalPraveen Hanchinal

Artificial intelligence by praveen hanchinalPraveen Hanchinal

Research Issues, Challenges and Directions in IoT (Internet of Things)Praveen Hanchinal

Cloud based mobile app development cit 2017Praveen Hanchinal

This document discusses building cross-platform mobile apps using PhoneGap and cloud services. PhoneGap allows developing apps using HTML, CSS, and JavaScript that can be compiled for iOS and Android. The document provides an example PhoneGap app and recommends using cloud services like Google App Engine, Firebase, and Kinvey for data storage. It includes useful links and encourages readers to learn more about virtualization and experiment with building innovative apps using these techniques.

Cloud based development cit-2017Praveen Hanchinal

Cloud computing simplified cit 2017Praveen Hanchinal

Women and WebPraveen Hanchinal

Google App Engine (Introduction)Praveen Hanchinal

Artificial Intelligence (AI): Applications in Life Science | Davangere Univer...Praveen Hanchinal

TensorFlow based Machine Learning VTU 2019 by pravin hanchinalPraveen Hanchinal

Internet of things | Research Directions in Green IoT and Case StudiesPraveen Hanchinal

Artificial Intelligence and Machine Learning by Praveen HanchinalPraveen Hanchinal

Economy and Big Data | Praveen HanchinalPraveen Hanchinal

Artificial intelligence by praveen hanchinalPraveen Hanchinal

Research Issues, Challenges and Directions in IoT (Internet of Things)Praveen Hanchinal

Cloud based mobile app development cit 2017Praveen Hanchinal

Cloud based development cit-2017Praveen Hanchinal

Cloud computing simplified cit 2017Praveen Hanchinal

Women and WebPraveen Hanchinal

Google App Engine (Introduction)Praveen Hanchinal

Recently uploaded (20)

Build Your Own Copilot & Agents For DevsBrian McKeiver

Semantic Cultivators : The Critical Future Role to Enable AIartmondano

Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Complete Guide to Advanced Logistics Management Software in Riyadh.pdfSoftware Company

Explore the benefits and features of advanced logistics management software for businesses in Riyadh. This guide delves into the latest technologies, from real-time tracking and route optimization to warehouse management and inventory control, helping businesses streamline their logistics operations and reduce costs. Learn how implementing the right software solution can enhance efficiency, improve customer satisfaction, and provide a competitive edge in the growing logistics sector of Riyadh.

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB

Want to learn practical tips for designing systems that can scale efficiently without compromising speed? Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development. As you explore key principles of designing low-latency systems with Rust, you will learn how to: - Create and compile a real-world app with Rust - Connect the application to ScyllaDB (NoSQL data store) - Negotiate tradeoffs related to data modeling and querying - Manage and monitor the database for consistently low latencies

Electronic_Mail_Attacks-1-35.pdf by xploitniftliyevhuseyn

Cyber Awareness overview for 2025 month of securityriccardosl1

DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxJustin Reock

Building 10x Organizations with Modern Productivity Metrics 10x developers may be a myth, but 10x organizations are very real, as proven by the influential study performed in the 1980s, ‘The Coding War Games.’ Right now, here in early 2025, we seem to be experiencing YAPP (Yet Another Productivity Philosophy), and that philosophy is converging on developer experience. It seems that with every new method we invent for the delivery of products, whether physical or virtual, we reinvent productivity philosophies to go alongside them. But which of these approaches actually work? DORA? SPACE? DevEx? What should we invest in and create urgency behind today, so that we don’t find ourselves having the same discussion again in a decade?

Procurement Insights Cost To Value Guide.pptxJon Hansen

HCL Nomad Web – Best Practices and Managing Multiuser Environmentspanagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-and-managing-multiuser-environments/ HCL Nomad Web is heralded as the next generation of the HCL Notes client, offering numerous advantages such as eliminating the need for packaging, distribution, and installation. Nomad Web client upgrades will be installed “automatically” in the background. This significantly reduces the administrative footprint compared to traditional HCL Notes clients. However, troubleshooting issues in Nomad Web present unique challenges compared to the Notes client. Join Christoph and Marc as they demonstrate how to simplify the troubleshooting process in HCL Nomad Web, ensuring a smoother and more efficient user experience. In this webinar, we will explore effective strategies for diagnosing and resolving common problems in HCL Nomad Web, including - Accessing the console - Locating and interpreting log files - Accessing the data folder within the browser’s cache (using OPFS) - Understand the difference between single- and multi-user scenarios - Utilizing Client Clocking

IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...organizerofv

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts! 📕 Agenda Welcome & Introductions Orchestrator API Overview Exploring the Swagger Interface Test Manager API Highlights Streamlining Automation & Testing with APIs (Demo) Q&A and Open Discussion Perfect for developers, testers, and automation enthusiasts! 👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/ This session streamed live on April 29, 2025, 18:00 CET. Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies

#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada

Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next. Link to recording, transcript, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/ Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Talk at the final event of Data Fusion Dynamics: A Collaborative UK-Saudi Initiative in Cybersecurity and Artificial Intelligence funded by the British Council UK-Saudi Challenge Fund 2024, Cardiff Metropolitan University, 29th April 2025 https://alandix.com/academic/talks/CMet2025-AI-Changes-Everything/ Is AI just another technology, or does it fundamentally change the way we live and think? Every technology has a direct impact with micro-ethical consequences, some good, some bad. However more profound are the ways in which some technologies reshape the very fabric of society with macro-ethical impacts. The invention of the stirrup revolutionised mounted combat, but as a side effect gave rise to the feudal system, which still shapes politics today. The internal combustion engine offers personal freedom and creates pollution, but has also transformed the nature of urban planning and international trade. When we look at AI the micro-ethical issues, such as bias, are most obvious, but the macro-ethical challenges may be greater. At a micro-ethical level AI has the potential to deepen social, ethnic and gender bias, issues I have warned about since the early 1990s! It is also being used increasingly on the battlefield. However, it also offers amazing opportunities in health and educations, as the recent Nobel prizes for the developers of AlphaFold illustrate. More radically, the need to encode ethics acts as a mirror to surface essential ethical problems and conflicts. At the macro-ethical level, by the early 2000s digital technology had already begun to undermine sovereignty (e.g. gambling), market economics (through network effects and emergent monopolies), and the very meaning of money. Modern AI is the child of big data, big computation and ultimately big business, intensifying the inherent tendency of digital technology to concentrate power. AI is already unravelling the fundamentals of the social, political and economic world around us, but this is a world that needs radical reimagining to overcome the global environmental and human challenges that confront us. Our challenge is whether to let the threads fall as they may, or to use them to weave a better future.

Cybersecurity Identity and Access Solutions using Azure ADVICTOR MAESTRE RAMIREZ

HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungenpanagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-und-verwaltung-von-multiuser-umgebungen/ HCL Nomad Web wird als die nächste Generation des HCL Notes-Clients gefeiert und bietet zahlreiche Vorteile, wie die Beseitigung des Bedarfs an Paketierung, Verteilung und Installation. Nomad Web-Client-Updates werden “automatisch” im Hintergrund installiert, was den administrativen Aufwand im Vergleich zu traditionellen HCL Notes-Clients erheblich reduziert. Allerdings stellt die Fehlerbehebung in Nomad Web im Vergleich zum Notes-Client einzigartige Herausforderungen dar. Begleiten Sie Christoph und Marc, während sie demonstrieren, wie der Fehlerbehebungsprozess in HCL Nomad Web vereinfacht werden kann, um eine reibungslose und effiziente Benutzererfahrung zu gewährleisten. In diesem Webinar werden wir effektive Strategien zur Diagnose und Lösung häufiger Probleme in HCL Nomad Web untersuchen, einschließlich - Zugriff auf die Konsole - Auffinden und Interpretieren von Protokolldateien - Zugriff auf den Datenordner im Cache des Browsers (unter Verwendung von OPFS) - Verständnis der Unterschiede zwischen Einzel- und Mehrbenutzerszenarien - Nutzung der Client Clocking-Funktion

The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfAbi john

Build Your Own Copilot & Agents For DevsBrian McKeiver

Semantic Cultivators : The Critical Future Role to Enable AIartmondano

Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Complete Guide to Advanced Logistics Management Software in Riyadh.pdfSoftware Company

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB

Electronic_Mail_Attacks-1-35.pdf by xploitniftliyevhuseyn

Cyber Awareness overview for 2025 month of securityriccardosl1

DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxJustin Reock

Procurement Insights Cost To Value Guide.pptxJon Hansen

HCL Nomad Web – Best Practices and Managing Multiuser Environmentspanagenda

IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...organizerofv

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies

#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Cybersecurity Identity and Access Solutions using Azure ADVICTOR MAESTRE RAMIREZ

HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungenpanagenda

The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfAbi john

Big Data simplified

1. BIG DATA simplified! Pravin Hanchinal pravinhanchinal.com

2. Before we start...

3. Big Data

5. What you can do with Big Data?

6. Big Data Big Data is a cluster of many technologies and tools that are used in various scenarios. (Hadoop + HDFS+ Hcatalog+Flume+PowerView) (HortonWorks + PowerView)

7. What you can do in Big Data? Fetching Processing Visualizing

8. How Big is Big Data? Byte of data: one grain of rice Kilobyte: cup of rice Megabyte: 8 bags of rice Gigabyte: 3 container of lorries Terabyte: 2 container ships Petabyte: covers Mumbai Exabyte: covers India Zettabyte: fills Indian Ocean

9. Big Data Industry Overview

11. MapReduce •MapReduce is a processing technique and a program model for distributed computing based on java. •The MapReduce algorithm contains two important tasks, namely Map and Reduce.

12. Mapreduce

16. Hadoop Cluster

19. What you can do on Big Data? Get Started with this: CloudEra HortonWorks

20. Why Big Data? Business Intelligence

21. HortonWorks

22. Cloud Era

23. Why Hadoop? -> Hadoop modeling and development: MapReduce, Pig, Mahout -> Hadoop storage and data management: HDFS, HBase, Cassandra -> Hadoop data warehousing, summarization and query: Hive, Sqoop -> Hadoop data collection, aggregation and analysis: Chukwa, Flume -> Hadoop metadata, table and schema management: HCatalog -> Hadoop cluster management, job scheduling and workflow: ZooKeeper, Oozie and Ambari -> Hadoop Data serialization: Avro

24. Big Data in Nutshell

26. Got questions? Text/WhatsApp on 974-086-1099

27. Stay connected pravinhanchinal.com

28. What Next? Dive in and Explore

29. Typical Use Case

30. Resources http://pravinhanchinal.com/what-is-for-what-hadoop-tools https://blog.cloudera.com/blog/2014/01/how-to-create-a-simple-hadoop-cluster-wit h-virtualbox/ http://pingax.com/install-apache-hadoop-ubuntu-cluster-setup/ https://de.slideshare.net/EdurekaIN/ha-webinar-48976388

31. Resources https://ayende.com/blog/4435/map-reduce-a-visual-explanation MultiNode on Amazon: https://dzone.com/articles/how-set-multi-node-hadoop https://ayende.com/blog/4435/map-reduce-a-visual-explanation Run Sample MapReduce Examples: MapReduce examples: http://www.informit.com/articles/article.aspx?p=2190194&seqNum=3 https://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-pig/

Big Data simplified

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Big Data simplified (20)

More from Praveen Hanchinal (12)

Recently uploaded (20)

Big Data simplified