Top 5 Tasks Of A Hadoop Developer Webinar
Download as PPTX, PDF1 like520 views
This document discusses the top 5 tasks of a Hadoop developer: 1) development and implementation of Hadoop applications, 2) loading data from disparate data sets, 3) performing analysis on big data, 4) securing data on Hadoop clusters, and 5) managing and deploying Hadoop clusters. It provides examples for tasks 2 and 3, discussing loading heterogeneous data from different sources and how companies analyze huge amounts of user data. The document is part of a presentation on Hadoop developer skills and roles.
Ad
More Related Content
What's hot (20)
Viewers also liked (9)
Ad
Similar to Top 5 Tasks Of A Hadoop Developer Webinar (20)
Ad
More from Skillspeed (8)
Recently uploaded (20)
Top 5 Tasks Of A Hadoop Developer Webinar
- 1. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Top 5 Tasks of a Hadoop Developer
- 2. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Session Objectives This session will cover ᗍ Introduction to Big Data and Hadoop ᗍ Roles & Scope of a Hadoop Developer ᗍ Top 5 Tasks of Hadoop Developers ᗍ Introduction to Hadoop Clusters & HBase ᗍ Job Trends for Hadoop
- 3. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Big Data Challenges
- 4. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Why Hadoop?
- 5. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com What is Hadoop? ᗍ Hadoop is open source framework for big data. Both distributed storage and processing ᗍ Hadoop is reliable and fault tolerant with no rely on hardware for these properties ᗍ Hadoop has unique horizontal scalability
- 6. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Tasks of a Hadoop Developer The following are the tasks of a Hadoop Developer: ᗍ Development and implementation ᗍ Loading from disparate data sets ᗍ Pre-processing ᗍ Designing, building, installing, configuring and supporting Hadoop ᗍ Translate complex functional and technical requirements into detailed design ᗍ Perform analysis on big data ᗍ Securing Data ᗍ Create scalable and high-performance web services for data tracking ᗍ High-speed querying ᗍ Managing and deploying
- 7. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Let us Look at the Top 5 Tasks of a Hadoop Developer with Examples
- 8. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Task 1: Development and Implementation A Hadoop developer is responsible for the actual coding/programming of Hadoop applications One of the most important component of Hadoop is MapReduce in which you need to write Java programs – all you need is a basic Java background
- 9. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Task 2: Loading from Disparate Data Sets Disparate Data are heterogeneous data They are neither similar nor can be easily integrated with an organizations database management system. It differs in one or more aspects of an information system Disparate data may be characterized by these basic problems: ᗍ Implementing a database system in an organization, there is no complete and integrated inventory of all its data ᗍ High data redundancy all over the organization ᗍ High variability of data formats and contents
- 10. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Task 2: Loading from Disparate Data Sets – Scenario Consider a web application where a user can send a query on getting a variety of information about various aspects such as crime statistics, weather, hotels, demographics, etc. in a particular city Traditionally, the information must be stored in a single database with a single schema But it would be difficult for any single enterprise to process and expensive to collect
- 11. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Task 3: Perform Analysis on Big Data A Hadoop Developer perform and analyses the big data
- 12. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Task 3: Perform Analysis on Big Data – Example Have you ever wondered how Google, Facebook or LinkedIn manages to store and utilize the huge data? Today, it is becoming a problem for all of us to manage such BIG DATA….
- 13. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Task 4: Securing Data One of the biggest concerns in our present age revolves around the security and protection of sensitive information Network security breaches from internal and external attackers are on the rise, often taking months to be detected, and affects the organizations terribly
- 14. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Task 5: Managing and Deploying Managing the Hadoop Cluster. This is done using HBase
- 15. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Job Trends – Hadoop
- 16. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Course Topics Module 1 Introduction to Big Data and Hadoop Module 2 HDFS Internals, Hadoop Configurations and Data Loading Module 3 Introduction to Map Reduce Module 4 Advanced Map Reduce Concepts Module 5 Introduction to Pig Module 6 Advanced Pig and Introduction to Hive Module 7 Advanced Hive Concepts Module 8 Extending Hive and HBase Introduction Module 9 Advanced HBase and Oozie Introduction Module 10 Project Set-up Discussion
- 17. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Why SkillSpeed? Course Curriculum from Industry Experts Instructor Led Live Virtual Sessions Lifetime access to Course Content via LMS 100% Placement Assistance 24x7 Support
- 18. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Corporate Partners
- 19. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com Lines open 24/7 To know more about the course, Please contact: IND +91-90660-20904 USA 1866-607-6547 (Toll Free) Or reach us at [email protected] Contact us..
- 20. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com References http://bigdatascroll.com/what-is-hadoop-an-introduction/ http://www.rudraitservices.com/web-development.html http://www.datameer.com/product/data-visualization.html
Editor's Notes
- #9: A Hadoop developer is responsible for the actual coding/programming of Hadoop applications. Ideally the candidate should have at least 2 years of experience as a programmer This role is synonymous to software developer or application developer - refers to the same role but in the Big Data domain One component of Hadoop is MapReduce in which you need to write Java programs - so if you have basic knowledge of Java its sufficient - even if you don't have java knowledge but know any other programming language you can quickly catch up
- #14: Organizations that have not properly controlled access to their data sets are facing lawsuits, negative publicity, and regulatory fines
- #18: SkillSpeed offer virtual instructor lead courses designed to bridge the time to competency gap experienced by the technology companies. USP of SkillSpeed is the subject matter expert (SME). SMEs are industry experts and has a good understanding and hands-on industry experience of the technology. This industry expert designs, develops, and delivers the course. SkillSpeed provides you: Course Curriculum from Industry Experts Instructor Led Live Virtual Sessions Real life industry case studies - Live Virtual Interactions Interaction with industry experts - Lifetime access to all course content via the LMS - 24*7 support - 100% placement assistance