Data scientist versus big data
0 likes•17 views
The document discusses the responsibilities of a data scientist working with big data. A data scientist must ensure smooth loading of large amounts of data from various sources, fetch the data quickly without errors, and create a big data dictionary that can be used to present information to end users according to business rules. The data scientist should understand the entire data flow, including business rules and processes, and present information to users in an accessible way. They require expertise in technologies like Hive, Spark, MapReduce, Pig and Cassandra to work with large datasets.
Data scientist versus big data
- 1. Ensure huge data loading smoothly and fetching those data for preparing big data dictionary which can be easily used for presenting end-use by applying business rules. We should have knowledge of the entire flow, including business rules, current organization business track and user-friendly presentation for an end-user. Should have knowledge of huge data loading smoothly from various sources, and fetching data as quickly as possible without any mistake. Those guys have clear ideas on data loading and data fetching related technologies or tools. There normally experts on Hive, Spark, MapReduce, Pig, Cassandra, etc. DATA SCIENTIST BIG DATA Ensure end to end the flow of data lake architecture, starting from data loading till presentation to end- user. Data Scientist normally has an idea of all the technologies or processing tools like Hive, Map Reduce, R, Spark or the related technologies or tools. Less costly since MapReduce model Costlier than Hadoop since it has an in- V E R S U S COMPARING THE 2 TYPES OF SOFTWARE COURSES