0% found this document useful (0 votes)
14 views

Chapter 2 - Data Science

Uploaded by

Dawit Sebhat
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
14 views

Chapter 2 - Data Science

Uploaded by

Dawit Sebhat
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 20

.

Chapter 2

Data Science
outlines
 data science
 data vs. information
 data types and representation
 data value chain
 basic concepts of big data.
2.1. An Overview of Data Science
 Data science is a multi-disciplinary field that uses scientific methods, processes,
algorithms, and systems to extract knowledge and insights from structured, semi-
structured and unstructured data.
 Data science is much more than simply analyzing data.

 Data can be defined as a representation of facts, concepts, or instructions in a


formalized manner, which should be suitable for communication, interpretation, or
processing, by human or electronic machines.
 Data can be described as unprocessed facts and figures.

 Data is represented with the help of characters such as alphabets (A-Z, a-z), digits
(0-9) or special characters (+, -, /, *, , =, etc.).
Information
 Information is the processed data on which decisions and actions are based. It is
data that has been processed into a form that is meaningful to the recipient and is
of real or perceived value in the current or the prospective action or decision of
recipient. Furtherer more, information is interpreted data; created from organized,
structured, and processed data in a particular context.
Data Processing Cycle

 Data processing is the re-structuring or re-ordering of data by people or machines


to increase their usefulness and add values for a particular purpose.
 These three steps constitute the data processing cycle.
Data types from Computer programming perspective

 A data type is simply an attribute of data that tells the compiler or interpreter how
the programmer intends to use the data.

Common data types include:-


 Integers(int)- is used to store whole numbers, mathematically known as integers

 Booleans(bool)- is used to represent restricted to one of two values: true or false

 •Characters(char)- is used to store a single character

 Floating-point numbers(float)- is used to store real numbers

 Alphanumeric strings(string)- used to store a combination of characters and


numbers
Data types from Data Analytics perspective
 From a data analytics point of view, there are three common types of data types or

structures:

1. Structured:- adheres to a pre-defined data model and is therefore straightforward


to analyze.

Eg:- Excel files or SQL databases.

2. Semi-structured:- does not conform with the formal structure of data models
associated with relational databases or other forms of data tables.
 it is also known as a self-describing structure

Eg:- JSON and XML

3. Unstructured:- either does not have a predefined data model or is not organized in
a pre-defined manner.
Metadata – Data about Data

 It is one of the most important elements for Big Data analysis and big data
solutions.
 Metadata is data about data.

 It provides additional information about a specific set of data.

Example:- In a set of photographs, metadata could describe when and where the
photos were taken.

Because of this reason, metadata is frequently used by Big Data solutions for initial
analysis.
Data value Chain

 It is introduced to describe the information flow within a big data system as a


series of steps needed to generate value & useful insights from data.
 The Big Data Value Chain identifies the following key high-level activities:
Data Acquisition

 It is the process of gathering, filtering, and cleaning data before it is put in a data

warehouse or any other storage solution on which data analysis can be carried out

 Data acquisition is one of the major big data challenges in terms of infrastructure

requirements.
Data Analysis

 Data analysis involves exploring, transforming, and modeling data with the goal

of highlighting relevant data, synthesizing and extracting useful hidden

information with high potential from a business point of view

 It is concerned with making the raw data acquired amenable to use in decision-

making as well as domain-specific usage.

Example:- Related areas include data mining, business intelligence, and machine

learning.
Data Curation

 It is the active management of data over its life cycle to ensure it meets the

necessary data quality requirements for its effective usage.

 Data curators (also known as scientific curators or data annotators) hold the

responsibility of ensuring that data are trustworthy, discoverable, accessible,

reusable and fit their purpose

 Data curation processes can be categorized into different activities such as

content creation, selection, classification, transformation, validation, and

preservation
Data Storage

 It is the persistence and management of data in a scalable way that satisfies the

needs of applications that require fast access to the data.

 Relational Database Management Systems (RDBMS) have been the main, and

almost unique, a solution to the storage paradigm for nearly 40 years.

 However, the ACID (Atomicity, Consistency, Isolation, and Durability) properties

that guarantee database transactions lack flexibility

 NoSQL technologies have been designed with the scalability goal.


Data Usage

 It covers the data-driven business activities that need access to data, its analysis,

and the tools needed to integrate the data analysis within the business activity.

Data usage in business decision-making can enhance competitiveness through:-

 the reduction of costs,

 increased added value,

 any other parameter that can be measured against existing performance criteria
Basic concepts of big data

 Big data is the term for a collection of data sets so large and complex that it

becomes difficult to process using on-hand database management tools or

traditional data processing applications.

 Big data is characterized by 3V and more:

 Volume: large amounts of data Zeta bytes/Massive datasets

 Velocity: Data is live streaming or in motion

 Variety: data comes in many different forms from diverse sources

 Veracity: can we trust the data? How accurate is it? etc.


…cont. Big Data in 4V

Figure 2.4 Characteristics of big data


Clustered Computing

 Big data clustering software combines the resources of many smaller machines,

seeking to provide a number of benefits:

 Resource Pooling:-Combining the available storage space to hold data is a clear

benefit

 High Availability:- Clusters guarantees to prevent hardware or software failures

from affecting access to data & processing

 Easy Scalability:- Clusters make it easy to scale horizontally by adding additional

machines to the group.

Example:- software like Hadoop’s YARN


Hadoop

 Hadoop is an open-source framework intended to make interaction with big data


easier.
 It is inspired by a technical document published by Google.

The four key characteristics of Hadoop are:


 Economical: Its systems are highly economical as ordinary computers can be used
for data processing.
 Reliable: it stores copies of the data on different machines & is resistant to
hardware failure.
 Scalable: both, horizontally & vertically. A few extra nodes help in scaling up the
framework.
 Flexible: stores as much structured & unstructured data
Hadoop Ecosystem

 Hadoop has an ecosystem that has evolved from its four core components: data
management, access, processing, and storage.
Big Data Life Cycle

1. Ingesting data into the system:- transferred to Hadoop from various sources

such as relational databases, systems, or local files

2. Processing the data in storage:- The data is stored in the distributed file system,

HDFS, and the NoSQL distributed data, HBase. Spark and MapReduce perform

data processing.

3. Computing and analyzing data:- the data is analyzed by processing frameworks

such as Pig, Hive, & Impala

4. Visualizing the results:- which is performed by tools such as Hue and Cloudera

Search. the analyzed data can be accessed by users.

You might also like