Introduction to Big Data Analytics

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 1
 Introduction to Big Data Analytics

Your Thoughts?

What is Big Data?

What makes data, “Big” Data?

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 2
 Big Data Defined
• “Big Data” is data whose scale, distribution, diversity,
and/or timeliness require the use of new technical
architectures and analytics to enable insights that unlock
new sources of business value.
 Requires new data architectures, analytic sandboxes
 New tools
 New analytical methods
 Integrating multiple skills into new role of data scientist

• Organizations are deriving business benefit from analyzing

ever larger and more complex data sets that increasingly
require real-time or near-real time capabilities
Source: McKinsey May 2011 article Big Data: The next frontier for innovation, competition, and productivity

2
EMC PROVEN PROFESSIONAL
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA
 Key Characteristics of Big Data
1. Data Volume
 44x increase from 2010 to 2020
(1.2zettabytes to 35.2zb)

2. Processing Complexity
 Changing data structures
 Use cases warranting additional transformations and
analytical techniques

3. Data Structure
 Greater variety of data structures to mine and analyze

2
EMC PROVEN PROFESSIONAL
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA
 Big Data Characteristics:
S Data Structures
Data Growth e is Increasingly Unstructured
“ m
Q i • Data containing a defined data type, format, structure
u -
• Example: Transaction data and OLAP
a S
s t • Textual data files with a discernable pattern,
i r
More Structured

enabling parsing

” u • Example: XML data files that are self

U c describing and defined by an xml schema

n S t • Textual data with erratic data formats, can

be formatted with effort, tools, and time
s t u
t r r • Example: Web clickstream data that
may contain some inconsistencies in data
r u e values and formats
• Data that has no inherent
u c d structure and is usually stored
as different types of files.
c t
t u • Example: Text documents,
PDFs, images and video
u r
r ePROFESSIONAL
2
EMC PROVEN

e2012 EMC Corporation. All Rights Reserved.

Copyright © Module 1: Introduction to BDA 5
 Four Main Types of Data Structures
Structured Data Quasi-Structured Data

Semi-Structured Data
View  Source

http://www.google.com/
#hl=en&sugexp=kjrmc&cp=8&gs_id=2m&xhr=t&q=data+scientist&pq=big+data&pf=p&sclien
t=psyb&source=hp&pbx=1&oq=data+sci&aq=0&aqi=g4&aql=f&gs_sm=&gs_upl=&bav=on.2,
or.r_gc.r_pw.,cf.osb&fp=d566e0fbd09c8604&biw=1382&bih=651

Unstructured Data
The Red Wheelbarrow, by
William Carlos Williams

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 6
 Data Repositories, An Analyst Perspective
Data Islands Data Warehouses Analytic Sandbox
“Spreadmarts”
Centralized data containers Data assets gathered from multiple
Isolated data marts in a purpose-built space sources and technologies for analysis

• Spreadsheets and low- • Supports BI and reporting, but • Enables high performance analytics
volume DB‘s for restricts robust analyses using in-db processing
recordkeeping • Analyst dependent on IT & • Reduces costs associated with data
• Analyst dependent on DBAs for data access and replication into "shadow" file
data extracts schema changes systems
• Analysts must spend significant • “Analyst-owned” rather than “DBA
time to get extracts from owned”
multiple sources

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 7
 Introduction to Big Data Analytics: Mini-Case Study
Yoyodyne Bank Scenario
• Evolving from small community bank to a global bank
• Needs to move away from its legacy mainframes to an environment that
supports more robust analytics
• Growing through mergers and acquisitions
• Subject to many new regulatory requirements
• Increasing customer base and increased product offerings
Your Thoughts?

Discussion Questions
1. Discuss how the bank’s data would change under these circumstances.
2. How are their needs changing with these business changes?
3. What do you need to consider from an analyst point of view? What are
some things to consider implementing as the bank grows?

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 9
 Business Drivers for Analytics
Current Business Problems Provide Opportunities for Organizations to
Become More Analytical & Data Driven
Driver Examples
1
Desire to optimize business
Sales, pricing, profitability, efficiency
operations

2
Desire to identify business risk Customer churn, fraud, default

3
Predict new business opportunities Upsell, cross-sell, best new customer prospects

4
Comply with laws or regulatory
Anti-Money Laundering, Fair Lending, Basel II
requirements

2
EMC PROVEN PROFESSIONAL
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 10
 Analytical Approaches for Meeting Business Drivers
Business Intelligence vs. Data Science
Predictive Analytics & Data Mining
(Data Science)
Typical • Optimization, predictive modeling,
Techniques & forecasting, statistical analysis
Data Types • Structured/unstructured data, many types
of sources, very large data sets

High Common • What if…..?

Questions • What’s the optimal scenario for our
business ?
• What will happen next? What if these
trends continue? Why is this happening?
Data
Science Business Intelligence
BUSINESS Typical • Standard and ad hoc reporting,
Techniques & dashboards, alerts, queries, details on
VALUE Data Types demand
Business • Structured data, traditional sources,
Intelligence manageable data sets
Common • What happened last quarter?
Questions • How many did we sell?
• Where is the problem? In which
situations?
Low

Past TIME Future

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 11
 A Typical Analytical Architecture
1 Data
Sources

Non-Agile Models

2 Departmental
“Spread
Marts”
Warehouse

Enterprise 4
Departmental Applications
Warehouse
3 Prioritized
Operational
Processes

Static schemas
accrete over time Reporting Siloed
Analytics

Non-Prioritized Data Provisioning

Errant data & marts

2
EMC PROVEN PROFESSIONAL
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 12
 Implications of Typical Architecture for Data Science

• High-value data is hard to reach and leverage

• Predictive analytics & data mining activities are last
in line for data
 Queued after prioritized operational processes
• Data is moving in batches from EDW to local Slow
“time-to-insight”
analytical tools
&
 In-memory analytics (such as R, SAS, SPSS, Excel)
reduced
 Sampling can skew model accuracy business impact
• Isolated, ad hoc analytic projects, rather than
centrally-managed harnessing of analytics
 Non-standardized initiatives
 Frequently, not aligned with corporate business goals

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 14
 Opportunities for a New Approach to Analytics
New Applications Driving Data Volume

MEASURED IN MEASURED IN WILL BE MEASURED IN

LARGE TERABYTES PETABYTES EXABYTES
1TB = 1,000GB 1PB = 1,000TB 1EB = 1,000PB
VOLUME OF INFORMATION

SMALL

1990’s 2000’s 2010’s

(RDBMS & DATA (CONTENT & DIGITAL ASSET (NO-SQL & KEY/VALUE)
WAREHOUSE) MANAGEMENT)
2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 15
 Opportunities for a New Approach to Analytics
Big Data Ecosystem
1
Data
Devices
Individual

Analytic Medical Information

Services Brokers Advertising Marketers Employers
Law
Enforcement
Government Internet
2
Data
Websites
3
Collectors Data
Aggregators

Data
Users/Buyers
Catalog
4 Co-Ops
Phone/TV Retail
Media

Private
Media Credit List Investigators
Archives Bureaus Financial Brokers Delivery /Lawyers
Banks Service
Government

2
EMC PROVEN PROFESSIONAL
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 16
 Considerations for Big Data Analytics
Criteria for Big Data Projects New Analytic Architecture

Analytic Sandbox
Data assets gathered from multiple sources
1. Speed of decision making and technologies for analysis

2. Throughput
• Enables high performance analytics
using in-db processing
3. Analysis flexibility • Reduces costs associated with data
replication into "shadow" file
systems
• “Analyst-owned” rather than “DBA
owned”

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 19
 State of the Practice in Analytics: Mini-Case Study
Big Data Enabled Loan Processing at Yoyodyne

Traditional Big Data Enabled

Underwriting Underwriting Your Thoughts?
Risk Level Risk Level
Underwriting Risk

e t al
om on en y ing ais
c
In ati ym o r c or tory pr
ic plo is
t S is Ap
e rif Em H edit d H
V Cr An

TRADITIONAL DATA LEVERAGED BIG DATA LEVERAGED

2
EMC PROVEN PROFESSIONAL
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 20
 Three Key Roles of the New Data Ecosystem

Data Scientists Role Role Description

Projected U.S. People with advanced training in
talent gap: Deep Analytical quantitative disciplines, such as
140,000 to Talent mathematics, statistics, and machine
190,000 learning.
People with a basic knowledge of statistics
Data Savvy
and/or machine learning, who can define
Analysts & Data Professionals
key questions that can be answered using
Savvy Managers advanced analytics
Projected U.S. People providing technical expertise to
talent gap: 1.5 Technology & Data support analytical projects. Skills sets
million Enablers including computer programming and
database administration

Note: Figures above reflect a projected talent gap in US in 2018, as shown in McKinsey May 2011 article Big Data: The next frontier for innovation,
competition, and productivity

2
EMC PROVEN PROFESSIONAL
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 21
 Roles Needed for Analytical Projects
Data Scientist Key Activities
Data Scientists
Key Activities Data Data Bl LOB
Engineers Analyst Analyst
• Reframe business User

challenges as analytics
challenges Analytic Productivity Platform

• Design, implement and

Tools & Services
deploy statistical models
and data mining Data
Data Access & Query Platform
techniques on big data Admin

• Create insights that lead

to actionable
recommendations Cloud Infrastructure

2
EMC PROVEN PROFESSIONAL
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Intro duction to BDA 22
 Profile of a Data Scientist

Quantitative

Curious &
Technical
Creative

Skeptical Communicative
& Collaborative

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 23
 Big Data Analytics: Industry Examples

1
Health Care
• Reducing Cost of Care Medical

2 Public Services Government Internet

• Preventing Pandemics
3 Life Sciences Data
Collectors
• Genomic Mapping

4 IT Infrastructure
• Unstructured Data Analysis
Phone/TV Retail

5 Online Services
Financial
• Social Media for Professionals

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 24
 1
Big Data Analytics: Healthcare

• Poor police response and problems with medical care, triggered

Situation by shooting of a Rutgers student
• The event drove local doctor to map crime data and examine
local health care

• Dr. Jeffrey Brenner generated his own crime maps from medical
Use of Big Data billing records of 3 hospitals

• City hospitals & ER’s provided expensive care, low quality care
• Reduced hospital costs by 56% by realizing that 80% of city’s
Key medical costs came from 13% of its residents, mainly low-
Outcomes income or elderly
• Now offers preventative care over the phone or through home
visits

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 25
 2
Big Data Analytics: Public Services

• Threat of global pandemics has increased exponentially

Situation
• Pandemics spreads at faster rates, more resistant to antibiotics

• Created a network of viral listening posts

• Combines data from viral discovery in the field, research in
Use of Big Data disease hotspots, and social media trends
• Using Big Data to make accurate predications on spread of new
pandemics
• Identified a fifth form of human malaria, including its origin

Key • Identified why efforts failed to control swine flu

Outcomes
• Proposing more proactive approaches to preventing outbreaks

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 26
 3
Big Data Analytics: Life Sciences

Situation • Broad Institute (MIT & Harvard) mapping the Human Genome

• In 13 yrs, mapped 3 billion genetic base pairs; 8 petabytes

Use of Big Data

• Developed 30+ software packages, now shared publicly, along
with the genomic data

• Using genetic mappings to identify cellular mutations causing

Key cancer and other serious diseases
Outcomes
• Innovating how genomic research informs new pharmaceutical
drugs

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 27
 4
Big Data Analytics: IT Infrastructure

• Explosion of unstructured data required new technology to

Situation
analyze quickly, and efficiently

• Doug Cutting created Hadoop to divide large processing tasks

into smaller tasks across many computers
Use of Big Data
• Analyzes social media data generated by hundreds of
thousands of users

• New York Times used Hadoop to transform its entire public

Key archive, from 1851 to 1922, into 11 million PDF files in 24 hrs
Outcomes
• Applications range from social media, sentiment analysis,
wartime chatter, natural language processing

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 28
 5
Big Data Analytics: Online Services

Situation • Opportunity to create social media space for professionals

• Collects and analyzes data from over 100 million users

Use of Big Data
• Adding 1 million new users per week

• LinkedIn Skills, InMaps, Job Recommendations, Recruiting

Key
Outcomes • Established a diverse data scientist group, as founder believes
this is the start of Big Data revolution

2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 29