Big data/Hadoop/HANA Basics

[

How Big Data Technologies Provide
Solutions for Big Data Problems
John Choate – PMMS SIG Chair
David Burdett – Strategic Technology Advisor, SAP
Henrik Wagner, Global SAP Lead-Alliances, EMC Corp

[ The Challenge of Big Data

Decision-Maker

Customer

LOB User

Data
IT Developer

2

Analyst

[ The 5 Part Series
 Webinar 1: Why Big Data matters, how it can fit into your Business and
Technology Roadmap, and how it can enable your business!
 Webinar 2: How Big Data technologies provide Solutions for Big Data
problems
 Webinar 3: Using Hadoop in an SAP Landscape with HANA

 Webinar 4: Leveraging Hadoop with SAP HANA smart data access
 Webinar 5: Using SAP Data Services with Hadoop and SAP HANA

Resources …
Webinar Registration
1. Go to www.saphana.com
2. Search “ASUG Big Data Webinar”
3. Registration links in blog …
Big Data, Hadoop and Hana – How they Integrate and How they Enable your Business!
Info on SAP and Big Data – go to www.sapbigdata.com

3

[ AREAS TO COVER
SETTING THE STAGE

MARKET

TECHNOLOGY

USE CASES

SUMMARY

4

[

How did we get here?

Facebook: 1 billion users; 600 mobile users; more
than 42 million pages and 9 million apps
Youtube:
More people have mobile 4 billion views per day
phones thanGoogle+: 400 million registered users
electricity or
safe drinking watermillion monthly connected users
Skype: 250

REAL TIME

3,000,000
1,000,000+
SOLD

people had access to
internet worldwide

BIG DATA

SOCIAL
MOBILE

PERSONAL
COMPUTER AND
CLIENT SERVER
DATABASE
(CIRCA 1980)

1990

B2B / B2C

WWW

ANALYTICS
(CIRCA 1980)

PREDICTIVE ANALYTICS
(CIRCA 1980)

2000

SEMANTIC ANALYTICS
(CIRCA 1980)

2005

2010

2015
2013

5

[

How big is Big Data?

Today we measure available data
in zettabytes (1 trillion gigabytes)
IN 2011, THE AMOUNT
OF DATA SURPASSED
90% OF THE WORLD DATA TODAY
has been created
in the last two years alone!

1.8
ZETTABYTES

Eight 32GB iPads
per person alive
in the world

6

[ Big Data Simplified
Definition
 “Big data” is high-volume, velocity and -variety
information assets that
demand cost-effective,
innovative forms of
information processing for
enhanced insight and
decision making

Gartner

7

Three Key Parts
 Part One: 3V’s – Volume,
Velocity, Variety
 Part Two: Cost-Effective,
Innovative Forms of
Information Processing
 Part Three: Enhanced
insight for “Real Time”
decision making

[ The 7 Key Drivers Behind the Big Data Movement? *
* http://hortonworks.com/blog/7-key-drivers-for-the-big-data-market/

 Business
 Opportunity to enable innovative new business models
 Potential for new insights that drive competitive advantage

 Technical
 Data collected and stored continues to grow exponentially
 Data is increasingly everywhere and in many formats
 Traditional solutions are failing under new requirements

 Financial
 Cost of data systems, as a percentage of IT spend, continues to
grow
 Cost advantages of commodity hardware & open source
software
8

[ Todays Key Challenges in Big Data




Data Analytics
1.
Data Capture & Retention – What data should be kept and why
2.
Behavioral Analytics – Understanding and leveraging customer behavior
3.
Predictive Analytics – Using new data types (sentiment, clickstream, video, image and
text) to predict future events



9

Information Strategy
1.
Which investments will deliver most business value and ROI?
2.
Governance – New expectations for data quality and management
3.
Talent – How will you assemble the right teams and align skills?

Enterprise Information Management (EIM)
1.
User expectations – Making “Big Data” accessible for the end user in “real-time”
2.
Costs – How to provide access to big data in a rapid and cost-effective way to support
better decision-making?
3.
Tools – Have you identified the processes, tools and technologies you need to support
big data in your enterprise?

[PRESENTATION CONTENT
 SETTING THE STAGE

 MARKET

 TECHNOLOGY

 USE CASES

 SUMMARY
10

[ The RAPIDLY GROWING Market
“By 2015, 4.4 million IT jobs globally will
be created to support big data, generating
1.9 million IT jobs in the United States”
Peter Sondergaard, Senior Vice President at Gartner and
global head of Research
http://www.gartner.com/newsroom/id/2207915

“The Global big data market is estimated to be
$14.87 billion in 2013 and expected to
grow to $46.34 billion … an estimated
Compounded Annual Growth Rate (CAGR) of
25.52% from 2013 to 2018”
“IDC expects the Big Data technology and
services market to grow at a 31.7% compound
annual growth rate through 2016”
http://www.idc.com/getdoc.jsp?containerId=238746

11

http://www.marketsandmarkets.com/PressReleases/big-data.asp

[ Products and Services under the Umbrella of Big Data
 Hadoop software and related
hardware
 NoSQL database software and
related hardware
 Next-generation data
warehouses/analytic database
software and related hardware
 Non-Hadoop Big Data platforms,
software, and related hardware
 In-memory – both DRAM and
flash – databases as applied to Big
Data workloads
 Data integration and data quality
platforms and tools as applied to
Big Data deployments
12

 Advanced analytics and data
science platforms and tools
 Application development
platforms and tools as applied to
Big Data use cases
 Business intelligence and data
visualization platforms and tools as
applied to Big Data use cases
 Analytic and transactional
applications as applied to Big Data
use cases
 Big Data support, training, and
professional services

[ WHO IS SPENDING $$$ ON BIG DATA ?
 COMPANIES

 INDUSTRIES

 Median = $10M

 MOST

 25% Spend less $2.5M
 15% Spend greater
$100M
 7% Spend greater than
$500M
13






Banking
High Tech
Telecommunications
Travel

 LEAST
 Energy/Resources
 Life Sciences
 Retail
2012 Tata Consulting Services (TCS)
Global Study

[ How the market is growing

Wikibon:
http://wikibon.org/wiki/v/Big_Data_Vendor_Revenue_
and_Market_Forecast_2012-2017

Wikibon:
http://wikibon.org/vault/Special:FilePath/2012BigDataSegment
Growth20112017.png

Fastest growing area is Applications (49% CAGR), 2012-17
14

[ Big Data Vendor Revenue

Big Data vendors are a
mix of established
players and pure-plays
Source Data:
http://wikibon.org/wiki/v/Big_Data_Vendor_Revenue_and_Marke
t_Forecast_2012-2017

15

[ 10 Big Data Trends Changing the Face of Business
1. Machine Data and the Internet of
Things Takes Center Stage

6. Large Companies Are Increasingly
Turning to Big Data

2. Compound Applications That
Combine Data Sets to Create Value

7. Most Companies Spend Very Little, A
Few Spend A Lot

3. Explosion of Innovation Built on
Open Source Big Data Tools
4. Companies Taking a Proactive
Approach to Identifying Where Big Data
Can Have an Impact

5. There Are More Actual Production
Big Data Projects
16

8. Investments Are Geared Toward
Generating and Maintaining Revenue
9. The Greatest ROI of Big Data Is
Coming from the Logistics and Finance
Functions
10. The Biggest Challenges Are as Much
Cultural as Technological


 MARKET

 TECHNOLOGY

 USE CASES

 SUMMARY
17

[ Aspect of Time Value of Data
 “HOT” Data may be better suited for “In Memory” HANA
residency. This data largely derived from structured SAP
sources.
 “WARM” and “COLD” Data may be better suited for
HADOOP residency. This data is largely unstructured in
nature and may present very large data sets (multi PB).
 Business value reflected by Use Cases may consist of queries
and data structures in three different ways:
 Enabled by SAP HANA
 Enabled by HADOOP
 Enabled by HANA and HADOOP simultaneously

18

EMC
Corporation

[ SAP’s Technology Use Case View

EMC Corporation

19

[ Big Data High Level Software Architecture
Big Data Storage holds the data in memory or on
SSD/HDD



Big Data Database Software manages data in the Big
Data Storage. Includes SQL and NoSQL DBMS.



Processing Engines are software that can process /
manipulate data in the Big Data Storage


Processing Engines
Software

Analytic Software analyzes data using the Processing
Engines or Big Data DB Software



Big Data Applications provide solutions for specific
business problems

In-memory



Development Software is used to build Big Data
Applications



Visualization Software presents the results to end
users from Analytic Software or Big Data Applications

Data Capture Software



Data Capture Software on-boards and manages data
from multiple Data Sources

Data Sources

Development
Software





Management Software handles operational of the Big
Data implementation / solution

Big Data
Applications

Analytic
Software

Big Data
Database Software

Visualization
Software

Management
Software

Big Data Storage

20

SSD
HDD

Big Data
Hive/HBase
Database Software

Visualization
Software

Mahout/
Processing Engines
Software
Giraph, etc

Big Data
Applications

Analytic
Software

Big Data
Cassandra
Database Software

Data Sources

Hadoop

Big Data Storage
Cassandra
Management
Software

Management
Software


Big Data
Applications

Analytic
Software

Big Data
MongoDB
Database Software

Data Sources

Cassandra

Software

In-memory

Big Data Storage
MongoDB

SSD

HDD

Visualization

Processing Engines
Software

In-memory

SSD

HDD

Software

Processing Engines
Software

In-memory

Big Data Storage
Hadoop HDFS

Visualization

Development
Software

Analytic
Software

Management
Software

Big Data
Applications

Development
Software

Development
Software

[ Big Data Software Other Solutions

Data Sources

MongoDB

Big Data Software solutions only handle part of the problem
21

SSD

HDD

[ Big Data Software Architecture and HANA

Development
HANA Studio
Software

ANALYZE – Analytics!

Big Data
Applications

Analytic
SAP BI
Software
Tools

Big Data
HANA / Sybase IQ
Database
Software

Visualizatio
SAP Lumira
n Software

Processing
“R” Engine, Text
Engines
Analytics, etc.
Software

SAP Landscape
Management
Management
Software

In-memory

22

SAP HANA
Sybase IQ
Big Data Storage
Hadoop HDFS

DataSAP Data Services
Capture Software
Data Sources

SSD
HDD

 Analyze and visualize Big Data using tools that best serve your
business needs.
 Reduce delays associated with complex analysis of large data sets
using in-memory analytics.
 New opportunities and expose hidden risks using algorithms, R
integration, and predictive analysis.
 Enable business users to access and visualize insight using charts,
graphs, maps, and more.
 Uncover hidden value from unstructured data with text analytics.

ACELERATE – “Real Time” Visibility
 Increase business speed with cost-performance data processing
options
 In-memory processing with SAP HANA to massively parallel
processing with the SAP Sybase IQ database
 Distributed processing of large data sets with Hadoop.

ACQUIRE – Meet the Expanding Data Demand
 Acquire and store large volumes of data from a variety of data sources.
 Flexible data management capabilities delivered via the SAP HANA
platform.
 Best option based on business requirements for accessibility,
complexity of analytics, processing speed, and storage costs.
See: http://www.sapbigdata.com/platform/


 MARKET

 TECHNOLOGY

 USE CASES

 SUMMARY
23

[ Looking for Big Data Potential in your Company
ACQUIRE – Meet the Expanding Data Demand

1. Acquire and store large volumes of data from a variety of data sources.
2. Flexible data management capabilities delivered via the SAP HANA platform.
3. Best option based on business requirements for accessibility, complexity of analytics, processing speed,
and storage costs.
ACELERATE – “Real Time” Visibility
1. Increase business speed with cost-performance data processing options
2. In-memory processing with SAP HANA to massively parallel processing with the SAP Sybase IQ
database
3. Distributed processing of large data sets with Hadoop.
ANALYZE – Analytics!
1. Analyze and visualize Big Data using tools that best serve your business needs.

2.
3.
4.
5.

24

Reduce delays associated with complex analysis of large data sets using in-memory analytics.
New opportunities and expose hidden risks using algorithms, R integration, and predictive analysis.
Enable business users to access and visualize insight using charts, graphs, maps, and more.
Uncover hidden value from unstructured data with text analytics.

[ OVERCOMING OBJECTIONS – USE CASES
1. Big Data Projects are too expensive
2. Big Data is Technology in search of a Business Problem to solve!
3. Big Data is an IT project, we don’t need to involve the business.
4. Big Data is just the new Buzzword phrase, just like Cloud! Soon another
trend and new buzzword will come along.
5. We don’t have the skills to use Big Data Solutions.

25

[ Big Data and Competitive Advantage
Utilize your data to gain a
competitive advantage!
Competitiveness of fact-finders vs. fumblers

Fumblers

Fumblers

Leading businesses can outpace the competition
because they can:
• Base decisions on the latest, granular
multi-structured data
• Make decisions on analytics rather than
intuition

Factfinders

Factfinders

• Frequently reassess forecasts and plans
• Utilize analytics to support a spectrum
of strategic, operational and tactical decision making

• Rapidly evaluate alternative scenarios
Laggards

Leaders
n=1,002
Source: IDC‘s SAP HANA Market Assessment, August 2011

26

[ Soliciting Allies
 REVENUE

 ROI

 Sales

 Finance

 Marketing

 Logistics

 Customer Service

 Marketing

 R&D/NPI

 Sales

 IT
 Finance

 Greater 25%

 HR
27

2012 Tata Consulting
Services (TCS) Global
Study

[ T-Mobile USA, Inc.

Telecom – Optimize Marketing Campaigns Effectiveness
Product: Agile Datamart

56x faster analysis

5 Billion+ records
for 33M customers
report executed in 9
seconds

Business Challenges
 Proliferation of offers/micro-offers increasingly strategic in a highly
competitive market
 Marketing Operations needs to collect, analyze and report on results of
campaigns/offers very quickly and with great flexibility
 Current and future campaigns have to be fine tuned to improve
customer adoption and profitability
Technical Challenges
 Data for 33M customers required a lot of time to be explored and
analyzed in detail with previous technology

Benefits
 Dynamic read outs on the upsell/cross sell performance of store and
call centers
 Easy, fast assess to the performance of all campaigns (e.g. by geo, by
store, etc)
 Quicker forecast of the financial impact of marketing campaigns

“ ”

Based on the rapid analytics that we’re performing on SAP HANA, we are now able to quickly fine tune our current and future campaigns to
improve the customer adoption rate, reduce churn and increase profit
Alison Bessho, Director, Enterprise Systems Business Solutions, T-Mobile USA

28

[ University of Kentucky
Higher Education – Student Retention
Business Challenges

$1.1M increase in
revenue with 1% increase
in retention rate

 Enable the University to increase student retention and thus increase the
Graduation Rate from 60% to 70% over a 10 Year period
 Huge costs and longer turnaround time for student classification to
improve student satisfaction and the retention rate

420x improvement


in reporting speed: It

 Lack of speed, accuracy and visibility into data analysis

took 2-3 seconds as
against the
competition Oracle DW
which took 15-20
minutes

 Handling Big data efficiently: SAP ECC V6 production system is 1.5 TB and
SAP BW V7 and Oracle Data Warehouse combined is 4 TB
Benefits

15x improvement in

 Increased Student Retention Rate, fast collect new information related to
student interactions and various student behaviors

Query load time

 Reduced IT Infrastructure Costs and increased IT FTE productivity

“”

 Allow the University to retire several systems including
Informatica, BI Web Focus (IBI), and Oracle (DB)

SAP HANA offers an effective real-time data driven system which is essential to giving immediate performance feedback and increase
retention rate of students, increasing millions in revenue for the University every year.
Vince Kellen, CIO University of Kentucky

29

[ Hardware Preventative Maintenance
Business Challenges
 A computer server manufacturer wants to implement effective preventative maintenance
by identifying problems as they arise then take prompt action to prevent the problem
occurring at other customer sites
 Identifying problems by analyzing text data from call centers, customer questionnaires
together with server logs generated by their hardware
 Combining results with CRM, sales and manufacturing data to predict which servers are
likely to have problems in the future
Solution
 Use SAP Data Services to analyze call center data and questionnaires stored in Hadoop and
identify potential problems
 Use HANA to merge results from Hadoop with server logs to identify indicators in those
logs of potential problems
 Combine with CRM, bill of material and production/manufacturing data to identify cases
where preventative maintenance would help

30

[ Data Warehouse Migration
Business Challenges
 A high tech company with a major web presence uses non-SAP software for its data warehouse to analyze
the activity on their web site properties and combine it with data in SAP Business Suite
 They want to both reduce the cost and improve the responsiveness of their data warehouse solutions by
moving to a combination of SAP HANA and Hadoop

 How to complete the migration without disrupting existing reporting processes
Solution – this was a four step process

 Step 1. Replicate Data in Hadoop. SAP Data Services is used to replicate in Hadoop all data from web
logs and SAP Business Suite being captured by the current Data Warehouse
 Step 2. Aggregate Data in Hadoop. The aggregation process in the existing Data Warehouse is reimplemented in Hadoop and the aggregate results fed back to the existing Data Warehouse
significantly reducing its workload.
 Step 3. Copy the Aggregate Data to HANA. The aggregate data created by Hadoop is also copied to
HANA together with historical aggregate data already in the existing Data Warehouse. The result is
that eventually HANA has a complete copy of the data in the existing Data Warehouse.
 Step 4. Replace Reporting by SAP HANA. New reports are developed in HANA to replace reports in the
original Data Warehouse. Once complete, the original Data Warehouse will be decommissioned.
The end result is a faster, more responsive and lower cost Data Warehouse built on HANA and Hadoop.

31


 MARKET

 TECHNOLOGY

 CASE STUDY

 SUMMARY
32

[ SUMMARY
1. The Big data Market Is Not Going Away!
2. There are 3 Distinct Components of BD Market
3. Its Not a New Trend but way for Technology To

Enable Your Business
4. Case Studies HELP to visualize your own Companies
BD Opportunities – Benchmark & Assess!

5. Don’t go the Journey Alone – There are many
resources available to make your Journey Successful!

33

[ The 5 Part Series
 Webinar 1: Why Big Data matters, how it can fit into your Business and
Technology Roadmap, and how it can enable your business!

 Webinar 2: How Big Data technologies provide Solutions for Big Data problems
 Webinar 3: Using Hadoop in an SAP Landscape with HANA

 Webinar 4: Leveraging Hadoop with SAP HANA smart data access
 Webinar 5: Using SAP Data Services with Hadoop and SAP HANA

Resources …
Webinar Registration
1. Go to www.saphana.com
2. Search “ASUG Big Data Webinar”
3. Registration links in blog …
Big Data, Hadoop and Hana – How they Integrate and How they Enable your Business!

Info on SAP and Big Data – go to www.sapbigdata.com

35

THANK YOU FOR PARTICIPATING.
SESSION CODE:

Learn more year-round at www.asug.com

Big data/Hadoop/HANA Basics

Recommended

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to Big data/Hadoop/HANA Basics (20)

More from Global Business Solutions SME (9)

Recently uploaded (20)

Big data/Hadoop/HANA Basics