Oracle Openworld Presentation with Paul Kent (SAS) on Big Data Appliance and SAS on Engineered Systems

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Appliance
for
Customers and Partners
Jean-Pierre Dijcks
Oracle
Big Data Product Management
Paul Kent
SAS
VP Big Data

Oracle Big Data Appliance for Customers and Partners
Big Data Appliance Recap
Why You Should Consider Big Data Appliance
Driving Business Value with SAS on Big Data Appliance
Q&A
2
1
2
3
4

Oracle Big Data Management System
SOURCES
Oracle Database
Oracle Industry
Models
Oracle Advanced
Analytics
Oracle Spatial & Graph
BigDataAppliance
Cloudera Hadoop
Oracle NoSQL Database
Oracle R Advanced
Analytics for Hadoop
Oracle R Distribution
Oracle Database
Oracle Advanced Security
Oracle Advanced
Analytics
Oracle Spatial & Graph
Oracle Exadata
Oracle Big Data
Connectors
Oracle Data
Integrator
B
OracleBigDataSQL

Recap: Big Data Appliance Overview
Big Data Appliance X4-2
Sun Oracle X4-2L Servers with per server:
• 2 * 8 Core Intel Xeon E5 Processors
• 64 GB Memory
• 48TB Disk space
Integrated Software:
• Oracle Linux, Oracle Java VM
• Oracle Big Data SQL*
• Cloudera Distribution of Apache Hadoop – EDH Edition
• Cloudera Manager
• Oracle R Distribution
• Oracle NoSQL Database
4
* Oracle Big Data SQL is separately licensed

Recap: Standard and Modular
5
 Starter Rack is a fully cabled and
configured for growth with 6 servers
 In-Rack Expansion delivers 6 server
modular expansion block
 Full Rack delivers optimal blend of
capacity and expansion options
 Grow by adding rack – up to 18 racks
without additional switches

Recap: Harness Rapid Evolution
6
b
b b
BDA
4.0
BDA 4.0 – Sept 2014
•Big Data SQL
•Node Migration
BDA 3.x – April 2014
•CDH 5.0 (MR2 & YARN)
•AAA Security
•Encryption
BDA 2.x – April 2013
•Starter Rack
•In-Rack Expansion
•EM Integration
BDA 1.0 – Jan 2012
•Initial BDA
•Mammoth Install

Operational Simplicity Simplify Access to ALL Data
7
Core Design Principles for Big Data Appliance

8
• Oracle Big Data SQL
– Oracle SQL on ALL your data
– All Native Oracle SQL Operators
– Smart Scan for Optimized Performance
• Oracle Security
– Govern all Data through a Single Set of
Security Policies

Oracle Big Data SQL – A New Architecture
• Powerful, high-performance SQL on Hadoop
– Full Oracle SQL capabilities on Hadoop
– SQL query processing local to Hadoop nodes
• Simple data integration of Hadoop and Oracle Database
– Single SQL point-of-entry to access all data
– Scalable joins between Hadoop and RDBMS data
• Optimized hardware
– Balanced Configurations
– No bottlenecks
Oracle Confidential – Internal/Restricted/Highly Restricted 9

Big Data SQL
10
SELECT w.sess_id, c.name
FROM web_logs w, customers c
WHERE w.source_country = ‘Brazil’
AND w.cust_id = c.customer_id;
Relevant SQL runs on BDA nodes
10’s of Gigabytes of Data
Only columns and rows needed to
answer query are returned
Hadoop Cluster
B B B
Big Data SQL
Oracle Database
CUSTOMERSWEB_LOGS

Big Data SQL
11
SELECT w.sess_id, c.name
FROM web_logs w, customers c
WHERE w.source_country = ‘Brazil’
AND w.cust_id = c.customer_id;
Relevant SQL runs on BDA nodes
10’s of Gigabytes of Data
Only columns and rows needed to
answer query are returned
Hadoop Cluster
B B B
Big Data SQL
Oracle Database
CUSTOMERSWEB_LOGS
SQL Push Down in Big Data SQL
• Hadoop Scans on Unstructured Data
• WHERE Clause Evaluation
• Column Projection
• Bloom Filters for Better Join Performance
• JSON Parsing, Data Mining Model Evaluation

Feedback Loop
Data Management
Big Data
Platform
(Hadoop/NoSQL)
Relational
Data Warehouse
(OCDM)
Analytic Apps
Customer
Experience
Operations
Monetization
Adapters
ETL/ELT
Adapters
Real-Time
Adapters
Third
Party
Data
Sources
Oracle Comms Apps
(BSS/OSS)
Oracle Comms
Ntwk Products
(Tekelec & Acme)
Other Oracle Apps
(CRM, ERP, etc.)
Third Party Sources
Oracle Communications Data Model
Reference Architecture
To Other Apps
B

13

• No Bottlenecks
• Full Stack Install and Upgrades
• Simplified Management
– Cluster Growth
– Critical Node Migration
• Always Highly Available
• Always Secure
• Very Competitive Price Point
14

Successful Big Data Systems Grow
From Cluster Install with HA to Large Clusters to Dealing with Operational Issues
15
Day 1
• 12 node BDA for Production
• Hadoop HA and Security Set-up
• Ready to Load Data
RCK_1
Full install with a single command:
./mammoth –i rck_1

16
N
N Example Service: Hadoop Name Nodes
Day 1
RCK_1

17
N
RCK_1 RCK_2
Day 90
Add 12 New Nodes across two Racks
N
Cluster expansion with a single command:
mammoth –e newhost1,…,newhostn

18
N
RCK_1 RCK_2
This expansion automatically optimizes HA
setup across multiple racks
N
Cluster Expansion with a single command:
mammoth –e newhost1,…,newhostn
Because of uniform nodes and IB networking,
no data is moved

19
N
RCK_1 RCK_2
N
Day n
Critical Node Failure => Primary Name Node

20
N
RCK_1 RCK_2
N
• Automatic Failover to other
NameNode
• Automatic Service Request to
Oracle for HW Failure

21
N
RCK_1 RCK_2
N
• Restore HA with a Single command
bdacli admin_cluster migrate N1
• Reinstate the Repaired Node with a Single
Command:
bdacli admin_cluster reprovision N1

Operational Simplicity
22
30%
21%
Quicker to Deploy
Cheaper to Buy
“Oracle Big Data Appliance is an excellent
choice for customers looking to work with
the full suite of Cloudera’s leading
Hadoop-based technology. It’s more cost-
effective and quicker to deploy than a DIY
cluster.”
⁻ Mike Olson, Cloudera founder, Chief Strategy
Officer, and Chairman of the Board
Source: ESG White Paper

 Real-time access to better data means better
insights, which means better decisions and better
business results
 Integrate data associated with customer
telemetry, configurations, service history,
diagnostics, knowledge & support information
Big Data Initiative @ Oracle Global Support Services
Anticipate Detect Predict Automate Delight

Operational Simplicity
Simplify Access to ALL Data
24
Core Design Principles Enable Success

There is one more thing…
25
Business Value = Applications

Big Data Appliance powers instant Business Value
26
B BB
Customer Experience
Management
Cyber Security
Solutions
Communications
Data Model

Introducing
27
Paul Kent - SAS

Copyright © 2014, SAS Institute Inc. All rights reserved.
Big Data and Big Analytics –
So Much more Gunpowder!
Paul Kent
VP BigData, SAS Research and Development

1. Change 2. Safari Pics

Copyright © 2014, SAS Institute Inc. All rights reserved.Copyright © 2014, SAS Institute Inc. All rights reserved.
[CON8279] Oracle Big Data Appliance:
Deep Dive and Roadmap for Customers and Partners
Oracle Big Data Appliance is the premier Hadoop appliance in the
market. This session describes the roadmap for customers in the
areas of high-performance SQL on Hadoop and securing big data,
plus overall performance improvements for Hadoop.
A special focus in the session is the roadmap and benefits Oracle
Big Data Appliance brings to Oracle partners.
To illustrate the benefits of running on a standardized and
optimized Hadoop platform, SAS presents the findings of its tests of
SAS In-Memory Analytics on Oracle Big Data Appliance.

Agenda
1. SAS & Oracle Partnership
2. Family Stories
1. Hadoop
2. Oracle Engineered Systems Family
3. SAS Software Family
3. Deployment Patterns

 Reflection on a stronger partnership than ever
 Both leaders in Big Data –
 Jointly solving the most difficult and demanding Big
Data Problems
 Providing simplicity and agility to create flexible
configurations
 Extensive engineering collaboration
 Can we answer:
 How Does it Work?
 How Does it Perform?
2014

Copyr ight © 2012, SAS Institute Inc. All rights reser ved.
THE
TAMOXIFEN
DILEMMA
SOURCE: http://commons.wikimedia.org/wiki/File:Tamoxifen-3D-vdW.png

Elephant :: 3 Good Ideas !!
1. Never forgets
2. Is a good (hard) worker
3. Is a Social Animal (teamwork)

 MPP (Massively Parallel) hardware running database-like software
 “data” is stored in parts, across multiple worker nodes
 “work” operates in parallel ,on the different parts of the table
Controller Worker Nodes
Hadoop – Simplified View

Head Node Data 1 Data 2 Data 3 Data 4…
MYFILE.TXT
..block1 -> block1
..block2 -> block2
..block3 -> block3
Idea #1 - HDFS. Never forgets!

MYFILE.TXT
..block1 -> block1 block1 copy2
..block3 -> block3 copy2 block3

MYFILE.TXT
..block1 -> block1 block1copy2
..block3 -> block3 copy2 block3

Redundancy Wins!

Idea #2 – MapReduce – Send the work to the Data
 We Want the Youngest Person in the Room
 Each Row in the audience is a data node
 I’ll be the coordinator
• From outside to center, accumulate MIN
• Sweep from back to front.
• Youngest Advances

Recap: Standard and Modular
44
 Starter Rack is a fully cabled and
configured for growth with 6 servers
 In-Rack Expansion delivers 6 server
modular expansion block
 Full Rack delivers optimal blend of
capacity and expansion options
 Grow by adding rack – up to 18 racks
without additional switches

Oracle Big Data SQL – A New Architecture
• Powerful, high-performance SQL on Hadoop
– Full Oracle SQL capabilities on Hadoop
– SQL query processing local to Hadoop nodes
• Simple data integration of Hadoop and Oracle Database
– Single SQL point-of-entry to access all data
– Scalable joins between Hadoop and RDBMS data
• Optimized hardware
– Balanced Configurations
– No bottlenecks
Oracle Confidential – Internal/Restricted/Highly Restricted 45

Diversity. It’s a good thing!
Impala Nyala

4 Important Things
#1 Join the Family

HADOOP
Hive QL
SAS
SERVER
SAS ACCESS to Hadoop
#2 Be Familiar

SAS / High Performance Analytics
HADOOP
SAS HPA
Procedures
SAS
SERVER
#3 Use the Cluster!

Prepare Explore / Transform Model
• HPDS2
• HPDMDB
• HPSAMPLE
• HPSUMMARY
• HPCORR
• HPREDUCE
• HPIMPUTE
• HPBIN
• HPLOGISTIC
• HPREG
• HPNEURAL
• HPNLIN
• HPCOUNTREG
• HPMIXED
• HPSEVERITY
• HPFOREST
• HPSVM
• HPDECIDE
• HPQLIM
•HPLSO
•HPSPLIT
•HPTMINE
•HPTMSCORE

Controller
Client

#1 Join the Family
#2 Be Familiar
#3 Use the cluster
#4 Have a pretty face!

4 Important Things (for cluster friendly software)
1.Join the Family
2.Be Familiar
3.Performance
4.Have a pretty face

62
SAS BIG DATA ON BIG DATA APPLIANCE
• Flexible Architectural options for SAS deployments
• Can run on Starter, Half and Full configurations
• Optionally select nodes “N, N-1, N-2, …” for additional SAS
Services such as SAS Compute Tier, SAS MidTier
• Optionally select node subset “N, N-1, N-2, N-3, …) for more
dedicated resources for SAS Analytic Compute Environment by
shifting Big Data Appliance roles
• Option to selectively add more memory on a per node basis
depending on specific workload distribution

63
SAS
Midtier
STARTER BDA
…
…
SAS Visual Analytics
Metadata
Server
SAS
Compute
SAS
HPA
Root
Node
SAS VISUAL ANALYTICS, HIGH-PERFORMANCE ANALYTIC COMPUTE
ENVIRONMENT CO-LOCATED WITH HADOOP

64
SAS
Midtier
STARTER BDA
…
…
Metadata
Server
SAS
Compute
SAS
HPA
Root
Node
Consider:
Extra Memory for 5,6?

65
SAS
Midtier
FULL RACK BDA
…
LASR
Worker
17
HDFS
Data
17
…
…
Metadata
Server
SAS
Compute
SAS
HPA
Root
Node
LASR
Worker
18
HDFS
Data
18

66
FULL RACK BDA ASSEMBLED IN OSC, SYDNEY AUSTRALIA

67

68

69
Basic Smoke Tests Confirmed:
Interoperate with Hadoop and Map Reduce
Read and Write text files to/from HDFS
Read and Write Tabular files to/from Hive (will confirm Oracle BIGSQL in OSC-SC)
Read and Write SAS binary format files to/from HDFS
High Degree Of Parallelism (DOP) reads via Map-Only jobs
SAS LASR server co-exists on/with datanodes
SAS HPA tasks scheduled on datanodes

70
Table 1: Summation of 5/20/100/200 columns;
Baseline: DOP=1 (no parallelism)
120M rows, 400 columns, reg_simtbl_400
SAS High-Performance Analytics Performance
SAS Format Data (SASHDAT)
1107 var
11.795 Mobs
97GB
5.7GB/node
1107 var
73.744 Mobs
608GB
35.7GB/node 6x
Create 208.79 sec 2284.29 sec 11
Scan/Count 24.60 sec 259.38 sec 10.5
HPCORR 295.20 1410.40 4.7
HPCNTREG 336.79 1547.59 4.6
HPREDUCE (u) 236.55 2467.76 10.4
HPREDUCE (s) 219.50 2037.74 9.3

71
OSC-AU FullRack BDA
• 408 Threads
• 600 GB dataset
• 17 servers
Your Problem solved ASAP

72

73

74

75
EXADATA
INTEGRATION
SAS EMBEDDED PROCESSING (EP) TO EXADATA
LEVERAGING BIG DATA SQL
…
SAS
Midtier
LASR
Worker
18
…
HDFS
Data
18
Metadata
Server
SAS
Compute
SAS
HPA
Root
Node
SAS
EP
Big Data SQL

76
SAS EP Parallel Data Feeders
DOP=1 DOP=24 DOP=24
(flash cache)
Add(5) 1.25min 1.5min .5min
Add(20) 2.5min 1.5min .5min
Add(100) 13min 1.5min .6min
Add(200) 16min ~2min 1.25min (10x)

77
Table 2: Scan times for 2 tables
(200 columns, 400 columns, 120M rows);
Baseline: SAS/ACCESS vs. HPA EP feeder
SAS EP Parallel Data Feeders
Access Access /
DBSlice
SAS HPA
Using EP
Reg_sim_200 1:01:12 0:28:37 0:08:00
Reg_sim_400 1:49:11 0:55:33 0:16:05 (7x!)

78
SAS Format Data (SASHDAT) and Oracle EXADATA
1107 var
11.795 Mobs
97GB
5.7GB/node
SASHDAT
907 var
11.795 Mobs
79.7GB
4.7GB/node
EXADATA
1107 var
73.744 Mobs
608GB
35.7GB/node
SASHDAT
Create 208.79 sec 931.22 sec 2284.29 sec
Scan/Count 24.60 sec 956.16 sec 259.38 sec
HPCORR 295.20 833.24 1410.40
HPCNTREG 336.79 756.97 1547.59
HPREDUCE (u) 236.55 1055.11 2467.76
HPREDUCE (s) 219.50 1051.93 2037.74

79
ORACLE ENGINEERED SYSTEMS FOR
SuperClusterExaData ExaLogic Virtual
Compute
Appliance
ZFS
Storage
Appliance
Big Data
Appliance

SAS AND ORACLE WORKING TOGETHER TO CREATE CUSTOMER VALUE
• Joint R & D development and
Product Management teams
in Cary and Redwood
Shores
• Focus on driving SAS
technology components to
run natively in Oracle
database
• Joint performance
engineering optimizations
• Template physical architectures
developed based on use-cases
• Physically tested and
benchmarked together
• Reduction in physical effort
• Overall reduction in lifecycle
costs
• Best Practice papers
• SAS and Oracle Engineers
provide joint "Sizing and
Architecture Analysis and
Design"

SAS AND ORACLE
BETTER TOGETHER
Paul.Kent @ sas.com
@hornpolish
paulmkent

Oracle Openworld Presentation with Paul Kent (SAS) on Big Data Appliance and SAS on Engineered Systems

Oracle Openworld Presentation with Paul Kent (SAS) on Big Data Appliance and SAS on Engineered Systems

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Oracle Openworld Presentation with Paul Kent (SAS) on Big Data Appliance and SAS on Engineered Systems (20)

Recently uploaded (20)

Oracle Openworld Presentation with Paul Kent (SAS) on Big Data Appliance and SAS on Engineered Systems