Meetup Oracle Database BCN: 2.1 Data Management Trends

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted
Data Management Trends
David Mauri Gómez
Cloud Platform Solution Architect – Oracle
April 10, 2018

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 2
Data Management Evolution
Transactional
Data Warehouse
SQL
Social, Web
Data Lake
IoT
Fast Data

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Decision Diagram
3
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
Concurrency
Complex Query Response Times
Single Record Read/Write
Performance
Bulk Write Performance
Privileged User Security
General User Security
Governance Tools
System per TB Cost
Backup per TB Cost
Skills Acquisition Cost
RDBMS
NoSQL DB
Hadoop

High-level Comparison
HDFS NoSQL RDBMS
Data Type Chunk Record Transaction
Write Type Synchronous Eventually Consistent ACID Compliant
Data Preparation No Parsing No Parsing Parsing and Validation
DR Type Second Cluster Node Replica Second RDBMS
DR Unit File Record Transaction
DR Timing Batch Record Transaction
Complex Analytics? Yes No Yes
Query Speed Slow Fast for simple questions Fast
# of Data Access Methods One (full table scan) One (index lookup) Many (Optimized)
4
IngestDRAcces
Affordable Scale Low Predictable Latency Flexible Performance

Unified Data Management
Data of any type
Any data source
Analysis of any typeSQL GraphSpark Spatial
Machine
Learning
SQL
Access with any language
node.jsJavaREST Python ScalaR

Big Data SQL
The Best of Both Worlds
6
SQL

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 7
Oracle Unified Data Management Solution
Conventional view of
Data Management
Emerging view of
Data Management
Oracle Big Data SQL

Storage Layer
8
Big Data SQL: Another Hadoop Processing Engine
Filesystem (HDFS)
NoSQL Databases
(Oracle NoSQL DB, HBase)
Resource Management (YARN, cgroups)
Processing Layer
MapReduce
and Hive
Spark Impala Search
Big Data
SQL
Meta
data
Store

Hive
DN
DN
DN
DN
ORACLE SQL Engine
Storage
Table Table
Big Data-enabled
Oracle Tables
Python GraphRnode.js JavaREST SQL
Data Local Processing
Big Data SQL Cells
Leverage Metadata
Big Data SQL Architecture
Oracle Big Data SQL
9

Anatomy of a Big Data SQL Cell
10
Smart Scan
I/O Stream Data Transfer
Convert to
Oracle “block”
format
Apply
Smart Scan and
other optimizations

Big Data SQL Goals
Easily access any data
across big data stores
Provides a unified security
model across the sources
Analyze all data using
Oracle’s rich SQL dialect
Fast performance using Big
Data SQL Smart Scan
11

I/O Elimination
• Storage index
• Hive partition pruning
• Predicate and column
pushdown parquet and ORC
12
Big Data SQL key features
Data Movement
Elimination
• Smart Scan performs final
filtering pass to ensure only
requested elements are sent
to Oracle Database
Security
• Apply Oracle Database
security policies on non-
Oracle data stores

Big Data SQL Security Features
Hadoop Security
ACL’s | Sentry | HDFS Encryption | Encryption in Motion
13

Big Data SQL Security Features
• Same security models apply
to a wider range of data
stores
• Advanced features such as
data redaction can now be
applied enabling joins
between disparate sources
• Oracle security layers on
top of existing Hadoop
functionality
Hadoop Security
ACL’s | Sentry | HDFS Encryption | Encryption in Motion
14

Data Lifecycle Management & Query Offload
More data on-line and available at a lower cost
Move Partition to BDA
Oracle Big Data SQL
Rolling 13
months
Month 14-n
Big Data Rolling Windows
• Process
• Copy older partition to BDA
• Update views
• Drop older Exadata partition
• Offloaded data can be accessed
via Oracle & Hadoop
• No Application changes required
15

Database Data in HDFS
16
Hybrid Partitioned Tables
…JAN 2014 FEB 2014 MAR 2014 OCT 2016 NOV 2016 DEC 2016
HDFS
Orders
Database
OCT 2016 NOV 2016 DEC 2016
JAN 2014 FEB 2014 MAR 2014
1 All Partitions are stored internally
2 Some Partitions are moved Externally
3 Mixed Storage for Partitions
No top level changes to Orders

Archive Data: Big Data SQL Implementation Options
1. Table Storage Split Across Tiers 2. View Combines Data Sources
HDFS
HDFSDATABASE
VIEW
DATABASE
TABLE
HDFS
DATABASE

Traditional vs. Oracle Machine Learning/Predictive Analytics
• Traditional— “Move the data” —“Don’t move the data!”
18

Traditional vs. Oracle Machine Learning/Predictive Analytics
• Traditional— “Move the data” — “Move the algorithms”
19
Simpler, Smarter Data Management
+ Analytics / Machine Learning Architecture

Data Science
Oracle Confidential – Internal/Restricted/Highly Restricted 20

• Strengths
– Powerful & Extensible
– Graphical & Extensive statistics
– Free—open source (CRAN + 9000 components)
– Standard for Data Scientist
• Challenges
– Memory constrained
– Single threaded
– Outer loop—slows down process
– Not Enterprise Oriented
R environment
R—Widely Popular
R is a statistics language similar to Base SAS or SPSS statistics
+ =
Enterprise

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 22
Oracle Machine Learning Tools for Data Scientist
Oracle R Enterprise
Oracle Advanced Analytics
- On-premise Database Option
- Included in Cloud EE Database
Oracle R Advanced
Analytics for Hadoop
RStudio Notebooks
(Zepelin, Jupyter)
Oracle Data
Mining (ODM)
Interface
Hadoop based
- Option for BDA
- Included with BDCS
- Planned for BDC
Oracle Big Data
Spatial & Graph
Oracle Database
& Hadoop
Data Mining Enterprise AA4H BDSGSpatial &
Graph

Oracle Advanced Analytics
• R-SQL Transparency Framework overloads R
functions for scalable in-database execution
• Function overload for data transforms,
statistical functions and advanced analytics
• Interactive display of graphical results and
flow control as in standard R
• Submit user-defined R functions for
execution at database server under control
of Oracle Database
• Scale to large datasets
• Access tables, views, and external tables, as
well as data through DB LINKS
• Leverage database SQL parallelism
• Leverage new and existing in-database
statistical and data mining capabilities
R Engine Other R
packages
Oracle R Enterprise packages
User R Engine on desktop
• Database can spawn multiple R engines for
database-managed parallelism
• Efficient data transfer to spawned R engines
• Emulate map-reduce style algorithms and
applications
• Enables production deployment and
automated execution of R scripts
1
User tables
Oracle DatabaseSQL
Results
Database Compute Engine
2
R Engine Other R
packages
Oracle R Enterprise packages
R Engine(s) spawned by Oracle DB
R
Results
3
?x
R
Open Source
Oracle R Enterprise Compute Engines

Oracle R Advanced Analytics for Hadoop
ORAAH = Oracle R Advanced Analytics for Hadoop, part of Big Data
Software Connectors Suite (Oracle Big Data Appliance Option)
• ORAAH transparency layer enables certain overloaded R functions to operate on Hive
tables using R syntax and behavior (transparently translating R to HiveQL)
• R interface for manipulating HDFS data and writing mapper and reducer functions in R –
where you can leverage open source CRAN packages – and invoke those Hadoop jobs
from R
• Provides a range of predictive algorithms that execute on the Hadoop cluster with data
in HDFS in a parallel/distributed manner.
Oracle Internal - Proprietary 24

Hadoop Cluster
with Oracle R Advanced Analytics for Hadoop
Oracle R Advanced Analytics for Hadoop:
Using Hadoop and HIVE, plus R Engine and Open-Source R Packages
R Analytics
Oracle R Advanced
Analytics for Hadoop
R Client
• ORAAH Spark algorithms: Deep Neural, GLM, LM
• Spark MLlib algorithms: LM, GLM, LASSO, Ridge
Regression, Decision Trees, Random Forests, SVM,
k-Means, PCA
• Open-source R packages distributed via Map-Red
function in R
HQL Basic Statistics, Data
Prep, Joins and View creation
25
HQL
+
HDFS Access, Store, Load,
Data Prep and Transform.
SQL Developer
Other SQL Apps
SQL Client
Oracle Database Server
with Advanced Analytics option
BigDataSQL

• Classification
– Naïve Bayes
– Logistic Regression (GLM)
– Decision Tree
– Random Forest
– Neural Network
– Support Vector Machine
– Explicit Semantic Analysis
– Gaussian Mixture Models
• Clustering
– Hierarchical K-Means
– Hierarchical O-Cluster
– Expectation Maximization (EM)
• Anomaly Detection
– One-Class
Support Vector Machine (SVM)
• Regression
– Generalized Linear Model
– Support Vector Machine (SVM)
– Random Forest
– Linear Model
– Stepwise Linear regression
– LASSO
• Association Rules
– A priori
• Attribute Importance
– Minimum Description Length
– Principal Component Analysis (PCA)
– Unsupervised Pair-wise KL Divergence
• Predictive Queries
• Statistical Functions
– Basic statistics: median, stdev, t-test,
F-test, Pearson’s, Chi-sq, Anova, etc.
• Algorithm Support for Text
– Algorithms support text type
– Tokenization and theme extraction
– Explicit Semantic Analysis (ESA) for
document similarity
• Feature Extraction
– Principal Component Analysis (PCA)
– Non-negative Matrix Factorization
– Singular Value Decomposition (SVD)
• Time Series
– Single Exponential Smoothing
– Double Exponential Smoothing
• Open Source ML Algorithms
– CRAN R Algorithm Packages through
Embedded R Execution
– Spark MLlib algorithm integration
Oracle’s Adv. Analytics Machine Learning Algorithms
A1 A2 A3 A4 A5 A6
A7

Oracle Spatial Analysis
• Análisis de proximidad y
contención
• Datos de localización
enriquecidos
• Preparación de datos
raster y vectoriales
27
Oracle Graph Analysis
• Gráficos de propiedades
• Análisis de relaciones
• Análisis de valor (churn)
• Ciberseguridad
• Reconocimiento de objetos
Oracle’s Spatial and Graph

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | CVC Spatial Update for DWR
Enable Spatial and Graph use cases on every platform
Oracle’s Spatial and Graph Strategy
NoSQL
Oracle Big Data Spatial and Graph Spatial and Graph in
Cloud Offerings
Oracle Database
Spatial and Graph
Big Data:
Single Model Data Store
Database 12c:
Polyglot (Multi-model)
Data Store
Oracle Big Data Cloud Service
Oracle Database Cloud Service
• Enterprise Edition High Performance
• Enterprise Edition Extreme Performance

Overview of Graph
• What is a graph?
– A set of vertices and edges (with optional properties)
– A graph is simply linked data
• Why do we care?
– Graphs are everywhere
• Road networks, power grids, biological networks
• Social networks/Social Web (Facebook, Linkedin, Twitter, Baidu, Google+,…)
• Knowledge graphs (RDF, OWL)
– Graphs are intuitive and flexible
• Easy to navigate, easy to form a path, natural to visualize
• Do not require a predefined schema
E
A D
C B
F
2

Graph Analysis Examples
Reachability
Quickly identify multi-hop
relations between (a set
of) vertices and how they
are connected under
various constraints.
Anomaly Detection
Analyze the link
relationships between
data entities to detect
subsets of data that are
different from others.
Centrality Analysis
Analyze the topology of the
network, in addition to data
values, in order to identify
data entities that are more
important than others.
Link Prediction
Inspect similarities
between data entities
under overall network
structure, and predict
potential future links.
e.g. product
recommendation
e.g. security breach
trace
e.g. influencer
identificatione.g. Fraud detection
Confidential – Oracle Internal 30

Rich set of built-in parallel graph algorithms … and parallel graph mutation operations
Computational Analytics: Built-in Package
40+ built-in algorithms, highly parallelized, highly performant
Detecting Components and
Communities
Tarjan’s, Kosaraju’s,
Weakly Connected
Components, Label
Propagation (w/ variants),
Soman and Narang’s
Spacification
Ranking and Walking
Pagerank, Personalized Pagerank,
Betwenness Centrality (w/ variants),
Closeness Centrality, Degree
Centrality,
Eigenvector Centrality, HITS,
Random walking and sampling (w/
variants)
Evaluating Community Structures
∑ ∑
Conductance,
Modularity
Clustering Coefficient
(Triangle Counting)
Adamic-Adar
Path-Finding
Hop-Distance (BFS)
Dijkstra’s,
Bi-directional Dijkstra’s
Bellman-Ford’s
Link Prediction SALSA
(Twitter’s Who-to-follow)
Other Classics Vertex Cover
Minimum Spanning-Tree
(Prim’s)
a
d
b e
g
c i
f
h
The original graph
a
d
b e
g
c i
f
h
Create Undirected
Graph
Simplify Graph
a
d
b e
g
c i
f
h
Left Set: “a,b,e”
a d
b
e
g
c
i
Create Bipartite
Graph
ge b d i a f c h
Sort-By-Degree (Renumbering)
Filtered
Subgraph
d
b
g
i
e
31

Information Management Reference Architecture
32
Actionable
Events
Streaming Engine Data Lake Enterprise Data & Reporting
Discovery Lab
Actionable
Metrics
Actionable
Data Sets
Input
Events
Execution
Innovation
Discovery
Output
Data
Structured
Enterprise
Data

Big Data SQL Simplifies Analyses
33
Streaming Engine Data Lake Enterprise Data & Reporting
Discovery Lab
Input
Events
Execution
Innovation
Discovery
Output
Data
Structured
Enterprise
Data
Notebooks/Analytic Services
Big Data SQL
Object Store Hadoop/HDFS
Your Application

Meetup Oracle Database BCN: 2.1 Data Management Trends

Meetup Oracle Database BCN: 2.1 Data Management Trends

Recommended

More Related Content

What's hot (20)

Similar to Meetup Oracle Database BCN: 2.1 Data Management Trends (20)

More from avanttic Consultoría Tecnológica (20)

Recently uploaded (20)

Meetup Oracle Database BCN: 2.1 Data Management Trends