Overview of running R in the Oracle Database

www.oraly)cs.com
t
:
@brendan)erney
e
:
brendan.)erney@oraly)cs.com

Running
R
in
the
Database
using

Oracle
R
Enterprise

Brendan Tierney
Code
Demo

www.oraly)cs.com
t
:
@brendan)erney
e
:

§  Data
Warehousing
since
1997

§  Data
Mining
since
1998

§  Analy)cs
since
1993

Brendan
Tierney

www.oraly)cs.com
t
:
@brendan)erney
e
:

Agenda

•  What
is
R?

•  Oracle
Advanced
Analy)cs
Op)on

•  Oracle
R
Technologies
&
Oracle
R
Enterprise

•  Examples
of
using
ORE

•  Crea)ng
&
running
R
in
the
Database

•  How
to
run
R
in
the
Database
using
SQL

•  Using
ORE
with
other
products

Code
Demo

www.oraly)cs.com
t
:
@brendan)erney
e
:

Agenda

•  What
is
R?

•  Oracle
Advanced
Analy)cs
Op)on

•  Oracle
R
Technologies
&
Oracle
R
Enterprise

•  Examples
of
using
ORE

•  Crea)ng
&
running
R
in
the
Database

•  How
to
run
R
in
the
Database
using
SQL

•  Using
ORE
with
other
products

www.oraly)cs.com
t
:
@brendan)erney
e
:

R

§  R
Open
source
sta)s)cal
compu)ng
and
graphics

language

§  Started
in
1993
as
an
alterna)ve
to
SAS,
SPSS
and

other
proprietary
sta)s)cal
packages

•  Originally
called
S,
renamed
to
R
in
1996

§  R
is
a
client
and
server
bundled
together
as
one

executable

•  It
is
a
single
user
tool

•  It
is
not
mul)-‐threaded

•  Constrained
to
a
single
CPU

§  Millions
of
R
users
worldwide

§  Thousands
of
libraries
available
at

•  hXp://cran.r-‐project.org

§  Free

Milestones:

2017-‐01-‐09:
9870
packages

2016-‐06-‐01:
8492
packages

2015-‐03-‐13:
6400
packages

2015-‐02-‐15:
6325
packages

2014-‐10-‐29:
6000
packages

2013-‐11-‐08:
5000
packages

2012-‐08-‐23:
4000
packages

2011-‐05-‐12:
3000
packages

2009-‐10-‐04:
2000
packages

2007-‐04-‐12:
1000
packages

2004-‐10-‐01:
500
packages

2003-‐04-‐01:
250
packages

www.oraly)cs.com
t
:
@brendan)erney
e
:

§  Is
used
every
where

§  Par)cularly
in
USA

–  And
else
where

www.oraly)cs.com
t
:
@brendan)erney
e
:

Examples
of
R

www.oraly)cs.com
t
:
@brendan)erney
e
:

> library(RJDBC)
> # Create connection driver and open
> connectionjdbcDriver <- JDBC(driverClass="oracle.jdbc.OracleDriver", classPath="c:/ojdbc6.jar")
> jdbcConnection <- dbConnect(jdbcDriver, "jdbc:oracle:thin:@//localhost:1521/orcl", "dmuser", "dmuser")
> #list the tables in the schema
> #dbListTables(jdbcConnection)
> #get the DB connections details - it get LOTS of info - Do not run unless it is really needed
> dbGetInfo(jdbcConnection)
> # Query on the Oracle instance name.
> #instanceName <- dbGetQuery(jdbcConnection, "SELECT instance_name FROM v$instance")
TABLE_NAME1
1 INSUR_CUST_LTV_SAMPLE2
2 OUTPUT_1_2
> #print(instanceName)tableNames <- dbGetQuery(jdbcConnection, "SELECT table_name from user_tables where
table_name not like 'DM$%' and table_name not like 'ODMR$%'")
> print(tableNames)
> viewNames <- dbGetQuery(jdbcConnection, "SELECT view_name from user_views")print(viewNames)
1 MINING_DATA_APPLY_V
2 MINING_DATA_BUILD_V
3 MINING_DATA_TEST_V
4 MINING_DATA_TEXT_APPLY_V
5 MINING_DATA_TEXT_BUILD_V
6 MINING_DATA_TEXT_TEST_V
> dbDisconnect(jdbcConnection)
Using
RJDBC

www.oraly)cs.com
t
:
@brendan)erney
e
:

> library(ROracle)
> drv <- dbDriver("Oracle")
> # Create the connection string
> host <- "localhost"
> port <- 1521
> sid <- "orcl"
>connect.string <- paste("(DESCRIPTION=”, "(ADDRESS=(PROTOCOL=tcp)(HOST=", host, ")(PORT=", port, "))",
> "(CONNECT_DATA=(SID=", sid, ")))", sep = "")
> con <- dbConnect(drv, username = "dmuser", password = "dmuser",dbname=connect.string)
> rs <- dbSendQuery(con, "select view_name from user_views")
> # fetch records from the resultSet into a data.frame
> data <- fetch(rs)
> # extract all rows
> dim(data)
[1] 6 1
> data
VIEW_NAME
1 MINING_DATA_APPLY_V
2 MINING_DATA_BUILD_V
3 MINING_DATA_TEST_V
4 MINING_DATA_TEXT_APPLY_V
5 MINING_DATA_TEXT_BUILD_V
6 MINING_DATA_TEXT_TEST_V
> dbCommit(con)
> dbClearResult(rs)
> dbDisconnect(con)
Using
ROracle

Needs
Oracle
Client

in
the
search
path

Pulls
the
data
to
the

Client

Has
a
set
of
R

func)ons
tuned
for

the
Oracle
DB

www.oraly)cs.com
t
:
@brendan)erney
e
:

The
Challenges

§  Scalability

§  Regardless
of
the
number
of
cores
on
your
CPU,
R
will
only
use
1
on
a

default
build

§  Performance

§  R
reads
data
into
memory
by
default.
Easy
to
exhaust
RAM
by
storing

unnecessary
data.
Typically
R
will
throw
an
excep)on
at
2GB.

§  Paralleliza)on
can
be
challenge.
Is
not
Default.
Packages
available

§  Produc)on
Deployment

§  Diﬃcul)es
deploying
R
in
produc)on

§  Typically
need
to
re-‐code
in
…..

www.oraly)cs.com
t
:
@brendan)erney
e
:

Comprehensive
Advanced
Analy)cs
Plaform

www.oraly)cs.com
t
:
@brendan)erney
e
:

Technique
Algorithms
Applicability

Classiﬁca)on
Logis)c
Regression
(GLM)

Decision
Trees

Naïve
Bayes

Support
Vector
Machine

Classical
Sta)s)cal
Technique

Popular
/
Rules
/
Transparency

Embedded

Wide
/
Narrow
Data
/
Text

Regression

Mul)ple
Regression

Support
Vector
Machine

Classical
Sta)s)cal
Technique

Wide
/
Narrow
Data
/
Text

Anomaly
Detec)on

One
Class
SVM
Lack
Examples

AXribute
Importance

Minimum
Descrip)ve
Length
AXribute
Reduc)on

Iden)fy
Useful
Data

Reduce
Data
Noise

Associa)on
Rules

Apriori
Market
Basket
Analysis

Link
Analysis

Clustering
Enhanced
K-‐Means

O-‐Cluster

Expecta)on
Maximiza)on

Product
Grouping

Text
Mining

Gene
and
Protein
Analysis

Feature
Extrac)on
Non-‐Nega)ve
Matrix
Factoriza)on

Principal
Components
Analysis

Singular
Vector
Decomposi)on

Text
Analysis

Feature
Reduc)on

In-‐Database
Data
Mining
Algorithms

www.oraly)cs.com
t
:
@brendan)erney
e
:

Oracle
Data
Mining

§  PL/SQL
Package

§  DBMS_DATA_MINING

§  DBMS_DATA_MINING_TRANSFORM

§  DBMS_PREDICTIVE_ANALYTICS

§  SQL
Func)ons

–  PREDICTION

–  PREDICTION_PROBABILITY

–  PREDICTION_BOUNDS

–  PREDICTION_COST

–  PREDICTION_DETAILS

–  PREDICTION_SET

–  CLUSTER_ID

–  CLUSTER_DETAILS

–  CLUSTER_DISTANCE

–  CLUSTER_PROBABILITY

–  CLUSTER_SET

–  FEATURE_ID

–  FEATURE_DETAILS

–  FEATURE_SET

–  FEATURE_VALUE

§  12c
–
Predic)ve
Queries

§  aka

Dynamic
Queries

§  Transi)ve
dynamic
Data
Mining
models

§  Can
scale
to
many
100+
models
all
in
one

statement

www.oraly)cs.com
t
:
@brendan)erney
e
:

Sta)s)cal
Func)ons
in
Oracle

All
of
these
are

FREE

with
the
Database

These
are
oren

forgoXen
about

www.oraly)cs.com
t
:
@brendan)erney
e
:

Oracle
R
Technologies

R
Distribu)on
Oracle's
supported
redistribu)on
of
open
source
R,
provided
as
a
free

download
from
Oracle,
enhanced
with
dynamic
loading
of
high

performance
linear
algebra
libraries.

Oracle
R
Enterprise
Integra)on
of
R
with
Oracle
Database.
A
component
of
the
Oracle

Advanced
Analy)cs
Op)on.
Oracle
R
Enterprise
makes
the
open
source
R

sta)s)cal
programming
language
and
environment
ready
for
the

enterprise
with
scalability,
performance,
and
ease
of
produc)on

deployment.

Oracle
R
Advanced
Analy)cs
for

Hadoop

High
performance
na)ve
access
to
the
Hadoop
Distributed
File
System

(HDFS)
and
MapReduce
programming
framework
for
R
users.
Oracle
R

Advanced
Analy)cs
for
Hadoop
is
a
component
of
Oracle
Big
Data

Connectors
sorware
suite.

ROracle
An
open
source
R
package,
maintained
by
Oracle
and
enhanced
to
use
the

Oracle
Call
Interface
(OCI)
libraries
to
handle
database
connec)ons
-‐

providing
a
high-‐performance,
na)ve
C-‐language
interface
to
Oracle

Database.

www.oraly)cs.com
t
:
@brendan)erney
e
:

Oracle
R
Enterprise

§  R
installed
in
ORACLE_HOME

§  Fully
integrated
with
the
database

§  Overcomes
the
limita)ons
of
R

§  U)lizes
the
DB
performance
and
scalability

features

§  Full
integra)on
into
the
DB
engine

§  Can
run
R
inside
the
DB

§  Can
store
R
object
in
the
DB

§  Can
run
R
objects
using
SQL
&
PL/SQL

§  Easily
integrated
into
other
Oracle
Tools

and
Applica)ons

§  Greatly
expands
the
sta)s)cs
&
analy)cs

§  Easily
integrates
new
“bells
&
whistles”

package
comes
available

www.oraly)cs.com
t
:
@brendan)erney
e
:

Oracle
R
Enterprise

www.oraly)cs.com
t
:
@brendan)erney
e
:

The
Magic

:
The
Transparency
Layer

§  No
need
to
learn
a
diﬀerent
programming
paradigm
or
environment

•  If
you
are
an
R
programmer

§  Operate
on
database
data
as
though
they
were
R
objects
using
R
syntax

§  Require
minimal
change
to
base
R
scripts
for
database
data

§  Implicitly
translates
R
to
SQL
for
in-‐database
execu)on,
performance,
and

scalability

www.oraly)cs.com
t
:
@brendan)erney
e
:

> AggData <- aggregate(CUSTOMER_V$CUST_ID,
by = list(CUST_GENDER = CUSTOMER_V$CUST_GENDER), 
FUN = length)  
> # Display the results  
> AggData  
CUST_GENDER x  
F F 18325  
M M 37175
select
cust_gender,
count(*)
X

from

customer_v

group
by
cust_gender;

CUST_GENDER X
----------- ----------
F 18325
M 37175
Oracle
R
Enterprise

Transparency
Layer

1
2
3
4
5
6

www.oraly)cs.com
t
:
@brendan)erney
e
:

The
Magic

:
The
Transparency
Layer

§  R
interface
to
in-‐Database
Sta)s)cal
func)ons

www.oraly)cs.com
t
:
@brendan)erney
e
:

The
Magic

:
The
Transparency
Layer

§  R
interface
to
in-‐Database
Predic)ve
Analy)cs
func)ons

www.oraly)cs.com
t
:
@brendan)erney
e
:

The
Magic

:
The
Transparency
Layer

§  In
R
all
commands
are
executed
immediately

§  In
ORE,
commands
are
not
executed
immediately

§  They
are
stacked
and
accumulated

§  When
a
ﬁnal
result
is
needed
(for
computa)on
or
viewing)

§  Oracle
will
perform
some
op)miza)on

reorganiza)on
of
the
commands

§  Oracle
will
perform
query
op)miza)on

§  Oracle
will
translate
into
in-‐Database
SQL
and
PL/SQL
func)on/
procedures/packages

§  Execute
the
SQL
(
R)
commands
on
the
data
in
the
database

§  Oracle
will
manage
the
results

§  The
returned
results
will
be
translated
back
into
R
format

www.oraly)cs.com
t
:
@brendan)erney
e
:

Agenda

•  What
is
R?

•  Oracle
Advanced
Analy)cs
Op)on

•  Oracle
R
Technologies

Oracle
R
Enterprise

•  Examples
of
using
ORE

•  Crea)ng
R
objects
in
the
Database

•  How
to
run
R
in
the
Database
using
SQL

•  Using
ORE
with
other
products

You
can
run
all
the

example
code
if
you

have
the

ODM
Demo
Schema

created

Code
Demo

www.oraly)cs.com
t
:
@brendan)erney
e
:

www.oraly)cs.com
t
:
@brendan)erney
e
:

Agenda

•  What
is
R?

•  Oracle
Advanced
Analy)cs
Op)on

•  Oracle
R
Technologies

Oracle
R
Enterprise

•  Examples
of
using
ORE

•  Crea)ng

running
R
in
the
Database

•  How
to
run
R
in
the
Database
using
SQL

•  Using
ORE
with
other
products

www.oraly)cs.com
t
:
@brendan)erney
e
:

Embedded
R
Execu)on
using
SQL

www.oraly)cs.com
t
:
@brendan)erney
e
:

--
-- Now let us use the Demo_GLM_Batch script to score data in Real-Time
-- The data values are passed to the GLM model
--
select * from table(rqTableEval(
cursor(select 'M' CUST_GENDER,
23 AGE,
'Married' CUST_MARITAL_STATUS,
'United States of America' COUNTRY_NAME,
'B: 30,000 - 49,999' CUST_INCOME_LEVEL,
'Assoc-A' EDUCATION,
'3' HOUSEHOLD_SIZE,
5 YRS_RESIDENCE
from dual),
cursor(select 'myDatastore' datastore_name, 1 ore.connect from dual),
'select CUST_GENDER, AGE, CUST_MARITAL_STATUS, COUNTRY_NAME, CUST_INCOME_LEVEL, EDUCATION,
HOUSEHOLD_SIZE, YRS_RESIDENCE, 1 PRED from MINING_DATA_APPLY',
'Demo_GLM_Batch')) order by 1, 2, 3;
What-‐if
analysis

You
can
easily
use
this

in-‐Database
R
code
in

your
applica)ons

www.oraly)cs.com
t
:
@brendan)erney
e
:

Integra)ng
with
OBIEE,
BI
Publisher

Any
other
Language
or
Tool

Database
Server

Machine

Oracle
Database

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐

Tables

In-‐Database

SQL
Func)ons

R
Language
Installa)on

ORE
Installed
Packages

R

Scripts

www.oraly)cs.com
t
:
@brendan)erney
e
:

--
-- Create an embedded R script
-- Called using the ORE SQL API
-- - performs an aggregration of the data
-- - creates a Graphic Plot
--
begin
sys.rqScriptDrop('AgeProfile');
sys.rqScriptCreate('AgeProfile',
'function(dat) {
mdbv - dat
aggdata - aggregate(mdbv$AFFINITY_CARD,
by = list(Age = mdbv$AGE),
FUN = length)
res - plot(aggdata$Age, aggdata$x, type = l) } ');
end;
/
--
-- Execute the embedded R Script
-- - Graphic created in PNG format for import into OBIEE
-- - change PNG to XML for BI Publisher
--
select * from table(rqTableEval( cursor(select * from MINING_DATA_BUILD_V),
cursor(select 1 ore.connect from dual),
'PNG','AgeProfile'));
ORE
Demo

ORE
Demo
Data
Chart

www.oraly)cs.com
t
:
@brendan)erney
e
:

The
Challenges
:
With
ORE
“Yes
we
can”


§  Regardless
of
the
number
of
cores
on
your
CPU,
R
will
only
use
1
on
a

default
build


§  R
reads
data
into
memory
by
default.
Easy
to
exhaust
RAM
by
storing

unnecessary
data.
Typically
R
will
throw
an
excep)on
at
2GB.

can
be
challenge.
Is
not
Default.
Packages
available

§  Produc)on
Deployment

deploying
R
in
produc)on

§  May
need
to
re-‐code
in
…..

✔

✔

✔

Scale
with
the

Database

In-‐Database
R
execu)on.

Easy
integra)on
with
all

your
applica)ons.

www.oraly)cs.com
t
:
@brendan)erney
e
:

Data
Mining
/
Data
Science
in
Oracle

Is
Just
SQL

(Oracle
Data
Mining

SQL
Sta)s)cs
func)ons)

+

R
(Oracle
R
Enterprise)

www.oraly)cs.com
t
:
@brendan)erney
e
:

The
Challenges
:
With
ORE
“Yes
we
can”


§  Regardless
of
the
number
of
cores
on
your
CPU,
R
will
only

use
1
on
a
default
build


§  R
reads
data
into
memory
by
default.
Easy
to
exhaust
RAM
by
storing

unnecessary
data.
Typically
R
will
throw
an
excep)on
at
2GB.

can
be
challenge.
Is
not
Default.
Packages
available

§  Produc)on
Deployment

deploying
R
in
produc)on

§  May
need
to
re-‐code
in
…..

✔

✔

✔

Scale
with
the

Database

In-‐Database
R
execu)on.

Easy
integra)on
with
all

your
applica)ons.

What
if
I
want
to
use
a
new
R
Package?

Easy,
just
install
it
on
the
DB
server
and
oﬀ

you
go
!!!

www.oraly)cs.com
t
:
@brendan)erney
e
:


@brendan)erney

www.oraly)cs.com

ie.linkedin.com/in/brendan)erney

www.oraly)cs.com
t
:
@brendan)erney
e
:

Word
Cloud
of
the
Oracle
Advanced

Analy)cs
web-‐pages

hXp://www.oraly)cs.com/2015/01/crea)ng-‐word-‐cloud-‐of-‐oracle-‐oaa.html

Overview of running R in the Oracle Database

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Overview of running R in the Oracle Database (20)

Recently uploaded (20)

Overview of running R in the Oracle Database