Geekier Analytics for SaaS data

Geekier analytics for
SaaS data
Sumit Sarkar
Chief Data Evangelist
Progress DataDirect
@SAsInSumit
www.linkedin.com/in/meetsumit

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.2
Agenda
§  Demand for geekier analytics
§  In depth: SaaS marketing data
lake
§  Our guidance

Demand for geekier analytics

As Lines of Business Control Technology Select,
SaaS Adoption Continues to Accelerate
http://www.forbes.com/sites/louiscolumbus/2015/01/24/roundup-of-cloud-computing-forecasts-and-market-estimates-2015/#192c1957740c

Are SaaS apps getting left behind the analytics revolution?
Current state Opportunity
Data silos in each SaaS App
behind process APIs
Open access to $200 billion analytics
market with standard interfaces for
analytics
Embedded BI provides limited
analytics options
Engage data skills and tools in IT for
richer insights
?

What are geekier analytics?
Business Intelligence Data Integration
ODBC, JDBC
or OData
?

What are geekier analytics?
JDBC
?

Who is getting geeky today with analytics connectivity?

In depth: SaaS marketing data lake

A data lake is a large-scale storage repository and
processing engine. A data lake provides "massive
storage for any kind of data, enormous processing
power and the ability to handle virtually limitless
concurrent tasks or jobs”
- SAS Institute
What is a Marketing Data Lake?

Benefits of a Marketing Data Lake?
Some of the benefits of a data lake include:
§  Store data in all shapes and sizes
§  Flexible analytics with “schema on read”
§  Query data using SQL or big data
programming frameworks
§  Eliminate data silos

Why Marketing Data?
§  CMOs will outspend CIOs on technology by 2017
(Gartner)
§  Oracle spent $3B on a martech aquisition spree to
gain CMO mindshare.
§  Expect more collaboration between CMO and CIO
(CIO.com)
§  Marketing Data Warehouse/Lake Webinars ~750
registrations (Progress)

It’s easy to forget that it’s still about solving real business problems.
Relevant data
Transaction / behavior history
Manage
Data
Perform
Analytics
Drive
Decisions
Insights
continuous feedback loop
Appropriate
data sources
Answers to
business questions
Strategy (Thinking) Moves Right to Left
Implementation Moves Left to Right
Before you think data, think decisions!

Our marketing data is almost all in the cloud
And it’s almost all complex, stream data – which means APIs that only
give aggregations aren’t too useful

How to ingest data directly from SaaS applications into HDFS

How to ingest data directly from SaaS applications into Spark

JDBC access to SaaS data
Progress DataDirect
JDBC Connector
Schema Manager
Apache Sqoop
Salesforce.com
Schema
User Defined
Schema
Driver uses
§  SOAP API
§  Bulk API
§  Metadata API

Geek Speak
$ sqoop help import
usage: sqoop import [GENERIC-ARGS] [TOOL-ARGS]
Common arguments:
--connect <jdbc-uri> Specify JDBC connect string
--connect-manager <jdbc-uri> Specify connection manager class to use
--driver <class-name> Manually specify JDBC driver class to use
--hadoop-mapred-home <dir>+ Override $HADOOP_MAPRED_HOME
--help Print usage instructions
-P Read password from console
--password <password> Set authentication password
--username <username> Set authentication username
--verbose Print more information while working
--hadoop-home <dir>+ Deprecated. Override $HADOOP_HOME

Geek Speak
val dataframe_salesforce =
sqlContext.read.format("jdbc").option("url","jdbc:datadirect:sforce://
login.salesforce.com;").option("driver","com.ddtek.jdbc.sforce.SForceDriver").
option("dbtable","SFORCE.<table_name>").option("user","<Username>").option("p
assword","<password>").option("securitytoken","<security_token>").load()
dataframe_salesforce.registerTempTable("account")
dataframe_salesforce.sqlContext.sql("select * from
account").collect.foreach(println)

Our guidance

Data in SaaS Applications is Siloed, Protected by Proprietary APIs Designed
for Process Integration, not Data Integration

Partner Summit registration report from SFDC

Each SaaS API is vastly different
§  Get OData, JDBC, ODBC interfaces on top of any API
Data Source API
Marketo Web Services API (REST/SOAP)
Bulk and non-Bulk APIs
No query language
Oracle Service Cloud Web Services APIs (REST/SOAP)
ROQL
Google Analytics Hypercube (query limits of 10 metrics grouped by
max of 7 dimensions)
Salesforce SOAP, BULK, Metadata APIs
SOQL

#DontBeJeff
§  http://prgress.co/dontbejeff

Detail is important because this digital data is true big data
The
relationship
between
events is
critical

We’re almost never solving for one problem with a big data system
Reporting Analytics
Summarized
Data
Segmented
Data
Detail
Data
We can’t just aggregate / We can’t not aggregate
Dashboarding
Campaign
Optimization
Customer
Drill-down
Attribution, CLTV,
Experience,
Personalization
Targeting
Forecasting

Expose variety of detailed data using standard interfaces
SaaS vendors
•  Expose bulk, non-bulk and analytics data model
•  Leverage standard interfaces for SQL and REST
•  Direct secure database access option
SaaS data consumers
•  Support same standard interfaces
•  Turn to trusted data connectivity partners or dedicate
significant headcount

Other SQL to SaaS platforms

Progress DataDirect
Embed SaaS Connectors into the Data Access Layer

Q&A with R&D
§  How do you handle the varying quality of services across
SaaS APIs?
§  With analytics style connectivity, are SaaS vendors concerned
about scalability against large extracts from multiple tenants?
§  Which SaaS vendors are easier to work with?

Ingest data across 200+ data sources (beyond marketing data sources)
Big Data/NoSQL
§  Apache Hadoop Hive
§  Cloudera
§  Hortonworks
§  Pivotal HD
§  MapR
§  EMR
§  Pivotal HAWQ
§  Cloudera Impala
§  MongoDB
§  Spark SQL
§  Cassandra
§  SAP HANA
Data Warehouses
§  Amazon Redshift
§  SAP Sybase IQ
§  Teradata
§  Pivotal Greenplum
Relational
§  Oracle DB
§  Microsoft SQL Server
§  IBM DB2
§  MySQL
§  PostgreSQL
§  IBM Informix
§  SAP Sybase
§  Pervasive SQL
§  Progress OpenEdge
§  Progress Rollbase
SaaS/Cloud
§  Salesforce.com
§  Database.com
§  FinancialForce
§  Veeva CRM
§  ServiceMAX
§  Any Force.com App
§  Hubspot
§  Marketo
§  Microsoft Dynamics CRM
§  Microsoft SQL Azure
§  Oracle Eloqua
§  Oracle Service Cloud
§  Google Analytics
EDI/XML/Text
§  EDIFACT
§  EDIG@S
§  EANCOM
§  X12
§  IATA
§  Healthcare EDI: X12, HIPAA,
ICD-10, HL7
§  Custom EDI
§  Flat files: CSV, TSV, dBase,
Clipper, Foxpro, Paradox
§  Text Files
Any
§  SDK
§  SequeLink Socket Server
§  Customer Engineering

Geekier Analytics for SaaS data

Geekier Analytics for SaaS data

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Geekier Analytics for SaaS data (20)

More from Progress (19)

Recently uploaded (20)

Geekier Analytics for SaaS data