How to Streamline DataOps on AWS

IT & DATA MANAGEMENT RESEARCH, INDUSTRY
ANALYSIS & CONSULTING
John Santaferraro
Research Director
EMA
How to Streamline
DataOps on AWS
Modernizing Data Management in the Cloud
Will Davis
Sr. Director of Product Marketing
Trifacta
Nikki Rouda
Principal Product Marketing Manager, Amazon
Web Services

Watch the On-Demand Webinar
Slide 2
• EMA How to Streamline DataOps on AWS On-Demand webinar is
available here: http://info.enterprisemanagement.com/how-to-
streamline-dataops-on-aws-webinar-ws
• Check out upcoming webinars from EMA here:
http://www.enterprisemanagement.com/freeResearch

Featured Speakers
John Santaferraro, Research Director, EMA
John is the research director for analytics, business intelligence, and data management at EMA. He has 23 years of
experience in data and analytics, from startups to executive positions at Fortune 50 companies. His deep
understanding of the industry comes from years of leadership in implementation, product and marketing organizations,
along with multiple big data imagineering efforts for finance, communications, retail, manufacturing, healthcare, events,
oil and gas, and utilities. John's coverage area also includes data integration, data discovery, metadata management,
artificial intelligence, machine learning, data science, digital marketing, and innovation.
Will Davis, Sr. Director of Product Marketing, Trifacta
Will drives go-to-market and product marketing efforts at Trifacta having spent the past ten years managing the
marketing initiatives for several high-growth data companies. Prior to Trifacta, Will worked with a variety of companies
focused on data infrastructure, analytics and visualization, including GoodData, Greenplum and ClearStory Data. Will
leads Trifacta’s marketing strategy to rapidly expand business growth and brand awareness.
Nikki Rouda, Principal Product Marketing Manager, Amazon Web Services
Nikki is the principal product marketing manager for data lakes and big data at AWS. Nikki has spent 20+ years helping
enterprises in 40+ countries develop and implement solutions to their analytics and IT infrastructure challenges. Nikki
holds an MBA from the University of Cambridge and an ScB in geophysics and math from Brown University.

Logistics for Today’s Webinar
An archived version of the event recording will be
available at www.enterprisemanagement.com
• Log questions in the chat panel located on the lower
left-hand corner of your screen
• Questions will be addressed during the Q&A session
of the event
QUESTIONS
EVENT RECORDING
A PDF of the speaker slides will be distributed
to all attendees
PDF SLIDES

Agenda
• Top Cloud Challenges and Opportunities
• Modernization and DataOps in the Cloud
• Data Lakes and Analytics on AWS
• Trifacta DataOps on AWS
• Question and Answer
Slide 5 © 2018 Enterprise Management Associates, Inc.

Top Cloud Challenges and
Opportunities

MORE PLATFORMS:
Almost 8 of 10 participants
indicated they have between 3
and 7 different platforms in
their big data environment.

FASTER SPEEDS:
Almost 3 of 4 participants
indicated they were adopting
real-time processing
strategies.

FASTER SPEEDS:
Streaming platforms take the
#1 spot for platforms
implemented in 2018.

MORE COMPLEXITY:
Almost 3 of 4 respondents
indicated they were adopting
complex workloads like data
science and machine learning.

MORE CLOUD:
More than 3 of every 4 big
data projects are using some
form of cloud implementation.

Modernization and DataOps in the Cloud

EMA Hybrid Data Ecosystems
H/S
AP
DW
DM
NSSP
RS
OS
DP
AP - Analytic Platforms
DP - Discovery Platforms
H/S - Hadoop/Spark
DW - Data Warehouse
DM - Data Marts
NS - NoSQL
OS - Operational Systems
SP - Streaming Platforms

H/S
AP
DW
DM
NSSP
SS
OS
RDS
DP
H/S - Hadoop/Spark
DW - Data Warehouse
DM - Data Marts
NS - NoSQL
SS - Simple Storage
- in the cloud

H/S
EMR
AP
Redshift
DW
Redshift
DM
RDS
NS
DynomoDB
SP
Kenesis
SS
S3
OS
RDS
DP
Several
H/S - Hadoop/Spark
DW - Data Warehouse
DM - Data Marts
NS - NoSQL
SS - Simple Storage
- Example: AWS

7 Principles of DataOps for Cloud Data Ecosystems
• Multi-model data access
replaces single model
• Interoperability replaces
integration
• Data preparation and pipelines
replace data cleansing
• Automation replaces manual
data everything
• Elasticity replaces enterprise
scalability
• Multidimensional agility
replaces extensibility
• Automated governance
replaces simple metadata

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential
Data lakes and analytics on
AWS
Nikki Rouda, AWS
February 21, 2019

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential
Driving business with analytics
• Fighting fraud
• Quoting car & truck
prices
• Movies & TV on demand
• Delivering software
quality
• Mitigating safety issues
• Targeted marketingWhat do all these business challenges have in
common?
They are solved with AWS data lakes and analytics.
• Finding new revenue
• Improving health
• Serving retail customers
• Valuing real estate
• Reducing advertising costs
• Making music

Companies want more value from their data
Complications:
Siloed approaches don’t work anymore
It’s too expensive and limiting to store data
on-premises
Data is:
Implication:
A new approach is needed to extract insights
and value
Growing
exponentially
From new
sources
Increasingly
diverse
Used by
many people
Analyzed by
many applications

Cloud data lakes are the future
Customers want:
To eliminate siloes of data
To move to a single store, i.e. a data lake in the cloud
To store data securely in standard formats
To grow to any scale, with low costs
To analyze their data in a variety of ways
To have real-time analytics
To predict future outcomes
Data Lake

Why choose AWS for data lakes and analytics
Most
comprehensive
Most
secure
Most
scalable
Most
cost-effective

Our portfolio
Broadest and deepest portfolio, purpose-built for builders
Migration & Streaming Services
Infrastructure Data Catalog
& ETL
Security &
Management
Dashboards Machine Learning
Data
Warehousing
Big Data
Processing
Interactive
Query
Operational
Analytics
Real time
Analytics
Serverless
Data
processing
Visualization & machine
learning
Data movement
Analytics
Data lake

Data movement
Analytics
Data lake
Our portfolio
Broadest and deepest portfolio, purpose-built for builders
QuickSight SageMaker
Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams | Managed Streaming for
Kafka
Redshift EMR Athena Elasticsearc
h Service
Kinesis Data
Analytics
Glue
S3/Glacier GlueLake
Formation
Visualization & machine
learning

More data lakes and analytics than anywhere else

125+ million players
Create a constant feedback
loop
for game designers
Up-to-the-minute
understanding
of gamer satisfaction to
guarantee gamers are engaged
Resulting in the most popular
game played in the world
Fortnite

Migrated from on-premises
data warehouse
Built a data warehouse with
Amazon Redshift and data lake
with Amazon S3
Analytics on data lake with
Amazon Athena, Amazon
Redshift Spectrum, and Amazon
EMR
Report delivery went from
months to days, at far lower cost

Needed to analyze data to find insights,
identify opportunities and evaluate business
performance
The Oracle data warehouse did not scale, was
difficult to maintain and costly
Deployed a data lake with Amazon S3, and run
analytics with Amazon Redshift, Amazon
Redshift Spectrum, and Amazon EMR
Result: They doubled the data stored (100PB),
lowered costs, and was able to gain insights
faster
50 PB of data
600,000 analytics jobs/day

Enabling all types of data-driven analytics
Retrospective
analysis and
reporting
Here-and-now
real-time processing
and dashboards
Predictions
to enable smart
applications

Trifacta on AWS
How to Streamline DataOps on AWS: Modernizing Data Management in the Cloud
Will Davis, Sr. Director of Product Marketing, Trifacta

DataOps Definition
32 Proprietary & Confidential
“DataOps is the function within an organization that controls
the data journey from source to value.”
Jarah Euston, What is DataOps?, (Nexla, 2017), https://www.nexla.com/define-dataops/

Data Platforms
Databases
Log Files
Spreadsheets
IoT Sensors
Apps
80%
Usage Patterns
Data Onboarding
ML/AI
Analytics
“It’s impossible to overstress this:
80% of the work in any data project
is in cleaning the data.”
— DJ Patil, Former Chief Data Scientist of the United States
Proprietary & Confidential.33
What is Biggest Challenge in Streamlining DataOps?

Analysis
Enterprise Data Warehouse
AI
Business Intelligence
Data Analyst Data Engineer Data Scientist
ValidatingDiscovering Structuring Cleaning Enriching Deploying
Data Platforms
Databases
Log Files
Spreadsheets
IoT Sensors
Apps
And it Impacts Your Entire Data Team...

“Poor data quality is enemy number
one to the widespread, profitable use
of machine learning.”
—Harvard Business Review
“So, while there is a visible arms race as
companies bring on machine learning coders
and kick off AI initiatives, there is also a
behind-the-scenes, panicked race for new
and different data.”
—MIT Sloan Management Review
The Rise of Machine Learning & AI Compounds the Problem

"The hard part of
AI
is data wrangling.”
wrangles
For AI
—SWAMI SIVASUBRAMANIAN
VP – AMAZON MACHINE LEARNING
#reInvent2018

WHAT TO DO?

Data Platforms
Databases
Log Files
Spreadsheets
IoT Sensors
Apps
Analysis
AI
DATA WRANGLING

Data Platforms
Databases
Log Files
Spreadsheets
IoT Sensors
Apps
Analysis
AI
DATA WRANGLING
• Empowers domain experts with intelligent visual
interfaces that automate assessment and
transformation of data
• Enable IT to collaboratively curate and operationalize
data pipelines authored by domain experts
• Establish an enterprise-wide platform that refines data
from a variety of sources, supporting a range of users
and use cases

Predictive
Modeling
Business
Intelligence
Data
Onboarding
Risk &
Compliance
Audit, Testing
&
Validation
Data Migration
OPERATIONAL
Data Platforms
Databases
Log Files
Spreadsheets
IoT Sensors
Apps
ValidatingDiscovering Structuring Cleaning Enriching Deploying
ANALYTIC
Data Analyst Data Engineer Data Scientist

And We’re Natively Integrated into AWS
• Native storage
• Native processing
• Native security

50+ Trifacta Customers Deployed on AWS

Why Trifacta?
QUALITY
SPEED
EFFICIENCY

QUALITY
SPEED
EFFICIENCY
Empower the people who know the data best
While maintaining governance and lineage
Intuitive, visual
interface
Intuitive,
visual interface
Self-documenting
lineage

QUALITY
SPEED
EFFICIENCY
Faster to Design Preparation Workflows
Instant previews,
continuous validation
ML-driven
suggestion
s

QUALITY
SPEED
EFFICIENCY
Faster to Put Workflows into Production
Automate data pipelines
Share, test &
version control

QUALITY
SPEED
EFFICIENCY
Retire legacy
solutions
Utilize native
cloud elasticity

Industry Analysts All Rank Trifacta #1
Self Service Data
Preparation Wave
“Customer references
can't say enough about
Trifacta’s ease of use"
A perfect score in 14 of
17 categories.
#1
with Gartner
#1
with Ovum,
Dresner,
and Bloor

49 Proprietary & Confidential.
How to Get
Started?

Different Editions & Deployments Options for Any Use Case
FOR INDIVIDUALS
Free
FOR TEAMS & DEPARTMENTS
Starts at $5K per user*
FOR ENTERPRISE DEPLOYMENT
Contact Us
• Trifacta Managed Cloud
• Works with desktop files
• Functional, data volume, and
processing limitations
• Community support
• Trifacta Managed Cloud
• Files, relational, cloud
connectivity
• Job Scheduling & Collaboration
• Phone/email support
• Customer Managed Cloud
• Unlimited volume & scalability
• Broad connectivity
• Advanced security, access
controls, and governance
• Enterprise support & dedicated
customer success manager

Available on the AWS Marketplace

Start Wrangling Today with Free Wrangler

How to Streamline DataOps on AWS

More Related Content

What's hot (20)

Similar to How to Streamline DataOps on AWS (20)

More from Enterprise Management Associates (20)

Recently uploaded (20)

How to Streamline DataOps on AWS