Analytics in a Day Ft. Synapse Virtual Workshop

Analytics in a day
Cloud analytics in the age of
self-service and data science

Agenda
Workshop Instruction & Discussion
9:00 – 9:30 Harness the power of analytics
9:30 – 10:15 Experience a new class of analytics with Azure Synapse
Break for 15 min and Set Up Lab
Hands-on lab
10:30 – 1:00 Hands-on lab using Azure Synapse and Power BI
Times are approximate and will be fluid with the class

Housekeeping
Please message Sami
with any questions,
concerns or if you
need assistance during
this workshop.
Please mute your line!
We will be applying mute.
This session will be
recorded.
If you do not want to be
recorded, please disconnect at
this time.
Links:
See chat window.
Worksheet:
See handouts.
To make presentation
larger, draw the
bottom half of screen
‘up’.

James McAuliffe,
Cloud Solution Architect
James McAuliffe is a Cloud Solution Architect with over 20 years of technology
industry experience. During this journey into data and analytics, he’s held all of the
traditional Business Intelligence Solution project roles, ranging from design and
development to complete life cycle BI implementations. He is a Microsoft Preferred
Partner Solutions expert and has worked with clients of all sizes, from local
businesses to Fortune 500 companies.
And I like old Italian cars.
linkedin.com/in/jamesmcauliffesql/

A premier Microsoft partner, CCG uses leading cloud
platforms to develop solutions and provide analytics that
help customers advance their digital strategies.
6
PARTNERSHIP SPOTLIGHT: MICROSOFT
Certifications
Gold Partner
Independent System Vendor (ISV)
and Co-Seller
AI Inner Circle Partner
Technologies
Azure Data Services
Azure Data Factory
Azure Data Lake Store
Azure Databricks
Azure Cognitive Services
Azure Machine Learning
Azure Stream Analytics
Azure Analysis Services
Azure Synapse Analytics
Power BI Platform

Offerings Overview
7
Data and
Analytics Strategy
Advanced Analytics,
Machine Learning, and AI
Data Management
and Data Governance
Enterprise Business
Intelligence
Cloud Strategy, Migration,
And Management

Harness the power of analytics
Section 1
All businesses are data businesses
Section 2
The cloud for modern analytics
Section 3
The paradox of analytics
Section 4
The analytics continuum

Section 1
All businesses are data businesses

Today, all businesses are
data businesses
Data is the lifeblood of modern work

All data businesses need
to be analytic businesses
Without analytics data is a cost center,
not a resource

Analytic businesses need
to evolve data science
Every business has opportunities to make
analytics faster, easier, and more insightful

Section 2
The cloud for modern analytics

Modern businesses
succeed in the cloud
The cloud is the default environment for
new technology initiatives

Price, performance, and agility
A cloud analytics platform
is an economic breakthrough

Structured, unstructured, and streaming data
integrated in a single, scalable, environment
A cloud analytics platform
is the hub for all data models

Section 3
The paradox of analytics

Data Landscape – Volume and Pressure
IDC Data Age 2025 - The Digitization of the World

While data grows 400%, less than 30% gets analyzed
2025
2020
44ZB 175ZB

* Harvard Business Review (2019), Understanding why analytics strategies fall short for some, but not for others
Analytics & AI is the #1 investment for business leaders,
however they struggle to maximize ROI
80% 55%
?

More analytics solutions lead to more silos
Data Technologies Skills

Each new technology creates
another siloed operation
Big data
Data integration
Machine learning
Business intelligence
Data governance
Security
paradox

Section 4
The analytics continuum

Real-time
BI
Data
integration
Artificial
intelligence
Machine
learning
Data
exploration
Data
warehousing

Data
exploration
Real-time
BI
Data
integration
Artificial
intelligence
Machine
learning
Data
warehousing

©Microsoft Corporation
Azure
The importance of data models
BI models Power BI
 Built and maintained by business users or BI developers
 Use enterprise models, departmental data, and external sources
 Focused on a single subject area, but often widely shared
Machine learning
models
Azure Synapse
Analytics
 Built and maintained by data scientists
 Mostly developed from raw sources in the data lake
 Often experimental, needing a data engineer for production use
Azure Synapse
Analytics
Enterprise models
 Built and maintained by IT architects
 Consolidated data from many systems
 Centralized as an authoritative source for reporting and analysis

Azure
If business users are
tech-smart and data
literate, why do they
need enterprise
models?
Enterprise models in the
self-service environment
Consistency
Some business processes can be built once and shared as a
corporate standard
Governance
Certain data sets need complex security and privacy controls
Efficiency
No need to repeat design, preparing, and loading or securing
Line-of-business sources
Data ingestion &
transformation
Enterprise models
Azure Synapse
Analytics
Power
BI

Azure
If enterprise models
are so important, why
do users need self-
service BI models?
BI models in the
enterprise environment
Flexibility
Some data sets are temporary, external, or ad-hoc don’t need to
be consolidated
Efficiency
Tech-smart business users have fresh and innovative ideas they
need to explore with agility
Ad-hoc, departmental and
external sources
Line-of-business sources
Data ingestion &
transformation
Power
BI
Enterprise models
Azure Synapse
Analytics
BI models

Azure
What is the role of
the data warehouse
with data science?
Data science models in the
enterprise environment
Integrating results with enterprise models
Making the results of data science easily available for business functions
Serving enterprise data for data scientists
Helps ensure consistency across diverse analyses
Power
BI
Azure Synapse
Analytics
Azure
Databricks
Enterprise models
Azure Synapse
Analytics
Data science results

Analytics in a Day Ft. Synapse Virtual Workshop

Limitless analytics service with
unmatched time to insight

The first unified, cloud native
platform for converged analytics
Azure Synapse is the only unified platform for
analytics, blending big data, data warehousing,
and data integration into a single cloud native
service for end-to-end analytics at cloud scale.

Powered by a new cloud native
distributed SQL engine

Flexible consumption models
Serverless pay-per-query ideal for ad-hoc data lake
exploration and transformation
Dedicated clusters optimized mission-critical data
warehouse workloads
Serverless Dedicated

➢ Comprehensive security and compliance
➢ Streamlined data integration
➢ Flexible data warehousing
➢ Real-time operational analytics
➢ Integrated machine learning
➢ Power BI + Azure Synapse

Security and compliance
Unified governance controls

Category Feature
Data Protection
Data in transit
Data encryption at rest
Data discovery and classification
Access Control
Object level security (tables/views)
Row level security
Column level security
Dynamic data masking
Column level encryption
Authentication
SQL login
Azure active directory
Multi-factor authentication
Network Security
Managed virtual network
Custom virtual network
Firewall
Azure ExpressRoute
Azure Private Link
Threat protection
Threat detection
Auditing
Vulnerability assessment
Isolation
Dedicated metadata store
Hosted in customer tenant
Best-in-class security
Customer & System Managed Keys
All data encrypted by default
Up to 3x levels of data encryption at rest
Democratize data at scale with fine-grained ACL
Proactive protection
Comprehensive Compliance

Eliminate network maintenance
One-click enables automated management of virtual
networks between cluster endpoints
Synapse resources only ever interop with private
endpoints
No management of subnets or IP Ranges
Prevents data exfiltration
Compliance boundary

More than just data security
Native integration with Azure Purview
Automatically discover and classify data assets
End-to-end data lineage

Data warehouse
Scalable and secure analytics platform for SQL workloads

Fully-managed elastic platform
Elastic compute that can be easily optimized to
different classes of workload
All features available in a single tier
Infinite cost effective PAYG storage

SQL Editor
Automatic code completion (Intellisense)
Script collaboration within the Workspace
Built-in visualizations
Easily switch between clusters

TPC-H and TPC-DS Leader
Price/performance leadership relative to other cloud
data warehouses
“Polaris” is the only query engine to successfully
complete TPC-H at 1PB scale
https://aka.ms/synapse-dqp
11.5
7
62
9.5
28
2.5
30.5
48.5
99
22.5
9
5
11.5
6.5
2 5
27
99.5
18.5 21
95.5
8
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22
TPC-H 1 petabyte execution times
$47
$152
$564
$51
DW30000C 4X-Large BigQuery dc2.8xlarge
60N
Test-H 30TB
Price/performance @30TB
($ per query per hour) lower is better
$153
$286 $309
$570
Azure
Synapse
Redshift Snowflake
Enterprise
BigQuery Flat
Rate
Test-DS 30TB
Price/performance comparison (lower is better)
* “GigaOm Analytics Field Test-H Benchmark Report” January 2019; “GigaOm Analytics Field Test-DS Benchmark Report” April 2019

Only platform to complete TPC-H
benchmark at 1 Petabyte
Massive concurrency
Global workload graph
Workload aware query scheduling
https://aka.ms/synapse-dqp

Workload management
Azure Synapse supports a more diverse set of workload management tools through workload importance, intra-cluster isolation, and elastic clusters.
Scale in Scale out
Workload Importance Workload Isolation
Workload Group B 40%
Elastic Cluster (Scale Up)
2000 cDWU

Chooses the most secure cloud DW, Azure Synapse Analytics,
to transform two business critical Teradata systems
Challenge
Solution

Real-time operational analytics
Eliminate latency and accelerate decision making

Integrating operational data with analytics systems

Integrating operational data with analytics systems
Azure Synapse Link

Enable real-time data analytics

Real-time operational analytics
One-click enablement in Azure Portal
No data integration pipelines required
Near-zero impact on operational systems
Latency <90s at 99th percentile
Azure Cosmos DB
Analytical Store
Column store optimized
for analytical queries
Transactional Store
Row store optimized for
transactional operations
Azure Synapse
Analytics
Cloud-Native HTAP
Azure Synapse Link

Event Hubs
IoT Hub
T-SQL language
Built-in streaming ingestion & analytics
Streaming Ingestion Dedicated SQL Pool
Synapse SQL
IoT ingestion without aggregation
Ingress up to 720 gigabytes/hour of raw events
Analyze data in-flight using SQL language Azure
Stream Analytics
Join streaming data with other data assets in the
data warehouse and data lake

“Azure data services have been
seamlessly integrated into existing
infrastructure, which was especially
helpful with respect to authentication
and user access management.”
New ways for optimizing Snow production
and operational costs with Azure Synapse
Challenge
Solution

Machine learning
Empower everyone with predictive insights

Democratize predictive power
Synapse makes predictive analytics accessible to all
Notebooks provides a code authoring experience for
complex predictive models
Automatic ML graphical interface provides a no-
code experience for creating ML models
Native integration with Azure Cognitive Search
provides access to pre-built models
All Code Low/no-Code Pre-built models

Code-first ML model development
PySpark, Scala, and C# languages supported
Automatic code completion (Intellisense)
Author multiple languages in a single notebook
Analyze data from the data warehouse, data lake,
and real-time operational data from one place

Data + Languages
Languages such as SQL, PySpark, Scala and
C# in support of data science and data
warehouse workloads
The data lake supports and unlimited set of file
formats including Parquet, ORC and Json as well as
audio, image, and video formats
Language
Data

All you need is data
Fully automated feature exploration

Code-free in Synapse Studio
No-code creation on Machine Learning models
Democratize ML to everyone since no data science
domain knowledge required
Support for ensemble models
Supports classification, regression, and
time-series forecasting

Code-free in Synapse Studio
No-code references to machine learning models
Democratize ML to everyone since no data science
domain knowledge required
Easily embed in SQL stored procedures for
transformation of Views for reporting

SELECT d.*, p.Score FROM PREDICT(MODEL = @onnx_model, …
In-engine ML scoring
Machine learning models executed using SQL
“In-engine” for performance and scalability
No data leaves the platform for scoring
No additional cost for scoring
T-SQL Language
Synapse SQL
Model Data Predictions

Power BI + Azure Synapse
An unmatched combination

Where do you find yourself on the curve?
Hindsight Insight Foresight
Value
Difficulty
What happened?
Descriptive Analysis
Why did it happen?
Diagnostic Analysis
What will happen?
Predictive Analysis
How can we make it happen?
Prescriptive Analysis

Where do you find yourself on the curve?
Value
Difficulty
What happened?
Why did it happen?
Diagnostic Analysis
What will happen?
Predictive Analysis
BI

BI + Analytics unlock the door to AI, machine learning, and
real-time insights
Value
Difficulty
What happened?
Why did it happen?
Diagnostic Analysis
What will happen?
Predictive Analysis
Analytics
BI

Unified experience to enrich data
Automated ML for rapid development
Seamless collaboration

Build dashboard in Synapse Studio
Code-free experience for development rich
visualizations
One-click publishing to for secure consumption
across the enterprise

Visualize
and report
Power BI
Model
& serve
Azure Synapse
Analytics
CDM
Folders
Azure Data
Lake Storage
Respond instantly
Enable instant response times with
Power BI Aggregations on massive
datasets when querying at the
aggregated level
Get granular with your data
Queries at the granular level are
sent to Azure Synapse Analytics
with DirectQuery leveraging its
industry-leading performance
Save money with
industry-leading performance
Azure Synapse Analytics is up to
14x faster and 94% cheaper than
other cloud providers
View reports with
a single pane of glass
Skip the configuration when
connecting to Power BI with
integrated Power BI-authoring
directly in the Azure Synapse Studio
Accelerate business value with a powerful analytics platform

Customers using Azure Synapse & Power BI today
are transforming their business with purpose
27%
Faster time
to insights
271% Average ROI
26%
Lower total cost
of ownership
60%
Increased customer
satisfaction
* Forrester, October 2019, “The Total Economic Impact of Microsoft Azure Analytics with Power BI”

Understanding insights in the consumer
goods market with Azure Synapse Analytics
Challenge
Solution

Business intelligence
Data integration
Data management Machine learning and AI

Accelerate time to solution
Azure Open Data sets
Pre-built samples to accelerate development
• SQL Scripts
• Notebooks
• Data Pipelines

Hands-on lab
Build an end-to-end analytics solution
with Azure Synapse & Power BI

Azure
Lab Contents Overview

Exercises
Getting setup
You will develop a Power BI model that connects to the Azure Synapse Analytics data
warehouse
You must use the lab Azure credentials to connect to Azure Synapse and publish to Power BI
© 2021 Microsoft. All rights reserved.

Power BI Exercises
Exercise 05: Develop a Power BI Model 45 minutes
Exercise 06: Optimize the Power BI Model 15 minutes
Exercise 07: Create a Model Using DQ to Power BI 20 minutes
Exercise 08: Author a Power BI Report (Optional) 45 minutes

Data integration
Code-free hybrid data integration

Cloud native ETL/ELT
95+ connectors available
Secure connectivity to on-premise data sources,
other clouds, and SaaS applications
Code-first and low/no code design interfaces
Schedule and Event based triggering
Code-free

No/low-code data
transformation
Excel-like interface is familiar to users
Transform data to desired shape completely visually
Operationalize into pipelines
Code-free data
wrangling

Real-time
operational analytics
No data integration pipelines required
Near-zero impact on operational systems
Latency <90s at 99th percentile

Section 02
Deliver Discipline at the Core
Analytics in a Day
Azure Synapse + Power BI better together

Section outline
02: Deliver Discipline at the Core
• Why Power BI data models?
• Power BI data modeling
• Developing models

Why Power BI data models?
• Power BI models are the way to effectively connect decision makers
with the data warehouse
• Semantic layer, providing intuitive browsing and exploration
• Deliver high performance (with the assistance of cached aggregations)
• Encapsulate complex calculation logic
• Source for rich interactive reports, and dashboards

• Power BI data models can scale to query large data stores, such as a
data warehouse
• Typically, these models:
• Are designed and developed by BI developers
• Represent large data volumes
• Support high report user concurrency
• Consideration must be given to the Power BI capacity that will host
the model
Capacities are introduced
later in this section

Architecture
Reports
Dashboards
Azure
Synapse
Analytics
Power BI
service

Power BI data modeling
Model basics
• Power BI is built upon the mature foundations of Analysis Services
• Specifically, it is based on tabular modeling technologies
• Models can source data from practically any data source
• Cloud or on-premises
• File, database, web page, or SaaS provider
• When published to Power BI, a model is known as a dataset
CRM

Model basics
• Models have tables, and tables have columns
• Table are related for purposes of filter propagation
• Often, dimensional modeling (star schema) is the optimal design
approach for Power BI models
• Dimension tables define business entities
• Fact tables define business activity or observations
• A star schema can comprise multiple stars, each based on a single
fact table

Model basics » Star schema
*
1
Fact
Dim
Dim
Dim
Dim
Dim

Model basics » Star schema

Model basics
• Well designed tables and relationships are the model foundations
• Power Query is the technology used to create model tables
• Upon the foundations, modelers can:
• Enhance column properties with default summarization, data categorization,
sorting, or formatting
• Add hierarchies to support drill down/up
• Add calculations to create new tables, columns, or measures, which summarize
model data
• Organize model fields into display folders
• Add roles to enforce row-level security (RLS) for different report user
audiences

Model basics » From source to visualization
Source
Azure
Synapse
Analytics
Model
Modeler view Report author view
Visualization

Storage modes
• Each model table has a storage mode:
• Import (Vertipaq)
• DirectQuery
• Dual—both Import and DirectQuery
• Possible model frameworks:
Table storage Model framework
All tables are Import storage mode Import
All tables are DirectQuery storage mode DirectQuery
Model has Mixed storage mode Composite

Model framework » Import
• Import storage mode caches table data in compressed and
optimized column stores
• Generally, it provides the fastest query performance
• But—requires periodic data refresh to keep data current

• Develop an Import model when:
• The entire model fits into available memory
• Data latency between data refreshes can be tolerated
• Power Query transformations are complex
• Model calculations are complex
• Because data warehouses represent large volumes of data, usually an
Import model can only be considered when:
• Caching high level summarizations
• Caching only recent history

Model framework » DirectQuery
• DirectQuery storage mode is an alternative to Import storage
mode
• When queried, a DirectQuery table uses native queries to retrieve
data from an underlying data source
• Generally, it allows:
• Creating tables over large data volumes
• Achieving near real-time reporting

• Develop a DirectQuery model when:
• Data volume cannot fit into memory
• Data latency must be low
• Power Query transformations are simple
• Model calculations are simple
• This model mode is well-suited for data warehouses
• The model can query all data
• Reports display the latest data

Exercise 06
45 minutes
Develop a Power BI Model
You must use the lab Azure credentials to connect to Azure Synapse and publish to Power BI
• Section 1: Create the Model
• Section 2: Develop the Model
• Section 3: Test the Model

Get started today
Create a free Azure account and get started with Azure Synapse Analytics:
https://azure.microsoft.com/en-us/free/synapse-analytics/
Submit info for free proof of value package for modernizing SAP workloads:
https://aka.ms/synapse-qlik
Submit info for free proof of value package for migrating on-prem data warehouse:
http://aka.ms/synapse-informatica
Learn more:
https://aka.ms/synapse

Use the limited time offer to save up to 76
percent when migrating to Azure Synapse
https://aka.ms/synapse-migration-offer

Analytics in a Day
Thank You!
James McAuliffe
jmcauliffe@ccganalytics.com
https://www.linkedin.com/in/jamesmcauliffesql/
https://ccganalytics.com/

Supply Chain Intelligence
and Finance Analytics Migrate and Modernize
Data from
SAP ECC
Legacy DW
appliance
Each motion has a customized free proof of value package

Azure Synapse + Power BI + Qlik Data Integration
Free Proof of Value for SAP ERP
Free Deep Dive Session
Free ½ Day Solution
Architecture Workshop
Free Software
Subscriptions
Free Technical and
Subject Matter Expertise
Azure Synapse Expert Support
Azure Synapse Expert Support
(data warehousing) + Power BI
(for duration of POV)
Azure Synapse Analytics and
Power BI Expertise
SAP Real-time Data Integration
Expert Support
1:1 Deep-Dive with Qlik Data
Integration Team
Qlik Data Integration for Data
Warehouse Automation
Qlik Data Integration and
SAP Expertise

Get started today with your free proof
of value from Microsoft and Qlik
https://aka.ms/synapse-qlik

Azure Synapse + Informatica
Free Proof of Value
Free Hands-On
Migration Workshop
TCO Assessment
Free Data Catalog
& ETL Migration
Free Code Conversion
Global Black Belt Support
On-premises Data Warehouse
Code Conversion
Azure Synapse Analytics (data
warehousing) Subscription
(for duration of PoV up to 30 days)
Cloud Data Warehouse Modernization
Workshop on Azure
Customized Migration
TCO Assessment
Enterprise Data Catalog (EDC)
Intelligent Cloud Services (IICS)
(for duration of PoV up to 30 days)
Free Azure Synapse
Subscription

Get started today with your free proof
of value from Microsoft and Informatica
http://aka.ms/synapse-informatica

Analytics in a Day Ft. Synapse Virtual Workshop

Recommended

More Related Content

What's hot (20)

Similar to Analytics in a Day Ft. Synapse Virtual Workshop (20)

More from CCG (20)

Recently uploaded (20)

Analytics in a Day Ft. Synapse Virtual Workshop