Exploring Microsoft Azure Infrastructures

Data Analytics Meetup (DAM)
March 27, 2018

AGENDA
Disruption = Opportunity
Traditional to Modern
Azure DB vs DW
Azure Data Integration
Analysis Services
Power BI
Wrap-Up

Widely available
unstructured data
Deliver powerful insights
over big data
Massive
data growth
Build data
into all your apps
50ZB
1010
0101
0010
{ }
Analytics driving
digital transformation
Modernize your apps
with built-in analytics
New Opportunities
in a Changing Data Landscape

SINCE 2000, 52% OF THE FORTUNE 500 COMPANIES HAVE
DISAPPEARED
0
20
40
60
80
1955 2015
Avg. Life
Expectancy
for a
Company
(years)
Source: Digital Transformation by Mark Baker

67
Years
25
Years
15
Years
1920 1930 1940 1950 1960 1970 1980 1990 2000 2010's 2020's
The Time to Adapt to Disruption is Shrinking
Source: BBC
A hundred years ago, the average lifespan of a company
listed on the S&P 500 index was 67 years
In the 2020s, 75% of the
S&P 500 will be new companies

Exploring Microsoft Azure Infrastructures

Digital Transformation is a Team Effort
CIO
Business
Program and Portfolio
Management
Sourcing and
Vendor Management
Applications
Enterprise
Architecture and
Technology Innovation
Data and Analytics
Security
and Risk
Infrastructure and
Operations

Invest in developing PEOPLE
to support analytic adoption &
create a data-driven culture
Embrace agile
PROCESS that supports
analytics at speed
Connect and collect
DATA up, across
and out
Integrated and Inclusive Strategy
Unlock VALUE
& Insights
Take advantage of the modern
TECHNOLOGY to drive self-
service, embedded and viral
analytics

Traditional to Modern Architecture
Transitioning BI Architecture to Azure

Traditional Business Intelligence Architecture

Azure PaaS
Key Benefits
Less Administration - free up
Administrative resources
Scalability - When using SQL
Database, you pay-as-you-go
with options to scale up or
out for greater power with no
interruption
Faster Deployments
No Operating System
Cost/Updates Overhead
Backups are done by default
Development continues
without any major changes.

Modern Business Intelligence Landscape

Modern BI with Advanced Analytics

Azure SQL DB and Azure SQL DW
Azure Databases

Azure SQL Database DTU – Database Transaction Unit
Number of Dedicated SQL database
resources (CPU, Memory, Read and
writes) available for SQL Database
Doubling the DTUs by increasing the
performance level of a database
equates to doubling the set of
resource available to that database.
For example, a Premium P11
database with 1750 DTUs provides
350x more DTU compute power than
a Basic database with 5 DTUs

Azure SQL Database eDTU – Elastic Database Transaction Unit
Rather than providing dedicated set of DTUs to a SQL database you can add
a eDTU pool shared by multiple databases to accommodate unpredictable
periods of usage by individual databases.
SQL Database elastic pools are a simple, cost-effective solution for
managing and scaling multiple databases that have varying and
unpredictable usage demands
While the eDTU unit price for a pool is 1.5x greater than the DTU unit price
for a single database, pool eDTUs can be shared by many databases and
fewer total eDTUs are needed.
Pool eDTU can be used by applications Databases during daily Operations
activities and Processes like ETL/Data load Databases can utilize the same
eDTU for Nightly Processing.

Application benefits and BI
SQL Database in Azure
Limitations – 4TB max is a sizing limitation in Single SQL database but can
overcome by Scaling out called Sharding.
Sharding – Scaling out - way to overcome single database size restriction by
distributing data across multiple databases
Azure SQL Database Managed Instance – New offering currently in preview
– Managed instance allows existing SQL Server customers to lift and shift
their on-premises applications to the cloud with minimal application and
database changes. At the same time, Managed Instance preserves all
PaaS capabilities (automatic patching and version updates, backup, high-
availability), that drastically reduces management overhead and total cost
of ownership

Azure SQL Data Warehouse
SQL Data Warehouse is a cloud-based Enterprise
Data Warehouse (EDW) that leverages Massively
Parallel Processing (MPP) to quickly run complex
queries across petabytes of data
Data is ingested into big data stores from a variety
of sources.
Once in a big data store, Hadoop, Spark, and
machine learning algorithms prepare and train the
data
When the data is ready for complex analysis, SQL
Data Warehouse uses PolyBase to query the big
data stores. PolyBase uses standard T-SQL queries
to bring the data into SQL Data Warehouse.

Azure SQL Data Warehouse (DWU)
Azure DW leverages Scale Out architecture to distribute computational
processing of data across multiple nodes. The unit of scale is an
abstraction of compute power that is known as data warehouse unit
(DWU)
Azure DW uses a Node based architecture. Applications connect and
issue Tsql commands to a control node. The control node runs the MPP
engine which optimizes queries for parallel processing.
Data Movement Service (DMS) Coordinates the data movements
between compute nodes.
Distribution is the basic unit of storage and processing for parallel
queries. Each compute node manages one or more of 60 distributions.

Azure DW Distributions
Types of Distributions –
Hash – Provides the highest query performance for joins and aggregation on Large tables
– Fact – use Hash distribution with clustered column store index. Performance improves when two has tables are
joined on the same distribution column. Large Dimensions or any large tables to store on each compute node
use hash distributed.
Round Robin – Simplest table to create and delivers fast Performance when used as staging table for loads
– Staging tables- Round robin is ideal for staging tables in Azure DW
Replicated – Provides fastest query performance for small tables
– Small Dimensions – Smaller tables can utilize replicated Distribution.

Azure DW Distributions (Cont.)

Traditional on premises installations of SQL server the OLTP and OLAP the systems sat on different server but
the software was the same. OLTP instance has more 3rd Normal form structures for
Operational/Transactional processing vs the OLAP install favored Dimensional Modelling for reporting
purposes.
When connecting to Azure SQL Database and Azure SQL data warehouse with SSMS might have the same
look and feel on surface. They both are PaaS offerings but under the covers they are very different
Azure SQL Database is a single database and the concept of SQL server in Azure is more of a container for
logically grouping the Azure SQL Databases
Azure SQL Data Warehouse was built for OLAP systems. Massively Parallel processing system or MPP are
made up of multiple nodes each with their own resources and they work together to provide increased
performance. Azure SQL Databases is a Paas offering of an OLTP database.
Azure SQL DB vs Azure SQL Data Warehouse

SQL Database vs SQL Data Warehouse

BI Tools
Azure Data Factory and SSIS
Azure BI Development tools

Development Tools Azure Data Factory V2
Azure Data Factory is a hybrid data integration service that allows you to create, schedule and orchestrate your ETL/ELT workflows at
scale wherever your data lives, in cloud or self-hosted network.
Schedule and manage your data transformation and analysis process, add a Hadoop processing step for big or semi-structured data, a
stored procedure invocation step for structured data, a machine-learning step for analytics, or insert your own custom code as a
processing step in any pipeline.
With Data Factory Version 2 you can lift and deploy your on perm SSIS/SSDT package to SQL Data Factory.

SSIS/SSDT Connection Managers for Azure
SSIS/SSDT and Azure
SSIS/SSDT Control Flow components for Azure
SSIS/SSDT Data Flow components for Azure

Demo – SSIS/SSDT – Data Factory
SSIS/SSDT and Data Factory

Azure Analysis Services
Tabular PaaS Offering

Azure Analysis Services – PaaS Offering
Azure Analysis Services supports tabular models at the 1200 and 1400 compatibility levels. Partitions, row-level security, bi-directional
relationships, and translations are all supported. In-memory and DirectQuery modes mean lightning fast queries over massive and
complex datasets.
For Scaling and Cost management you can change plans up or down within the same tier, or upgrade to a higher tier, but you cannot
downgrade from a higher tier to a lower tier. You pay for only what you use so you can pause the instance during non business hours.
Its easy to migrate on premises Tabular cubes using SSDT or using SSMS. If your cube uses On Premises data sources you need to
install and configure on premises data gateway.

Reporting
Power BI – SSRS – Other technologies

Power BI Reporting
A Power BI report is a multi-perspective
view into a dataset, with visualizations
that represent different findings and
insights from that dataset. A report can
have a single visualization or pages full of
visualizations
Data sources pointed to Local Instance
and Azure Analysis services instance.
Simple change

Demo - Cube and Power BI Sample
Power BI

On Premises Gateway for On Premises Data Sources
The on-premises data gateway acts as a bridge,
providing secure data transfer between on-premises
data sources and your Azure Analysis Services servers
in the cloud.
The gateway also works with Azure Logic Apps, Power
BI, Power Apps, and Microsoft Flow. You can associate
multiple services in the same region with a single
gateway.

On Premises Gateway for On Premises Data sources (Cont.)
Few steps in getting this setup:
– Download and run setup - Installs a gateway service on a computer in your organization using an account in your tenant's
Azure AD. Azure B2B (guest) accounts are not supported.
– Register your gateway - Specify a name and recovery key for your gateway and select a region, registering your gateway
with the Gateway Cloud Service.
– Create a gateway resource in Azure - create a gateway resource in your Azure subscription.
– Connect your servers to your gateway resource - Once you have a gateway resource in your subscription, you can begin
connecting your servers to it. You can connect multiple servers and other resources to it.

1 2 3
BI Supporting Tools for development
Azure Storage Explorer
Easily manage the
contents of your storage
account with Azure
Storage Explorer. Upload,
download, and manage
blobs, files, queues,
tables, and Cosmos DB
entities. Gain easy access
to manage your virtual
machine disks.
Link to Azure Storage
Explorer
SSIS Azure Feature Pack
SQL Server Integration
Services (SSIS) Feature
Pack for Azure is an
extension that provides
the components listed on
this page for SSIS to
connect to Azure
services, transfer data
between Azure and on-
premises data sources,
and process data stored
in Azure
Link to Azure Feature
Pack for SSIS
On Premises Gateway
The on-premises data
gateway acts as a bridge,
providing quick and secure
data transfer between on-
premises data (data that is
not in the cloud) and the
Power BI, Microsoft Flow,
Logic Apps, and PowerApps
services.
Link to On Premises
Gateway

Compliance and Regulations
Azure

Compliance - HIPPA/HITECH, HITRUST and more.
– Link to compliance offerings
Privacy – Microsoft is governed by strict standards regarding the privacy and protection of customer data. Its now General Data
Protection Regulation (GDPR) compliant. More info Click link below
– Link to privacy information
Transparency – Where is my data? How is it Secured? Who can Access it?
– Link to transparency information
Azure Government - environment includes a unique cloud instance, exclusively for government customers and their solution
providers, and hardened US data centers that are operated by extensively screened personnel.
– Link to Azure government information
Azure Industries - Azure solutions are used by multiple industries, including banking, healthcare, manufacturing, and
telecommunications.
– Link to Azure industries information
Microsoft Azure

Additional Resources
Azure 101 -
– Interactive website to learn all Azure offerings
– Link to Azure 101
Azure SLA (Service Level Agreements)
– Link to Azure SLA
Azure Updates – Releases
– Link to Azure Updates
Azure Pricing Calculator
– Link to Pricing Calculator

41 © 2017 Gartner, Inc. and/or its affiliates. All rights reserved.
Abundance Data and information
Insight and action
Data, and the ability to analyze and apply data, is central to how
firms compete and collaborate.
The Paradox

Expectations are High, Analytics are at the Center of it
Percentage of respondents to select each technology as the top three
Q. In your opinion, which three of these technologies have the most
potential to change your organization over the next five years?
81% 48% 43% 40%
Advanced Analytics Internet of Things Digital Security
Business
Algorithms
Machine
Learning
Virtual
Customer
Assistants
Augmented
Reality
Blockchain Autonomous
Vehicles
Smart
Robots
22% 19% 13%
10% 7% 6%
For more details on this research, see "The 2017 CIO Agenda: Seize the Digital Ecosystem Opportunity."

Scarcity = Opportunity
Chief Data Officer (CDO) Chief Analytics Officer (CAO)
Informatio
n
Architects
Data Scientists
Academia
Information
and Analytics
Technology
Providers
Information
and Analytics
Service
Providers
CEOs, CIOs and Boards
"Skill shortages are the biggest barrier
facing CIOs in all three performance
bands, especially top performers. Drilling
down shows skill gaps across a broad
spectrum, from information and analytics to
basic IT skills." — Gartner CIO Survey,
2017
Organizations need data and analytics professionals and
business hybrids to succeed in digital business

Holistic View
– People – Skillsets, Roles,
Responsibilities, Culture,
Literacy
– Process – Collaboration,
Agile Delivery, Governance,
Self-Service, Prototyping,
Security, bimodal
– Technology – Investments,
capabilities, Consolidation,
Security
– Data – Unstructured,
Structured, Internal,
External, Security, Literacy
Technology Enablement
Process Enablement
People Enablement
Data Enablement

Feedback?
Email questions or comments to:
DAMfeedback@ccgbi.com

Thank you
www.ccganalytics.com | (813) 968-3238

Exploring Microsoft Azure Infrastructures

Recommended

More Related Content

What's hot (20)

Similar to Exploring Microsoft Azure Infrastructures (20)

More from CCG (20)

Recently uploaded (20)

Exploring Microsoft Azure Infrastructures