SlideShare a Scribd company logo

PaaSport to Paradise: Back to the Future with SSIS in Azure Data Factory

Download as PPTX, PDF
••
S
Sandy Winarko

The document outlines the migration and operationalization strategies for SQL Server Integration Services (SSIS) into Azure Data Factory (ADF) and the benefits of utilizing a Platform as a Service (PaaS) model. It compares traditional ETL processes with modern cloud-based solutions, emphasizing the ease of transitioning workloads while highlighting challenges such as on-premises data access and the complexities of scheduling and moving packages. It also discusses licensing considerations and provides guidance through documentation and resources for effective implementation.

Data & Analytics
1 of 66
Downloaded 16 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
PaaSport to Paradise: Back to the Future with SSIS in Azure Data Factory
Agenda  Secure Data Access from SSIS PaaS SSIS PaaS
Learning Objectives
Learning Objectives prepare migrate operationalize
Learning Objectives prepare migrate operationalize  Complement
Learning Objectives prepare migrate operationalize  Complement innovate
SSIS + ADF = SSIS in ADF
SSIS Analogy • Running traditional ETL workloads on premises is like being on the ground floor
SSIS + ADF Analogy • Running traditional ETL workloads on premises is like being on the ground floor • Running modern ELT workloads in the cloud on a fully managed Platform as a Service (PaaS) is like being on the 7th floor, so jumping 7 stories straight up is virtually impossible
SSIS + ADF = SSIS in ADF Analogy • Running traditional ETL workloads on premises is like being on the ground floor • Running modern ELT workloads in the cloud on a fully managed Platform as a Service (PaaS) is like being on the 7th floor, so jumping 7 stories straight up is virtually impossible • Running SSIS in ADF is like taking an escalator that seamlessly carries you to the 7th floor step-by-step with the option to get on/off anytime at any floor.
SQL Migration to an All PaaS Solution
SQL Migration to an All PaaS Solution Cloud On-premises
SQL Migration to an All PaaS Solution Cloud On-premises
SQL Migration to an All PaaS Solution Cloud On-premises
SQL Migration to an All PaaS Solution Cloud On-premises
SQL Migration to an All PaaS Solution Cloud On-premises
SQL Migration to an All PaaS Solution Cloud On-premises
SSIS On-Premises-to-Cloud Evolution
Linux Scale-Out SSIS On-Premises-to-Cloud Evolution On-premises data stores Cloud On-premises SQL Server/ File System MSDB SSISDB
Project + Package Deployment Models Azure Files SSIS IR Azure-enabled SSDT SSMS dtutil dtexec DF UI Azure SQL MI Agent custom setups licensing BYOL AHB here SSIS On-Premises-to-Cloud Evolution On-premises data stores Azure SQL DB/Managed Instance (MI)/Azure Files Azure Data Factory (ADF) Cloud data stores Cloud On-premises SQL Server/ File System MSDB SSISDB MSDB SSISDB
VNet proxy here Windows authentication AKV AAD first-class chain/group Azure Monitor Azure Data Catalog (ADC) GA SSIS On-Premises-to-Cloud Evolution On-premises data stores Azure SQL DB/Managed Instance (MI)/Azure Files VNet/ Proxy Azure Data Factory (ADF) Cloud data stores Cloud On-premises SQL Server/ File System MSDB SSISDB MSDB SSISDB
Secure Data Access from SSIS PaaS
Secure Data Access from SSIS PaaS
Alternative Methods to Access On-Prem Data
Joining SSIS IR to a VNet Connected to On-Prem Network
Forced Tunneling for Outbound Traffic Inspection
Configuring Self-Hosted IR as a Proxy for SSIS IR
Extensibility Features for SSIS PaaS
Enterprise Edition – Introduction
Custom Setup – Introduction use strong cryptography/more secure network protocol, etc.
3rd Party Licensing – Introduction
SSIS IR node Container ISV Setup1. Specify Product Key in setup script ISV Activation Server 2. Get Activation Key by submitting Cluster ID + Product Key Local Store (e.g. Registry) 3. Write Activation Key SSIS Executor ISV Extension 4. Read Activation Key and validate it with Cluster ID Setup Runtime 4. Get Cluster ID 4. Report on Node Count (Optional) SSIS Runtime 2. Activation Key 3rd Party Extensibility – Implementation
3rd Party Ecosystem – ISV Partners
SSIS Migration Playbook
Batch Assess DBA Database Migration Assistant (DMA)/SSDT
Batch Assess Edit TestAssess DBA BI Developer SSDT Database Migration Assistant (DMA)/SSDT
Batch Assess Batch Migrate Edit TestAssess DBA BI Developer SSDT Database Migration Assistant (DMA)/SSDT Azure Files/file system Database Migration Service (DMS)/Deployment Wizard/dtutil utility/manual copy SSISDB hosted by Azure SQL DB/MI, MSDB hosted by Azure SQL MI
Source: SQL Server + SQL Agent Destination: Azure SQL MI + MI Agent Destination: Azure SQL DB + ADF Storage Types Package Assessment Package Migration Job Migration Package Migration Job Migration SSISDB • DMA • SSDT • SSISDB to SSISDB via DMS • SQL Agent to MI Agent via DMS (TBD) • SQL Agent to MI Agent via PowerShell/T-SQL/C# script • Recreate in MI Agent via SSMS • SSISDB to SSISDB via DMS/SSMS (TBD) • Redeploy to SSISDB via SSDT/SSMS • SQL Agent to ADF via SSMS (TBD) • Recreate in ADF via SSMS • Recreate in ADF
Source: SQL Server + SQL Agent Destination: Azure SQL MI + MI Agent Destination: Azure SQL DB + ADF Storage Types Package Assessment Package Migration Job Migration Package Migration Job Migration SSISDB • DMA • SSDT • SSISDB to SSISDB via DMS • SQL Agent to MI Agent via DMS (TBD) • SQL Agent to MI Agent via PowerShell/T-SQL/C# script • Recreate in MI Agent via SSMS • SSISDB to SSISDB via DMS/SSMS (TBD) • Redeploy to SSISDB via SSDT/SSMS • SQL Agent to ADF via SSMS (TBD) • Recreate in ADF via SSMS • Recreate in ADF File System • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • SQL Agent to ADF via SSMS • Recreate in ADF via SSMS • Recreate in ADF
Source: SQL Server + SQL Agent Destination: Azure SQL MI + MI Agent Destination: Azure SQL DB + ADF Storage Types Package Assessment Package Migration Job Migration Package Migration Job Migration SSISDB • DMA • SSDT • SSISDB to SSISDB via DMS • SQL Agent to MI Agent via DMS (TBD) • SQL Agent to MI Agent via PowerShell/T-SQL/C# script • Recreate in MI Agent via SSMS • SSISDB to SSISDB via DMS/SSMS (TBD) • Redeploy to SSISDB via SSDT/SSMS • SQL Agent to ADF via SSMS (TBD) • Recreate in ADF via SSMS • Recreate in ADF File System • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • SQL Agent to ADF via SSMS • Recreate in ADF via SSMS • Recreate in ADF MSDB • Same as above • MSDB to MSDB via DMS (TBD) • Redeploy to MSDB via dtutil • Reconfigure w/ SSIS IR package store • Same as above • Export to Azure Files/file system via SSMS/dtutil • Recreate in ADF via SSMS • Recreate in ADF
Source: SQL Server + SQL Agent Destination: Azure SQL MI + MI Agent Destination: Azure SQL DB + ADF Storage Types Package Assessment Package Migration Job Migration Package Migration Job Migration SSISDB • DMA • SSDT • SSISDB to SSISDB via DMS • SQL Agent to MI Agent via DMS (TBD) • SQL Agent to MI Agent via PowerShell/T-SQL/C# script • Recreate in MI Agent via SSMS • SSISDB to SSISDB via DMS/SSMS (TBD) • Redeploy to SSISDB via SSDT/SSMS • SQL Agent to ADF via SSMS (TBD) • Recreate in ADF via SSMS • Recreate in ADF File System • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • SQL Agent to ADF via SSMS • Recreate in ADF via SSMS • Recreate in ADF MSDB • Same as above • MSDB to MSDB via DMS (TBD) • Redeploy to MSDB via dtutil • Reconfigure w/ SSIS IR package store • Same as above • Export to Azure Files/file system via SSMS/dtutil • Recreate in ADF via SSMS • Recreate in ADF SSIS Package Store • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Redeploy to MSDB via dtutil • Reconfigure w/ SSIS IR package store • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Export to Azure Files/file system via SSMS/dtutil • Same as above
SSIS PaaS Preparation – Demos
SSIS PaaS Preparation – Demos
SSIS PaaS Preparation – Demos
SSIS Migration – Demos
SSIS Migration – Demos
SSIS Operationalization – Demos
SSIS Operationalization – Demos
Appendix
Why Should You Care? When? How?
Why Should You Care? Better TCO & ROI! Source: The Business Value of Microsoft Azure SQL Database Services, IDC, Mar 2015 The Total Economic Impact of Microsoft Azure SQL Database Managed Instance, Forrester Consulting, Sep 2018
Why Should You Care? BYOL to Get AHB!  Maximize the value of on-premises SQL Server license investments by migrating SSIS in ADF/Azure-SSIS Integration Runtime (IR) Editions SQL Server with Software Assurance
Why Should You Care? Huge Opportunity! Source: Worldwide Database Server Forecast, IDC, Dec 2016 Worldwide Database Server Forecast Modernizing opportunity with SQL Server 37% of all units run Microsoft, >50% of them run 2008/R2 or older versions Non-Relational Open Source PaaS & IaaS Microsoft Oracle IBM SAP New SQL Other MySQL No SQL
Why Should You Care? Do The Math! Source: Feb 2019 snapshot of active SSIS instances extrapolated from SSIS 2016/2017 telemetry & CSS case volume for SQL 2008-2017 SSIS 2008-2017 on premises/Azure VMs/AWS VMs (100%) SSIS 2008-2017 on premises (88%) SSIS 2008/R2 EOS w/o SSISDB (13%) SSIS 2012-2017 w/ SSISDB (21%) Total Addressable Market Present Lift & Shift Target Immediate Target Past Target
When Should You Care? NOW!
How Should You Care? Fear is a Great Motivator!  Fear of missing out  Separation of storage/DB engine from compute/ETL engine  High Availability (HA) via multi-node cluster  Disaster Recovery (DR) via standby SSIS IR in a paired region  Dynamic scalability (in/out/up/down)
 Fear of losing investments  Too many SSIS packages to convert from using OLEDB connection via Win Auth to using ADO.NET connection via AAD Auth w/ ADF managed identity  Too many SSIS packages to convert from Package Deployment Model w/o SSISDB to Project Deployment Model w/ SSISDB  Too many SSIS jobs scheduled by 3rd party orchestrators (e.g. Active Batch, Control-M, etc.) invoking dtexec  Too many packages using absolute paths w/ hard-coded drives (e.g. C:)  SQL MI Agent doesn’t support SSIS environment/32-bit runtime/property override  SQL MI Agent can’t invoke dtexec to execute SSIS packages stored in MSDB/file system/SSIS package stores  Network configuration/policy is too complex/strict to access on-premises data via VNet  Custom setups w/ script are too complex/error-prone How Should You Care? Fear is a Great Motivator!  Fear of missing out  Separation of storage/DB engine from compute/ETL engine  High Availability (HA) via multi-node cluster  Disaster Recovery (DR) via standby SSIS IR in a paired region  Dynamic scalability (in/out/up/down)
How Should You Care? Fear is a Great Motivator!  Fear of losing investments  Too many SSIS packages to convert from using OLEDB connection via Win Auth to using ADO.NET connection via AAD Auth w/ ADF managed identity – Support OLEDB connection via AAD Auth w/ ADF managed identity, switch at run-time via property overrides  Too many SSIS packages to convert from Package Deployment Model w/o SSISDB to Project Deployment Model w/ SSISDB – Support file system/Azure Files/MSDB hosted by SQL MI to store packages, turn SSISDB from a blocker into an upsell​  Too many SSIS jobs scheduled by 3rd party orchestrators (e.g. Active Batch, Control-M, etc.) invoking dtexec – Azure-enable dtexec to invoke Execute SSIS Package activities in ADF pipelines  Too many packages using absolute paths w/ hard-coded drives (e.g. C:) – Replace absolute paths w/ UNC paths at run-time via property overrides​  SQL MI Agent doesn’t support SSIS environment/32-bit runtime/property override – Achieve feature parity w/ SQL Server Agent  SQL MI Agent can’t invoke dtexec to execute SSIS packages stored in MSDB/file system/SSIS package stores – ADF-enable SQL MI Agent to invoke executions of packages stored in file system/Azure Files/SSIS IR package stores  Network configuration/policy is too complex/strict to access on-premises data via VNet – Support Self-Hosted IR as proxy for Azure-SSIS IR to access on-premises data  Custom setups w/ script are too complex/error-prone – Support custom setups w/o script  Fear of missing out  Separation of storage/DB engine from compute/ETL engine  High Availability (HA) via multi-node cluster  Disaster Recovery (DR) via standby SSIS IR in a paired region  Dynamic scalability (in/out/up/down)
How Should You Care? Fear is a Great Motivator!  Minimizing fear of losing investments – No more lift & shift blockers Rebalancing your risks/fears  Maximizing fear of missing out  Separation of storage/DB engine from compute/ETL engine  High Availability (HA) via multi-node cluster  Disaster Recovery (DR) via standby SSIS IR in a paired region  Dynamic scalability (in/out/up/down)  ADO.NET/OLEDB/Storage connection via AAD Auth w/ ADF managed identity  AKV support on Execute SSIS Package activity  SSIS ADF connectors (80+)  PQ Source  DQS migration to cloud alternatives running on SSIS in ADF  More cloud-first/cloud-only features, etc.
Resources
sawinark@microsoft.com https://techcommunity.microsoft.com/t5/SQL-Server-Integration- Services/bg-p/SSIS Contacts
https://www.slideshare.net/SandyWinarko/paasport-to-paradise-back-to-the-future- with-ssis-in-azure-data-factory Documentations Provisioning Azure-SSIS IR Using UI: https://docs.microsoft.com/azure/data-factory/tutorial-deploy-ssis-packages-azure Using PSH: https://docs.microsoft.com/azure/data-factory/tutorial-deploy-ssis-packages-azure-powershell Using ARM template: https://docs.microsoft.com/azure/data-factory/create-azure-ssis-integration-runtime#azure- resource-manager-template Provisioning Azure-SSIS IR using Azure SQL DB/Managed Instance w/ VNet service endpoint/private endpoint to host SSISDB Using UI/PSH: https://docs.microsoft.com/azure/data-factory/create-azure-ssis-integration-runtime Joining a VNet: https://docs.microsoft.com/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network Provisioning Azure-SSIS IR using AAD Auth w/ ADF managed identity Using UI/PSH: https://docs.microsoft.com/azure/data-factory/create-azure-ssis-integration-runtime Enabling AAD Auth: https://docs.microsoft.com/azure/data-factory/enable-aad-authentication-azure-ssis-ir Provisioning Azure-SSIS IR to access data on premises Using UI/PSH: https://docs.microsoft.com/azure/data-factory/create-azure-ssis-integration-runtime Joining a VNet: https://docs.microsoft.com/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network Using Self-Hosted IR as a proxy: https://docs.microsoft.com/azure/data-factory/self-hosted-integration-runtime-proxy- ssis Monitoring Azure-SSIS IR Using UI/PSH: https://docs.microsoft.com/azure/data-factory/monitor-integration-runtime#azure-ssis-integration- runtime Using Azure Monitor: https://docs.microsoft.com/azure/data-factory/monitor-using-azure-monitor#monitor-ssis- operations-with-azure-monitor
Documentations Configuring Azure-SSIS IR Using UI/PSH: https://docs.microsoft.com/azure/data-factory/manage-azure-ssis-integration-runtime Configuring Azure-SSIS IR to use Enterprise Edition Using UI/PSH: https://docs.microsoft.com/azure/data-factory/how-to-configure-azure-ssis-ir-enterprise-edition Configuring Azure-SSIS IR to use custom setups Using UI/PSH: https://docs.microsoft.com/azure/data-factory/how-to-configure-azure-ssis-ir-custom-setup Configuring Azure-SSIS IR for high performance Using UI/PSH: https://docs.microsoft.com/azure/data-factory/configure-azure-ssis-integration-runtime- performance Configuring Azure-SSIS IR for disaster recovery Using UI/PSH: https://docs.microsoft.com/azure/data-factory/configure-bcdr-azure-ssis-integration-runtime Deploying projects/packages to run on Azure-SSIS IR Connecting SSMS to SSISDB: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-connect-to- catalog-database?view=sql-server-2017 Using UI/PSH: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-deploy-run-monitor- tutorial?view=sql-server-2017 Using dtutil: https://docs.microsoft.com/azure/data-factory/azure-ssis-integration-runtime-package- store#deploying-multiple-packages-with-dtutil
Documentations Connecting to data stores using Win Auth on Azure- SSIS IR Using 4 methods: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-connect-with-windows- auth?view=sql-server-2017 Connecting to data stores using AAD Auth w/ ADF managed identity on Azure-SSIS IR Using Azure Storage connections: https://docs.microsoft.com/sql/integration-services/connection- manager/azure-storage-connection-manager#managed-identities-for-azure-resources-authentication Using ADO.NET connections: https://docs.microsoft.com/sql/integration-services/connection-manager/ado-net- connection-manager#managed-identities-for-azure-resources-authentication Using OLEDB connections: https://docs.microsoft.com/sql/integration-services/connection-manager/ole-db- connection-manager#managed-identities-for-azure-resources-authentication Connecting to file shares and processing files on Azure- SSIS IR Using local file system/file shares: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-files- file-shares?view=sql-server-2017 Using Win Auth: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-connect-with-windows- auth?view=sql-server-2017 Running packages on Azure-SSIS IR From SSDT: https://docs.microsoft.com/azure/data-factory/how-to-invoke-ssis-package-ssdt Using SSMS/stored procedures/scripts/code: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis- azure-run-packages?view=sql-server-2017 Using AzureDTExec: https://docs.microsoft.com/azure/data-factory/how-to-invoke-ssis-package-azure-enabled- dtexec As Execute SSIS Package activities in ADF pipelines: https://docs.microsoft.com/azure/data-factory/how-to- invoke-ssis-package-ssis-activity As Stored Procedure activities in ADF pipelines: https://docs.microsoft.com/azure/data-factory/how-to-invoke- ssis-package-stored-procedure-activity
Documentations Scheduling package executions on Azure-SSIS IR Using SSMS/ADF/other methods: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure- schedule-packages?view=sql-server-2017 Using SSMS: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-schedule-packages- ssms?view=sql-server-2017 As Execute SSIS Package activities in ADF pipelines: https://docs.microsoft.com/azure/data-factory/how-to- invoke-ssis-package-ssis-activity Using Azure SQL MI Agent: https://docs.microsoft.com/azure/data-factory/how-to-invoke-ssis-package- managed-instance-agent Scheduling Azure-SSIS IR provisioning on demand/just in time Using Web activities: https://docs.microsoft.com/azure/data-factory/how-to-schedule-azure-ssis-integration- runtime Cleaning up package execution logs in SSISDB Using Elastic Jobs: https://docs.microsoft.com/azure/data-factory/how-to-clean-up-ssisdb-logs-with-elastic-jobs Assessing SSIS packages for migration Using DMA: https://docs.microsoft.com/sql/dma/dma-assess-ssis?view=sql-server-ver15 Using SSDT: https://docs.microsoft.com/azure/data-factory/how-to-invoke-ssis-package-ssdt#assess-ssis- projectpackages-for-executions-in-azure Migrating SSIS packages SSIS Migration Playbook: https://docs.microsoft.com/azure/data-factory/scenario-ssis-migration-overview Using DMS: https://docs.microsoft.com/azure/dms/how-to-migrate-ssis-packages-managed-instance Practicing SSIS CI/CD Using Azure DevOps: https://docs.microsoft.com/sql/integration-services/devops/ssis-devops-overview?view=sql- server-ver15 Developing premium/licensed components for Azure- SSIS IR Using Cluster ID: https://docs.microsoft.com/azure/data-factory/how-to-develop-azure-ssis-ir-licensed- components Procuring 3rd party components for Azure-SSIS IR Partner list: https://techcommunity.microsoft.com/t5/SQL-Server-Integration-Services/Enterprise-Edition-Custom- Setup-and-3rd-Party-Extensibility-for/ba-p/388360
Thank you!

More Related Content

PPTX
What’s New with SSIS in ADF
Sandy Winarko
 
PPTX
Embrace and Extend - First-Class Activity and 3rd Party Ecosystem for SSIS in...
Sandy Winarko
 
PPTX
PaaSport to Paradise - Azure SQL and SSIS in Azure Data Factory - Better Toge...
Sandy Winarko
 
PPTX
Running SSIS 2017 at Scale Everywhere
Sandy Winarko
 
PPT
Make Web, Not War - Installfest: Extend Your Web Server, Rodney Buike
Make Web Not War
 
PDF
Best Practices for Running Microsoft SQL Server on AWS
Gianluca Hotz
 
PDF
Help, I need to migrate my On Premise Database to Azure, which Database Tier ...
Erwin de Kreuk
 
PDF
AWS Data migration services
Arun Sirimalla
 
What’s New with SSIS in ADF
Sandy Winarko
 
Embrace and Extend - First-Class Activity and 3rd Party Ecosystem for SSIS in...
Sandy Winarko
 
PaaSport to Paradise - Azure SQL and SSIS in Azure Data Factory - Better Toge...
Sandy Winarko
 
Running SSIS 2017 at Scale Everywhere
Sandy Winarko
 
Make Web, Not War - Installfest: Extend Your Web Server, Rodney Buike
Make Web Not War
 
Best Practices for Running Microsoft SQL Server on AWS
Gianluca Hotz
 
Help, I need to migrate my On Premise Database to Azure, which Database Tier ...
Erwin de Kreuk
 
AWS Data migration services
Arun Sirimalla
 

Similar to PaaSport to Paradise: Back to the Future with SSIS in Azure Data Factory (20)

PPTX
PaaSport to Paradise: Lifting & Shifting with Azure SQL Database/Managed Inst...
Sandy Winarko
 
PPTX
New capabilities for modern data integration in the cloud
Microsoft Tech Community
 
PPTX
New capabilities for modern data integration in the cloud
Gaurav Malhotra
 
PPTX
Modern dataintegration azuredatafactory_ssis
Gaurav Malhotra
 
PDF
SQL Server Integration Services – Enterprise Manageability
Dan English
 
PDF
Migrating Enterprise BI to Azure
Wlodek Bielski
 
PDF
Modern Data Platform Part 1: Data Ingestion
Nilesh Shah
 
PPTX
how to move data from on premise to ssis in google cloud platform ,azure, sno...
KeerthuBabu
 
PPTX
Embrace and extend first-class activity and 3rd party ecosystem for ssis in adf
Tillmann Eitelberg
 
PPTX
Azure SQL DB Managed Instances Built to easily modernize application data layer
Microsoft Tech Community
 
PDF
Migrate SQL Workloads to Azure
Antonios Chatzipavlis
 
PDF
SQL Server 2008 Migration
Mark Ginnebaugh
 
PPTX
SQL to Azure Migrations
Datavail
 
PPTX
Migrating SSIS to the cloud
KoenVerbeeck
 
PDF
DesignMind SQL Server 2008 Migration
Mark Ginnebaugh
 
PDF
Azure SQL Database Managed Instance - technical overview
George Walters
 
PPTX
Upgrade your SQL Server like a Ninja
Amit Banerjee
 
PPT
Saying goodbye to SQL Server 2000
ukdpe
 
PDF
J1 T1 4 - Azure Data Factory vs SSIS - Regis Baccaro
MS Cloud Summit
 
PDF
SQL Server 2012 : réussir la migration - Stéphane Haby - Antonio De Santo - d...
dbi services
 
PaaSport to Paradise: Lifting & Shifting with Azure SQL Database/Managed Inst...
Sandy Winarko
 
New capabilities for modern data integration in the cloud
Microsoft Tech Community
 
New capabilities for modern data integration in the cloud
Gaurav Malhotra
 
Modern dataintegration azuredatafactory_ssis
Gaurav Malhotra
 
SQL Server Integration Services – Enterprise Manageability
Dan English
 
Migrating Enterprise BI to Azure
Wlodek Bielski
 
Modern Data Platform Part 1: Data Ingestion
Nilesh Shah
 
how to move data from on premise to ssis in google cloud platform ,azure, sno...
KeerthuBabu
 
Embrace and extend first-class activity and 3rd party ecosystem for ssis in adf
Tillmann Eitelberg
 
Azure SQL DB Managed Instances Built to easily modernize application data layer
Microsoft Tech Community
 
Migrate SQL Workloads to Azure
Antonios Chatzipavlis
 
SQL Server 2008 Migration
Mark Ginnebaugh
 
SQL to Azure Migrations
Datavail
 
Migrating SSIS to the cloud
KoenVerbeeck
 
DesignMind SQL Server 2008 Migration
Mark Ginnebaugh
 
Azure SQL Database Managed Instance - technical overview
George Walters
 
Upgrade your SQL Server like a Ninja
Amit Banerjee
 
Saying goodbye to SQL Server 2000
ukdpe
 
J1 T1 4 - Azure Data Factory vs SSIS - Regis Baccaro
MS Cloud Summit
 
SQL Server 2012 : réussir la migration - Stéphane Haby - Antonio De Santo - d...
dbi services
 
Ad

Recently uploaded (20)

PPTX
Future_of_AI_Presentation for everyone.pptx
boranamanju07
 
PPTX
Data-Driven Machine Learning for Rail Infrastructure Health Monitoring
Sione Palu
 
PPT
Real Life Application of Set theory, Relations and Functions
manavparmar205
 
PPTX
short term internship project on Data visualization
JMJCollegeComputerde
 
PPTX
INFO8116 - Week 10 - Slides.pptx data analutics
guddipatel10
 
PPTX
Pipeline Automatic Leak Detection for Water Distribution Systems
Sione Palu
 
PPTX
White Blue Simple Modern Enhancing Sales Strategy Presentation_20250724_21093...
RamNeymarjr
 
PPTX
The whitetiger novel review for collegeassignment.pptx
DhruvPatel754154
 
PDF
SUMMER INTERNSHIP REPORT[1] (AutoRecovered) (6) (1).pdf
pandeydiksha814
 
PDF
TIC ACTIVIDAD 1geeeeeeeeeeeeeeeeeeeeeeeeeeeeeer3.pdf
Thais Ruiz
 
PPTX
Data-Users-in-Database-Management-Systems (1).pptx
dharmik832021
 
PDF
Practical Measurement Systems Analysis (Gage R&R) for design
Rob Schubert
 
PPTX
IP_Journal_Articles_2025IP_Journal_Articles_2025
mishell212144
 
PDF
202501214233242351219 QASS Session 2.pdf
lauramejiamillan
 
PPTX
Introduction-to-Python-Programming-Language (1).pptx
dhyeysapariya
 
PPTX
Fuzzy_Membership_Functions_Presentation.pptx
pythoncrazy2024
 
PPTX
Probability systematic sampling methods.pptx
PrakashRajput19
 
PDF
Blitz Campinas - Dia 24 de maio - Piettro.pdf
fabigreek
 
PPTX
Blue and Dark Blue Modern Technology Presentation.pptx
ap177979
 
PDF
Blue Futuristic Cyber Security Presentation.pdf
tanvikhunt1003
 
Future_of_AI_Presentation for everyone.pptx
boranamanju07
 
Data-Driven Machine Learning for Rail Infrastructure Health Monitoring
Sione Palu
 
Real Life Application of Set theory, Relations and Functions
manavparmar205
 
short term internship project on Data visualization
JMJCollegeComputerde
 
INFO8116 - Week 10 - Slides.pptx data analutics
guddipatel10
 
Pipeline Automatic Leak Detection for Water Distribution Systems
Sione Palu
 
White Blue Simple Modern Enhancing Sales Strategy Presentation_20250724_21093...
RamNeymarjr
 
The whitetiger novel review for collegeassignment.pptx
DhruvPatel754154
 
SUMMER INTERNSHIP REPORT[1] (AutoRecovered) (6) (1).pdf
pandeydiksha814
 
TIC ACTIVIDAD 1geeeeeeeeeeeeeeeeeeeeeeeeeeeeeer3.pdf
Thais Ruiz
 
Data-Users-in-Database-Management-Systems (1).pptx
dharmik832021
 
Practical Measurement Systems Analysis (Gage R&R) for design
Rob Schubert
 
IP_Journal_Articles_2025IP_Journal_Articles_2025
mishell212144
 
202501214233242351219 QASS Session 2.pdf
lauramejiamillan
 
Introduction-to-Python-Programming-Language (1).pptx
dhyeysapariya
 
Fuzzy_Membership_Functions_Presentation.pptx
pythoncrazy2024
 
Probability systematic sampling methods.pptx
PrakashRajput19
 
Blitz Campinas - Dia 24 de maio - Piettro.pdf
fabigreek
 
Blue and Dark Blue Modern Technology Presentation.pptx
ap177979
 
Blue Futuristic Cyber Security Presentation.pdf
tanvikhunt1003
 
Ad

PaaSport to Paradise: Back to the Future with SSIS in Azure Data Factory

  • 2. Agenda  Secure Data Access from SSIS PaaS SSIS PaaS
  • 7. SSIS + ADF = SSIS in ADF
  • 8. SSIS Analogy • Running traditional ETL workloads on premises is like being on the ground floor
  • 9. SSIS + ADF Analogy • Running traditional ETL workloads on premises is like being on the ground floor • Running modern ELT workloads in the cloud on a fully managed Platform as a Service (PaaS) is like being on the 7th floor, so jumping 7 stories straight up is virtually impossible
  • 10. SSIS + ADF = SSIS in ADF Analogy • Running traditional ETL workloads on premises is like being on the ground floor • Running modern ELT workloads in the cloud on a fully managed Platform as a Service (PaaS) is like being on the 7th floor, so jumping 7 stories straight up is virtually impossible • Running SSIS in ADF is like taking an escalator that seamlessly carries you to the 7th floor step-by-step with the option to get on/off anytime at any floor.
  • 11. SQL Migration to an All PaaS Solution
  • 12. SQL Migration to an All PaaS Solution Cloud On-premises
  • 13. SQL Migration to an All PaaS Solution Cloud On-premises
  • 14. SQL Migration to an All PaaS Solution Cloud On-premises
  • 15. SQL Migration to an All PaaS Solution Cloud On-premises
  • 16. SQL Migration to an All PaaS Solution Cloud On-premises
  • 17. SQL Migration to an All PaaS Solution Cloud On-premises
  • 19. Linux Scale-Out SSIS On-Premises-to-Cloud Evolution On-premises data stores Cloud On-premises SQL Server/ File System MSDB SSISDB
  • 20. Project + Package Deployment Models Azure Files SSIS IR Azure-enabled SSDT SSMS dtutil dtexec DF UI Azure SQL MI Agent custom setups licensing BYOL AHB here SSIS On-Premises-to-Cloud Evolution On-premises data stores Azure SQL DB/Managed Instance (MI)/Azure Files Azure Data Factory (ADF) Cloud data stores Cloud On-premises SQL Server/ File System MSDB SSISDB MSDB SSISDB
  • 21. VNet proxy here Windows authentication AKV AAD first-class chain/group Azure Monitor Azure Data Catalog (ADC) GA SSIS On-Premises-to-Cloud Evolution On-premises data stores Azure SQL DB/Managed Instance (MI)/Azure Files VNet/ Proxy Azure Data Factory (ADF) Cloud data stores Cloud On-premises SQL Server/ File System MSDB SSISDB MSDB SSISDB
  • 22. Secure Data Access from SSIS PaaS
  • 23. Secure Data Access from SSIS PaaS
  • 24. Alternative Methods to Access On-Prem Data
  • 25. Joining SSIS IR to a VNet Connected to On-Prem Network
  • 26. Forced Tunneling for Outbound Traffic Inspection
  • 27. Configuring Self-Hosted IR as a Proxy for SSIS IR
  • 29. Enterprise Edition – Introduction
  • 30. Custom Setup – Introduction use strong cryptography/more secure network protocol, etc.
  • 31. 3rd Party Licensing – Introduction
  • 32. SSIS IR node Container ISV Setup1. Specify Product Key in setup script ISV Activation Server 2. Get Activation Key by submitting Cluster ID + Product Key Local Store (e.g. Registry) 3. Write Activation Key SSIS Executor ISV Extension 4. Read Activation Key and validate it with Cluster ID Setup Runtime 4. Get Cluster ID 4. Report on Node Count (Optional) SSIS Runtime 2. Activation Key 3rd Party Extensibility – Implementation
  • 33. 3rd Party Ecosystem – ISV Partners
  • 35. Batch Assess DBA Database Migration Assistant (DMA)/SSDT
  • 37. Batch Assess Batch Migrate Edit TestAssess DBA BI Developer SSDT Database Migration Assistant (DMA)/SSDT Azure Files/file system Database Migration Service (DMS)/Deployment Wizard/dtutil utility/manual copy SSISDB hosted by Azure SQL DB/MI, MSDB hosted by Azure SQL MI
  • 38. Source: SQL Server + SQL Agent Destination: Azure SQL MI + MI Agent Destination: Azure SQL DB + ADF Storage Types Package Assessment Package Migration Job Migration Package Migration Job Migration SSISDB • DMA • SSDT • SSISDB to SSISDB via DMS • SQL Agent to MI Agent via DMS (TBD) • SQL Agent to MI Agent via PowerShell/T-SQL/C# script • Recreate in MI Agent via SSMS • SSISDB to SSISDB via DMS/SSMS (TBD) • Redeploy to SSISDB via SSDT/SSMS • SQL Agent to ADF via SSMS (TBD) • Recreate in ADF via SSMS • Recreate in ADF
  • 39. Source: SQL Server + SQL Agent Destination: Azure SQL MI + MI Agent Destination: Azure SQL DB + ADF Storage Types Package Assessment Package Migration Job Migration Package Migration Job Migration SSISDB • DMA • SSDT • SSISDB to SSISDB via DMS • SQL Agent to MI Agent via DMS (TBD) • SQL Agent to MI Agent via PowerShell/T-SQL/C# script • Recreate in MI Agent via SSMS • SSISDB to SSISDB via DMS/SSMS (TBD) • Redeploy to SSISDB via SSDT/SSMS • SQL Agent to ADF via SSMS (TBD) • Recreate in ADF via SSMS • Recreate in ADF File System • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • SQL Agent to ADF via SSMS • Recreate in ADF via SSMS • Recreate in ADF
  • 40. Source: SQL Server + SQL Agent Destination: Azure SQL MI + MI Agent Destination: Azure SQL DB + ADF Storage Types Package Assessment Package Migration Job Migration Package Migration Job Migration SSISDB • DMA • SSDT • SSISDB to SSISDB via DMS • SQL Agent to MI Agent via DMS (TBD) • SQL Agent to MI Agent via PowerShell/T-SQL/C# script • Recreate in MI Agent via SSMS • SSISDB to SSISDB via DMS/SSMS (TBD) • Redeploy to SSISDB via SSDT/SSMS • SQL Agent to ADF via SSMS (TBD) • Recreate in ADF via SSMS • Recreate in ADF File System • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • SQL Agent to ADF via SSMS • Recreate in ADF via SSMS • Recreate in ADF MSDB • Same as above • MSDB to MSDB via DMS (TBD) • Redeploy to MSDB via dtutil • Reconfigure w/ SSIS IR package store • Same as above • Export to Azure Files/file system via SSMS/dtutil • Recreate in ADF via SSMS • Recreate in ADF
  • 41. Source: SQL Server + SQL Agent Destination: Azure SQL MI + MI Agent Destination: Azure SQL DB + ADF Storage Types Package Assessment Package Migration Job Migration Package Migration Job Migration SSISDB • DMA • SSDT • SSISDB to SSISDB via DMS • SQL Agent to MI Agent via DMS (TBD) • SQL Agent to MI Agent via PowerShell/T-SQL/C# script • Recreate in MI Agent via SSMS • SSISDB to SSISDB via DMS/SSMS (TBD) • Redeploy to SSISDB via SSDT/SSMS • SQL Agent to ADF via SSMS (TBD) • Recreate in ADF via SSMS • Recreate in ADF File System • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • SQL Agent to ADF via SSMS • Recreate in ADF via SSMS • Recreate in ADF MSDB • Same as above • MSDB to MSDB via DMS (TBD) • Redeploy to MSDB via dtutil • Reconfigure w/ SSIS IR package store • Same as above • Export to Azure Files/file system via SSMS/dtutil • Recreate in ADF via SSMS • Recreate in ADF SSIS Package Store • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Redeploy to MSDB via dtutil • Reconfigure w/ SSIS IR package store • Same as above • Redeploy to Azure Files via dtutil/manual copy • Keep in file system to access via VNet/Self-Hosted IR • Export to Azure Files/file system via SSMS/dtutil • Same as above
  • 50. Why Should You Care? When? How?
  • 51. Why Should You Care? Better TCO & ROI! Source: The Business Value of Microsoft Azure SQL Database Services, IDC, Mar 2015 The Total Economic Impact of Microsoft Azure SQL Database Managed Instance, Forrester Consulting, Sep 2018
  • 52. Why Should You Care? BYOL to Get AHB!  Maximize the value of on-premises SQL Server license investments by migrating SSIS in ADF/Azure-SSIS Integration Runtime (IR) Editions SQL Server with Software Assurance
  • 53. Why Should You Care? Huge Opportunity! Source: Worldwide Database Server Forecast, IDC, Dec 2016 Worldwide Database Server Forecast Modernizing opportunity with SQL Server 37% of all units run Microsoft, >50% of them run 2008/R2 or older versions Non-Relational Open Source PaaS & IaaS Microsoft Oracle IBM SAP New SQL Other MySQL No SQL
  • 54. Why Should You Care? Do The Math! Source: Feb 2019 snapshot of active SSIS instances extrapolated from SSIS 2016/2017 telemetry & CSS case volume for SQL 2008-2017 SSIS 2008-2017 on premises/Azure VMs/AWS VMs (100%) SSIS 2008-2017 on premises (88%) SSIS 2008/R2 EOS w/o SSISDB (13%) SSIS 2012-2017 w/ SSISDB (21%) Total Addressable Market Present Lift & Shift Target Immediate Target Past Target
  • 55. When Should You Care? NOW!
  • 56. How Should You Care? Fear is a Great Motivator!  Fear of missing out  Separation of storage/DB engine from compute/ETL engine  High Availability (HA) via multi-node cluster  Disaster Recovery (DR) via standby SSIS IR in a paired region  Dynamic scalability (in/out/up/down)
  • 57.  Fear of losing investments  Too many SSIS packages to convert from using OLEDB connection via Win Auth to using ADO.NET connection via AAD Auth w/ ADF managed identity  Too many SSIS packages to convert from Package Deployment Model w/o SSISDB to Project Deployment Model w/ SSISDB  Too many SSIS jobs scheduled by 3rd party orchestrators (e.g. Active Batch, Control-M, etc.) invoking dtexec  Too many packages using absolute paths w/ hard-coded drives (e.g. C:)  SQL MI Agent doesn’t support SSIS environment/32-bit runtime/property override  SQL MI Agent can’t invoke dtexec to execute SSIS packages stored in MSDB/file system/SSIS package stores  Network configuration/policy is too complex/strict to access on-premises data via VNet  Custom setups w/ script are too complex/error-prone How Should You Care? Fear is a Great Motivator!  Fear of missing out  Separation of storage/DB engine from compute/ETL engine  High Availability (HA) via multi-node cluster  Disaster Recovery (DR) via standby SSIS IR in a paired region  Dynamic scalability (in/out/up/down)
  • 58. How Should You Care? Fear is a Great Motivator!  Fear of losing investments  Too many SSIS packages to convert from using OLEDB connection via Win Auth to using ADO.NET connection via AAD Auth w/ ADF managed identity – Support OLEDB connection via AAD Auth w/ ADF managed identity, switch at run-time via property overrides  Too many SSIS packages to convert from Package Deployment Model w/o SSISDB to Project Deployment Model w/ SSISDB – Support file system/Azure Files/MSDB hosted by SQL MI to store packages, turn SSISDB from a blocker into an upsell​  Too many SSIS jobs scheduled by 3rd party orchestrators (e.g. Active Batch, Control-M, etc.) invoking dtexec – Azure-enable dtexec to invoke Execute SSIS Package activities in ADF pipelines  Too many packages using absolute paths w/ hard-coded drives (e.g. C:) – Replace absolute paths w/ UNC paths at run-time via property overrides​  SQL MI Agent doesn’t support SSIS environment/32-bit runtime/property override – Achieve feature parity w/ SQL Server Agent  SQL MI Agent can’t invoke dtexec to execute SSIS packages stored in MSDB/file system/SSIS package stores – ADF-enable SQL MI Agent to invoke executions of packages stored in file system/Azure Files/SSIS IR package stores  Network configuration/policy is too complex/strict to access on-premises data via VNet – Support Self-Hosted IR as proxy for Azure-SSIS IR to access on-premises data  Custom setups w/ script are too complex/error-prone – Support custom setups w/o script  Fear of missing out  Separation of storage/DB engine from compute/ETL engine  High Availability (HA) via multi-node cluster  Disaster Recovery (DR) via standby SSIS IR in a paired region  Dynamic scalability (in/out/up/down)
  • 59. How Should You Care? Fear is a Great Motivator!  Minimizing fear of losing investments – No more lift & shift blockers Rebalancing your risks/fears  Maximizing fear of missing out  Separation of storage/DB engine from compute/ETL engine  High Availability (HA) via multi-node cluster  Disaster Recovery (DR) via standby SSIS IR in a paired region  Dynamic scalability (in/out/up/down)  ADO.NET/OLEDB/Storage connection via AAD Auth w/ ADF managed identity  AKV support on Execute SSIS Package activity  SSIS ADF connectors (80+)  PQ Source  DQS migration to cloud alternatives running on SSIS in ADF  More cloud-first/cloud-only features, etc.
  • 62. https://www.slideshare.net/SandyWinarko/paasport-to-paradise-back-to-the-future- with-ssis-in-azure-data-factory Documentations Provisioning Azure-SSIS IR Using UI: https://docs.microsoft.com/azure/data-factory/tutorial-deploy-ssis-packages-azure Using PSH: https://docs.microsoft.com/azure/data-factory/tutorial-deploy-ssis-packages-azure-powershell Using ARM template: https://docs.microsoft.com/azure/data-factory/create-azure-ssis-integration-runtime#azure- resource-manager-template Provisioning Azure-SSIS IR using Azure SQL DB/Managed Instance w/ VNet service endpoint/private endpoint to host SSISDB Using UI/PSH: https://docs.microsoft.com/azure/data-factory/create-azure-ssis-integration-runtime Joining a VNet: https://docs.microsoft.com/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network Provisioning Azure-SSIS IR using AAD Auth w/ ADF managed identity Using UI/PSH: https://docs.microsoft.com/azure/data-factory/create-azure-ssis-integration-runtime Enabling AAD Auth: https://docs.microsoft.com/azure/data-factory/enable-aad-authentication-azure-ssis-ir Provisioning Azure-SSIS IR to access data on premises Using UI/PSH: https://docs.microsoft.com/azure/data-factory/create-azure-ssis-integration-runtime Joining a VNet: https://docs.microsoft.com/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network Using Self-Hosted IR as a proxy: https://docs.microsoft.com/azure/data-factory/self-hosted-integration-runtime-proxy- ssis Monitoring Azure-SSIS IR Using UI/PSH: https://docs.microsoft.com/azure/data-factory/monitor-integration-runtime#azure-ssis-integration- runtime Using Azure Monitor: https://docs.microsoft.com/azure/data-factory/monitor-using-azure-monitor#monitor-ssis- operations-with-azure-monitor
  • 63. Documentations Configuring Azure-SSIS IR Using UI/PSH: https://docs.microsoft.com/azure/data-factory/manage-azure-ssis-integration-runtime Configuring Azure-SSIS IR to use Enterprise Edition Using UI/PSH: https://docs.microsoft.com/azure/data-factory/how-to-configure-azure-ssis-ir-enterprise-edition Configuring Azure-SSIS IR to use custom setups Using UI/PSH: https://docs.microsoft.com/azure/data-factory/how-to-configure-azure-ssis-ir-custom-setup Configuring Azure-SSIS IR for high performance Using UI/PSH: https://docs.microsoft.com/azure/data-factory/configure-azure-ssis-integration-runtime- performance Configuring Azure-SSIS IR for disaster recovery Using UI/PSH: https://docs.microsoft.com/azure/data-factory/configure-bcdr-azure-ssis-integration-runtime Deploying projects/packages to run on Azure-SSIS IR Connecting SSMS to SSISDB: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-connect-to- catalog-database?view=sql-server-2017 Using UI/PSH: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-deploy-run-monitor- tutorial?view=sql-server-2017 Using dtutil: https://docs.microsoft.com/azure/data-factory/azure-ssis-integration-runtime-package- store#deploying-multiple-packages-with-dtutil
  • 64. Documentations Connecting to data stores using Win Auth on Azure- SSIS IR Using 4 methods: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-connect-with-windows- auth?view=sql-server-2017 Connecting to data stores using AAD Auth w/ ADF managed identity on Azure-SSIS IR Using Azure Storage connections: https://docs.microsoft.com/sql/integration-services/connection- manager/azure-storage-connection-manager#managed-identities-for-azure-resources-authentication Using ADO.NET connections: https://docs.microsoft.com/sql/integration-services/connection-manager/ado-net- connection-manager#managed-identities-for-azure-resources-authentication Using OLEDB connections: https://docs.microsoft.com/sql/integration-services/connection-manager/ole-db- connection-manager#managed-identities-for-azure-resources-authentication Connecting to file shares and processing files on Azure- SSIS IR Using local file system/file shares: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-files- file-shares?view=sql-server-2017 Using Win Auth: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-connect-with-windows- auth?view=sql-server-2017 Running packages on Azure-SSIS IR From SSDT: https://docs.microsoft.com/azure/data-factory/how-to-invoke-ssis-package-ssdt Using SSMS/stored procedures/scripts/code: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis- azure-run-packages?view=sql-server-2017 Using AzureDTExec: https://docs.microsoft.com/azure/data-factory/how-to-invoke-ssis-package-azure-enabled- dtexec As Execute SSIS Package activities in ADF pipelines: https://docs.microsoft.com/azure/data-factory/how-to- invoke-ssis-package-ssis-activity As Stored Procedure activities in ADF pipelines: https://docs.microsoft.com/azure/data-factory/how-to-invoke- ssis-package-stored-procedure-activity
  • 65. Documentations Scheduling package executions on Azure-SSIS IR Using SSMS/ADF/other methods: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure- schedule-packages?view=sql-server-2017 Using SSMS: https://docs.microsoft.com/sql/integration-services/lift-shift/ssis-azure-schedule-packages- ssms?view=sql-server-2017 As Execute SSIS Package activities in ADF pipelines: https://docs.microsoft.com/azure/data-factory/how-to- invoke-ssis-package-ssis-activity Using Azure SQL MI Agent: https://docs.microsoft.com/azure/data-factory/how-to-invoke-ssis-package- managed-instance-agent Scheduling Azure-SSIS IR provisioning on demand/just in time Using Web activities: https://docs.microsoft.com/azure/data-factory/how-to-schedule-azure-ssis-integration- runtime Cleaning up package execution logs in SSISDB Using Elastic Jobs: https://docs.microsoft.com/azure/data-factory/how-to-clean-up-ssisdb-logs-with-elastic-jobs Assessing SSIS packages for migration Using DMA: https://docs.microsoft.com/sql/dma/dma-assess-ssis?view=sql-server-ver15 Using SSDT: https://docs.microsoft.com/azure/data-factory/how-to-invoke-ssis-package-ssdt#assess-ssis- projectpackages-for-executions-in-azure Migrating SSIS packages SSIS Migration Playbook: https://docs.microsoft.com/azure/data-factory/scenario-ssis-migration-overview Using DMS: https://docs.microsoft.com/azure/dms/how-to-migrate-ssis-packages-managed-instance Practicing SSIS CI/CD Using Azure DevOps: https://docs.microsoft.com/sql/integration-services/devops/ssis-devops-overview?view=sql- server-ver15 Developing premium/licensed components for Azure- SSIS IR Using Cluster ID: https://docs.microsoft.com/azure/data-factory/how-to-develop-azure-ssis-ir-licensed- components Procuring 3rd party components for Azure-SSIS IR Partner list: https://techcommunity.microsoft.com/t5/SQL-Server-Integration-Services/Enterprise-Edition-Custom- Setup-and-3rd-Party-Extensibility-for/ba-p/388360

Editor's Notes

  • #5: Repeat the mantra: Rehost – Enhance – Extend.
  • #6: Repeat the mantra: Rehost – Enhance – Extend.
  • #7: Repeat the mantra: Rehost – Enhance – Extend.
  • #9: Continuous and increased product investments for different customer cohorts.
  • #10: Continuous and increased product investments for different customer cohorts.
  • #11: Continuous and increased product investments for different customer cohorts.
  • #13: Typical SQL Server estate including primary Value-Added Services (VAS), excluding secondary ones (MDS/DQS).
  • #14: Full compatibility rehost for DB + ETL engines.
  • #15: Full compatibility rehost vs. partial rehost/complete replacement.
  • #16: SSIS in ADF as glue between disparate resources on premises and in the cloud. Pop quiz: What is AWS ETL service called?
  • #17: ADF complements/fills gaps in SSIS and vice versa.
  • #18: We have cloud alternatives of DQS built as extensions running on SSIS in ADF.
  • #20: 2017 is our break-out year.
  • #21: Preserved skillsets by ensuring that traditional on-premises tools can still be used in the cloud.
  • #22: Transformed ADF into a first-class job scheduler for Azure SQL DB.
  • #52: Lift & shift motivation/migration drive from the economic/financial angle for enterprises.
  • #53: Lift & shift motivation/migration drive from the economic/financial angle for enterprises. Pop quiz: If you have on-premises SQL Server license for 100 cores and you’ve traded in 50 cores for Azure SQL DB/MI, how many cores can you use for SQL on IaaS/SSIS in ADF?
  • #54: Lift & shift motivation/migration drive from the business opportunity angle for System Integrators (SIs) and DB/BI consultancies.
  • #55: Lift & shift motivation/migration drive from the business opportunity angle for System Integrators (SIs) and DB/BI consultancies.
  • #56: Lift & shift motivation/migration drive from the timing angle for all. Pop quiz: What kind of insect is in my photo?
  • #57: Lift & shift motivation/migration drive from the risk/fear angle (pros & cons) for enterprises. Pros = fear of missing out.
  • #58: Lift & shift motivation/migration drive from the risk/fear angle (pros & cons) for enterprises. In the past, pros = fear of missing out < cons = fear of losing investments.
  • #59: We’ve addressed/are addressing each fear of losing investments.
  • #60: Now, pros = fear of missing out > cons = fear of losing investments.