Introduction to Microsoft SQL Server 2008 R2 Integration Services

Microsoft SQL Server 2008 R2

Introduction to Integration Services

http://techmaster.vn

SQL Server 2008 R2 BI Technologies


Contents
• Understanding the Data Integration
• Understanding the SQL Server 2008 R2
Integration Services
• Understanding the SSIS Packages
• Understanding the SSIS Control Flow
• Understanding the SSIS Data Flow

SQL Server 2008 R2 BI Structure
Reporting and Visualization Tools (Dashboard, KPI,
Presentation Layer
Scorecard,…)

Turn data into information (analysis)
Analytical Layer
Multidimensional OLAP Database

Data Storage and Retrieval Layer Data Warehouse in RDBMS

1. Extract the data from the multiple sources
Data Transformation Layer 2. Modify the data to consistent
3. Load the data into Data Storage system

Data Source Layer Text, MS Excel, MS Access, MS SQL, Oracle,…| External Sources


Microsoft Business Intelligence Platform

Analytic Scorecards, Analytics, Planning
Applications (PerformancePoint Service)

Portal
(SharePoint)
Data Delivery Report Builder End-user Analysis
SSRS (Excel)

Integrate Analyze Report
(SQL Integration Services) (SQL Analysis Services) (SQL Reporting Services)
Infrastructure
Platform Data Warehouse, Data Marts,
Operational Data
(SQL Server 2008 R2)

Office SQL

Data Integration in Real World

Extract data Transform the Load data into
from sources data data stores


Data Integration Challenges
• Multiple sources with different formats.
• Structured, semi-structured, and unstructured
data.
• Huge data volumes.
Enterprises spend 60%–80% of their resources developing
and testing their ETL processes


SQL SERVER 2008 R2 INTEGRATION SERVICES


Introducing Integration Services 2008 R2
• Primarily designed to implement ETL
processes
• Provides a robust, flexible, fast, scalable and
extensible architecture
• Challenges traditional ETL design approaches


Introducing Integration Services 2008 R2
• Its capabilities are useful in many other
scenarios:
– Assessing data quality
– Cleansing and standardizing data
– Merging data from heterogeneous data stores
– Implementing ad hoc data transfers
– Automating administrative tasks

SSIS Architecture
• SQL Server Integration
Services (SSIS) service
• SSIS object model
• Two distinct runtime
engines:
– Control flow
– Data flow

SSIS Architecture
• SSIS Designer
– Graphical tool to create and maintain
Integration Services packages.
• Integration Services Runtime
– Saves the layout of packages, runs
packages, and provides support for
logging, breakpoints, configuration,
connections, and transactions.
• Tasks and other executable:
– The Integration Services run-time
executables are the package,
containers, tasks, and event handlers

SSIS Architecture

• Data Flow engine (pipeline)
– In-memory buffers
• Data Flow components
– Sources,
– Transformations
– Destinations


SSIS Architecture
• Object Model
– Allow for creating custom
components for use in packages
• Integration Services Service
– Lets you monitor running
Integration Services packages and
to manage the storage of
packages.



PACKAGE ESSENTIALS


What’s IS Package
• A package is the object that implements Integration
Services functionality to extract, transform, and load
data.
• Creation tools:
– SSIS Designer in BI Development Studio.
– SQL Server Import and Export Wizard
– Integration Services Connections Project Wizard
• Saved in XML format to the file system or SQL Server

Package Elements
• Connection managers
• Control flow components
• Data flow components
• Variables
• Event handlers
• Configurations

Connection Managers
• Logical representation of
a connection
• Stored in the package
and cannot be shared
between packages
• Used by package
elements
• Do not need to connect
to SQL Server


CONTROL FLOW


Control Flow
• Control flow is the process-oriented
workflow engine
• A package consists of a single
control flow
• Control flow elements:
– Containers
– Tasks

– Precedence constraints

– Variables
– Event handlers

Containers
• Provide structure and services for
– Grouping tasks

– Implementing repeating flows

• Execute in sequence defined by precedence constraints in the control flow

• Manage variable and transactional boundaries


Tasks
• Perform discrete operations at runtime
• Execute in sequence defined by precedence
constraints in the control flow
• Use properties configured at design time or
assigned dynamically at runtime by using
expressions


Task Categories
Task Descriptions
Data Flow The Data Flow task defines and runs data flows that extract data, apply transformations,
and load data
Data Preparation Data preparation tasks copy files and directories, download files and data, save data
returned by Web methods, or work with XML documents
Workflow Workflow tasks communicate with other processes to run packages or programs, send
and receive messages between packages, send e-mail messages, read Windows
Management Instrumentation (WMI) data, or watch for WMI events.
SQL Server SQL Server tasks access, copy, insert, delete, or modify SQL Server objects and data
Analysis Services Analysis Services tasks create, modify, delete, or process Analysis Services objects
Scripting Scripting tasks extend package functionality through custom scripts
Maintenance Maintenance tasks perform administrative functions, such as backing up and shrinking
SQL Server databases, rebuilding and reorganizing indexes, and running SQL Server Agent
jobs


Precedence Constraints
• Precedence constraints link executables, containers,
and tasks in packages into a control flow, and specify
conditions that determine whether executables run
• Configure conditions that determine whether the
executable runs:
– Success, Failure, or Completion constraints
– Expressions
– Logical AND/OR for
multiple constraints

Variables
• Variables customize package
behavior by changing expression
values or object properties
• System variables store values
collected during package
execution
• All variables use case-sensitive
names
• Variables can be scoped at
package, container, or task level

Event Handlers
• At run time executables raise events
• Event handlers can be defined to respond to
these events
• Creating an event handler is similar to
building a package; an event handler has
tasks and containers, which are sequenced
into the control flow

Event Handlers
• Common events used to trigger event handlers:
– OnPreExecute
– OnPostExecute
– OnError
• Examples:
– Retrieve system information to assess resource
availability before the package runs
– Send an e-mail message when an error occurs


DATA FLOW


Data Flow
• Data Flow is optional elements
– Extract data
– Modify data
– Load data into data sources.
• The main data flow elements are
– Sources
– Transformations
– Destinations.

Data Flow Sources
• Sources extract data from:
– Relational tables and views
– Files
– Analysis Services databases


Data Flow Transformations
• Aggregate, merge, distribute, or modify data
• Include error outputs in some cases
• Transformation categories
– Row
– Rowset
– Split and Join
– Script
– Other

Row Transformations
Transformation Description
Character Map The transformation that applies string functions to character data.
The transformation that adds copies of input columns to the
Copy Column transformation output.
The transformation that converts the data type of a column to a
Data Conversion different data type.
The transformation that populates columns with the results of
Derived Column expressions.
Export Column The transformation that inserts data from a data flow into a file.
The transformation that reads data from a file and adds it to a data
Import Column flow.
The transformation that uses script to extract, transform, or load
Script Component data.
The transformation that runs SQL commands for each row in a data
OLE DB Command flow. http://techmaster.vn

Rowset Transformations
The transformation that performs aggregations such as
Aggregate
AVERAGE, SUM, and COUNT.
Sort The transformation that sorts data.
Percentage The transformation that creates a sample data set using a
Sampling percentage to specify the sample size.
The transformation that creates a sample data set by
Row Sampling
specifying the number of rows in the sample.
The transformation that creates a less normalized version
Pivot
of a normalized table.
The transformation that creates a more normalized
Unpivot
version of a nonnormalized table.

Split and Join Transformations
Conditional Split The transformation that routes data rows to different outputs.
The transformation that distributes data sets to multiple
Multicast
outputs.
Union All The transformation that merges multiple data sets.
Merge The transformation that merges two sorted data sets.
The transformation that joins two data sets using a FULL, LEFT,
Merge Join
or INNER join.
The transformation that looks up values in a reference table
Lookup
using an exact match.
The transformation that writes data from a connected data
source in the data flow to a Cache connection manager that
Cache
saves the data to a cache file. The Lookup transformation
performs lookups on the data in the cache file. http://techmaster.vn

The Script Transformation
• Extends the capabilities of the data flow
• Similar to the Script Task, develop VB.NET or C# .NET
scripts to introduce custom logic into the data flow
• Can be configured for these roles:
– Source
– Destination
– Transformation
• Delivers optimized performance because it is precompiled


Other Transformations
• Add audit information
• Populate lookup caches
• Export and import data
• Count rows
• Manage slowly changing dimensions



DEMO


THANK YOU


Introduction to Microsoft SQL Server 2008 R2 Integration Services

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Introduction to Microsoft SQL Server 2008 R2 Integration Services (20)

More from Quang Nguyễn Bá (16)

Recently uploaded (20)

Introduction to Microsoft SQL Server 2008 R2 Integration Services

Editor's Notes