5\9 SSIS 2008R2_Training - DataFlow Basics

5.5. Data Flow BasicsData Flow Basics
Presented by:
Pramod Singla
Pramod_singla@yahoo.co.in
.

ContentContent
 Recap and Q&A
 Data Flow Task
 Pipeline Architecture
 Data Sources
◦ Demo: ADO.NET Source
◦ Demo: Excel Source
◦ Demo: Flat File Source
◦ Demo: OLE DB Source
◦ Demo: XML Source
◦ Demo: Raw File Destination
◦ Demo: Raw File Source
 Data Destinations
◦ Demo: OLE DB Destination
◦ Demo: DataReader Destination
◦ Demo: Excel Destination
◦ Demo: Flat File Destination
◦ Demo: SQL Server Destination
 Analysis Services Destinations
◦ Demo: Dimension Processing
◦ Demo: Partition Processing
◦ Demo: Data Mining Model Training
 Summary

Recap and Q&ARecap and Q&A
Variables
◦ Demo: Introducing the Variables Window
◦ Demo: Working with Variables
◦ Demo :Using variables in Execute SQL task
Expressions
◦ Demo: Creating Time stamped log file
◦ Demo: Operators and More Variables
Expression Usage
◦ Demo: Using a Variable in an Expression
◦ Demo: Using an Expression with a Connection Manager
◦ Demo: Using Expressions with Precedence Constraints

Data Flow TaskData Flow Task
 Encapsulates the data flow
engine that Extract, transform, and
load data
 A Data Flow task can include multiple
data flows
 When order is important, use
multiple Data Flow tasks, each task
containing one data flow.
 Diagram includes the inputs,
outputs, ,error outputs external
columns.

Pipeline ArchitecturePipeline Architecture
 The run-time engine implements
the control flow and package
management infrastructure
 The data flow engine
is a specialized, high performance
engine that is exclusively
dedicated to extracting,
transforming, and loading data.
 The data flow engine provides the
in-memory buffers
 Arrow in dataflow are called
Service path

ADO.NET SourceADO.NET Source
 The ADO NET source consumes data from a .NET provider and makes the
data available to the data flows
 SSIS 2012 onwards, you can use the ADO NET source to connect to
Microsoft Windows Azure SQL Database
 The ADO.net source provides two different data access modes for
extracting data:
◦ A table or view.
◦ The results of an SQL statement.

Excel SourceExcel Source
 The Excel source extracts data
from worksheets or ranges in
Microsoft Excel workbooks
 The Excel source provides four
different data access modes
for extracting data:
◦ A table or view specified in a
variable.
◦ The results of an SQL statement
stored in a variable.

Flat File SourceFlat File Source
 The Flat File source reads data from a
text file. The text file can be in
delimited, fixed width, or mixed format.
 Delimited format uses column and row
delimiters to define columns and rows.
 Fixed width format uses width to define
columns and rows. This format also
includes a character for padding fields to
their maximum width.
 Ragged right format uses width to define
all columns, except for the last column,
which is delimited by the row delimiter.

OLE DB SourceOLE DB Source
 The OLE DB source extracts
data from a variety of OLE DB-
compliant relational databases
 The OLE DB source provides
four different data access
modes for extracting data:
◦ A table or view specified in a
variable.
◦ The results of an SQL statement. The
query can be a parameterized query.
◦ The results of an SQL statement
stored in a variable.

XML SourceXML Source
 The XML Source does not
require a Connection Manager,
but instead references a XML
data source either as a
◦ direct reference to a file
◦ a variable containing a file reference
or
◦ a variable containing XML data.

Raw File DestinationRaw File Destination
 The Raw File destination writes
raw data to a file. Because the
format of the data is native to
the destination
 write data more quickly than
other destinations
 You use the Raw File source
to retrieve raw data that was
previously written by the
destination
 This source does not use a
connection manager.

Raw File SourceRaw File Source
 The Raw File source reads raw
data from a file
 Reads data more quickly than
other sources
 Retrieve raw data that was
previously written by the Raw
File destination
 This source does not use a
connection manager.

OLEDB DestinationOLEDB Destination
 The OLE DB destination loads data into a variety of OLE DB-compliant databases
 The OLE DB destination provides five different data access modes for loading
data:
◦ A table or view. You can specify an existing table or view, or you create a new table.
◦ A table or view using fast-load options. You can specify an existing table or create a new table.
◦ A table or view specified in a variable.
◦ A table or view specified in a variable using fast-load options.
 Does not support parameters. To execute a parameterized INSERT statement,
use OLE DB Command transformation
 If OLE DB destination uses fast-load data access mode, specify following
options:
◦ Keep identity values from the imported data file or use unique values assigned by SQL Server.
◦ Retain a null value during the bulk load operation.
◦ Check constraints on the target table or view during the bulk import operation.
◦ Acquire a table-level lock for the duration of the bulk load operation.
◦ Specify the number of rows in the batch and the commit size.

DataReader DestinationDataReader Destination
 Exposes the data in a data flow by using the ADO.NET DataReader interface
 The data can then be consumed by other application

Excel DestinationExcel Destination
 The Excel destination loads data into
worksheets or ranges in Microsoft
Excel workbooks
 Can create new sheet based on input
 In Excel, a worksheet or range is the
equivalent of a table or view.
 The lists of available tables in the
Excel Source and Destination editors
display only existing worksheets

Flat File DestinationFlat File Destination
 The Flat File destination writes data to a text file
 This destination uses a Flat File connection manager to
access the text file
 Specify whether to overwrite a data in a destination file
that has the same name.

SQL Server DestinationSQL Server Destination
 Connects to a local SQL Server database and bulk loads data
 Cannot access a SQL Server database on a remote server
 Users who execute packages that include the SQL Server destination require
the "Create global objects" permission.
 Assigning "Create Global Objects" user right to that account:
 Click Start, point to Administrative Tools, and then click Local Security Policy.
 Expand Local Policies, and then click User Rights Assignment.
 In the right pane, double-click Create global objects
 In the Local Security Policy Setting dialog box, click Add.
 In the Select Users or Group dialog box, click the user accounts that you want to add,
click Add, and then click OK.
 Click OK.
 If you are running the package from either SQL Server Management
Studio (SSMS) or Business Intelligence Development Studio (BIDS)
or DTExecUI.exe, launch those tools under the elevated Administrator
account

Dimension ProcessingDimension Processing
 The Dimension Processing
destination loads and processes an
SQL Server Analysis Services
dimension
 The Dimension Processing
destination includes the following
features:
◦ Options to perform incremental, full, or update
processing.
◦ Error configuration, to specify whether
dimension processing ignores errors or stops
after a specified number of errors.
◦ Mapping of input columns to columns in
dimension tables.

Partition ProcessingPartition Processing
 The Partition Processing destination loads and processes an SQL Server Analysis
Services partition.
 The Partition Processing destination includes the following features:
◦ Options to perform incremental, full, or update processing.
◦ Error configuration, to specify whether processing ignores errors or stops after a specified number of errors.
◦ Mapping of input columns to partition columns.

SummarySummary
 Data Flow Task
 Pipeline Architecture
 Data Sources
◦ Demo: ADO.NET Source
◦ Demo: Excel Source
◦ Demo: Flat File Source
◦ Demo: OLE DB Source
◦ Demo: XML Source
◦ Demo: Raw File Destination
◦ Demo: Raw File Source
 Data Destinations
◦ Demo: OLE DB Destination
◦ Demo: DataReader Destination
◦ Demo: Excel Destination
◦ Demo: Flat File Destination
◦ Demo: SQL Server Destination
 Analysis Services Destinations
◦ Demo: Dimension Processing
◦ Demo: Partition Processing

DEMO
5_DataFlowBasics.dtsx 5_DataFlowBasics_OnlinDemo.dtsx

5\9 SSIS 2008R2_Training - DataFlow Basics

Resources & QuestionsResources & Questions
Contact me :
◦ Pramod_singla@yahoo.co.in
◦ http://pramodsingla.wordpress.com/
Microsoft Resources:
◦ http://pluralsight.com/training/courses/TableOfContents?courseName=ssis-basic&highlight=stacia-misn
◦ http://technet.microsoft.com/en-
us/library/microsoft.sqlserver.dts.pipeline.wrapper(v=sql.105).aspx
◦ http://technet.microsoft.com/en-us/library/bb522498(v=sql.105).aspx
◦ http://www.mssqltips.com/sqlservertip/2557/ssis-architecture-and-internals-interview-
questions/
◦ http://msdn.microsoft.com/en-us/library/ms141122.aspx
◦ http://www.bimonkey.com/support/ssis-component-samples-list/
◦ http://technet.microsoft.com/en-us/library/ms137622.aspx
◦ http://support.microsoft.com/kb/2009672
◦ https://www.simple-talk.com/sql/reporting-services/using-sql-server-integration-services-to-
bulk-load-data/

5\9 SSIS 2008R2_Training - DataFlow Basics

Recommended

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to 5\9 SSIS 2008R2_Training - DataFlow Basics (20)

Recently uploaded (20)

5\9 SSIS 2008R2_Training - DataFlow Basics