Part 3,4,5,6 & 7: Informatica: Informatica Overview and Transformations
Part 3,4,5,6 & 7: Informatica: Informatica Overview and Transformations
Informatica
Overview and
Transformations
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Introduction:
• Is GUI based ETL product from Informatica corporation.
• Is a client server technology.
• Is developed using JAVA language.
• Is an integrated tool set (To Design, To Run, To Monitor)
Versions:
• 5.0
• 6.0
• 7.1.1
• 8.1.1
• 8.5
• 8.6
• 9.0
• 9.1
• 9.5
• 9.6
• 10
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Informatica Architecture
Client Components
1. Administrator Console
2. Repository Manager
3. Mapping Designer
4. Work flow Manager
5. Work flow Monitor
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Mapping Designer Workflow Manager Workflow Monitor
(M_xyz)
Save
| Start
Repository
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Domain:
A domain is the primary unit for management and administration of services in PowerCenter.
A domain contain one or more nodes.
Node:
A node is the logical representation of a machine in a domain.
Two kinds of nodes.
1. Gateway node - can run management services for the domain, and they are also the ones
that communicate with the domain database. The worker node can run application services,
but cannot communicate with the domain database.
Only one gateway node can run the management services at a time, and only one node
can talk to the domain database at a time, regardless of how many nodes are in the domain.
The node that performs these tasks is the master gateway node. While there is no upper limit on
the number of nodes in a domain, each domain has a minimum of one gateway node
2. Worker node - A worker node runs a Service Manager process, and it can run application services.
The worker node cannot run the extra management processes, nor does it communicate with the
domain database. This can be good, because it does not require the extra resources for
management, but it cannot take over as a master gateway node.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Integration Services
Responsible for the movement of data from sources to targets
The Integration Service reads mappings and session information from the repository.
It extract the data from the mapping source stores in the memory (Staging Area) where it
applies the transformation rule that you can configure in the mapping.
The Integration Service loads the transformed data into the mapping targets.
The integration service connects to the repository through repository service to fetch the
metadata.
Client components:
Mapping Designer:
It is a GUI based client component which allows you to design the plan of ETL process called mapping.
The following types of metadata objects can be created using designer client.
• Create Source Definition
• Create Target Definition
• Design Mapping with or without a Transformation rule.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Workflow Manager:
It is a GUI based client component which allows you to create the following task.
• Create session for each mapping
• Create workflow
• Execute workflow
• Schedule workflow
Workflow Monitor:
It is a GUI based client component which provides the following information:
• Give the workflow and session status (Succeeded or Failed)
• Get Session Log from the repository.
• Start, Stop sessions and workflows.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Repository Manager:
The Repository manager is GUI based administrative client which allows you to create following objects.
• Create, Edit and Delete folders which are required to organize the metadata and the repository.
• Create used, user groups, assign permissions and privileges.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Session: A Session is a set of instruction which perform extraction, transformation and loading. A session Created to make the
mapping available for execution.
Workflow: A Workflow is a start task which contains a set of instruction to execute the other task such as session. Workflow is a
top object in the power center development hierarchy.
Schedule Workflow: A Schedule workflow is an administrative task which specifies the data and time to run the workflow.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Transformation:
A transformation is an object used to define business logic for processing the data.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Active Transformation:
A transformation which can affect the number of rows while data is going from source to target is
known as active transformation.
The following are the list of active transformation used for processing the data.
• Source Qualifier Transformation
• Filter Transformation
• Aggregator Transformation
• Joiner Transformation
• Router Transformation
• Rank Transformation
• Sorter Transformation
• Update Strategy Transformation
• Transaction Control Transformation
• Union Transformation
• Normalizer Transformation
• XML Source Qualifier
• Java Transformation
• SQL Transformation
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Passive Transformation:
A transformation which does not affect the number of rows when the data is moving from source to
target is known as passive transformation.
The following are the list of passive transformation used for processing the data.
• Expression Transformation
• Sequence Generator Transformation
• Stored Procedure Transformation
• Lookup Transformation
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Filter Transformation:
This is a type of an active transformation which allows you to filter the data based on given condition.
A condition is created with the three elements
• Port
• Operator
• Operand
The integration service evaluates the filter condition against each input record, returns TRUE or FALSE.
The integration service returns TRUE when the records is satisfied with the condition and the records are
given for further processing or loading the data into the target.
The integration service returns FALSE when the input record is not satisfied with the condition and those
records are rejected from filter transformation.
Router transformation is a type of active transformation which allows to apply multiple condition, to
load multiple target table.
This is a type of passive transformation which allows you to calculate the expression for each record.
The expression can be calculated only in the output ports.
Used expression transformation to perform data cleansing and data scrubbing activities.
Expression transformations define only on the output port.
Ex: Price/Quantity
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Aggregator Transformation:
This is a type of an Active transformation which allows you to calculate the summary for a group of records.
This is of type an Active Transformation which sorts the data in ascending or in descending order.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Sequence Generator Transformation:
Sequence generator transformation is a passive and connected transformation. The sequence generator
transformation is used for
Ex: 1, 2, 3,…
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Rank Transformation:
This is of type an active transformation which allows you to identify the TOP and BOTTOM
performers.
The rank transformation can be created with following types of ports.
1. Input Port
2. Output Port
3. Rank Port (R)
4. Variable Prot (V)
Rank Port: - The port based on which rank is determined is known as Rank Port.
Variable Port: - A port which can store the data temporally is known as a variable port.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Joiner Transformation:
This is of type of an Active transformation which allows you to combine the data from multiple
sources into a single output based on given join condition.
The joiner transformation is created with the following types of ports.
1. Input Port
2. Output Port
3. Master Port (M)
A Source which is defined with lesser number of records than other source is designated as master source.
A master source is created with the master ports. The joiner transformation can be created with following
types of join.
1. Normal join (Equi Join)
2. Master outer join
3. Detail outer join
4. Full outer join.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
The default type of joiner transformation is Normal join (Equi Join).
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Normal join Master outer join Detail outer join
M M M
D D D
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Merging
Horizontally Vertically
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Source Qualifier Transformation:
• Joins
• Filter rows
• Sorting Input
• Distinct rows
• Custom SQL Query
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Lookup Transformation:
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Connected look up – Receives source data, performs a look up and returns data to the pipeline.
Unconnected lookup – Received source data from :LKP expression, performs a lookup and returns one
column data at a time to the calling transformation.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Update Stratergy Transformation:
In the informatica, you can set the update strategy at two different levels:
• Session Level: Configuring at session level instructs the integration service to either treat all rows
in the same way (Insert or update or delete) or use instructions coded in the session mapping to
flag for different database operations.
• Mapping Level: Use update strategy transformation to flag rows for inert, update, delete or reject.
An important note, Update strategy works only when we have a primary key on the target table.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
SQL Transformation:
SQL Transformation is a connected transformation used to process SQL queries in the midstream of a
pipeline. We can insert, update, delete and retrieve rows from the database at run time using the SQL
transformation.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Transaction control Transformation:
Transaction Control is an active and connected transformation. The transaction control transformation is used
to control the commit and rollback of transactions. You can define a transaction based on varying number of input
rows.
Use the following built-in variables in the expression editor of the transaction control transformation,
• TC_CONTINUE_TRANSACTION – IS does not perform any change in the transaction for this row.
• TC_COMMIT_BEFORE – IS Commits the transaction, begins a new transaction, and writes the current
row to the target. The current row is in the new transaction.
• TC_COMMIT_AFTER – IS writes the current row to the target, commits the transaction, and begins a
new transaction. The current row is in the committed transaction.
• TC_ROLLBACK_BEFORE – IS rolls back the current transaction, begins a new transaction, and writes
the current row to the target. The current row is in the new transaction.
• TC_ROLLBACK_AFTER – IS writes the current row to the target, rolls back the transaction, and begins
a new transaction. The current row is in the rolled back transaction.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Stored Procedure Transformation:
Stored Procedure Transformation is a passive transformation. Stored procedure transformation can be used in
both connected and unconnected mode. Stored procedures are stored and run within the database. Stored procedures
contain a pre-compiled collection of PL-SQL statements.
The stored procedures in the database are executed using the Execute or Call statements. Informatica provides the
stored procedure transformation which is used to run the stored procedures in the database. It contains connected and
unconnected transformation.
The property, "Stored Procedure Type" is used to specify when the stored procedure runs. The different values of this
property are shown below:
• Normal.
• Pre-load of the source.
• Post-load of the source.
• Pre-load of the target.
• Post-load of the target.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Normalizer Transformation:
Normalizer transformation type is Active & Connected. The Normalizer transformation is used in place of
Source Qualifier transformations when you wish to read the data from the cobol copybook source.
Also, a Normalizer transformation is used to convert column-wise data to row-wise data. This is similar to the
transpose feature of MS Excel. You can use this feature if your source is a cobol copybook file or relational database
table. The Normalizer transformation converts columns to rows and also generates an index for each converted row.
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved
Thank You
Email us – [email protected]
Visit us - https://intellipaat.com
Intellipaat Software Solutions Pvt. Ltd. © Copyright Intellipaat.com All rights reserved