DATA LOAD FROM TEXT FILE TO

ORACLE TABLE IN ODI 12C

ABSTRACT
Step by step process along with respective snapshot to understand the process of data
loading from text file to Oracle database table through ODI 12C
BY
TAMOGHNA BANDYOPADHYAY

1
1. PREREQUISITE:
1.1. Text file with data of a table, which will be imported in Oracle table. This data must be fixed size data and fields
are separated by TAB and records are separated by TAB
1.2. Tools: TOAD, ODI 12C

2. BEFORE WE START:
2.1. Create two users/schemas in Oracle (TOAD) in SYS schema and give the few permissions to that user. These
users / schemas are used as master repository and work repository in ODI 12C.
(Note: Master Repository: This is a data structure containing information on the topology of the company’s IT
resources, on security and on version management of projects and data models. This repository is stored on a
relational database accessible in client/server mode from the different Oracle Data Integrator modules.
Work Repository: This is a data structure containing information on data models, projects, and their use. This
repository is stored on a relational database accessible in client/server mode from the different Oracle Data
Integrator modules. Several work repositories can be created with several master repositories if necessary.
However, a work repository can be linked with only one master repository for version management purposes.
Query:
CREATE USER <MASTER_REPOSITORY_NAME> IDENTIFIED BY <MASTER_REPOSITORY_PASSWORD>;
GRANT CONNECT, RESOURCE,DBA TO <MASTER_REPOSITORY_NAME>;
CREATE USER <WORK_REPOSITORY_NAME> IDENTIFIED BY <WORK_REPOSITORY_PASSWORD>;
GRANT CONNECT, RESOURCE,DBA TO <WORK_REPOSITORY_NAME>;

2.2. Now open ODI 12C tool. After opening the tool, you can see there is an icon like shown below. Click on that icon.

FIGURE 2.2.1
After clicking on the icon another window will be opened and there we can create the master repository for
ODI. (Note: This master repository is designed for ODI. Previously we have created the schema for it. All pre-
defined tables will be created in <MASTER_REPOSITORY> schema for Master Repository in ODI 12C.

2
 FIGURE 2.2.2
After click on OK, a window appears. There we will fill up the details as below.

FIGURE 2.2.3
Technology: Oracle (As we are creating the environment in Oracle)
JDBC Driver: Oracle.jdbc.OracleDriver (This JDBC driver is used to connect Oracle database from ODI 12C)
JDBC URL: By default the URL appears as jdbc:oracle:thin:@<host>:<port/ServiceName> . Put the host as localhost,
as we are creating the environment in our local system. Put port/ServiceName as 1521/orcl
User: Give the Master Repository name, which is created in Database as Master Repository.
Password: Password will be Master Repository’s Password in Database
DBA User: Put sys as sysdba
DBA Password: password is Manger1. It will appears as (.).
Now Click on Test Connection Button. If the data of each field is true and associated with each other, a dialogue
bow will appeared with the message “Successful Connection”.
Then Click on Next > button to proceed. After clicking on the Next > button a window will be appeared like below.

3
 FIGURE 2.2.4

In this window, we will put the details of ODI supervisor. Supervisor user is already appears in the Supervisor User
field.
Supervisor User: SUPERVISOR
Supervisor Password: Put the password as sunopsys
Confirm Password: Re-enter the password.

Now you can click on either Next > or Finish Button. (If you click on Next > button, an option will appears, which
states the default storage of password. Don’t make any changes. Just click on Finish to complete the process)

2.3. As the wizard closed, ODI will create Master Repository default tables in MASTER_REPOSITORY schema. It will
take some time. At the time of creation of pre-defined tables, you see the Logs of table generation is population
below of ODI 12C tool’s interface. Now disconnect the session.

2.4. Reconnect the session with clicking the symbol.

FIGURE 2.4.1
A window will be appeared like

FIGURE 2.4.2
4
2.5. Click on (+) symbol to add new connection details. After click on (+) symbol, a window will be appeared. We will
fill up data details like below.

FIGURE 2.5.1
Login Name: Give a unique login name, which will identified later.
User: this user is supervisor user. So we will put the data of ODI supervisor data. We put SUPERVISOR here.
Password: Put SUPERVISOR’s password here. It is sunopsys.
Database Connection (Master Repository):
User: Here we put the data of MASTER_REPOSITORY’s data, which is created in Oracle database as Master
repository. So we put the name of MASTER_REPOSITORY.
Password: Put the password of MASTER_REPOSITORY user.
Driver List: Choose Oracle JDBC Driver as the environment is in Oracle.
Driver Name: As we choose driver list from dropdown box, Driver Name is automatically populated. It is like
oracle.jdbc.OracleDriver.
URL: URL is jdbc:oracle:thin:@localhost:1521/orcl. Modification of URL has been discussed before.
Work repository: when you create Master Repository for first time, Master Repository only will be checked.
Now click on the Test button to check the connection. If all credentials are ok, then a dialogue box will appears
with the message “Successful Connection”. Then press OK to complete the process.
After completing the process, if you disconnect the session and reconnect to session, newly created session
name is appeared in the dropdown box of Repository Name.
2.6. Now we connect the work repository to the master repository in ODI 12C. Connect the newly created Master
repository. After connection go to Topology Pane in top left corner as shown below. Expand the Repositories
Option. Choose Work repository option. Right click on Work Repository Option. Choose New Work Repository.

5
 FIGURE 2.6.1
After clicking on New Work Repository, A Window will be appeared like below

FIGURE 2.6.2
All attributes are automatically populated, as these all are fetched from Master Repository in ODI. Now click on
Next > Button and set a new name and password for work repository as below and click on Finish to create work
repository.

FIGURE 2.6.3
6
2.7. Work repository in ODI is created and it is linked with Master Repository in ODI. But one manual step is required
to link the Work Repository with Master Repository at the time of connecting the session. Disconnect and
reconnect the session of ODI. AT the time of connecting, choose the Master Repository name and click on Edit
button as below.

FIGURE 2.7.1

After clicking on edit button the window will be appeared as below. We are familiar with this window, at the
time of creation the master repository. Now we click on the radio button of Work Repository option. Then we
click on search option. We can see the newly created work repository name is appeared in a dialogue bow.
Choose that work repository and then click on Test button to check the connection. Lastly click on OK. Then
connect the master repository in ODI.

FIGURE 2.7.2

7
 After creation of master repository and work repository and users/schema in Oracle, We will create target database
table in the work repository schema. Here we use default EMPLOYEES table of Oracle from HR schema. We will
create EMPLOYEES table in WORK_REPOSITORY schema with same specification of EMPLOYEES table of Oracle from
HR schema.

3. TEXT FILE TO ORACLE TABLE DATA LOAD:

All things ae set for our project. Now we will proceed for the data loading process from text file to Oracle table. Here
we create the topology for the project. There are two types of topology. A) Physical topology B) Logical topology.
(NOTE: The Oracle Data Integrator Topology is the physical and logical representation of the Oracle Data Integrator
architecture and components.)

3.1. CREATING OF PHYSICAL TOPOLOGY OF SOURCE TEXT FILE:

Connect the master repository in ODI. Go to topology pane. Expand the physical Architecture option. Then
expand the technology option. If you don’t see many types of Technology, then uncheck the Hide unused
Technologies option as below.

FIGURE 3.1.1
Now right click on File and choose New Data Server option.

FIGURE 3.1.2 FIGURE 3.1.3

8
After clicking on New Data Server option a new window will open. Here we set the physical data server name
and Host of the data server as localhost, as the environment is standalone. We don’t not need to change any
other value in any other tab. Then we click on Test Connection button to check the connection. Once we click
on that button, a dialogue box will come as below.

FIGURE 3.1.4
We choose Local (No Agent) as physical agent and click on test. Another dialogue box will appeared for
successful connection. Then we save the physical source topology of text file in ODI. After saving the topology,
it will appeared under File option as below.
Still now text file is not physically linked with ODI. Now select the newly created physical topology. Right click
on it and choose New Physical Schema Option.

FIGURE 3.1.5 FIGURE 3.1.6

A new window will appeared after clicking on the New Physical Schema option. We will fill up the attributes of
that window as below.

FIGURE 3.1.7

9
 We put the source file’s parent folder’s path in Directory (Schema) and Directory (Work Schema) field and save
the current changes. At the time of saving the current changes a dialogue box will appeared as below. Click on
OK to continue.

FIGURE 3.1.8

3.2. CREATING LOGICAL TOPOLOGY OF SOURCE TEXT FILE:

After creating physical topology, it is time to create logical topology. Logical topology creation is described below
step by step. Creating logical topology is under Logical Architecture. Choose New Logical Schema to create
logical topology. Put a unique name in Name field for logical schema. Now click on the down arrow under
Physical Schemas column. Choose the newly created physical schema name. No need to change any other value
in any other tab. Save the current changes.

FIGURE 3.2.1 FIGURE 3.2.2

We successfully created the logical architecture and physical architecture of source text file.

3.3. CREATING PHYSICAL TOPOLOGY FOR TARGET TABLE:

Here we will create physical topology for the target file, which is in Oracle table format. Basic steps are same as
creating physical topology of text file. The Only change is in Types and attributes of topology. As the target file
type is Oracle, we will choose Oracle under Technology tab and right click on the Oracle option and choose New
Data Server. After clicking on New Data Server, a window will be appeared. We will fill up the field of window
as below.

10
 FIGURE 3.3.1

FIGURE 3.3.2

Here in Definition Tab,

Name: put a unique name of target physical topology.
Instance/Dblink (Data Server): localhost
Connection:
User: Put work repository name, which is created in oracle.
Password: Work repository’s password.
IN JDBC tab:
JDBC Driver: oracle.jdbc.OracleDriver (By default it is set)
JDBC URL: jdbc:oracle:thin:@localhost:1521/orcl (we already know how to set the URL)
Now click on Test Connection Button. After clicking on the Test Connection button one dialogue box will be
appeared like below. Please click on OK to continue and choose Local (No agent) to test the connection. If
all credentials are ok, then connection will be successful. Then close the current page.

FIGURE 3.3.3
11
 Click on New Physical schema as shown below. Now click on the down arrow, associated with Schema
(Schema) and Schema (Work Schema) field. Choose the Work Repository schema name.

FIGURE 3.3.4 FIGURE 3.3.5

Save the current changes. While saving a dialogue box will be appeared like below. Please click on OK to
continue. Close the current page.

3.4. CREATING LOGICAL TOPOLOGY OF TARGET TABLE:

Creating logical topology is also same as previous one. The only change is in type and attributes to be filled up.
Step by step process has been shown below. Click on New Logical Schema as shown below. Choose the physical
target topology under Physical Schema column and save the current changes.

FIGURE 3.4.1 FIGURE 3.4.2

Logical schema is completed.

12
So far, we have created Physical topology and logical topology for both source text file and target oracle table. Now we
are going to create the models of source and target file.

3.5. CREATING MODEL OF SOURCE TEXT FILE:

For creating model, navigate to designer pane. Then expand the Models option. Click on the folder symbol and
choose New Model Folder option. Using Model folder is good practice in a large project. We can give a proper
name of the folder to identify it later. Please look below for reference.

FIGURE 3.5.1 FIGURE 3.5.2

After creating the folder, right click on the folder and choose New Model option to create the new model. Here
we will fill up few attributes value, shown in below.

FIGURE 3.5.3 FIGURE 3.5.4

In Definition tab,
Name: Put a unique name to identify the model for source file.
Code: This will be automatically populated once you type the Name.
Technology: Technology will be File as the source file is in Text format.
Logical Schema: Choose the appropriate logical schema for the source text file. Here as we choose Technology as
File, Logical schema for Oracle will not be shown in the dropdown box. In current scenario, there is only one logical
schema for Technology type File. So, only one schema is shown in the dropdown box.
Don’t change any other attribute in any other Tab. Save the current changes.

13
Once you save the changes, the model will be appeared under its folder. Right click on the model and choose New
Datastore option to add the text file.

FIGURE 3.5.5

In Definition Tab, Put the name of the source text file. Now click on search icon to find the physical location of the
file in local system.

FIGURE 3.5.6

FIGURE 3.5.7

14
Choose the file and click on Open button. Now click on Files Tab and fill the value as shown below.

FIGURE 3.5.8 FIGURE 3.5.9

A part of the source text file is shown above. First row is the Header of the data. That is why we mentioned in
Heading (Number of Lines) in Files Tab and field separator is Tab and record separator in MS-DOS. In Attribute tab,
click on Reverse Engineer button. The main purpose of Reverse Engineer to populate metadata from Data server.
After clicking reverse Engineer a pop up box will be appeared as below. Change the value of Number of Rows to
Fetch to the maximum and then click on OK to continue.

FIGURE 3.5.10

Once you click on OK, Attribute Setup Wizard will be appeared. Here you can select each column and set the input
datatype of the column. It is recommended that datatype of each column should be as String. Please note few
things have to be modified at the time of setup the input data. By default there in only one column heading is
there as C1. Once you move the cursor near the number scale, you can setup the range of the column. When all
data of a particular column come under one block, please click to mention end of the column. Then modify
Attribute Name field with the actual column name. If you put the same column name which will be imported to
the table’s column, then it will be easier to importing data at the time of mapping. Once you setup all the
attribute. Click on OK to continue. You can modify the column name in next window too. After modification, save
the current changes. If all setup are ok, columns of the text file will be populated under the text file model as
shown below.

15
 FIGURE 3.5.11

FIGURE 3.5.12

FIGURE 3.5.12

3.6. CREATING THE MODEL OF TARGET DATABASE FILE:

We have already created model of source text file. Now we are going to create the model for target database
table. Creation of model folder and creation of model are same as creation of model of text file. At the time of
creation model of database table, we choose technology as Oracle and logical schema for target database. No
need to modify anything in any tab. Save the current changes.

16
 FIGURE 3.6.1
Now we can see the newly created model, under the target model folder. Right click on the model and choose
the reverse engineer option. Here the concept of Reverse Engineer comes into play. As reverse engineer
option has been clicked, ODI will populate all the tables of the schema, which is linked with the model.

FIGURE 3.6.2 FIGURE 3.6.3

3.7. CREATING PROJECT TO LOAD DATA WITH TEXT MODEL & DATABASE MODEL
In Designer tab, please click on New Project as shown below. Give an appropriate name of the project and
save the changes.

FIGURE 3.7.1 FIGURE 3.7.2

17
After creating the Project folder, some sub folders are automatically created under the project folder.
Here we have to import few things before we start the mapping.

FIGURE 3.7.3

IMPORTING KNOWLEDGE MODULES:

Knowledge modules take important part for data mapping. There are many types of Knowledge Modules like
a. Integration Knowledge Module (IKM)
b. Check Knowledge Module (CKM)
c. Loading knowledge Module (LKM)
d. Reverse-Engineering Knowledge Module (RKM)
e. Journalizing Knowledge Module (JKM)
f. Service Knowledge Module (SKM)

IKM: Integrates data from the staging area to a target.

CKM: Check consistency of data against constraint
LKM: Loads heterogeneous data to a staging area

Right click on Knowledge Modules and choose Import Knowledge Modules. If the path is not set for the
importing knowledge modules, please follow the path as shown below. Select necessary Knowledge Modules
and click on OK. In current scenario we will import CKM Oracle, LKM File to Oracle (EXTERNAL TABLE), IKM
Oracle Control Append. There are many more Knowledge Modules which will come into play in another
scenario. Now under newly created project folder, there is folder named First Folder. Expand that folder and
you can see, there is an option named Mapping. Right click on it and choose New Mapping as shown below.
Put a unique name of that folder and save. Once you save that, a workspace will be appeared, where you can
create mapping between source and target table.

FIGURE 3.7.4

18
 Now expand the Model option. Then navigate to the source text file and select the text file. Drag and drop the
text file on the workspace. A logical data structure will be appeared on the work space for source text file.
Follow the same process for getting the logical data structure of target database table.

FIGURE 3.7.5
We can see there is a small circle at outer surface of logical schema of source table. Click on that circle. Drag the circle
and release the point at lower left corner of target table. Now we are going to link both logical table as shown below

FIGURE 3.7.6

As soon as you release the pointer, a pop up box will be appeared on the screen. If all columns of source file, have same
name with target file, choose the match option as By Name. If Column names are not appropriate but the sequence of
both source file and target file are same, choose By Position option. If you choose By Name option, but there and very
few columns that do not match the name with target source file’s column’s name, then those columns will not be
mapped automatically. We have to map those columns manually by drag the source text file’s column and drop on the
target database file’s column. If any column of target database table is unmapped, the color of arrow will be lighter than
the mapped one.

19
 FIGURE 3.7.7 FIGURE 3.7.8

Here we can see attributes of source file and target table in logical mapping option. We select database table. So we can
see its attributes under property inspector.

FIGURE 3.7.9

Now click on Physical Tab. Here we can see, a staging table has been created and the data from text file will be moved to
that staging table through LKM File to Oracle. LKM File to Oracle is pre-defined set of codes, which helps to move the
data from text file to staging table. As we already imported the LKM, It is directly embed the LKM to its appropriate
position. If there are multiple LKMs, you can choose the correct one according to your choice. You can see these
attributes while you select the staging table.

20
 FIGURE 3.7.10

While you select on the target table, you can modify the IKM and CKM as shown below. In IKM, IKM Oracle Control
Append is set by default. Change it to IKM Oracle Control Append. CKM is already present as CKM is already imported.

FIGURE 3.7.11

Now all is set for execution. Save the current changes and click on the Run button as shown below. Before execution, a
pop up box will be appeared. Here you can tick the simulation option. If simulation option is ticked, then the project will
executed in disconnected mode, means all steps will be executed, but data will not be imported. We can check the steps
or errors from the simulation result without executing the process. Set the log level to 5 to view the highest level of log,
means it will show the errors or execution steps in details.

21
 FIGURE 3.7.12

Once you execute the process with uncheck the simulation option, a pop up box will be appeared, states that the
session has been started.

FIGURE 3.7.13

If you want to see the execution or error steps please navigate as shown below. Here we can see, there is an error in
Oracle stated that Not a Valid Month. It means, while load the data from Text file to Oracle table, Date column of text
file was mentioned as string earlier, but in Oracle table, it is in date format. So conversion to Date from String is not
possible until we write some condition at the time of mapping.

FIGURE 3.7.14

22
Navigate to Advance Expression Editor. It is in logical schema. Select the Date field in Target Table. Here we can see, in
the properties of HIRE_DATE field, expression field holds the value as SRC.GIRE_DATE. SRC means the source text file
name and HIRE_DATE is column name. As we know that HIRE_DATE of source file is set as string, we have to modify the
expression. By clicking on the Expression Advanced Editor, modify it to TO_DATE(SRC.HIRE_DATE, ‘MM/DD/YYYY’).
Check the date format according to data and save the changes

FIGURE 3.7.15

After successful execution of mapping a green tick will be appeared before the log details. By seeing this we can confirm
that the data has been loaded successfully.

FIGURE 3.7.16

23