Scribd
Upload
191 views

Talend - Case Study

The document describes 7 case studies related to ETL processes. Case Study 1 involves bulk loading of data from files to a database table, handling duplicate and error records. Case Study 2 focuses on validating numeric and date fields and capturing rejected records. Case Study 3 requires reading an Excel file containing futures and options data and aggregating values before writing to output files. Case Study 4 describes concatenating text values for duplicate records. The remaining case studies involve tasks like parsing COBOL copy books, generating Excel output with multiple sheets, and parsing vector records to normalize transaction data.

Uploaded by

Rahul Birla
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
191 views

Talend - Case Study

The document describes 7 case studies related to ETL processes. Case Study 1 involves bulk loading of data from files to a database table, handling duplicate and error records. Case Study 2 focuses on validating numeric and date fields and capturing rejected records. Case Study 3 requires reading an Excel file containing futures and options data and aggregating values before writing to output files. Case Study 4 describes concatenating text values for duplicate records. The remaining case studies involve tasks like parsing COBOL copy books, generating Excel output with multiple sheets, and parsing vector records to normalize transaction data.

Uploaded by

Rahul Birla
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
You are on page 1/ 5

Case Study 1 : Bulk

Loading Job

Requirement :
1) Need to load millions of data stored in file to DB
2) We should not load the same file multiple times, If any one tried to
load then your job should send alert message saying that file already
loaded.
3) Should proved the reload option to job, if you set the reload option
=Yes then your job should delete the already loaded records from table and
load the same file again.
4) If job failed after loading of 1000 row and again if you restart the job
again that should skip 1001 (error row) and start loading from 1002
5) In case of failure you should send the error row as attachment to the
failure mail
6) We should not consider the Seq_NO columen in file and while loading
and generate the SEQ_NO from oracle sequence (load_seq.nexval()) for
each row
7) Job should exit if any error.
8) We should update the max_seq_no of bank table for each account.

File Format:
Account_no,Seq_NO, Amount,BUS_DATE,FILE_NO

Table - ITEM
Account_no,Seq_NO, Amount,BUS_DATE,FILE_NO

Table - BANK
Account_no, Max_Seq_No

Script Usage: load_job.ksh <input_file> <YES/NO>


--> Yes - reload the file, delete already loaded
data and load again
--> NO - exit and send alert if anyone trying
to load same file again.

Case Study 2 : Malformed records capture

Requirement:
1. It can be any input file
2. Validate Numeric and date fields in a row.
3. Capture the rejected records into output file.

Case Study 3 : Read Excel

Requirement :

A. Input File Format Excel File (File Name : SourceFile.xls)


B. Input file contains 2 types of records (Futures and Options
Records)
C. futures Records are grouped together if Put_call = " "
D. Options records are grouped together if Put_call = "P" or
Put_call = "C"

Implement the below Aggregate logic and split the files into 2
based on whether the records is futures record or options record.

Mapping Logic :
Futures (If put_call= " ")
Criteria to group by-

account_num
contract_symbol
tenor_month
tenor_year

Sum Up-

quantity
market_value

Options (If put_call= "Call" or "Put")


Criteria to group by-

account_num
contract_symbol
tenor_month
tenor_year
put_call
trade_price

Sum Up-

quantity
market_value

In the output file, the sum of market_value should be


divided with quantity to get column called Average
Weighted Market Price.

The output file should contain the below fields

account_num
contract_symbol
tenor_month
tenor_year
put_call (only for options output file)
trade_price (only for options output file)
sum(quantity)
sum(market_value)

Average Weighted Market Price

Attched the sample source file to work on and also sample data
that is expected .

Note : when divide operation is performed pls ensure that


we don't get any scientific notations . So, consider data
type of the input field accordingly.

Sample_data.xls SourceFile.xls

Case Study 4: String Concatination

Requirement :
Pls achieve the following scenario in Talend.If combination of
CORR_ACC_NO and GIN is same then do Concatenation for NOTE
column.

Input data is below

CORR_ACC_NO GIN NOTE

101 201 AAA

102 101 BBB

101 201 CCC

103 203 DDD

102 101 XXX

Output data sholud be showed as below

CORR_ACC_NO GIN NOTE

101 201 AAACCC

102 101 BBBXXX

103 203 CCC

Case Study 5: Cobol


Copy Book Parsing

How to parse Cobol copy book in


Talend

Case Study 6 : Generate Excel File

How to generate multiple outputs to same excel with different sheets.

Case Study 7 : Parse Vectors Records

Requirement:

Read the input file. And parse the vector records to calculate Principle
Amount,Net Amount and normalize the records based on Contra_no(length
field for normalise) to get multiple records for each trailer record.

Sample Input File Layout:


decimal(10) PS800_TRD_ACC_NO;
decimal(10) PS800_TRD_GIN;
string(8) PS800_TRD_SETT_DATE;

string(8) PS800_TRD_TRD_DATE;
decimal(3,0) PS800_TRD_CONTRA_CNT;
record // PS800-FIGURED-DATA
decimal(3,0) PS800_TRD_FIG_DATA_CNT;
record // PS800-SLB-FIG-DATA
record // PS800-TRD-FIG-CODE
string(1) PS800_TRD_FIG_TYPE;
string(3) PS800_TRD_FIG_ITEM;
end PS800_TRD_FIG_CODE;
string(3) PS800_SLB_FIG_CURR;
decimal(15,2) PS800_TRD_FIG_AMOUNT;
end[15] PS800_SLB_FIG_DATA;
record // PS800-VAR-PART-2
record // PS800-TRD-CONTRA-BRKR-DATA
decimal(17,6) PS800_SLB_MNR_QTY;
string(6) PS800_SLB_MNR_EXEC_BRKR;
string(6) PS800_SLB_MNR_EXEC_BADGE;
string(6) PS800_SLB_MNR_CL_BRKR;
decimal(7,0) PS800_TRD_TIME_OF_EXEC;
string(6) PS800_SLB_MAJ_EXEC_BRKR_VAR;
string(6) PS800_SLB_MAJ_EXEC_BADGE_VAR;
end[99] PS800_TRD_CONTRA_BRKR_DATA;
end PS800_VAR_PART_2;

Output File Layout :

string("|") acc_no;
string("|") gin;
string("|") trade_date;
string("|") settle_date;
decimal(“|”) quantity;
string("|") pri_amount;
string("|") int_amount;
decimal("|") mnr_quantity;
string("|") badge;

Mapping :

acc_no -- PS800_TRD_ACC_NO;
gin -- PS800_TRD_GIN;
trade_date -- PS800_TRD_TRD_DATE (YYMMDD)
settle_date -- PS800_TRD_SETT_DATE (YYMMDD)
quantity -- PS800_SLB_QTY -- Fail if quantity is null/empty
pri_amount -- PS800_TRD_FIG_AMOUNT when
PS800_TRD_FIG_ITEM=='PRI'
int_amount -- PS800_TRD_FIG_AMOUNT when
PS800_TRD_FIG_ITEM=='INT'
MNR_QUANTITY -- PS800_SLB_MNR_QTY
badge -- PS800_SLB_MAJ_EXEC_BADGE_VAR

You might also like