IBC Exercises SolutionsDataEngineer
IBC Exercises SolutionsDataEngineer
INTELLIGENT
BUSINESS CLOUD
2. Why is a Connector needed to facilitate the connection between an on-premise source system and the Intelligent Business Cloud?
• On-premise systems typically can’t communicate with external applications. [true]
• The Intelligent Business Cloud can’t communicate directly with other systems. [false]
• The Connector allows for a more stable and performant way to extract data. [true]
2
IBC Data Engineer Training
1. Access your training team in the Intelligent Business Cloud as described in the video and navigate to Event Collection. The source system
we are using in this training is a MS SQL database. Please follow the instructions below step by step to connect to the training source
system:
3
IBC Data Engineer Training
1
Access the Event
Collection…
2
… and create a
new Data Pool.
IBC Data Engineer Training
1
Name it
accordingly…
2
… and click on
“Save”.
IBC Data Engineer Training
Go to the data
connections.
IBC Data Engineer Training
Click on “New
Data Connection”.
IBC Data Engineer Training
1
Give the data
connection a
unique name…
2
… and choose “Microsoft
SQL Server (native)”.
IBC Data Engineer Training
2
… and click on
“Save”.
IBC Data Engineer Training
C
The connection has
been set up properly.
IBC Data Engineer Training
12
IBC Data Engineer Training
6. Which tables can be used for translating the technical table and column names?
• CDPOS [false]
• CDHDR [false]
• DD02T [true]
• DD03M [true]
7. When changing both the quantity and the price of a purchase order item and saving the changes with one click, how many rows are
created in CDPOS and CDHDR?
• One in CDPOS, one in CDHDR [false]
• Two in CDPOS, one in CDHDR [true]
• One in CDPOS, two in CDHDR [false]
• Two in CDPOS, two in CDHDR [false]
14
IBC Data Engineer Training
15
IBC Data Engineer Training
1. Which of the options are available to configure the extraction for each table?
• Option to only extract specific columns [true]
• Option the pseudonymize specific columns [true]
• Option to change the data type during the extraction [true]
• Option to set a filter statement to restrict the data extract [true]
16
IBC Data Engineer Training
1
Navigate to
“Data Jobs”…
2
… and click on
“New Data Job”.
IBC Data Engineer Training
1
Give it a proper
name …
2
… and select the Data
Connection you set up
previously.
IBC Data Engineer Training
1
Choose the
tables listed in
the exercise… 2
… and click on “Save”.
IBC Data Engineer Training
C
On the left you can see
all the tables you
selected before. Let’s
move on and define the
filters for the extraction!
IBC Data Engineer Training
• EKKO
• Filter Statement:
AEDAT >= 01.06.2014 00:00
• EKPO
• Join table EKKO
• Custom join condition:
EKPO.MANDT = EKKO.MANDT AND
EKPO.EBELN = EKKO.EBELN
• Filter on joined table:
EKKO.AEDAT >= ‘01.06.2014’
2. Why does it make sense to join table EKKO in the extraction of table EKPO?
• If we do not join the tables, they can’t be connected in our data model later on. [false]
• Because we only want to extract Purchase order items for which we also extracted the corresponding purchase order. [true]
24
IBC Data Engineer Training
2
… and open the
time filter.
1
The first table we
want to adjust is
EKKO. Click on it …
IBC Data Engineer Training
1
Activate the creation
date filter …
2
… and choose
the field AEDAT.
IBC Data Engineer Training
2
... and tick the box
to join this table
with EKKO.
1
Choose table
EKPO …
IBC Data Engineer Training
1
Enter the table
EKKO…
2
… specify the
join condition…
3
… enter the same
time filter as before…
4
… and click on
“Save”.
IBC Data Engineer Training
• EBAN
• Filter Statement:
BADAT >= 01.04.2014
• LFA1
• No configuration needed → Extract full table
3. Why do we use a different date for the Filter Statement of table EBAN compared to the Filter Statement of table EKKO?
• Purchase Orders which were created on the 01.06.2014 most likely have a purchase requisition which was created on an earlier point
in time. This is why we extract all Purchase Requisitions which were created two months in advance of our first Purchase Order as
well. [true]
• It is always better to extract as much data as possible to keep the load times at a minimum. [false]
• LFA1 doesn’t contain a date on which the table records were created. [false]
• LFA1 is the Vendor Master table and it doesn’t make sense to filter on the creation date of the table records. [true]
32
IBC Data Engineer Training
2
We can use the creation
date filter again …
1
The next table we
configure is EBAN.
3
… and choose
the field BADAT.
IBC Data Engineer Training
1
Enter the
correct date …
1
… and click on
“Save”.
IBC Data Engineer Training
C
We can leave the
settings for LFA1 as they
are as we want to
extract the whole master
data table.
IBC Data Engineer Training
• CDHDR
• Filter Statement:
UDATE >= 01.06.2014
• CDPOS
• Custom join condition:
CDPOS.MANDANT = CDHDR.MANDANT AND
CDPOS.OBJECTCLAS = CDHDR.OBJECTCLAS AND
CDPOS.OBJECTID = CDHDR.OBJECTID AND
CDPOS.CHANGENR = CDHDR.CHANGENR
• Filter on joined table:
• CDHDR.UDATE >= ‘01.06.2014’
• Because we only want to extract Changes which occurred after or on the 01.06.2014. [true]
• Because we only want to extract Purchase Orders which went through at least one change. [false]
36
IBC Data Engineer Training
1
Click on CDHDR …
2
… enable the
creation date filter …
3
… and choose
UDATE.
IBC Data Engineer Training
1
Open the configuration 2
of CDPOS … … tick the box …
3
… and add the
join.
IBC Data Engineer Training
Don’t forget to
click on “Save”.
IBC Data Engineer Training
• DD02T
• No configuration needed → Extract full table
• DD03M
• No configuration needed → Extract full table
• Name Mapping tables like DD02T and DD03M are one of the larger tables in a source system and typically take a long time to extract. [false]
• Name Mapping tables like DD02T and DD03M are one of the smallest tables in a source system and typically take only a few seconds to
extract. [true]
7. Exit the extraction task and execute the entire Data Job! Which information can you derive from the logs?
42
IBC Data Engineer Training
2
… so let’s go back to the
data job overview.
1
For both DD02T and
DD03M we don’t have
to set any filters …
IBC Data Engineer Training
Click on “Execute
Data Job”.
IBC Data Engineer Training
1 2
Make sure to … and press the
reload all data … execution button.
IBC Data Engineer Training
C
In the logs you can
monitor the execution of
every task.
IBC Data Engineer Training
2. Which of the following statements is/are true for the communication between the Intelligent Business Cloud and another cloud system?
• The Cloud Connector of the Intelligent Business Cloud establishes a connection to the other Cloud System via public API endpoints of
the Cloud Service. [true]
• The Data Engineer defines which tables to extract in the Cloud Connector interface. [false]
• The Celonis data storage transforms all files to parquet files. [false]
47
IBC Data Engineer Training
Hint: You can use the SQL statement “SELECT COUNT (*) FROM TABLENAME”;
49
IBC Data Engineer Training
Click on “New
Transformation”.
IBC Data Engineer Training
1
Give the task a
proper name …
2
… and click on
“Save”.
IBC Data Engineer Training
C
Adjust the statement for
EKPO and execute it again.
IBC Data Engineer Training
C
Adjust the statement for
EBAN and execute it again.
IBC Data Engineer Training
57
IBC Data Engineer Training
C
Adjust the statement for
LFA1 and execute it again.
C
Adjust the statement for
CDHDR and execute it again.
C
Adjust the statement for
CDPOS and execute it again.
C
Adjust the statement for
DD02T and execute it again.
IBC Data Engineer Training
In case you got one of the questions wrong, please use the solutions in the next section to correct your extraction task and run the extraction
again. When running the extraction again, please make sure to activate the toggle “Reload all data”. By activating this toggle all existing data is
deleted and you won’t receive any duplicates.
62
IBC Data Engineer Training
C
Adjust the statement for
DD03M and execute it again.
IBC Data Engineer Training
64
IBC Data Engineer Training
Click on “Delete”.
IBC Data Engineer Training
1
Give it a proper
name …
2
… and click on
“Save”.
IBC Data Engineer Training
1
If you click on the
“Refresh” button, …
2
… you can see the table
and all its columns in
your Schema Explorer.
IBC Data Engineer Training
• 136 [true]
• 2340 [false]
• 417 [false]
• None of the above [false]
3. Now, create a new transformation for the activity Create Purchase Order Item. Use the code for the first activity as guideline. Which tables
do you need for the activity "Create Purchase Order Item"?
• EKPO [true]
• EKKO [true]
• LFA1 [false]
• EBAN [false]
73
IBC Data Engineer Training
1
Insert a name …
2
… and click on
“Save”.
IBC Data Engineer Training
5. Is it necessary to include any restrictions (WHERE clause) when adding the activity "Create Purchase Order Item"?
• Yes. The purchasing document category needs to be a purchase requisition (EKKO.BSTYP='B’) [false]
• Yes. The purchasing document category needs to be a purchase order (EKKO.BSTYP='F’) [true]
• No. No restrictions are needed. [false]
6. Which fields should be selected as Case Key for the activity "Create Purchase Order Item"?
• LIFNR (Vendor Number) [false]
• MANDT (Client) [true]
• EBELN (Purchasing Document Number) [true]
• EBELP (Purchasing Document Item Number) [true]
7. Which field should you select as event time for the activity "Create Purchase Order Item"?
• FRGDT (Purchase Requisition Release Date, table EBAN) [false]
• ERDAT (Date on which vendor record was created, table LFA1) [false]
• AEDAT (Date on which purchase order item record was created, table EKKO) [true]
• BEDAT (Purchasing Document Date, table EKKO) [false]
79
IBC Data Engineer Training
8. Given the current sorting 10 for "Create Purchase Requisition Item", which sorting value is appropriate for the activity "Create Purchase
Order Item"?
• 0 [false]
• 5 [false]
• 10 [false]
• 20 [true]
8. Run the script to insert the activity “Create Purchase Order Item”. How many rows were added to the Activity Table?
• 27,937 [true]
• 3,860 [false]
• 25,390 [false]
• 1,210 [false]
80
IBC Data Engineer Training
1
Create a new
transformation,
insert a meaningful
name …
2
… and click on
“Save”.
IBC Data Engineer Training
2. Which columns should you select for the characteristics activity and timestamp, respectively? Please select the right columns. Then skip
the sorting column and click on “Finish”.
• _CASE_KEY [false]
• EVENTTIME [true]
• _ACTIVITY_EN [true]
3. Where can you a posteriori change the specification (case key, activity column etc.) of the Activity Table? Specify the Sorting Column and
click on “Save”.
• By clicking on the three dots on the Activity Table and then clicking on “Edit” [false]
• By clicking on the three dots on the Activity Table and then clicking on “Edit activity columns” [true]
• "Name mapping" tab in the data model editor [false]
4. In which tab(s) in the data model editor can you do a complete reload of the data model? Please do so.
• Permissions [false]
• Calendar [false]
• Data Loads [true]
• Model [false] 85
IBC Data Engineer Training
1
Go to “Process
Data Models” …
2
… and create a
new Data Model.
IBC Data Engineer Training
1
Name the Data
Model reasonably …
2
… and click on
“Save”.
IBC Data Engineer Training
Click on “Next”.
IBC Data Engineer Training
1
The third column contains
the timestamp.
2
Skip the other steps
and click on “Finish”.
IBC Data Engineer Training
Click on “Edit
activity columns”.
IBC Data Engineer Training
Specify the
sorting column.
IBC Data Engineer Training
Let’s do the
complete reload.
IBC Data Engineer Training
C
Wait until the tables
have been loaded
successfully.
IBC Data Engineer Training
6. How many cases start with the creation of the purchase requisition item?
• 349 [false]
• 421 [false]
• 136 [true]
• 260 [false]
7. What is the average throughput time between purchase requisition item and purchase order item creation?
• 20 days [false]
• 27 days [false]
• 25 days [true]
101
IBC Data Engineer Training
2
1 … and click on
Switch to “New Workspace”.
“Process Analytics” …
IBC Data Engineer Training
2
... select your Data
Model…
1
Choose the Data Pool
you created earlier…
3
… and click on
“Connect”.
IBC Data Engineer Training
1
Give your Workspace a
proper name… 2
… and click on
“Create”.
IBC Data Engineer Training
Create a new
analysis.
IBC Data Engineer Training
1
Give your analysis a
proper name … 2
… and create it.
IBC Data Engineer Training
Click on the
analysis to open it.
IBC Data Engineer Training
2
… and you can see that for 136
purchase order items the process
starts with the creation of the
purchase requisition item. 1
Move the slider
up …
IBC Data Engineer Training
2. Which table stores the name of the table and column that has been changed?
• CDPOS (Change Document Items) [true]
• CDHDR (Change Document Header) [false]
113
IBC Data Engineer Training
4. Have a look at the output, how many activities have been added to the activity table?
• 57 [true]
• 602 [false]
• 77 [false]
• 31 [false]
Create another transformation task (“Insert Activity: Change Quantity”) and write the code for the activity "Change Quantity". Answering the
following questions will help you accomplish this:
5. Which tables do you have to join for creating the activity "Change Quantity"?
• EKKO [true]
• CDPOS [true]
• CDHDR [true]
• EKPO [true]
• EKBE [false]
• LFA1 [false]
114
IBC Data Engineer Training
1
Name it
accordingly …
2
… and click on
“Save”.
IBC Data Engineer Training
Insert the
attached code.
IBC Data Engineer Training
57 price changes
have been added to
the Activity Table.
IBC Data Engineer Training
7. You already know how to link the pairs EKPO/EKKO and CDPOS/CDHDR. How can you link the change tables to the purchasing tables?
• Connect CDPOS and EKPO [true]
• Connect CDPOS and EKKO [false]
• Connect CDHDR and EKPO [false]
• Connect CDHDR and EKKO [false]
8. Which CDPOS columns do you have to consider for joining the table to the changed purchase order items in EKPO?
• TABNAME [true]
• CHANGENR [false]
• TABKEY [true]
• VALUE_NEW [false]
120
IBC Data Engineer Training
10. In the WHERE clause of the activity SQL statement, we already restrict the selection to EKKO.BSTYP='F'. Which other columns have to be
restricted when looking at quantity changes?
• EKPO.MENGE [false]
• CDPOS.FNAME [true]
• CDPOS.VALUE_OLD [false]
• EKPO.NETPR [false]
11. Where is the information on the purchase quantity of a given purchase order item stored?
• EKPO.NETPR [false]
• EKPO.MENGE [true]
• EKPO.MEINS [false]
• EKKO.MENGE [false]
121
IBC Data Engineer Training
13. Let’s focus on the SELECT part of the statement. What to use as case key?
• CDHDR.CHANGENR [false]
• CDPOS.TABNAME + CDPOS.FNAME [false]
• EKKO.MANDT + EKKO.EBELN [false]
• EKPO.MANDT || EKPO.EBELN || EKPO.EBELP [true]
14. Which value should you use for the activity column?
• 'Change Price’ [false]
• CDPOS.FNAME [false]
• 'Change Quantity’ [true]
• EKPO.MENGE [false]
15. Which time information would you choose for the event time?
• EKKO.AEDAT [false]
• CAST(CDHDR.UDATE AS DATE) + CAST(CDHDR.UTIME AS TIME) [true]
• CDHDR.UDATE [false]
• EKPO.AEDAT [false] 122
IBC Data Engineer Training
17. Execute your SQL script. How many quantity changes are inserted into the Activity Table?
• 13,423 [false]
• 257 [true]
• 4,352 [false]
• 158 [false]
18. Reload the data model and look at the updated process flow in Process Analytics. What is the average throughput time (in days) between
PO item creation and quantity change?
• 10 [false]
• 5 [false]
• 8 [true]
• 13 [false]
123
IBC Data Engineer Training
1
Create a new
transformation,
name it, …
2
… and click on
“Save”.
IBC Data Engineer Training
2
… and access the one
you previously
created.
1
Go to the Data
Models …
IBC Data Engineer Training
1
Open the data
loads tab …
2
… and conduct a
complete reload.
IBC Data Engineer Training
3. What are possible reasons for limiting the entries in a table (e.g. on the table LFA1) as opposed to embedding all rows of the source table?
• Avoid duplicates [false]
• Consider only vendors with purchase orders [true]
• Consider only top 10 vendors [false]
• There is no reason to do so [false]
131
IBC Data Engineer Training
4. Create the table “P2P_EKKO”, restricted to purchase orders with at least one position in table “EKPO”. How many rows does the table
contain?
• 24,675 rows [false]
• 9,753 rows [false]
• 13,829 rows [true]
• 267 rows [false]
5. Add the tables “P2P_EKKO” and “P2P_EKPO”to your data model and connect the tables. Which columns do you have to connect?
• EBELN [true]
• EBELP [false]
• MANDT [true]
• AEDAT [false]
6. Connect the two tables to the Activity Table. Which tables(s) do you have to connect?
• EKKO with Activity Table [false]
• EKPO with Activity Table [true]
• EKPO and EKKO with Activity Table [false]
132
IBC Data Engineer Training
1
Create a new
transformation,
name it …
2
… and click on
“Save”.
IBC Data Engineer Training
2
You can also refresh
the browser.
1
The code has been
executed
successfully.
IBC Data Engineer Training
Click on
“Execute”.
IBC Data Engineer Training
1
Insert an SELECT COUNT(*) FROM
statement to count the number of
rows in your new table.
2
EKKO consists
of 13, 829 rows.
IBC Data Engineer Training
1
Go back to your
Data Model … 2
… and click on “Add Data”
to add EKPO and EKKO.
IBC Data Engineer Training
1
Select both tables.
2
… and click
on “Finish”.
IBC Data Engineer Training
Choose the
second table.
IBC Data Engineer Training
The columns to
connect are
MANDT and
Click on “New
EBELN. Simply
Data
click on the key
Connection”
and the keyhole
symbol to
connect each line.
IBC Data Engineer Training
C
The first two tables
are connected.
IBC Data Engineer Training
8. In the same way as before, create the table “P2P_LFA1” for the vendor master data, restricted to relevant vendors (i.e. subject to the
“P2P_EKKO” table created in the last exercise). How many different vendors remain?
• 249 [true]
• 354 [false]
• 645 [false]
• 43 [false]
9. Add the “P2P_LFA1” to your data model. Which existing table(s) in the data model do you have to connect it to?
• _CEL_P2P_ACTIVITIES [false]
• EKPO [false]
• EKKO [true]
• None of the above [false]
149
IBC Data Engineer Training
1
To connect the Activity
Table with EKPO click on
“New Foreign Key” …
2
… and on
“Connect to
Target Table”.
IBC Data Engineer Training
1
The tables are
uniquely connected
via the case key.
2
Simply click on
“Save”.
IBC Data Engineer Training
C
Right now, all three
tables are connected.
Now, go back to the
data jobs.
IBC Data Engineer Training
Create a new
transformation.
IBC Data Engineer Training
1
Count the entries of
LFA1 as before.
2
The table
contains 249 rows.
IBC Data Engineer Training
Confirm your
selection.
IBC Data Engineer Training
LFA1 has to be
connected to EKKO.
IBC Data Engineer Training
Connect
MANDT and
LIFNR with
each other.
IBC Data Engineer Training
If you have
selected both
columns, click on
“Save”.
IBC Data Engineer Training
166
IBC Data Engineer Training
Click on the
configuration icon
on your Activity
Table and assign
EKPO as your case
table
IBC Data Engineer Training
C
The case table is
always marked
with this icon.
IBC Data Engineer Training
2
Then go back to
“Process Analytics”.
1
Don’t forget to do a
complete reload.
IBC Data Engineer Training
Click on “Add” to
define the
dimensions of the
table.
IBC Data Engineer Training
3
… and proceed to
the KPI menu.
1 2
Click on … select
P2P_LFA1, … NAME1, …
IBC Data Engineer Training
1
Add the case
count …
2
… and click on
“Done”.
IBC Data Engineer Training
2. In SAP, which tables provide information with respect to table and column translations?
• EKPO [false]
• DD02T [true]
• DD03M [true]
• MAKT [false]
6. Configure “Name mappings from pool” for your data model by using the tables DD03M and DD02T. Which columns in DD02T contain the
technical table name, the translation text as well as the language key?
• DDTEXT [true]
• TABNAME [true]
• AS4LOCAL [false]
• DDLANGUAGE [true]
177
IBC Data Engineer Training
Click on the
configuration
icon of LFA1 and
click on “Edit”.
IBC Data Engineer Training
Enter “LFA1” as an
alias for “P2P_LFA1”
to make sure that
the name mapping
can become
effective later on.
IBC Data Engineer Training
2
Then go to “Name
Mapping”.
1
Repeat the
same steps for
EKKO and EKPO.
IBC Data Engineer Training
1
Choose DD02T …
2
… and click on
“Done”.
IBC Data Engineer Training
8. Click on “Load Mapping From Pool” and reload your data model for the name mapping to become effective. How many columns were
translated for the Language Key “E”?
• 510 [true]
• 3 [false]
• 506 [false]
• 157 [false]
184
IBC Data Engineer Training
1
For the column
name mapping
select DD03M …
2
… and click on
“Done”.
IBC Data Engineer Training
1 2
Select all the … and load the
columns … mappings from
pool.
IBC Data Engineer Training
C
You can see that the name
mapping successfully
translated 3 tables (EKKO,
EKPO, LFA1) and 510
columns for the Language
key “E”