Integration Technical Document - v4 5 PDF
Integration Technical Document - v4 5 PDF
Author:
Ejada
Creation Date:
17-07-2009
Last Updated:
11-04-2010
Version:
4.5
1. Document Control
Date
Author
Version
Change Summary
17-7-2009
03-11-2009
22-01-2010
14-02-2010
26-02-2010
03-03-2010
09-03-2010
18-03-2010
25-03-2010
26-03-2010
01-04-2010
11-04-2010
Ejada
Ejada
Ejada
Ejada
Ejada
Ejada
Ejada
Ejada
Ejada
Ejada
Ejada
Ejada
0.1
0.2
0.7
1.0
1.5
2.0
2.7
3.0
3.5
4.0
4.1
4.5
First version
Introduction section
Integration Architecture section
FMS interface
Informatica part
CRM part
FIN part
TR part
HR part
EAM part
Workflow guide
Adding a New Workshop-Depot Mapping
1.2 Contributors
Contributor
Role
Position
Ahmed Kandil
Emad Sarhan
1.3 Reviewers
Name
Role
Position
Document Status
Date Reviewed
Ejada
Ejada
Template
First version
17-7-2009
1.5
4.5
26-02-2010
11-04-2010
Emad Sarhan
Emad Sarhan
Comments
Incorporated
Page ii of 65
Contents
1.
2.
3.
1.2
Contributors ................................................................................................................ii
1.3
Reviewers ...................................................................................................................ii
Introduction .............................................................................................. 1
2.1
2.2
2.3
Audience ................................................................................................................... 1
2.4
Overview ................................................................................................... 2
3.1
Abbreviations ............................................................................................................. 2
3.2
3.3
4.
4.2
4.3
4.4
4.5
4.6
4.7
Page iii of 65
5.
5.2
6.
5.4
6.3
6.4
6.5
6.6
6.7
8.
5.3
6.1
7.
7.2
8.2
Page iv of 65
8.2.1
8.2.2
8.2.3
9.
Functionality .................................................................................................. 58
Benefits .......................................................................................................... 58
Implementation .............................................................................................. 58
Appendix ................................................................................................. 59
Page v of 65
2. Introduction
2.3 Audience
This document is targeted for technical readers that will maintain the integration project. It
is expected that the reader has some technical background related to the technologies
mentioned in the previous paragraph.
Technology
URL
http://www.oracle.com/technology/products/soa/soasuite/index.html
Informatica
www.informatica.com/
www.oracle.com
http://www.oracle.com/technology/products/ias/bpel/index.html
Page 1 of 65
3. Overview
The goal for the integration project is to integrate different company system according the
project scope of work. The document should help in monitoring and managing day-to-day
integration activities
3.1 Abbreviations
The table below surmises the abbreviations used in the integration project
Abbreviation
Meaning
TR
FMS
CRM
FIN
HR
EAM
Page 2 of 65
Name
Usage
Products
10.10.2.162
Integp1
10.10.2.174
Integp3
10.10.2.172
Integp2
All servers have Windows2003 Enterprise Edition SP2 installed and Oracle RDBMS
10gR2 (10.2.0.3)
Schema location
STAGE_INTEG_USER
TR_INTEG_USER
FIN_INTEG_USER
CRM_INTEG_USER
Service name
Extract Fleet Trips Data
Process Fleet Trips Data
Export Fleet Data Trips Data
Export Stations data
Export Lines data
Export Carriers data
Producer
FMS
DH
DH
DH
DH
DH
Consumer(s)
DH
DH
TR
FIN
FIN
FIN
Page 3 of 65
DH
DH
DH
TR
TR
TR
TR
TR
FIN
CRM
TR
CRM
CRM
CRM
FIN
FIN
FIN
TR
FMS
FMS
FMS
FMS
FMS
FMS
HR
DH
DH
DH
DH
HR
FIN
FIN
FIN
FIN
FIN
TR
TR
CRM
FIN
TR
FIN
CRM
CRM
CRM
CRM
EAM
EAM
EAM
EAM
EAM
TR
FMS
TR
All
Service
Export Fleet Trips Data
Description
Protocol
Informatica
Informatica
Informatica
Informatica
Informatica
Informatica
Informatica
Informatica
Page 4 of 65
Informatica
Informatica
Informatica
Informatica
Informatica
SOAP over HTTP
Informatica
Informatica
Informatica
Informatica
Informatica
SOAP over HTTP
Page 5 of 65
DataIntegration /
SOAP over HTTP
LookupsWS /
SOAP over HTTP
Service name
Extract Fleet Trips Data
Processing Fleet Trips
Data
Export Fleet Trips Data
Export Stations data
Export Lines data
Export Carriers data
Export Drivers data
Export Vehicles data
Export Drivers Trip
Allowance data
Export TR Tickets data
Export TR Services data
Export TR Luggage data
Export TR International
Carriers data
Export TR Services
Lookups data
Export FIN Agent
Export CRM Agreements
Export TR Agreements
update
Export CRM Accounts
data
Export TR Passenger
Data
Create CRM Payment
Create Complete Order
Create Order Payment
Lookup
Get Account Amount
Export CRM customers
Data
Check In Maintenance
Stay
Create Maintenance Stay
Modify Maintenance Stay
Delete Maintenance Stay
Export Vehicle Dispatch
Data
Push Fleet Dispatch Data
Export HR Data
Employee
Location
wf1_call_FMS_WS
wf2_call_FMS_TR_data_processing
PROD_FMS_TR
PROD_FMS_TR
wf2_call_FMS_TR_data_processing
wf3_Fill_DH
wf3_Fill_DH
wf3_Fill_DH
wf3_Fill_DH
wf3_Fill_DH
wf_DH_HR_DECODE
PROD_FMS_TR
PROD_FMS_TR
PROD_FMS_TR
PROD_FMS_TR
PROD_FMS_TR
PROD_FMS_TR
PROD_FMS_TR
wf_TR_FIN_TICKETS
wf_TR_FIN_SERVICE
wf_TR_FIN_LUGGAGE
wf_TR_FIN_INT_CARR_LOOKUP
PROD_FIN_TR
PROD_FIN_TR
PROD_FIN_TR
PROD_FIN_TR
wf_TR_FIN_SERVICES_LOOKUP
PROD_FIN_TR
wf_ FIN_TR_AGENTS
wf_CRM_AGREEMENT_TR_CONTRACT
wf_TR_CONTRACTS_CRM_AGREEMENTS
PROD_FIN_TR
PROD_TR_CRM
PROD_TR_CRM
wf1_CRM_ACCOUNTS_FIN_CUSTOMERS
/ wf2_CRM_AccountAddress_FIN_Site
wf_m_TR_CUSTOMER_CRM_CONTACT
PROD_CRM_FIN
Create_CRM_Payment
WEB-completeOrder
WEB-createOrderPaymentLookup
BPEL Server
BPEL Server
BPEL Server
WEB-getOnAccountAmount
TRExportInterface- BaseDataPortType
BPEL Server
BPEL Server
WorkshopService-checkinMaintenanceStay
BPEL Server
WorkshopService-createMaintenanceStay
WorkshopService-modifyMaintenanceStay
WorkshopService-deleteMaintenanceStay
VehicleDispatchVehicleWorking-export
VehicleDispatchVehicleWorking
TripPortType-importDispatchedTrips
PersonnalImport-importEmployees
BPEL Server
BPEL Server
BPEL Server
BPEL Server
PROD_TR_CRM
BPEL Server
BPEL Server
Page 6 of 65
Note: If you are not connected to a repository, you can remove the repository from the Navigator.
Select the repository in the Navigator and click Edit > Delete. The Workflow Monitor displays a
message verifying that you want to remove the repository from the Navigator list. Click Yes to
remove the repository. You can connect to the repository again at any time.
Page 7 of 65
Note: You can also open an Integration Service in the Navigator without connecting to it. When
you open an Integration Service, the Workflow Monitor gets workflow run information stored in the
repository. It does not get dynamic workflow run information from currently running workflows.
4.4 Filtering Tasks and Integration Services
You can filter tasks and Integration Services in both Gantt Chart view and Task view. Use
the Filters menu to hide tasks and Integration Services you do not want to view in the
Workflow Monitor.
To filter tasks:
1. Click Filters > Tasks.
2. Clear the tasks you want to hide, and select the tasks you want to view.
3. Click OK.
Note: When you filter a task, the Gantt Chart view displays a red link between tasks to indicate a
filtered task. You can double-click the link to view the tasks you hid.
Page 8 of 65
Tip: To filter an Integration Service in the Navigator, right-click it and select Filter Integration
Service.
4.5 Opening and Closing Folders
You can select which folders to open and close in the Workflow Monitor. When you open
a folder, the Workflow Monitor displays the number of workflow runs that you configured
in the Workflow Monitor options. For more information.
You can open and close folders in the Gantt Chart and Task views. When you open a
folder, it opens in both views. To open a folder, right-click it in the Navigator and select
Open. Or, you can double-click the folder.
Page 9 of 65
Page 10 of 65
Monitor. Verify succeeded or failed workflows and which workflows are scheduled.
Get Log Events for workflows and sessions logs: used to get internal Informatica Log screen.
Page 11 of 65
Monitor
Page 12 of 65
Page 13 of 65
5.1.3 Configure Informatica Extraction period:To allow integration to be parameterized to accept the extraction period from FMS to TR system Integration stuff
should change parameter file mentioned below to be reflected at daily scheduled workflow which extracted Trips data
into TR system. As below table
Parameter
$$DAYES_BEFORE_SYSDATE
-30
$$DAYES_OF_WINDOW
31
91
91
5.2 BPEL Processes Monitoring:In this section we describe how to monitor BPEL (Business Process Execution Language) Processes, Use below way
to do that
1. Oracle BPEL Console
Page 14 of 65
Page 15 of 65
Page 16 of 65
Integration Point
CRM-FIN
TR-FIN
3
4
5
FMS-TR
FMS-HR
CRM-TR
Scheduling Time
1:00 AM
Run every two
hours starting from
1:00 AM each day
2:30 AM
2:30 AM
12:00 AM
4:00 AM
8:00 AM
12:00 PM
4:00 PM
8:00 PM
Page 17 of 65
Page 18 of 65
3456-
7- Go to Step Instances and you will find all the details related to the execution steps for the required
workflow instance.
Page 19 of 65
Page 20 of 65
10111213-
14- Go to Step Instances and you will find all the details related to the execution steps for the required
workflow instance.
Page 21 of 65
error.log
hql.log
server.log
sql.log
wrapper.log
Administrative problems like network failures, out-of-memory problems etc. can all be found in
boot.log or server.log
wrapper.log (this is a catch all log written by the Windows service wrapper for Java).
error.log
However, not every Error log message in these files points to administrative problems or a real problem at all. For
example, errors reporting EJBTransactionRolledBackException are generally handled by the concerned IVU.plan
applications or interfaces.
Page 22 of 65
IVU-specific logs
These log files contain technical log messages of IVU.plan applications or interfaces. They are generally only
understandable by the respective IVU developers. The set of these files and their configuration might change any time
without notice.
Currently, these include ivu*.log, saptco-*.log and usecase.log.
For the online interface, this includes cases where the call to the TR online webservice failed or where that
service returned a non-success error. In these cases, the online export will be tried again in the interfaces
next run. Thus, the situation is only problematic if these files appear repeatedly for subsequent exports.
For the offline interface, this includes cases where the response to the TR offline service failed. The TR
system should recognize and handle such cases.
For both interfaces, it also includes cases where the export data could not be obtained (e.g. due to illegal appserver state or if the database if offline). In these cases, a technical Exception and stack trace is included. It
should first be checked whether app-server and database are running OK (e.g. by starting Personnel
Dispatch). Otherwise, the error should be reported to IVU.
online-export-<timestamp>.xml
The data payload of the respective online-export.
offline-export-<timestamp>.xml
The data payload of the respective offnline-export.
Page 23 of 65
6.3.6 HR Interface
Any problems are reported in the response of the personnel import service to the HR system. This includes
Inconsistent data messages: they point out illegal import data regarding e.g. inconsistencies of staff
membership and depot assignments or unavailabilities. They should be resolved by HR system users.
FMS update or other error messages: these point to technical import failures. It should first be checked
whether app-server and database are running OK (e.g. by starting Personnel Dispatch). Otherwise, the error
should be reported to IVU.
Page 24 of 65
Page 25 of 65
[2] Query for the request whose name starts with INTEGRATION , as follows :
Page 26 of 65
Page 27 of 65
[3] Check the execution of the interface on the last night as follows:
Go to the line of the last nights execution (the line just under the pending request line):
Check the Two Columns Phase and Status.
The phase should be Completed and the status should be Normal. If status is Error, you should check the log, as
we will see shortly.
To make sure of the date of execution, click the View Details Button
Page 28 of 65
[4] Check the Date Started, Date Completed of the request, as follows:
Page 29 of 65
Page 30 of 65
Page 31 of 65
Page 32 of 65
Page 33 of 65
Page 34 of 65
Page 35 of 65
[3] Check the execution of the interface on the last night as follows:
Go to the line of the last nights execution:
Check the Two Columns Phase and Status.
The phase should be Completed and the status should be Normal. If status is Error, you should check the log, as
we will see shortly.
To make sure of the date of execution, click the View Details Button
Page 36 of 65
[4] Check the Date Started, Date Completed of the request, as follows:
Page 37 of 65
Page 38 of 65
Then you should check the output by pressing the View output button, as follows:
Page 39 of 65
[6] If there is any error code with any ticket you should solve this problem by reference to the error code and its
description at the end of the report
Page 40 of 65
Page 41 of 65
Page 42 of 65
[2] Query for the request whose name starts with INTEGRATION: Preventive, as follows:
Page 43 of 65
Page 44 of 65
[3] Check the execution of the interface on the last night as follows:
Go to the line of the last nights execution (the line just under the pending request line):
Check the Two Columns Phase and Status.
The phase should be Completed and the status should be Normal. If status is Error, you should check the log, as
we will see shortly.
To make sure of the date of execution, click the View Details Button
Page 45 of 65
Page 46 of 65
Page 47 of 65
[2] Query for the request whose name starts with INTEGRATION: Corrective, as follows:
Page 48 of 65
Page 49 of 65
[5] In case a work order of a bus is not sent, you can check the log, by pressing the View Log button, as follows:
Page 50 of 65
To add the mapping between the network point and the workshop, do the following:
[1] Under the Enterprise Asset Management Responsibility
Navigate to the Lookups window
Setup > Lookups
Page 51 of 65
Page 52 of 65
Page 53 of 65
The code fields contains the code of the fleet system network point code (whether its a depot or any other network
point)
The Meaning column contains the maintenance workshop (department) code
The Description column contains the name of place where the fleet network point exist.
Page 54 of 65
[3] Create a new record and add the data of the new mapping as follows:
Page 55 of 65
7. Backup process
To keep last up and running version of Stage DataBase users, the below schemas have
to be backup regularly.
Page 56 of 65
Page 57 of 65
Tip: To perform maintenance on a node without service interruption, disable the service process
on the node so that the service will fail over to a backup node.
8.2.1 Functionality
Oracle RAC allows multiple computers to run Oracle RDBMS software simultaneously while accessing a single
database, thus providing a clustered database.
In a non-RAC Oracle database, a single instance accesses a single database. Where the "database" consists of a
collection of data files, control files, and redo logs located on disk; the "instance" comprises the collection of Oraclerelated memory and operating system processes that run on a computer system.
In an Oracle RAC environment, two or more computers (each with an instance) concurrently access a single
database. This allows an application or user to connect to either computer and have access to a single coordinated
set of data.
8.2.2 Benefits
Since Oracle RAC allows multiple computers (instances) to access a single database simultaneously, it addresses
several areas of database management. These areas include:
fault tolerance
load balancing
scalability
8.2.3 Implementation
Oracle RAC depends on the infrastructure component Oracle Cluster ware to coordinate multiple servers and their
sharing of data storage.
Page 58 of 65
9. Appendix
Buffer memory
Buffer memory allocated to a session. The Integration Service uses buffer memory to move data from sources to
targets. The Integration Service divides buffer memory into buffer blocks.
Buffer memory size
Total buffer memory allocated to a session specified in bytes or as a percentage of total memory.
Concurrent workflow
A workflow configured to run multiple instances at the same time. When the Integration Service runs a concurrent
workflow, you can view the instance in the Workflow Monitor by the workflow name, instance name, or run ID.
High availability
A PowerCenter option that eliminates a single point of failure in a domain and provides minimal service interruption in
the event of failure.
Log Agent
A Service Manager function that provides accumulated log events from session and workflows. You can view session
and workflow logs in the Workflow Monitor. The Log Agent runs on the nodes where the Integration Service process
runs.
Log Manager
A Service Manager function that provides accumulated log events from each service in the domain. You can view logs
in the Administration Console. The Log Manager runs on the master gateway node.
Mapping
A set of source and target definitions linked by transformation objects that define the rules for data transformation.
Node
A logical representation of a machine or a blade. Each node runs a Service Manager that performs domain operations
on that node.
One-way mapping
A mapping that uses a web service client for the source. The Integration Service loads data to a target, often triggered
by a real-time event through a web service request.
PowerCenter resource
Any resource that may be required to run a task. PowerCenter has predefined resources and user-defined resources.
PowerCenter services
The services available in the PowerCenter domain. These consist of the Service Manager and the application
services. The application services include the Repository Service, Integration Service, Reporting Service, Metadata
Manager Service, Web Services Hub, and SAP BW Service.
Real-time data
Data that originates from a real-time source. Real-time data includes messages and messages queues, web services
messages, and changed source data.
Real-time processing
On-demand processing of data from operational data sources, databases, and data warehouses. Real-time
processing reads, processes, and writes data to targets continuously.
Real-time session
A session in which the Integration Service generates a real-time flush based on the flush latency configuration and all
transformations propagate the flush to the targets.
Page 59 of 65
Service workflow
A workflow that contains exactly one web service input message source and at most one type of web service output
message target. Configure service properties in the service workflow.
Session
A session is a set of instructions or a type of task that tells the Integration Service how and when to move data from
sources to targets.
Workflow
A set of instructions that tells the Integration Service how to run tasks such as sessions, email notifications, and shell
commands.
Workflow instance
The representation of a workflow. You can choose to run one or more workflow instances associated with a
concurrent workflow. When you run a concurrent workflow, you can run one instance multiple times concurrently, or
you can run multiple instances concurrently.
Workflow run ID
A number that identifies a workflow instance that has run.
Worklet
A worklet is an object representing a set of tasks created to reuse a set of workflow logic in multiple workflows.
BPEL
Business Process Execution Language for Web services is an XML-based language designed to enable task-sharing
for a distributed computing or grid computing environment - even across multiple organizations - using a combination
of Web services.
Page 60 of 65