Oracle GoldenGate

Overview & Demo / Instructions

Background, Overview of GG, Next Steps
etc.
 Agenda

• Oracle Golden Gate

– Overview
– Deep-dive Architecture
• Demo

November 2010 2
Overview of Oracle Golden Gate

• Oracle GoldenGate provides real-time data integration to support high

availability capabilities across heterogeneous systems to achieve
business continuity.

• GoldenGate provides the ability to capture and deliver updates of

critical data as the changes occur and provide continuous synchronized
data across heterogeneous environments.

• Use Cases: Online banking, Claims processing and Online shopping

can use GoldenGate to operate without any disruption.

November 2010 3
Overview of Golden Gate … Contd.
Provides
Low-impact capture
Routing Zero Downtime
Transformation and Migration and New DB/
Delivery of transactional data across Upgrades
OS/HW/App
Heterogeneous environments in
Real time
Active-Active High Fully Active
Availability Distributed Database

Key Features Log Based, Real-

Time Change Data
Capture Query Offloading
Reporting
Database
Non-intrusive, low-impact, GoldenGate
Performance
sub-second latency
ETL

ODS EDW
ETL
Open, modular architecture
Flexible /
- Supports heterogeneous
Extensible Real-time BI
sources and targets Heterogeneous EDW
Source Systems

Reliable Maintains transactional Data Distribution

Global Data Centers
integrity - Resilient against
interruptions and failures

SOA/EDA

November 2010 4
Oracle GoldenGate
Capture: Committed transactions are captured (and can be filtered) real time, from
the DB Redo logs.
Trail: stages and queues data for routing.
Pump: distributes data for routing to target(s).
Route: data is compressed,
encrypted for routing to target(s).
Delivery: applies data with transaction
integrity, transforming the data as required.

Trail Trail
Capture Pump Delivery
LAN/WAN
Internet
TCP/IP

Source Target
Oracle & Non-Oracle Oracle & Non-Or
Database(s) Database(s)

November 2010 5
Oracle GoldenGate
Capture: Committed transactions are captured (and can be filtered) real time, from
the DB Redo logs.
Trail: stages and queues data for routing.
Pump: distributes data for routing to target(s).
Route: data is compressed,
encrypted for routing to target(s).
Delivery: applies data with transaction
integrity, transforming the data as required.

Trail Trail
Capture Pump Delivery (SQL)
LAN/WAN
Internet
TCP/IP

Source Target
Oracle & Non-Oracle Bi-directional Database(s)
Database(s)

November 2010 6
GoldenGate Deployment Topologies
Unidirectional Bi-Directional Peer-to-Peer
Query Offloading Hot Standby or Load Balancing,
Zero-Downtime Migration Active-Active for HA Multi-Master

Broadcast /Fan-out Integration/Consolidation/Fan-in Data Distribution

Data Distribution Data Warehouse via Messaging

BPM

November 2010 7
Architecture Components (Processes)
CAPTURE DELIVERY
• Log-based • Native database API
• VAM-based • ODBC
• Local/Remote Queuing • Batched Operations
• Filtering • Filtering
• Parallel Coordination • Transaction/Operations Grouping
• Local/Remote • Parallel Coordination
• Local/Remote

PUMP (routes from one side to other) MANAGER (Start/Stop/Report on thresholds, trail purging etc.)
• Filtering • Queue (Trail) Management
• Local/Remote • Process Management (Dynamic)
• TCP/IP • Monitoring and Administration (Lag reports)
• Compression • Reporting (Events, Errors, Thresholds)
• Encryption • Receive/Route requests from UI

On-disk Components
• Trail Files
• Checkpoint Files (Capture, Pump, Delivery)
• Data Definition Files (SOURCEDEFS/TARGETDEFS)
• Configuration Files
• Discard Files
• Report/Log Files
• Executables

November 2010 8
Oracle GoldenGate Components
Communications and Startup

GGSCI GGSCI
Interactive Manager Manager Interactive
Interface Interface

Network

Source Trail Target Trail

Source
Source Capture Pump Collector Delivery Target
Database
Target
Target

TCP/IP
Process Start
Shared Memory (for Monitoring)

November 2010 9
 GoldenGate Capture
GoldenGate
GoldenGate
Event Log
Parameter File Messaging
Messaging Subsystem
Subsystem

Checkpoint
Checkpoint Manager
Manager
Parser
Parser

Memory
Memory Manager
Manager
GoldenGate
Transaction Checkpoint File
Logs

Redo
Redo API
API
Transaction
Transaction Transformation
Transformation Serialization
Serialization
(Log
(Log reader
reader Formatting
Formatting Filtering
Filtering
Log
Log Reader
Reader and
and Mapping
Mapping and
and Routing
Routing
interface)
interface)

GoldenGate
Metadata Metadata SQL Transformation Trail Files
Metadata Metadata SQL Transformation
Manager
Manager Manager
Manager Interface
Interface Engine
Engine

Database
Database Access
Access –– To
To gather
gather Metadata
Metadata (to
(to interpret
interpret log
log data)
data) && for
for Command
Command
SQL
SQL interface
interface for
for data
data lookup,
lookup, advanced
advanced filtering
filtering etc.
etc. and
and Control
Control

Capture
Capture GoldenGate
Source
Command
Database
Interpreter

November 2010 10
GoldenGate Delivery
GoldenGate
GoldenGate
Event Log
Parameter File Messaging
Messaging Subsystem
Subsystem

Checkpoint
Checkpoint Manager
Manager
Parser
Parser

Memory
Memory Manager
Manager
GoldenGate
Checkpoint File

GoldenGate
GoldenGate Transformation
Transformation SQL
SQL
Read
Read API
API Formatting
Formatting Filtering
Filtering
Trail
Trail Reader
Reader and
and Mapping
Mapping Generation
Generation

GoldenGate
Trail Files
Metadata
Metadata SQL
SQL Transformation
Transformation
Manager
Manager Interface
Interface Engine
Engine

Command
Command
and
and Control
Control
Database
Database Access
Access –– For
For Metadata
Metadata access
access && SQL
SQL interface
interface

GoldenGate Delivery
Delivery Target
Command
Database
Interpreter

November 2010 11
GoldenGate Instantiation
• Start delivery at or after given backup CSN (245 in this case)
– Current source Commit Sequence Number (CSN) is 356
• Once Delivery is current the source and target are in sync

Consistent as
Current CSN is Deliver of CSN 356
356 CSNs 222 transactions In Sync!
through 356 after CSN 245

Commit Ordered Commit Ordered

Capture Source Trail Pump Target Trail Delivery
Source Target
Database Database

Take Backup Apply Backup

as of CSN as of CSN
245 245

November 2010 12
Initial Data Load
Source:
• Create the Initial data extract process ‘load1′
– ADD EXTRACT load1, SOURCEISTABLE
– Create Parameter file for Load1: Edit params load1

Target:
• Create initial data load task ‘load2’
– ADD REPLICAT load2, SPECIALRUN
– Create Parameter file for Load2: Edit params load2

• Start Initial Load:

– START EXTRACT load1
– INFO EXTRACT load1

November 2010 13
Online Change Synchronization
1. Create a GoldenGate Checkpoint table
• EDIT PARAMS ./GLOBALS
– GGSCHEMA GGS_OWNER
CHECKPOINTTABLE GGS_OWNER.CHKPTAB

• DBLOGIN USERID ggs_owner, PASSWORD ggs_owner

ADD CHECKPOINTTABLE GGS_OWNER.CHKPTAB
2. Create an Extract groups on Source
• ADD EXTRACT ext1, TRANLOG, BEGIN NOW
• ADD EXTRACT dpump, EXTTRAILSOURCE /ora/app/stage/GoldenGate-Zip/ggs/dirdat/lt

3. Create a parameter file for the online Extract group

• EDIT PARAMS ext1
4. Create a Trail on Source & Target
• On the source/target system, the trail files will be created having a prefix ‘rt’/’lt’ which will be used by the Replicat/Extract
processes also running on the target /source system(s)
• ADD EXTTRAIL /ora/app/stage/GoldenGate-Zip/ggs/dirdat/lt, EXTRACT EXT1
• ADD RMTTRAIL /ora/app/stage/GoldenGate-Zip/ggs/dirdat/rt, EXTRACT dpump

5. Create a Replicat group on the Target

• ADD REPLICAT rep1, exttrail /ora/app/stage/GoldenGate-Zip/ggs/dirdat/rt, checkpointtable chkptab
6. Create a parameter file for the online Replicat group
• EDIT PARAMS rep1

Start Online Change Sync:

• START EXTRACT ext1
• START EXTRACT dpump
• START REPLICAT rep1

November 2010 14
Significance of Data Pump
• When Data Pump is not used, the Extract process writes to a remote trail that is located
on the target system using TCP/IP
• When Data Pump is configured, Extract process writes to a local trail for Data Pump to
read and write over the network to the remote trail located on the target system
• The advantages of Data Pump can be seen as it protects against a network failure as in
the absence of a storage device on the local system, the Extract process writes data into
memory before the same is sent over the network
• Any failures in the network could then cause the Extract process to abort (“abend”)
• Additionally, if there are any complex data transformations or filtering, Data Pump can be
used to perform these

November 2010 15