Oracle Data Guard Operation Reference V1.0
Oracle Data Guard Operation Reference V1.0
IT Operation
Technical Documents
Setup Options
Start and stop replication
Monitoring Scripts and informational views
Basic Administration Tasks
Troubleshooting Scenarios
Setup Options
Oracle Data guard can be settled via various methods and option manual, via OEM, via Grid
Control, .. etc., we will consider only here manual options as it easier to debug and discover
errors if occurred, simply first you have to prepare the primary DB, then create the standby via
rman duplicate for standby command, or manual via regular backup and restore options.
Assumptions
You have two servers with an operating system and Oracle installed on them.
The primary server has a running database instance.
The standby server has a software only installation.
Oracle 11g used as example during the document and version differences mentioned
during document.
Logging
Check that the primary database is in archivelog mode, and if not set it on archivelog mode.
LOG_MODE
------------
NOARCHIVELOG
Initialization Parameters
Check the setting for the DB_NAME and DB_UNIQUE_NAME parameters. In this case they are
both set to "DB11G" on the primary database.
The DB_NAME of the standby database will be the same as that of the primary, but it must have
a different DB_UNIQUE_NAME value. The DB_UNIQUE_NAME values of the primary and
standby database should be used in the DG_CONFIG setting of the LOG_ARCHIVE_CONFIG
parameter. For this example, the standby database will have the value "DB11G_STBY".
Set suitable remote archive log destinations. In this case I'm using the fast recovery area for the
local location, but you could specify a location explicitly if you prefer. Notice the SERVICE and
the DB_UNIQUE_NAME for the remote location reference the standby location.
In addition to the previous setting, it is recommended to make sure the primary is ready to
switch roles to become a standby. For that to work properly we need to set the following
parameters. Adjust the *_CONVERT parameters to account for your filename and path
differences between the servers.
Remember, some of the parameters are not modifiable, so the database will need to be
restarted before they take effect.
Service Setup
Entries for the primary and standby databases are needed in the
"$ORACLE_HOME/network/admin/tnsnames.ora" files on both servers. You can create these
using the Network Configuration Utility (netca) or manually. The following entries were used
during this setup.
DB11G =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = ol5-112-dga1)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = DB11G.WORLD)
)
)
DB11G_STBY =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = ol5-112-dga2)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = DB11G.WORLD)
)
)
Listener Setup
This setup is important and should be typically performed on both servers, can be summarized
in flowing points
Use “netca” to add required service, or edit listener.ora file as following example
# Global Name, hostname listed with listener is different based on Server Role
#
SID_LIST_LISTENER =
(SID_LIST =
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = ol5-112-dga1.localdomain)(PORT =
1521))
)
(DESCRIPTION =
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
)
)
ADR_BASE_LISTENER = /u01/app/oracle
2. Start Listener
$ lsnrctl reload
or
$ lsnrctl stop
$ lsnrctl start
$ rman target=/
Amend the PFILE making the entries relevant for the standby database. I'm making a replica of
the original server, so in my case I only had to amend the following parameters.
*.db_unique_name='DB11G_STBY'
*.fal_server='DB11G'
*.log_archive_dest_2='SERVICE=db11g ASYNC
VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=DB11G'
$ mkdir -p /u01/app/oracle/oradata/DB11G
$ mkdir -p /u01/app/oracle/fast_recovery_area/DB11G
$ mkdir -p /u01/app/oracle/admin/DB11G/adump
Standby controlfile
Backups and archivelogs
Parameter files
Password file
$ # Parameter file.
$ scp oracle@ol5-112-dga1:/tmp/initDB11G_stby.ora /tmp/initDB11G_stby.ora
Notice, the backups were copied across to the standby server as part of the FRA copy. If your
backups are not held within the FRA, you must make sure you copy them to the standby server
and make them available from the same path as used on the primary server.
Start Listener
Make sure the listener setup performed in previous steps and started on the standby server.
$ lsnrctl start
$ lsnrctl status
Restore Backup
Create the SPFILE form the amended PFILE.
In addition to the online redo logs, you should create standby redo logs on both the standby and
the primary database (in case of switchovers). The standby redo logs should be at least as big as
Copy Files
Create the necessary directories on the standby server.
$ mkdir -p /u01/app/oracle/oradata/DB11G
$ mkdir -p /u01/app/oracle/fast_recovery_area/DB11G
$ mkdir -p /u01/app/oracle/admin/DB11G/adump
Standby controlfile
Parameter files
Password file
$ # Parameter file.
$ scp oracle@ol5-112-dga1:/tmp/initDB11G_stby.ora /tmp/initDB11G_stby.ora
Start Listener
When using active duplicate, the standby server requires static listener configuration in a
"listener.ora" file. In this case I used the following configuration.
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = ol5-112-dga2.localdomain)(PORT =
1521))
)
(DESCRIPTION =
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
)
)
ADR_BASE_LISTENER = /u01/app/oracle
$ lsnrctl start
$ lsnrctl status
$ export ORACLE_SID=DB11G
Connect to RMAN, specifying a full connect string for both the TARGET and AUXILLARY
instances. DO not attempt to use OS authentication.
RUN
{
allocate channel prmy1 type disk;
allocate channel prmy2 type disk;
allocate channel prmy3 type disk;
allocate channel prmy4 type disk;
allocate channel prmy5 type disk;
allocate channel prmy6 type disk;
allocate channel prmy7 type disk;
FOR STANDBY: This tells the DUPLICATE command is to be used for a standby, so it will
not force a DBID change.
FROM ACTIVE DATABASE: The DUPLICATE will be created directly from the source
datafile, without an additional backup step.
DORECOVER: The DUPLICATE will include the recovery step, bringing the standby up to
the current point in time.
SPFILE: Allows us to reset values in the spfile when it is copied from the source server.
NOFILENAMECHECK: Destination file locations are not checked.
*Note: if you use different file path don’t forget to set convert* option parameter.
# Background redo apply. Control is returned to the session once the apply
process is started.
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;
If you need to cancel the apply process, issue the following command.
If you prefer, you can set a delay between the arrival of the archived redo log and it being
applied on the standby server using the following commands.
Provided you have configured standby redo logs, you can start real-time apply using the
following command.
Create an Oracle Data Guard Broker configuration for the primary database.
You use the “dgmgrl” command to create the Oracle Data Guard Broker configuration. You
need to know the name of the Oracle Data Guard Broker configuration that you want to create,
the name of the primary database, and the net service name through which to connect. You will
If you find errors when you connect to the Oracle Data Guard Broker, check the
${ORACLE_HOME}/admin/sales/bdump/alert_prim_sid.log file. You can check that the
configuration has been created by using the following command:
Add the standby database to the Oracle Data Guard Broker configuration.
You need to know the name of the standby database, the net service name through which to
connect, and the type of standby (physical or logical).
To verify that the Oracle Data Guard Broker configuration is working correctly,
1. enable the configuration.
2. If you have successfully performed all steps, you can check the status of the
configuration by using the following command:
3. Verify that the Oracle Data Guard Broker configuration can switch over.
Database dismounted.
ORACLE instance shut down.
Operation requires shutdown of instance "salesdr1" on database "salesdr"
Shutting down instance "salesdr1"...
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance "sales1" on database "sales"
Starting instance "sales1"...
ORACLE instance started.
Database mounted.
Operation requires startup of instance "salesdr1" on database "salesdr"
Starting instance "salesdr1"...
ORACLE instance started.
Database mounted.
Switchover succeeded, new primary is "salesdr"
Database dismounted.
ORACLE instance shut down.
Operation requires shutdown of instance "sales1" on database "sales"
Shutting down instance "sales1"...
ORA-01109: database not open
Database dismounted.
Check the new archived redo log has arrived at the standby server and been applied.
Issue the following query to show information about the protection mode, the protection level,
the role of the database, and switchover status:
On the standby database, query the V$ARCHIVED_LOG view to identify existing files in the
archived redo log.
Or
SELECT THREAD#, MAX(SEQUENCE#) AS "LAST_APPLIED_LOG" FROM V$LOG_HISTORY GROUP
BY THREAD#;
On the standby database, query the V$ARCHIVED_LOG view to verify the archived redo log files
were applied.
The V$DATAGUARD_STATUS fixed view displays events that would typically be triggered by any
message to the alert log or server process trace files.
Determining Which Log Files Were Not Received by the Standby Site last day.
If a delayed apply has been specified or an archive log is missing then switchover may take
longer than expected.
Check v$managed_standby
OR alternatively:
The redo data generated by a transaction must have been received by every enabled
destination that has the SYNC attribute before that transaction can commit.
On primary databases and logical standbys, destinations 1 through 10 default to ASYNC
(real-time cascading).
On physical standbys, snapshot standbys, and far sync instances, destinations 1 through
10 default to ARCH transport mode. (Note that the ARCH attribute is deprecated; the use
of ARCH in this situation simply indicates non-real-time cascading.)
Note* Destinations 11 through 31 always default to ASYNC. (Only)
COMPRESSION
The COMPRESSION attribute is used to specify whether redo data is compressed before
transmission to a redo transport destination.
Usage Notes
The COMPRESSION attribute is optional. If it is not specified, the default compression behavior is
DISABLE.
*Note: For Oracle Data Guard SYNC connection strings that also use the Oracle Data Guard
COMPRESSION attribute, the SQLNET.COMPRESSION configuration parameter should be
disabled (set to off) in the sqlnet.ora file.
DELAY
The DELAY attribute specifies a minimum time lag between when redo data from the primary
site is archived on a standby site and when the archived redo log file is applied to the standby
database or any standbys cascaded from it.
Usage Notes
The DELAY attribute is optional. By default, there is no delay.
The DELAY attribute indicates the archived redo log files at the standby destination are not
available for recovery until the specified time interval has expired. The time interval is expressed
in minutes, and it starts when the redo data is successfully transmitted to, and archived at, the
standby site.
The DELAY attribute may be used to protect a standby database from corrupted or erroneous
primary data. However, there is a tradeoff because during failover it takes more time to apply all
the redo up to the point of corruption.
The DELAY attribute does not affect the transmittal of redo data to a standby destination.
If you have real-time apply enabled, then any delay that you set is ignored.
You can override the specified delay interval at the standby site, as follows:
The DELAY value that a cascaded standby use is the value that was set for the
LOG_ARCHIVE_DEST_n parameter on the primary that shipped the redo to the cascading
standby.
Examples:
You can use the DELAY attribute to set up a configuration where multiple standby databases are
maintained in varying degrees of synchronization with the primary database. However, this
protection incurs some overhead during failover, because it takes Redo Apply more time to
apply all the redo up to the corruption point.
For example, assume primary database A has standby databases B and C. Standby database B is
set up as the disaster recovery database and therefore has no time lag. Standby database C is
set up with a 2-hour delay, which is enough time to allow user errors to be discovered before
they are propagated to the standby database.
The following example shows how to specify the DELAY attribute for this configuration:
LOG_ARCHIVE_DEST_1='LOCATION=/arch/dest MANDATORY'
LOG_ARCHIVE_DEST_STATE_1=ENABLE
LOG_ARCHIVE_DEST_2='SERVICE=stbyB SYNC AFFIRM'
LOG_ARCHIVE_DEST_STATE_2=ENABLE
LOG_ARCHIVE_DEST_3='SERVICE=stbyC DELAY=120'
LOG_ARCHIVE_DEST_STATE_3=ENABLE
Note: You should compare several serial and parallel recovery runs to determine optimal
recovery performance
SHUTDOWN IMMEDIATE;
STARTUP MOUNT;
ALTER DATABASE OPEN READ ONLY;
In 11g, Oracle introduced the Active Data Guard feature. This allows the standby database to be
open in read-only mode, but still apply redo information. This means a standby can be available
for querying, yet still be up to date. There are licensing implications for this feature, but the
following commands show how active data guard can be enabled.
SHUTDOWN IMMEDIATE;
STARTUP MOUNT;
ALTER DATABASE OPEN READ ONLY;
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;
Since managed recovery continues with active data guard, there is no need to switch back to
managed recovery from read-only mode in this case.
By default, for a newly created standby database, the primary database is in maximum
performance mode.
PROTECTION_MODE
--------------------
MAXIMUM PERFORMANCE
The mode can be switched using the following commands. Note the alterations in the redo
transport attributes.
-- Maximum Availability.
-- Maximum Performance.
> ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='SERVICE=db11g_stby NOAFFIRM ASYNC
VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=DB11G_STBY';
> ALTER DATABASE SET STANDBY DATABASE TO MAXIMIZE PERFORMANCE;
-- Maximum Protection.
> ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='SERVICE=db11g_stby AFFIRM SYNC
VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=DB11G_STBY';
SHUTDOWN IMMEDIATE;
STARTUP MOUNT;
> ALTER DATABASE SET STANDBY DATABASE TO MAXIMIZE PROTECTION;
Database Switchover
A database can be in one of two mutually exclusive modes (primary or standby). These roles can
be altered at runtime without loss of data or resetting of redo logs. This process is known as a
Switchover and can be performed using the following statements.
Once this is complete, test the log transport as before. If everything is working fine, switch the
primary database back to the original server by doing another switchover. This is known as a
switchback.
Since the standby database is now the primary database it should be backed up immediately.
The original primary database can now be configured as a standby. then, the whole setup
process must be followed, but this time using the original primary server as the standby before
switch back.
Flashback Database
It was already mentioned in the previous section, but it is worth drawing your attention to
Flashback Database once more. Although a switchover/switchback is safe for both the primary
and standby database, a failover renders the original primary database useless for converting to
a standby database. If flashback database is not enabled, the original primary must be scrapped
and recreated as a standby database.
An alternative is to enable flashback database on the primary (and the standby if desired) so in
the event of a failover, the primary can be flashed back to the time before the failover and
quickly converted to a standby database. That process is shown here.
1. Pre-configuration steps
2. Enable flashback
db_recovery_file_dest
db_recovery_file_dest_size
db_flashback_retention_target
do following steps:
on standby
on standby
on standby
on primary
on standby
If you are using RAC, turn off all but one of the RAC instances. Make sure the instance is in
MOUNT mode.
SHUTDOWN IMMEDIATE;
STARTUP MOUNT;
Convert the standby to a snapshot standby. The following example queries the V$DATABASE
view to show that flashback database is not enabled prior to the conversion operation.
FLASHBACK_ON
------------------
NO
FLASHBACK_ON
------------------
RESTORE POINT ONLY
You can now do treat the standby like any read-write database.
To convert it back to the physical standby, losing all the changes made since the conversion to
snapshot standby, issue the following commands.
SHUTDOWN IMMEDIATE;
STARTUP MOUNT;
ALTER DATABASE CONVERT TO PHYSICAL STANDBY;
SHUTDOWN IMMEDIATE;
STARTUP NOMOUNT;
ALTER DATABASE MOUNT STANDBY DATABASE;
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT;
SELECT flashback_on FROM v$database;
FLASHBACK_ON
------------------
NO
The standby is once again in managed recovery and archivelog shipping is resumed. Notice that
flashback database is still not enabled.
Connect to the primary (cdb1) database and convert the standby database (cdb1_stby) to a
snapshot standby.
$ dgmgrl sys/Password1@cdb1
DGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production
When you are finished with the snapshot standby, convert it back to a standby database.
$ dgmgrl sys/Password1@cdb1
DGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production
The standby is once again in managed recovery and archivelog shipping is resumed. Notice that
flashback database is still not enabled.
Configuration - my_dg_config
Configuration Status:
SUCCESS (status updated 38 seconds ago)
DGMGRL>
Note:
If you use 'DataGuard Broker', then it should be stopped before starting with step 1 and re-
started again when all steps finished.
disable configuration;
Note: For the remaining steps, the standby database must be in a MOUNT state.
You need to use the 'lowest SCN' from the queries below:
CURRENT_SCN
--------------
169363633809
MIN(F.FHSCN)
----------------
169363632617
in this example is SCN: 3162298. Therefore, From the above you need to backup from SCN
3162298
Use RMAN, connect to the primary database and create an incremental backup from the SCN
derived in the previous step:
RMAN> run
{
ALLOCATE CHANNEL ch1 DEVICE TYPE DISK FORMAT '/u02/backup/ForStandby_%U';
BACKUP INCREMENTAL FROM SCN 169363632617 DATABASE FORMAT
'/u02/backup/ForStandby_%U' tag 'FORSTANDBY';
All backups created by step #3 must be transferred from the primary to the standby system. If
the backup is written to NFS device, this step can be skipped.
Do you really want to catalog the above files (enter YES or NO)? YES
cataloging files...
cataloging done
7. In RMAN, connect to the PRIMARY database and create a standby control file backup:
RMAN> run
{
ALLOCATE CHANNEL ch1 DEVICE TYPE DISK FORMAT '/u02/backup/ForStandby_%U';
BACKUP CURRENT CONTROLFILE FOR STANDBY FORMAT
'/u02/backup/ForStandbyCTRL.bck';
}
spool datafile_names_step8.txt
set lines 200
col name format a60
select file#, name from v$datafile order by file#;
spool off
10. From RMAN, connect to STANDBY database and restore the standby control file:
RMAN> SHUTDOWN;
RMAN> STARTUP MOUNT;
Since the controlfile is restored from PRIMARY the datafile locations in STANDBY controlfile will
be same as PRIMARY database, if the directory structure is different between the standby and
primary or you are using Oracle managed file names, catalog the datafiles in STANDBY will do
the necessary rename operations. If the primary and standby have identical structure and
datafile names, this step can be skipped.
Perform the below step in STANDBY for each diskgroup where the datafile directory structure
between primary and standby are different.
Do you really want to catalog the above files (enter YES or NO)? YES
cataloging files...
cataloging done
If any datafiles have been added to Primary AFTER scn 3162298 they will also have to be
restored to the standby host (see Note 1531031.1 Steps to perform for Rolling forward a
standby database using RMAN incremental backup when datafile is added to primary) and
cataloged as shown above before doing the switch.
To determine if any files have been added to Primary since the standby current SCN:
If the above query returns with 0 zero rows, you can switch the datafiles. This will rename the
datafiles to its correct name at the standby site:
Note:
a) At this point, you can compare the query output from step 9 for any discrepancy (other
than newly added datafiles) to ensure we have all the datafiles added in standby.
b) run the queries from step 2 again to confirm that the incremental apply has moved the
datafiles forward. The SCN should now be bigger than its initial values.
13. Configure the STANDBY database to use flashback (optional)
If the STANDBY database needs to be configured for FLASHBACK run the following commands:
Test configuration
enable configuration;
if you got the below error
DGMGRL> enable configuration;
ORA-16541: database is not enabled
Configuration details cannot be determined by DGMGRL
On Primary
> DGMGRL
DGMGRL> remove configuration;
DGMGRL> CREATE CONFIGURATION TB02ADM_DG AS PRIMARY DATABASE IS TB02ADM CONNECT
IDENTIFIER IS TB02ADM;
DGMGRL> show configuration;
DGMGRL> add database TB02ADM_SAH as connect identifier is TB02ADM_SAH;
DGMGRL> enable configuration;