SR Detail Página 1 de 7

SR 3-9813614841 : Problem with DataGuard replication

Severity 3-Standard Status Closed
Escalation Status Never Escalated Opened 31-Oct-2014 11:28 (Friday)
Last Updated 04-Nov-2014 12:20 (19+ hours ago)
alertgenp.zip, alertlogs.zip,
new_dg_prim_diag_genp_Oct31_0424.html,
Bug Reference No Related Bugs Attachments
new_dg_psby_diag_GEAG_Oct31_0425.html,
Output_querys.log
Related Articles 1577401.1, 1577406.1 Related SRs No Related SRs
Support Identifier 10406457
Account Name IBERDROLA SISTEMAS, S.A.
Primary Contact DBA DBA Alternate Contact
System Host No Related Hosts
Product Oracle Database - Enterprise Edition
Product Version 11.2.0.3
Operating System Microsoft Windows Itanium (64-bit) OS Version 2008 R2
Hi,

We have a problem with a Oracle RAC on 11.2.0.3 because we have got a Dataguard but since one week
it didn't replicate the archives.
We launch the command "ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;" but it
finished by timeout.
The dataguard looks like ok:
DGMGRL> show configuration
Configuration - dg_genp
Protection Mode: MaxPerformance
Databases:
Problem Description
genp - Primary database
geag - Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

Could you help us ASAP?

Regards,

Javi Maillo

History
ODM Knowledge Content Oracle Support- 04-Nov-2014 12:20 (19+ hours ago)
Knowledge Review completed
Notes Oracle Support- 04-Nov-2014 12:17 (19+ hours ago)
Hi Sir

You're quite welcome and thank you for your update.

I am glad to know that the issue is resolved.... it's really a pleasure working with you!

Thank you for the opportunity to assist you.

Have a nice day

Thank you ,
Mohamed ElKady
Engineer Shift time: 12:30pm to 09:00pm (IST) | 07:00am to 3:30pm (GMT) | 12:00am to 08:30am (PT) - Mondays to Fridays
Global Customer Services
ODM Answer Oracle Support- 04-Nov-2014 12:16 (19+ hours ago)
Could you please restart the standby database
ODM Question Oracle Support- 04-Nov-2014 12:16 (19+ hours ago)
We have a problem with a Oracle RAC on 11.2.0.3 because we have got a Dataguard but since one week it didn't replicate the
archives.
We launch the command "ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;" but it finished by timeout.
The dataguard looks like ok:
DGMGRL> show configuration
Configuration - dg_genp
Protection Mode: MaxPerformance
Databases:
genp - Primary database

https://support.oracle.com/epmos/faces/SrDetailPrint?sysmsg=true&_afrLoop=34448... 05/11/2014
SR Detail Página 2 de 7

geag - Physical standby database

Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS
Update from Customer [email protected] - 04-Nov-2014 09:58 (22+ hours
ago)
Hi,

The restart worked!!! thanks for your help. You can close the incident.

Thanks for all again and kind regards,

Javi Maillo
ODM Action Plan Oracle Support- 03-Nov-2014 16:27 (Monday)
Hi Sir,

Thanks for your update, Could you please restart the standby database

Thank you,
Mohamed ElKady
Engineer Shift time: 12:30pm to 09:00pm (IST) | 07:00am to 3:30pm (GMT) | 12:00am to 08:30am (PT) - Mondays to Fridays
Global Customer Services
ODM Data Collection Oracle Support- 03-Nov-2014 15:24 (Monday)
SQL> -- Verify the last sequence# received and the last sequence# applied to standby database.
SQL>
SQL> SELECT al.thrd "Thread", almax "Last Seq Received", lhmax "Last Seq Applied" FROM (select thread# thrd, MAX
(sequence#) almax FROM v$archived_log WHERE resetlogs_change#=(SELECT resetlogs_change# FROM v$database) GROUP
BY thread#) al, (SELECT thread# thrd, MAX(sequence#) lhmax FROM v$log_history WHERE resetlogs_change#=(SELECT
resetlogs_change# FROM v$database) GROUP BY thread#) lh WHERE al.thrd = lh.thrd;

Thread Last Seq Received Last Seq Applied

1 877 853
2 847 823

2 rows selected.

STANDBY
************
SQL> select thread#, max(sequence#) "Last Standby Seq Received"
2 from v$archived_log
3 where first_time between (sysdate-7) and (sysdate+1)
4 group by thread# order by 1;

THREAD# Last Standby Seq Received

---------- -------------------------
1 889
2 859

SQL> select thread#, max(sequence#) "Last Standby Seq Applied"

2 from v$archived_log where applied='YES'
3 and first_time between (sysdate-7) and (sysdate+1)
4 group by thread# order by 1;

no rows selected

SQL> SELECT al.thrd "Thread", almax "Last Seq Received", lhmax "Last Seq Applied" FROM (select thread# thrd, MAX
(sequence#) almax FROM v$archived_log WHERE re
WHERE resetlogs_change#=(SELECT resetlogs_change# FROM v$database) GROUP BY thread#) lh WHERE al.thrd = lh.thrd;

Thread Last Seq Received Last Seq Applied

---------- ----------------- ----------------
2 859 823
1 889 853

SQL>
SQL>
SQL> SELECT thread#, process, pid, status, client_process, client_pid, sequence#, block#, active_agents, known_agents FROM
gv$managed_standby ORDER BY thread#

THREAD# PROCESS PID STATUS CLIENT_P

https://support.oracle.com/epmos/faces/SrDetailPrint?sysmsg=true&_afrLoop=34448... 05/11/2014
SR Detail Página 3 de 7

---------- --------- ---------- ------------ --------

CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------
KNOWN_AGENTS
------------
0 ARCH 109012 CONNECTED ARCH
109012 0 0 0
0

0 ARCH 4640 CONNECTED ARCH

4640 0 0 0
0

THREAD# PROCESS PID STATUS CLIENT_P

---------- --------- ---------- ------------ --------
CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------
KNOWN_AGENTS
------------

0 RFS 25132 IDLE UNKNOWN

8212 0 0 0
0

0 RFS 77508 IDLE ARCH

96252 0 0 0

THREAD# PROCESS PID STATUS CLIENT_P

---------- --------- ---------- ------------ --------
CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------
KNOWN_AGENTS
------------
0

0 RFS 57860 IDLE UNKNOWN

32588 0 0 0
0

0 RFS 24916 IDLE UNKNOWN

THREAD# PROCESS PID STATUS CLIENT_P

---------- --------- ---------- ------------ --------
CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------
KNOWN_AGENTS
------------
7060 0 0 0
0

0 RFS 25164 IDLE ARCH

8396 0 0 0
0

THREAD# PROCESS PID STATUS CLIENT_P

---------- --------- ---------- ------------ --------
CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------
KNOWN_AGENTS
------------
0 RFS 22860 IDLE UNKNOWN
70380 0 0 0
0

1 ARCH 88352 CLOSING ARCH

88352 888 10240 0
0

THREAD# PROCESS PID STATUS CLIENT_P

---------- --------- ---------- ------------ --------
CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------

https://support.oracle.com/epmos/faces/SrDetailPrint?sysmsg=true&_afrLoop=34448... 05/11/2014
SR Detail Página 4 de 7

KNOWN_AGENTS
------------

1 ARCH 95072 CLOSING ARCH

95072 887 67584 0
0

1 ARCH 19700 CLOSING ARCH

19700 889 1 0

THREAD# PROCESS PID STATUS CLIENT_P

---------- --------- ---------- ------------ --------
CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------
KNOWN_AGENTS
------------
0

1 RFS 31276 IDLE LGWR

73784 890 18248 0
0

2 ARCH 6480 CLOSING ARCH

THREAD# PROCESS PID STATUS CLIENT_P

---------- --------- ---------- ------------ --------
CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------
KNOWN_AGENTS
------------
6480 859 1 0
0

2 ARCH 5684 CLOSING ARCH

5684 858 61440 0
0

THREAD# PROCESS PID STATUS CLIENT_P

---------- --------- ---------- ------------ --------
CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------
KNOWN_AGENTS
------------
2 ARCH 4188 CLOSING ARCH
4188 857 176128 0
0

2 MRP0 30408 APPLYING_LOG N/A

N/A 824 24033 13
13

THREAD# PROCESS PID STATUS CLIENT_P

---------- --------- ---------- ------------ --------
CLIENT_PID SEQUENCE# BLOCK# ACTIVE_AGENTS
---------------------------------------- ---------- ---------- -------------
KNOWN_AGENTS
------------

2 RFS 17056 IDLE LGWR

7352 860 27300 0
0

17 rows selected.
Update from Customer [email protected] 03-Nov-2014 13:25 (Monday)
Hi Mohamed,

Thanks for your reply, I attached a file with the output.

I don't understand what do you mean with rfs,arc and mrp?

Regards,

https://support.oracle.com/epmos/faces/SrDetailPrint?sysmsg=true&_afrLoop=34448... 05/11/2014
SR Detail Página 5 de 7

Javi Maillo
Update from Customer [email protected] 03-Nov-2014 13:24 (Monday)
Upload to TDS successful for the file Output_querys.log.
Notes Oracle Support- 03-Nov-2014 12:38 (Monday)
Hi Sir,

Thanks for your update , Please upload the output of the following :

Primary: SQL > select thread#, max(sequence#) "Last Primary Seq Generated"
from v$archived_log
where first_time between (sysdate-7) and (sysdate+1)
group by thread# order by 1;

PhyStdby: SQL > select thread#, max(sequence#) "Last Standby Seq Received"
from v$archived_log
were first_time between (sysdate-7) and (sysdate+1)
group by thread# order by 1;

PhyStdby: SQL > select thread#, max(sequence#) "Last Standby Seq Applied"
from v$archived_log where applied='YES'
and first_time between (sysdate-7) and (sysdate+1)
group by thread# order by 1;

SELECT al.thrd "Thread", almax "Last Seq Received", lhmax "Last Seq Applied" FROM (select thread# thrd, MAX(sequence#)
almax FROM v$archived_log WHERE resetlogs_change#=(SELECT resetlogs_change# FROM v$database) GROUP BY thread#) al,
(SELECT thread# thrd, MAX(sequence#) lhmax FROM v$log_history WHERE resetlogs_change#=(SELECT resetlogs_change#
FROM v$database) GROUP BY thread#) lh WHERE al.thrd = lh.thrd;

SELECT thread#, process, pid, status, client_process, client_pid, sequence#, block#, active_agents, known_agents FROM
gv$managed_standby ORDER BY thread#, process;

Also please check for rfs,arc traces at primary and mrp and its slave traces at standby if generated any around the time of the
issue

Thank you,
Mohamed ElKady
Engineer Shift time: 12:30pm to 09:00pm (IST) | 07:00am to 3:30pm (GMT) | 12:00am to 08:30am (PT) - Mondays to Fridays
Global Customer Services
Update from Customer [email protected] 03-Nov-2014 11:14 (Monday)
The customer is pressing our, please could you say us any news?
Update from Customer [email protected] 31-Oct-2014 16:28 (Friday)
Upload to TDS successful for the file alertgenp.zip.
Update from Customer [email protected] 31-Oct-2014 16:27 (Friday)
Upload to TDS successful for the file new_dg_psby_diag_GEAG_Oct31_0425.html.
Update from Customer [email protected] 31-Oct-2014 16:27 (Friday)
Upload to TDS successful for the file new_dg_prim_diag_genp_Oct31_0424.html.
Update from Customer [email protected] 31-Oct-2014 16:27 (Friday)
Uploaded files
ODM Action Plan Oracle Support- 31-Oct-2014 15:22 (Friday)
HI Sir,

Thanks for using My Oracle Support.

1.Please upload the alert.log from the primary database.

2.Please upload the output of :

- Note 1577401.1 Script to Collect Data Guard Primary Site Diagnostic Information
- Note 1577406.1 Script to Collect Data Guard Physical Standby Diagnostic Information

Thank you,
Mohamed ElKady
Engineer Shift time: 12:30pm to 09:00pm (IST) | 07:00am to 3:30pm (GMT) | 12:00am to 08:30am (PT) - Mondays to Fridays
Global Customer Services

https://support.oracle.com/epmos/faces/SrDetailPrint?sysmsg=true&_afrLoop=34448... 05/11/2014
SR Detail Página 6 de 7

Notes Oracle Support- 31-Oct-2014 14:22 (Friday)

HI Sir,

Your issue was transferred to me. My name is Mohamed from Data Guard team and I will be helping you with your issue.

Thank you for your patience,

Mohamed ElKady
Engineer Shift time: 12:30pm to 09:00pm (IST) | 07:00am to 3:30pm (GMT) | 12:00am to 08:30am (PT) - Mondays to Fridays
Global Customer Services
Update from Customer [email protected] 31-Oct-2014 11:28 (Friday)
Upload to TDS successful for the file alertlogs.zip.
Customer Problem Description [email protected] 31-Oct-2014 11:28 (Friday)
Customer Problem Description
---------------------------------------------------

Problem Summary
---------------------------------------------------
Problem with DataGuard replication

Problem Description
---------------------------------------------------
Hi,

We have a problem with a Oracle RAC on 11.2.0.3 because we have got a Dataguard but since one week it didn't replicate the
archives.
We launch the command "ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;" but it finished by timeout.
The dataguard looks like ok:
DGMGRL> show configuration
Configuration - dg_genp
Protection Mode: MaxPerformance
Databases:
genp - Primary database
geag - Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

Could you help us ASAP?

Regards,

Javi Maillo

Error Codes
---------------------------------------------------

Problem Category/Subcategory
---------------------------------------------------
Database Backup/Recovery and RMAN, Duplicate and DataGuard StandBy setup, GDS, OSB/Data Guard (Create Standby, Redo
Transport & Apply, Role Changes, Broker, etc)

Uploaded Files
---------------------------------------------------
File: alertlogs.zip

Interactive Solution Details

---------------------------------------------------
Question: In which area of Oracle Data Guard do you have a problem or question ?
Answer: Physical standby database

Question: My issue with physical standby is in

Answer: Active Data Guard and real-time apply

Diagnosis: Review the following documents to troubleshoot this issue:

Read Note:828274.1 Data Guard Real-Time Apply FAQ

Read Note:1672293.1 MAA Whitepaper: Active Data Guard 11g Best Practices (includes best practices for Redo Apply)

If the above solution does not resolve your issue, follow the steps detailed in the document below:

https://support.oracle.com/epmos/faces/SrDetailPrint?sysmsg=true&_afrLoop=34448... 05/11/2014
SR Detail Página 7 de 7

Read Note:1910064.1 SRDC - Collect Physical Standby Database Information

We recommend saving this session as a draft Service Request (SR). Once data gathering is complete, proceed with the next step
to upload requested data and submit SR.

https://support.oracle.com/epmos/faces/SrDetailPrint?sysmsg=true&_afrLoop=34448... 05/11/2014