Sandesh_Rao_Navigating Oracle Troubleshooting- AHF Insights for Database 23ai - AIOUG July 2024.pdf
0 likes86 views
Navigating Oracle Troubleshooting: AHF Insights for Database 23ai
![§ For troubleshooting one needs to know :
§ What type of system does user have ?
§ What’s going on around the time of issue ?
§ Can I get a full picture across all nodes ?
§ Can I zoom into specific timeframe ?
§ Can I look at the data from various perspectives ?
Customer Complains of “Grid failure - CRS-8503 []” in SR](https://image.slidesharecdn.com/sandeshraonavigatingoracletroubleshooting-ahfinsightsfordatabase23ai-aiougjuly2024-241119052650-596b7836/85/Sandesh_Rao_Navigating-Oracle-Troubleshooting-AHF-Insights-for-Database-23ai-AIOUG-July-2024-pdf-32-320.jpg)
Ad
More Related Content
Similar to Sandesh_Rao_Navigating Oracle Troubleshooting- AHF Insights for Database 23ai - AIOUG July 2024.pdf (20)
More from Sandesh Rao (20)
Ad
Recently uploaded (20)
Ad
Sandesh_Rao_Navigating Oracle Troubleshooting- AHF Insights for Database 23ai - AIOUG July 2024.pdf
- 1. A unified reporting & correlation tool for AHF Stack Navigating Oracle Troubleshooting: AHF Insights for Database 23ai Sandesh Rao VP AIOps and Machine Learning , Autonomous Database @sandeshr https://www.linkedin.com/in/raosandesh/ https://www.slideshare.net/SandeshRao4
- 2. AHF Stack
- 3. § Purpose § Provides bird's eye view of the system from diagnostic perspective. § Offers insights for effective issue resolution with guidance & co-relation. § Unifies the AHF stack under a single user interface § Usage § Reactive, Proactive. § Target Users § Customers, Operations, Support, Development § To generate AHF Insights, run : § Collected by Default in AHF Collections from AHF 24.2 AHF Insights
- 4. AHF Insights (over command-line)
- 5. AHF Insights as part of AHF collection
- 6. AHF Insights as part of AHF collection
- 7. § All Manual Collections § This is included in all AHF manual collections since 24.2 § SRDCs (Subset) Automatic Insights Report with AHF collection § dbinstancecrash § dbunixresources § crsdbhangperf § dbhangperflite § dbperf § dbrac § dbracperf § exadata § dbdataguard § dbhangperflite_auto
- 10. Need configurations details while troubleshooting an issue
- 11. § BareMetal system (DomU has access to query storage server and fabric switch details) § DomU has limited access Configuration Details
- 12. Cluster Details
- 13. ASM Details
- 14. Database Details
- 19. Troubleshoot : Issue due to inconsistent RPMs
- 20. § Example Scenario - Inconsistent glibc Package Versions § Issue - Node Eviction and Clusterware Instability § Example § In a four-node Oracle RAC cluster, nodes 1 and 2 have glibc version 2.17-307 installed, while nodes 3 and 4 have glibc version 2.17-307.el7.1 installed. This discrepancy can cause several problems. § Impact § Node Eviction - Due to the different versions of glibc, nodes 3 and 4 might face eviction as the clusterware detects inconsistencies in the environment. § Clusterware Instability - The inconsistency in glibc can cause instability in Oracle Clusterware, leading to startup failures and communication errors. Issue due to inconsistent RPMs
- 22. Troubleshoot : Issue due to software version lower than MAA Software Recommendations
- 23. Recommended Software All software should be updated regularly. Maintaining software at current or recent releases provides the following benefits: § Better software security § More stable maintenance releases § Continued compatibility with newer related software § Better support and faster resolution of issues § Ability to receive fixes for newly discovered issues.
- 24. Troubleshoot : Issue due to recent changes on the system
- 25. § Issue due to Application of new Patch § Issue due to Changes on ASM / Database parameter § Issue due to New OS package installed § Issue due to New Oracle Software installed Issue due to recent changes on the system
- 26. System Changes
- 27. Troubleshoot : Space Usage Issues
- 29. Troubleshoot : Issue due to Best Practice Violations
- 31. Troubleshoot : Major Events happening across the cluster
- 32. § For troubleshooting one needs to know : § What type of system does user have ? § What’s going on around the time of issue ? § Can I get a full picture across all nodes ? § Can I zoom into specific timeframe ? § Can I look at the data from various perspectives ? Customer Complains of “Grid failure - CRS-8503 []” in SR
- 33. Customer’s System around the time of Issue
- 34. Major Events around the time of issue
- 35. Major Events around the time of issue
- 36. Major Events around the time of issue
- 37. Troubleshoot : Operating System Issues
- 38. Customer’s System undergoes Node eviction
- 39. Customer’s System undergoes Node eviction
- 40. Customer’s System undergoes Node eviction High Memory Pressure Increase in RSS consumption by ‘extract’ process
- 41. Customer’s System undergoes Node eviction 50GB RSS hogged by extract process
- 43. Database Anomalies as observed in the Database Anomaly Advisor
- 44. Database Anomalies as observed in the Database Anomaly Advisor
- 45. Troubleshoot : Node Eviction Problems
- 46. Node eviction due to Huge Page over-allocation
- 47. Node eviction due to Huge Page over-allocation
- 48. Node eviction due to Huge Page over-allocation
- 49. Node eviction due to Huge Page over-allocation
- 50. Node eviction due to Huge Page over-allocation
- 51. Node eviction due to Huge Page over-allocation
- 52. Node eviction due to Huge Page over-allocation
- 53. How customers can log SRs using diagnostics
- 57. Demo
- 58. Thank you 46 Confidential – © 2019 Oracle Internal/Restricted/Highly Restricted