Sandesh_Rao_Unlocking Oracle Database Mysteries AHF Insights and the AI-LLM Duo.pdf

Unlocking Oracle Database Mysteries:
AHF Insights and the AI-LLM Duo
Sandesh Rao
VP AIOps and Machine Learning , Autonomous Database
@sandeshr
https://www.linkedin.com/in/raosandesh/
https://www.slideshare.net/SandeshRao4

Copyright © 2024, Oracle and/or its affiliates
2
1. AHF AIOPs platform
2. Challenges faced by Support & DevOps Engineers
3. Diagnostic Assistant - Solution
4. Diagnostic Assistant - High Level Architecture
5. Benefits of using NLU techniques and LLMs together
6. Support User Workflow
7. Intent Classification Overview
8. LLM - RAG Pipeline
Agenda

3
Observe Engage
Automate
AHF
AIOps
Realtime
data
Historical
data
Notification
Collaboration
Compliance
Incident
detection
Diagnostic
collection
Issue / Service Request
Cause & solution
identification
Runbooks
Machine
Learning
Health
Availability
Performance
Capacity
Logs
Scripts
Health
Checks Anomaly
Detection
Email
Pager
Jira
Bug
SR
Slack
ACR Sanitization

4
Fleet Level - Data Collection Fleet Level - Event Distribution

5
Fleet Level - Best Practice Issue Distribution

Cluster Level - AHF Insights

Event Information Best Practice Violations

Operating System Metrics

Anomaly Identification & Root Cause Analysis

10
Software Recommendations
RPMs

11
1. Loss of time
• Delays in information retrieval for triaging & debugging
• Manual and repetitive tasks that could be automated
• Managing a high volume of support tickets and requests
• Delayed resolution results in customer dissatisfaction
2. Information Overload
• Complex Systems and Integrations
• Misinterpretation of logs and symptoms leading to incorrect troubleshooting steps
3. Information silos / Navigation Issues
• Issue navigating to right source of information
• Learning curve to get accustomed to different applications
Challenges faced by Support & DevOps Engineers

12
When was the last patch applied ?
Any OS Issues identified 15 minute
before 3 AM ?
What are the CRS issues from alert
log around 3 AM ?
What are the major events between
8AM to 9AM ?
What is a node eviction issue ?
Are there any slow queries or
database performance issues?
What is the purpose of log writer
process ?
Support / DevOps
Information Engineers Need for troubleshooting
Diagnostics Specific
Product Specific
Diagnostics

13
1. AI Chatbot with Natural Language Support
• On-demand information retrieval using natural language queries
• Makes troubleshooting simpler and faster
2. Leverages
• Intent Classification
• Entity Recognition
• Retrieval-Augmented Generation (RAG) – LLM
- Oracle Database Documentation, Knowledge Management Documents
• Structured diagnostic data & analysis
- AHF Collection, AHF Insights
• API Integration with various internal data sources
3. Persona based user workflow automation
Diagnostic Assistant - Solution

14
Any OS Issues identified 15 minute
before 3AM ?
What are the CRS issues from alert
log around 3AM ?
What is a node eviction issue ?
What is the purpose of log writer
process ?
Support / DevOps
Diagnostic Assistant’s on-demand responses
Diagnostics Specific
Product Specific

15
• LLMs are advanced AI models trained on vast amounts of data to understand and generate human-like text
• They employ transformer architectures, which enable them to capture and process relationships between
words and phrases effectively
• GPT-3,3.5,4o (Generative Pre-trained Transformer) Developed by OpenAI , Cohere, Anthropic (Claude) ,
LLama 2,3 by Meta are examples of these LLMS
• They are known for their large scale and ability to generate coherent human-like text across a variety of
tasks
• BERT (Bidirectional Encoder Representations from Transformers) developed by Google
• BERT focuses on improving the understanding of context in natural language processing tasks
• After pre-training on this data, LLMs can be fine-tuned on specific tasks or domains by exposing them to
additional data relevant to the task at hand
Lets get the basics first

16
• LLMs maintain context over multiple turns of conversation, which is crucial for building coherent and
engaging chatbots.
• Besides chatbots, LLMs are used for content generation, translation, summarization, sentiment analysis
• Basic steps to build a chatbot
• Collect a diverse dataset of conversations (e.g., customer service, healthcare, education)
• Clean and preprocess the data by tokenizing sentences (e.g., punctuation, stopwords)
• Use a pre-trained LLM and fine-tune it on your specific dataset using techniques like transfer learning
• Adjust the model's parameters and training to optimize performance
• Integrate the fine-tuned LLM into a chatbot application framework (e.g Hugging Face)
• Test the chatbot rigorously to ensure it performs well across different scenarios and user inputs
• Gather user feedback and iterate on the chatbot's responses and functionalities
Lets get the basics first

17
Diagnostic Assistant - High Level Architecture

18
1. NLU (Intent + Entity Recognition)
1. Advantages
1. Provides accurate & repeatable results
2. Provides structured responses
3. Easy to reinforce new training data
4. Ability to score intent confidence
5. Items can be tailored through entity
recognition
6. Cheaper to perform the operation
2. Disadvantages
1. If low scoring intents are identified, there
are no results
2. Can support areas where APIs
corresponding to intents are available
Benefits of using NLU techniques and LLMs together
1. LLM
1. Advantages
1. Provides generative natural language
answers for a wider scope of natural
language queries
2. RAG approaches significantly increase their
scope of operations with additional
knowledge sources
2. Disadvantages
1. Does not provide repeatable results
2. Structured response require fine tuning,
prompt engineering and tailoring
3. Costly to perform the operation

19
Intent
Response
LLM
Response
Support User Workflow

20
Intent
Response

21
Intent
Response

22
Intent Classification Implementation & Feedback Loop

Confidential – Oracle Internal
25
Behind the scenes
for this bug,
change the support
contact to Marco
and the severity
to 4
Input query
0.14
-0.23
0.81
0.01
-0.97
-0.26
0.50
0.62
Embedding
0.01
0.08
0.03
0.11
0.02
0.36
0.09
0.03
Sentence
Encoder
0.01
0.27
Intent classifier
Complexity
classifier

26
Confusion Matrix
• Helps to visualize
• which APIs are being mis-classified
• which other API are being confused
• Helps better understand
• when there is a poor representation of a
word.
• Note
• Correct Classification are find over the
diagonal of the matrix.
• Mis-Classifications are any other position
out of the diagonal, indicating which actual
intent (row) was mis-classified as another
intent (column)

27
Intent Samples
Intent Samples used for training

31
Intent Testing and Training Interface

32
User Feedback for Intent Training and LLM response improvement

33
LLM - RAG Pipeline

34
1. Separating the knowledge base into
fixed-size chunks.
2. Vectorizing each chunk with an
embedding model.
3. Vectorizing the input/query at
inference time and using vector
search to find relevant chunks.
4. Adding relevant chunks into the
LLM’s prompt.
RAG Pipeline with customizations as per Use Case

35
1. No need to create very
low level tailored APIs
2. Context can be
restricted to a specific
diagnostic document
3. Better vocab and
context understanding
4. Better prompt results
from LLM (Generative,
Summarization)
Custom Diagnostic Documentation Creation
Templatization
Structured Data

37
1. Query : “What were the major events between
8 PM to 10 PM ?”
2. Intent Mapping : “get_bug_events”
3. Scoring: 0.30
4. What was the training dataset ?
- show me event list
- present all events for this bug
- give all detected events
- send list events
- display me all listed events
- retrieve me event types
- list me events from database obelyx_sty
- list events from host stbm000060-vm1
- give events from 30/09/2023 to 22/10/2024
- get all events from Feb to Jun
Natural Language Query -> Intent Classifier -> LLM (x)

38
1. Query : “How do you create a table in
Oracle database ?”
2. Intent Mapping : NO_API
3. Scoring : 0.18
4. Request passed onto LLM
Natural Language Query -> Intent Classifier (x) -> LLM

Thank you
39

Sandesh_Rao_Unlocking Oracle Database Mysteries AHF Insights and the AI-LLM Duo.pdf

Recommended

More Related Content

Similar to Sandesh_Rao_Unlocking Oracle Database Mysteries AHF Insights and the AI-LLM Duo.pdf (20)

More from Sandesh Rao (20)

Recently uploaded (20)

Sandesh_Rao_Unlocking Oracle Database Mysteries AHF Insights and the AI-LLM Duo.pdf