6Different

RAG
Techniques Part 3

@Bhavishya Pandit
REVEAL: Retrieval-Augmented Visual-Language Model

This technique enhances AI models by combining

reasoning with task-specific actions and external
knowledge for decision-making.
It reduces errors by grounding reasoning in real-world
facts, minimizing inaccuracies & hallucinations.
The method produces clear, human-like task-solving steps,
increasing transparency.
REVEAL delivers strong performance across tasks with
fewer training examples, improving efficiency and
adaptability.
Its flexibility allows for interactive adjustments, making
models more controllable and responsive in real-world
applications.

@Bhavishya Pandit
REACT: Retrieval-Enhanced Action generation

The ReAct technique combines reasoning & action,

starting with the model receiving an observation from its
environment.
It updates its context with past actions and thoughts to
maintain situational awareness.
The model generates a thought that guides its next action,
ensuring decisions are logical and task-aligned.
After executing the action, new feedback helps refine its
understanding.
This blend of reasoning and action reduces errors, adapts
to real-time changes, and leads to more transparent,
reliable decisions.

@Bhavishya Pandit
 REPLUG: Retrieval Plugin

REPLUG enhances LLMs by retrieving relevant external

documents to improve predictions.
It treats the language model as a fixed "black box",
prepending retrieved information to the input.
This flexible design can be easily applied to existing
models without modifying them.
By integrating external knowledge, REPLUG reduces errors
like hallucinations and expands the model’s understanding
of niche information.
The retrieval component can be fine-tuned using feedback
from the language model, improving alignment with the
model’s needs.

@Bhavishya Pandit
MEMO RAG: Memory-Augmented RAG

Memo RAG combines memory and retrieval to

handle complex queries.
A memory model generates draft answers that
guide the search for external information.
The retriever then gathers relevant data from
databases, which a more powerful language
model uses to create a comprehensive final
answer.
This method helps Memo RAG manage
ambiguous queries and efficiently process
large amounts of information across various
tasks.

@Bhavishya Pandit
ATLAS: Attention-based retrieval Augmented Sequence generation

ATLAS improves language models by retrieving external

documents to boost accuracy in tasks like question
answering.
It uses a dual-encoder retriever to search large text corpora
and find the top-K relevant documents for a query.
These documents are processed by a Fusion-in-Decoder
model, integrating query and document data to generate the
final response.
With fewer parameters, it reduces reliance on memorization,
using dynamic document retrieval instead.
The document index can be updated without retraining,
keeping its current and effective for knowledge-intensive
tasks

@Bhavishya Pandit
RETRO: Retrieval-Enhanced Transformer

RETRO splits input text into smaller chunks and retrieves

relevant information from a large text database.
Using pre-trained BERT embeddings, it pulls in similar
chunks from external data to enrich context.
By integrating these chunks through chunked cross-
attention, it improves predictions without significantly
increasing model size.
This approach enables better access to external
knowledge, enhancing tasks like question answering and
text generation.
It achieves greater efficiency, handling large amounts of
information without the heavy computational demands
of larger models.

@Bhavishya Pandit
 Standard RAG Speculative RAG
Uses a small specialist model for
Combines retrieval with large drafting and a larger generalist Agentic RAG
language models for accurate,
model for verification, ensuring Uses adaptive agents for real-time
context-aware responses.
efficiency and accuracy. strategy adjustments in information
Breaks documents into chunks for
Parallel Drafting: Speeds up retrieval.
efficient information retrieval. Accurately interprets user intent
responses by generating multiple
Aims for 1-2 second response times for relevant, trustworthy
drafts simultaneously.
for real-time use. responses.
Enhances answer quality by
Superior Accuracy: Outperforms Modular design enables easy
leveraging external data sources. standard RAG systems. integration of new data sources and
Efficient Processing: Offloads features.
Enhances parallel processing and
complex tasks to specialized
performance on complex tasks by
models, reducing computational running agents concurrently.
load.
Corrective RAG Fusion RAG
Focuses on identifying and fixing
Integrates multiple retrieval Self RAG
methods and data sources for
errors in generated responses. Uses the model's own outputs as
enhanced response quality.
Uses multiple passes to improve retrieval candidates for better
Provides comprehensive answers
outputs based on feedback. contextual relevance.
Aims for higher precision and by leveraging diverse data inputs. Refines responses iteratively,
user satisfaction compared to Increases system resilience by improving consistency and
standard RAG. reducing dependence on a single coherence.
Leverages user feedback to source. Grounds responses in prior
enhance the correction process . Adapts retrieval strategies outputs for increased accuracy.
dynamically based on query Adapts retrieval strategies based
on the conversation's evolving
context.
context.
@Bhavishya Pandit
@Bhavishya Pandit @Bhavishya Pandit

Have you
Graph RAG Adaptive RAG

Graph RAG constructs a knowledge graph on-the-fly,

linking relevant entities during retrieval.
It leverages node relationships to decide when and
read the first It dynamically decides when to retrieve external
knowledge, balancing internal and external
knowledge​.
It uses confidence scores from the language
model's internal states to assess retrieval necessity.
An honesty probe helps the model avoid

two parts?
how much external knowledge to retrieve. hallucinations by aligning its output with its actual
Confidence scores from the graph guide expansion, knowledge​.
avoiding irrelevant additions. It reduces unnecessary retrievals, improving both
This approach improves efficiency and response accuracy efficiency and response accuracy​.
by keeping the knowledge graph compact and relevant.
@Bhavishya Pandit @Bhavishya Pandit

REALM: Retrieval augmented language model pre-training RAPTOR: Recursive Abstractive Processing for Iterative RAG
Tree-Organized Retrieval
REFEED: Retrieval Feedback

Unlike traditional retrieval, iterative RAG performs

multiple retrieval steps, refining its search based on
feedback from previously selected documents​.
REALM retrieves relevant documents from large corpora RAPTOR builds a hierarchical tree by clustering and REFEED refines model outputs using retrieval feedback Retrieval decisions follow a Markov decision process​.
like Wikipedia to enhance model predictions​. without fine-tuning​. Reinforcement learning improves retrieval
summarizing text recursively​.
The retriever is trained with masked language modeling, Initial answers are improved by retrieving relevant
It enables retrieval at different abstraction levels, performance.
optimizing retrieval to improve prediction accuracy​. documents and adjusting the response based on the
combining broad themes with specific details​. The iterative retriever maintains an internal state,
It uses Maximum Inner Product Search to efficiently find
RAPTOR outperforms traditional methods in complex new information. allowing it to adjust future retrieval steps based on
relevant documents from millions of candidates during
question-answering tasks​. Generates multiple answers to improve retrieval the accumulated knowledge from previous iterations.
training​.
Offers tree traversal and collapsed tree methods for accuracy.
REALM outperforms previous models in Open-domain
efficient information retrieval​. Combines pre- and post-retrieval outputs using a
Question Answering by integrating external knowledge​.
ranking system to enhance answer reliability​.
@Bhavishya Pandit @Bhavishya Pandit
@Bhavishya Pandit
@Bhavishya Pandit

@Bhavishya Pandit
 Bhavishya Pandit

Share your Save for Like this

thoughts later post