Scribd
Upload
2 views

RAGHACKCosmosDBNoSQL

Azure Cosmos DB is a highly scalable NoSQL database designed for AI applications, offering features like serverless options, automatic sharding, and low latency. It supports various GenAI use cases such as chat history, retrieval, and augmented generation, with successful implementations like KymChat and ChatGPT. The platform includes built-in vector indexing and search capabilities, making it suitable for multi-tenant applications and real-time data transactions.

Uploaded by

Asma Laaribi
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
2 views

RAGHACKCosmosDBNoSQL

Azure Cosmos DB is a highly scalable NoSQL database designed for AI applications, offering features like serverless options, automatic sharding, and low latency. It supports various GenAI use cases such as chat history, retrieval, and augmented generation, with successful implementations like KymChat and ChatGPT. The platform includes built-in vector indexing and search capabilities, making it suitable for multi-tenant applications and real-time data transactions.

Uploaded by

Asma Laaribi
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 15

Scalable RAG with

Azure Cosmos DB for NoSQL


James Codella, PhD
Principal Product Manager
Azure Cosmos DB
 What is Azure Cosmos DB
Agenda
 Who is using it?
 GenAI use cases
 Vector Search and more
 Demo
Azure Cosmos DB
is the world’s most scalable AI database

Serverless or Provisioned Throughput


Automatic sharding/partitioning
High elasticity with instant autoscale
Low latency, real-time data transactions
Mission critical reliability (99.999%)
5 levels of multitenancy
Built-in vector index and search with DiskANN
AI scenarios with Azure Cosmos DB for NoSQL

Chat history Retrieval Multi-tenant


Augmented AI apps
Generation (RAG)

Real-time Real-time Multi-Agent


Recommendatio Anomaly Detection AI
ns
KymChat
KymChat is an AI agent to streamline KPMG
employee operational tasks such as research,
drafting proposals, documents, and communications.

Leveraging Vector Search in Azure Cosmos DB for


MongoDB vCore enabled KPMG to provide value to
their employees at scale. KymChat demo at Ignite 2023

Accuracy Performance Scalability


PCI, a key relevancy 7,000+ employees Performance
metric increased from Up to 50% productivity improvements enabled
50% to 90%+ gain rollout to all KPMG
ChatGPT scales with OpenAI
Azure Cosmos DB
Challenge
• Meet incredible demand from traffic spikes,
without having to worry about database
Open AI stores ChatGPT conversations
operations
and all other user interactions in Azure
Cosmos DB, 40+ workloads Outcomes
• Rapidly and seamlessly scaled as service grew,
with zero downtime

• Able to iterate fast on data shapes thanks to


schemaless flexibility

• Maintained high performance and availability

Key Azure products used:

Azure Azure Azure


Cosmos DB Kubernetes Service AI Search
Azure Cosmos DB + AI
Today
GenAI use cases with Azure Cosmos DB
What Why When
Drastically reduces latency
Saves on Token consumption Slow moving / static content
Semantic Caching
Reduces costs and latency for FAQs, Policies…
LLM
Conversational context
UX improvements A MUST for Chat sessions
Chat History
LLM optimizations Improving cost & performance
Auditing a
Retrieval Personalize LLM on your data
Augmented Cheaper than fine tuning Any workload for GenAI apps
Generation (RAG) Faster iteration on new data

Vector + No ETL Data & vectors together


Operational Consistent data Cosmos DB scale &
Database Reduce complexity & costs performance
Azure Cosmos DB for
NoSQL
May 2024

Native Vector
Indexing and DiskANN Index Integrations
Search (Gated Preview) Semantic Kernel
(Public Preview) LangChain

Microsoft Confidential: Content is shared under NDA with Microsoft CAB


members
Vector Search in Azure Cosmos DB for NoSQL

Store data + Vector Search


vectors together + Query Filters Flexible
Indexing
Reduced Complexity & Cost Combine with equality,
Transactional Data & Vectors range Flat, quantized flat, and
Optimized for App & spatial filters DiskANN indexing available
Developers Optimize query focus

Azure Cosmos DB for NoSQL Capabilities


Serverless or provisioned throughput
Built-in multitenancy
Instant & dynamic autoscale
<10ms point-reads
Globally-replicated
Industry-leading 99.999% SLA
Multi-modal database with built-in DiskANN*
Scalable and cost effective, ideal for multi-tenant
apps
Vector Storage and Algorithms
compression graph construction
Large Vectors
{ D1, D2, D3, D4, D5, …, D99,
D100 } RAM
Compressed vectors
Quantizatio
n

SSD
Compressed Full vectors + graph
Vectors
{ D1, D2 .., D10 }

Unlimited scale Low latency Robust to data changes Serverless

*In preview, GA November ‘24


DEMO
Using Python + Azure Cosmos DB for NoSQL + Azure
Open AI to build a chatbot featuring
• RAG
• Chat History
• Semantic Caching
Links

• This demo: aka.ms/CosmosDBRAGwithPython


• Docs, demos, videos, and more: aka.ms/CosmosAISamples
• Azure Cosmos DB Vector Search:
aka.ms/CosmosVectorSearch
Azure AI Advantage free offer

Up to $6,000 Azure Cosmos


DB free for 90 days1
Eligibility: customers using Azure AI Services or GitHub Copilot

Why Azure Cosmos DB for Era of


AI

Flexibility and
AI ready Guaranteed Mission critical
efficiency
performance and scale

Learn more:
Aka.ms/AzureAIAdvantageBlog
*Terms & conditions: Offer available to all Azure customers who are using or will use Azure AI Services (including Azure OpenAI Service), Azure Machine Learning,
Azure AI Search) and/or GitHub Copilot. Offer provides 40,000 RU/s for free for a 90 day period. Offer only available where Azure Cosmos DB available.
Customers must create a new Azure Cosmos DB account and register via Microsoft Form at https://aka.ms/AzureAIAdvantageSignupForm. Time period: the last
day for customers to opt-in for the offer via the signup form is June 30, 2024.

You might also like