Neo4j and Generative AI: New Frontiers in Data Intelligence

Neo4j and Generative AI:
New Frontiers in Data
Intelligence
Malaysia GraphTalk-Monday 14th October
Bryan Lee, Professional Services Architect

2 Neo4j Inc. All rights reserved 2024
● Located in Singapore
● Neo4j Professional
Services Team
● Helping customers
achieve success
throughout their Neo4j
journey

Agenda
● Current State of GenAI
● Why RAG?
● Can Knowledge Graph help?
● Why GraphRAG? Why RAG?
● GenAI Use Cases
● LLM Graph Builder Demo
Neo4j Inc. All rights reserved 2024
3

The State of Generative AI
4

The Good 󰙤
5

6
AI
Breakthroughs
Personalization at Scale
Enhance Decision-Making
Process Optimization
Boost Creativity
Predictive Analytics
Improve Data Retrieval
and Automation
Generate Content and Code
Improve Customer Experiences

The Good 󰙤

The Good 󰙤
The Bad 󰗭

The Good 󰙤
The Bad 󰗭
9

The Good 󰙤 The Bad 󰗭
The Ugly 😱

The Good 󰙤 The Bad 󰗭 The Ugly 😱
GenAI Alone != Right Outcomes 🤯

Challenges with GenAI: Stochastic Parrot?
● Lack of enterprise domain knowledge
● Inability to verify answers
● Hallucination
● Ethical and data bias concerns
● and more
GenAI
PARROT

13
Managing AI risk
is the biggest
barrier to scaling
AI initiatives1
Skepticism: Over half of business leaders are
skeptical in adopting GenAI.2
Explainability: Over 80% of executives worry
about non-transparent nature of GenAI could
result in poor or unlawful decisions.2
Reliability: Inaccuracy and hallucination are two
of the most-cited risks of adopting GenAI
technology at all levels of an organisation.3
1. Deloitte’s State of AI in the Enterprise 2. BCG’s Digital Acceleration Index Study 2023 3. McKinsey: The state of AI in 2023

How can organizations use
domain-speciﬁc knowledge
to rapidly build accurate,
contextual, and explainable
GenAI applications?
The Big
Question

Why RAG?
And what is it anyway…

Retrieval Augmented Generation:
The ability to dynamically query a large
text corpus to incorporate relevant factual
knowledge into the responses generated
by the underlying language model

Retrieval-Augmented Generation Is Becoming
an Industry Standard
RAG augments LLMs by retrieving
up-to-date, contextual external data
to inform responses:
● Reduce hallucinations with
veriﬁed data
●
Provide domain-speciﬁc,
relevant responses
●
Enable traceability back
to sources
Retrieval Augmented Generation
Database of Truth

Why RAG With Vector Databases Fall Short
1
3
2
4
Similarity is insufﬁcient for rich enterprise reasoning
Only leverage a fraction of
your data: Beyond simple
“metadata”, vector databases
alone fail to capture relationships
from structured data
Miss critical context: Struggle to
capture connections across
nuanced facts, making it
challenging to answer multi-step,
domain-speciﬁc, questions
Vector Similarity ≠ Relevance:
Vector search uses an incomplete
measure of similarity. Relying on it
solely can result in irrelevant and
duplicative results
Lack explainability:
The black-box nature of
vectors lacks transparency
and explainability

Why GraphRAG? Why RAG?
And what is it anyway…

GraphRAG
Technique for richly
understanding text datasets
by combining text extraction,
network analysis, LLM
prompting and summarization
into a single end-to-end
system
20

21
RAG with Neo4j
Find similar documents
and content
Identify entities associated
to content and patterns
in connected data
Improve GenAI inferences
and insights. Discover new
relationships and entities
Unify vector search, knowledge graph and data science
capabilities to improve RAG quality and effectiveness
Vector Search
Graph Data
Science
Knowledge
Graph

Elevate relevance with domain
context and inferences
Accelerate GenAI
application development
Build your next
GenAI breakthrough
with context and
deep explainability
Explain and improve
GenAI applications
Neo4j is the GenAI Enabler

Elevate Relevance with Domain
Context and Inferences
Improve context with facts from a knowledge graph
Enhance personalization and quality
using graph pattern matching
Expand insights and inferences using
graph data science and machine learning

Improve Context
with Facts from a
Knowledge Graph
Combine native vector search with
multi-hop graph traversals to add
domain-speciﬁc context from a
knowledge graph
Incorporate hybrid retrieval, including
text search and lookups based off date,
numeric, and geopoint indexes

LLM
Application
User
Neo4j Database
5. Answer provided
1. Asks question
2. Question is translated to
a Cypher statement
4. Result from database is
converted to natural language
3. Generated
Cypher is used
to query Neo4j
database
Improve Context
with Facts from a
Knowledge Graph
knowledge graph
Convert user questions to Cypher
queries to perform RAG with explicit
query logic

Improve Context
with Facts from a
Knowledge Graph
knowledge graph
Convert user questions to Cypher
queries to perform RAG with explicit
query logic
Uniﬁed Copilot Service Layer
Including Text2Cypher
data
importer
Customer
applications

Use graph patterns to rank and
score vector search results
based on domain knowledge
Retrieve more reﬁned results
to power better personalized
LLM Responses
from langchain.vectorstores.neo4j_vector import
Neo4jVector
kg_personalized_search =
Neo4jVector.from_existing_index(
embedding=embedding_model,
url=NEO4J_URI,username=NEO4J_USERNAME,password=NEO4J_PA
SSWORD,
index_name='product-text-embeddings',
retrieval_query=f"""
WITH node AS product, score AS vectorSearchScore
OPTIONAL MATCH
(product) [:VARIANT_OF]-(:Article) [:PURCHASED]-
(:Customer)-[:PURCHASED] (a:Article) [:PURCHASED]-
(:Customer {{customerId: ' {CUSTOMER_ID }'}})
WITH count(a) AS purchaseScore,
RETURN
""")
Enhance
Personalization
and Quality

Expand Insights and
Inferences Using
Data Science and ML
28
Quickly incorporate context from
structured data in RAG vector search
using Neo4j graph embeddings with a
native vector index
Deeper insights for AI by enriching
your knowledge graph with link
prediction, community detection, and
classiﬁcation using graph algorithms
and machine learning
Largest Catalog of Graph
Algorithms
Native Graph
Analytics Workspace
Graph Embeddings for
Enhanced Retrieval

29
Explain and Improve GenAI Applications
Easily trace sources and explain retrieval logic
Understand and explain patterns in AI grounding
Improve GenAI grounding data at scale

Easily Trace
Sources
30
Add citation metadata
to nodes and relationships
within the knowledge graph
Enable source citation in
LLM responses by
integrating with
frameworks like LangChain
and LlamaIndex

Explain
Retrieval Logic
31
Inspect and explain retrieval
logic by converting user
questions to explicit Cypher
queries that are human-
readable and traceable

Understand and Explain
Patterns in AI Grounding
Easily understand grounding data
by representing it in a Neo4j knowledge
graph-both a human-friendly and
machine/GenAI-readable format
Introspect and visualize connections
between data using Cypher queries
and low-code/no-code interfaces like
Neo4j Bloom
Analyze patterns in grounding data
using graph algorithms like KNN,
community detection and centrality to
understand semantic similarities and
connections between sources

Improve GenAI
Grounding Data at Scale
Improve data quality using Cypher and
graph algorithms for duplicate
identiﬁcation, outlier detection, and
revealing trends and biases
Assess LLM performance and usage
patterns to identify areas of improvement
–analyze the connections between
prompts, responses, and grounding data
using Cypher and graph algorithms

34
Power your AI applications with robust and dynamic data representation
Rapidly build a knowledge graph for AI use cases
Integrate with the GenAI ecosystem
Accelerate GenAI Application Development

Empower your AI projects
with Robust and Dynamic
Data Representation
Harness all your data in one place with a
property graph data model, including
vector storage and search capabilities
Future-proof with an adaptable,
dev-friendly schema to add new data
types and entities, while maintaining data
quality
Enable real-time data ingestion into Neo4j
for immediate use in GenAI apps, with
streaming support via our Kafka Connector
●
Car
●
Person
Person
Name: “Andre”
Born: May 29, 1970
Twitter: “@dan”
Name: “Mica”
Born: Dec 5, 1975
Car
Brand “Volvo”
Model: “V70”
Description: “An executive car manufactured and…”
DescEmbedding: [0.1, -0.3, 0.4, …, -0.7]
DescSource:”https://en.wikipedia.org/wiki/Volvo_V70”
Since:
Jan 10, 2011
LOVES
KNOWS
KNOWS
LIVES WITH
O
W
N
S
D
R
I
V
E
S

Rapidly Build a
Knowledge Graph
for AI Use Cases
Car
Car
Jumpstart knowledge graph
creation from unstructured
data with named entity
recognition
Rapidly model structured data
as nodes and relationships in a
knowledge graph
Ingest text, audio, and other
content embeddings as node
vector properties and index
them for use in RAG
Embedding (Unstructured)
Structured
Unstructured
Data Sources
Named Entity Recognition
Structured Data
Generative AI Models
Embedding Models
Knowledge Graph

Integrate with the
GenAI Ecosystem
GenAI Stack
Application
Generative AI & Embedding Models
Orchestration
Grounding Knowledge Graph
Neo4jGraph
Neo4jVector
GraphCypherQAChain
Neo4jGraphStore
Neo4jVectorStore
KnowledgeGraphIndex
Neo4j GenAI Integrations
Text | Chat | Embedding
NL Query | Image Gen
Neo4j Drivers
Java
Python JavaScript
Call LLM APIs natively via
Cypher using our GenAI
procedures or open-source
APOC library
Integrate Neo4j with leading LLM
open-source frameworks such as
LangChain and LlamaIndex
Agnostic LLM orchestration
connecting graphs to OpenAI,
AWS Bedrock, GCP Vertex AI,
Azure, Anthropic, Hugging Face,
and other proprietary and open
source foundation models

38
Summary
Elevate relevance with domain
context and inferences
Accelerate GenAI application
development
Explain and improve GenAI
applications

40
Challenge: Global Retailer wants to retain and
grow talent, but ﬁnding the right new opportunity
is in the hands of the associate. This is a time
consuming and often limiting approach because
the current system forces the associate to search
and browse roles without visibility into all the
options and without knowledge about how roles
and skills come together
Solution: Build a career opportunity
recommendation system based on where
associates live, work today, their skills,
and past work experience
Impact: Engage associates at all levels of their
career path and guide them to their best next
position. Proactively recruit candidates for
promotion based on their attributes
Roles
(Email, Custom Cards)
Associate
Global Retail Giant uses LLM with Knowledge Graph
to Recommend the Best Opportunities

41
Investment Bank Summarizes Quarterly Earnings
Reports Across Companies
Challenge: Investment bankers need a
quick way to understand the detail of
quarterly reporting across all the
companies in their portfolios, but data is
stored in multiple formats and locations
Solution: Use vector embeddings to look
for the most relevant information, while
the Neo4j knowledge graph adds the
exact match for ﬁnancial statements
Impact: Quickly review quarterly reports
across companies
Investment Banker
Prompt
LLM
Embeddings API
Vector Search in
Vector Index
Embedding
Relevant Results /
Documents
Knowledge graph of
ﬁnancial statements
LLM
Chat API
Response

42
Pharmaceutical Company Democratize Access to
Supply Chain Risk Analysis
Challenge: Supply Chain risk analysis
knowledge graph was accessible only by 300
experts familiar with graph analysis. They
wanted to open up these insights to
thousands more employees
Solution: Conversational AI solution to allow
natural language access to supply chain KG.
Any employee can ask a question in plain
english. It gets converted into Cypher query
that can pull relevant risk assessment data
from the graph. The query results get passed
back into natural language generator to
create an an easy to understand response to
the initial question
Impact: Democratize access to supply chain
risk analysis
Supply Chain
Knowledge
Graph
Prompt (Description of
Desired Products)
Cypher Query
Relevant Results
Employee
LLM API
LLM API
Response

43
Government Procurement Entity Automates
Complex Processes
Challenge: Government entities can overspend
or acquire goods and services redundantly
because the volume of RFPs makes reviewing
each of them resource prohibitive
Solution: Use LLM to read the nature of RFP
and classify it accordingly. Compare the new
RFP against the knowledge graph of active
and historic RFPs and associated spend.
Recommend opportunities for consolidation or
terms negotiation with suppliers
Impact: Save tax dollars for other important
projects
Predeﬁned Prompt with
Results
LLM API
Generates personalized
text
User is Emailed with
Recommendation
Procurement Ofﬁcer
Supplier
LLM API Categorizes
RFP
Enterprise App
Ingests New RFP
Knowledge
graph of RFPs
and spending
patterns

FREE to use and test:
https://llm-graph-builder.neo4jlabs.com/
Github:
https://github.com/neo4j-labs/llm-graph-
builder
LLM Graph
Builder

Neo4j and Generative AI: New Frontiers in Data Intelligence

Recommended

More Related Content

Similar to Neo4j and Generative AI: New Frontiers in Data Intelligence (20)

More from Neo4j (20)

Recently uploaded (20)

Neo4j and Generative AI: New Frontiers in Data Intelligence