Neo4j: The path to success with Graph Database and Graph Data Science
1 like273 views
This document provides an overview of the Neo4j graph data platform and its capabilities for data science and analytics. It discusses Neo4j's native graph architecture, tools for data scientists and analysts, and how Neo4j enables graph data science across the machine learning lifecycle from feature engineering to model deployment. Algorithms, embeddings, and machine learning pipelines in Neo4j are highlighted. Integration with common data ecosystems is also covered.
![© 2023 Neo4j, Inc. All rights reserved.
© 2023 Neo4j, Inc. All rights reserved.
Faster and
Easier
Queries
24
K-HOP queries speedup
New and Enhanced Indexes
Graph Pattern Matching (GPM)
Quantified relationship patterns
Quantified path patterns
Benefit: orders of magnitude faster!
e.g., 1000x faster than v4 for 8-hop queries
New RANGE and POINT indexes
Benefit: Up to 100x faster for queries involving CONTAINS and ENDS
WITH clauses
Benefits: The new expressions can make the MATCH much easier to
read. It is another step towards making Cypher GQL compliant.
MATCH (n:Person)-[:KNOWS]->+(m:Person) RETURN n,m
(start)((a:Person)-[k:KNOWS WHERE k.since<date("2001-01-11”)]->
(b:Person))*
(end)](https://image.slidesharecdn.com/graphsummit2023telaviv-yafo-thepathtosuccesswithgraphdatabaseanddatasciencecopy-230621085738-fe53d757/85/Neo4j-The-path-to-success-with-Graph-Database-and-Graph-Data-Science-23-320.jpg)
Neo4j: The path to success with Graph Database and Graph Data Science
- 1. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. The Path To Success With Graph Database and Data Science Gal Bello, Senior Field Engineer 1
- 2. © 2023 Neo4j, Inc. All rights reserved. © 2022 Neo4j, Inc. All rights reserved. Neo4j Graph Data Platform 2 BUSINESS USERS DEVELOPERS DATA SCIENTISTS DATA ANALYSTS Enterprise Ready Data Science & MLOps Graph Data Science OLAP Data Science and Analytics Tools, algorithms, and Integrated ML framework AutoML Integrations Discovery & Visualization Low-code querying, data modeling and exploration tools Neo4j Bloom BI Connectors Neo4j Browser Language interfaces Application Development Tools & Frameworks Tools and APIs for rapid prototyping and development Graph Query Language Cypher and GQL as the lingua franca for graphs Transactions Analytics Graph Database Data Consolidation Contextualization OLTP Native Graph Database The core component of Neo4j platform Runs Anywhere Run by yourself or as DBaaS by Neo4j, in the cloud or on premises Data Connectors Ecosystem & Integrations Rich set of connectors to plug into existing data ecosystems Data Sources
- 3. © 2023 Neo4j, Inc. All rights reserved. Engineering Expertise >1000 person-years investment First mover advantage Maturity, Most enterprise deployments Largest graph community Growing at 80%+ annually Neo4j Graph Database Capabilities Hybrid Workloads Native Graph Architecture Powers Graph Data Science Rich Toolset Enterprise Trust Runs Anywhere 3
- 4. © 2023 Neo4j, Inc. All rights reserved. 4 Native Graph Architecture Native Graph Storage Native Graph Processing • No mismatch • Data integrity / ACID • Schema flexible • 1000x faster than relational • K-Hop now 10-1000x faster than version 4 Fabric • Federation of scaled out shards • Instant composite database Composite DB Autonomous Clustering • Elastic scale-out for high throughput • 100s of machines across clusters Data integrity and high speed also true in scaled out situations
- 5. © 2023 Neo4j, Inc. All rights reserved. A Graph Is... ...a set of discrete entities, each of which has some set of relationships with the other entities
- 6. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. WHEN 6
- 7. © 2023 Neo4j, Inc. All rights reserved. It’s Not What You Know
- 8. © 2023 Neo4j, Inc. All rights reserved. It’s How it’s Connected
- 9. © 2023 Neo4j, Inc. All rights reserved. It’s How it’s Connected And Other Insights it brings
- 10. © 2023 Neo4j, Inc. All rights reserved. Hybrid Workload Duality 10 Intelligent Applications Transactions - Security - Performance & Scalability - ACID Consistency - Intelligent Modeling - Extensive & Supported Algo Library - Scalable - Graph Visualization - Graph Transformations Graph Transactions Graph Analytics & Data Science
- 11. © 2023 Neo4j, Inc. All rights reserved. Powers Neo4j Graph Data Science Graph Data Science MACHINE LEARNING Analytics Feature Engineering Data Exploration Graph Data Science TensorFlow KNIME Python 11 Project your graph for in-memory analytics ● Unparalleled analytical processing ● .. with 60+ Algorithms for predictive analytics ● .. and pipeline to supervised AI/ML models ● Making AI smarter!
- 12. © 2023 Neo4j, Inc. All rights reserved. Developer Productivity: Rich tooling and easy onramp ops manager 12 data importer Visualize and explore your data Query editor and results visualizer Code-free data loader and modeler AuraWorkspace Unified Workspace
- 13. © 2023 Neo4j, Inc. All rights reserved. 13 Plugs into your data and development ecosystem Neo4j BI Connector Apache Spark Connector Apache Kafka Connector Data Warehouse Connector Java Python .NET JavaScript Go
- 14. © 2023 Neo4j, Inc. All rights reserved. ● Real-time Performance at Scale ● Automatic Upgrades, Patches, Backups ● Scale on Demand, No Downtime ● High Availability ● Multi Cloud, Any Region ● Enterprise-grade Security ● Simple Capacity-Based Pricing 14 Run Anywhere: self managed, or by Neo4j ● Full administrative control ● On-premises or via cloud marketplace ● Fit where cloud isn’t appropriate (e.g. special compliance scenarios) ● Easy migration to AuraDB Self-Managed
- 15. 15 Neo4j Database: Index-free adjacency Nodes and relationships are stored on disk as a graph for fast navigational access using pointers.
- 16. Neo4j, Inc. All rights reserved 2021 16 Index Free Adjacency in 2 Minutes My house Yossi’s house
- 17. Neo4j, Inc. All rights reserved 2021 17 Index Free Adjacency in 2 Minutes My house Yossi’s house
- 18. 18 IFA Benefits IFA uses pointers to the target address ● No index lookups or table scans ● Reduced duplication of foreign keys Anne Links 5 1 USA Links 3 2 3 4 5 Alex Links 2 James Links 5 Germany Links 1, 4
- 19. E J D A 19 IFA Question: Deletion 1 2 3 4 5 6 7 8 9 10 How to avoid wasted space on deletion and insertion? B C F G H
- 20. C E A 20 IFA Solution: Chunking 1 2 3 4 5 6 7 8 9 10 All data objects are chunked into same-size pieces A B C D F C F
- 21. Neo4j, Inc. All rights reserved 2021 Neo4j, Inc. All rights reserved 2021 Fixed Sized Records “Joins” on Creation Spin Spin Spin through this data structure Pointers instead of Lookups 1 2 3 4 Neo4j Secret Sauce
- 22. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. Neo4j v5.x: Unbounded Scale, Performance, and Agility 23
- 23. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. Faster and Easier Queries 24 K-HOP queries speedup New and Enhanced Indexes Graph Pattern Matching (GPM) Quantified relationship patterns Quantified path patterns Benefit: orders of magnitude faster! e.g., 1000x faster than v4 for 8-hop queries New RANGE and POINT indexes Benefit: Up to 100x faster for queries involving CONTAINS and ENDS WITH clauses Benefits: The new expressions can make the MATCH much easier to read. It is another step towards making Cypher GQL compliant. MATCH (n:Person)-[:KNOWS]->+(m:Person) RETURN n,m (start)((a:Person)-[k:KNOWS WHERE k.since<date("2001-01-11”)]-> (b:Person))* (end)
- 24. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. Easy Scale- Out 25 Autonomous clustering Instant Fabric Composite database
- 25. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. Seamless Operational Agility 26 Neo4j Ops Manager Differential backup Continuous Improvement Any-to-any rolling upgrade Full Diff Diff Full2 Diff Diff Full Recover Restore until time or tx id Neo4j 5.0 Neo4j 5.n Neo4j 5.5
- 26. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. Neo4j Graph Data Science
- 27. © 2023 Neo4j, Inc. All rights reserved. What’s important? Prioritization Who has the most connections? Who has the highest page rank? Who is an influencer? What’s unusual? Anomaly & Fraud Detection Where is a community forming? What are the group dynamics? What’s unusual about this data? What’s next? Predictions What’s the most common path? Who is in the same community? What relationship will form? 28 Pl ay s Lives_in In_sport Likes F a n _ o f Plays_for K n o w s Knows Knows K n o w s Graph Structure Improves Data Science Outcomes
- 28. © 2023 Neo4j, Inc. All rights reserved. 29 With The Largest Catalog of Graph Algorithms Pathfinding & Search Centrality & Importance Community Detection Supervised Machine Learning Heuristic Link Prediction Similarity Graph Embeddings …and more Graph algorithms are a set of instructions that visit the nodes of a graph to analyze the relationships in connected data.
- 29. © 2023 Neo4j, Inc. All rights reserved. And created Neo4j Graph Data Science: Eliminate Pain & Optimize Data Science Workflows with the data you already have Eliminates Pain Optimizes Data Science Flows Complex joins operations Mining Multiple Tables Tedious Manual Approximations Brute Force Comparisons Fractured Data 95% reduction in computation time 500x faster than open source libraries Improves Customer Outcomes 20-30% improvement in model performance 600% improvement in traffic $5 Million of additional fraud detected 3x better churn predictability 5x reduction in factory production lead time 30
- 30. © 2023 Neo4j, Inc. All rights reserved. 31 Data Scientists > Native Python Client > Apache Arrow integration > Unified ML pipelines We invest in four key areas Built by data scientists, for data scientists Better Predictions > 65+ Graph algorithms & embeddings > Graph native ML Pipelines > Vertex AI & SageMaker Integrations The best graph data science and ML engine Ecosystem > Apache Spark & Kafka Connectors > Native BI Connector > Data Warehouse Connector > GNN library support Seamlessly works with your data stack and pipeline Production Ready > Compatible with all major clouds > Enterprise Scale & Security > Deploy anywhere Go to production with speed, scale, and security
- 31. © 2023 Neo4j, Inc. All rights reserved. And Full Support Across the entire ML Lifecycle Feature Engineering Model Training & Tuning Model Deployment Data Collection & Preparation Exploratory Data Analysis Model Evaluation & Selection Drivers, Connectors, Fast Import/Export Graph Queries, Algorithms, and Visualization Graph Embeddings & Algorithms Predict APIs, Model/Graph Catalog Operations, Connectors Graph Native ML Pipelines Unsupervised Graph Algorithms Graph Features -> External ML Pipelines 32
- 32. © 2023 Neo4j, Inc. All rights reserved. And made it seamless for all ecosystems and pipelines Graph Data Science BI & VISUALIZATIONS INGEST STORE PROCESS Apache Kafka MACHINE LEARNING Cloud Functions Neo4j Bloom PubSub DataProc Analytics Feature Engineering Data Exploration Graph Data Science Business Applications & Existing Systems Files (unstructured, structured) TensorFlow KNIME Python Cloud Storage AWS Lambda 33 Graph Database
- 33. © 2023 Neo4j, Inc. All rights reserved. View the most well connected and influential nodes Recommendations from shared user interactions and associations Our visualizations make analysis easy to understand 34
- 34. © 2023 Neo4j, Inc. All rights reserved. 35 What’s in it for you: ● Improve model accuracy by 30% ● Simplify processes and remove headaches ● More projects into production without additional hiring Neo4j Graph Data Science Analytics Feature Engineering Data Exploration Graph Data Science Queries & Search Machine Learning Visualization
- 35. © 2023 Neo4j, Inc. All rights reserved. 37 How to get started… 3. Graph Native Machine Learning Learn features in your graph that you don’t even know are important yet using embeddings. Predict links, labels, and missing data with in-graph supervised ML models. Identify associations, anomalies, and trends using unsupervised machine learning. 2. Graph Algorithms 1. Knowledge Graphs Find the patterns you’re looking for in connected data
- 36. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. What’s New in Graph Data Science
- 37. © 2023 Neo4j, Inc. All rights reserved. Algos & Embeddings HashGNN Embedding: Faster approach than GNNs for knowledge graphs KMeans Cluster data based on properties like graph embeddings Leiden Algorithm: Fast and scalable modularity based community detection New Image courtesy of: Traag, V.A., Waltman, L. & van Eck, N.J. Image courtesy of: javatpoint.com Leiden Algorithm: K-means Clustering: 39
- 38. © 2023 Neo4j, Inc. All rights reserved. ML Pipelines Autotuning: Find optimal hyperparameters to improve model performance Multilayer Perceptrons (MLPs): Fully connected neural networks now available for Link Prediction and Node Classification New 40
- 39. © 2023 Neo4j, Inc. All rights reserved. GNN Support Graph Sampling: sample a representative subgraph from a larger graph for training complex models Graph Export: use our projections in other graph ML libraries like Deep Graph Library (DGL), PyG, and Tensorflow GNN New Image courtesy of Google Cloud 41
- 40. © 2023 Neo4j, Inc. All rights reserved. GNN Support Graph Sampling: sample a representative subgraph from a larger graph for training complex models Graph Export: use our projections in other graph ML libraries like Deep Graph Library (DGL), PyG, and Tensorflow GNN New Image courtesy of Google Cloud 42
- 41. © 2023 Neo4j, Inc. All rights reserved. Continue your graph journey Connect with passionate graphistas Free online training and certification • dev.neo4j.com/learn • dev.neo4j.com/datasets Graph expert group - The Ninjas • dev.neo4j.com/ninjas Connect with the community: • dev.neo4j.com/chat • dev.neo4j.com/forum • dev.neo4j.com/newsletter Next developer events • Live Streams - Weekly & Online • Local Meetups neo4j.com/events
- 42. © 2023 Neo4j, Inc. All rights reserved. Meet the Neo4j Ninjas Masters of Graphs Ninjas are: Active graph bloggers, presenters, GitHub contributors, professors, user group leaders, and researchers - all sharing their graph expertise Benefits: Ninjas benefit from exclusive access to Neo4j experts, VIP event experience, special giveaways and much more Interested? For more information visit:
- 43. © 2023 Neo4j, Inc. All rights reserved. APOC Documentation Other Neo4j Resources Neo4j Graph Data Science Documentation Neo4j Cypher Manual Neo4j Driver Manual Cypher Style Guide Arrows App • APOC is a great plugin to level up your cypher • This documentation outlines different commands one could use • Link to APOC documentation • The Cypher manual can be used to get more information about Cypher commands • Link to cypher manual • Neo4j Graph Data Science documentation is a great reference to see which algorithms to use • Show how to use different algorithms • Link to Graph Data Science documentation • The driver manual provides the official drivers that are supported by Neo4j • Link to Neo4j driver manual • The cypher style guide provide recommendations for building clean, easy to read Cypher queries • Link to Cypher style guide • The Arrows app allows one to design a graph without using Cypher • Link to Arrows app Cypher Cheat Sheet • This page gives quick examples of how to write different queries within Cypher • Link to Cypher cheat sheet GraphGists • GraphGists has many different use cases and examples for specific industries • Link to GraphGists Neo4j Sandbox • The Neo4j sandbox provides a quick deployment of a Neo4j server • It does not require a download • Comes with example projects • Link to Neo4j Sandbox
- 44. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. 46 THANK YOU Share feedback at slido.com #GraphSummitTelAvivYafo2023