SlideShare a Scribd company logo

MongoDB Europe 2016 - Graph Operations with MongoDB

MongoDB
MongoDB

The document discusses the evolution of MongoDB and its capabilities in graph operations, particularly focusing on the $graphLookup operator that allows for efficient recursive queries. It highlights the strengths of using MongoDB as a native graph database, such as index-free adjacency and the first-class treatment of relationships, while also addressing the challenges and complexities associated with graph databases. Various use cases and example scenarios illustrate the application of MongoDB's graph features in real-world contexts, like social networks and travel options.

Data & Analytics
1 of 52
Downloaded 108 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
Graph Operations Rich Cullen Manager, Solutions Architecture
#MDBE16 Agenda MongoDB Evolution01 $graphLookup Operator03Graph Concepts02 Example Scenarios 05 Summary 06Demo 05
MongoDB Evolution
“ I’ve got too many applications. And I’ve got too many different technologies and tools being used to build and support those applications. I want to significantly reduce my technology surface area.”
#MDBE16 Functionality Timeline 2.0 – 2.2 Geospatial Polygon support Aggregation Framework New 2dsphere index Aggregation Framework efficiency optimisations Full text search 2.4 – 2.6 3.0 – 3.2 Join functionality Increased geo accuracy New Aggregation operators Improved case insensitivity Recursive graph traversal Faceted search Multiple collations 3.4
MongoDB 3.4 - Multi-Model Database Document Rich JSON Data Structures Flexible Schema Global Scale Rela,onal Le9-Outer Join Views Schema Valida?on Key/Value Horizontal Scale In-Memory Search Text Search Mul?ple Languages Faceted Search Binaries Files & Metadata Encrypted Graph Graph & Hierarchical Recursive Lookups GeoSpa,al GeoJSON 2D & 2DSphere
Graph Concepts
#MDBE16 Common Use Cases •  Networks •  Social – circle of friends/colleagues •  Computer network – physical/virtual/application layer •  Mapping / Routes •  Shortest route A to B •  Cybersecurity & Fraud Detection •  Real-time fraud/scam recognition •  Personalisation/Recommendation Engine •  Product, social, service, professional etc.
#MDBE16 Graph Key Concepts •  Vertices (nodes) •  Edges (relationships) •  Nodes have properties •  Relationships have name & direction
Why MongoDB For Graph?
#MDBE16 Native Graph Database Strengths •  Relationships are first class citizens of the database •  Index-free adjacency •  Nodes “point” directly to other nodes •  Efficient relationship traversal
#MDBE16 Index Free Adjacency •  Each node maintains direct references to adjacent nodes •  Bi-directional joins “pre-computed” and stored as relationships •  Query times independent of overall graph size •  Query times proportional to graph search area •  BUT… efficiency relies on native graph data storage •  Fixed-sized records •  Pointer-like record IDs •  File-system and object cache
#MDBE16 Native Graph Database Challenges •  Complex query languages •  Lack of widely available skills •  Poorly optimised for non-traversal queries •  Difficult to express •  Inefficient, memory intensive •  Less often used as System Of Record •  Synchronisation with SOR required •  Increased operational complexity •  Consistency concerns
Data Modeling
#MDBE16 Relational DBs Lack Relationships •  “Relationships” are actually JOINs •  Raw business or storage logic and constraints – not semantic •  JOIN tables, sparse columns, null-checks •  More JOINS = degraded performance and flexibility
#MDBE16 Relational DBs Lack Relationships •  How expensive/complex is: •  Find my friends? •  Find friends of my friends? •  Find mutual friends? •  Find friends of my friends of my friends? •  And so on…
#MDBE16 NoSQL DBs Lack Relationships •  “Flat” disconnected documents or key/value pairs •  “Foreign keys” inferred at application layer •  Data integrity/quality onus is on the application •  Suggestions re difficulty of modeling ANY relationships efficiently with aggregate stores. •  However…
#MDBE16 Friends Network – Document Style { _id: 0, name: "Bob Smith", friends: ["Anna Jones", "Chris Green"] }, { _id: 1, name: "Anna Jones", friends: ["Bob Smith", "Chris Green", "Joe Lee"] }, { _id: 2, name: "Chris Green", friends: ["Anna Jones", "Bob Smith"] }
#MDBE16 Schema Design – before $graphLookup •  Options •  Store an array of direct children in each node •  Store parent in each node •  Store parent and array of ancestors •  Trade-offs •  Simple queries… •  …vs simple updates 5 13 14 16 176 3 15121094 2 7 8 11 1
#MDBE16 •  Options •  Store immediate parent in each node •  Store immediate children in each node •  Traverse in multiple directions •  Recurse in same collection •  Join/recurse into another collection •  Aggregation Framework pipeline stage •  Can use multiple times per pipeline 5 13 14 16 176 3 15121094 2 7 8 11 1 Schema Design – with $graphLookup
$graphLookup
#MDBE16 Syntax $graphLookup: { from: <target lookup table>, startWith: <expression for value to start from>, connectToField: <field name to connect to>, connectFromField: <field name to connect from – recurse from here>, as: <field name alias for result array>, maxDepth: <max number of iterations to perform>, depthField: <field name alias for number of iterations required to reach this node>, restrictSearchWithMatch: <match condition to apply to lookup> }
#MDBE16 Things To Note •  startWith value is an expression •  Referencing a field directly requires the ‘$’ prefix •  Can do things like { $toLower: “name” } •  Handles array-based fields •  connectToField and connectFromField take field names only •  restrictSearchWithMatch requires standard query expressions •  Does not support aggregation framework operators/expressions
#MDBE16 Things To Note •  Cycles are automatically detected •  Can be used with views: •  Define a view •  Recurse across existing view (‘base’ or ‘from’) •  Supports Collations
Example Scenarios
#MDBE16 Scenario: Calculate Friend Network { _id: 0, name: "Bob Smith", friends: ["Anna Jones", "Chris Green"] }, { _id: 1, name: "Anna Jones", friends: ["Bob Smith", "Chris Green", "Joe Lee"] }, { _id: 2, name: "Chris Green", friends: ["Anna Jones", "Bob Smith"] }
#MDBE16 Scenario: Calculate Friend Network [ { $match: { "name": "Bob Smith" } }, { $graphLookup: { from: "contacts", startWith: "$friends", connectToField: "name", connectFromField: "friends”, as: "socialNetwork" } }, { $project: { name: 1, friends:1, socialNetwork: "$socialNetwork.name"} } ] No maxDepth set
#MDBE16 Scenario: Calculate Friend Network { "_id" : 0, "name" : "Bob Smith", "friends" : [ "Anna Jones", "Chris Green" ], "socialNetwork" : [ "Joe Lee", "Fred Brown", "Bob Smith", "Chris Green", "Anna Jones" ] } Array
#MDBE16 Scenario: Determine Air Travel Options ORD JFK BOS PWM LHR { "_id" : 0, "airport" : "JFK", "connects" : [ "BOS", "ORD" ] } { "_id" : 1, "airport" : "BOS", "connects" : [ "JFK", "PWM" ] } { "_id" : 2, "airport" : "ORD", "connects" : [ "JFK" ] } { "_id" : 3, "airport" : "PWM", "connects" : [ "BOS", "LHR" ] } { "_id" : 4, "airport" : "LHR", "connects" : [ "PWM" ] }
#MDBE16 [ { "$match": {"name":"Lucy"} }, { "$graphLookup": { from: "airports", startWith: "$nearestAirport", connectToField: "airport", connectFromField: "connects", maxDepth: 2, depthField: "numFlights", as: "destinations” } } ] Scenario: Determine Air Travel Options Record the number of recursions
#MDBE16 { name: "Lucy”, nearestAirport: "JFK", destinations: [ { _id: 0, airport: "JFK", connects: ["BOS", "ORD"], numFlights: 0 }, { _id: 1, airport: "BOS", connects: ["JFK", "PWM"], numFlights: 1 }, { _id: 2, airport: "ORD", connects: ["JFK"], numFlights: 1 }, { _id: 3, airport: "PWM", connects: ["BOS", "LHR"], numFlights: 2 } ] } Scenario: Determine Air Travel Options
#MDBE16 ORD JFK BOS PWM LHR ATL AA, BA AA, BA Scenario: Determine Air Travel Options
#MDBE16 { "_id" : 0, "airport" : "JFK", "connects" : [ { "to" : "BOS", "airlines" : [ "UA", "AA" ] }, { "to" : "ORD", "airlines" : [ "UA", "AA" ] }, { "to" : "ATL", "airlines" : [ "AA", "DL" ] }] } { "_id" : 1, "airport" : "BOS", "connects" : [ { "to" : "JFK", "airlines" : [ "UA", "AA" ] }, { "to" : "PWM", "airlines" : [ "AA" ] } ]] } { "_id" : 2, "airport" : "ORD", "connects" : [ { "to" : "JFK", "airlines" : [ "UA”,"AA" ] }] } { "_id" : 3, "airport" : "PWM", "connects" : [ { "to" : "BOS", "airlines" : [ "AA" ] }] } Scenario: Determine Air Travel Options
#MDBE16 [ { "$match":{"name":"Lucy"} }, { "$graphLookup": { from: "airports", startWith: "$nearestAirport", connectToField: "airport", connectFromField: "connects.to”, maxDepth: 2, depthField: "numFlights”, restrictSearchWithMatch: {"connects.airlines":"UA"}, as: ”UAdestinations" } } ] Scenario: Determine Air Travel Options We’ve added a filter
#MDBE16 { "name" : "Lucy", "from" : "JFK", "UAdestinations" : [ { "_id" : 2, "airport" : "ORD", "numFlights" : NumberLong(1) }, { "_id" : 1, "airport" : "BOS", "numFlights" : NumberLong(1) } ] } Scenario: Determine Air Travel Options
#MDBE16 Scenario: Calculate Friend Network Andrew Ron MeagenEliot Dev Elyse Richard Eliot Kirsten
#MDBE16 Scenario: Calculate Friend Network { "_id" : 1, "name" : "Dev", "title" : "CEO" } { "_id" : 2, "name" : "Eliot", "title" : "CTO", "boss" : 1 } { "_id" : 3, "name" : "Meagen", "title" : "CMO", "boss" : 1 } { "_id" : 4, "name" : "Carlos", "title" : "CRO", "boss" : 1 } { "_id" : 5, "name" : "Andrew", "title" : "VP Eng", "boss" : 2 } { "_id" : 6, "name" : "Ron", "title" : "VP PM", "boss" : 2 } { "_id" : 7, "name" : "Elyse", "title" : "COO", "boss" : 2 } { "_id" : 8, "name" : "Richard", "title" : "VP PS", "boss" : 1 } { "_id" : 8, "name" : "Kirsten", "title" : "VP People", "boss" : 1 }
#MDBE16 Scenario: Calculate Friend Network { $graphLookup: { from: "emp", startWith: "$boss", connectToField: "_id", connectFromField: "boss", as: "allBosses", depthField: "levelAbove” } }
#MDBE16 Scenario: Calculate Friend Network { "_id" : 5, "name" : "Andrew", "title" : "VP Eng", "boss" : 2, "allBosses" : [ { "_id" : 1, "name" : "Dev", "title" : "CEO", "levelAbove" : NumberLong(1) }, { "_id" : 2, "name" : "Eliot", "title" : "CTO", "boss" : 1, "levelAbove" : NumberLong(0) } ] }
#MDBE16 Scenario: Product Categories Mugs Kitchen & Dining Commuter & Travel Glassware & Drinkware Outdoor Recreation Camping Mugs Running Thermos Red Run Thermos White Run Thermos Blue Run Thermos
#MDBE16 Scenario: Product Categories Get all children 2 levels deep – flat result
#MDBE16 Scenario: Product Categories Get all children 2 levels deep – nested result
#MDBE16 Scenario: Article Recommendation 1 98 9 1 8 15 7 2 6 8 5 38 4 12 3 4 2 75 Depth 1 Depth 2 Depth 0 43 19 content id conversion rate recommendation
#MDBE16 Scenario: Article Recommendation 1 98 9 1 8 15 7 2 6 8 5 38 4 12 3 4 2 75 Depth 1 Depth 2 Depth 0 43 19 content id conversion rate recommendation Recommendations for Target #1 Recommendation for Targets #2 and #3 Target #1 (best) Target #2 Target #3
#MDBE16 Syntax
#MDBE16 Syntax
Demo
#MDBE16
Summary
#MDBE16 MongoDB $graphLookup •  Efficient, index-based recursive queries •  Familiar, intuitive query language •  Use a single System Of Record •  Cater for all query types •  No added operational overhead •  No synchronisation requirements •  Reduced technology surface area
MongoDB Europe 2016 - Graph Operations with MongoDB
Queries & Results
Ad

Recommended

PPTX
Social engineering
Vishal Kumar
 
PPTX
Cyber Threat Hunting Workshop
Digit Oktavianto
 
PDF
OSINT 2.0 - Past, present and future
Christian Martorella
 
PPTX
Security Operations, MITRE ATT&CK, SOC Roles / Competencies
Harry McLaren
 
PPTX
Цифровая гигиена. Что это и зачем нужна?
Vitaliy Yakushev
 
PPTX
Effective Threat Hunting with Tactical Threat Intelligence
Dhruv Majumdar
 
PDF
ATT&CKING Containers in The Cloud
MITRE ATT&CK
 
PPTX
DDoS Attack Presentation.pptx
HusamKhan1
 
PPT
DDOS Attack
Ahmed Salama
 
PDF
No Easy Breach DerbyCon 2016
Matthew Dunwoody
 
PDF
DSS_Enterprise MDM MAM Mobile Security - MobileIron Overview_2013
Andris Soroka
 
PPT
Malminkartanon muinaisuus ja kivikautinen kylätie
Malminkartanon kirjasto / Malminkartano Library
 
PDF
Threat Hunting Report
Morane Decriem
 
PDF
The ATT&CK Latin American APT Playbook
MITRE ATT&CK
 
PPTX
Social engineering
Maulik Kotak
 
PDF
MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...
MITRE - ATT&CKcon
 
PDF
From SIEM to SOC: Crossing the Cybersecurity Chasm
Priyanka Aash
 
PDF
Cyber Attack Methodologies
Geeks Anonymes
 
PPTX
Social Engineering new.pptx
Santhosh Prabhu
 
PDF
What We’ve Learned Building a Cyber Security Operation Center: du Case Study
Priyanka Aash
 
PDF
ATT&CK Updates- ATT&CK's Open Source
MITRE ATT&CK
 
PPT
Engenharia Social - A arte de enganar
Anderson Zardo
 
PPTX
The Diamond Model for Intrusion Analysis - Threat Intelligence
ThreatConnect
 
PPTX
RMF Roles and Responsibilities (Part 1)
Donald E. Hester
 
PDF
Exploring how Students Map Social Engineering Techniques to the ATT&CK Framew...
MITRE ATT&CK
 
PDF
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
MITRE ATT&CK
 
PPTX
How Internal Human Intelligence Networks (HUMINT) Develop External Primary So...
IntelCollab.com
 
PDF
Tracking Noisy Behavior and Risk-Based Alerting with ATT&CK
MITRE ATT&CK
 
PDF
Webinar: Working with Graph Data in MongoDB
MongoDB
 
PPTX
Doing Joins in MongoDB: Best Practices for Using $lookup
MongoDB
 

More Related Content

What's hot (20)

PPT
DDOS Attack
Ahmed Salama
 
PDF
No Easy Breach DerbyCon 2016
Matthew Dunwoody
 
PDF
DSS_Enterprise MDM MAM Mobile Security - MobileIron Overview_2013
Andris Soroka
 
PPT
Malminkartanon muinaisuus ja kivikautinen kylätie
Malminkartanon kirjasto / Malminkartano Library
 
PDF
Threat Hunting Report
Morane Decriem
 
PDF
The ATT&CK Latin American APT Playbook
MITRE ATT&CK
 
PPTX
Social engineering
Maulik Kotak
 
PDF
MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...
MITRE - ATT&CKcon
 
PDF
From SIEM to SOC: Crossing the Cybersecurity Chasm
Priyanka Aash
 
PDF
Cyber Attack Methodologies
Geeks Anonymes
 
PPTX
Social Engineering new.pptx
Santhosh Prabhu
 
PDF
What We’ve Learned Building a Cyber Security Operation Center: du Case Study
Priyanka Aash
 
PDF
ATT&CK Updates- ATT&CK's Open Source
MITRE ATT&CK
 
PPT
Engenharia Social - A arte de enganar
Anderson Zardo
 
PPTX
The Diamond Model for Intrusion Analysis - Threat Intelligence
ThreatConnect
 
PPTX
RMF Roles and Responsibilities (Part 1)
Donald E. Hester
 
PDF
Exploring how Students Map Social Engineering Techniques to the ATT&CK Framew...
MITRE ATT&CK
 
PDF
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
MITRE ATT&CK
 
PPTX
How Internal Human Intelligence Networks (HUMINT) Develop External Primary So...
IntelCollab.com
 
PDF
Tracking Noisy Behavior and Risk-Based Alerting with ATT&CK
MITRE ATT&CK
 
DDOS Attack
Ahmed Salama
 
No Easy Breach DerbyCon 2016
Matthew Dunwoody
 
DSS_Enterprise MDM MAM Mobile Security - MobileIron Overview_2013
Andris Soroka
 
Malminkartanon muinaisuus ja kivikautinen kylätie
Malminkartanon kirjasto / Malminkartano Library
 
Threat Hunting Report
Morane Decriem
 
The ATT&CK Latin American APT Playbook
MITRE ATT&CK
 
Social engineering
Maulik Kotak
 
MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...
MITRE - ATT&CKcon
 
From SIEM to SOC: Crossing the Cybersecurity Chasm
Priyanka Aash
 
Cyber Attack Methodologies
Geeks Anonymes
 
Social Engineering new.pptx
Santhosh Prabhu
 
What We’ve Learned Building a Cyber Security Operation Center: du Case Study
Priyanka Aash
 
ATT&CK Updates- ATT&CK's Open Source
MITRE ATT&CK
 
Engenharia Social - A arte de enganar
Anderson Zardo
 
The Diamond Model for Intrusion Analysis - Threat Intelligence
ThreatConnect
 
RMF Roles and Responsibilities (Part 1)
Donald E. Hester
 
Exploring how Students Map Social Engineering Techniques to the ATT&CK Framew...
MITRE ATT&CK
 
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
MITRE ATT&CK
 
How Internal Human Intelligence Networks (HUMINT) Develop External Primary So...
IntelCollab.com
 
Tracking Noisy Behavior and Risk-Based Alerting with ATT&CK
MITRE ATT&CK
 

Similar to MongoDB Europe 2016 - Graph Operations with MongoDB (20)

PDF
Webinar: Working with Graph Data in MongoDB
MongoDB
 
PPTX
Doing Joins in MongoDB: Best Practices for Using $lookup
MongoDB
 
PPTX
MongoDB Schema Design: Practical Applications and Implications
MongoDB
 
PPT
Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.
InfiniteGraph
 
PPT
Making sense of the Graph Revolution
InfiniteGraph
 
PPT
Webinar: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.
InfiniteGraph
 
PPTX
Jumpstart: Introduction to Schema Design
MongoDB
 
PDF
MongoDB Europe 2016 - The Rise of the Data Lake
MongoDB
 
PPTX
Schema Design
MongoDB
 
PDF
MongoDB Stitch Introduction
MongoDB
 
PDF
MongoDB in FS
MongoDB
 
PDF
Complex queries in a distributed multi-model database
Max Neunhöffer
 
PPT
10. Graph Databases
Fabio Fumarola
 
PPTX
tranSMART Community Meeting 5-7 Nov 13 - Session 2: MongoDB: What, Why And When
David Peyruc
 
PDF
Your Database Cannot Do this (well)
javier ramirez
 
PPTX
Graph Database Query Languages
Jay Coskey
 
PPT
NOSQL Now! Presentation, August 24, 2011: Graph Databases: Connecting the Dot...
InfiniteGraph
 
PDF
Which Questions We Should Have
Oracle Korea
 
PDF
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB
MongoDB
 
PPTX
MongoDB Schema Design
Joe Drumgoole
 
Webinar: Working with Graph Data in MongoDB
MongoDB
 
Doing Joins in MongoDB: Best Practices for Using $lookup
MongoDB
 
MongoDB Schema Design: Practical Applications and Implications
MongoDB
 
Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.
InfiniteGraph
 
Making sense of the Graph Revolution
InfiniteGraph
 
Webinar: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.
InfiniteGraph
 
Jumpstart: Introduction to Schema Design
MongoDB
 
MongoDB Europe 2016 - The Rise of the Data Lake
MongoDB
 
Schema Design
MongoDB
 
MongoDB Stitch Introduction
MongoDB
 
MongoDB in FS
MongoDB
 
Complex queries in a distributed multi-model database
Max Neunhöffer
 
10. Graph Databases
Fabio Fumarola
 
tranSMART Community Meeting 5-7 Nov 13 - Session 2: MongoDB: What, Why And When
David Peyruc
 
Your Database Cannot Do this (well)
javier ramirez
 
Graph Database Query Languages
Jay Coskey
 
NOSQL Now! Presentation, August 24, 2011: Graph Databases: Connecting the Dot...
InfiniteGraph
 
Which Questions We Should Have
Oracle Korea
 
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB
MongoDB
 
MongoDB Schema Design
Joe Drumgoole
 
Ad

More from MongoDB (20)

PDF
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB
 
PDF
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
PDF
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB
 
PDF
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB
 
PDF
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB
 
PDF
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB
 
PDF
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
PDF
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB
 
PDF
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB
 
PDF
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB
 
PDF
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB
 
PDF
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB
 
PDF
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB
 
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB
 
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB
 
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB
 
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB
 
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB
 
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB
 
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB
 
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB
 
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB
 
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB
 
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB
 
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB
 
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB
 
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB
 
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB
 
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB
 
Ad

Recently uploaded (20)

PPTX
最新版美国约翰霍普金斯大学毕业证（JHU毕业证书）原版定制
Taqyea
 
PDF
Informatics Market Insights AI Workforce.pdf
karizaroxx
 
PDF
Allotted-MBBS-Student-list-batch-2021.pdf
subhansaifi0603
 
PDF
Predicting Titanic Survival Presentation
praxyfarhana
 
PPTX
Indigo_Airlines_Strategy_Presentation.pptx
mukeshpurohit991
 
PPTX
一比一原版(TUC毕业证书)开姆尼茨工业大学毕业证如何办理
taqyed
 
PPTX
Model Evaluation & Visualisation part of a series of intro modules for data ...
brandonlee626749
 
PDF
Microsoft Power BI - Advanced Certificate for Business Intelligence using Pow...
Prasenjit Debnath
 
PPT
Camuflaje Tipos Características Militar 2025.ppt
e58650738
 
PDF
NVIDIA Triton Inference Server, a game-changing platform for deploying AI mod...
Tamanna36
 
DOCX
Starbucks in the Indian market through its joint venture.
sales480687
 
PPTX
UPS and Big Data intro to Business Analytics.pptx
sanjum5582
 
PDF
Shifting Focus on AI: How it Can Make a Positive Difference
1508 A/S
 
PDF
11_L2_Defects_and_Trouble_Shooting_2014[1].pdf
gun3awan88
 
PDF
presentation4.pdf Intro to mcmc methodss
SergeyTsygankov6
 
PPTX
RESEARCH-FINAL-GROUP-3, about the final .pptx
gwapokoha1
 
PPTX
英国毕业证范本利物浦约翰摩尔斯大学成绩单底纹防伪LJMU学生证办理学历认证
taqyed
 
PPTX
最新版美国威斯康星大学河城分校毕业证（UWRF毕业证书）原版定制
taqyea
 
PDF
Boost Business Efficiency with Professional Data Entry Services
eloiacs eloiacs
 
PDF
All the DataOps, all the paradigms .
Lars Albertsson
 
最新版美国约翰霍普金斯大学毕业证（JHU毕业证书）原版定制
Taqyea
 
Informatics Market Insights AI Workforce.pdf
karizaroxx
 
Allotted-MBBS-Student-list-batch-2021.pdf
subhansaifi0603
 
Predicting Titanic Survival Presentation
praxyfarhana
 
Indigo_Airlines_Strategy_Presentation.pptx
mukeshpurohit991
 
一比一原版(TUC毕业证书)开姆尼茨工业大学毕业证如何办理
taqyed
 
Model Evaluation & Visualisation part of a series of intro modules for data ...
brandonlee626749
 
Microsoft Power BI - Advanced Certificate for Business Intelligence using Pow...
Prasenjit Debnath
 
Camuflaje Tipos Características Militar 2025.ppt
e58650738
 
NVIDIA Triton Inference Server, a game-changing platform for deploying AI mod...
Tamanna36
 
Starbucks in the Indian market through its joint venture.
sales480687
 
UPS and Big Data intro to Business Analytics.pptx
sanjum5582
 
Shifting Focus on AI: How it Can Make a Positive Difference
1508 A/S
 
11_L2_Defects_and_Trouble_Shooting_2014[1].pdf
gun3awan88
 
presentation4.pdf Intro to mcmc methodss
SergeyTsygankov6
 
RESEARCH-FINAL-GROUP-3, about the final .pptx
gwapokoha1
 
英国毕业证范本利物浦约翰摩尔斯大学成绩单底纹防伪LJMU学生证办理学历认证
taqyed
 
最新版美国威斯康星大学河城分校毕业证（UWRF毕业证书）原版定制
taqyea
 
Boost Business Efficiency with Professional Data Entry Services
eloiacs eloiacs
 
All the DataOps, all the paradigms .
Lars Albertsson
 

MongoDB Europe 2016 - Graph Operations with MongoDB

  • 1. Graph Operations Rich Cullen Manager, Solutions Architecture
  • 4. “ I’ve got too many applications. And I’ve got too many different technologies and tools being used to build and support those applications. I want to significantly reduce my technology surface area.”
  • 5. #MDBE16 Functionality Timeline 2.0 – 2.2 Geospatial Polygon support Aggregation Framework New 2dsphere index Aggregation Framework efficiency optimisations Full text search 2.4 – 2.6 3.0 – 3.2 Join functionality Increased geo accuracy New Aggregation operators Improved case insensitivity Recursive graph traversal Faceted search Multiple collations 3.4
  • 6. MongoDB 3.4 - Multi-Model Database Document Rich JSON Data Structures Flexible Schema Global Scale Rela,onal Le9-Outer Join Views Schema Valida?on Key/Value Horizontal Scale In-Memory Search Text Search Mul?ple Languages Faceted Search Binaries Files & Metadata Encrypted Graph Graph & Hierarchical Recursive Lookups GeoSpa,al GeoJSON 2D & 2DSphere
  • 8. #MDBE16 Common Use Cases •  Networks •  Social – circle of friends/colleagues •  Computer network – physical/virtual/application layer •  Mapping / Routes •  Shortest route A to B •  Cybersecurity & Fraud Detection •  Real-time fraud/scam recognition •  Personalisation/Recommendation Engine •  Product, social, service, professional etc.
  • 9. #MDBE16 Graph Key Concepts •  Vertices (nodes) •  Edges (relationships) •  Nodes have properties •  Relationships have name & direction
  • 10. Why MongoDB For Graph?
  • 11. #MDBE16 Native Graph Database Strengths •  Relationships are first class citizens of the database •  Index-free adjacency •  Nodes “point” directly to other nodes •  Efficient relationship traversal
  • 12. #MDBE16 Index Free Adjacency •  Each node maintains direct references to adjacent nodes •  Bi-directional joins “pre-computed” and stored as relationships •  Query times independent of overall graph size •  Query times proportional to graph search area •  BUT… efficiency relies on native graph data storage •  Fixed-sized records •  Pointer-like record IDs •  File-system and object cache
  • 13. #MDBE16 Native Graph Database Challenges •  Complex query languages •  Lack of widely available skills •  Poorly optimised for non-traversal queries •  Difficult to express •  Inefficient, memory intensive •  Less often used as System Of Record •  Synchronisation with SOR required •  Increased operational complexity •  Consistency concerns
  • 15. #MDBE16 Relational DBs Lack Relationships •  “Relationships” are actually JOINs •  Raw business or storage logic and constraints – not semantic •  JOIN tables, sparse columns, null-checks •  More JOINS = degraded performance and flexibility
  • 16. #MDBE16 Relational DBs Lack Relationships •  How expensive/complex is: •  Find my friends? •  Find friends of my friends? •  Find mutual friends? •  Find friends of my friends of my friends? •  And so on…
  • 17. #MDBE16 NoSQL DBs Lack Relationships •  “Flat” disconnected documents or key/value pairs •  “Foreign keys” inferred at application layer •  Data integrity/quality onus is on the application •  Suggestions re difficulty of modeling ANY relationships efficiently with aggregate stores. •  However…
  • 18. #MDBE16 Friends Network – Document Style { _id: 0, name: "Bob Smith", friends: ["Anna Jones", "Chris Green"] }, { _id: 1, name: "Anna Jones", friends: ["Bob Smith", "Chris Green", "Joe Lee"] }, { _id: 2, name: "Chris Green", friends: ["Anna Jones", "Bob Smith"] }
  • 19. #MDBE16 Schema Design – before $graphLookup •  Options •  Store an array of direct children in each node •  Store parent in each node •  Store parent and array of ancestors •  Trade-offs •  Simple queries… •  …vs simple updates 5 13 14 16 176 3 15121094 2 7 8 11 1
  • 20. #MDBE16 •  Options •  Store immediate parent in each node •  Store immediate children in each node •  Traverse in multiple directions •  Recurse in same collection •  Join/recurse into another collection •  Aggregation Framework pipeline stage •  Can use multiple times per pipeline 5 13 14 16 176 3 15121094 2 7 8 11 1 Schema Design – with $graphLookup
  • 22. #MDBE16 Syntax $graphLookup: { from: <target lookup table>, startWith: <expression for value to start from>, connectToField: <field name to connect to>, connectFromField: <field name to connect from – recurse from here>, as: <field name alias for result array>, maxDepth: <max number of iterations to perform>, depthField: <field name alias for number of iterations required to reach this node>, restrictSearchWithMatch: <match condition to apply to lookup> }
  • 23. #MDBE16 Things To Note •  startWith value is an expression •  Referencing a field directly requires the ‘$’ prefix •  Can do things like { $toLower: “name” } •  Handles array-based fields •  connectToField and connectFromField take field names only •  restrictSearchWithMatch requires standard query expressions •  Does not support aggregation framework operators/expressions
  • 24. #MDBE16 Things To Note •  Cycles are automatically detected •  Can be used with views: •  Define a view •  Recurse across existing view (‘base’ or ‘from’) •  Supports Collations
  • 26. #MDBE16 Scenario: Calculate Friend Network { _id: 0, name: "Bob Smith", friends: ["Anna Jones", "Chris Green"] }, { _id: 1, name: "Anna Jones", friends: ["Bob Smith", "Chris Green", "Joe Lee"] }, { _id: 2, name: "Chris Green", friends: ["Anna Jones", "Bob Smith"] }
  • 27. #MDBE16 Scenario: Calculate Friend Network [ { $match: { "name": "Bob Smith" } }, { $graphLookup: { from: "contacts", startWith: "$friends", connectToField: "name", connectFromField: "friends”, as: "socialNetwork" } }, { $project: { name: 1, friends:1, socialNetwork: "$socialNetwork.name"} } ] No maxDepth set
  • 28. #MDBE16 Scenario: Calculate Friend Network { "_id" : 0, "name" : "Bob Smith", "friends" : [ "Anna Jones", "Chris Green" ], "socialNetwork" : [ "Joe Lee", "Fred Brown", "Bob Smith", "Chris Green", "Anna Jones" ] } Array
  • 29. #MDBE16 Scenario: Determine Air Travel Options ORD JFK BOS PWM LHR { "_id" : 0, "airport" : "JFK", "connects" : [ "BOS", "ORD" ] } { "_id" : 1, "airport" : "BOS", "connects" : [ "JFK", "PWM" ] } { "_id" : 2, "airport" : "ORD", "connects" : [ "JFK" ] } { "_id" : 3, "airport" : "PWM", "connects" : [ "BOS", "LHR" ] } { "_id" : 4, "airport" : "LHR", "connects" : [ "PWM" ] }
  • 30. #MDBE16 [ { "$match": {"name":"Lucy"} }, { "$graphLookup": { from: "airports", startWith: "$nearestAirport", connectToField: "airport", connectFromField: "connects", maxDepth: 2, depthField: "numFlights", as: "destinations” } } ] Scenario: Determine Air Travel Options Record the number of recursions
  • 31. #MDBE16 { name: "Lucy”, nearestAirport: "JFK", destinations: [ { _id: 0, airport: "JFK", connects: ["BOS", "ORD"], numFlights: 0 }, { _id: 1, airport: "BOS", connects: ["JFK", "PWM"], numFlights: 1 }, { _id: 2, airport: "ORD", connects: ["JFK"], numFlights: 1 }, { _id: 3, airport: "PWM", connects: ["BOS", "LHR"], numFlights: 2 } ] } Scenario: Determine Air Travel Options
  • 33. #MDBE16 { "_id" : 0, "airport" : "JFK", "connects" : [ { "to" : "BOS", "airlines" : [ "UA", "AA" ] }, { "to" : "ORD", "airlines" : [ "UA", "AA" ] }, { "to" : "ATL", "airlines" : [ "AA", "DL" ] }] } { "_id" : 1, "airport" : "BOS", "connects" : [ { "to" : "JFK", "airlines" : [ "UA", "AA" ] }, { "to" : "PWM", "airlines" : [ "AA" ] } ]] } { "_id" : 2, "airport" : "ORD", "connects" : [ { "to" : "JFK", "airlines" : [ "UA”,"AA" ] }] } { "_id" : 3, "airport" : "PWM", "connects" : [ { "to" : "BOS", "airlines" : [ "AA" ] }] } Scenario: Determine Air Travel Options
  • 34. #MDBE16 [ { "$match":{"name":"Lucy"} }, { "$graphLookup": { from: "airports", startWith: "$nearestAirport", connectToField: "airport", connectFromField: "connects.to”, maxDepth: 2, depthField: "numFlights”, restrictSearchWithMatch: {"connects.airlines":"UA"}, as: ”UAdestinations" } } ] Scenario: Determine Air Travel Options We’ve added a filter
  • 35. #MDBE16 { "name" : "Lucy", "from" : "JFK", "UAdestinations" : [ { "_id" : 2, "airport" : "ORD", "numFlights" : NumberLong(1) }, { "_id" : 1, "airport" : "BOS", "numFlights" : NumberLong(1) } ] } Scenario: Determine Air Travel Options
  • 36. #MDBE16 Scenario: Calculate Friend Network Andrew Ron MeagenEliot Dev Elyse Richard Eliot Kirsten
  • 37. #MDBE16 Scenario: Calculate Friend Network { "_id" : 1, "name" : "Dev", "title" : "CEO" } { "_id" : 2, "name" : "Eliot", "title" : "CTO", "boss" : 1 } { "_id" : 3, "name" : "Meagen", "title" : "CMO", "boss" : 1 } { "_id" : 4, "name" : "Carlos", "title" : "CRO", "boss" : 1 } { "_id" : 5, "name" : "Andrew", "title" : "VP Eng", "boss" : 2 } { "_id" : 6, "name" : "Ron", "title" : "VP PM", "boss" : 2 } { "_id" : 7, "name" : "Elyse", "title" : "COO", "boss" : 2 } { "_id" : 8, "name" : "Richard", "title" : "VP PS", "boss" : 1 } { "_id" : 8, "name" : "Kirsten", "title" : "VP People", "boss" : 1 }
  • 38. #MDBE16 Scenario: Calculate Friend Network { $graphLookup: { from: "emp", startWith: "$boss", connectToField: "_id", connectFromField: "boss", as: "allBosses", depthField: "levelAbove” } }
  • 39. #MDBE16 Scenario: Calculate Friend Network { "_id" : 5, "name" : "Andrew", "title" : "VP Eng", "boss" : 2, "allBosses" : [ { "_id" : 1, "name" : "Dev", "title" : "CEO", "levelAbove" : NumberLong(1) }, { "_id" : 2, "name" : "Eliot", "title" : "CTO", "boss" : 1, "levelAbove" : NumberLong(0) } ] }
  • 40. #MDBE16 Scenario: Product Categories Mugs Kitchen & Dining Commuter & Travel Glassware & Drinkware Outdoor Recreation Camping Mugs Running Thermos Red Run Thermos White Run Thermos Blue Run Thermos
  • 41. #MDBE16 Scenario: Product Categories Get all children 2 levels deep – flat result
  • 42. #MDBE16 Scenario: Product Categories Get all children 2 levels deep – nested result
  • 43. #MDBE16 Scenario: Article Recommendation 1 98 9 1 8 15 7 2 6 8 5 38 4 12 3 4 2 75 Depth 1 Depth 2 Depth 0 43 19 content id conversion rate recommendation
  • 44. #MDBE16 Scenario: Article Recommendation 1 98 9 1 8 15 7 2 6 8 5 38 4 12 3 4 2 75 Depth 1 Depth 2 Depth 0 43 19 content id conversion rate recommendation Recommendations for Target #1 Recommendation for Targets #2 and #3 Target #1 (best) Target #2 Target #3
  • 47. Demo
  • 50. #MDBE16 MongoDB $graphLookup •  Efficient, index-based recursive queries •  Familiar, intuitive query language •  Use a single System Of Record •  Cater for all query types •  No added operational overhead •  No synchronisation requirements •  Reduced technology surface area