How Shutl Delivers Even Faster Using Neo4J
0 likes778 views
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1uRYaAR. Volker Pacher, Sam Phillips present key differences between relational databases and graph databases, and how they use the later to model a complex domain and to gain insights into their data. Filmed at qconlondon.com. Sam Phillips is Head of Engineering for eBay's Local Delivery team, bringing super fast delivery to customers in the UK and US. Volker Pacher is a Senior Developer at eBay Local Delivery. Before its acquisition by eBay, he was a member of the core team at Shutl helping to transition from a monolithic application to SOA and introducing new technologies, among them Neo4j.
![find all the companies my friends work for
MATCH (person{ name:’Volker’ }) -[:friends]
- (person) - [:works_for]-> company
RETURN company](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-103-320.jpg)
![find all the companies my friends work for
MATCH (person{ name:’Volker’ }) -[:friends]
- (person) - [:works_for]-> company
RETURN company](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-104-320.jpg)
![find all the companies my friends work for
MATCH (person{ name:’Volker’ }) -[:friends]
- (person) - [:works_for]-> company
RETURN company
Person
name: Sam
Person
name: Volker
:friends
:knows
since: 2005
Person
name: Megan
:friends
Company
name: eBay
:works_for
:works_for
Person
name: Jim
:friends
:works_for
Company
name: neotech](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-105-320.jpg)
![find all the companies my friend’s friends work for
MATCH (person{ name:’Volker’ }) -
[:friends*2..2]-(person) - [:works_for]
-> company
RETURN company](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-106-320.jpg)
![find all the companies my friend’s friends work for
MATCH (person{ name:’Volker’ }) -
[:friends*2..2]-(person) - [:works_for]
-> company
RETURN company](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-107-320.jpg)
![find all the companies my friend’s friends work for
MATCH (person{ name:’Volker’ }) -
[:friends*2..2]-(person) - [:works_for]
-> company
RETURN company
Person
name: Sam
Person
name: Volker
:friends
:knows
since: 2005
Person
name: Megan
:friends
Company
name: eBay
:works_for
:works_for
Person
name: Jim
:friends
:works_for
Company
name: neotech](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-108-320.jpg)
![find all my friends who work for neotech
MATCH (person{ name:’Volker’ }) -[:friends]
-(friends) - [:works_for]-> company
WHERE company.name = ‘neotech’
RETURN friends](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-109-320.jpg)
![find all my friends who work for neotech
MATCH (person{ name:’Volker’ }) -[:friends]
-(friends) - [:works_for]-> company
WHERE company.name = ‘neotech’
RETURN friends](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-110-320.jpg)
![find all my friends who work for neotech
MATCH (person{ name:’Volker’ }) -[:friends]
-(friends) - [:works_for]-> company
WHERE company.name = ‘neotech’
RETURN friends
Person
name: Sam
Person
name: Volker
:friends
:knows
since: 2005
Person
name: Megan
:friends
Company
name: eBay
:works_for
:works_for
Person
name: Jim
:friends
:works_for
Company
name: neotech](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-111-320.jpg)
![MATCH (store{ id:’ebay_store’ }) -[:located]
-> (locality) <- [:operates]- carrier
RETURN carrier
the query](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-117-320.jpg)
![the query
MATCH (store{ id:’ebay_store’ }) -[:located]
-> (locality) <- [:operates]- carrier
RETURN carrier](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-118-320.jpg)
![MATCH (store{ id:’ebay_store’ }) -[:located]
-> (locality) <- [:operates]- carrier
RETURN carrier
the query
Locality
id = california
Locality
id = 94902
Locality
id = marin_county
Locality
id = 94901
:contains
Store
id = ebay_store
:located
:contains
Locality
id = 94903
:contains :contains
:operates :operates
Carrier
id = carrier_1](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-119-320.jpg)
![MATCH (store{ id:’ebay_store’ }) -[:located]
-> () <- [:contains*0..2] - (locality)
<- [:operates]- carrier
RETURN carrier
the query](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-120-320.jpg)
![MATCH (store{ id:’ebay_store’ }) -[:located]
-> () <- [:contains*0..2] - (locality)
<- [:operates]- carrier
RETURN carrier
the query](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-121-320.jpg)
![MATCH (store{ id:’ebay_store’ }) -[:located]
-> () <- [:contains*0..2] - (locality)
<- [:operates]- carrier
RETURN carrier
the query
Locality
id = california
Locality
id = marin_county
Locality
id = 94901
:contains
Store
id = ebay_store
:located
:contains
Locality
id = 94903
:contains :contains
Locality
id = 94902
:operates Carrier
id = carrier_2
:operates :operates
Carrier
id = carrier_1](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-122-320.jpg)
![MATCH (store{ id:’ebay_store’ }) -[:located]
-> () <- [:contains*0..2] - (locality)
<- [:operates]- carrier
RETURN carrier
the query
Locality
id = california
Locality
id = marin_county
Locality
id = 94901
:contains
Store
id = ebay_store
:located
:contains
Locality
id = 94903
:contains :contains
Locality
id = 94902
:operates Carrier
id = carrier_2
:operates :operates
Carrier
id = carrier_1](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-123-320.jpg)
![MATCH (store{ id:’ebay_store’ }) -[:located]
-> () <- [:contains*0..2] - (locality)
<- [:operates]- carrier
RETURN carrier
the query
Locality
id = california
Locality
id = marin_county
Locality
id = 94901
:contains
Store
id = ebay_store
:located
:contains
Locality
id = 94903
:contains :contains
Locality
id = 94902
:operates Carrier
id = carrier_2
:operates :operates
Carrier
id = carrier_1](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-124-320.jpg)
![MATCH (store{ id:’ebay_store’ }) -[:located]
-> () <- [:contains*0..2] - (locality)
<- [:operates]- carrier
RETURN carrier
the query
Locality
id = california
Locality
id = marin_county
Locality
id = 94901
:contains
Store
id = ebay_store
:located
:contains
Locality
id = 94903
:contains :contains
Locality
id = 94902
:operates Carrier
id = carrier_2
:operates :operates
Carrier
id = carrier_1](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-125-320.jpg)
![MATCH (store{ id:’ebay_store’ }) -[:located]
-> () <- [:contains*0..2] - (locality)
<- [:operates]- carrier
RETURN carrier](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-129-320.jpg)
![find all events on a specific day
START root=node(0)
MATCH root - [:year_2014] -> () -[:month_05] ->
()- [:day_24] -> () - [:happens] -> event
RETURN event](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-131-320.jpg)
![find all events on a specific day
START root=node(0)
MATCH root - [:year_2014] -> () -[:month_05] ->
()- [:day_24] -> () - [:happens] -> event
RETURN event](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-132-320.jpg)
![find all events on a specific day
START root=node(0)
MATCH root - [:year_2014] -> () -[:month_05] ->
()- [:day_24] -> () - [:happens] -> event
RETURN event root (0)
:year_2014
Year: 2013
:year_2015
Year: 2015
:month_06
:month_05 :month_01
Month: 05 Month: 01
Month: 06
:day_24 :day_25
Day: 24 Day: 25
:day_26
Day: 26
:happens :happens :happens :happens
Event 1 Event 2 Event 3](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-133-320.jpg)
![find all events on a specific day
START root=node(0)
MATCH root - [:year_2014] -> () -[:month_05] ->
()- [:day_24] -> () - [:happens] -> event
RETURN event root (0)
:year_2014
Year: 2013
:year_2015
Year: 2015
:month_06
:month_05 :month_01
Month: 05 Month: 01
Month: 06
:day_24 :day_25
Day: 24 Day: 25
:day_26
Day: 26
:happens :happens :happens :happens
Event 1 Event 2 Event 3](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-134-320.jpg)
![find all events on a specific day
START root=node(0)
MATCH root - [:year_2014] -> () -[:month_05] ->
()- [:day_24] -> () - [:happens] -> event
RETURN event root (0)
:year_2014
Year: 2013
:year_2015
Year: 2015
:month_06
:month_05 :month_01
Month: 05 Month: 01
Month: 06
:day_24 :day_25
Day: 24 Day: 25
:day_26
Day: 26
:happens :happens :happens :happens
Event 1 Event 2 Event 3](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-135-320.jpg)
![find all events on a specific day
START root=node(0)
MATCH root - [:year_2014] -> () -[:month_05] ->
()- [:day_24] -> () - [:happens] -> event
RETURN event root (0)
:year_2014
Year: 2013
:year_2015
Year: 2015
:month_06
:month_05 :month_01
Month: 05 Month: 01
Month: 06
:day_24 :day_25
Day: 24 Day: 25
:day_26
Day: 26
:happens :happens :happens :happens
Event 1 Event 2 Event 3](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-136-320.jpg)
![find all events on a specific day
START root=node(0)
MATCH root - [:year_2014] -> () -[:month_05] ->
()- [:day_24] -> () - [:happens] -> event
RETURN event root (0)
:year_2014
Year: 2013
:year_2015
Year: 2015
:month_06
:month_05 :month_01
Month: 05 Month: 01
Month: 06
:day_24 :day_25
Day: 24 Day: 25
:day_26
Day: 26
:happens :happens :happens :happens
Event 1 Event 2 Event 3](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-137-320.jpg)
![find all events on a specific day
START root=node(0)
MATCH root - [:year_2014] -> () -[:month_05] ->
()- [:day_24] -> () - [:happens] -> event
RETURN event root (0)
:year_2014
Year: 2013
:year_2015
Year: 2015
:month_06
:month_05 :month_01
Month: 05 Month: 01
Month: 06
:day_24 :day_25
Day: 24 Day: 25
:day_26
Day: 26
:happens :happens :happens :happens
Event 1 Event 2 Event 3](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-138-320.jpg)
![all together
MATCH (store{ id:’ebay_store’ }) -[:located]
-> (locality) <- [:operates]- carrier -
[available:available] -> () <-
[:hour_10] - () <- [:day_24] - ()
[:month_05] - () [:year_2014] - ()
RETURN carrier, available.premium as premium](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-140-320.jpg)
![all together
MATCH (store{ id:’ebay_store’ }) -[:located]
-> (locality) <- [:operates]- carrier -
[available:available] -> () <-
[:hour_10] - () <- [:day_24] - ()
[:month_05] - () [:year_2014] - ()
RETURN carrier, available.premium as premium](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-141-320.jpg)
![all together
MATCH (store{ id:’ebay_store’ }) -[:located]
-> (locality) <- [:operates]- carrier -
[available:available] -> () <-
[:hour_10] - () <- [:day_24] - ()
[:month_05] - () [:year_2014] - ()
RETURN carrier, available.premium as premium
Locality
id = california
Locality
id = marin_county
:contains
Locality
id = 94901
:contains
Store
id = ebay_store
:located
:available {premium: 1.5}
Carrier
id = carrier_1
:operates
root (0)
:year_2014
Year: 2013
:month_05
Month: 05
:day_24
Day: 24
hour 09
:hour_09
hour 10
:hour_10
hour 11 :hour_11
:available {premium: 1}](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-142-320.jpg)
![Testing was a challenge
• Setting up scenarios for tests was tedious
• Built our own tool based on the geoff syntax developed by Nigel Small
• Geoff allows modelling of graphs in textual form and provides an
interface to insert them into an existing graph
(A) {“name”: “Alice”}
(B) {“name”: “Bob”}
(A) -[:KNOWS] -> (B)](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-158-320.jpg)
![Testing was a challenge
• Setting up scenarios for tests was tedious
• Built our own tool based on the geoff syntax developed by Nigel Small
• Geoff allows modelling of graphs in textual form and provides an
interface to insert them into an existing graph
(A) {“name”: “Alice”}
(B) {“name”: “Bob”}
(A) -[:KNOWS] -> (B)
• We created a Ruby dsl for modelling a graph and inserting it into the db
that works with factory_girl](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-159-320.jpg)
![Testing was a challenge
• Setting up scenarios for tests was tedious
• Built our own tool based on the geoff syntax developed by Nigel Small
• Geoff allows modelling of graphs in textual form and provides an
interface to insert them into an existing graph
(A) {“name”: “Alice”}
(B) {“name”: “Bob”}
(A) -[:KNOWS] -> (B)
• We created a Ruby dsl for modelling a graph and inserting it into the db
that works with factory_girl
• Open source - https://github.com/shutl/geoff](https://image.slidesharecdn.com/untitled-140922062958-phpapp02/85/How-Shutl-Delivers-Even-Faster-Using-Neo4J-160-320.jpg)
Ad
More Related Content
Viewers also liked (14)
Similar to How Shutl Delivers Even Faster Using Neo4J (20)
Ad
More from C4Media (20)
Ad
Recently uploaded (20)
How Shutl Delivers Even Faster Using Neo4J
- 2. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /shutl-neo4j-graph-db InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month
- 3. Presented at QCon London www.qconlondon.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
- 4. How Shutl Delivers Even Faster Using Neo4j Sam Phillips and Volker Pacher @samsworldofno @vpacher
- 5. Sam Phillips Volker Pacher
- 7. Graphs at Shutl • Graph databases are awesome
- 8. Graphs at Shutl • Graph databases are awesome • We’ve seen lots of the talks about modelling
- 9. Graphs at Shutl • Graph databases are awesome • We’ve seen lots of the talks about modelling • But querying is important too
- 10. Graphs at Shutl • Graph databases are awesome • We’ve seen lots of the talks about modelling • But querying is important too • So let’s talk about querying too!
- 11. Show of hands
- 12. Show of hands • Who has used graph databases before?
- 13. Show of hands • Who has used graph databases before? • Who has used Neo4j before?
- 14. Shutl
- 15. Shutl
- 16. ECOMMERCE IS QUICK & CONVENIENT
- 17. ECOMMERCE IS QUICK & CONVENIENT
- 18. PAYPAL FOR AWESOME DELIVERY
- 19. PAYPAL FOR AWESOME DELIVERY
- 20. PAYPAL FOR AWESOME DELIVERY Branded, super quick delivery that people trust, embedded in merchant websites
- 21. HUB & SPOKE A B
- 22. HUB & SPOKE A B
- 23. HUB & SPOKE A B Only cost effective means to deliver 10+ miles but slow and unpredictable
- 24. HUB & SPOKE Only cost effective means to deliver 10+ miles but slow and unpredictable A B
- 25. HUB & SPOKE Only cost effective means to deliver 10+ miles but slow and unpredictable A B POINT TO POINT A B
- 26. HUB & SPOKE Only cost effective means to deliver 10+ miles but slow and unpredictable A B POINT TO POINT A B Fast and predictable but cost prohibitive over longer distances
- 27. HUB & SPOKE 97% Courier, Express & Parcel Market
- 28. POINT TO POINT 3% Courier, Express & Parcel Market
- 29. POINT TO POINT +7,500 more! 3% Courier, Express & Parcel Market
- 30. POINT TO POINT
- 31. SHOP
- 32. Shutl generates a quote from each relevant carrier within platform SHOP $$ $$$ $ $$ $ $
- 33. Shutl generates a quote from each relevant carrier within platform Optimum picked based on price & quality rating SHOP $$ $$$ $ $$ $ $$
- 34. SHOP SHOP
- 35. On checkout, delivery sent via API into chosen carrier’s transportation system SHOP $$ SHOP
- 36. On checkout, delivery sent via API into chosen carrier’s transportation system Courier collects from nearest store and delivers to shopper SHOP $$
- 38. Delivery status updated in real-time, performance compared against SLA & carrier quality rating updated Better performing carriers get more deliveries & can demand higher prices
- 39. Delivery status updated in real-time, performance compared against SLA & carrier quality rating updated Better performing carriers get more deliveries & can demand higher prices
- 40. Delivery status updated in real-time, performance compared against SLA & carrier quality rating updated Better performing carriers get more deliveries & can demand higher prices
- 41. Track your order online…
- 42. FEEDBACK Quality paramount since we are motivated by LTV of shopper
- 43. FEEDBACK Quality paramount since we are motivated by LTV of shopper
- 44. FEEDBACK Shutl sends feedback email to consumer seconds after they have received delivery asking to rate qualitative aspects of experience
- 45. FEEDBACK Feedback streamed unedited to shutl.com/feedback & facebook
- 46. FEEDBACK
- 47. FEEDBACK
- 48. FEEDBACK
- 49. FEEDBACK
- 51. SHUTL IS NOW AN COMPANY
- 52. Version One Ruby 1.8, Rails 2.3 and MySQL
- 53. Version One Ruby 1.8, Rails 2.3 and MySQL
- 54. Version One Ruby 1.8, Rails 2.3 and MySQL • Well-known tale: built quickly, worked slowly, tough to maintain • Getting a quote for an hour time-slot took over 4 seconds
- 55. Here is the Shutl price calendar
- 56. Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds)
- 57. Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds) Then, they would have to call Shutl to generate a quote for each slot - for two days of store opening, that’s 20+ slots
- 58. Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds) Then, they would have to call Shutl to generate a quote for each slot - for two days of store opening, that’s 20+ slots So, that’s 2 + (20 x 4) seconds, 1:22 to generate the data for this calendar
- 59. Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds) Then, they would have to call Shutl to generate a quote for each slot - for two days of store opening, that’s 20+ slots So, that’s 2 + (20 x 4) seconds, 1:22 to generate the data for this calendar In V1, this UX could never have happened.
- 60. V2
- 61. • Broke app into services V2 • Services focused around functions like quoting, booking, and giving feedback • Key goal for the project was improving the speed of the quoting operation, which is where we used graph databases
- 62. V1 V2
- 63. V1 V2 • Quoting for 20 windows down from 82000 ms to 800 ms
- 64. V1 V2 • Quoting for 20 windows down from 82000 ms to 800 ms • Code complexity much reduced
- 65. V1 V2 • Quoting for 20 windows down from 82000 ms to 800 ms • Code complexity much reduced
- 66. A large part of the success of our rewrite was down to the graph database.
- 67. What is a graph anyway?
- 69. a simple graph a collection of vertices (nodes) connected by edges (relationships)
- 70. a short history the seven bridges of Königsberg (1735)! Leonard Euler
- 71. the seven bridges of Königsberg (1735)!
- 72. the seven bridges of Königsberg (1735)!
- 73. the seven bridges of Königsberg (1735)!
- 74. the seven bridges of Königsberg (1735)!
- 75. the seven bridges of Königsberg (1735)!
- 76. Euler walk each node has an even degree
- 77. Euler walk
- 78. Euler walk
- 79. Euler walk two nodes have an odd degree
- 80. Euler walk two nodes have an odd degree
- 81. Euler walk no two nodes have an odd degree
- 82. directed graph each relationship has a direction or one start node and one end node
- 83. property graph nodes contain properties (key, value) relationships have a type and are always directed relationships can contain properties too Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for
- 84. The Case for Graph Databases
- 85. relationships are explicit stored
- 88. traversals of relationships are easy and very fast
- 89. DB performance remains relatively constant as queries are localised to its portion of the graph. O(1) for same query
- 90. a graph is its own index (constant query performance)
- 91. a graph is its own index (constant query performance)
- 92. a graph is its own index (constant query performance)
- 93. the case for Neo4j
- 94. standalone or embedded in jvm
- 95. ruby/jruby
- 96. ruby libraries - neo4j gem by Andreas Ronge (https://github.com/andreasronge/neo4j)
- 97. cypher
- 98. the neotech guys are awesome
- 99. Querying the graph: Cypher declarative query language specific to neo4j easy to learn and intuitive use specific patterns to query for (something that looks like ‘this’) inspired partly by SQL (WHERE and ORDER BY) and SPARQL (pattern matching) focuses on what to query for and not how to query for it switch from a mySQl world is made easier by the use of cypher instead of having to learn a traversal framework straight away
- 100. cypher clauses START: Starting points in the graph, obtained via index lookups or by element IDs. MATCH: The graph pattern to match, bound to the starting points in START. WHERE: Filtering criteria. RETURN: What to return. CREATE: Creates nodes and relationships. DELETE: Removes nodes, relationships and properties. SET: Set values to properties. FOREACH: Performs updating actions once per element in a list. WITH: Divides a query into multiple, distinct parts
- 101. cypher clauses START: Starting points in the graph, obtained via index lookups or by element IDs. MATCH: The graph pattern to match, bound to the starting points in START. WHERE: Filtering criteria. RETURN: What to return. CREATE: Creates nodes and relationships. DELETE: Removes nodes, relationships and properties. SET: Set values to properties. FOREACH: Performs updating actions once per element in a list. WITH: Divides a query into multiple, distinct parts
- 102. an example Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
- 103. find all the companies my friends work for MATCH (person{ name:’Volker’ }) -[:friends] - (person) - [:works_for]-> company RETURN company
- 104. find all the companies my friends work for MATCH (person{ name:’Volker’ }) -[:friends] - (person) - [:works_for]-> company RETURN company
- 105. find all the companies my friends work for MATCH (person{ name:’Volker’ }) -[:friends] - (person) - [:works_for]-> company RETURN company Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
- 106. find all the companies my friend’s friends work for MATCH (person{ name:’Volker’ }) - [:friends*2..2]-(person) - [:works_for] -> company RETURN company
- 107. find all the companies my friend’s friends work for MATCH (person{ name:’Volker’ }) - [:friends*2..2]-(person) - [:works_for] -> company RETURN company
- 108. find all the companies my friend’s friends work for MATCH (person{ name:’Volker’ }) - [:friends*2..2]-(person) - [:works_for] -> company RETURN company Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
- 109. find all my friends who work for neotech MATCH (person{ name:’Volker’ }) -[:friends] -(friends) - [:works_for]-> company WHERE company.name = ‘neotech’ RETURN friends
- 110. find all my friends who work for neotech MATCH (person{ name:’Volker’ }) -[:friends] -(friends) - [:works_for]-> company WHERE company.name = ‘neotech’ RETURN friends
- 111. find all my friends who work for neotech MATCH (person{ name:’Volker’ }) -[:friends] -(friends) - [:works_for]-> company WHERE company.name = ‘neotech’ RETURN friends Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
- 112. a good place to try it out: ! http://console.neo4j.org/ ! http://gist.neo4j.org/
- 113. coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains :contains Locality id = 94903 :contains :contains Locality id = 94902
- 114. coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates :operates Carrier id = carrier_1
- 115. coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates :operates Carrier id = carrier_1
- 116. coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
- 117. MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier RETURN carrier the query
- 118. the query MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier RETURN carrier
- 119. MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = 94902 Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains :operates :operates Carrier id = carrier_1
- 120. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query
- 121. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query
- 122. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
- 123. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
- 124. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
- 125. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
- 126. SELECT * FROM carriers LEFT JOIN locations ON carrier.location_id = location.id LEFT JOIN stores ON stores.location_id = carrier.location_id WHERE stores.name = ‘ebay_store’
- 127. SELECT * FROM carriers LEFT JOIN locations ON carrier.location_id = location.id OR carrier.location_id = location.parent_id LEFT JOIN stores ON stores.location_id = carrier.location_id WHERE stores.name = ‘ebay_store’
- 128. ?
- 129. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier
- 130. representing dates/times root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
- 131. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event
- 132. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event
- 133. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
- 134. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
- 135. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
- 136. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
- 137. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
- 138. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
- 139. all together Locality id = california Locality id = marin_county :contains Locality id = 94901 :contains Store id = ebay_store :located :available {premium: 1.5} Carrier id = carrier_1 :operates root (0) :year_2014 Year: 2013 :month_05 Month: 05 :day_24 Day: 24 hour 09 :hour_09 hour 10 :hour_10 hour 11 :hour_11 :available {premium: 1}
- 140. all together MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier - [available:available] -> () <- [:hour_10] - () <- [:day_24] - () [:month_05] - () [:year_2014] - () RETURN carrier, available.premium as premium
- 141. all together MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier - [available:available] -> () <- [:hour_10] - () <- [:day_24] - () [:month_05] - () [:year_2014] - () RETURN carrier, available.premium as premium
- 142. all together MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier - [available:available] -> () <- [:hour_10] - () <- [:day_24] - () [:month_05] - () [:year_2014] - () RETURN carrier, available.premium as premium Locality id = california Locality id = marin_county :contains Locality id = 94901 :contains Store id = ebay_store :located :available {premium: 1.5} Carrier id = carrier_1 :operates root (0) :year_2014 Year: 2013 :month_05 Month: 05 :day_24 Day: 24 hour 09 :hour_09 hour 10 :hour_10 hour 11 :hour_11 :available {premium: 1}
- 143. Other graph uses
- 144. Other graph uses • Recommendation engines
- 145. Other graph uses • Recommendation engines • Organisational analysis
- 146. Other graph uses • Recommendation engines • Organisational analysis • Graphing your infrastructure
- 147. Some gotchas
- 148. Some gotchas • There was a learning curve in switching from a relational mentality to a graph one
- 149. Some gotchas • There was a learning curve in switching from a relational mentality to a graph one • Tooling not as mature as in the relational world
- 150. Some gotchas • There was a learning curve in switching from a relational mentality to a graph one • Tooling not as mature as in the relational world • No out of the box solution for db migrations
- 151. Some gotchas • There was a learning curve in switching from a relational mentality to a graph one • Tooling not as mature as in the relational world • No out of the box solution for db migrations • Seeding an embedded database was unfamiliar
- 152. Testing was a challenge
- 153. Testing was a challenge • Setting up scenarios for tests was tedious
- 154. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small
- 155. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph
- 156. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”}
- 157. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”}
- 158. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”} (A) -[:KNOWS] -> (B)
- 159. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”} (A) -[:KNOWS] -> (B) • We created a Ruby dsl for modelling a graph and inserting it into the db that works with factory_girl
- 160. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”} (A) -[:KNOWS] -> (B) • We created a Ruby dsl for modelling a graph and inserting it into the db that works with factory_girl • Open source - https://github.com/shutl/geoff
- 161. Wrap Up
- 162. Wrap Up • Neo4j and graph theory enabled Shutl to achieve big performance increases in its most important operation - calculating delivery prices
- 163. Wrap Up • Neo4j and graph theory enabled Shutl to achieve big performance increases in its most important operation - calculating delivery prices • It’s a new tool based on tested theory, and cypher is the first language that allows you to query graphs in a declarative way (like SQL)
- 164. Wrap Up • Neo4j and graph theory enabled Shutl to achieve big performance increases in its most important operation - calculating delivery prices • It’s a new tool based on tested theory, and cypher is the first language that allows you to query graphs in a declarative way (like SQL) • Tooling and adoption is immature but getting better all the time
- 165. Thank you! ! Any questions? Sam Phillips Head of Engineering ! @samsworldofno http://samsworldofno.com [email protected] Volker Pacher Senior Developer ! @vpacher https://github.com/vpacher [email protected]
- 167. our
- 168. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/shutl-neo4j- graph-db