SlideShare a Scribd company logo
Persistent Graphs in
                   Python with Neo4j

                                 twitter: @thobe / #neo4j
       Tobias Ivarsson           email: tobias@neotechnology.com
                                 web: http://www.neo4j.org/
       Hacker @ Neo Technology   web: http://www.thobe.org/

Sunday, February 21, 2010
We all know the
                                                                                relational model.
           Attendees                                                            It has been predominant
                                                                                for a long time.
           username              fullname       registration    tutorials   payment


          guido             Guido van Rossum   null            yes          0

          thobe             Tobias Ivarsson    2009-12-12      no           300

          joe               John Doe           2010-02-05      yes          700

                   ...               ...              ...            ...             ...




                                                                                       2

Sunday, February 21, 2010
Attendees
                                                                                                        The relational model has
                            username         fullname        registration    tutorials     payment      a few problems, such as:
                                                                                                        •poor support for sparse
                                                                                                        data
                                                                                                        •modifying the data
                            guido       Guido van Rossum    null            yes           0             model is almost
                                                                                                        exclusively done through
                                                                                                        adding tables
                            thobe       Tobias Ivarsson     2009-12-12      no            300

                            joe         John Doe            2010-02-05      yes           700

                                  ...            ...                 ...           ...          ...

                            Location
                            username        latitude         longitude            title       publish


                            thobe       55°36'47.70"N     12°58'34.50"E     Malmö         yes

                                                                            San
                            joe         37°49'36.00"N     122°25'22.00"W                  no
                                                                            Francisco

                                  ...          ...                 ...             ...          ...

                                                                                                              3

Sunday, February 21, 2010
Attendees                                                                      Sessions
             username            fullname        registration    tutorials     payment      id       title time room ...

                                                                                            ...        ...        ...   ...        ...
            guido           Guido van Rossum    null            yes           0
                                                                                            ...        ...        ...   ...        ...
            thobe           Tobias Ivarsson     2009-12-12      no            300
                                                                                            Session attendance
            joe             John Doe            2010-02-05      yes           700                 session               user

                  ...                ...                 ...           ...          ...             ...                  ...

             Location                                                                               ...                  ...
             username           latitude         longitude            title       publish

                                                                                            More complication...
            thobe           55°36'47.70"N     12°58'34.50"E     Malmö         yes                   ...                  ...
                                                                                                      ...                  ...
     After a while, modeling                                                                             ...                  ...
     complex relationships                                                                                 ...                  ...
     leads to complicated
                                                                                                    ......               ......
                                                                San                                      ......
     schemasjoe            37°49'36.00"N      122°25'22.00"W                  no                                              ......
                                                                Francisco                           ......               ......
                                                                                                         ......               ......
                  ...              ...                 ...             ...          ...

                                                                                                                          4

Sunday, February 21, 2010
A number of companies
                            have realized that the
                            relational model is
                            insufficient and are
                            working on alternative
                            database solutions.



                                    5

Sunday, February 21, 2010
Most focus on scaling to large numbers



                                           192.168.0.15   192.168.0.16




                                                                         192.168.0.21
                            192.168.0.14




                                                                                        6

Sunday, February 21, 2010
Graph Databases focuses on structure of data




                                                 7

Sunday, February 21, 2010
Positioning w.r.t. other NOSQL DBs
                      Size
                            Key/Value stores

                                               Bigtable clones

                                                                 Document databases

                                                                                      Graph databases




                                                                                                Complexity

                                                                                                        8

Sunday, February 21, 2010
Positioning w.r.t. other NOSQL DBs
                      Size
                            Key/Value stores

                                               Bigtable clones

                                                                 Document databases

                                                                                      Graph databases
                                                                                                  Billions of nodes
                                                                                                  and relationships




                                                     > 90% of use cases

                                                                                                Complexity

                                                                                                         8

Sunday, February 21, 2010
What is Neo4j?
        ๏ Neo4j is a Graph Database
                 • Non-relational (“#nosql”), transactional (ACID), embedded
                 • Data is stored as a Graph / Network
                       ‣Nodes and relationships with properties
                       ‣“Property Graph” or “edge-labeled multidigraph”
        ๏ Neo4j is Open Source / Free (as in speech) Software
                 • AGPLv3
                                                                          Prices are available at
                                                                          http://neotechnology.com/



                 • Commercial (“dual license”) license available
                                                                          Contact us if you have
                                                                          questions and/or special
                                                                          license needs (e.g. if you
                                                                          want an evaluation license)

                       ‣Free (as in beer) for “small” installations
                       ‣Inexpensive (as in startup-friendly) when you grow           9

Sunday, February 21, 2010
More about Neo4j
        ๏ Neo4j is stable
                 • In 24/7 operation since 2003
        ๏ Neo4j is in active development
                 • Neo Technology got VC funding October 2009
        ๏ Neo4j delivers high performance graph operations
                 • traverses 1’000’000+ relationships / second
                     on commodity hardware




                                                                 10

Sunday, February 21, 2010
The Neo4j Graph data model
        ๏ Nodes are connected to one another through relationships
        ๏ A Relationship is a connection between two nodes
                 • Relationships have types
                 • Relationships have a direction
                 • Relationships are traversed equally fast in either direction
        ๏ Properties are mappings from a string key to a primitive value
                 • Both Nodes and Relationships have properties
                 • Primitive values are any of these (or an array of these):
                       ‣String
                       ‣Numbers: float, double, integers (1-8 byte)             11

Sunday, February 21, 2010
The Neo4j Graph data model
                                                                       name: “Mary”
                                                        LOVES
                   name: “James”                                       age: 35
                   age: 32                   LIVES WITH
                   twitter: “@spam”                   LOVES



                                      OWNS
                             property type: “car”             DRIVES



                                                      brand: “Volvo”
                                                      model: “V70”



                                                                                12

Sunday, February 21, 2010
Graphs are all around us
                            A                        B           C             D           ...
                  1                 17                  3.14          3   17.79333333333

                  2                 42               10.11           14            30.33

                  3              316                    6.66          1          2104.56

                  4                 32                  9.11     592      0.492432432432

                  5         Even if this spread sheet looks
                            like it could be a fit for a RDBMS
                                                                          2153.175765766
                            it isn’t:
                            •RDBMSes have problems with
                 ...        extending indefinitely on both
                            rows and collumns
                            •Formulas and data
                            dependencies would quickly lead
                            to heavy join operations

                                                                                           13

Sunday, February 21, 2010
Graphs are all around us
                            A      B      C         D            ...
                  1         17    3.14     3    = A1 * B1 / C1

                  2         42    10.11   14    = A2 * B2 / C2

                  3         316   6.66     1    = A3 * B3 / C3

                  4         32    9.11    592   = A4 * B4 / C4

                  5                             = SUM(D2:D5)

                 ...

                                                                 14

Sunday, February 21, 2010
Graphs are all around us
                            A      B      C         D            ...
                  1         17    3.14     3    = A1 * B1 / C1

                  2         42    10.11   14    = A2 * B2 / C2

                  3         316   6.66     1    = A3 * B3 / C3

                  4         32    9.11    592   = A4 * B4 / C4

                  5                             = SUM(D2:D5)

                 ...

                                                                 14

Sunday, February 21, 2010
Graphs are all around us                              If we add external data
                                                             sources the problem
                                                             becomes even more
                                                             interesting...




                            17    3.14     3    = A1 * B1 / C1

                            42    10.11   14    = A2 * B2 / C2

                            316   6.66     1    = A3 * B3 / C3

                            32    9.11    592   = A4 * B4 / C4

                                                = SUM(D2:D5)




                                                                     15

Sunday, February 21, 2010
Graphs are all around us                              If we add external data
                                                             sources the problem
                                                             becomes even more
                                                             interesting...




                            17    3.14     3    = A1 * B1 / C1

                            42    10.11   14    = A2 * B2 / C2

                            316   6.66     1    = A3 * B3 / C3

                            32    9.11    592   = A4 * B4 / C4

                                                = SUM(D2:D5)




                                                                     15

Sunday, February 21, 2010
Graphs are whiteboard friendly




                                        16

Sunday, February 21, 2010
Graphs are whiteboard friendly


                                        *
                                1
                                            *
                            *       1




                            *                   1
                                    *

                                1
                                        *



                                                    16

Sunday, February 21, 2010
Graphs are whiteboard friendly

                                   thobe



                                              Joe project blog


                                             Wardrobe Strength


                            Hello Joe

                            Modularizing Jython

                               Neo4j performance analysis

                                                                 16

Sunday, February 21, 2010
Query Languages
        ๏ Traversal API
        ๏ Sparql - “SQL for linked data”
               SELECT ?person WHERE {
                   ?person neo4j:KNOWS ?friend .
                   ?friend neo4j:KNOWS ?foe .
                   ?foe neo4j:name “Larry Ellison” .
               }

        ๏ Gremlin - “perl for graphs”
               ./outE[@label='KNOWS']/inV[@age > 30]/@name




                                                             17

Sunday, February 21, 2010
Python integration for Neo4j
        ๏ Mapping of the core Neo4j API for Python
                 • Making it feel “Pythonic”
        ๏ Available from the Neo4j repository (and soon from PyPI)
                 • http://components.neo4j.org/neo4j.py
                       ‣svn co http://svn.neo4j.org/components/neo4j.py/trunk neo4j-python
        ๏ Works with both Jython and CPython
                 • The threading of Jython is a plus with an embedded db...
        ๏ Comes with Django empowering batteries included
                 • Could have support for other frameworks in the future
                                                                                     18

Sunday, February 21, 2010
Simple interaction
       import neo4j
       graphdb = neo4j.GraphDatabase(“var/neo”)

       with graphdb.transaction:
          james = graphdb.node(name=“James”, age=32, twitter=“@spam”)
          mary = graphdb.node(name=“Mary”, age=35)
          the_car = graphdb.node(brand=“Volvo”, model=“V70”)

               james.LOVES( mary )
               mary.LOVES( james )
               james.LIVES_WITH( mary )
               james.OWNS( the_car, property_type=“car” )   Creates the graph we saw
                                                            in the first example.
               mary.DRIVES( the_car )




                                                                            19

Sunday, February 21, 2010
Graph traversals                                                                         name: “The Architect”
                                                  disclosure: “public”
       name: “Thomas Anderson”
       age: 29                                                            name: “Cypher”
                                                                          last name: “Reagan”
                               KNOWS name: “Morpheus”
                         KNOWS                                  KNOWS
                                     rank: “Captain”                                                  CODED BY
                   LOVES             occupation: “Total badass”                          KNOWS
                                         KNOWS
                       name: “Trinity”                            disclosure: “secret”
                                                                                            name: “Agent Smith”
                                                                                            version: “1.0b”
          since: “meeting the oracle”            since: “a year before the movie”
                                                                                            language: “C++”
                                                 cooperates on: “The Nebuchadnezzar”
       import neo4j
       class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
          types = [ neo4j.Outgoing.KNOWS ]
          order = neo4j.BREDTH_FIRST
          stop = neo4j.STOP_AT_END_OF_GRAPH
          returnable = neo4j.RETURN_ALL_BUT_START_NODE

       for friend_node in Friends(mr_anderson):
          print “%s (@ depth=%s)” % ( friend_node[“name”],
             friend_node.depth )                                                                         20

Sunday, February 21, 2010
Graph traversals                                                                         name: “The Architect”
                                                  disclosure: “public”
       name: “Thomas Anderson”
       age: 29                                                            name: “Cypher”
                                                                          last name: “Reagan”
                               KNOWS name: “Morpheus”
                         KNOWS                                  KNOWS
                                     rank: “Captain”                                                  CODED BY
                   LOVES             occupation: “Total badass”                          KNOWS
                                         KNOWS
                       name: “Trinity”                            disclosure: “secret”
                                                                                            name: “Agent Smith”
                                                                                            version: “1.0b”
          since: “meeting the oracle”            since: “a year before the movie”
                                                                                            language: “C++”
                                                 cooperates on: “The Nebuchadnezzar”
       import neo4j
       class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
          types = [ neo4j.Outgoing.KNOWS ]
          order = neo4j.BREDTH_FIRST
          stop = neo4j.STOP_AT_END_OF_GRAPH
          returnable = neo4j.RETURN_ALL_BUT_START_NODE

       for friend_node in Friends(mr_anderson):
          print “%s (@ depth=%s)” % ( friend_node[“name”],
             friend_node.depth )                                                                         20

Sunday, February 21, 2010
Graph traversals                                                                         name: “The Architect”
                                                  disclosure: “public”
       name: “Thomas Anderson”
       age: 29                                                            name: “Cypher”
                                                                          last name: “Reagan”
                               KNOWS name: “Morpheus”
                         KNOWS                                  KNOWS
                                     rank: “Captain”                                                  CODED BY
                   LOVES             occupation: “Total badass”                          KNOWS
                                         KNOWS
                       name: “Trinity”                            disclosure: “secret”
                                                                                            name: “Agent Smith”
                                                                                            version: “1.0b”
          since: “meeting the oracle”            since: “a year before the movie”
                                                                                            language: “C++”
                                                 cooperates on: “The Nebuchadnezzar”
       import neo4j
       class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
          types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
          order = neo4j.BREDTH_FIRST
          stop = neo4j.STOP_AT_END_OF_GRAPH
          returnable = neo4j.RETURN_ALL_BUT_START_NODE

       for friend_node in Friends(mr_anderson):
          print “%s (@ depth=%s)” % ( friend_node[“name”],
             friend_node.depth )                                                                         20

Sunday, February 21, 2010
Graph traversals                                                                         name: “The Architect”
                                                  disclosure: “public”
       name: “Thomas Anderson”
       age: 29                                                            name: “Cypher”
                                                                          last name: “Reagan”
                               KNOWS name: “Morpheus”
                         KNOWS                                  KNOWS
                                     rank: “Captain”                                                  CODED BY
                   LOVES             occupation: “Total badass”                          KNOWS
                                         KNOWS
                       name: “Trinity”                            disclosure: “secret”
                                                                                            name: “Agent Smith”
                                                                                            version: “1.0b”
          since: “meeting the oracle”            since: “a year before the movie”
                                                                                            language: “C++”
                                                 cooperates on: “The Nebuchadnezzar”
       import neo4j
       class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
          types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
          order = neo4j.BREDTH_FIRST                     Trinity (@ depth=1)
          stop = neo4j.STOP_AT_END_OF_GRAPH
          returnable = neo4j.RETURN_ALL_BUT_START_NODE

       for friend_node in Friends(mr_anderson):
          print “%s (@ depth=%s)” % ( friend_node[“name”],
             friend_node.depth )                                                                         20

Sunday, February 21, 2010
Graph traversals                                                                         name: “The Architect”
                                                  disclosure: “public”
       name: “Thomas Anderson”
       age: 29                                                            name: “Cypher”
                                                                          last name: “Reagan”
                               KNOWS name: “Morpheus”
                         KNOWS                                  KNOWS
                                     rank: “Captain”                                                  CODED BY
                   LOVES             occupation: “Total badass”                          KNOWS
                                         KNOWS
                       name: “Trinity”                            disclosure: “secret”
                                                                                            name: “Agent Smith”
                                                                                            version: “1.0b”
          since: “meeting the oracle”            since: “a year before the movie”
                                                                                            language: “C++”
                                                 cooperates on: “The Nebuchadnezzar”
       import neo4j
       class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
          types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
          order = neo4j.BREDTH_FIRST                     Trinity (@ depth=1)
          stop = neo4j.STOP_AT_END_OF_GRAPH
                                                         Cypher (@ depth=2)
          returnable = neo4j.RETURN_ALL_BUT_START_NODE

       for friend_node in Friends(mr_anderson):
          print “%s (@ depth=%s)” % ( friend_node[“name”],
             friend_node.depth )                                                                         20

Sunday, February 21, 2010
Graph traversals                                                                         name: “The Architect”
                                                  disclosure: “public”
       name: “Thomas Anderson”
       age: 29                                                            name: “Cypher”
                                                                          last name: “Reagan”
                               KNOWS name: “Morpheus”
                         KNOWS                                  KNOWS
                                     rank: “Captain”                                                  CODED BY
                   LOVES             occupation: “Total badass”                          KNOWS
                                         KNOWS
                       name: “Trinity”                            disclosure: “secret”
                                                                                            name: “Agent Smith”
                                                                                            version: “1.0b”
          since: “meeting the oracle”            since: “a year before the movie”
                                                                                            language: “C++”
                                                 cooperates on: “The Nebuchadnezzar”
       import neo4j
       class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
          types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
          order = neo4j.BREDTH_FIRST                     Trinity (@ depth=1)
          stop = neo4j.STOP_AT_END_OF_GRAPH
                                                         Cypher (@ depth=2)
          returnable = neo4j.RETURN_ALL_BUT_START_NODE
                                                                                           Agent Smith (@ depth=3)

       for friend_node in Friends(mr_anderson):
          print “%s (@ depth=%s)” % ( friend_node[“name”],
             friend_node.depth )                                                                         20

Sunday, February 21, 2010
Graph traversals                                                                         name: “The Architect”
                                                  disclosure: “public”
       name: “Thomas Anderson”
       age: 29                                                            name: “Cypher”
                                                                          last name: “Reagan”
                               KNOWS name: “Morpheus”
                         KNOWS                                  KNOWS
                                     rank: “Captain”                                                  CODED BY
                   LOVES             occupation: “Total badass”                          KNOWS
                                         KNOWS
                       name: “Trinity”                            disclosure: “secret”
                                                                                            name: “Agent Smith”
                                                                                            version: “1.0b”
          since: “meeting the oracle”            since: “a year before the movie”
                                                                                            language: “C++”
                                                 cooperates on: “The Nebuchadnezzar”
       import neo4j
       class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
          types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
          order = neo4j.BREDTH_FIRST                     Trinity (@ depth=1)
          stop = neo4j.STOP_AT_END_OF_GRAPH
                                                         Cypher (@ depth=2)
          returnable = neo4j.RETURN_ALL_BUT_START_NODE
                                                                                           Agent Smith (@ depth=3)

       for friend_node in Friends(mr_anderson):
          print “%s (@ depth=%s)” % ( friend_node[“name”],
             friend_node.depth )                                                                         20

Sunday, February 21, 2010
Batteries for Django
       from neo4j.model import django_model as models

       class Movie(models.NodeModel):
          title = models.Property(indexed=True)
          year = models.Property()
          href = property(lambda self: ('/movie/%s/' %
             (self.node.id,)))
          def __unicode__(self):
              return self.title

       class Actor(models.NodeModel):
          name = models.Property(indexed=True)
          href = property(lambda self: ('/actor/%s/' %
             (self.node.id,)))
          def __unicode__(self):
              return self.name

       # etc. ...

                                                         21

Sunday, February 21, 2010
“My ORM already does this”
        ๏ ORMs and model evolution is a hard problem
                 • virtually unsupported in Django
        ๏ SQL is a “compatible” across many RDBMSs
                 • data is still locked in
        ๏ Each ORM maps object models differently
                 • Moving to another ORM == legacy schema support
                       ‣except your legacy schema is strange auto-generated
        ๏ Object/Graph Mapping is always done the same
                 • allows you to keep your data through application changes
                 • or share data between multiple implementations             22

Sunday, February 21, 2010
What your ORM doesn’t do
        ๏ Drop down to underlying graph model
                 • Traversals
                 • Graph algorithms
                 • Shortest path(s)
                 • etc.



                                                23

Sunday, February 21, 2010
Buzzword summary                                                         http://neo4j.org/



                                     SPARQL
               AGPLv3
                                                  Open Source
                                                                        ACID
                             Object mapping                                    Shortest path
                                                     NOSQL
                            startup friendly
                                                                 whiteboard friendly
                Traversal
                                                  Query language
                               Embedded
                                                                     Beer

                                               Software Transactional Memory
            polyglot persistence
                                                             Free Software
                                 Scaling to complexity
                                                                                       24

Sunday, February 21, 2010
http://neotechnology.com

Sunday, February 21, 2010
Ad

More Related Content

Viewers also liked (8)

Sistemas de recomendações e neo4J na cloud computing
Sistemas de recomendações e neo4J na cloud computingSistemas de recomendações e neo4J na cloud computing
Sistemas de recomendações e neo4J na cloud computing
Priscila Mayumi
 
No sql std
No sql stdNo sql std
No sql std
Arthur Azevedo
 
Neo4j - Rede de relacionamentos baseada em grafos
Neo4j - Rede de relacionamentos baseada em grafosNeo4j - Rede de relacionamentos baseada em grafos
Neo4j - Rede de relacionamentos baseada em grafos
Evandro Venancio
 
Introdução ao neo4j
Introdução ao neo4jIntrodução ao neo4j
Introdução ao neo4j
Jhonathan de Souza Soares
 
GraphConnect Europe 2016 - NoSQL Polyglot Persistence: Tools and Integrations...
GraphConnect Europe 2016 - NoSQL Polyglot Persistence: Tools and Integrations...GraphConnect Europe 2016 - NoSQL Polyglot Persistence: Tools and Integrations...
GraphConnect Europe 2016 - NoSQL Polyglot Persistence: Tools and Integrations...
Neo4j
 
Neo4j Spatial - GIS for the rest of us.
Neo4j Spatial - GIS for the rest of us.Neo4j Spatial - GIS for the rest of us.
Neo4j Spatial - GIS for the rest of us.
Peter Neubauer
 
Banco de dados de grafos
Banco de dados de grafosBanco de dados de grafos
Banco de dados de grafos
Priscila Mayumi
 
OrientDB vs Neo4j - Comparison of query/speed/functionality
OrientDB vs Neo4j - Comparison of query/speed/functionalityOrientDB vs Neo4j - Comparison of query/speed/functionality
OrientDB vs Neo4j - Comparison of query/speed/functionality
Curtis Mosters
 
Sistemas de recomendações e neo4J na cloud computing
Sistemas de recomendações e neo4J na cloud computingSistemas de recomendações e neo4J na cloud computing
Sistemas de recomendações e neo4J na cloud computing
Priscila Mayumi
 
Neo4j - Rede de relacionamentos baseada em grafos
Neo4j - Rede de relacionamentos baseada em grafosNeo4j - Rede de relacionamentos baseada em grafos
Neo4j - Rede de relacionamentos baseada em grafos
Evandro Venancio
 
GraphConnect Europe 2016 - NoSQL Polyglot Persistence: Tools and Integrations...
GraphConnect Europe 2016 - NoSQL Polyglot Persistence: Tools and Integrations...GraphConnect Europe 2016 - NoSQL Polyglot Persistence: Tools and Integrations...
GraphConnect Europe 2016 - NoSQL Polyglot Persistence: Tools and Integrations...
Neo4j
 
Neo4j Spatial - GIS for the rest of us.
Neo4j Spatial - GIS for the rest of us.Neo4j Spatial - GIS for the rest of us.
Neo4j Spatial - GIS for the rest of us.
Peter Neubauer
 
Banco de dados de grafos
Banco de dados de grafosBanco de dados de grafos
Banco de dados de grafos
Priscila Mayumi
 
OrientDB vs Neo4j - Comparison of query/speed/functionality
OrientDB vs Neo4j - Comparison of query/speed/functionalityOrientDB vs Neo4j - Comparison of query/speed/functionality
OrientDB vs Neo4j - Comparison of query/speed/functionality
Curtis Mosters
 

More from Tobias Lindaaker (10)

NOSQL Overview
NOSQL OverviewNOSQL Overview
NOSQL Overview
Tobias Lindaaker
 
Building Applications with a Graph Database
Building Applications with a Graph DatabaseBuilding Applications with a Graph Database
Building Applications with a Graph Database
Tobias Lindaaker
 
JDK Power Tools
JDK Power ToolsJDK Power Tools
JDK Power Tools
Tobias Lindaaker
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j Internals
Tobias Lindaaker
 
Choosing the right NOSQL database
Choosing the right NOSQL databaseChoosing the right NOSQL database
Choosing the right NOSQL database
Tobias Lindaaker
 
[JavaOne 2011] Models for Concurrent Programming
[JavaOne 2011] Models for Concurrent Programming[JavaOne 2011] Models for Concurrent Programming
[JavaOne 2011] Models for Concurrent Programming
Tobias Lindaaker
 
NOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4jNOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4j
Tobias Lindaaker
 
A Better Python for the JVM
A Better Python for the JVMA Better Python for the JVM
A Better Python for the JVM
Tobias Lindaaker
 
A Better Python for the JVM
A Better Python for the JVMA Better Python for the JVM
A Better Python for the JVM
Tobias Lindaaker
 
Exploiting Concurrency with Dynamic Languages
Exploiting Concurrency with Dynamic LanguagesExploiting Concurrency with Dynamic Languages
Exploiting Concurrency with Dynamic Languages
Tobias Lindaaker
 
Building Applications with a Graph Database
Building Applications with a Graph DatabaseBuilding Applications with a Graph Database
Building Applications with a Graph Database
Tobias Lindaaker
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j Internals
Tobias Lindaaker
 
Choosing the right NOSQL database
Choosing the right NOSQL databaseChoosing the right NOSQL database
Choosing the right NOSQL database
Tobias Lindaaker
 
[JavaOne 2011] Models for Concurrent Programming
[JavaOne 2011] Models for Concurrent Programming[JavaOne 2011] Models for Concurrent Programming
[JavaOne 2011] Models for Concurrent Programming
Tobias Lindaaker
 
NOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4jNOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4j
Tobias Lindaaker
 
A Better Python for the JVM
A Better Python for the JVMA Better Python for the JVM
A Better Python for the JVM
Tobias Lindaaker
 
A Better Python for the JVM
A Better Python for the JVMA Better Python for the JVM
A Better Python for the JVM
Tobias Lindaaker
 
Exploiting Concurrency with Dynamic Languages
Exploiting Concurrency with Dynamic LanguagesExploiting Concurrency with Dynamic Languages
Exploiting Concurrency with Dynamic Languages
Tobias Lindaaker
 
Ad

Recently uploaded (20)

Linux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded DevelopersLinux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Toradex
 
Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfComplete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Software Company
 
Linux Professional Institute LPIC-1 Exam.pdf
Linux Professional Institute LPIC-1 Exam.pdfLinux Professional Institute LPIC-1 Exam.pdf
Linux Professional Institute LPIC-1 Exam.pdf
RHCSA Guru
 
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...
Vishnu Singh Chundawat
 
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
BookNet Canada
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc
 
Rusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond SparkRusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond Spark
carlyakerly1
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Aqusag Technologies
 
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfThe Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
Abi john
 
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business Consulting
Trs Labs
 
How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?
Daniel Lehner
 
Build Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For DevsBuild Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For Devs
Brian McKeiver
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxProcurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptx
Jon Hansen
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
Special Meetup Edition - TDX Bengaluru Meetup #52.pptx
Special Meetup Edition - TDX Bengaluru Meetup #52.pptxSpecial Meetup Edition - TDX Bengaluru Meetup #52.pptx
Special Meetup Edition - TDX Bengaluru Meetup #52.pptx
shyamraj55
 
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded DevelopersLinux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Toradex
 
Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfComplete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Software Company
 
Linux Professional Institute LPIC-1 Exam.pdf
Linux Professional Institute LPIC-1 Exam.pdfLinux Professional Institute LPIC-1 Exam.pdf
Linux Professional Institute LPIC-1 Exam.pdf
RHCSA Guru
 
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...
Vishnu Singh Chundawat
 
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
BookNet Canada
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc
 
Rusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond SparkRusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond Spark
carlyakerly1
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Aqusag Technologies
 
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfThe Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
Abi john
 
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business Consulting
Trs Labs
 
How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?
Daniel Lehner
 
Build Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For DevsBuild Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For Devs
Brian McKeiver
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxProcurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptx
Jon Hansen
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
Special Meetup Edition - TDX Bengaluru Meetup #52.pptx
Special Meetup Edition - TDX Bengaluru Meetup #52.pptxSpecial Meetup Edition - TDX Bengaluru Meetup #52.pptx
Special Meetup Edition - TDX Bengaluru Meetup #52.pptx
shyamraj55
 
Ad

Persistent graphs in Python with Neo4j

  • 1. Persistent Graphs in Python with Neo4j twitter: @thobe / #neo4j Tobias Ivarsson email: [email protected] web: http://www.neo4j.org/ Hacker @ Neo Technology web: http://www.thobe.org/ Sunday, February 21, 2010
  • 2. We all know the relational model. Attendees It has been predominant for a long time. username fullname registration tutorials payment guido Guido van Rossum null yes 0 thobe Tobias Ivarsson 2009-12-12 no 300 joe John Doe 2010-02-05 yes 700 ... ... ... ... ... 2 Sunday, February 21, 2010
  • 3. Attendees The relational model has username fullname registration tutorials payment a few problems, such as: •poor support for sparse data •modifying the data guido Guido van Rossum null yes 0 model is almost exclusively done through adding tables thobe Tobias Ivarsson 2009-12-12 no 300 joe John Doe 2010-02-05 yes 700 ... ... ... ... ... Location username latitude longitude title publish thobe 55°36'47.70"N 12°58'34.50"E Malmö yes San joe 37°49'36.00"N 122°25'22.00"W no Francisco ... ... ... ... ... 3 Sunday, February 21, 2010
  • 4. Attendees Sessions username fullname registration tutorials payment id title time room ... ... ... ... ... ... guido Guido van Rossum null yes 0 ... ... ... ... ... thobe Tobias Ivarsson 2009-12-12 no 300 Session attendance joe John Doe 2010-02-05 yes 700 session user ... ... ... ... ... ... ... Location ... ... username latitude longitude title publish More complication... thobe 55°36'47.70"N 12°58'34.50"E Malmö yes ... ... ... ... After a while, modeling ... ... complex relationships ... ... leads to complicated ...... ...... San ...... schemasjoe 37°49'36.00"N 122°25'22.00"W no ...... Francisco ...... ...... ...... ...... ... ... ... ... ... 4 Sunday, February 21, 2010
  • 5. A number of companies have realized that the relational model is insufficient and are working on alternative database solutions. 5 Sunday, February 21, 2010
  • 6. Most focus on scaling to large numbers 192.168.0.15 192.168.0.16 192.168.0.21 192.168.0.14 6 Sunday, February 21, 2010
  • 7. Graph Databases focuses on structure of data 7 Sunday, February 21, 2010
  • 8. Positioning w.r.t. other NOSQL DBs Size Key/Value stores Bigtable clones Document databases Graph databases Complexity 8 Sunday, February 21, 2010
  • 9. Positioning w.r.t. other NOSQL DBs Size Key/Value stores Bigtable clones Document databases Graph databases Billions of nodes and relationships > 90% of use cases Complexity 8 Sunday, February 21, 2010
  • 10. What is Neo4j? ๏ Neo4j is a Graph Database • Non-relational (“#nosql”), transactional (ACID), embedded • Data is stored as a Graph / Network ‣Nodes and relationships with properties ‣“Property Graph” or “edge-labeled multidigraph” ๏ Neo4j is Open Source / Free (as in speech) Software • AGPLv3 Prices are available at http://neotechnology.com/ • Commercial (“dual license”) license available Contact us if you have questions and/or special license needs (e.g. if you want an evaluation license) ‣Free (as in beer) for “small” installations ‣Inexpensive (as in startup-friendly) when you grow 9 Sunday, February 21, 2010
  • 11. More about Neo4j ๏ Neo4j is stable • In 24/7 operation since 2003 ๏ Neo4j is in active development • Neo Technology got VC funding October 2009 ๏ Neo4j delivers high performance graph operations • traverses 1’000’000+ relationships / second on commodity hardware 10 Sunday, February 21, 2010
  • 12. The Neo4j Graph data model ๏ Nodes are connected to one another through relationships ๏ A Relationship is a connection between two nodes • Relationships have types • Relationships have a direction • Relationships are traversed equally fast in either direction ๏ Properties are mappings from a string key to a primitive value • Both Nodes and Relationships have properties • Primitive values are any of these (or an array of these): ‣String ‣Numbers: float, double, integers (1-8 byte) 11 Sunday, February 21, 2010
  • 13. The Neo4j Graph data model name: “Mary” LOVES name: “James” age: 35 age: 32 LIVES WITH twitter: “@spam” LOVES OWNS property type: “car” DRIVES brand: “Volvo” model: “V70” 12 Sunday, February 21, 2010
  • 14. Graphs are all around us A B C D ... 1 17 3.14 3 17.79333333333 2 42 10.11 14 30.33 3 316 6.66 1 2104.56 4 32 9.11 592 0.492432432432 5 Even if this spread sheet looks like it could be a fit for a RDBMS 2153.175765766 it isn’t: •RDBMSes have problems with ... extending indefinitely on both rows and collumns •Formulas and data dependencies would quickly lead to heavy join operations 13 Sunday, February 21, 2010
  • 15. Graphs are all around us A B C D ... 1 17 3.14 3 = A1 * B1 / C1 2 42 10.11 14 = A2 * B2 / C2 3 316 6.66 1 = A3 * B3 / C3 4 32 9.11 592 = A4 * B4 / C4 5 = SUM(D2:D5) ... 14 Sunday, February 21, 2010
  • 16. Graphs are all around us A B C D ... 1 17 3.14 3 = A1 * B1 / C1 2 42 10.11 14 = A2 * B2 / C2 3 316 6.66 1 = A3 * B3 / C3 4 32 9.11 592 = A4 * B4 / C4 5 = SUM(D2:D5) ... 14 Sunday, February 21, 2010
  • 17. Graphs are all around us If we add external data sources the problem becomes even more interesting... 17 3.14 3 = A1 * B1 / C1 42 10.11 14 = A2 * B2 / C2 316 6.66 1 = A3 * B3 / C3 32 9.11 592 = A4 * B4 / C4 = SUM(D2:D5) 15 Sunday, February 21, 2010
  • 18. Graphs are all around us If we add external data sources the problem becomes even more interesting... 17 3.14 3 = A1 * B1 / C1 42 10.11 14 = A2 * B2 / C2 316 6.66 1 = A3 * B3 / C3 32 9.11 592 = A4 * B4 / C4 = SUM(D2:D5) 15 Sunday, February 21, 2010
  • 19. Graphs are whiteboard friendly 16 Sunday, February 21, 2010
  • 20. Graphs are whiteboard friendly * 1 * * 1 * 1 * 1 * 16 Sunday, February 21, 2010
  • 21. Graphs are whiteboard friendly thobe Joe project blog Wardrobe Strength Hello Joe Modularizing Jython Neo4j performance analysis 16 Sunday, February 21, 2010
  • 22. Query Languages ๏ Traversal API ๏ Sparql - “SQL for linked data” SELECT ?person WHERE { ?person neo4j:KNOWS ?friend . ?friend neo4j:KNOWS ?foe . ?foe neo4j:name “Larry Ellison” . } ๏ Gremlin - “perl for graphs” ./outE[@label='KNOWS']/inV[@age > 30]/@name 17 Sunday, February 21, 2010
  • 23. Python integration for Neo4j ๏ Mapping of the core Neo4j API for Python • Making it feel “Pythonic” ๏ Available from the Neo4j repository (and soon from PyPI) • http://components.neo4j.org/neo4j.py ‣svn co http://svn.neo4j.org/components/neo4j.py/trunk neo4j-python ๏ Works with both Jython and CPython • The threading of Jython is a plus with an embedded db... ๏ Comes with Django empowering batteries included • Could have support for other frameworks in the future 18 Sunday, February 21, 2010
  • 24. Simple interaction import neo4j graphdb = neo4j.GraphDatabase(“var/neo”) with graphdb.transaction: james = graphdb.node(name=“James”, age=32, twitter=“@spam”) mary = graphdb.node(name=“Mary”, age=35) the_car = graphdb.node(brand=“Volvo”, model=“V70”) james.LOVES( mary ) mary.LOVES( james ) james.LIVES_WITH( mary ) james.OWNS( the_car, property_type=“car” ) Creates the graph we saw in the first example. mary.DRIVES( the_car ) 19 Sunday, February 21, 2010
  • 25. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] order = neo4j.BREDTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print “%s (@ depth=%s)” % ( friend_node[“name”], friend_node.depth ) 20 Sunday, February 21, 2010
  • 26. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] order = neo4j.BREDTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print “%s (@ depth=%s)” % ( friend_node[“name”], friend_node.depth ) 20 Sunday, February 21, 2010
  • 27. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREDTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print “%s (@ depth=%s)” % ( friend_node[“name”], friend_node.depth ) 20 Sunday, February 21, 2010
  • 28. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREDTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print “%s (@ depth=%s)” % ( friend_node[“name”], friend_node.depth ) 20 Sunday, February 21, 2010
  • 29. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREDTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH Cypher (@ depth=2) returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print “%s (@ depth=%s)” % ( friend_node[“name”], friend_node.depth ) 20 Sunday, February 21, 2010
  • 30. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREDTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH Cypher (@ depth=2) returnable = neo4j.RETURN_ALL_BUT_START_NODE Agent Smith (@ depth=3) for friend_node in Friends(mr_anderson): print “%s (@ depth=%s)” % ( friend_node[“name”], friend_node.depth ) 20 Sunday, February 21, 2010
  • 31. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREDTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH Cypher (@ depth=2) returnable = neo4j.RETURN_ALL_BUT_START_NODE Agent Smith (@ depth=3) for friend_node in Friends(mr_anderson): print “%s (@ depth=%s)” % ( friend_node[“name”], friend_node.depth ) 20 Sunday, February 21, 2010
  • 32. Batteries for Django from neo4j.model import django_model as models class Movie(models.NodeModel): title = models.Property(indexed=True) year = models.Property() href = property(lambda self: ('/movie/%s/' % (self.node.id,))) def __unicode__(self): return self.title class Actor(models.NodeModel): name = models.Property(indexed=True) href = property(lambda self: ('/actor/%s/' % (self.node.id,))) def __unicode__(self): return self.name # etc. ... 21 Sunday, February 21, 2010
  • 33. “My ORM already does this” ๏ ORMs and model evolution is a hard problem • virtually unsupported in Django ๏ SQL is a “compatible” across many RDBMSs • data is still locked in ๏ Each ORM maps object models differently • Moving to another ORM == legacy schema support ‣except your legacy schema is strange auto-generated ๏ Object/Graph Mapping is always done the same • allows you to keep your data through application changes • or share data between multiple implementations 22 Sunday, February 21, 2010
  • 34. What your ORM doesn’t do ๏ Drop down to underlying graph model • Traversals • Graph algorithms • Shortest path(s) • etc. 23 Sunday, February 21, 2010
  • 35. Buzzword summary http://neo4j.org/ SPARQL AGPLv3 Open Source ACID Object mapping Shortest path NOSQL startup friendly whiteboard friendly Traversal Query language Embedded Beer Software Transactional Memory polyglot persistence Free Software Scaling to complexity 24 Sunday, February 21, 2010