SlideShare a Scribd company logo
1 of 60
Download to read offline
Django and Neo4j
  Domain modeling that kicks ass!



                          twitter: @thobe / #neo4j
Tobias Ivarsson           email: tobias@neotechnology.com
                          web: http://www.neo4j.org/
Hacker @ Neo Technology   web: http://www.thobe.org/
It all started with this guy.
Emil Eifrem, CEO of Neo
Technology. We picked him
apart, and inside his brain
we found the base for a
database that models the
connections in bet ween
entities.




              2
It all started with this guy.
                     Emil Eifrem, CEO of Neo
                     Technology. We picked him
                     apart, and inside his brain
                     we found the base for a
                     database that models the
                     connections in bet ween
                     entities.




Image credits: US Army
                                   2
It all started with this guy.
Emil Eifrem, CEO of Neo
Technology. We picked him
apart, and inside his brain
we found the base for a
database that models the
connections in bet ween
entities.




              2
Neo4j
        It all started with this guy.
        Emil Eifrem, CEO of Neo
        Technology. We picked him
        apart, and inside his brain
        we found the base for a
        database that models the
        connections in bet ween
        entities.




                      2
NOSQL is a wide area


                  3
The problems NOSQL focuses on
                                                                                             Relational database

                                                                                             Requirement of application

                                    Focus area of many
                                     NOSQL Databases                                ๏ Huge amounts of data
        Performance




                                                                                    ๏ (mostly) Disjoint data
                      Salary List
                                                                                    ๏ Heavy load            most focus on...

                                             Majority of
                                             Webapps                                ๏ Many concurrent writers
                                                                            Social network

                                                                                                   Semantic Trading



All NOSQL databases focus
on solving problems where
RDBMSes fail.
                                                      While this handles the
                                                      load, it lacks in “social”
                                                                                   }     custom


                                                                                   Data complexity                    4
The evolution of data
                                                                                                          Giant
                                                                                                          Global
                                                                                                       Graph (GGG)


                                                                                          Ontologies


                                                                                 RDF


                                                                                                Folksonomies
  Information connectivity




                                                                              Tagging


                                                              Wikis            User-generated
                                                                                  content
                                                                      Blogs

                                                                                                 ... but it turns out that
                                                            RSS                                  data evolves to become
                                                                                                 MORE interconnected
                                            Hypertext                                            (as well as greater sizes)


                         Text documents
                                                  web 1.0               web 2.0                        “web 3.0”

                                          1990              2000                        2010                   2020           5
Neo4j is a Graph Database
   Graph databases FOCUS
   on the interconnection
   bet ween entities.




                            6
IS_A


Neo4j                      Graph Database
  Graph databases FOCUS
  on the interconnection
  bet ween entities.




                                      6
Scaling to size vs. Scaling to complexity
    Size
       Key/Value stores

                          Bigtable clones

                                            Document databases

                                                                 Graph databases




                                                                           Complexity

                                                                                   7
Scaling to size vs. Scaling to complexity
    Size
       Key/Value stores

                          Bigtable clones

                                            Document databases

                                                                 Graph databases
                                                                             Billions of nodes
                                                                             and relationships




                                > 90% of use cases

                                                                           Complexity

                                                                                    7
What is Neo4j?
๏ Neo4j is a Graph Database
   • Non-relational (“#nosql”), transactional (ACID), embedded
   • Data is stored as a Graph / Network
      ‣Nodes and relationships with properties
      ‣“Property Graph” or “edge-labeled multidigraph”
   • Schema free, bottom-up data model design
๏ Neo4j is Open Source / Free (as in speech) Software
                                                            Prices are available at
                                                            http://neotechnology.com/



   • AGPLv3
                                                            Contact us if you have
                                                            questions and/or special
                                                            license needs (e.g. if you


   • Commercial (“dual license”) license available
                                                            want an evaluation license)




      ‣First server is free (as in beer), next is inexpensive          8
More about Neo4j
๏ Neo4j is stable
   • In 24/7 operation since 2003
๏ Neo4j is in active development
   • Neo Technology received VC funding October 2009
๏ Neo4j delivers high performance graph operations
   • traverses 1’000’000+ relationships / second
       on commodity hardware
        (1000~2500 traversals/ms)



                                                       9
Building business applications with Neo4j
๏ Try it out! It’s all open source!
    • Build a prototype, find out your needs and how Neo4j matches
    • AGPL this stage thisusers should have access to your code
        - at
             says all your
                           is your employees / co-workers




                                                           10
Building business applications with Neo4j
๏ Try it out! It’s all open source!
    • Build a prototype, find out your needs and how Neo4j matches
    • AGPL this stage thisusers should have access to your code
        - at
             says all your
                           is your employees / co-workers
๏ Put it in front of users! The license is free for the first server!
    • Contact Neo Technology sales to get a free single server license
    • You’ll (probably) not have massive load the first days

                                                                   10
Building business applications with Neo4j
๏ Try it out! It’s all open source!
    • Build a prototype, find out your needs and how Neo4j matches
    • AGPL this stage thisusers should have access to your code
        - at
             says all your
                           is your employees / co-workers
๏ Put it in front of users! The license is free for the first server!
    • Contact Neo Technology sales to get a free single server license
    • You’ll (probably) not have massive load the first days
๏ As you grow, Neo4j grows with you!
    • Aslicenseneeds and revenue increase you can by an advanced
          your
                (prices are resonable)
                                                                   10
Graphs are all around us
          A                        B           C             D           ...
   1              17                  3.14          3   17.79333333333

   2              42               10.11           14            30.33

   3           316                    6.66          1          2104.56

   4              32                  9.11     592      0.492432432432

   5      Even if this spreadsheet looks
          like it could be a fit for a RDBMS
                                                        2153.175765766
          it isn’t:
          •RDBMSes have problems with
  ...     extending indefinitely on both
          rows and columns
          •Formulas and data
          dependencies would quickly lead
          to heavy join operations

                                                                         11
Graphs are all around us
                 A                B      C         D            ...
   1            17               3.14     3    = A1 * B1 / C1

   2            42               10.11   14    = A2 * B2 / C2

   3           316               6.66     1    = A3 * B3 / C3

   4            32               9.11    592   = A4 * B4 / C4

   5                                           = SUM(D2:D5)
        With data dependencies
  ...   the spread sheet turns
        out to be a graph.




                                                                12
Graphs are all around us
                 A                B      C         D            ...
   1            17               3.14     3    = A1 * B1 / C1

   2            42               10.11   14    = A2 * B2 / C2

   3           316               6.66     1    = A3 * B3 / C3

   4            32               9.11    592   = A4 * B4 / C4

   5                                           = SUM(D2:D5)
        With data dependencies
  ...   the spread sheet turns
        out to be a graph.




                                                                12
Graphs are all around us                      If we add external data
                                              sources the problem
                                              becomes even more
                                              interesting...




          17     3.14       3    = A1 * B1 / C1

          42     10.11     14    = A2 * B2 / C2

          316    6.66       1    = A3 * B3 / C3

          32     9.11      592   = A4 * B4 / C4

                                 = SUM(D2:D5)




                                                      13
Graphs are all around us                      If we add external data
                                              sources the problem
                                              becomes even more
                                              interesting...




          17     3.14       3    = A1 * B1 / C1

          42     10.11     14    = A2 * B2 / C2

          316    6.66       1    = A3 * B3 / C3

          32     9.11      592   = A4 * B4 / C4

                                 = SUM(D2:D5)




                                                      13
The Neo4j Graph data model




•Nodes
•Relationships bet ween Nodes
•Relationships have Labels
•Relationships are directed, but traversed at
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties              14
The Neo4j Graph data model




•Nodes
•Relationships bet ween Nodes
•Relationships have Labels
•Relationships are directed, but traversed at
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties              14
The Neo4j Graph data model


                                                      LIVES WITH
                                                               LOVES



                                         OWNS
                                                                       DRIVES

•Nodes
•Relationships bet ween Nodes
•Relationships have Labels
•Relationships are directed, but traversed at
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties                                        14
The Neo4j Graph data model

                                                                 LOVES

                                                      LIVES WITH
                                                               LOVES



                                         OWNS
                                                                       DRIVES

•Nodes
•Relationships bet ween Nodes
•Relationships have Labels
•Relationships are directed, but traversed at
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties                                        14
The Neo4j Graph data model
                                                                                name: “Mary”
                                                                 LOVES
             name: “James”                                                      age: 35
             age: 32                                  LIVES WITH
             twitter: “@spam”                                  LOVES



                                         OWNS
                                                                       DRIVES

•Nodes
•Relationships bet ween Nodes
•Relationships have Labels                                     brand: “Volvo”
•Relationships are directed, but traversed at                  model: “V70”
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties                                                 14
The Neo4j Graph data model
                                                                                name: “Mary”
                                                                 LOVES
             name: “James”                                                      age: 35
             age: 32                                  LIVES WITH
             twitter: “@spam”                                  LOVES



                                         OWNS
                                     item type: “car”                  DRIVES

•Nodes
•Relationships bet ween Nodes
•Relationships have Labels                                     brand: “Volvo”
•Relationships are directed, but traversed at                  model: “V70”
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties                                                 14
Graphs are Whiteboard Friendly   The domain I specify is the
                                 domain I implement.
                                 No mismatch, no ER-modeling.




                                               15
Graphs are Whiteboard Friendly                            The domain I specify is the
                                                          domain I implement.
                                          odin            No mismatch, no ER-modeling.

                                   thobe

                              dude
                #17
           #6
     #14                                          Wardrobe Strength
                                            Joe project
   #32


                      Call site caching

                          Hello world
                                 OSCON

                                Best panncakes
                                    Optimizing Jython                   15
Graphs are Whiteboard Friendly
                                     odin



                          dude
                #17
           #6
     #14    username: “thobe”             Wardrobe Strength
                                    Joe project
   #32      name: “Tobias Ivarsson”
            twitter: “@thobe”
            password: “**********”
                  Call site caching

                       Hello world
                            OSCON

                            Best panncakes
                                Optimizing Jython      16
Graphs are Whiteboard Friendly
                                     odin

                               thobe

                          dude
                #17
           #6
     #14    address: “http://journal.thobe.org”
   #32      title: “Wardrobe Strengthproject
                                        Joe
                                             ”
            tagline: “Good enough
                         thoughts”
                    Call site caching

                       Hello world
                            OSCON

                            Best panncakes
                                Optimizing Jython   17
Building a graph - the basic API
import neo4j

grapDb = neo4j.GraphDatabase( PATH_TO_YOUR_NEO4J_DATASTORE )


with graphDb.transaction: # All writes require transactions

   # Create Thomas 'Neo' Anderson
   mrAnderson = graphDb.node(name="Thomas Anderson", age=29)

   # Create Morpheus
   morpheus = graphDb.node(name="Morpheus", rank= "Captain",
     occupation= "Total bad ass")

   # Create relationship representing they know each other
   mrAnderson.KNOWS( morpheus )

   # ... similarly for Trinity, Cypher, Agent Smith, Architect

                                                         18
Graph traversals


                                                                                  name: “The Architect”
                                    disclosure: “public”
name: “Thomas Anderson”
age: 29                                                     name: “Cypher”
                                                            last name: “Reagan”
                   KNOWS name: “Morpheus”
             KNOWS                                  KNOWS
                         rank: “Captain”                                                CODED BY
       LOVES             occupation: “Total badass”                        KNOWS
                           KNOWS
         name: “Trinity”                            disclosure: “secret”
                                                                              name: “Agent Smith”
                                                                              version: “1.0b”
 since: “meeting the oracle”       since: “a year before the movie”
                                                                              language: “C++”
                                   cooperates on: “The Nebuchadnezzar”




                                                                                           19
Graph traversals                                                                  name: “The Architect”
                                    disclosure: “public”
name: “Thomas Anderson”
age: 29                                                     name: “Cypher”
                                                            last name: “Reagan”
                   KNOWS name: “Morpheus”
             KNOWS                                  KNOWS
                         rank: “Captain”                                                CODED BY
       LOVES             occupation: “Total badass”                        KNOWS
                           KNOWS
         name: “Trinity”                            disclosure: “secret”
                                                                              name: “Agent Smith”
                                                                              version: “1.0b”
 since: “meeting the oracle”       since: “a year before the movie”
                                                                              language: “C++”
                                   cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
   types = [ neo4j.Outgoing.KNOWS ]
   order = neo4j.BREADTH_FIRST
   stop = neo4j.STOP_AT_END_OF_GRAPH
   returnable = neo4j.RETURN_ALL_BUT_START_NODE



                                                                                           20
Graph traversals                                                                  name: “The Architect”
                                    disclosure: “public”
name: “Thomas Anderson”
age: 29                                                     name: “Cypher”
                                                            last name: “Reagan”
                   KNOWS name: “Morpheus”
             KNOWS                                  KNOWS
                         rank: “Captain”                                                CODED BY
       LOVES             occupation: “Total badass”                        KNOWS
                           KNOWS
         name: “Trinity”                            disclosure: “secret”
                                                                              name: “Agent Smith”
                                                                              version: “1.0b”
 since: “meeting the oracle”       since: “a year before the movie”
                                                                              language: “C++”
                                   cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
   types = [ neo4j.Outgoing.KNOWS ]
   order = neo4j.BREADTH_FIRST
   stop = neo4j.STOP_AT_END_OF_GRAPH
   returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
   print "%s (@ depth=%s)" % ( friend_node["name"],
     friend_node.depth )
                                                                                           20
Graph traversals                                                                  name: “The Architect”
                                    disclosure: “public”
name: “Thomas Anderson”
age: 29                                                     name: “Cypher”
                                                            last name: “Reagan”
                   KNOWS name: “Morpheus”
             KNOWS                                  KNOWS
                         rank: “Captain”                                                CODED BY
       LOVES             occupation: “Total badass”                        KNOWS
                           KNOWS
         name: “Trinity”                            disclosure: “secret”
                                                                              name: “Agent Smith”
                                                                              version: “1.0b”
 since: “meeting the oracle”       since: “a year before the movie”
                                                                              language: “C++”
                                   cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
   types = [ neo4j.Outgoing.KNOWS ]
   order = neo4j.BREADTH_FIRST
   stop = neo4j.STOP_AT_END_OF_GRAPH
   returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
   print "%s (@ depth=%s)" % ( friend_node["name"],
     friend_node.depth )
                                                                                           20
Graph traversals                                                                  name: “The Architect”
                                    disclosure: “public”
name: “Thomas Anderson”
age: 29                                                     name: “Cypher”
                                                            last name: “Reagan”
                   KNOWS name: “Morpheus”
             KNOWS                                  KNOWS
                         rank: “Captain”                                                CODED BY
       LOVES             occupation: “Total badass”                        KNOWS
                           KNOWS
         name: “Trinity”                            disclosure: “secret”
                                                                              name: “Agent Smith”
                                                                              version: “1.0b”
 since: “meeting the oracle”       since: “a year before the movie”
                                                                              language: “C++”
                                   cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
   types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
   order = neo4j.BREADTH_FIRST
   stop = neo4j.STOP_AT_END_OF_GRAPH
   returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
   print "%s (@ depth=%s)" % ( friend_node["name"],
     friend_node.depth )
                                                                                           20
Graph traversals                                                                  name: “The Architect”
                                    disclosure: “public”
name: “Thomas Anderson”
age: 29                                                     name: “Cypher”
                                                            last name: “Reagan”
                   KNOWS name: “Morpheus”
             KNOWS                                  KNOWS
                         rank: “Captain”                                                CODED BY
       LOVES             occupation: “Total badass”                        KNOWS
                           KNOWS
         name: “Trinity”                            disclosure: “secret”
                                                                              name: “Agent Smith”
                                                                              version: “1.0b”
 since: “meeting the oracle”       since: “a year before the movie”
                                                                              language: “C++”
                                   cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
   types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
   order = neo4j.BREADTH_FIRST                    Trinity (@ depth=1)
   stop = neo4j.STOP_AT_END_OF_GRAPH
   returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
   print "%s (@ depth=%s)" % ( friend_node["name"],
     friend_node.depth )
                                                                                           20
Graph traversals                                                                  name: “The Architect”
                                    disclosure: “public”
name: “Thomas Anderson”
age: 29                                                     name: “Cypher”
                                                            last name: “Reagan”
                   KNOWS name: “Morpheus”
             KNOWS                                  KNOWS
                         rank: “Captain”                                                CODED BY
       LOVES             occupation: “Total badass”                        KNOWS
                           KNOWS
         name: “Trinity”                            disclosure: “secret”
                                                                              name: “Agent Smith”
                                                                              version: “1.0b”
 since: “meeting the oracle”       since: “a year before the movie”
                                                                              language: “C++”
                                   cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
   types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
   order = neo4j.BREADTH_FIRST                    Trinity (@ depth=1)
   stop = neo4j.STOP_AT_END_OF_GRAPH
                                                  Cypher (@ depth=2)
   returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
   print "%s (@ depth=%s)" % ( friend_node["name"],
     friend_node.depth )
                                                                                           20
Graph traversals                                                                  name: “The Architect”
                                    disclosure: “public”
name: “Thomas Anderson”
age: 29                                                     name: “Cypher”
                                                            last name: “Reagan”
                   KNOWS name: “Morpheus”
             KNOWS                                  KNOWS
                         rank: “Captain”                                                CODED BY
       LOVES             occupation: “Total badass”                        KNOWS
                           KNOWS
         name: “Trinity”                            disclosure: “secret”
                                                                              name: “Agent Smith”
                                                                              version: “1.0b”
 since: “meeting the oracle”       since: “a year before the movie”
                                                                              language: “C++”
                                   cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
   types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
   order = neo4j.BREADTH_FIRST                    Trinity (@ depth=1)
   stop = neo4j.STOP_AT_END_OF_GRAPH
                                                  Cypher (@ depth=2)
   returnable = neo4j.RETURN_ALL_BUT_START_NODE
                                                                             Agent Smith (@ depth=3)
for friend_node in Friends(mr_anderson):
   print "%s (@ depth=%s)" % ( friend_node["name"],
     friend_node.depth )
                                                                                           20
Graph traversals                                                                  name: “The Architect”
                                    disclosure: “public”
name: “Thomas Anderson”
age: 29                                                     name: “Cypher”
                                                            last name: “Reagan”
                   KNOWS name: “Morpheus”
             KNOWS                                  KNOWS
                         rank: “Captain”                                                CODED BY
       LOVES             occupation: “Total badass”                        KNOWS
                           KNOWS
         name: “Trinity”                            disclosure: “secret”
                                                                              name: “Agent Smith”
                                                                              version: “1.0b”
 since: “meeting the oracle”       since: “a year before the movie”
                                                                              language: “C++”
                                   cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
   types = [ neo4j.Outgoing.KNOWS ]               Morpheus (@ depth=1)
   order = neo4j.BREADTH_FIRST                    Trinity (@ depth=1)
   stop = neo4j.STOP_AT_END_OF_GRAPH
                                                  Cypher (@ depth=2)
   returnable = neo4j.RETURN_ALL_BUT_START_NODE
                                                                             Agent Smith (@ depth=3)
for friend_node in Friends(mr_anderson):
   print "%s (@ depth=%s)" % ( friend_node["name"],
     friend_node.depth )
                                                                                           20
Finding a place to start
๏ Traversals need a Node to start from
    • QUESTION: How do I find the start Node?
    • ANSWER:You use an Index
๏ Indexes in Neo4j are different from Indexes in Relational Databases
    • RDBMSes use them for Joining
    • Neo4j use them for simple lookup
index = graphDb.index["name"]

mr_anderson = index["Thomas Anderson"]

performTraversalFrom( mrAnderson )

                                                              21
Indexes in Neo4j
๏ The Graph *is* the main index
   • Use relationship labels for navigation
   • Build index structures *in the graph*
     ‣Search trees, tag clouds, geospatial indexes, et.c.
     ‣Linked/skip lists or other data structures in the graph
     ‣We have utility libraries for this
๏ External indexes used *for lookup*
   • Finding a (number of) points to start traversals from
   • Major difference from RDBMS that use indexes for everything
                                                            22
Django integration
does all of this for you!

                      23
Implementing the domain

                     user



                             blog
     comment




                     entry
                                    24
from neo4j.model import django_model as models



                        user



                                    blog
     comment




                       entry
                                           25
from neo4j.model import django_model as models




          class User(models.NodeModel):
             username = models.Property(indexed=True)
             name = models.Property()
                                                    blog
     comment blogs = models.Relationship(Blog,
                         type=models.Outgoing.member_of,
                         related_name="users")
             def __unicode__(self):
                return self.name


                               entry
                                                           25
from neo4j.model import django_model as models



                                user


          class Blog(models.NodeModel):
             identifier = models.Property(indexed=True)
     comment title = models.Property()
             def __unicode__(self):
                 return self.title




                               entry
                                                         26
from neo4j.model import django_model as models



                                user

          class Entry(models.NodeModel):
             title = models.Property()
             text = models.Property()
                                                    blog
     comment date = models.Property()
             blog = models.Relationship(Blog,
                         type=models.Outgoing.posted_on,
                         single=True, optional=False,
                         related_name="articles")
             author = models.Relationship(User,
                         type=models.Outgoing.authored_by,
                         single=True, optional=False,
                         related_name="articles")
                                                             27
models.py
from neo4j.model import django_model as models
class Blog(models.NodeModel):
    identifier = models.Property(indexed=True)
    title = models.Property()
                                                    The rest of the code for
                                                    working with the domain
class User(models.NodeModel):                       objects is (mostly) the same
    username = models.Property(indexed=True)        as you are used to in Django.
    name = models.Property()
    blogs = models.Relationship(Blog,
                type=models.Outgoing.member_of,
                related_name="users")

class Entry(models.NodeModel):
    title = models.Property()
    text = models.Property()
    date = models.Property()
    blog = models.Relationship(Blog,
                type=models.Outgoing.posted_on,
                single=True, optional=False,
                related_name="articles")
    author = models.Relationship(User,
                type=models.Outgoing.authored_by,
                single=True, optional=False,
                                                                                    28
                related_name="articles")
Why not use an O/R mapper?
๏ Model evolution in ORMs is a hard problem
   • virtually unsupported in most ORM systems
๏ SQL is “compatible” across many RDBMSs
   • data is still locked in
๏ Each ORM maps object models differently
   • Moving to another ORM == legacy schema support
      ‣except your legacy schema is a strange auto-generated one
๏ Object/Graph Mapping is always done the same way
   • allows you to keep your data through application changes
   • or share data between multiple implementations         29
What an ORM doesn’t do

๏Deep traversals
๏Graph algorithms
๏Shortest path(s)
๏Routing
๏etc.
                          30
Path exists in social network
๏ Each person has on average 50 friends      The performance impact
                                             in Neo4j depends only on
                                             the degree of each node. in
             Tobias                          an RDBMS it depends on
                                             the number of entries in
                                             the tables involved in the
                                             join(s).
                                   Emil



                 Johan
                                                Peter


        Database               # persons query time
  Relational database                 1 000      2 000 ms
  Neo4j Graph Database                1 000          2 ms
  Neo4j Graph Database            1 000 000          2 ms
  Relational database             1 000 000 way too long...
                                                                    31
Path exists in social network
๏ Each person has on average 50 friends      The performance impact
                                             in Neo4j depends only on
                                             the degree of each node. in
             Tobias                          an RDBMS it depends on
                                             the number of entries in
                                             the tables involved in the
                                             join(s).
                                   Emil



                 Johan
                                                Peter


        Database               # persons query time
  Relational database                 1 000      2 000 ms
  Neo4j Graph Database                1 000          2 ms
  Neo4j Graph Database            1 000 000          2 ms
  Relational database             1 000 000 way too long...
                                                                    31
On-line real time routing with Neo4j
๏ 20 million Nodes - represents places
๏ 62 million Edges - represents direct roads between places
   • These edges have a length property, for the length of the road
๏ Average optimal route, 100 separate roads, found in 100ms
๏ Worst case route we could find:
   • Optimal route is 5500 separate roads
   • Total length ~770km                             There’s a difference


   • Found in less than 3 seconds
                                                     bet ween least
                                                     number of hops and
                                                     least cost.

๏ Uses A* “best first” search
                                                                    32
Jython vs. CPython
๏ Neo4j with the Python bindings work in both
   • Requires no code modification in your code
๏ Neo4j at its core is an Embedded (in-process) database
   • CPython manages concurrency by forking multiple processes
   • Jython has full concurrency support in the same JVM
   • Stand-alone Neo4jon
        is being worked
                         server-process with (C)Python client


๏ Neo4j has a RESTful interface
   • There are Python clients
   • The API differs slightly (no transactions)            33
Finding out more
๏ http://neo4j.org/ - project website - main place for getting started
      ‣Contains screen casts, download links, et.c.
      ‣http://api.neo4j.org/ and http://components.neo4j.org/
        ‣Specifically http://components.neo4j.org/neo4j.py/
      ‣http://wiki.neo4j.org/ - HowTos, Tutorials, Examples, FAQ, et.c.
      ‣http://planet.neo4j.org/ - aggregation of blogs about Neo4j
      ‣http://github.com/neo4j-examples - small example applications
๏ https://lists.neo4j.org/ - community mailing list
๏ http://twitter.com/neo4j/team - follow the Neo4j team
๏ http://neotechnology.com/ - commercial licensing               34
Helping out!
๏ Neo4j and the Python integration is all Open Source
๏ The Python bindings in particular would benefit from more devs...
   • Integrate more of the Neo4j components
      ‣Neo4j Spatial
      ‣The Graph Algorithms package
      ‣The Graph Matching component
   • Trimming off the rough edges in the Django integration
   • Native client for CPython
                                                              35
Buzzword summary                                                   http://neo4j.org/


                                              Semi structured
                    SPARQL
  AGPLv3
                                                            ACID transactions
                                    Open Source

              Object mapping                      Gremlin        Shortest path
                                      NOSQL
  Traversal
                          RESTful             Software Transactional Memory
                                    Query language
    whiteboard friendly
                                                      Beer
A* routing                      Embedded
                                                            Schema free
      Scaling to complexity
                                              Free Software

                     Polyglot persistence
                                                                          36

More Related Content

What's hot

Neo4j Graph Use Cases, Bruno Ungermann, Neo4j
Neo4j Graph Use Cases, Bruno Ungermann, Neo4jNeo4j Graph Use Cases, Bruno Ungermann, Neo4j
Neo4j Graph Use Cases, Bruno Ungermann, Neo4jNeo4j
 
ntroducing to the Power of Graph Technology
ntroducing to the Power of Graph Technologyntroducing to the Power of Graph Technology
ntroducing to the Power of Graph TechnologyNeo4j
 
Intro to Neo4j presentation
Intro to Neo4j presentationIntro to Neo4j presentation
Intro to Neo4j presentationjexp
 
Introduction to Graph Databases.pdf
Introduction to Graph Databases.pdfIntroduction to Graph Databases.pdf
Introduction to Graph Databases.pdfNeo4j
 
The Graph Database Universe: Neo4j Overview
The Graph Database Universe: Neo4j OverviewThe Graph Database Universe: Neo4j Overview
The Graph Database Universe: Neo4j OverviewNeo4j
 
Master Real-Time Streams With Neo4j and Apache Kafka
Master Real-Time Streams With Neo4j and Apache KafkaMaster Real-Time Streams With Neo4j and Apache Kafka
Master Real-Time Streams With Neo4j and Apache KafkaNeo4j
 
Graph Data Modeling Best Practices(Eric_Monk).pptx
Graph Data Modeling Best Practices(Eric_Monk).pptxGraph Data Modeling Best Practices(Eric_Monk).pptx
Graph Data Modeling Best Practices(Eric_Monk).pptxNeo4j
 
Training Week: Create a Knowledge Graph: A Simple ML Approach
Training Week: Create a Knowledge Graph: A Simple ML Approach Training Week: Create a Knowledge Graph: A Simple ML Approach
Training Week: Create a Knowledge Graph: A Simple ML Approach Neo4j
 
Demo Showcase: Graphs for Cybersecurity in Action
Demo Showcase: Graphs for Cybersecurity in ActionDemo Showcase: Graphs for Cybersecurity in Action
Demo Showcase: Graphs for Cybersecurity in ActionNeo4j
 
Neo4j for Discovering Drugs and Biomarkers
Neo4j for Discovering Drugs and BiomarkersNeo4j for Discovering Drugs and Biomarkers
Neo4j for Discovering Drugs and BiomarkersNeo4j
 
Neo4j Webinar: Graphs in banking
Neo4j Webinar:  Graphs in banking Neo4j Webinar:  Graphs in banking
Neo4j Webinar: Graphs in banking Neo4j
 
The Customer Journey Is a Graph
The Customer Journey Is a GraphThe Customer Journey Is a Graph
The Customer Journey Is a GraphNeo4j
 
Neo4j in Production: A look at Neo4j in the Real World
Neo4j in Production: A look at Neo4j in the Real WorldNeo4j in Production: A look at Neo4j in the Real World
Neo4j in Production: A look at Neo4j in the Real WorldNeo4j
 
Neo4j Demo: Using Knowledge Graphs to Classify Diabetes Patients (GlaxoSmithK...
Neo4j Demo: Using Knowledge Graphs to Classify Diabetes Patients (GlaxoSmithK...Neo4j Demo: Using Knowledge Graphs to Classify Diabetes Patients (GlaxoSmithK...
Neo4j Demo: Using Knowledge Graphs to Classify Diabetes Patients (GlaxoSmithK...Neo4j
 
Intro to Graphs and Neo4j
Intro to Graphs and Neo4jIntro to Graphs and Neo4j
Intro to Graphs and Neo4jjexp
 
Improving Machine Learning using Graph Algorithms
Improving Machine Learning using Graph AlgorithmsImproving Machine Learning using Graph Algorithms
Improving Machine Learning using Graph AlgorithmsNeo4j
 
Introduction to Neo4j for the Emirates & Bahrain
Introduction to Neo4j for the Emirates & BahrainIntroduction to Neo4j for the Emirates & Bahrain
Introduction to Neo4j for the Emirates & BahrainNeo4j
 
Knowledge Graphs and Generative AI
Knowledge Graphs and Generative AIKnowledge Graphs and Generative AI
Knowledge Graphs and Generative AINeo4j
 
Workshop - Neo4j Graph Data Science
Workshop - Neo4j Graph Data ScienceWorkshop - Neo4j Graph Data Science
Workshop - Neo4j Graph Data ScienceNeo4j
 
Optimizing Your Supply Chain with the Neo4j Graph
Optimizing Your Supply Chain with the Neo4j GraphOptimizing Your Supply Chain with the Neo4j Graph
Optimizing Your Supply Chain with the Neo4j GraphNeo4j
 

What's hot (20)

Neo4j Graph Use Cases, Bruno Ungermann, Neo4j
Neo4j Graph Use Cases, Bruno Ungermann, Neo4jNeo4j Graph Use Cases, Bruno Ungermann, Neo4j
Neo4j Graph Use Cases, Bruno Ungermann, Neo4j
 
ntroducing to the Power of Graph Technology
ntroducing to the Power of Graph Technologyntroducing to the Power of Graph Technology
ntroducing to the Power of Graph Technology
 
Intro to Neo4j presentation
Intro to Neo4j presentationIntro to Neo4j presentation
Intro to Neo4j presentation
 
Introduction to Graph Databases.pdf
Introduction to Graph Databases.pdfIntroduction to Graph Databases.pdf
Introduction to Graph Databases.pdf
 
The Graph Database Universe: Neo4j Overview
The Graph Database Universe: Neo4j OverviewThe Graph Database Universe: Neo4j Overview
The Graph Database Universe: Neo4j Overview
 
Master Real-Time Streams With Neo4j and Apache Kafka
Master Real-Time Streams With Neo4j and Apache KafkaMaster Real-Time Streams With Neo4j and Apache Kafka
Master Real-Time Streams With Neo4j and Apache Kafka
 
Graph Data Modeling Best Practices(Eric_Monk).pptx
Graph Data Modeling Best Practices(Eric_Monk).pptxGraph Data Modeling Best Practices(Eric_Monk).pptx
Graph Data Modeling Best Practices(Eric_Monk).pptx
 
Training Week: Create a Knowledge Graph: A Simple ML Approach
Training Week: Create a Knowledge Graph: A Simple ML Approach Training Week: Create a Knowledge Graph: A Simple ML Approach
Training Week: Create a Knowledge Graph: A Simple ML Approach
 
Demo Showcase: Graphs for Cybersecurity in Action
Demo Showcase: Graphs for Cybersecurity in ActionDemo Showcase: Graphs for Cybersecurity in Action
Demo Showcase: Graphs for Cybersecurity in Action
 
Neo4j for Discovering Drugs and Biomarkers
Neo4j for Discovering Drugs and BiomarkersNeo4j for Discovering Drugs and Biomarkers
Neo4j for Discovering Drugs and Biomarkers
 
Neo4j Webinar: Graphs in banking
Neo4j Webinar:  Graphs in banking Neo4j Webinar:  Graphs in banking
Neo4j Webinar: Graphs in banking
 
The Customer Journey Is a Graph
The Customer Journey Is a GraphThe Customer Journey Is a Graph
The Customer Journey Is a Graph
 
Neo4j in Production: A look at Neo4j in the Real World
Neo4j in Production: A look at Neo4j in the Real WorldNeo4j in Production: A look at Neo4j in the Real World
Neo4j in Production: A look at Neo4j in the Real World
 
Neo4j Demo: Using Knowledge Graphs to Classify Diabetes Patients (GlaxoSmithK...
Neo4j Demo: Using Knowledge Graphs to Classify Diabetes Patients (GlaxoSmithK...Neo4j Demo: Using Knowledge Graphs to Classify Diabetes Patients (GlaxoSmithK...
Neo4j Demo: Using Knowledge Graphs to Classify Diabetes Patients (GlaxoSmithK...
 
Intro to Graphs and Neo4j
Intro to Graphs and Neo4jIntro to Graphs and Neo4j
Intro to Graphs and Neo4j
 
Improving Machine Learning using Graph Algorithms
Improving Machine Learning using Graph AlgorithmsImproving Machine Learning using Graph Algorithms
Improving Machine Learning using Graph Algorithms
 
Introduction to Neo4j for the Emirates & Bahrain
Introduction to Neo4j for the Emirates & BahrainIntroduction to Neo4j for the Emirates & Bahrain
Introduction to Neo4j for the Emirates & Bahrain
 
Knowledge Graphs and Generative AI
Knowledge Graphs and Generative AIKnowledge Graphs and Generative AI
Knowledge Graphs and Generative AI
 
Workshop - Neo4j Graph Data Science
Workshop - Neo4j Graph Data ScienceWorkshop - Neo4j Graph Data Science
Workshop - Neo4j Graph Data Science
 
Optimizing Your Supply Chain with the Neo4j Graph
Optimizing Your Supply Chain with the Neo4j GraphOptimizing Your Supply Chain with the Neo4j Graph
Optimizing Your Supply Chain with the Neo4j Graph
 

Viewers also liked

Persistent graphs in Python with Neo4j
Persistent graphs in Python with Neo4jPersistent graphs in Python with Neo4j
Persistent graphs in Python with Neo4jTobias Lindaaker
 
Introduction to py2neo
Introduction to py2neoIntroduction to py2neo
Introduction to py2neoNigel Small
 
Data Modeling with Neo4j
Data Modeling with Neo4jData Modeling with Neo4j
Data Modeling with Neo4jNeo4j
 
Managing Microservices with Neo4j
Managing Microservices with Neo4jManaging Microservices with Neo4j
Managing Microservices with Neo4jAshley Chloe
 
Best Practices for Front-End Django Developers
Best Practices for Front-End Django DevelopersBest Practices for Front-End Django Developers
Best Practices for Front-End Django DevelopersChristine Cheung
 
Word Puzzles with Neo4j and Py2neo
Word Puzzles with Neo4j and Py2neoWord Puzzles with Neo4j and Py2neo
Word Puzzles with Neo4j and Py2neoGrant Paton-Simpson
 
Its all about the domain honey
Its all about the domain honeyIts all about the domain honey
Its all about the domain honeyCarola Lilienthal
 
Raspberry Pi und Python
Raspberry Pi und PythonRaspberry Pi und Python
Raspberry Pi und PythonThomas Koch
 
An example graph visualization with processing
An example graph visualization with processingAn example graph visualization with processing
An example graph visualization with processingMax De Marzi
 
Neo4j Spatial - GIS for the rest of us.
Neo4j Spatial - GIS for the rest of us.Neo4j Spatial - GIS for the rest of us.
Neo4j Spatial - GIS for the rest of us.Peter Neubauer
 
Creative Data Analysis with Python
Creative Data Analysis with PythonCreative Data Analysis with Python
Creative Data Analysis with PythonGrant Paton-Simpson
 
Natural Language Processing and Graph Databases in Lumify
Natural Language Processing and Graph Databases in LumifyNatural Language Processing and Graph Databases in Lumify
Natural Language Processing and Graph Databases in LumifyCharlie Greenbacker
 
Round pegs and square holes
Round pegs and square holesRound pegs and square holes
Round pegs and square holesDaniel Greenfeld
 
A quick review of Python and Graph Databases
A quick review of Python and Graph DatabasesA quick review of Python and Graph Databases
A quick review of Python and Graph DatabasesNicholas Crouch
 
Introduction to Neo4j and .Net
Introduction to Neo4j and .NetIntroduction to Neo4j and .Net
Introduction to Neo4j and .NetNeo4j
 
Neo4j - graph database for recommendations
Neo4j - graph database for recommendationsNeo4j - graph database for recommendations
Neo4j - graph database for recommendationsproksik
 
Building social network with Neo4j and Python
Building social network with Neo4j and PythonBuilding social network with Neo4j and Python
Building social network with Neo4j and PythonAndrii Soldatenko
 
An Introduction to NOSQL, Graph Databases and Neo4j
An Introduction to NOSQL, Graph Databases and Neo4jAn Introduction to NOSQL, Graph Databases and Neo4j
An Introduction to NOSQL, Graph Databases and Neo4jDebanjan Mahata
 

Viewers also liked (20)

Persistent graphs in Python with Neo4j
Persistent graphs in Python with Neo4jPersistent graphs in Python with Neo4j
Persistent graphs in Python with Neo4j
 
Introduction to py2neo
Introduction to py2neoIntroduction to py2neo
Introduction to py2neo
 
Data Modeling with Neo4j
Data Modeling with Neo4jData Modeling with Neo4j
Data Modeling with Neo4j
 
Managing Microservices with Neo4j
Managing Microservices with Neo4jManaging Microservices with Neo4j
Managing Microservices with Neo4j
 
Lead Nurturing
Lead NurturingLead Nurturing
Lead Nurturing
 
Best Practices for Front-End Django Developers
Best Practices for Front-End Django DevelopersBest Practices for Front-End Django Developers
Best Practices for Front-End Django Developers
 
Word Puzzles with Neo4j and Py2neo
Word Puzzles with Neo4j and Py2neoWord Puzzles with Neo4j and Py2neo
Word Puzzles with Neo4j and Py2neo
 
Its all about the domain honey
Its all about the domain honeyIts all about the domain honey
Its all about the domain honey
 
Raspberry Pi und Python
Raspberry Pi und PythonRaspberry Pi und Python
Raspberry Pi und Python
 
An example graph visualization with processing
An example graph visualization with processingAn example graph visualization with processing
An example graph visualization with processing
 
Neo4j Spatial - GIS for the rest of us.
Neo4j Spatial - GIS for the rest of us.Neo4j Spatial - GIS for the rest of us.
Neo4j Spatial - GIS for the rest of us.
 
Creative Data Analysis with Python
Creative Data Analysis with PythonCreative Data Analysis with Python
Creative Data Analysis with Python
 
Natural Language Processing and Graph Databases in Lumify
Natural Language Processing and Graph Databases in LumifyNatural Language Processing and Graph Databases in Lumify
Natural Language Processing and Graph Databases in Lumify
 
Round pegs and square holes
Round pegs and square holesRound pegs and square holes
Round pegs and square holes
 
A quick review of Python and Graph Databases
A quick review of Python and Graph DatabasesA quick review of Python and Graph Databases
A quick review of Python and Graph Databases
 
Neo4j in Depth
Neo4j in DepthNeo4j in Depth
Neo4j in Depth
 
Introduction to Neo4j and .Net
Introduction to Neo4j and .NetIntroduction to Neo4j and .Net
Introduction to Neo4j and .Net
 
Neo4j - graph database for recommendations
Neo4j - graph database for recommendationsNeo4j - graph database for recommendations
Neo4j - graph database for recommendations
 
Building social network with Neo4j and Python
Building social network with Neo4j and PythonBuilding social network with Neo4j and Python
Building social network with Neo4j and Python
 
An Introduction to NOSQL, Graph Databases and Neo4j
An Introduction to NOSQL, Graph Databases and Neo4jAn Introduction to NOSQL, Graph Databases and Neo4j
An Introduction to NOSQL, Graph Databases and Neo4j
 

Similar to Django and Neo4j - Domain modeling that kicks ass

NOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4jNOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4jTobias Lindaaker
 
NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)
NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)
NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)Emil Eifrem
 
NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)
NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)
NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)Emil Eifrem
 
NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)
NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)
NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)Emil Eifrem
 
A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)
A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)
A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)Emil Eifrem
 
Spring Data Neo4j Intro SpringOne 2011
Spring Data Neo4j Intro SpringOne 2011Spring Data Neo4j Intro SpringOne 2011
Spring Data Neo4j Intro SpringOne 2011jexp
 
Silicon valley nosql meetup april 2012
Silicon valley nosql meetup  april 2012Silicon valley nosql meetup  april 2012
Silicon valley nosql meetup april 2012InfiniteGraph
 
No Sql Movement
No Sql MovementNo Sql Movement
No Sql MovementAjit Koti
 
MapReduce and Its Discontents
MapReduce and Its DiscontentsMapReduce and Its Discontents
MapReduce and Its DiscontentsDean Wampler
 
NoSQL – Back to the Future or Yet Another DB Feature?
NoSQL – Back to the Future or Yet Another DB Feature?NoSQL – Back to the Future or Yet Another DB Feature?
NoSQL – Back to the Future or Yet Another DB Feature?Martin Scholl
 
An Introduction to Big Data, NoSQL and MongoDB
An Introduction to Big Data, NoSQL and MongoDBAn Introduction to Big Data, NoSQL and MongoDB
An Introduction to Big Data, NoSQL and MongoDBWilliam LaForest
 
Big Data Israel Meetup : Couchbase and Big Data
Big Data Israel Meetup : Couchbase and Big DataBig Data Israel Meetup : Couchbase and Big Data
Big Data Israel Meetup : Couchbase and Big DataTugdual Grall
 
Soeren okfn greece meetup
Soeren okfn greece meetupSoeren okfn greece meetup
Soeren okfn greece meetupOKFN-GR
 
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQLA STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQLijscai
 
A Study on Graph Storage Database of NOSQL
A Study on Graph Storage Database of NOSQLA Study on Graph Storage Database of NOSQL
A Study on Graph Storage Database of NOSQLIJSCAI Journal
 
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQLA STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQLijscai
 

Similar to Django and Neo4j - Domain modeling that kicks ass (20)

NOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4jNOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4j
 
NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)
NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)
NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)
 
NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)
NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)
NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)
 
NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)
NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)
NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)
 
A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)
A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)
A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)
 
Spring Data Neo4j Intro SpringOne 2011
Spring Data Neo4j Intro SpringOne 2011Spring Data Neo4j Intro SpringOne 2011
Spring Data Neo4j Intro SpringOne 2011
 
Silicon valley nosql meetup april 2012
Silicon valley nosql meetup  april 2012Silicon valley nosql meetup  april 2012
Silicon valley nosql meetup april 2012
 
Eifrem neo4j
Eifrem neo4jEifrem neo4j
Eifrem neo4j
 
No Sql Movement
No Sql MovementNo Sql Movement
No Sql Movement
 
Anti-social Databases
Anti-social DatabasesAnti-social Databases
Anti-social Databases
 
MapReduce and Its Discontents
MapReduce and Its DiscontentsMapReduce and Its Discontents
MapReduce and Its Discontents
 
NoSQL – Back to the Future or Yet Another DB Feature?
NoSQL – Back to the Future or Yet Another DB Feature?NoSQL – Back to the Future or Yet Another DB Feature?
NoSQL – Back to the Future or Yet Another DB Feature?
 
An Introduction to Big Data, NoSQL and MongoDB
An Introduction to Big Data, NoSQL and MongoDBAn Introduction to Big Data, NoSQL and MongoDB
An Introduction to Big Data, NoSQL and MongoDB
 
Big Data Israel Meetup : Couchbase and Big Data
Big Data Israel Meetup : Couchbase and Big DataBig Data Israel Meetup : Couchbase and Big Data
Big Data Israel Meetup : Couchbase and Big Data
 
Soeren okfn greece meetup
Soeren okfn greece meetupSoeren okfn greece meetup
Soeren okfn greece meetup
 
Sql no sql
Sql no sqlSql no sql
Sql no sql
 
STI Summit 2011 - Digital Worlds
STI Summit 2011 - Digital WorldsSTI Summit 2011 - Digital Worlds
STI Summit 2011 - Digital Worlds
 
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQLA STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
 
A Study on Graph Storage Database of NOSQL
A Study on Graph Storage Database of NOSQLA Study on Graph Storage Database of NOSQL
A Study on Graph Storage Database of NOSQL
 
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQLA STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
 

More from Tobias Lindaaker

Building Applications with a Graph Database
Building Applications with a Graph DatabaseBuilding Applications with a Graph Database
Building Applications with a Graph DatabaseTobias Lindaaker
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j InternalsTobias Lindaaker
 
Choosing the right NOSQL database
Choosing the right NOSQL databaseChoosing the right NOSQL database
Choosing the right NOSQL databaseTobias Lindaaker
 
[JavaOne 2011] Models for Concurrent Programming
[JavaOne 2011] Models for Concurrent Programming[JavaOne 2011] Models for Concurrent Programming
[JavaOne 2011] Models for Concurrent ProgrammingTobias Lindaaker
 
A Better Python for the JVM
A Better Python for the JVMA Better Python for the JVM
A Better Python for the JVMTobias Lindaaker
 
A Better Python for the JVM
A Better Python for the JVMA Better Python for the JVM
A Better Python for the JVMTobias Lindaaker
 
Exploiting Concurrency with Dynamic Languages
Exploiting Concurrency with Dynamic LanguagesExploiting Concurrency with Dynamic Languages
Exploiting Concurrency with Dynamic LanguagesTobias Lindaaker
 

More from Tobias Lindaaker (9)

NOSQL Overview
NOSQL OverviewNOSQL Overview
NOSQL Overview
 
Building Applications with a Graph Database
Building Applications with a Graph DatabaseBuilding Applications with a Graph Database
Building Applications with a Graph Database
 
JDK Power Tools
JDK Power ToolsJDK Power Tools
JDK Power Tools
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j Internals
 
Choosing the right NOSQL database
Choosing the right NOSQL databaseChoosing the right NOSQL database
Choosing the right NOSQL database
 
[JavaOne 2011] Models for Concurrent Programming
[JavaOne 2011] Models for Concurrent Programming[JavaOne 2011] Models for Concurrent Programming
[JavaOne 2011] Models for Concurrent Programming
 
A Better Python for the JVM
A Better Python for the JVMA Better Python for the JVM
A Better Python for the JVM
 
A Better Python for the JVM
A Better Python for the JVMA Better Python for the JVM
A Better Python for the JVM
 
Exploiting Concurrency with Dynamic Languages
Exploiting Concurrency with Dynamic LanguagesExploiting Concurrency with Dynamic Languages
Exploiting Concurrency with Dynamic Languages
 

Recently uploaded

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 

Recently uploaded (20)

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 

Django and Neo4j - Domain modeling that kicks ass

  • 1. Django and Neo4j Domain modeling that kicks ass! twitter: @thobe / #neo4j Tobias Ivarsson email: tobias@neotechnology.com web: http://www.neo4j.org/ Hacker @ Neo Technology web: http://www.thobe.org/
  • 2. It all started with this guy. Emil Eifrem, CEO of Neo Technology. We picked him apart, and inside his brain we found the base for a database that models the connections in bet ween entities. 2
  • 3. It all started with this guy. Emil Eifrem, CEO of Neo Technology. We picked him apart, and inside his brain we found the base for a database that models the connections in bet ween entities. Image credits: US Army 2
  • 4. It all started with this guy. Emil Eifrem, CEO of Neo Technology. We picked him apart, and inside his brain we found the base for a database that models the connections in bet ween entities. 2
  • 5. Neo4j It all started with this guy. Emil Eifrem, CEO of Neo Technology. We picked him apart, and inside his brain we found the base for a database that models the connections in bet ween entities. 2
  • 6. NOSQL is a wide area 3
  • 7. The problems NOSQL focuses on Relational database Requirement of application Focus area of many NOSQL Databases ๏ Huge amounts of data Performance ๏ (mostly) Disjoint data Salary List ๏ Heavy load most focus on... Majority of Webapps ๏ Many concurrent writers Social network Semantic Trading All NOSQL databases focus on solving problems where RDBMSes fail. While this handles the load, it lacks in “social” } custom Data complexity 4
  • 8. The evolution of data Giant Global Graph (GGG) Ontologies RDF Folksonomies Information connectivity Tagging Wikis User-generated content Blogs ... but it turns out that RSS data evolves to become MORE interconnected Hypertext (as well as greater sizes) Text documents web 1.0 web 2.0 “web 3.0” 1990 2000 2010 2020 5
  • 9. Neo4j is a Graph Database Graph databases FOCUS on the interconnection bet ween entities. 6
  • 10. IS_A Neo4j Graph Database Graph databases FOCUS on the interconnection bet ween entities. 6
  • 11. Scaling to size vs. Scaling to complexity Size Key/Value stores Bigtable clones Document databases Graph databases Complexity 7
  • 12. Scaling to size vs. Scaling to complexity Size Key/Value stores Bigtable clones Document databases Graph databases Billions of nodes and relationships > 90% of use cases Complexity 7
  • 13. What is Neo4j? ๏ Neo4j is a Graph Database • Non-relational (“#nosql”), transactional (ACID), embedded • Data is stored as a Graph / Network ‣Nodes and relationships with properties ‣“Property Graph” or “edge-labeled multidigraph” • Schema free, bottom-up data model design ๏ Neo4j is Open Source / Free (as in speech) Software Prices are available at http://neotechnology.com/ • AGPLv3 Contact us if you have questions and/or special license needs (e.g. if you • Commercial (“dual license”) license available want an evaluation license) ‣First server is free (as in beer), next is inexpensive 8
  • 14. More about Neo4j ๏ Neo4j is stable • In 24/7 operation since 2003 ๏ Neo4j is in active development • Neo Technology received VC funding October 2009 ๏ Neo4j delivers high performance graph operations • traverses 1’000’000+ relationships / second on commodity hardware (1000~2500 traversals/ms) 9
  • 15. Building business applications with Neo4j ๏ Try it out! It’s all open source! • Build a prototype, find out your needs and how Neo4j matches • AGPL this stage thisusers should have access to your code - at says all your is your employees / co-workers 10
  • 16. Building business applications with Neo4j ๏ Try it out! It’s all open source! • Build a prototype, find out your needs and how Neo4j matches • AGPL this stage thisusers should have access to your code - at says all your is your employees / co-workers ๏ Put it in front of users! The license is free for the first server! • Contact Neo Technology sales to get a free single server license • You’ll (probably) not have massive load the first days 10
  • 17. Building business applications with Neo4j ๏ Try it out! It’s all open source! • Build a prototype, find out your needs and how Neo4j matches • AGPL this stage thisusers should have access to your code - at says all your is your employees / co-workers ๏ Put it in front of users! The license is free for the first server! • Contact Neo Technology sales to get a free single server license • You’ll (probably) not have massive load the first days ๏ As you grow, Neo4j grows with you! • Aslicenseneeds and revenue increase you can by an advanced your (prices are resonable) 10
  • 18. Graphs are all around us A B C D ... 1 17 3.14 3 17.79333333333 2 42 10.11 14 30.33 3 316 6.66 1 2104.56 4 32 9.11 592 0.492432432432 5 Even if this spreadsheet looks like it could be a fit for a RDBMS 2153.175765766 it isn’t: •RDBMSes have problems with ... extending indefinitely on both rows and columns •Formulas and data dependencies would quickly lead to heavy join operations 11
  • 19. Graphs are all around us A B C D ... 1 17 3.14 3 = A1 * B1 / C1 2 42 10.11 14 = A2 * B2 / C2 3 316 6.66 1 = A3 * B3 / C3 4 32 9.11 592 = A4 * B4 / C4 5 = SUM(D2:D5) With data dependencies ... the spread sheet turns out to be a graph. 12
  • 20. Graphs are all around us A B C D ... 1 17 3.14 3 = A1 * B1 / C1 2 42 10.11 14 = A2 * B2 / C2 3 316 6.66 1 = A3 * B3 / C3 4 32 9.11 592 = A4 * B4 / C4 5 = SUM(D2:D5) With data dependencies ... the spread sheet turns out to be a graph. 12
  • 21. Graphs are all around us If we add external data sources the problem becomes even more interesting... 17 3.14 3 = A1 * B1 / C1 42 10.11 14 = A2 * B2 / C2 316 6.66 1 = A3 * B3 / C3 32 9.11 592 = A4 * B4 / C4 = SUM(D2:D5) 13
  • 22. Graphs are all around us If we add external data sources the problem becomes even more interesting... 17 3.14 3 = A1 * B1 / C1 42 10.11 14 = A2 * B2 / C2 316 6.66 1 = A3 * B3 / C3 32 9.11 592 = A4 * B4 / C4 = SUM(D2:D5) 13
  • 23. The Neo4j Graph data model •Nodes •Relationships bet ween Nodes •Relationships have Labels •Relationships are directed, but traversed at equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
  • 24. The Neo4j Graph data model •Nodes •Relationships bet ween Nodes •Relationships have Labels •Relationships are directed, but traversed at equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
  • 25. The Neo4j Graph data model LIVES WITH LOVES OWNS DRIVES •Nodes •Relationships bet ween Nodes •Relationships have Labels •Relationships are directed, but traversed at equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
  • 26. The Neo4j Graph data model LOVES LIVES WITH LOVES OWNS DRIVES •Nodes •Relationships bet ween Nodes •Relationships have Labels •Relationships are directed, but traversed at equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
  • 27. The Neo4j Graph data model name: “Mary” LOVES name: “James” age: 35 age: 32 LIVES WITH twitter: “@spam” LOVES OWNS DRIVES •Nodes •Relationships bet ween Nodes •Relationships have Labels brand: “Volvo” •Relationships are directed, but traversed at model: “V70” equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
  • 28. The Neo4j Graph data model name: “Mary” LOVES name: “James” age: 35 age: 32 LIVES WITH twitter: “@spam” LOVES OWNS item type: “car” DRIVES •Nodes •Relationships bet ween Nodes •Relationships have Labels brand: “Volvo” •Relationships are directed, but traversed at model: “V70” equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
  • 29. Graphs are Whiteboard Friendly The domain I specify is the domain I implement. No mismatch, no ER-modeling. 15
  • 30. Graphs are Whiteboard Friendly The domain I specify is the domain I implement. odin No mismatch, no ER-modeling. thobe dude #17 #6 #14 Wardrobe Strength Joe project #32 Call site caching Hello world OSCON Best panncakes Optimizing Jython 15
  • 31. Graphs are Whiteboard Friendly odin dude #17 #6 #14 username: “thobe” Wardrobe Strength Joe project #32 name: “Tobias Ivarsson” twitter: “@thobe” password: “**********” Call site caching Hello world OSCON Best panncakes Optimizing Jython 16
  • 32. Graphs are Whiteboard Friendly odin thobe dude #17 #6 #14 address: “http://journal.thobe.org” #32 title: “Wardrobe Strengthproject Joe ” tagline: “Good enough thoughts” Call site caching Hello world OSCON Best panncakes Optimizing Jython 17
  • 33. Building a graph - the basic API import neo4j grapDb = neo4j.GraphDatabase( PATH_TO_YOUR_NEO4J_DATASTORE ) with graphDb.transaction: # All writes require transactions # Create Thomas 'Neo' Anderson mrAnderson = graphDb.node(name="Thomas Anderson", age=29) # Create Morpheus morpheus = graphDb.node(name="Morpheus", rank= "Captain", occupation= "Total bad ass") # Create relationship representing they know each other mrAnderson.KNOWS( morpheus ) # ... similarly for Trinity, Cypher, Agent Smith, Architect 18
  • 34. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” 19
  • 35. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] order = neo4j.BREADTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE 20
  • 36. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] order = neo4j.BREADTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
  • 37. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] order = neo4j.BREADTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
  • 38. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
  • 39. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
  • 40. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH Cypher (@ depth=2) returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
  • 41. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH Cypher (@ depth=2) returnable = neo4j.RETURN_ALL_BUT_START_NODE Agent Smith (@ depth=3) for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
  • 42. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH Cypher (@ depth=2) returnable = neo4j.RETURN_ALL_BUT_START_NODE Agent Smith (@ depth=3) for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
  • 43. Finding a place to start ๏ Traversals need a Node to start from • QUESTION: How do I find the start Node? • ANSWER:You use an Index ๏ Indexes in Neo4j are different from Indexes in Relational Databases • RDBMSes use them for Joining • Neo4j use them for simple lookup index = graphDb.index["name"] mr_anderson = index["Thomas Anderson"] performTraversalFrom( mrAnderson ) 21
  • 44. Indexes in Neo4j ๏ The Graph *is* the main index • Use relationship labels for navigation • Build index structures *in the graph* ‣Search trees, tag clouds, geospatial indexes, et.c. ‣Linked/skip lists or other data structures in the graph ‣We have utility libraries for this ๏ External indexes used *for lookup* • Finding a (number of) points to start traversals from • Major difference from RDBMS that use indexes for everything 22
  • 45. Django integration does all of this for you! 23
  • 46. Implementing the domain user blog comment entry 24
  • 47. from neo4j.model import django_model as models user blog comment entry 25
  • 48. from neo4j.model import django_model as models class User(models.NodeModel): username = models.Property(indexed=True) name = models.Property() blog comment blogs = models.Relationship(Blog, type=models.Outgoing.member_of, related_name="users") def __unicode__(self): return self.name entry 25
  • 49. from neo4j.model import django_model as models user class Blog(models.NodeModel): identifier = models.Property(indexed=True) comment title = models.Property() def __unicode__(self): return self.title entry 26
  • 50. from neo4j.model import django_model as models user class Entry(models.NodeModel): title = models.Property() text = models.Property() blog comment date = models.Property() blog = models.Relationship(Blog, type=models.Outgoing.posted_on, single=True, optional=False, related_name="articles") author = models.Relationship(User, type=models.Outgoing.authored_by, single=True, optional=False, related_name="articles") 27
  • 51. models.py from neo4j.model import django_model as models class Blog(models.NodeModel): identifier = models.Property(indexed=True) title = models.Property() The rest of the code for working with the domain class User(models.NodeModel): objects is (mostly) the same username = models.Property(indexed=True) as you are used to in Django. name = models.Property() blogs = models.Relationship(Blog, type=models.Outgoing.member_of, related_name="users") class Entry(models.NodeModel): title = models.Property() text = models.Property() date = models.Property() blog = models.Relationship(Blog, type=models.Outgoing.posted_on, single=True, optional=False, related_name="articles") author = models.Relationship(User, type=models.Outgoing.authored_by, single=True, optional=False, 28 related_name="articles")
  • 52. Why not use an O/R mapper? ๏ Model evolution in ORMs is a hard problem • virtually unsupported in most ORM systems ๏ SQL is “compatible” across many RDBMSs • data is still locked in ๏ Each ORM maps object models differently • Moving to another ORM == legacy schema support ‣except your legacy schema is a strange auto-generated one ๏ Object/Graph Mapping is always done the same way • allows you to keep your data through application changes • or share data between multiple implementations 29
  • 53. What an ORM doesn’t do ๏Deep traversals ๏Graph algorithms ๏Shortest path(s) ๏Routing ๏etc. 30
  • 54. Path exists in social network ๏ Each person has on average 50 friends The performance impact in Neo4j depends only on the degree of each node. in Tobias an RDBMS it depends on the number of entries in the tables involved in the join(s). Emil Johan Peter Database # persons query time Relational database 1 000 2 000 ms Neo4j Graph Database 1 000 2 ms Neo4j Graph Database 1 000 000 2 ms Relational database 1 000 000 way too long... 31
  • 55. Path exists in social network ๏ Each person has on average 50 friends The performance impact in Neo4j depends only on the degree of each node. in Tobias an RDBMS it depends on the number of entries in the tables involved in the join(s). Emil Johan Peter Database # persons query time Relational database 1 000 2 000 ms Neo4j Graph Database 1 000 2 ms Neo4j Graph Database 1 000 000 2 ms Relational database 1 000 000 way too long... 31
  • 56. On-line real time routing with Neo4j ๏ 20 million Nodes - represents places ๏ 62 million Edges - represents direct roads between places • These edges have a length property, for the length of the road ๏ Average optimal route, 100 separate roads, found in 100ms ๏ Worst case route we could find: • Optimal route is 5500 separate roads • Total length ~770km There’s a difference • Found in less than 3 seconds bet ween least number of hops and least cost. ๏ Uses A* “best first” search 32
  • 57. Jython vs. CPython ๏ Neo4j with the Python bindings work in both • Requires no code modification in your code ๏ Neo4j at its core is an Embedded (in-process) database • CPython manages concurrency by forking multiple processes • Jython has full concurrency support in the same JVM • Stand-alone Neo4jon is being worked server-process with (C)Python client ๏ Neo4j has a RESTful interface • There are Python clients • The API differs slightly (no transactions) 33
  • 58. Finding out more ๏ http://neo4j.org/ - project website - main place for getting started ‣Contains screen casts, download links, et.c. ‣http://api.neo4j.org/ and http://components.neo4j.org/ ‣Specifically http://components.neo4j.org/neo4j.py/ ‣http://wiki.neo4j.org/ - HowTos, Tutorials, Examples, FAQ, et.c. ‣http://planet.neo4j.org/ - aggregation of blogs about Neo4j ‣http://github.com/neo4j-examples - small example applications ๏ https://lists.neo4j.org/ - community mailing list ๏ http://twitter.com/neo4j/team - follow the Neo4j team ๏ http://neotechnology.com/ - commercial licensing 34
  • 59. Helping out! ๏ Neo4j and the Python integration is all Open Source ๏ The Python bindings in particular would benefit from more devs... • Integrate more of the Neo4j components ‣Neo4j Spatial ‣The Graph Algorithms package ‣The Graph Matching component • Trimming off the rough edges in the Django integration • Native client for CPython 35
  • 60. Buzzword summary http://neo4j.org/ Semi structured SPARQL AGPLv3 ACID transactions Open Source Object mapping Gremlin Shortest path NOSQL Traversal RESTful Software Transactional Memory Query language whiteboard friendly Beer A* routing Embedded Schema free Scaling to complexity Free Software Polyglot persistence 36