Designing and Building a Graph Database Application - Ian Robinson (Neo Technology)

Twi$er:
@ianSrobinson
#neo4j

Designing
and
Building
a
Graph

Database
Applica5on

Roadmap

•  Complex
data

•  Designing
a
graph
data
model
and
queries

•  Tes@ng

Data
Complexity

complexity = f(size

, semi-structure, connectedness)

Network
Impact
Analysis

Securi@es
and
Debt

Image:
orgnet.com

Graph
Databases

•  Store

•  Manage

•  Query

data

Neo4j
is
a
Graph
Database

Labeled
Property
Graph

Designing
a
Graph
Model

Models

Purposeful
abstrac@on
of
a
domain
designed
to

sa@sfy
par@cular
applica@on/end-‐user
goals

Images:
en.wikipedia.org

Applica@on/End-‐User
Goals

As
an
emp

loyee

I
want
to
k
now
who
i
n
the
comp
has
similar
any

skills
to
m

e

So
that
we

can
excha
nge
knowl
edge

Ques@ons
To
Ask
of
the
Domain

As
an
emp
loyee

I
want
to
k
now
who
i
n
the
co
has
similar

skills
to
me mpany

So
that
we

can
excha
nge
knowle
dge

Which
people,
who
work
for
the
same
company

as
me,
have
similar
skills
to
me?

Iden@fy
En@@es

Which
people,
who
work
for
the
same
company

as
me,
have
similar
skills
to
me?

Person

Company

Skill

Iden@fy
Rela@onships
Between
En@@es

Which
people,
who
work
for
the
same
company

as
me,
have
similar
skills
to
me?

Person
WORKS_FOR
Company

Person
HAS_SKILL
Skill

Convert
to
Cypher
Paths

Rela@onship

Person
WORKS_FOR
Company

Person
HAS_SKILL
Skill

Label

(:Person)-[:WORKS_FOR]->(:Company),
(:Person)-[:HAS_SKILL]->(:Skill)

Consolidate
Paths

(:Person)-[:WORKS_FOR]->(:Company),
(:Person)-[:HAS_SKILL]->(:Skill)

(:Company)<-[:WORKS_FOR]-(:Person)-[:HAS_SKILL]->(:Skill)

Candidate
Data
Model


Express
Ques@on
as
Graph
Pa$ern

Which
people,
who
work
for
the
same
company

as
me,
have
similar
skills
to
me?

Cypher
Query

Which
people,
who
work
for
the
same
company

as
me,
have
similar
skills
to
me?

MATCH (company)<-[:WORKS_FOR]-(me:Person)-[:HAS_SKILL]->(skill),
(company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill)
WHERE me.name = {name}
RETURN colleague.name AS name,
count(skill) AS score,
collect(skill.name) AS skills
ORDER BY score DESC

Graph
Pa$ern

Which
people,
who
work
for
the
same
company

as
me,
have
similar
skills
to
me?

ORDER BY score DESC

Anchor
Pa$ern
in
Graph

Which
people,
who
work
for
the
same
company

as
me,
have
similar
skills
to
me?

ORDER BY score DESC

Search
nodes
labeled

‘Person’,
matching
on

‘name’
property

Create
Projec@on
of
Results

Which
people,
who
work
for
the
same
company

as
me,
have
similar
skills
to
me?

ORDER BY score DESC

From
User
Story
to
Model
and
Query

As
an
emp
loyee

I
want
to
k
now
who
i
n
the
co
has
similar

skills
to
me mpany

So
that
we

can
excha
nge
knowle
dge

ORDER BY score DESC

?

Which
people,
who
work
for
the
same

company
as
me,
have
similar
skills
to
me?

Person
WORKS_FOR
Company

Person
HAS_SKILL
Skill


Why
Test?

•  Ensure
model
is
ﬁt
for
queries

–  Rapid
feedback

•  Ensure
correctness
of
queries

•  Document
your
understanding
of
your
domain

–  Including
corner
cases
and
excep@ons

•  Provide
a
regression
test
suite

–  Allows
you
to
change
and
evolve
model
and

queries

Method

•  Develop
queries,
or
classes
that
encapsulate

queries,
using
unit
tests

•  Use
small,
well-‐understood
datasets
in
each
test

–  Create
data
in
test
setup

–  Test
dataset
expresses
your
understanding
of
(part
of)

the
domain

•  Inject
in-‐memory
graph
database
(or
Cypher

engine)
into
object
under
test

•  The
exact
strategy
you
use
depends
on
your

applica@on
architecture…

Applica@on
Architectures

•  Embedded

•  Server

•  Server
with
Extensions

Applica@on
Architectures

•  Embedded

–  Host
in
Java
process

–  Access
to
Java
APIs

•  Server

•  Server
with
Extensions

Applica@on

Java
APIs

Applica@on
Architectures

•  Embedded

•  Server

–  HTTP/JSON
interface

–  Server
wraps
embedded

instance

•  Server
with
Extensions

Applica@on

REST
Client

Write
LB

REST
API

REST
API

Read
LB

REST
API

Applica@on
Architectures

•  Embedded

•  Server

•  Server
with
Extensions

REST
API

–  Execute
complex
logic
on
server

–  Control
HTTP
request/response
format

Extensions

Embedded
Example

•  Company
social
network

•  Find
colleagues
with
similar
skills

•  Encapsulate
query
in
a
ColleagueFinder

Unit
Test
Fixture

public class ColleagueFinderTest {

private GraphDatabaseService db;
private ColleagueFinder finder;

@Before
public void init() {
db = new TestGraphDatabaseFactory().newImpermanentDatabase();
ExampleGraph.populate( db );
finder = new ColleagueFinder( new ExecutionEngine( db ) );
}

@After
public void shutdown() {
db.shutdown();
}
}

Create
Database



@Before
}

@After
db.shutdown();
}
}

Populate
Graph



@Before
}

@After
db.shutdown();
}
}

Create
Object
Under
Test



@Before
}

@After
db.shutdown();
}
}

Inject

Execu@onEngine

ImpermanentGraphDatabase

•  In-‐memory

•  For
tes@ng
only,
not
produc@on!

<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-kernel</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>

Create
Sample
Data

public static void populate( GraphDatabaseService db ) {

ExecutionEngine engine = new ExecutionEngine( db );

String cypher =
"CREATE ian:Person VALUES {name:'Ian'},n" +
"
bill:Person VALUES {name:'Bill'},n" +
"
lucy:Person VALUES {name:'Lucy'},n" +
"
acme:Company VALUES {name:'Acme'},n" +

// Cypher continues...

"
"
"
"

(bill)-[:HAS_SKILL]->(neo4j),n" +
(bill)-[:HAS_SKILL]->(ruby),n" +
(lucy)-[:HAS_SKILL]->(java),n" +
(lucy)-[:HAS_SKILL]->(neo4j)";

engine.execute( cypher );
}

Unit
Test

@Test
public void shouldFindColleaguesWithSimilarSkills() throws Exception {

// when
Iterator<Map<String, Object>> results = finder.findColleaguesFor( "Ian" );

// then
assertEquals( "Lucy", results.next().get( "name" ) );
assertEquals( "Bill", results.next().get( "name" ) );

assertFalse( results.hasNext() );
}

Execute
Object
Under
Test

@Test

// when

// then

}

Assert
Results

@Test

// when

// then

}

Ensure
No
More
Results

@Test

// when

// then

}

ColleagueFinder

public class ColleagueFinder {

private final ExecutionEngine executionEngine;

public ColleagueFinder( ExecutionEngine executionEngine ) {
this.executionEngine = executionEngine;
}

public Iterator<Map<String, Object>> findColleaguesFor( String name ) {
...
}
}

Inject
Execu@onEngine

public class ColleagueFinder {

private final ExecutionEngine executionEngine;

public ColleagueFinder( ExecutionEngine executionEngine ) {
this.executionEngine = executionEngine;
}

...
}
}

ﬁndColleaguesFor()
Method


String cypher =
"MATCH (me:Person)-[:WORKS_FOR]->(company),n" +
"
(me)-[:HAS_SKILL]->(skill),n" +
"
(colleague)-[:WORKS_FOR]->(company),n" +
"
(colleague)-[:HAS_SKILL]->(skill)n" +
"WHERE me.name = {name}n" +
"RETURN colleague.name AS name,n" +
"
count(skill) AS score,n" +
"
collect(skill.name) AS skillsn" +
"ORDER BY score DESC";

Map<String, Object> params = new HashMap<String, Object>();
params.put( "name", name );

return executionEngine.execute( cypher, params ).iterator();
}

Cypher
Query


String cypher =
"
"
"
"
"


}

Parameterized
Query


String cypher =
"
"
"
"
"


}

Execute
Query


String cypher =
"
"
"
"
"


}

Server
Extension
Example

•  Same
data
mode
and
query
as
before

•  This
@me,
we’ll
host
ColleagueFinder
in
a

server
extension

Server
Extension

@Path("/similar-skills")
public class ColleagueFinderExtension {
private static final ObjectMapper MAPPER = new ObjectMapper();
private final ColleagueFinder colleagueFinder;

public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) {
this.colleagueFinder = new ColleagueFinder( cypherExecutor.getExecutionEngine() );
}

@GET
@Produces(MediaType.APPLICATION_JSON)
@Path("/{name}")
public Response getColleagues( @PathParam("name") String name )
throws IOException {

String json = MAPPER
.writeValueAsString( colleagueFinder.findColleaguesFor( name ) );
return Response.ok().entity( json ).build();
}
}

JAX-‐RS
Annota@ons


}

@GET
@Path("/{name}")

}
}

Map
HTTP
Request
to
Object
+
Method


}

@GET
@Path("/{name}")

}
}

GET

/similar-‐skills
/Ian

CypherExecutor
Injected
by
Server

Ensures

Execu@onEngine

reused
across

resource
instances


}

@GET
@Path("/{name}")

}
}

Generate
and
Format
Response


}

@GET
@Path("/{name}")

}
}

Extension
Test
Fixture

public class ColleagueFinderExtensionTest {
private CommunityNeoServer server;

@Before
public void startServer() throws IOException
{
server = CommunityServerBuilder.server()
.withThirdPartyJaxRsPackage(
"org.neo4j.good_practices", "/colleagues" )
.build();
server.start();

ExampleGraph.populate( server.getDatabase().getGraph() );
}

@After
public void stopServer() {
server.stop();
}
}

Build
and
Conﬁgure
Server


@Before
{
.build();
server.start();

}

@After
server.stop();
}
}

Start
Server


@Before
{
.build();
server.start();

}

@After
server.stop();
}
}

Populate
Database


@Before
{
.build();
server.start();

}

@After
server.stop();
}
}

CommunityServerBuilder

•  Programma@c
conﬁgura@on

<dependency>
<groupId>org.neo4j.app</groupId>
<artifactId>neo4j-server</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
</dependency>

Tes@ng
Extension
Using
HTTP
Client

@Test
public void shouldReturnColleaguesWithSimilarSkills() throws Exception {

Client client = Client.create( new DefaultClientConfig() );

WebResource resource = client
.resource( "http://localhost:7474/colleagues/similar-skills/Ian" );

ClientResponse response = resource
.accept( MediaType.APPLICATION_JSON )
.get( ClientResponse.class );

List<Map<String, Object>> results = new ObjectMapper()
.readValue(response.getEntity( String.class ), List.class );

// Assertions

...

Create
HTTP
Client

@Test





// Assertions

...

Issue
Request

@Test





// Assertions

...

Parse
Response

@Test





// Assertions

...

Assert
Results

...

assertEquals( 200, response.getStatus() );
assertEquals( MediaType.APPLICATION_JSON,
response.getHeaders().get( "Content-Type" ).get( 0 ) );

assertEquals( "Lucy", results.get( 0 ).get( "name" ) );
assertThat( (Iterable<String>) results.get( 0 ).get( "skills" ),
hasItems( "Java", "Neo4j" ) );
}

ts gy
en lo
i m no
pl h
m ec
Co eo T
N
of

Graph
h
Databases

Thank
you

Twi$er:
@ianSrobinson

#neo4j

Neo4j
User
Group

30th
Oct

Skillsma$er

Ian Robinson,
Jim Webber & Emil Eifrem

github.com/iansrobinson/neo4j-‐good-‐prac@ces

Designing and Building a Graph Database Application - Ian Robinson (Neo Technology)

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Designing and Building a Graph Database Application - Ian Robinson (Neo Technology) (20)

More from jaxLondonConference (17)

Recently uploaded (20)

Designing and Building a Graph Database Application - Ian Robinson (Neo Technology)