neodj ”= Q Neo4j Admin import Contents ‘+ Import a small data set * CSV file delimiters ‘+ Using separate header files * Multiple input files * Using the same label for every node ‘+ Using the same relationship type for every relationship * Properties * ID space ‘+ Skip relationships referring to missing nodes * Skip nodes with same ID This tutorial provides detailed examples to illustrate the capabilities of importing data from CSV files with the command neo4j-admin import The neo4j-admin import is a command for loading large amounts of data from CSV files into an unused non-existing database. Importing data from CSV files with neo4j-admin import can only be done once into an unused database, it is used for initial graph population only. The neo4j-admin import command can be used on the local Neo4j instance even if the instance is running or not. ‘The neo4j-admin import command does not create a database, the command only imports data and make it available for the database, The database must not exist before the neo4j-admin import command has been executed, and the database should be created afterwards, The command will exit with an error message if the database already exists. Relationships are created by connecting node IDs, each node should have a unique ID to be able to be referenced when creating relationships between nodes. In the following examples, the node IDs arestored as properties on the nodes. If you do not want the IDs to persist as properties after the import completes, then do not specify a property name in the :10 field. The examples show how to import data in a standalone Neo4j DBMS. They use: ‘The Neodj tarball (Unix console application) $NEO4J_HOME as the current working directory. + The default database neo4j * The import directory of the Neo4j installation to store all the CSV files. However, the CSV files can be located in any directory of your file system + UNIX styled paths + The neodj-admin import command. To create a cluster based on imported data, see Seed a cluster using the import tool. Handy tips: + The details of CSV file header format can be found at CSV header format. + To show available databases, use the Cypher query SHOW DATABASES against the system database. + To remove a database, use the Cypher query DROP DATABASE database_name against the system database, + To create a database, use the Cypher query CREATE DATABASE database_name against the system database, Import a small data set In this example, you will import a small data set containing nodes and relationships. The data set is split into three CSV files, where each file has a header row describing the data. The data ‘The data set contains information about movies, actors, and roles, Data for movies and actors are stored as nodes and the roles are stored as relationships.The files you want to import data from are: * movies.csv * actors.csv * roles.csv Each movie in movies.csv has an ID,a title anda year , stored as properties in the node, All the nodes in movies.csv also have the label Movie. A node can have several labels, as you can see in movies. csv there are nodes that also have the label Sequel . The node labels are optional, they are very useful for grouping nodes into sets where all nodes that have a certain label belongs to the same set. movies.csy None Copy to Clipboard movieId:1D, title, year:int, :LABEL tt0193093, "The Matrix", 1999, Movie 10234215, "The Matrix Reloaded” ,2003, Movie;Sequel £t0242653, "The Matrix Revolutions", 2003, Movie; Sequel The actors data in actors.csv consist of an ID anda name, stored as properties in the node. The ID in this case a shorthand of the actors name. All the nodes in actors.csv have the label Actor actors.csv None Copy to Clipboard personId:1D, name, : LABEL keanu, "Keanu Reeves" ,Actor laurenee, "Laurence Fishburne" , Actor carrieanne, "Carrie-Anne Moss", Actor ‘The roles data in roles.csv have only one property, role. Roles are represented by relationship data that connects actor nodes with movie nodes, ‘There are three mandatory fields for relationship data: 1. :START_ID —ID refering to a node. 2, :END_ID —ID refering to a node. 3, :TYPE —The relationship type.In order to create a relationship between two nodes, the IDs defined in actors.csv and movies.csv are used for the :START_ID and :END_ID fields. You also need to provide a relationship type (in this case ACTED_IN) for the :TYPE field, roles.esv None Copy to Clipboard START_ID, role, :END_ID, :TYPE keanu, "Neo" , tt@133093, ACTED_IN keanu, "Neo", tt@234215, ACTED_IN keanu, "Neo" , tt@242652, ACTED_IN Laurence, "Morpheus" , tt@133@93, ACTED.IN Laurence, "Morpheus" ,tt@234215, ACTED.IN Laurence, "Morpheus" , tt242653, ACTED_IN carrieanne, "Trinity", tt@193893, ACTED_IN carrieanne, “Trinity” ,tt@234215, ACTED_IN carrieanne, "Trinity" ,tt@242653, ACTED_IN Importing the data * Paths to node data is defined with the --nodes option. ‘+ Paths to relationship data is defined with the --relationships option. The call to neo4j-admin import would look like this: shell None Copy to Clipboard bin/neo4j-admin import --database=neo4j --nodes=inport/movies.csv --nodes=import/actors.csv -- relationships=inport/roles.csv Query the data To query the data. Start Neo4j. ‘The default username and password is neo4j and neo4} shell None Copy to Clipboard bin/neoaj startTo query the imported data in the graph, try a simple Cypher query. shell None: Copy to Clipboard bin/cypher-shell --database=neo4j "MATCH (n) RETURN count(n) as nodes" Stop Neodj shell None Copy to Clipboard bin/neoaj stop CSV file delimiters You can customize the configuration options that the import tool uses (see Options) if your data does not fit the default format. The details of CSV file header format can be found at CSV header format, The data The following CSV files have: + --delimiter="; + --array-delimiter: + --quote= movies2.esv None Copy to Clipboard movield:10;title;year:int; :LABEL 00133099; The Matrix’ ;1999;Movie 1t8294215; "The Matrix Reloaded’ ;2003 ;Movie| Sequel, 1100242659; "The Matrix Revolutions’ ;2803 ;Movie| Sequel actors2.esv None Copy to ClipboardpersonId:1D;name; :LABEL Actor keanu; ‘Keanu Reeves’ Laurence; ' Laurence Fishburne’ ;Actor carrieanne; ‘Carrie-Anne Moss’ roles2.csv None 'START_ID; role; :END_ID; : TYPE Neo’ ;tt@133093 ;ACTED_IN Neo" ;tt@234215 ;ACTED_IN Neo" :tt@242653 :ACTED_IN Laurence Laurence Laurence; carrieanne; ‘Trinity’ ; "Worpheus’ ;tt8183093 ;ACTED_IN ‘"Worpheus’ ;tt8284215 ;ACTED_IN Morpheus" ;tt@242653 ;ACTED_IN 39099 ;ACTED_IN 1234215; ACTED_IN 1242653 ; ACTED_IN carrieanne; ‘Trinity’ ; carrieanne; ‘Trinity’ ; Importing the data The call to neo4j-admin import would look like this: shell None bin/neodj-admin inport --database=neo4j --delimiter: nodes=inport/movies2.csv ~-node: Using separate header files import /actors?.csv --relationship: Copy to Clipboard Copy to Clipboard array-delimiter="|" --quote: port/roles?.csv When dealing with very large CSV files, it is more convenient to have the header in a separate file, This makes it easier to edit the header as you avoit having to open a huge data file just to change it. The header file must be specified before the rest of the files in each file group. The import tool can also process single file compressed archives, for example: + --nodes=import/nodes.csv.gz + --relationships=import/relationships.zip The dataYou will use the same data set as in the previous example but with the headers in separate files. movies3-header.csv None movield:1D, title, year:int, :LABEL movies3.csv None £t0193093, "The Matrix", 1999, Movie ££0294215, "The Matrix Reloaded” ,2003, Movie;Sequel ££0242653, "The Matrix Revolutions", 2003, Movie; Sequel actors3-headercsv None personId:1D, name, : LABEL actors3.cev None keanu, "Keanu Reeves" Actor laurence, "Laurence Fishburne" , Actor carrieanne, "Carrie-Anne Moss" , Actor roles3-headercsv None START_ID, role, :END_1D, : TYPE roles3.sv None keanu, "Neo" ,tt@139093, ACTED_IN keanu, "Neo" , tt@234215, ACTED_IN keanu, "Neo" ,tt@242653, ACTED_IN Laurence, "Morpheus® , tt133093, ACTED_IN Laurence, "Morpheus" , tt234215, ACTED-IN Laurence, "Morpheus" , tt242653, ACTED.TN carrieanne, "Trinity", tt@133093, ACTED_IN carrieanne, "Trinity" ,tt@234215, ACTED_IN carrieanne, "Trinity" ,tt@242663, ACTED_IN Importing the data Copy to Clipboard Copy to Clipboard Copy to Clipboard Copy to Clipboard Copy to Clipboard Copy to ClipboardThe call to neo4j-admin import would look as follows: ‘The header line for a file group, whether itis the first line of a file in the group or a dedicated header file, must be the firstline in the file group. shell None Copy to Clipboard bin/neodj-admin inport --database=neodj --nodes=inport /movies3-header .csv, import /movies3.csv -- nodes=inport/actors3-header .csv, import /actors3.csv --relationships=inport/roles3- header.csv, import/roles3.csv Multiple input files In addition to using a separate header file you can also provide multiple nodes or relationships files Files within such an input group can be specified with multiple match strings, delimited by , , where each match string can be either the exact file name or a regular expression matching one or more files. Multiple matching files will be sorted according to their characters and their natural number sort order for file names containing numbers The data movies4-headeresy None Copy to Clipboard movieTd:1D, title, year movies4-parti.csv None Copy to Clipboard ££0133093, "The Matrix", 1999, Movie £t0234215, "The Matrix Reloaded” ,2003, Movie;Sequel movies4-part2.csv None Copy to Clipboard ££0242653, "The Matrix Revolutions”, 2003, Movie;Sequel actors4-headeresv None Copy to ClipboardpersonId:1D, name, :LABEL actors4-partL.csv None keanu, "Keanu Reeves Actor Laurence, "Laurence Fishburne", Actor actors4-part2.csv None carrieanne, "Carrie-Anne Moss”, Actor roles4-headeresv None :START_ID, role, :END_ID, :TYPE roles4-partLesv None keanu, "Neo" , tt@138093, ACTED_IN keanu, "Neo" ,tt@234215, ACTED_IN keanu, "Neo" ,tt@242653, ACTED_IN Laurence, "Morpheus" , tt8133@93, ACTED_IN Laurence, "Morpheus" , tt234215, ACTED.IN roles4-part2.csv None Laurence, "Morpheus" , tt242653, ACTED-IN carrieanne, “Trinity”, tt@733093, ACTED_IN carrieanne, "Trinity", tt@234215, ACTED_IN carrieanne, "Trinity", tt@242653, ACTED_IN Importing the data The call to neo4j-admin import would look like thi shell. None bin/neo4j-admin import --database-neo4j --node: part csv, import/movies4-part2.csv --node: Copy to Clipboard Copy to Clipboard Copy to Clipboard Copy to Clipboard Copy to Clipboard Copy to Clipboard import /movies4-header .csv, inport/movies4- import/actors4-header .csv, import /actors4-part! .csv, import /actors4-part2.csv --relationships=import/roles4-header .csv, import/roles4- part csv, import/roles4-part2.csv Regular expressions File names can be specified using regular expressions in order to simplify using the command line when there are many data source files. Each file name that matches the regular expression will be included. If using separate header files, for the import to work correctly, the header file must be the first in the file group. When using regular expressions to specify the input files, the list of files will be sorted according to the names of the files that match the expression. The matching is aware of numbers inside the file names and will sort them accordingly, without the need for padding with zeros. Example 1. Match order For example, let's assume that you have the following files * movies4-header.csv * movies4-data1.csv + movies4-data2.csv * movies4-data12.csv If you use the regular expression movies4 .*, the sorting will place the header file last and the import will fail. A better alternative would be to name the header file explicitly and use a regular expression that only matches the names of the data files. For example: --nodes “import/movies4-header .csv,movies-data.*" will accomplish this. Importing the data using regular expressions, the call to neo4j-admin import can be simplified to: shell None Copy to Clipboard bin/neod-admin import --database=neo4j --nodes="import/movies4-header .csv, import /movies4~ part.*" --nodes="inport/actors4-header csv, import/actors4-part.*" -— relationships="import /roles4-header .csv, import/roles4-part.*‘The use of regular expressions should not be confused with file globbing 7 ‘The expression .* means: "zero or more occurrences of any character except line break”. Therefore, the regular expression movies4..* will list all files starting with movies4 . Conversely, with file globbing, 1s movies4.* will list all files starting with movies4 Another important difference to pay attention to is the sorting order. The result of a regular expression matching will place the file movies4-part2.csv before the file movies4-part12.csv.Ifdoing 1s movies4-parts in a directory containing the above listed files, the file movies4-part12.csv will be listed before the file movies4-part2.csv Using the same label for every node If you want to use the sarne node label(s) for every node in your nodes file you can do this by specifying the appropriate value as an option to neo4j-admin import . There is then no need to specify the : LABEL column in the header file and each row (node) will apply the specified labels from the command line option. Example 2. Specify node labels option --nodes=Label0ne : Label Two= import /example-header .csv, import/example-datal .csv Itis possible to apply both the label provided in the file and the one provided on the command line to the node. The data In this example you want to have the label Movie on every node specified in moviesSa.csv, and you put the labels Movie and Sequel on the nodes specified in sequelsSa.csv. moviesSa.csv None Copy te Clipboard movieTd:1D, title, year:int tt@133093, "The Matrix", 1999sequelsSa.csv None movieTd:1D, title, year:int 10234215, "The Matrix Reloaded” 2003 100242653, "The Matrix Revolutions" 2083 actorsSa.csv None personId:1D, name keanu, "Keanu Reeves’ laurence, "Laurence Fishburne" carrieanne, "Carrie-Anne Moss” rolesSacsv None START_ID, role, :END_ID, :TYPE keanu, "Neo" , tt2133093, ACTED_IN keanu, "Neo" , tt@234215, ACTED_IN keanu, "Neo" , tt@242653, ACTED_IN Laurence, "Worpheus® , tt@133093, ACTED_IN Laurence, "Morpheus" , tt234215, ACTED-IN Laurence, “Morpheus” , tt242653, ACTED-IN carrieanne, "Trinity", tt@183093, ACTED_IN ‘carrieanne, "Trinity", tt2234215, ACTED_IN carrieanne, "Trinity" , tt2242653, ACTED_IN Importing the data The call to neo4j-admin import would look like this: shell None bin/neo4j-admin inport --database=neo4j ~-nodes-Novie-import/moviessa.csv ~ \ctor=import/actorssa.csv ~ nodes=Movie :Sequel=inport/sequels5a.csv --node: relationships=import/rolesSa.csv Copy to Clipboard Copy to Clipboard Copy to Clipboard Copy to Clipboard Using the same relationship type for every relationship If you want to use the same relationship type for every relationship in your rel done by specifying the appropriate value as an option to neo4j-admin import. ionships file this can beExample 3. Specify relationship type option relationship: YPE=import /example-header .csv, import/example-datal.csv If you provide a relationship type both on the command line and in the relationships file, the one in the file will be applied, The data In this example you want the relationship type ACTED_IN to be applied on every relationship specified in rolesSb.csv moviesSb csv None movield:1D, title, year:int, :LABEL 00133893, "The Matrix”, 1999, Movie 100234215, "The Matrix Reloaded”, 2003, Movie;Sequel +1t0242653, "The Matrix Revolutions” ,2803, Mavie;Sequel actorsSbesv None personTd:1D, name, :LABEL keanu, "Keanu Reeves" , Actor Laurence, "Laurence Fishburne", Actor carrieanne, "Carrie-Anne Moss”, Actor rolesSbesv None START_ID, role, :END_ID keanu, "Neo", tt0133093 keanu, "Neo" 100234215 keanu, "Neo" , tt0242653 Laurence, "Morpheus" , ¢t133093 Laurence, "Morpheus" ,tt@234215 Laurence, "Morpheus" , tt@242653 carrieanne, "Trinity" ,tt@793093 carrieanne, "Trinity" ,tte234215 carrieanne, “Trinity” ,tt@242653 Copy to Clipboard Copy to Clipboard Copy to ClipboardImporting the data The call to neo4j-admin import would look like this: shell None Copy to Clipboord bin/neodj-admin import --database=neo4j --nodes=inport/moviessb.csv ~-nodes=import/actorssb.csv import /rolessb.csv ~-relationships=ACTE Properties Nodes and relationships can have properties. The property type are specified in the CSV header row, see CSV header format. The data The following example creates a small graph containing one actor and one movie connected by one relationship. There is a roles property on the relationship which contains an array of the characters played by the actor in a movie: movies.csv None Copy to Clipboard movieTd:1D, title, year:int, :LABEL +tt@999892, "Joe Versus the Volcano", 1990, Movie actors6.cev None Copy to Clipboard personId:1D,name, : LABEL meg, "Meg Ryan", Actor roles6.csv None Copy to Clipboard SSTART_ID, roles:string[], :END_ID, :TVPE meg, "DeDe; Angelica Graynamore;Patricia Graynamore”, tt@099892, ACTED_INImporting the data The call to neo4j-admin import would look like this: shell None Copy to Clipboard bin/neosj-adnin import ~-database-neotj --nodes-inport/novies6 csv --nodes=inport/actors6.csv ~ ~relationships=inport/roles6.csv ID space ‘The import tool makes the assumption that identifiers are unique across node files. This may not be the case for data sets which use sequential, auto incremented or otherwise colliding identifiers. Those data sets can define ID spaces where identifiers are unique within their respective ID space. In cases where the node ID is only unique within files, using ID spaces is a way to ensure uniqueness across all nodes files, See Using ID spaces, Each node processed by neo4j-admin import must provide an ID if it is to be connected in any relationships. The node ID is used to find the start node and end node when creating a relationship. Example 4. ID space To define a ID space Movie-ID for movieId:ID the syntax will be movield: The data For example, if movies and people both use sequential identifiers, then you would define Movie and Actor ID spaces. movies7.esv None Copy to Clipboard movieId:1D(Movie-ID), title, year:int, :LABEL 1, "The Matrix", 1999, Movie 2,"The Matrix Reloaded" 2803, Movie; Sequel 3, "The Matrix Revolutions”, 2003, Movie;Sequelactors7.csv None Copy to Clipboard personId:1D(Actor-ID), name, :LABEL, 1, "Keanu Reeves", Actor 2,"Laurence Fishburne" , Actor 3,"Carrie-Anne Moss" Actor You also need to reference the appropriate ID space in your relationships file so it knows which nodes to connect together. roles7.csv None Copy to Clipboard START_ID(Actor-ID), role, :=ND_ID(Movie-ID) Neo Neo Neo 1 1 1 2,"Morpheus”,1 2,"Morpheus” 2 2, "Morpheus", 3 a, "Trinity",1 3, "Trinity",2 3,"Trinity",3 Importing the data The call to neo4j-admin import would look like this: shell None Copy to Clipboard bin/neodj-admin inport --database=neo4j --nodes=inport/movies? .csv --nodes=import/actors7.csv -relationships=ACTED_IN=import/roles7.csv Skip relationships referring to missing nodes The import tool has no tolerance for bad entities (relationships or nodes) and will fail the import on the first bad entity. You can specify explicitly that you want it to ignore rows that contain bad entities. There are two different types of bad input: 1. Bad relationships.2. Bad nodes. Relationships that refer to missing node IDs, either for :START_ID or :END_ID are considered bad relationships. Whether or not such relationships are skipped is controlled with skip-bad- relationships flag, which can have the values true or false or no value, which means true. The default is false, which means that any bad relationship is considered an error and will fail the import. For more information, see the --skip-bad-relationships option. The data In the following example there is a missing emil node referenced in the roles file. moviesBa.csv None Copy to Clipboard movield:1D, title, year:int, :LABEL 1t@199093, "The Matrix", 1999, Movie 110294215, "The Matrix Reloaded” , 2203, Movie;Sequel +1t0242653, "The Matrix Revolutions”, 2003, Movie;Sequel actorsBa.csv None Copy to Clipboard personId:1D, name, : LABEL keanu, "Keanu Reeves" , Actor laurenee, "Laurence Fishburne" , Actor carrieanne, "Carrie-Anne Moss”, Actor rolesBacsv None Copy to Clipboard |ART-ID, role, :END_ID, :TYPE keanu, "Neo", tt@138093, ACTED_IN keanu, "Neo" , tt@234215, ACTED_IN keanu, "Neo" , tt@242653, ACTED_IN Laurence, "Morpheus" ,tt133@93, ACTED_IN Laurence, "Worpheus" , tt234215, ACTED.IN Laurence, "Worpheus" ,tt242653, ACTED.IN carrieanne, “Trinity” ,tt@133093, ACTED_IN carrieanne, “Trinity” ,tt@234215, ACTED_IN carrieanne, “Trinity”, tt@242653, ACTED_IN emil, "Emil", tt@139093, ACTED_IN Importing the dataThe call to neo4j-admin import would look like this: shell None Copy to Clipboard bin/neodj-admin import --database=neodj --nodes=inport/moviesBa.csv --nodes=import/actors8a.csv --relationships=inport/roles#a.csv Since there was a bad relationship in the input data, the import process will fail completely. Let's see what happens if you append the --skip-bad-relationships flag: shell None Copy to Clipboard bin/neo4j-admin import --database=neo4j nodes=inport/actors8a.csv --relations! skip-bad-relationships --nodes=inport/movies8a.csv ~~ mport/rolesfa.csv The data files are successfully imported and the bad relationship is ignored. An entry is written to the import. report file. ignore bad relationships None Copy te Clipboard InputRelationship: source: rolesBa.csv:11 properties: [role, Emil] startNode: emil (global id space) endNode: tt@133093 (global id space) type: ACTED_IN referring to missing node enil Skip nodes with same ID Nodes that specify :ID which has already been specified within the ID space are considered bad nodes. Whether or not such nodes are skipped is controlled with --skip-duplicate-nodes flag which can have the values true or false orno value, which means true. The default is false, which means that any duplicate node is considered an error and will fail the import. For more information, see the --skip-duplicate-nodes option. The dataIn the following example there is a node ID, Laurence, that is specified twice within the same ID space. actorsBb.csv None Copy to Clipboard personId:1D,name, :LABEL keanu, "Keanu Reeves" , Actor laurence, "Laurence Fishburne" , Actor carrieanne, "Carrie-Anne Moss” Actor laurence, "Laurence Harvey", Actor Importing the data ‘The call to neo4j-admin import would look like this: shell None Copy to Clipboard bin/neo4j-admin inport --database=neo4j ~-node: inport/actors8b.csv Since there was a bad node in the input data, the import process will fail completely. Let's see what happens if you append the skip-duplicate-nodes flag: shell None Copy to Clipboard bin/neo4)-admin import --database=neo4j --skip-duplicate-nodes --nodes=import/actors8b.csv The data files are successfully imported and the bad node is ignored. An entry is written to the import. report file ignore bad nodes None Copy to Clipboard ID ‘laurence’ is defined more than once in global ID space, at least at actorssb.csv:3 and actors8b.csv:5 Next and use Fabri ‘ and restore a database in Causal Set ClusterWas this page helpful? Contents Import a small data set CSV file detimiters Using separate header fles Muttipte input les Using the same label for every node Using the same relationship type for every relationship Properties ID space Skip relationships referring to missing nodes Skip nodes with same ID 16-17 NOVEMBER 2022 NODES 2022 Join us fora free, two-day virtual conference of technical presentations by developers and data scientists, solving problems with graphs, Save your space © 2022 Neodj Inc. Terms | Privacy | Sitemap Neodj®, Neo Technotogy®, Cypher®, Neodi® Bloom™ and Neo4j® Aura™ are registered trademarks of Neodj, Inc. All other marks are owned by their respective companies. Contact Us» US: 1-855-636-4532 ‘Sweden +46 171 480 113 Uk: +44 20 3868 3223 France: +33 (0) 8 05 08 03 44# Sandbox & Neo} Community Site B Ne04j Developer Blog & Neo4j Videos ® GrephAcademy B Neodj Labs Social Twitter @ Moctups OQ Github, Stack Overtiow Want to Speak? Get $ back