/neo4j-elasticsearch

Neo4j ElasticSearch Integration

Primary LanguageJava

Neo4j Elastic{Search} Integration

Integrates Neo4j change-feed with an ElasticSearch cluster.

Approach

This Neo4j Kernel Extension updates an ElasticSearch instance or cluster with changes in the graph.

A transaction event listener checks changed Nodes against a given label, renders the whole node as json document and indexes all changes in bulk with ES.

Installation

  • Clone this repository and run mvn clean install -DskipTests

  • Copy the jar files in target/dependency to your Neo4j instance’s plugins folder

  • Also copy target/neo4j-elasticsearch-3.0-SNAPSHOT.jar to the plugins folder

  • Modify conf/neo4j.conf accordingly (see the Example section)

  • Restart Neo4j

Example

Suppose that we keep nodes in our Neo4j instance labeled Person and Place, and that we want to index the values of the first_name and last_name properties of the former and name of the latter in two separate ElasticSearch indices named people and places. For that, we would add the following directives to conf/neo4j.conf:

elasticsearch.host_name=http://localhost:9200
elasticsearch.index_spec=people:Person(first_name,last_name), places:Place(name)

With that in place, Neo4j will now track changes to nodes labeled Person or Place and keep our ES instance running on localhost:9200 in sync.

Suppose that we also want to index all nodes (independent of label or scheme) in a separate index called all_nodes, we would add the following directive below the previous:

elasticsearch.index_all=all_nodes

The first two directives (host_name and index_spec) are required. The latter (index_all) is optional.

To perform an initial import, you can force a commit by executing a Cypher query like:

MATCH n:Person
SET n.first_name = n.first_name, n.last_name = n.last_name;

MATCH n:Place
SET n.name = n.name;

ID / Labels fields

By default, the indexes created will contain fields for the Neo4j ID and Labels, named id and labels. These will be auto-created as searchable fields, but, if you’d prefer they not be included, simply add one or both of these lines to your conf/neo4j.conf file.

elasticsearch.include_id_field=false
elasticsearch.include_labels_field=false

Developing

To run the tests, run mvn test. Make sure that an elastic{search} server is running on localhost:9200.

Todo

  • Support indexing of relationships