/infranodus

A Node.Js / Neo4J tool that translates words and relations into network graphs and shows you how it all connects.

Primary LanguageJavaScript

InfraNodus is a text-to-network visualization tool, based on Neo4J Graph Database, Node.Js, Sigma.Js, and Graphology.

www.infranodus.com

InfraNodus is a web app that can represent plain text as a graph. Every word is a node, every co-occurrence is a connection. We build a graph and apply network science algorithms to get insights about the connections in this graph. So you can encode any text or anything you want into a graph using #hashtags and @mentions). InfraNodus will visualize it, so you can analyze it.

Try InfraNodus Online

www.infranodus.com - includes the Pro features, such as GPT-3 AI idea generation tool

Watch a Video Demo - a 3-minute introduction

Features

  • Visualize any text as a network graph
  • Advanced graph analysis algorithms to study text
  • Topic modeling
  • Most influential terms
  • Advanced text statistics
  • Graph / text comparison
  • Works with English, German, French, Russian, Spanish, Portugese (+ Japanese and Chinese in the cloud version)
  • Use InfraNodus to interface with your Neo4J database and to try some basic graph constructs using an easy input procedure via natural language, voice-to-text, or #hashtags

Works on mobile and desktop. Voice-to-text (EN, RU, FR, DE) is available.

The Science and the Method

InfraNodus is a non-commercial venture fiction project. Its main objective is to enhance perception and imagination, although it can also solve concrete problems and offer you an efficient way to retain, process, and make sense of information. It can:

  • Help researchers make sense of textual data
  • Help you retain and remember information
  • Help you connect disjointed bits and pieces of data
  • Visualize patterns in complex relations
  • Be used for text network analysis
  • Be used to quickly create graphs using text / voice input
  • Perform comparative visual analysis of texts
  • Identify the structure of discourse (dispersed / biased)
  • Reveal the gaps in a network (and, thus, the potential for new ideas/connections)
  • Provide network metrics, identify the main communities and top nodes

The method is mainly based on the approach outlined in Paranyushkin (2011) Identifying the Pathways of Meaning Circulation Using Text Network Analysis.

A more precise description of InfraNodus' algorithm is presented in Paranyushkin (2019) InfraNodus — Generating Insight Using Text Network Analysis (published in the Proceedings for the WWW'19 The Web Conference). If you use the tool, you can cite this paper.

A more easy-to-read article on the tool is published in Towards Data Science.

Introduction

The basic ideas behind InfraNodus are:

  • Graph DB model suitable for large graphs and collaboration.
  • Represent text as a network of interconnected nodes.
  • Works on desktop and mobile.
  • Make it easy to add nodes into a graph through a simple web interface.
  • Work with what people already know and use well: #hashtags and @mentions, not nodes and edges.
  • Use natural language to enter nodes in a graph (English and Russian morphology is supported)
  • Move away from binary edges (e.g. A < - > B) to hyperedges in one statement (e.g. A and B and C or D)
  • API and JSON export of all data;
  • Twitter, Evernote, Gmail, Google, YouTube subtitles, RSS import (TODO: gexf, xml, csv upload)
  • Enable people to use ше collaboratively, both locally and over the internet
  • Your suggestions? (open an issue)

Technology

InfraNodus is built on

  • Sigma.Js for graph visualization;
  • Neo4J ver 3.x for graph database storage
  • Graphology for graph metrics calculations (Modularity)
  • JSNetworksX for graph metrics calculations (BC)
  • Cytoscape for some graph metrics
  • jQuery for the front-end
  • Node.Js for the back-end
  • Express Node.Js library;
  • Node-Neo4J layer;
  • Textexture algorithm for text network visualization;
  • Open-source code from the people on StackOverflow and Neo4J community;

Installation Guide

If you just want to try out the tool, the best way to do that is to use the Cloud version of InfraNodus. We only charge €9 towards development and server costs, and it has more capabilities than the open-source version. The cloud version is also connected to multiple AI APIs, such as GPT-3 and BERT sentiment analysis, and can process bigger files faster.

Note, that due to the limited capacity we cannot provide support for installing the open-source version. However, you can post an Issue to this repository and maybe somebody from the community will be able to respond.

Run on the cloud

To run InfraNodus on the cloud, go to www.infranodus.com.

Run directly on your machine

To use this software you should install Neo4J 3.3.9 on your local computer (up to 3.5 is supported). To install Neo4J on a Mac use homebrew (see Neo4J instructions here) or you can also set up a cloud Neo4J instance on AWS or their own cloud service. For settings, check out How to Set Up Neo4J for InfraNodus. You may also find other wiki pages interesting, especially Neo4J installation guide – the section on setting up indexes and installing APOC plugin.

Git clone this project into a folder, then you will also need to have npm Node.Js package manager installed on your computer. After you install InfraNodus, run

npm install

in the main folder the project to install all the dependencies into node_modules folder.

Check out the config.json.sample file and edit it to add your own Evernote and Twitter API credentials. Then rename it to config.json. Create statsabove.ejs, statsbelow.ejs and statsheader.ejs files in the /views folder.

Run the application with node app.js

You can access the app from http://localhost:3000

To create an account on your local machine, visit http://localhost:3000/signup?invitation=secretcode - replace secretcode with the value of secrets.invitation in your config.json.

When you sign up, please, consider creating an account on www.infranodus.com to support the development of this tool. Subscribers get fast-track support and help on the technical issues of the installation.

Run inside a Vagrant Virtual Machine

  1. git clone the project
  2. cd infranodus to switch into the git project directory
  3. cp config.json.sample config.json and modidy the file as necessary - for example, add Evernote and/or Twitter credentials. Note: if you change the Neo4j database password here, you'll also need to change it in your local copy of /vagrant/setup-neo4j.sh.
  4. vagrant up in the project folder. This may take a while the first time, as it installs an Ubuntu VM and all dependencies.
  5. vagrant ssh to logon to the Virtual Machine
  6. cd /vagrant to get into the project directory on the Virtual machine
  7. node app.js to run the application
  8. You can access the app from http://192.168.66.101:3000 - if you want a different IP address, change the ip setting under config.vm.network in the vagrantfile.
  9. To create an account on your local machine, visit http://192.168.66.101:3000/signup?invitation=secretcode - replace secretcode with the value of secrets.invitation in your config.json. In this case, please, consider becoming a patron by signing up on www.infranodus.com to support the development of this open source tool. Patrons get fast-track support and help on the technical issues of the installation.

Data Model

The general rationale for the data model used in InfraNodus is outlined in

The main properties of this model are

  • There are 5 labels (types) for nodes: Concepts, Statements, Contexts, Users, and Narratives
  • Every expression of a user into the system is a Statement
  • Example: "#antibiotics can fight #bacteria which are #dangerous for #health" is the user's input. The system creates 4 :Concept nodes from the hashtags inside the :Statement, which is linked to the :Context (by default, "@private") and all those are attached to the :User.
  • Types of connections: :TO (between Concepts), :AT (Concepts to Context), :OF (Concepts to Statement), :IN (Statement to Context), :BY (to User)
  • Narrative is implemented through creating a :Narrative node, which is linked to from Statements and Concepts by :INTO type of connection (think of :Narrative as another :Context)
  • Narrative elements are linked to each other via :THRU type of connection.

This data model enables to create custom views for any kind of data and introduce the idea of narrative into holistic graph views.

This data model is derived from the Cognitive Graph DB Model draft created by Nodus Labs.

The current data model description utilized in InfraNodus app is available in https://github.com/noduslabs/graphdbmodel repository.

Mobile Interface

InfraNodus can also be used on a mobile.

Python Port

Together with Mattias Östmar (who did most of the work) we created a port of InfraNodus in Python, focusing on the measure of network diversity structure that it provides.

You can get it in DiscourseDiversity repo on GitLab. We're still working on it, but if you like Jupyter notebooks and network analysis, there's some good stuff to find in there!

Special Thanks

InfraNodus could not be built without the

  • Help from the people on StackOverflow and Neo4J community;

Also... :)

GPL License

This open source, free software is available under the GNU Affero General Public License version 3 (AGPLv3) license. You can make modifications to this code and binaries based on it, but only on the condition that you provide access to those modifications under the same license (including remotely through a computer network). It is provided as is, with no guarantees and no liabilities. You can re-use it as long as you keep this notice inside the code.

You are very welcome to join the project!

Created by Dmitry Paranyushkin of Nodus Labs, conceptualized via Polysingularity, inspired from ThisIsLike.Com, and KnowNodes (now Rhizi) co-developed at Center for Interdisciplinary Research.

This project would not be possible without the help from StackOverflow community. Thank you so much!

Copyright (C) Dmitry Paranyushkin | Nodus Labs and hopefully you also! (http://www.noduslabs.com) | info AT noduslabs DOT com

In some parts the code from the book "Node.js in Action" is used (c) 2014 Manning Publications Co. Any source code files provided as a supplement to the book are freely available to the public for download. Reuse of the code is permitted, in whole or in part, including the creation of derivative works, provided that you acknowledge that you are using it and identify the source: title, publisher and year.

Some parts of this code may come under a different license if specified within.