SouslesensVocables

SousLesensVocables is a set of tools developed to manage Thesaurus and Ontologies resources through SKOS , OWL and RDF standards and graph visualisation approaches.

It has functionalities to :

read : visualize, navigate and export SKOS or OWL resources
edit : create , modify, aggregate OWL resources

A key feature of SLSV is graph visualization and interaction performed using excellent visjs/vis-network open source solution

Annotate tool allows annotate textual corpus with several registered lexical resources and identify missing terms

Deploy a production instance

Prerequisites

In production, souslesensVocable is deployed using docker and docker-compose. Install docker following this link.

Then, install docker-compose with your package manager

sudo apt install docker-compose

Configure the deployment

Clone the souslesensVocables repository

git clone https://github.com/souslesens/souslesensVocables.git
cd souslesensVocables

Checkout the latest release

git checkout x.x.x  # replace with release number (e.g. 1.15.0)

Create a .env file from the template

cp env.template .env

Edit the .env file:

The DATA_ROOT_DIR is defined by default to /tmp, change it for data persistance.

variable	description
`TAG`	souslesensVocable release. Same as you checkout at the previous step
`VOCABLES_LISTEN_PORT`	Port of souslesensVocable that will be exposed outside
`VIRTUOSO_LISTEN_PORT`	Port of virtuoso that will be exposed outside
`JOWL_LISTEN_PORT`	Port of jowl that will be exposed outside
`JOWL_PATH`	Path of the jowl repository
`DATA_ROOT_DIR`	Where the data will be written
`USER_PASSWORD`	Password of the `admin` user automatically created at first start
`SA_PASSWORD`	Password of the sql server
`DBA_PASSWORD`	Password of the virtuoso server
`FORMAL_ONTOLOGY_SOURCE_LABEL`

Advanced configuration

SouslesensVocables

All entries from SouslesensVocables mainConfig.json can be overwritten with env variables. For example VOCABLES__SQLserver__user=toto will be converted into {"SQLserver": {"user": "toto"}}.

Virtuoso

Refer to the docker-virtuoso documentation.

Elasticsearch

Refer to the Elasticsearch documentation.

SQLServer

Refer to the SQLServer documentation.

Build docker image

We do not provide a docker image for SouslesensVocables. You have to build it yourself:

docker-compose build vocables

Create the data directories

Elasticsearch and SqlServer needs directories with specific right. Use the following commands to create directories usable by ElasticSearch and SqlServer.

bash scripts/init_directories.sh

Launch the docker stack

docker-compose up -d

SouslesensVocables will be available at localhost:3010.

Data loading

TODO:

Upgrading souslesens

First, fetch the latest changes

git fetch

Then, checkout the desired release

git checkout x.x.x  # replace with release number (e.g. 1.16.0)

Change the TAG number in the .env file

Rebuild the image

docker-compose build vocables

Finally, restart the docker stack

docker-compose up -d

Install locally (development instance)

Prerequisites

In development, souslesensVocable needs a Virtuoso instance, an ElasticSearch instance and a Sql and a Spacy server. We provide a docker-compose.test.yaml to deploy this dependencies.

SouslesensVocables is deployed locally using node and npm.

Docker

Install docker following this link.

Then, install docker-compose with your package manager

sudo apt install docker-compose

Node

Install nodejs from the nodesource package repository (detailed instruction here).

# Add nodesource repository
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | gpg --dearmor > /usr/share/keyrings/nodesource.gpg
echo "deb [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_16.x buster main" > /etc/apt/sources.list.d/nodesource.list
echo "deb-src [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_16.x buster main" >> /etc/apt/sources.list.d/nodesource.list
apt update
# Install node
apt install -y nodejs
# Check versions
node --version
npm --version

Configure

Run the config script to create a default configuration:

node scripts/init_configs.js

Then, edit the config/*.json to your needs or run the following script to copy the values from the docker-compose configuration into the config/*.json files.

node scripts/config_from_docker_compose.js

Start docker dependencies

Start the servers with docker-compose

docker-compose -f docker-compose.dev.yaml up -d

Load some data into virtuoso

bash tests/load_data.sh dev

Create a user account in MariaDB (for database authentication)

bash scripts/create_user_in_db.sh dev <login> <password>

Install souslesens

souslesensVocable is composed of a backend in node/express and a frontend in pure javascript and typescript/react.

# Install the server
npm ci
# Install the front
cd mainapp
npm ci
# Build Webpack files
npm run build
# Go back to root
cd ..

Run souslesens server

The following command will build and watch the react app and run and watch the node app.

npm run devserver

Run migration scripts

Certain version need migration of data. Run the script for the following version:

Release 1.27

node scripts/sources_access_control_migration_treeview.js -c config -w

Release 1.30

node scripts/controller-migration.js -c config -w

Release 1.33

node scripts/remove_admin_profile_migration.js -c config -w

Release 1.37

node scripts/migration_1.37_sources.js -c config -w

SouslesensVocables will be available at localhost:3010.

Contribute to souslesensVocable

To contribute to souslesensVocable, fork the repo and submit Pull Requests.

continuous integration

Style

We use Prettier for Javascript code formatting. Please run npm run format before each commit to format your code.

Tests

We provide a docker-compose.test.yaml and a tests/load_data.sh script to build a testing environment.

docker-compose -f docker-compose.test.yaml -d
bash tests/load_data.sh

Release

To create a new release:

npm run release:minor  # or patch or major

Then, edit the generated CHANGELOG.md if necessary and tag the release with

npm run release:tag

And push commits and tag to GitHub

git push --tags

GitHub releases and docker images are created on tags with GitHub Actions.

Plugins system

In root create a plugins folder

mkdir plugins

Each directory is named after the plugin we want to add.

plugins/
└── MyPlugin
    └── public
        └── MyPlugin.js

The plugin's directory must contain a public directory with the source code within it.

MyPlugin.js must export a single IIFE function.

const Toto = (function () {
    return { name: "toto" };
})();
export default Toto;

Once it done, don't forget to add the plugin's name to mainConfig.tools_available. If you still don't see the plugin in the jsTree, check that your user's profile allows to see this plugin.

xgaia/souslesensVocables

SouslesensVocables

Deploy a production instance

Prerequisites

Configure the deployment

Advanced configuration

SouslesensVocables

Virtuoso

Elasticsearch

SQLServer

Build docker image

Create the data directories

Launch the docker stack

Data loading

Upgrading souslesens

Install locally (development instance)

Prerequisites

Docker

Node

Configure

Start docker dependencies

Install souslesens

Run souslesens server

Run migration scripts

Release 1.27

Release 1.30

Release 1.33

Release 1.37

Contribute to souslesensVocable

continuous integration

Style

Tests

Release

Plugins system