/sonar

A p2p content database and search engine

Primary LanguageJavaScriptGNU General Public License v3.0GPL-3.0

sonar

A p2p database and search engine running on top of the dat stack.

This is the main repository of Sonar. The usual setup is that you run sonar-server on localhost and then interact with Sonar through the client, the UI running on http://localhost:9191 or the CLI. The CLI can be invoked with ./sonar from the root of this repository, and is also used to start the server.

The Docs website has more documentation (WIP).

Installation

npm install -g @arso-project/sonar-server
sonar help
sonar start

Development

Note: At the moment yarn 1 is recommended, please install it according to the instructions.

# clone the sonar repository
git clone https://github.com/arso-project/sonar.git
cd sonar
# install dependencies of all workspaces
yarn
# (re)build the user interface and docs
yarn run rebuild

Instead of yarn, lerna works too:

npm install -g lerna
lerna bootstrap

Yarn is recommended because it's much faster.

You can start sonar with ./sonar from the repository root.

If the start fails with errors related to sonar-tantivy, try to redownload or rebuild sonar-tantivy (the search engine included in sonar):

yarn run build:sonar-tantivy

# start the sonar server
./sonar start

# start the sonar server in dev mode
./sonar start --dev

Then, you can:

  • open the web UI on http://localhost:9191.
  • use the CLI: 
    ./sonar collection create default
./sonar db get
# etc.
# the cli has a built-in help that should list the available commands

Contributing

Sonar is a young open source project and all kinds of contributions are welcome. We're in the process of writing up more documentation and overviews of how things work and come together (in a book).

If in doubt, talk to us! For example on IRC in #dat on freenode (or in the browser through gitter.

More on this project on arso.xyz.

Repo layout

The repo is structured as a monorepo of different packages (that are interdependent at several points).

  • sonar-core it the core module. It manages collections, which are our notion of "a group of feeds*. Each collection has a kappa-record-db that's plugged into a search index through sonar-tantivy. Each collection has also a list of associated hyperdrives.

  • sonar-server provides a REST style HTTP api that's used both by the CLI and the UI to access and manage the data in sonar-core.

  • sonar-client is a Javascript client library. It's used by both the CLI and the UI. It speaks to sonar-server over HTTP.

  • sonar-ui is a single-page application to browse data in Sonar.

  • sonar-cli is a command-line application. It can manage collections, put and get into the database, upload and download files, and make search queries.