/permanent-seeder

WIP

Primary LanguageJavaScriptMIT LicenseMIT

permanent-seeder

Build Status JavaScript Style Guide standard-readme compliant

Made by GEUT

Permanent Seeder logo, a  beautiful seed surrounded with a round and exquisite black border

Install

npm i -g @geut/permanent-seeder

Alternatively you can git clone this repo and build it from source:

# install deps
npm i
npm run bootstrap

# build the dashboard
cd packages/dashboard
cp .env.example .env
npm run build

# start the seeder
cd ../cli
permanent-seeder stop
permanent-seeder start

Usage

First, create a base config file, you can tweak it later ;-)

$ permanent-seeder config:init

Then, start the Permanent Seeder daemon

$ permanent-seeder start

🚀

Commands

$ permanent-seeder [COMMAND] [--OPTIONS]

Config

$ permanent-seeder config:[init|get]
  • init: creates the base config file for the Permanent Seeder. This is a .toml file that will live in ~/permanent-seeder/settings.toml.
  • get: returns the settings from the CLI. Useful when you are changing values and want to be sure they are pick up by the Permanent Seeder.

Default settings:

# Permanent seeder path (will be completed on config:init)
path = 'permanent-seeder'

# Enable stats recording
save_stats = true

# keys.endpoints = array of configs per endpoint
[[keys.endpoints]]

  # Where to fetch keys
  url = 'http://localhost:3000'

  # Frequency between fetchs (in minutes)
  frequency = 5

  # Hook to parse response
  hook = 'endpoint-hook.js'

## To add another endpoint, uncomment and complete next lines:
# [[keys.endpoints]]
#   url =
#   frequency =
#   hook

Start

$ permanent-seeder start

Bootstrap a Permanent Seeder instance that will keep up running in the background. If you change settings, you will need to call start command again.

Status

$ permanent-seeder status

It will return instance status. If it is running and some basic stats.

Stop

$ permanent-seeder stop

Stops the current instance.

Dashboard

$ permanent-seeder dashboard

Opens the dashboard app in a browser. If you want to manually access the dashboard, it can be found in: localhost:3001

⚠️ Note: The dashboard app runs in http://localhost:3001. If you deploy the Permanent Seeder on a server and wants to access the dashboard from the outside, you would need to setup a reverse proxy.

Sample nginx reverse proxy config
upstream dashboard-nodejs {
        server  127.0.0.1:3001;
}

server {
	listen 80 default_server;
	listen [::]:80 default_server;

	server_name _;

        location / {
                proxy_pass              http://dashboard-nodejs;
                proxy_next_upstream     error timeout invalid_header http_500 http_502 http_503 http_504;
                proxy_redirect          off;
                proxy_buffering         off;

                proxy_set_header        Host                    $host;
                proxy_set_header        X-Real-IP               $remote_addr;
                proxy_set_header        X-Forwarded-For         $proxy_add_x_forwarded_for;
                proxy_set_header        X-Forwarded-Proto       $scheme;
        }

        location /socket.io/ {
                proxy_pass              http://dashboard-nodejs;
                proxy_redirect          off;

                proxy_http_version      1.1;

                proxy_set_header        Upgrade                 $http_upgrade;
                proxy_set_header        Connection              "upgrade";
                proxy_set_header        Host                    $host;
                proxy_set_header        X-Real-IP               $remote_addr;
                proxy_set_header        X-Forwarded-For         $proxy_add_x_forwarded_for;
        }
}

⚠️ Note2: We use the .env.example file as a placeholder for some environment variables that we use internally, like PUBLIC_URL. For development there are a few others variables that you can tweak, like the SOCKET_URL and API_URL. Checkout the full .env.example:

PUBLIC_URL=/

## Extend eslint config
EXTEND_ESLINT=true

## Optional ENVs only for development
#REACT_APP_SOCKET_URL=
#REACT_APP_API_URL=

Key Management 🔑

$ permanent-seeder key:[add|remove|remove-all]
  • add: Insert a new key Permanent Seeder db, it will start downloading and seeding ASAP.
  • remove: Removes a single key from the seeder db and also stops seeding it (e.g.: no more announcing to other peers)
  • remove-all: Removes and unnanounce all the keys in the db.

Logs

$ permanent-seeder logs --[live|all|error]
  • live: like doing a tail -f of the logs.
  • all: Show all the logs stored.
  • error: Display only error logs.

repl

$ permanent-seeder repl

Useful to inspect the Permanent Seeder under the hood. 🔬

Keys Endpoint

The Permanent Seeder can fetch keys from an external endpoint, i.e: perform a GET request against a particular endpoint. This can be useful if you maintain a service that stores hyperdrive's keys. If that is the case, then the Permament Seeder can fetch those keys regularly. You can think of this like a cron job.

Whilst, we internally expect an Array<{key}>, you can customize and parse the fetch response the way you need it.

To do this, you will need to modify $HOME/permanent-seeder/endpoint-hook.js.

That hook will be called after fetch the response.

You can also tweak the fetch frequency (defined in minutes) and the endpoint url. These options can be found in the settings.toml file:

[[keys.endpoints]]
url = "http://localhost:3000"
frequency = 5
hook = "$HOME/permanent-seeder/endpoint-hook.js"

Design

The Permanent Seeder is a CLI tool that can starts a daemon which will seed hyperdrive's keys that you pass into it. Using the CLI you can add, remove keys, check the status and inspect logs.

It also contains a dashboard that you can use to have a visual reference of what is going on with your hyperdrives.

As you can see the project does a couple of things. To do this we decided to use a microservices based approach. We choose to use moleculer as the structural framework behind the Permanent Seeder. This enables multiples processes to communicate each other and at the same time each of these will have a single responsibility/scope. This also give us some room to scale things up if needed. :sunglasses:

Issues

🐛 If you found an issue we encourage you to report it on github. Please specify your OS and the actions to reproduce it.

Contributing

👥 Ideas and contributions to the project are welcome. You must follow this guideline.

Built in collaboration with Liberate Science

Liberate Science

Acknowledgments

👏 Many thanks to @nicomanso for offering some basic infrastructure that was used for early testing and @krahser for his outstanding docker contribution (PR)

License

MIT © A GEUT project