polkadot-deployer

polkadot-deployer is a general tool for deploying Polkadot nodes, aiming to make it easy to deploy a network of nodes. To learn more about Polkadot explore the wiki or join the conversation at Riot.

polkadot-deployer allows you to create local or remote cloud deployments of polkadot. Currently it supports local deployments using Kind and remote deployments using Google Cloud Platform for the infrastructure and Cloudflare for the DNS settings that make your network accessible through websockets RPC.

Requirements

The tool is meant to work on Linux and MacOS machines. In order to be able to use the tool you will require to have installed recent versions of node (developed and tested with v10.7.0 and v10.15.1) and [docker](https://docs.docker.com/install/) for local deployments (developed and tested with 18.09.5). Once installed, you should also be able to [run docker` as a regular user](https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user). See the Troubleshooting section in case you have problems running the tool.

Local deployments (TL;DR method)

In order to deploy a number of polkadot nodes locally, we recommend using kubernetes-sigs/kind. The required steps to successfully deploy polkadot validator locally are as follows:

Download the latest polkadot deployer from git issuing the following command and change directory to polkadot-deployer:
git clone git@github.com:w3f/polkadot-deployer.git and run npm installto install all requirements.
You can deploy the polkadot deployer either using the interactive menu or by using a config file.

In order to create through the interactive menu issue the following command:
node . create --verbose
In order to deploy polkadot using the preset configuration file: config/create.local.sample.json and issue the following command:
node . create --config config/create.local.sample.json --verbose The process will start creating an instance of polkadot inside a your local kubernetes cluster that will be created as part of the procedure using kubernetes-sigs/kind. The entire procedure will take some time, so it might be a good idea to get some coffee at this point.

Once the local cluster is created, a kubeconfig file will be created in ~/.kube/ with a prefix of kind-config followed by the name of the created cluster (eg: kind-config-my-demo-testnet). This file should be moved or copied to your local .kube directory.
Attach kubectl to this kubeconfig file by issuing the command: export KUBECONFIG="$(kind get kubeconfig-path --name="kind")" You can verify your installation by using the following commands: kubectl cluster-info to see an overview of your local installation and kubectl get pods to see the pods running on your local cluster. You can also view all your local deployments using the command: node . list
At this point you can attach to the local polkadot web socket by visiting the websockets endpoint available at ws://127.0.0.1:11000 Furthermore you at this point you will be presented with the raw seeds for the created accounts, including the nodeKey, peerId, stash address and seed etc.
Once you are done with your local deployment of polkadot, you can delete your deployment using the destroy [name] command: node . destroy testnet5 More information on the polkadot-deployer usage commands can be found in the usage section.

Check the Troubleshooting section if something goes wrong with the installation.

Remote deployments

To perform a remote deployment of polkador to a public cloud provider we will follow the same general path. We will need to specify a number of extra attributes inside the json file with extra information such as our credentials etc. In order to be able to deploy remotely you will need:

A Linux machine to run this tool (macOS may fail, see the Troubleshooting section in case you have problems running the tool).
A project on GCP.
GCP service account and credentials in the form of the environment variable GOOGLE_APPLICATION_CREDENTIALS with the path of the json credentials file for your service account (see here for details and make sure the service account has sufficient privileges for GKE).
Cloudflare credentials as two environment variables CLOUDFLARE_EMAIL and CLOUDFLARE_API_KEY (see here for details about the API key, the email hould be the one used for registration. Also, your domain name registra should be Cloudflare since this tool relies on Cloudflare for generating SSL certification).
Enough quota on GCP to create the required resources (terraform will show the exact errors if this condition is not met).
Kubernetes Engine API and billing enabled for your project, see here.
Read through the usage section.

The required steps to successfully deploy polkadot validator on GCP are as follows:

Download the latest polkadot deployer from git issuing the following command:
git clone git@github.com:w3f/polkadot-deployer.git and run npm installto install all requirements.
In order to deploy polkadot using a the preset configuration file: my-gcp-testnet.json and issue the following command:
node . create --config config/create.remote.sample.json --verbose The process will start creating an instance of polkadot using GCP. A second cup of coffee is recommended at this point.
If you wish to delete your remote deployment of polkadot, you can use the destroy [name] command:
node . destroy testnet9

More information on the polkadot-deployer usage commands can be found in the usage section.

Usage

polkadot-deployer allows you to create, list, update and delete Polkadot networks of nodes, which we'll call deployments from now on. All the interaction with the command line tool is done through the following subcommands:

`create [options]`

Creates a deployment. It accepts a --config option with the path of a json file containing the definition of the deployment, like this for local deployments:

{
  "name": "testnet1",
  "type": "local",
  "nodes": 4
}

or this for remote deployments:

{
  "name": "testnet6",
  "type": "gcp",
  "nodes": 1,
  "remote": {
    "monitoring": true,
    "clusters": [
      {
        "location": "europe-west1-b",
        "projectID": "polkadot-benchmarks",
        "domain": "foo.bar"
      }
    ]
  }
}

These are the fields you can use:

name: unique string to distinguish your deployment in subsequent commands.
type: either local or remote, local or gcp allowed.
nodes: number of validators of the network, an integer between 2 and 20.
remote.monitoring: enable monitoring stack, see the Monitoring section
remote.clusters[i].location: region or zone to use for the deployment.
remote.clusters[i].projectID: id of the GCP project.
remote.clusters[i].domain: under which domain the tool will create the websockets endpoint.

You can also omit the --config option the tool will launch a wizard utility to get the deployment details.

Each deployment consists of two components, a cluster and a network.

The cluster is the common platform on top of which the network runs, and is currently based on kubernetes v1.13.
The network is composed of a set of polkadot nodes connected together, each of them created from this polkadot Helm chart. Helm charts are application packages for kubernetes, more about them here.

Once the deployment is created, the tool sets up a port forwarding process, so that one of the network nodes' websockets endpoint is available on your local host. The location of this port is shown at the end of the polkadot-deployer create call:

*********************************************************
 Websockets endpoint available at ws://127.0.0.1:11000
*********************************************************

You can use that url to access the network from PolkadotJS -> Settings

`list`

Shows details of all the created deployments:

┌──────────────┬─────────────────┬──────────────────────┬──────────┬─────────┬────────────────┐
│ Network name │ Deployment type │ WebSockets endpoint  │ Provider │ Workers │ Polkadot nodes │
├──────────────┼─────────────────┼──────────────────────┼──────────┼─────────┼────────────────┤
│ testnet1     │ local           │ ws://127.0.0.1:11000 │ kind     │ 1       │ 4              │
└──────────────┴─────────────────┴──────────────────────┴──────────┴─────────┴────────────────┘

`redeploy [name]`

Recreates a network on an existing cluster. It resets all the Polkadot nodes to their initial state, without having to creating the cluster again.

You can either pass the name of the deployment to recreate or let the wizard show a list of existing deployments.

`destroy [name]`

Destroy a deployment including cluster, network and portforwarding process.

You can either pass the name of the deployment to destroy or let the wizard show a list of existing deployments.

`benchmark`

Creates deployments and runs benchmarks on them. These are the type of benchmarks curretly supported:

finality: measures the mean time to finality on several networks with different number of validators. It accepts a --config option with the path of a json file containing the definition of the benchmark, like this:
```
{
  "name": "bm1",
  "type": "local",
  "benchmark": "finality",
  "startNodes": 2,
  "endNodes": 10,
  "blocks": {
    "offset": 10,
    "measure": 10
  },
  "reuseCluster": true
}
```
With this definition, the test will spin up deployments of 2 nodes through 10, measuring for each of them the mean time to finality of 10 consecutive blocks. These are the fields specific to this benchmark that you can use:
- benchmark: which tests to run, currently only finality implemented.
- startNodes: how many validators will be used in the first run.
- endNodes: how many validators will be used in the last run, the process will increment the number of nodes in 1 on each run.
- blocks.offset: number of blocks to wait before starting to the measurements.
- blocks.measure: number of blocks on which the metrics will be recorded.
- reuseCluster: if true the cluster will be cleaned but not deleted after the benchmark is finished and can be used in subsequent runs, this can speed up the execution.
You can pass also the destination directory of the result files in the --output parameter (current directory by default).

After the benchmarks are done the results are written to a json file, which includes all the metrics and the details of the execution, and to a file compatible with gnuplot, that can generate a png file with a plot of the results:
```
$ polkadot-deployer benchmark -c ./config/benchmark.finality.sample.json
Initializing nodes...
Done
Waiting for nodes ready...
Done
***************************************
 Starting benchmarks with 2 nodes
***************************************
New produced block: 1, timestamp: 1558369995883
Last finalized block: 1, finalized at: 1558370001258

...

***************************************
 Finished benchmarks with 6 nodes
***************************************
Done
Results writen to ./polkadot-deployer.benchmark.finality.20190520-063948.json
gnuplot writen to ./polkadot-deployer.benchmark.finality.20190520-063948.gnuplot

$ gnuplot /polkadot-deployer.benchmark.finality.20190520-063948.gnuplot
```
The previous gnuplot command will generate a benchmark.png file with the benchmark plot on the current directory.

Monitoring

You can enable monitoring for remote deployments, by setting remote.monitoring to true in the configuration. When enabled, polkadot-deployer will install a generic monitoring stack composed of prometheus, alertmanager, grafana and loki, and a more polkadot-specific set of tools around substrate-telemetry.

There will be a grafana instance deployed per cluster, and they will be accessible at https://grafana.<deployment_name>-<n>.<domain>, being n the order of the cluster in the config starting with 0, and can be accessed with username admin and password controlled by the envirnment variable GRAFANA_PASSWORD (grafanapassword if not set).

All the nodes in the deployment will be sending operational data to a substrate-telemetry backend deployed on the first cluster (according to the config definition order). Connected to that backend is also a frontend accessible at https://telemetry.<deployment_name>-0.<domain>.

A prometheus exporter is also deployed as part of the substrate-telemetry pod, there is a grafana dashboard called Polkadot Metrics showing information from the exposed metrics.

Troubleshooting

Below are some common problems found by users. If you have an issue and this suggestions don't help don't hesitate to open an issue. You can get more information about what is the actual adding the --verbose option to any polkadot-deployer command.

In some cases the installation process can produce errors from the secp256k1 dependency with messages related to the required python version, like:
```
gyp ERR! configure error
gyp ERR! stack Error: Python executable "/usr/local/opt/python/libexec/bin/python" is v3.7.3, which is not supported by gyp.
```
To solve this problem you can either define some alias from the command line before installing:
```
alias python=python2
alias pip=pip2
```
or call the install command with an additional option:
```
npm i -g --python=python2.7 polkadot-deployer
```
See this issue for details.
For local deployments, if after issuing a create command you find the deployer stuck with a message like:
```
wait-on(537) waiting for: http://127.0.0.1:10080/kubernetes-ready
```
then it is possible that the cluster is not able to be created using your local docker installation. Make sure that:
- The system has enough free disk space (at least 10Gb).
- There are no leftovers on your docker installation. You can clean up with:
```
docker system prune -a --volumes
```

In case you are experiencing problems and any of the above solution works for you don't hesitate to open an issue.

CI/CD Workflow

When a PR is proposed to this repo, the npm test task is run, this includes linting and unit tests.

After the PR is merged into master, when a semantic version tag (vX.Y.Z) is pushed the tests are run again and, if all is ok, the package is published to npm registry

DanielMagro97/polkadot-deployer