Avalanchego Operator

Propose of this operator is to ease the automation of test processes. Operator provides an abstraction layer, that allows programmatically create private Avalanche chains at any time and scale.

Creating Validators and custom nodes with autogenerated genesis

Deployment template

Avalanchego Operator extends kubernetes API with new kind of resources, Avalanchego. To new create a network from scratch, apply this object.

apiVersion: chain.avax.network/v1alpha1
kind: Avalanchego
metadata:
  name: avalanchego-test-validator
spec:
  # Add fields here
  deploymentName: test-validator
  nodeCount: 5
  image: avaplatform/avalanchego
  tag: v1.6.0
  imagePullSecrets:
  - name: regcred
  env:
  - name: AVAGO_LOG_LEVEL
    value: debug
  resources:
    limits:
      cpu: "1"
      memory: 2Gi
    requests:
      cpu: 500m
      memory: 1Gi

name this name is a K8s's reference to the chain validators, use this name to check the status or delete the chain

deploymentName is a suffix for downstream k8s objects (pods, services, secrets, etc.)

nodeCount initial number of validators, these nodes will be added to genesis.json as initial stakers

image and tag docker image and tag

env common configuration for chain nodes, check the full list here: https://github.com/ava-labs/avalanchego/blob/master/config/keys.go

resources amount of CPU and RAM, an individual node would be able to use (https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)

imagePullSecrets a map of preset secrets with dockerhub credentials. More information on how to generate and upload a dockerhub secret here: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/

Logic and deployment output

WARNING: currently operator does not support in-flight changes. Spin up a new node, and delete the existing one if not needed.

After applying a deployment template, the operator generates certificates and keys (nodeCount of them), calculates node id's, generates genesis.json and starts the validator group.

Operator updates deployment's status and emits events on every update:

apiVersion: chain.avax.network/v1alpha1
kind: Avalanchego
metadata:
  name: avalanchego-test-validator
spec:
...
status:
    bootstrapperURL: avago-test-validator-0-service
    genesis: '{"networkID":1,......."message":"Make time for fun"}'
    networkMembersURI:
    - avago-test-validator-0-service
...
    - avago-test-validator-4-service

networkMembersURI Addresses of all the validators, created

DISCLAIMER

operator does not check node health, it only outputs URI, after it is generated and applied
k8s is an asynchronous system, whenever you apply an object to k8s API and get success result, it only means that an object was successfully received, not that operation finished successfully

Creating custom nodes

To create a new node for an existing network or create a new chain with pregenerated certificates/genesis use this example:

apiVersion: chain.avax.network/v1alpha1
kind: Avalanchego
metadata:
  name: avalanchego-test-worker
spec:
  deploymentName: test-worker
  bootstrapperURL: avago-test-validator-0-service
  nodeCount: 1
  image: avaplatform/avalanchego
  tag: v1.6.0
  env:
  - name: AVAGO_LOG_LEVEL
    value: debug
  resources:
    limits:
      cpu: "1"
      memory: 2Gi
    requests:
      cpu: 500m
      memory: 1Gi
# Genesis is in deployed validator status
  genesis: ''
  certificates:
  - cert: ''
    key: ''

bootstrapperURL mandatory; set bootstrapper URL to attach to an existing network, leave empty to start a new one

genesis mandatory; genesis information in JSON format

certificates optional; an array of certificates/keys; autogenerates, if empty; length of the array should be equal to

nodeCount

For fully custom deployment see config/samples/chain_v1alpha1_avalanchego_static.yaml

Exposing an Avalanchego node

To expose a node to external networks (Internet), please create an ingress object (namnespace should match) Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: HTTP
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
  name: avago-test-val-0-ing
spec:
  rules:
  # URI
  - host: avago-test-val-0.avax-dev.network
    http:
      paths:
      - backend:
          service:
            # Service name, get it from the status
            name: avago-test-val-0-service
            port:
              number: 9650
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - avago-test-val-0.avax-dev.network
    # pre created in integration cluster
    secretName: cloudflare-avax-dev-tls

Developing

This operator was created with operator-SDK (https://sdk.operatorframework.io/docs/) Please, read the docs before committing any changes. Make a PR and ask DevOps for a review.

DO NOT update ./config manually, unless you know what you are doing

Upgrading

update Makefile with a new version
run make docker-build and make docker-push this will also update operator's deployment template with a new version
merge changes to main, argocd will automatically pick all the changes and deploy new version in the cluster

lebdron/avalanchego-operator