/novops

Platform agnostic secret and config aggregator

Primary LanguageRustGNU Lesser General Public License v3.0LGPL-3.0

Novops

novops-features

Novops is like .env, but on steroïd 💪

  • Load secrets directly in memory from any source (AWS, GCloud, Azure...)
  • Manage multiple environments
  • Set plain variables, secrets, files and generate temporary credentials
  • Stop spreading secrets across CI tools and dev environments

Use Novops to easily setup secrets and variables in your development environment or CI platform. Stop having developers setup their own .env.prod, .bashrc - Novops takes care of it and make sure everyone's on the same page.

Getting Started

Let's deploy an application with secret password and SSH key from Hashicorp Vault and temporary AWS credentials.

Install static binary (or use Nix):

curl -L "https://github.com/PierreBeucher/novops/releases/latest/download/novops-X64-Linux.zip" -o novops.zip
unzip novops.zip
sudo mv novops /usr/local/bin/novops

Create .novops.yml and commit it safely - it does not contain any secret:

environments:
  dev:
    
    # Environment variables for dev environment
    variables:
      
      # Fetch Hashicorp Vault secrets
      - name: DATABASE_PASSWORD
        value:
          hvault_kv2:
            path: app/dev
            key: db_password

      # Plain string are also supported
      - name: DATABASE_USER
        value: root
    
    # Load files in memory (not written on disk)
    files:
      - variable: APP_SSH_KEY # Will point to generated file
        content:
          hvault_kv2:
            path: app/dev
            key: ssh_key
    
    # Generate temporary AWS credentials for IAM Role
    # Provide environment variables:
    # - AWS_ACCESS_KEY_ID
    # - AWS_SECRET_ACCESS_KEY
    # - AWS_SESSION_TOKEN
    aws:
      assume_role:
        role_arn: arn:aws:iam::12345678910:role/dev_deploy

Load secrets as environment variables:

# Run a sub-process with secrets
# Secrets are cleaned-up on exit
novops run -- sh

# Or source directly into your shell
source <(novops load)

Secrets are now available:

echo $DATABASE_PASSWORD
# passxxxxxxx

echo $APP_SSH_KEY
# /run/user/1000/novops/... 
# Files are not written on disk but remain in memory

env | grep AWS
# AWS_ACCESS_KEY_ID=AKIAXXX
# AWS_SECRET_ACCESS_KEY=xxx
# AWS_SESSION_TOKEN=xxx

🔐 Security

Novops loads secrets in memory and does not write anything to disk. Secrets are loaded temporarily and kept only for as long as they are needed.

See Novops Security Model for details

Features

  • Securely load secrets and generate temporary credentials directly in memory as environment variables or temporary files
  • Fetch secrets at their source. No more syncing secrets between local tool, CI/CD, and Cloud secret service
  • Fetch secrets from anywhere: Hashicorp Vault, AWS, Google Cloud, Azure...
  • Provide secrets directly to process, easing usage of IaC tools like Terraform, Pulumi, Ansible...
  • Manage multi-environments setup
  • Easy installation with fully static binary or Nix

Example usage

Shell

Novops run in any shell

# Run a sub-process with secrets
# Secrets are cleaned-up on exit
novops run -- sh

# Or source directly into current shell
source <(novops load)

🐳 Docker & Podman

Load environment variables directly into containers:

docker run -it --env-file <(novops load -f dotenv -e dev) alpine sh
podman run -it --env-file <(novops load -f dotenv -e dev) alpine sh

Multi-environment context

.novops.yml support multi-environment context:

environments:
  dev:
    variables:      
      - name: DATABASE_PASSWORD
        value:
          hvault_kv2:
            path: app/dev
            key: db_password
  prod:
    variables:      
      - name: DATABASE_PASSWORD
        value:
          hvault_kv2:
            path: app/prod
            key: db_password

novops prompts for environment by default

novops load 
# Select environment: dev, prod

Use -e ENV to avoid prompt and run directly

novops load -e dev

Or specify a default environment in .novops.yml

config:
  default:
    environment: dev

Temporary files

Novops can write secret in files such as SSH keys. Files are not written to disk but in memory under a secure directory, see Novops Security Model.

environments:
  dev:   
    files:
      
      # Each file entry generates a file AND an environment variable
      # pointing to generated file such as
      # ANSIBLE_PRIVATE_KEY=/run/user/1000/novops/.../file_ANSIBLE_PRIVATE_KEY
      - variable: ANSIBLE_PRIVATE_KEY
        content:
          hvault_kv2:
            path: app/dev
            key: ssh_key

Plain strings

Like .env, Novops support plain strings. This can be useful to specify both user and passwords or some generic configs.

environments:
  dev:
    variables:      
      # Plain string will be loaded as DATABASE_USER="app-dev"
      - name: DATABASE_USER
        value: app-dev

      - name: DATABASE_PASSWORD
        value:
          hvault_kv2:
            path: crafteo/app/dev
            key: db_password
    files:
      # File with plain string content
      - variable: APP_CONFIG
        content: |
            db_host: localhost
            db_port: 3306

More examples: Nix, GitLab, GitHub, Ansible, Terraform, Pulumi...

Modules: load secrets and generate temporary credentials

Novops uses modules to load and generate temporary secrets from various platforms and providers. Configure them in .novops.yml:

Hashicorp Vault

Supported Hashicorp Vault Secret Engines:

  • Key Value v1/v2
  • AWS: generate temporary STS credentials
environments:
  dev:
    variables:
      
      # Key Value v2
      - name: DATABASE_PASSWORD
        value:
          hvault_kv2:
            path: app/dev
            key: db_password
      
      # Key Value v1
      - name: SECRET_TOKEN
        value:
          hvault_kv1:
            path: app/dev
            key: token
            mount: kv1

    # Hashivault module with AWS secret engine
    hashivault:
      aws:
        name: dev_role
        role_arn: arn:aws:iam::111122223333:role/dev_role
        role_session_name: dev-session
        ttl: 2h

AWS

Multiple AWS services are supported:

  • Secrets Manager
  • STS Assume Role for temporary IAM Role credentials
  • SSM Parameter Store
environments:
  dev:

    variables:
      # SSM Parameter Store
      - name: SOME_PARAMETER_STORE_SECRET
        value:
          aws_ssm_parameter:
            name: secret-parameter
      
      # Secrets Manager
      - name: SOME_SECRET_MANAGER_PASSWORD
        value:
          aws_secret:
            id: secret-password
    
    # Generate temporary AWS credentials for IAM Role
    # Generate environment variables:
    # - AWS_ACCESS_KEY_ID
    # - AWS_SECRET_ACCESS_KEY
    # - AWS_SESSION_TOKEN
    aws:
      assume_role:
        role_arn: arn:aws:iam::12345678910:role/dev_deploy

All modules: AWS, Hashicorp Vault, GCloud, Azure...

Full Documentation

Checkout full documentation

Roadmap

The following modules are expected to be implemented:

Feel free to create an issue and contribute a PR !

Contributing

We welcome contributions: bug reports/fixes, modules, proposals... :) To get started you can check Novops internal architecture and:

Inspiration and similar tools

License

GNU Lesser General Public License

Acknowledgment

Novops was initially developed and used at Novadiscovery who graciously transferred code ownership. Thanks Nova's team for your help in designing and developing Novops.