GRUA (Generic Resources Unifier and Allocator)

GRUA is a Puppet External Node Classifier (ENC) application. Here are some of its key features:

Multi-tenant support: It is possible to manage multiple Puppet Servers (called Master Zones) with a single GRUA application;
Multi-environments support: GRUA supports multiple Puppet Servers environments and it can automatically sync information about them;
Access control: It is possible to establish selective restrictions of access to Master Zone, depending on the user permissions. Also, GRUA's API is protected and requires authentication;
Group-orientation: Nodes can only be associated to groups. And groups are abstractions that are associated with a Puppet Server and one of its environments, that may contain classes and parameters.

Tech

GRUA uses the following technologies:

Database: PostgreSQL 9.6.9
Backend: Python 3.6 + Django 2 + Django Rest;
Frontend: HTML5 + Sass + JS (ES6 + Babel) + Gulp.
Docker image: postgres + contribsys/faktory + python:3-alpine

How to run (with Docker)

GRUA is very easy to install and deploy in Docker containers, through Docker Compose.

Set environment variables on docker-compose.yml on scheduler and worker services:

  ...
  scheduler:
    ...
    environment:
      - WEBAPP_USER=<username to be created>
      - WEBAPP_PASS=<user pass to be created>
  ... 
  worker:
    ...
    environment:
      - WEBAPP_USER=<username to be created>
      - WEBAPP_PASS=<user pass to be created>
  ...

Build the containers with:

docker-compose build

Apply database migrations with:

docker-compose run webapp python manage.py migrate

Create an admin user with the information provided in WEBAPP_USER and WEBAPP_PASS:

docker-compose run webapp python manage.py shell -c "from django.contrib.auth.models import User; User.objects.create_superuser('<chosen username>', '<user email>', '<chosen password>')"

Spin up the containers with:

docker-compose up

By default, the Docker will expose port 8080. Therefore, you can access the main webapp at localhost:8000.

Usage and documentation

A more complete documentation of GRUA can be found on our wiki.

Setting up a Puppet Server to communicate with GRUA

In order to a Puppet Server correctly communicate with GRUA there are two viable options:

Use our puppet-grua module on each Puppet Server on your infrastructure that should be communicating with GRUA;
Manually perform the following actions:
1. Create a Master Zone for the Puppet Server:
2. Obtain the Master Zone id (UUID) via GRUA's API (on /api/master_zones):
3. Access the Puppet Server responsible for the Puppet CA;
4. Generate a certificate for GRUA via Puppet CA;
5. Sign this certificate via Puppet CA;
6. Create/obtain a token on GRUA's admin page for an admin user (on /admin/authtoken/token/add/):
7. Config Puppet Server's API permissions to allow requests with the generated certificate on the classes/environments endpoints (using Master Zone's UUID value and GRUA's auth token);
8. Restart the Puppet Server;
9. Upload the certificate file and the key file for the PuppetServer (Master Zone) via a PUT on GRUA API (on /api/master_zones/<master_zone_id>):
10. Create a file on the Puppet Server to consult GRUA's API node_classifier endpoint (on /api/nodes/node_classifier/?certname=<node_certname>&master_id=<master_zone_id>);
11. Restart the Puppet Server
12. Repeat steps 9, 10 and 11 for each compiler on a non-monolithic server architecture.

We absolutely recommend you to choose the first option, as it is much easier and the straight-forward one to go with.

API

Our API is documented using Swagger and can be accessed on GRUA via /docs/ URL path.

Development (with Docker)

Want to contribute? Awesome!

Contribution steps

First and foremost: adhere to our Code of Conduct;
Fork it (https://github.com/instruct-br/grua/fork);
Create your feature branch (git checkout -b feature/fooBar);
Commit your changes with an informative message (git commit);
Push to the branch (git push origin feature/fooBar);
Create a new Pull Request following the project's pull request template (available on Github interface).

Development environment

GRUA has a docker-compose.dev.yml file specifically created to help the development process. It is responsible to mount volumes on Docker so the changes you made on the code are reflected on the containers.

Build the containers with:

docker-compose -f docker-compose.yml -f docker-compose.dev.yml build

Apply database migrations with:

docker-compose run webapp python manage.py migrate

Create a generic admin user:

docker-compose run webapp python manage.py shell -c "from django.contrib.auth.models import User; User.objects.create_superuser('admin', 'admin@example.com', 'admin')"

Spin up the containers with:

docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

Frontend static assets are mapped with local files in the development containers. To generate assets locally you need to install JS dependencies: npm 6.4.1 and node v8.12.0. These assets can be created with:

cd webapp/
npm install
npm run build
cd ..

Style guide

We mostly use PEP8 as our style guide (aside of us allowing bigger line lengths).

You can use Black to format your code (while the containers are up):

docker exec -it grua_webapp_1 black .

And Flake8 to check it (while the containers are up):

docker exec -it grua_webapp_1  pipenv run flake8 --exclude=*/migrations/* --exclude node_modules/ --ignore=E501 .

Tests

We also encourage developers to implement tests when contributing to a new feature or a bug fix. It is possible to run tests with:

docker-compose -f docker-compose.yml -f docker-compose.dev.yml run webapp python manage.py test

Release History

0.1.0
- Public release
0.1.1
- Fix node-sass version on package.json
- Update info about environment variables WEBAPP_USER and WEBAPP_PASS

thiagopena/grua