/baldr

Reporting to increase visibility of barriers to automation

Primary LanguagePLpgSQLMIT LicenseMIT

postgresql express node ##test

Perfecto Project Baldr

Named after the Norse God of Light, this server generates daily reports that increase visibility of automation obstacles. The server can be accessed at https://baldr.perfecto.io/?cloud=FQDN&date=DATE-ISO-8601&securityToken=YOUR-SECURITY-TOKEN with the appropriate substitutions for the parameters listed. This article explains how to request a security token.

Getting Started

Prerequisites for macOS

Local Project Setup

  1. Launch Terminal on your Mac.

  2. Make sure prerequisites are installed. If Homebrew was already installed, be sure to run brew doctor and address any issues before installing other brew packages.

  3. Type brew services start postgresql to start PostgreSQL (if not already running)

  4. From your main development directory (I use ~/dev), type git clone https://github.com/nstuyvesant/baldr.git to clone the project

  5. Type cd baldr && npm install to connect to the directory and install NodeJS dependencies

  6. Type psql -d postgres -f db_create.sql to create the baldr database and populate it with sample data for a cloud called demo.perfectomobile.com with a snapshot for 2018-06-20

  7. Type npm start

Testing

  1. To retrieve a customer-facing report, go to http://localhost:3000/?cloud=demo.perfectomobile.com&date=2018-06-19&securityToken=YOUR-SECURITY-TOKEN replacing YOUR-SECURITY-TOKEN with the value you got from the Prerequisite step

  2. To view JSON returned by the API, go to http://localhost:3000/api/?cloud=demo.perfectomobile.com&date=2018-06-19&securityToken=YOUR-SECURITY-TOKEN replacing YOUR-SECURITY-TOKEN with the value you got from the Prerequisite step

  3. To edit previously submitted JSON directly in the editor, go to http://localhost:3000/editor.html?cloud=demo.perfectomobile.com&date=2018-06-19&securityToken=YOUR-SECURITY-TOKEN replacing YOUR-SECURITY-TOKEN with the value you got from the Prerequisite step

  4. To submit JSON via the API, go to http://localhost:3000/editor.html?cloud=demo.perfectomobile.com&securityToken=YOUR-SECURITY-TOKEN replacing YOUR-SECURITY-TOKEN with the value you got from the Prerequisite step

Production Setup

  1. To run in production on Ubuntu 18.04, login to your server then type sudo useradd baldr to create a low-privileged user.

  2. Type cd /home && sudo git clone https://github.com/nstuyvesant/baldr.git

  3. Type sudo chown -R baldr:baldr /home/baldr

  4. Type sudo chmod 755 /home/baldr && sudo chmod 744 -R /home/baldr/*

  5. Type cd /home/baldr

  6. Type su - postgres

  7. Type psql -d postgres -f db_create.sql to create the database then exit

  8. Type su - baldr to open a shell as the baldr user

  9. Type npm install to install project dependencies then exit

  10. Type sudo cp baldr.service /etc/systemd/system/ to copy the SystemD configuration file to the required directory

  11. Type sudo systemctl daemon-reload to reload the list of daemons

  12. Type sudo systemctl start baldr to start the Baldr Report Server on TCP port 3000

  13. Type sudo systemctl enable baldr to enable it to run on startup

  14. Setup a reverse proxy for http://fqdn:3000 with an SSL certificate

  15. Test by typing curl https://fqdn/editor.html

  16. Create a cron job for the report analyzer jar by adding this line to your crontab 0 0 * * * cd /home/baldr/uploader/; java -jar ReportAnalyzer-3.0.jar

  17. Create a cron job that populates the quality score 0 2 * * * python /home/baldr/python/score_quality.py

Overview of files

  • db_create.sql - Creates the PostgreSQL database, functions to generate JSON and populates with sample data

  • app.js - Web server using ExpressJS with pg-native to serve report's UI and API for required JSON

  • editor.html - JSON editor page. Submissions from this page are stored in the underlying PostgreSQL database, vr, replacing any contents for the snapshot date.

  • sample-input.json - JSON sample loaded by default in the editor. It represents the valid format for JSON snapshot upserts.

  • sample-output.json - JSON sample returned by the HTTP GET handler in index.js. The difference from the input are the two properties: last7d and last14d. These are not part of the input as they are calculated from previous daily snapshots.