With Vagrant and VirtualBox installed:
vagrant up
vagrant ssh -c 'cd /vagrant && bundle exec jekyll serve --force_polling --host 0.0.0.0'
To run other interactive commands, run vagrant ssh, then cd /vagrant.
To stop Vagrant: vagrant halt
- Make sure you have the latest LTS version of NodeJS installed.
- Make sure you have the latest version of Yarn installed.
- Run
yarnin the root folder of the repository to install all dependancies.
We are in the process of incrementally moving from a bespoke Jekyll-Preact architecture to a standard React (via Create React App) architecture.
In order to build the current Jekyll-Preact assets you can run the following in the project root folder:
yarn build
The new React environment is integrated into the existing architecture via a single JavaScript file generated by means of a Yarn monorepo workspaces (in /packages/workspace/). It also follows standard conventions for monorepos. You can run the following scripts in /packages/webapp/:
yarn start: This is where you will spend the majority of your time, this is a library that contains (and documents) all the components/views that will be included in the built JavaScript file.yarn build: Builds the production JavaScript file that needs to be added to the existing Vulekamali stack.
Create a python 2 virtual environment, e.g. with virtualenv2 env
Activate it and install python dependencies
source env/bin/activate
pip install -r requirements.txt
yarn build
Be sure to run Jekyll and have a quick look whether things look and work right.
bundle exec jekyll serve --watch
Regenerating the site can take over a minute on a typical dev machine. You can speed it up by working on a subset of the site data. We delete a bunch of the routing files and tell git to ignore those changes in your local repository using git update-index --assume-unchanged. This means only national departments and departments from Mpumalanga and the Eastern Cape will exist, bringing regeneration time down to about 10s.
git ls-files|grep provincial|egrep -v '(mpumalanga|eastern-cape)' | grep html | tr '\n' ' ' | xargs git update-index --assume-unchanged
git ls-files|grep provincial|egrep -v '(mpumalanga|eastern-cape)' | grep html | tr '\n' ' ' | xargs rm
To remove assume-unchanged from all files with that currently configured:
git ls-files -v|grep '^h'| sed 's/^h//' | xargs git update-index --no-assume-unchanged
Every branch push results in a Travis CI build, but by default it won't regenerate data files.
Add [ci] to the commit message of the latest commit in the pushed branch to opt into generating data files.
Add [staging] to that commit message to use the staging web data server instead of production. This is useful if we want to see what the data will look like for a development branch of the web data server.
If any files changed, Travis CI will add and commit the changes and push that back to the branch. That means you'll need to pull the branch and perhaps use pull --rebase if you've since committed other changes. Don't trust the changes because it was a robot - check that the site works as expected and the diff looks reasonable before merging your PR
Best practise is to place these tags as the first part of the first line of the commit message. This makes it easy to see whether staging or production was used in the pull request list of commits.
To create a commit without any code changes to trigger a build, use --allow-empty. e.g.
git commit -m "[ci][staging] trigger data regeneration for new department xyz data" --allow-emptysource env/bin/activate
python generate/from_dynamic.py
Then use git status and git diff to get an idea of what changed. If it looks sensible, add the updated files and commit and PR back into the branch where you want changes reflected.
When modifying the Dynamic Budget Portal server, you might want to point to your development server:
PORTAL_URL=http://localhost:8000/ python generate/from_dynamic.py
You can update one file at a time as follows:
curl -o _data/2016-17/national/departments/planning-monitoring-and-evaluation.yaml https://dynamicbudgetportal.openup.org.za/2016-17/national/departments/planning-monitoring-and-evaluation.yaml