This is a container repository that makes developing against the QMK web services (Configurator, API, and the Compiler) easier. After cloning this repo you can easily setup a web stack using either Docker or the python and jekyll development servers on your local machine.
We use Docker Compose to start and run the services required for QMK Configurator. Setting up a dev environment is a pretty simple process.
- Install Docker or Docker Desktop
- Clone qmk_web_stack:
git clone -r https://github.com/qmk/qmk_web_stack - Build the Docker containers
cd qmk_web_stack
docker-compose build - Edit the file
qmk_configurator/assets/js/script.js. Changebackend_baseurlfromhttps://api.qmk.fm/tohttp://127.0.0.1:5001 - Start the containers
docker-compose up - Populate the database
./populate_api.sh - Visit your local configurator: http://127.0.0.1:5000/
You are now ready to develop on your local environment. There are some things you can do outside the normal docker compose workflow that are helpful when developing features.
There are some functional tests for the qmk_compiler code. You can run them with the provided script:
cd ~/qmk_web_stack
./run_tests.sh
One of the key features of the API is the parsing of qmk_firmware to build a database about QMK keyboards and keymaps. There are two ways to start this process:
populate_api.sh: This script will run the update process directly, bypassing the normal infrastructure.trigger_update.sh: This script triggers an update using the same pathway asqmk_bot- it sets a flag in redis which is acted upon byqmk_api_tasks.
Normally you will want to use method #1. If you need to test method #2 you will also need to start the qmk_api_tasks service, which is not started by default.
The qmk_api_tasks service serves two roles- it continually tests keyboards to ensure they're compatible with Configurator, and it handles routine maintenance such as qmk_firmware updates and S3 cleanup. You typically will not need to run this service except for specific purposes, such as working on qmk_api_tasks itself or exercising the backend infrastructure.
To start qmk_api_tasks you will use the run_qmk_api_tasks.sh script:
cd ~/qmk_web_stack
./run_qmk_api_tasks.sh
If you need to manually clean your S3 storage you can use the cleanup_storage.sh script:
cd ~/qmk_web_stack
./cleanup_storage.sh
If you don't want to or can't use Docker you can setup and run the services locally. The process for doing so is rather involved, it's recommended that you use Docker if at all possible.
There are several services you need to setup to run the whole stack locally.
You will need an S3 compatible storage service backing your install. You can use S3, Spaces, or a local service such as Minio. The development environment scripts assume you're using Minio on your local machine.
We use RQ to decouple compiling the firmware from the webserver. It also handles administrative tasks like cleaning up old firmware hexes. Installing and administering Redis in a production environment is beyond the scope of this document.
For development purposes, you can simply install and run redis with the default configuration.
You will need to setup a python development environment where each service can access the qmk_compiler code. You can use the setup_virtualenv script to do so:
cd ~/qmk_web_stack
./setup_virtualenv
This will leave an activate script in the venv directory. Make sure to source this script whenever you need to start these services:
cd ~/qmk_web_stack
source venv/activate-<python-version>
The frontend uses jekyll so that we can ultimately push our work to gh-pages. Setup is pretty simple.
- Edit the file
qmk_configurator/assets/js/script.js. Changebackend_baseurlfromhttps://api.qmk.fm/tohttp://127.0.0.1:5001 - Setup the dependencies
cd ~/qmk_web_stack/qmk_configurator
bundle install
You can start a local instance of minio using the provided script. You need to do this each time you work on the stack.
cd ~/qmk_web_stack
./start_minio
You can start a local instance of the backend using start_local_server:
cd ~/qmk_web_stack
./start_local_server
Use jekyll to start the frontend service:
cd ~/qmk_web_stack/qmk_configurator
bundle exec jekyll serve
The frontend will now be available on port 4000: http://localhost:4000/
You will have to reload between changes as jekyll in it's default configuration doesn't support live reloading. If someone wants to contribute that please do.