Archweb README
To get a pretty version of this document, run
$ markdown README > README.html
License
See LICENSE file.
Authors
See AUTHORS file.
Dependencies
- python
- rsync (optional for mirrorcheck with rsync mirrors)
Python dependencies
More detail in requirements.txt
and requirements_prod.txt
; it is best to
use virtualenv and pip to handle these. But if you insist on (Arch Linux)
packages, you will probably want the following:
- python-django
- python-psycopg2
- python-markdown
- python-memcached
Testing Installation
-
Run
python -m venv env
.cd /path/to/archweb && python -m venv ./env/
-
Activate the virtualenv.
source ./env/bin/activate
-
Install dependencies through
pip
.pip install -r requirements.txt
-
Copy
local_settings.py.example
tolocal_settings.py
and modify. Make sure to uncomment the appropriate database section (either sqlite or PostgreSQL). -
Migrate changes.
./manage.py migrate
-
Load the fixtures to pre populate some data. If you don't want some of the provided data, adjust the file glob accordingly.
./manage.py loaddata main/fixtures/*.json ./manage.py loaddata devel/fixtures/*.json ./manage.py loaddata mirrors/fixtures/*.json ./manage.py loaddata releng/fixtures/*.json
-
Use the following commands to start a service instance
./manage.py runserver
-
To optionally populate the database with real data:
wget http://mirrors.kernel.org/archlinux/core/os/x86_64/core.db.tar.gz ./manage.py reporead x86_64 core.db.tar.gz # Package file listing wget http://mirrors.kernel.org/archlinux/core/os/x86_64/core.files.tar.gz ./manage.py reporead --filesonly x86_64 core.files.tar.gz
Alter architecture and repo to get x86_64 and packages from other repos if needed.
-
Database Updates for Added/Removed packages
sqlite3 archweb.db < packages/sql/update.sqlite3.sql
For PostgreSQL use packages/sql/update.postgresql_psycopg2.sql
Testing SMTP server
To be able to create an account on your test environment an SMTP server is
required. A simple debugging SMTP server can be setup using Python and aiosmtpd
.
Install aiosmtpd
pip install aiosmtpd
Run the server
python -m aiosmtpd -n -l localhost:1025
In local_settings.py add entries to set EMAIL_HOST to 'localhost' and EMAIL_PORT to 1025.
Running tests and coverage
Install the test dependencies:
pip install -r requirements_test.txt
To the unittests execute the following commands:
make collectstatic
make test
Running coverage:
make coverage
make open-coverage
Django Debug toolbar
To use the Django Debug toolbar install django-debug-toolbar and in local_settings.py set DEBUG_TOOLBAR to True.
Management commands
Archweb provides multiple management commands for importing various sorts of data. An overview of commands:
- generate_keyring - Assemble a GPG keyring with all known developer keys.
- pgp_import - Import keys and signatures from a given GPG keyring.
- read_rebuilderd_status - Import rebuilderd status into Archweb.
- rematch_developers - Rematch flag requests and packages where user_id/packager_id is NULL to a Developer.
- reporead - Parses a repo.db.tar.gz, repo.files.tar.gz file and updates the Arch database with the relevant changes.
- reporead_inotify - Watches a templated patch for updates of *.files.tar.gz to update Arch databases with.
- donor_import - Import a single donator from a mail passed to stdin
- mirrorcheck - Poll every active mirror URLs to store the lastsnyc time and record network timing details.
- mirrorresolv - Poll every active mirror URLs and determine wheteher they have IP4 and/or IPv6 addresses.
- populate_signoffs - retrieves the latest commit message of a signoff-eligible package.
- update_planet - Import all feeds for users who have a valid website and website_rss in their user profile.
- read_links - Reads a repo.links.db.tar.gz file and updates the Soname model.
- read_links_inotify - Watches a templated patch for updates of *.links.tar.gz to update Arch databases with.
Updating iPXE images
The binaries required for iPXE based netboot are updated by copying them from
the ipxe package to
the static content directory (with the run_ipxe
script the binaries may be tested beforehand):
cp -v /usr/share/ipxe/x86_64/ipxe-arch.efi /usr/share/ipxe/ipxe-arch.{ipxe,lkrn} sitestatic/releng
Afterwards a detached PGP signature using a valid WKD enabled packager key is created for each file:
gpg --sender "User Name <your@mail.address>" --detach-sign sitestatic/netboot/*.{efi,ipxe,lkrn}
Production Installation
Arch Linux has an Ansible role for Archweb in their infrastructure repo.
vim: set syntax=markdown et: