A bash script to deploy a django project for production sites. The script sets up the environment under which the django project can be run safely. The environment will have the following characterstics:
- Designed to be run using Gunicorn/WSGI and as an app under NGINX.
- Each site will be run in a dedicated automation user account sandbox.
- A new PostgreSQL database with the same name as the project specified in command line that can be used by Django.
- Python virtualenv setup with basic packages such as pip and django installed.
- Supervisor, the python based daemon control process installed and configured with the necessary conf file to control the WSGI process.
- An script to autostart supervisor and control it like other Ubuntu services.
The primary end-user facing components of the package are the two scripts:-
install_os_prereq.shdeploy_django_project.sh
The names ought to be self explanatory.
In any case, the first script ensures that the necessary packages are installed in the OS. Run this first when deploying a django project to a fresh Ubuntu image.
The second script creates the automation account, virtual environment, PostgreSQL DB & and sets up the assortment of scripts that make up the website deployment.
install_os_prereq.sh needs to be run only once per machine, typically after
a fresh OS image installation. Thereafter deploy_django_project.sh can be run
as many times as necessary, once for each domain that you want to support on
the same machine (using HTTP virtual hosts).
The old command
create_django_project_run_env.shis essentially a wrapper that calls the above two scripts in sequence. Unless you have a specific reason (I can't think of any), it's best that you don't use this script at all. Instead use the new scriptsinstall_os_prereq.shanddeploy_django_project.sh.
Use the commands as follows:
install_os_prereq.sh [<python-version>]
deploy_django_project.sh <project> <domain> [<python-version>]
where:
<project> is the name of the parent project you use to refer to the
solution. This should be a single word without space or other special charactes.
A new user account with this name will be created under group webapps and
the django project will be served from this account's home folder
/webapps/<project>_project. If group webapps does not exist, it will
be created.
<domain> is the domain name where the website is to be to be deployed.
Specify this without the www prefix. Appropriate NGINX configuration files
will be generated to direct requets to both <domain> and
www.<domain> to the django app.
<python-version> is either 2 or 3, which is the python version that you
wish to use for the project. In most cases this can be omitted as it defaults
to 3.
For example, for deploying the domain example.com, use the following command:
$ install_os_prereq.sh
$ deploy_django_project.sh example example.com
Note that in both the above commands the default Python version of 3 is used.
This will create a new user account example under group webapps with home
folder set to /webapps/example_project/. Under this folder, it will create a
python virtual environment and a gunicorn startup script, that will be auto
started using the Supervisor process control system. Ngix configuration will be
updated such that requests to domain example.com (and wwww.example.com) will be
proxied to this gunicon WSGI instance. A placeholder Django app, aptly named
<project> will also be created under the home folder which will serve as the
the WSGI endpoint.
A production Django app is deployed using the WSGI proxy mechanism which serves requests proxied from an HTTP server such as NGINX or Apache. Here the HTTP server does little more than forwarding the incoming requests to the configured WSGI backend application server and forward the received response from the appserver to the client. The HTTP server will be configured such that for specific domain name(s), a specific WSGI app server instance will be used. For Django apps, the recommended HTTP server and WSGI app servers are NGINX and Gunicorn respectively.
Since deploying a production web app involves multiple components and each of them with their own configuration, it's imperative that all these varied steps are either documented or captured in the form of script files such that these can be replicated across multiple server instances. Even for the most simple web application that uses a single server instance, a production site requires a staging server where the code needs to be deployed and tested for runtime and deployment issues before launching it live. And for sites expecting medium to heavy traffic and load, the code would have be deployed on multiple servers with the HTTP server deployed as a load balancer or a dedicated load balancer distributing the HTTP request load equally across multiple application servers. Therefore the need for consistent configuration of multiple servers cannot be overemphasized.
An even better solution would be standardizing the deployment characteristics of a Django web application such that multiple Django applications, serving multiple sites, all are deployed in a certain fixed configuration. This would make troubleshooting and subsequent fixing of any deployment issues easy as all sites retain the same characteristics.
This script is an attempt to achieve this deployment standardization for all Django apps.
- Script is written to work with Ubuntu Linux 14.04, though it should work fine on Ubuntu versions >= 12.04. Script does install the prerequisite Linux packages if they are not installed and therefore a vanilla OS installation is all that is necessary.
- PostgreSQL is used as the database backend. PostgreSQL is considered the best DB backend for Django apps and provides more sophisticated RDBMS features than MySQL.
- As already mentioned HTTP server is provided by NGNIX and WSGI is served by Gunicorn. NGINX and Gunicorn are considered the best match for Django apps.
- Supervisor process control system is managed to Gunicorn appserver processes. Using supervisor provides automatic restart of the Gunicorn appserver daemon, should it crash for some reason, making the overall deployment that much more robust.
- The app server is served using python runtime from a dedicated virtual environment and therefore the system-wide python distribution is not touched.
The sequence of steps taken by the script can be summed up as:
As part of the installation, the following OS packages are installed:
- git
- build-essential
- nginx
- postgresql
- libpq-dev
- python-dev|python3-dev
- python-pip|python3-pip
Note that the package python-pip & python3-pip is a python package, though
it is installed from the OS package distribution mechanism.
After successful completion of the above, necessary global python packages are installed. These are:
- virtualenv
- supervisor
These are installed using Python Package Installer, which itself installed in
the previous step(through
python-pip).
A dedicated user account is created for the app server. This helps isolate
the app server's run environment from other normal user accounts which are
typically used to login to the machine to do management tasks. This is an
automation account and disabled for interactive login. This user account name
defaults the <project> argument of the command line and is made a member of
group webapps. If this group does not exist, it will be created.
Home folder for this account will be set to /webapps/<project>_project.
Post user/group creation, the runtime evironment for the app is created. First
the python virtual environment is created. This is created in the home folder
of the dedicated automation user account. Therefore python and related
binaries will be installed to ~/bin folder and python packages will be
installed at ~/local/lib/python2.x or ~/local/lib/python3.x.
The following packages will be installed to the dedicated python virtual environment just created:
- django
- psycopg2
- gunicorn
- setproctitle
The script then generates two bash scripts that will be used to start the app server. The script is split into two files such that it can be used for interactive shell for manual interaction with the Django app server. These scripts are:
- prepare_env.sh
- gunicorn_start.sh
The former is an environment script and is to be sourced from the interactive shell whereas the latter is the master script that is used to the start Gunicorn app server (Supervisor will be configured to start the app server through this script).
So if manual interaction with the production site is desired, one may sudo into
the <project> user account and source it as:
$ sudo -u <project> -i
$ source ./prepare_env.sh
A PostgreSQL database, with the same name as <project> will be created for
use by the Django app. A dedicated PostgreSQL role with the same name will
also be created. This role is configured with a random password and this
password is stored in ~/.django_db_password.
A Nginx conf file for the requested domain will be created in
/etc/nginx/sites-available and it will be enabled by creating the necessary
soft link in /etc/nginx/sites-enabled. This configuration will be setup such
all requests to the domain (specified in the command line argument) will be
proxied to the Gunicorn server started through the script created earlier.
Supervisor installation does not create its own configuration file,
/etc/supervisord.conf. Also Supervisor installation does not create the
necessary init.d script to start it automatically and manage it interactively.
This script addresses both by creating the necessary files for this --
/etc/supervisord.conf and /etc/init.d/supervisord, a script to manage it
using the Ubuntu standard service <daemon> {start|stop} commands.
The script will also create /etc/supervisor/<project>/conf file which will
contain the configuration for the Gunicorn app server.
A placeholder Django app will be created in
/webapps/<project>_project/<project>. This placeholder app can be replaced
with the real production code.
Django applications generated using its admin command, will put the secret key
in the generated settings file. Since the Django source is shared between
different developers and since the code is typically stored in an online
repository, this key gets shared across different machines and usere. Therefore
production sites should not use this embedded key and instead should use its
own key. To automate this the script will generate a random string which is
stored in ~/.django_secret_key.
Both the secret key and the database role password (stored in
~/.django_secret_key and ~/.django_db_password) is made available to
the Django app through the environment variables SECRET_KEY and
DB_PASSWORD respectively. These variables will be set through the script
~/prepare_env.sh.
Additionally, the settings file to be used with the Django app is set
through the environment variable DJANGO_SETTINGS_MODULE.
Providing these three environment variables, allows the Django app to use different values for these three Django app parameters in production environment than what is used in development.
The script, ~/gunicorn_start.sh is the only connection between the Gunicorn
configuration and the Django app server. So as long as running this script
results in a valid app server instance, the rest of the system will remain
of the expected integrity. So if the Django app being deployed requires
different configuration, any requisite changes can be made from this file.
~/prepare_env.sh is sourced from this file to setup the environment. So this
file is also available for further tweaking to suit the deployed app's
runtime environment.