/The-Django-gunicorn-fabfile-project

This is a bare Django project that includes a Fabric file, commonly known as a fabfile, to setup an Ubuntu 11.10 server with Django, Nginx, gunicorn and upstart.

Primary LanguagePython

The Django gunicorn fabfile project
======================================

This is a bare Django project that includes a Fabric file, commonly known as a fabfile, to setup an Ubuntu 11.10 server with the following environment:

- Nginx to serve static files,
- gunicorn to serve a Django project,
- Python 2.7.2, which is the version in Ubuntu 11.10,
- Django 1.4,
- PostgreSQL 9.1 and pyscopg2 2.4.1 (which is compatible with the test framework in Django 1.3.1),
- git,
- upstart to easily start and stop the Django project.

The fabfile can create a production ready environment and optionally an staging environment, which is highly recommended.

All the project files are cloned from a git repository to a temporal directory, which can be used by staging, and rsynced to a production directory.

The fabfile has been tested with Fabric >= 1.3.1.

Prerequisites
===============

An Ubuntu server with the following packages installed via apt-get:

git-core
python-pip
python-dev

And Fabric installed via pip.

You can install these using other methods but apt-get and pip were the ones used for my tests and I think are the easiest and fastest ones.

SSH authentication setup for any remote hosts that you will need to access for obtaining your code. This will help to avoid typing passwords.

If you are using github follow the instructions on http://help.github.com/mac-set-up-git/

Setup a directory with your settings to be copied over, I usually create a directory like ~/djsettings and put myproject_local_settings.py in there with the correct settings for your environments.

How to use
===============

1. This step is optional. If you still haven't created a user to run the project you can start with an existing user to create one.
$ fab -H existing_user@host add_user:user
That will create user with a random password and sudo permissions.

2. Fill configuration details in settings.py.

3. Run setup to install the server applications, create virtualenvs, install basic Python packages and configuration files for one or more environments.
Start creating a development environment on the development box.
Then create a staging environment, ideally on one of the production boxes, as it will be used to get code from repositories and then rsync to production.
$ fab -H user@host setup:production,staging,development,mirror=y

4. Install or update project and apps for one environment.
$ fab -H user@host update_site:env=production,update_settings=y,upgrade_apps=y

5. To start, stop or restart the site on one environment.
$ fab -H user@host start_site:env=production
$ fab -H user@host stop_site:env=production
$ fab -H user@host restart_site:env=production

6. Work on the development environment and use this to commit from time to time.

$ fab -H user@host commit:env=development,message='commit message and escaping comma\, this way',push=n,test=y

Parameters:
env: 'production', 'staging', 'development'.
mirror: 'y', 'n'. Default: 'n'.

Development can be accessed at http://PROJECT_DOMAIN_DEVELOPMENT:development_port
Staging can be accessed at http://PROJECT_DOMAIN_STAGING:staging_port
Production can be accessed at http://PROJECT_DOMAIN:port

The two hosts interacting in the basic usage scenario are the deployer, which is the GNU/Linux workstation on which you will run the fabfile, and the target, which is the GNU/Linux server where the Django project will be setup when the fabfile is run. There are other hosts involved, like the one hosting the git repository with your code, but the deployer and the target are the ones where you have full control, including ssh root access.

This fabfile has been developed and tested using Ubuntu 11.10 and the standard software packages that are available via the Ubuntu repositories and PyPi, but it could be adapted to run with other GNU/Linux distributions.

To start, follow these steps:

1. Download or git clone this project to the deployer,

2. Install Fabric, ideally while in a virtualenv, in the deployer  with: 
$ pip install Fabric

3. Make sure the target is running Ubuntu 11.10 (unless you want to modify the fabfile to make it work with other versions or distributions), openssh-server and has a user with sudo permission. Ideally you should have ssh key authentication working between the deployer and the target, and between the target and the git host.

4. Now you have two choices, copy some files from this project to your own Django project and make some changes or start building your own Django project on top of this template. Either way will take you to the same place.

4a. To copy this project to your own Django project, you just need to copy the file fabfile.py and the directory deploy, both in the root of this project, to the root of your project and then add the following at the end of your project's settings.py:

4b. To start building your project on top of this one change the SECRET_KEY in settings.py to make it unique for your project and proceed to step 5.

5. Copy local_settings_template.py from the root of this project to some directory in the deployer and adapt for your project, feel free to change the name of the file as the full path will be later used in the PROJECT_STAGING_SETTINGS_PATH constant in the fabfile.py configuration section.

There are some important points to notice in the local_settings_template.py. DEBUG is initially set to True and conditionals based on the constant are used to make changes for either staging or production. I assume that DEBUG will be True when running in staging and False when running in production. In local_settings_template.py there is a commented piece of code showing how to add paths to the PYTHONPATH, for adding applications to your project, based on the value of DEBUG. A similar logic is used for setting up staging and production databases.

Modify INSTALLED_APPS as needed. For now the fabfile doesn't support running Django's syncdb and supposes you have a database already setup for the apps that you need.

6. Change to your project's directory and add these lines at the bottom of your settings.py file:

# Use local_settings.py to override settings.
# This file should be outside of control version. 
# Copy local_settings_template.py as a starting point.
try:
    from local_settings import *
except ImportError:
    pass
    
Then modify your configuration parameters on fabconfig.py, which is found at the root level of your project, together with fabfile.py.

How to run the fabfile
------------------------

1. This step is optional. If you still haven't created a user to run the project you can start with an existing user to create one.
$ fab -H existing_user@host add_user:user
That will create user with a random password and sudo permissions.

2. Fill configuration details in settings.py.

3. Run setup to install the server applications, create virtualenvs, install basic Python packages and configuration files for one or more environments.
Start creating a development environment on the development box.
Then create a staging environment, ideally on one of the production boxes, as it will be used to get code from repositories and then rsync to production.
$ fab -H user@host setup:production,staging,development,mirror=y

4. Install or update project and apps for one environment.
$ fab -H user@host update_site:env=production,update_settings=y,upgrade_apps=y

5. To start, stop or restart the site on one environment.
$ fab -H user@host start_site:env=production
$ fab -H user@host stop_site:env=production
$ fab -H user@host restart_site:env=production

6. Work on the development environment and use this to commit from time to time.

$ fab -H user@host commit:env=development,message='commit message and escaping comma\, this way',push=n,test=y

Parameters:
env: 'production', 'staging', 'development'.
mirror: 'y', 'n'. Default: 'n'.

Development can be accessed at http://PROJECT_DOMAIN_DEVELOPMENT:development_port
Staging can be accessed at http://PROJECT_DOMAIN_STAGING:staging_port
Production can be accessed at http://PROJECT_DOMAIN:port

Keep an eye on the prompt for the sudo password and, possibly, the git repository password. These should appear right after you run each command so you can enter the password and go grab a coffee while the process works. The first command, setup, usually takes longer (I counted 15 minutes in my test servers); the start_project command depends on how big your git repository, for a medium project is very quick.

That should be all.

TODO
==========

- Distribute via PyPi.
- Make the project generic to work with other versions of Ubuntu and Python.
- Add support for other version control systems.
- Support for Django test framework.
- setup git user and email with git config before starting with commit/push, using
  git config --global user.name "Your Name"
  git config --global user.email you@example.com
- Support for easier commit and push in the deploy function.
- Support for syncdb and database distribution. Is it needed or better left to do manually?
- Memcached setup.
- Varnish setup.

Latest changes
===============

Modified the project directory layout to work with Django 1.4.

Common erorrs
===============