mapit: A Python repository from annapowellsmith

MaPit
=====

pylib/mapit is a standard GeoDjango project; the rest is mostly to fit within
mySociety's standard deployment.

Installation
------------

Our version is running on Debian lenny - to install PostGIS and GeoDjango I
just followed the instructions (including creating the template) at
http://docs.djangoproject.com/en/dev/ref/contrib/gis/install/

However, not only only is the PostGIS that is installed missing SRID 900913, as
the GeoDjango docs tell you, but both SRID 27700 (British National Grid) and
SRID 29902 (Irish National Grid) are incorrect (and they're quite important
for this application!). After you've installed and got a PostGIS template, log
in to it and update the proj4text column of SRID 27700 to include
+datum=OSGB36, and update SRID 29902 to have +datum=ire65. (This may not be
necessary, depending on your version of PostGIS, but do check.)

Here are the basic instructions to install OS OpenData and NSPD Open:

1. Make sure DEBUG = False in settings.py, otherwise you'll soon run out of
memory as it tries to remember all the SQL queries made.
2. Optional: turn off escape_string_warning in Postgres (unless you want your
server to run out of disc space logging all the errors, as ours did the first
time I imported overnight and didn't realise).
3. Download Code-Point-Open-2010-05, Boundary-Line and NSPD-2010-05.csv
from http://parlvid.mysociety.org:81/os/ , and save in /data.
4. Create database tables and start the site:
./manage.py syncdb
./manage.py runserver
5. Go to the admin interface and create an inactive generation.
6. Run the following (altering file paths to data to suit):
./manage.py import_boundary_line --control=controls/2009-10 `ls /home/matthew/mapit/data/Boundary-Line/2009-10/*.shp|grep -v high_water`
./manage.py import_codepoint /home/matthew/mapit/data/Code-Point-Open-2010-05/*.csv
./manage.py find_parents
# Not importing NI here, as that only has the current boundaries.
./manage.py scilly /home/matthew/mapit/data/Code-Point-Open-2010-05/tr.csv
7. At this point, go back to the admin interface. Make the generation active,
and add another inactive generation. Then run:
./manage.py import_boundary_line --control=controls/2010-05 `ls /home/matthew/mapit/data/Boundary-Line/2010-05/*.shp|grep -v high_water`
# import_codepoint not needed as it's the same and there's no P-in-P tests!
./manage.py find_parents
./manage.py scilly /home/matthew/mapit/data/Code-Point-Open-2010-05/tr.csv # I doubt the boundaries change! But updates the generation.
./manage.py import_nspd /home/matthew/mapit/data/NSPD-2010-05.csv # This is now split into two scripts, see below.
7. Go to the admin interface, and make this generation active.

If you want to add all historic data, there are a number of extra complications
to maintain IDs, names and so on from the old version of this service....

At this point, I then matched up all IDs and names using the scripts in bin,
manually inserted some generation 10 areas (in data) for FixMyStreet, and
manually added our test/fake areas that used to be in code but can now
happily sit in the database along with everything else.

Obviously if you don't care about older Boundary-Line, you can just do the one
Boundary-Line, Code-Point, NSPD imports. [Make these default instructions.]

Notes on future releases
------------------------

When a new Code-Point is released, you should just be able to run import_codepoint
and scilly; when new NSPD is out, import_nspd_ni if it's only postcodes that
have changed, or import_nspd_ni_areas first if boundary changes too (this is
incomplete, it doesn't use a control file like import_boundary_line does);
when new Boundary-Line, import_boundary_line and find_parents.

Though I haven't had to do this myself yet, so there's probably some updating as
opposed to creation niggles to find.

Improvements / patches
----------------------

Are welcome :)

ATB,
Matthew

annapowellsmith/mapit