MinnPost Crime

An application that explores crime in Minnesota.

Development and Install

Install Git. On Mac with Homebrew: brew install git
Install NodeJS. On Mac with Homebrew: brew install node
Install Grunt: npm install -g grunt-cli
Install Bower: npm install -g bower
Install Sass: gem install sass
Because Leaflet comes unbuilt, we need to build it with Jake: npm install -g jake

Clone the repository with something like: git clone git@github.com:MinnPost/minnpost-crime.git
Go into the code directory: cd minnpost-crime

npm install
bower install
Because Leaflet comes unbuilt, we need to build it: cd bower_components/leaflet/ && npm install && jake; cd -;

Us grunt to watch files for changes and server the files with: grunt server-watch
Go to localhost:8080/index.html in your browser.

To turn MN Compass neighborhood profile data to JSON: node data-processing/mncompass-2010-xlsx-json.js
Create final Minneapolis neighborhood data JSON:
1. Run: node data-processing/minneapolis-neighborhoods.js

Scraper of Minneapolis Monthly Reports. Original reports can be found on the MPD stats page. These reports are very similar to the FBI Uniform Crime Reports, except that they use the date of the offense (not reported date) and count each offense of the report (not the worse offense) see details.
- Due to the fact that the MPD has decided not to release Excel files anymore. This means that PDF's are manually parsed with Tabula and put into the data/crime/mpls-monthly-reports-manual-folder. These are then uploaded to S3 and used in the scraper.
- Run node data-processing/download-mpls-monthly-reports.js to download the existing Excel files locally. Then do grunt mp-source-data to upload these files to S3.

MN Compass has a zipped Excel file of demographic data for Twin Cities neighborhoods that is downloaded here for processing. This is mostly 2010 population data with 2005-2009 employment and related data.
The City of Minneapolis has a 2000 race and ethnicity report. A scraper of this data has been made. Download with: wget -O data/demographics/2000/minneapolis-demographics-2000.json "https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=json&name=minneapolis_neighborhood_census_population_2000&query=select%20*%20from%20%60swdata%60"
- Note that in 2005, the Phillips neighborhood was split up into Ventura Village, Phillips West, Easy Phillips, and Midtown Phillips (see neighborhood profile). The 2000 population is taken from Wikipedia:

Definitive list of Minneapolis Neighborhoods last updated 2006-01-01. A scraper of this data has been made. Downloaded locally (data/neighborhoods/minneapolis/minneapolis-neighborhoods-2012.json). These contain keys that should be used throughout the application.
- Download JSON with this command, though this data should not change anytime soon: wget -O data/neighborhoods/minneapolis/minneapolis-neighborhoods-2012.json "https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=json&name=minneapolis_neighborhoods&query=select%20*%20from%20%60swdata%60"
- Download CSV: wget -O data/neighborhoods/minneapolis/minneapolis-neighborhoods-2012.csv "https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=minneapolis_neighborhoods&query=select%20*%20from%20%60swdata%60"
Downloaded shapefile of the neighborhoods was provided by the City of Minneapolis: wget -O data/neighborhoods/minneapolis/minneapolis-neighborhoods.shp.zip "http://www.minneapolismn.gov/www/groups/public/@bis/documents/webcontent/wcms1p-106980.zip". This was extracted with: cd data/neighborhoods/minneapolis/ && unzip minneapolis-neighborhoods.shpfile.zip -d minneapolis-neighborhoods.shpfile; cd -;
- This has been converted to a GeojSON file with: ogr2ogr -f "GeoJSON" data/neighborhoods/minneapolis/minneapolis-neighborhoods.geo.json data/neighborhoods/minneapolis/minneapolis-neighborhoods.shpfile/NEIGHBORHOODS.shp -t_srs "EPSG:4326"