mapmill: An HTML repository from jywarren

README¶ ↑

MapMill 2 is a complete rewrite of MapMill – an application developed during the BP oil spill, which enables large groups of people to sort through and rank very large image collections (1000s of images).

The new version now allows any user with a PublicLab.org user account to create new image sets and upload new images. To make this affordable to run, all image processing is performed on the client side and images are stored in AWS S3 cloud storage.

MapMill 2 uses the jquery-file-upload plugin to handle file uploads. See <a href=“blueimp.github.io/jQuery-File-Upload/”>blueimp.github.io/jQuery-File-Upload/</a> It also uses blueimp’s gallery for as image gallery. See <a href=“blueimp.github.io/Gallery/”>blueimp.github.io/Gallery/</a>

Thanks to Fabio Barone (publiclab.org/profile/faboolous) and Melissa Nunes (publiclab.org/profile/MelissaN) for their work on this new version of MapMill.

Installation¶ ↑

Run ‘bundle install` to install the required gems. (See ’Windows Note’ if installing on Windows.) Run ‘bower install` to install the required static dependencies. The current version is configured with a sqlite database. Installing rvm for ruby management is recommended.

MapMill 2 uses:

Ruby: 2.1.5
Rails: 4.1.8
- Mysql 2 (optional)

The default ‘config/database.yml.sqlite3` uses SQLite. Copy this to `config/database.yml` (and make changes to use MySQL if desired).

Create a database first (rake db:create), create a user and grant privileges to that user. Then run rake db:migrate to run all needed migrations.

In order to successfully configure AWS S3 stuff, edit the file ‘config/app_environment_vars.rb` file in the config directory, and add bucket name, AWS access key and secret as needed. A .sample file is provided in the same directory.

DO NOT CHECK IN THAT FILE INTO VERSION CONTROL, AS THIS WOULD SERIOUSLY COMPROMISE SECURITY, MAKING YOUR KEYS ACCESSIBLE TO THE WORLD.

‘config/initializers/aws.rb` contains needed initialization for AWS, which loads its values from the `app_environment_vars.rb`, but you should not need to edit this file.

S3 CORS Configuration Sample¶ ↑

If you are using AWS S3, you can use this CORS configuration:

<CORSRule>
    <AllowedOrigin>http://localhost:3000</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>x-csrf-token</AllowedHeader>
    <AllowedHeader>x-requested-with</AllowedHeader>
    <AllowedHeader>x-amz-request-id</AllowedHeader>
    <AllowedHeader>content-type</AllowedHeader>
    <AllowedHeader>content-range</AllowedHeader>
    <AllowedHeader>content-disposition</AllowedHeader>
    <AllowedHeader>content-description</AllowedHeader>
</CORSRule>

</CORSConfiguration>

Running with local S3¶ ↑

It is possible to run a local S3 (such as fakes3) so you do not need to connect to AWS. This can be accomlished by adding the following lines to your ‘config/app_environment_vars.rb` file (example values provided):

ENV['S3_DEV_PROTOCOL']='http'
ENV['S3_DEV_HOST']='localhost'
ENV['S3_DEV_PORT']='10001'

You will also need to make ‘[bucketname].localhost’ point to ‘127.0.0.1’ (by editing yout hosts file).

To install a fakes3 with updated CORS handling, run:

git clone https://github.com/scoopergit/fake-s3.git
cd fake-s3
gem build fakes3.gemspec
gem install fakes3-0.1.7.gem

Then run the fakes3 command:

fakes3 -r [root] -p [port] -o [allowed origin; such as: http://localhost:3000]

If you still encounter problems with cross-domain errors, you can disable these checks in your browser. If you are using Chrome, this can be done by running it with ‘google-chrome –disable-web-security`.

Usage¶ ↑

The app only allows image upload after login. Login is only allowed with PublicLab’s OpenID infrastructure.

The front page contains the form for a site creation. We need to create a temporary object (Sitetmp) if the user is not logged in, in order to redirect to the OpenID provider, otherwise the form info gets lost when returning from the OpenID login process. A nonce is generated and sent as GET parameter for OpenID login. If Login went successful, the nonce is passed again and the real site object is created.

Users can upload pictures, and rate its quality. Images are grouped in sets. Sets can be viewed and browsed. A map view is provided per set. If a user has rated an image, further voting is disabled. Vote tracking is done via cookie.

The upload works like this:

First images need to be selected via upload button
…or, drag and drop
Images which really will be uploaded need to be selected first with the mouse
Click the upload button
After images have been successfully uploaded, a thumbnail is automatically generated
As this is all CLIENT SIDE, this implies:
- The client sends the image to AWS S3
- If this went OK, a handle is returned. The client **saves the image on the server**
- The server returns an id for the new image to the client; the client generates a thumbnail and **saves this thumbnail on AWS S3**
- If this all went ok, the client associates the new thumbnail with the id of the image on the server
- If that went ok, the process is finished

Contributors¶ ↑

Join the Public Lab developers mailing list “plots-dev” at publiclab.org/wiki/developers

We are hoping to add image sorting and analysis scripts to help users select in-focus images (perhaps based on file size).