Web-based Geo mapping

Use of species distribution maps in a web application. Currently prototype development using web-based mapping packages and D3js visualization and interactivity library.

IMG_SITE_OLIVES branch ready for development of PDFs as maps


1 Overview

1.1 To Do (9 Aug, 2016):

Make species pages by the following steps:

  • Add "clade, iucn, coverage, uncertain" columns to "SPECIES SPECIES pearldata.csv" file
  • Create routes/SPECIES_SPECIES.js
  • Copy views/SAMPLE_PAGE.ejs to views/SPECIES_SPECIES.ejs
  • Make future map in public/pdf/web/species-pages/SPECIES_SPECIES/future-maps directory
  • Move future map into public/pdf/web/species-pages/SPECIES_SPECIES/future-maps directory
  • Add source of where data was collected per species (into routes/SPECIES_SPECIES.js)

Steps:

# Step 1: in excel
Copy 'clade, iucn, coverage, uncertain, sources' columns from "public/csv/data_summary.csv" into the correct species row in "public/pdf/web/species-pages/NEXT_SPECIES/NEXT SPECIES pearldata.csv"

####################################################################
####################################################################
####################################################################

# Step 2: in command line (Terminal)
$ cp views/SAMPLE_PAGE.ejs views/NEXT_SPECIES.ejs
$ cp routes/SAMPLE_ROUTE.js routes/NEXT_SPECIES.js

####################################################################
####################################################################
####################################################################

# Step 3: go into "routes/NEXT_SPECIES.js" file (Javascript)
Search (use the find command) for "SPECIES_SPECIES" and replace with "NEXT_SPECIES"
and then search for "SPECIES SPECIES" and replace with "NEXT SPECIES" but then copy over the lowercase "Next species" into the 'title' attribute in the JS page (NOTE: make sure the 's' is lowercase - second name!)

####################################################################
####################################################################
####################################################################

# Step 4: make future.pdf (Terminal - but uses Python)
Go to "public/pdf/web/species-pages" in Terminal
> python mergePDF.py -p NEXT_SPECIES/future-maps/

This makes a pdf up at the top directory (where you currently are) so you need to move it into the "NEXT_SPECIES/future-maps" dir.

> mv future.pdf NEXT_SPECIES/future-maps/

####################################################################
####################################################################
####################################################################

# Step 5: Add to app.js (Javascript - top project directory)
var NEXT_SPECIES = ('./routes/NEXT_SPECIES')

...

app.use('/NEXT_SPECIES', NEXT_SPECIES)

####################################################################
########## THE END
####################################################################

2 Project structure and design:

2.1 Frontend (HTML + D3js)

  • The views/ directory contains html files for each page in our web site. These files have the extension .ejs because we are using a template with NodeJS on the server-side. Treat any .ejs file like it is a normal html file.

  • The public directory contains files for the html files in views/. These include css, js, fonts, images that live in their respective sub-directories. For example, the web page foo.ejs (that would live in views/foo.ejs) may have a styling file that lives in public/css/foo.css.

2.2 Backend (NodeJS + express)

  • The bin/ directory contains file wwww that starts up the server by executing the app.js file.

  • The node_modules/ directory contains all of the dependencies and packages being used for this project - this is organized / structured through the package.json file.

  • The routes directory contains the "middleware" between the server and the frontend web pages. Remember, our server is running NodeJS and our frontend is html + D3js. Each html page in views/ has a .js file in routes/ that controls executing that file and sending data to it (if data needs to be sent to it - we determine that).


3 Run this project

This project is using a frontend Bootstrap skeleton with D3js for data visualization - i.e. mapping. The project is running on an express engine with NodeJS in the backend.

NOTE: You need to download NodeJS with some packages to run this project!

3.1 Dependencies:

We are going to use XCODE and Homebrew at various times. Xcode is Apple's (OS X) development software package thingie that you need to do a lot of programming / development on an Apple (OS X) machine. It is extremely silly that this is a thing you need to download...but that is just my opinion of a very huge and successful company strategy.

We are also going to use Homebrew. Homebrew is a package manager. It makes installing things very, very easy in Terminal. For example: below I tell you how to install NodeJS through their website (like a normal software download)...but you could also download it through brew with the following command in Terminal: $ brew install node

NOTE: XCODE & Homebrew may or may not actually be needed but will make life better for work on this and future projects!...but I think we do need Xcode.

Here is a tutorial / webpage showing how to download all of this.


  • download NodeJS
  • NodeJS will also download the package manager npm
  • open Terminal
  • go to this dir in Terminal: pearl-geo$
  • check if NodeJS is installed: pearl-geo$ node -v
  • check if npm is installed: pearl-geo$ npm -v
  • NodeJS will show something like: v4.4.2
  • Npm will show something like: 2.15.0

NOTE: If NodeJS or npm not installed...

To run this project:

  • open Terminal
  • go to this dir in Terminal: pearl-geo$
  • start the NodeJS + Express server with this command: pearl-geo$ npm start
  • this will start a local server (on your machine) served at: localhost:5050
  • open any browser (Firefox, Chrome, Safari, etc.) at the local server: localhost:5050
  • ...in other words...just type in localhost:5050 in your browser like it is a website
  • model will be running - info + errors will appear in Terminal where you started with npm start

4 Resources

4.1 Code References

4.1.1 Git How To (initialization):

Refer to this for anything and everything related to command-line + git

Install Shell Commands in Atom:

    # Click on the "Atom" icon next to the "Apple" symbol in your menu bar...
    # Click on "Install Shell Commands"

Clone the repo (only for first time):

    git clone https://github.com/Thru-Echoes/pearl-geo.git

Go into repo dir:

    cd pearl-geo

Check status of repo:

    git status

What branch does it say you are on? You should be in your development branch! If not...

Go into development branch:

    git checkout gio-dev

Make changes - do whatever - time to add + commit + push:

    git add FILE-YOU-CHANGED.js
    git commit -m "Describe what you did"
    git push origin gio-dev

Always push changes into your own development branch!


4.1.2 HTML / CSS / JS:

Make font bold:

    <strong>Bold font</strong>
    <p><strong>Bold first four words</strong> in the paragraph.</p>

Make font italicized:

    *this is italic*

    And also this could be bold:

    **wow another way to do bold**

And you can make code: here is code


Add this is the text to add to the home and / or about page:

    PEARL - the first global Parasite Extinction Assessment & Red List - compiles the preliminary work of an international team of researchers working to produce a mainstream data portal for parasite conservation. An estimated XX% of endoparasites are threatened globally with coextinction from their hosts, and an additional XX% are likely to be driven to extinction by climate change. PEARL compiles the conservation status, distribution and host associations of nearly 500 parasites from a pilot risk assessment study, and makes that data freely available to researchers, managers and the public at large. Preliminary expansions to the dataset are scheduled for 2018.

And here are some thoughts from Colin and me:

    Colin:
    Echinococcus granulosus will definitely be one of the things in the carousel and we'll figure out the rest as we go... general overview showing the team would be cool with a separate team bio giving the history of PERP and detailed bios on everyone, search bar would be awesome, all that sounds good

    Attached is a GIF that I think would be really cool to have on the home page, showing the history of the dataset's creation (I saved these as we georeferenced!). It's not the coolest but it shows how long it took to put together?

    (GIF is in public/img)

    Oliver:
    Leaflet (JS mobile-friendly mapping library) + D3js!

    Leaflet: http://leafletjs.com/ Super lightweight - open source - mobile friendly - REALLY pretty (good UI / UX) - works well with my love D3js

    Leaflet + D3js: https://bost.ocks.org/mike/leaflet/

    Slightly more involved example: http://bl.ocks.org/d3noob/9267535

Note to Gio: I added the about and team pages but you have to fill them (i.e. they are blank).

4.2 References