CosmiQ/cw-eval

Determine what recipes should be included in the cookbook

Closed this issue · 4 comments

Steps:

  • Come up with a list of desired recipes for cookbook
  • Define how they'll be presented
  • Create a sample and run by the lab for review
  • run the list by the rest of the lab for review
  • Write issues for each individual recipe
  • Write issue for creating and linking to cookbook section in rtfd

List of recipes:

  1. Eval one chip (.geojson against .geojson)
  2. Eval one chip (.csv against .csv)
  3. Eval an entire competition (proposal uber csv against ground truth uber-csv)
  4. CLI spacenet_eval command to run eval on entire challenge

Presentation:

Use nbsphinx to render notebooks running examples.

Recipes to include:

  • .geojson scoring
  • Full challenge scoring: either using API or CLI
  • APLS

Resolved with #46.