CRF Ingredient Phrase Tagger

Fork Notes

This is a fork of the original NY Times ingredient-phrase-tagger. This fork is maintained by Michael Lynch

This fork maintains the design of the original ingredient-phrase-tagger, but adds bugfixes and additional features to aid in future development:

Adds a Docker image for easy deployment.
Adds a continuous integration build on every check-in.
Adds unit tests.
Adds end-to-end tests.
Enforces rules for source formatting and linting.

These improvements were described in a series of blog posts on mtlynch.io:

Zestful

Zestful is a managed ingredient-parsing service based on this library. It has higher accuracy and more frequent updates:

https://zestfuldata.com

Overview

This repo contains scripts to extract the Quantity, Unit, Name, and Comments from unstructured ingredient phrases. We use it on Cooking to format incoming recipes. Given the following input:

1 pound carrots, young ones if possible
Kosher salt, to taste
2 tablespoons sherry vinegar
2 tablespoons honey
2 tablespoons extra-virgin olive oil
1 medium-size shallot, peeled and finely diced
1/2 teaspoon fresh thyme leaves, finely chopped
Black pepper, to taste

Our tool produces something like:

{
    "qty":     "1",
    "unit":    "pound"
    "name":    "carrots",
    "other":   ",",
    "comment": "young ones if possible",
    "input":   "1 pound carrots, young ones if possible",
    "display": "<span class='qty'>1</span><span class='unit'>pound</span><span class='name'>carrots</span><span class='other'>,</span><span class='comment'>young ones if possible</span>",
}

We use a conditional random field model (CRF) to extract tags from labelled training data, which was tagged by human news assistants. We wrote about our approach on the New York Times Open blog. More information about CRFs can be found here.

On a 2012 Macbook Pro, training the model takes roughly 30 minutes for 130k examples using the CRF++ library.

Development

On OSX:

brew install crf++
python3 setup.py install

Docker:

docker pull mtlynch/ingredient-phrase-tagger

Quick Start

docker run -it mtlynch/ingredient-phrase-tagger bash

# Train a new model
MODEL_DIR=$(mktemp -d)
bin/train-prod-model "$MODEL_DIR"
MODEL_FILE=$(find $MODEL_DIR -name '*.crfmodel')

# Parse some ingredients
echo '
2 tablespoons honey
1/2 cup flour
Black pepper, to taste' | bin/parse-ingredients.py --model-file $MODEL_FILE

[
  {
    "display": "<span class='qty'>2</span><span class='unit'>tablespoons</span><span class='name'>honey</span>",
    "input": "2 tablespoons honey",
    "name": "honey",
    "qty": "2",
    "unit": "tablespoon"
  },
  {
    "display": "<span class='qty'>1/2</span><span class='unit'>cup</span><span class='name'>flour</span>",
    "input": "1/2 cup flour",
    "name": "flour",
    "qty": "1/2",
    "unit": "cup"
  },
  {
    "comment": "to taste",
    "display": "<span class='name'>Black pepper</span><span class='other'>,</span><span class='comment'>to taste</span>",
    "input": "Black pepper, to taste",
    "name": "Black pepper",
    "other": ","
  }
]

Authors

License

Apache 2.0.

tmb5cg/ingredient-phrase-tagger