/affinder-search

affinder search that assists in stretch search based on learned meta-data on yelp categories

Primary LanguagePython

Affinder Search

Installing and Running the App

To install, we need to (1) download the tfidf matrix data; (2) setup the needed packages and dependencies; (3) convert tfidf data into a memory friendly format.

  1. ./download-data.sh Check the comments on this file, to determine which data it downloads
  2. pipenv install
  3. python write_tfidf_from_numpy_to_hdf5.py

To run the microservice that powers the unlimited vocabulary search for affinder, we will (1) activate the right python environment and (2) run the flask app

  1. pipenv shell
  2. You can choose to run a memory-efficient implementation python main.py or one that might be a tad faster since data is loaded in memory python search_yelp_places_server.py

This is also encapsulated in the ./run.sh command.

Serving the App in Production

See this tutorial on serving a flask app with digital ocean

Open the port

sudo ufw allow 5000

When creating a new Digital Ocean server you'll might have to modify the affinder-search.service and affinder-search-ngnix files.

  1. Make sure the public IP is updated in the affinder-search-ngnix. You can find public IP via the console.
  2. Make sure affinder-search.service is pointing to the right pipenv environment path. After activating the pipenv, you can get that using echo $VIRTUAL_ENV

Should copy the system service file to the right place

sudo cp affinder-search.service /etc/systemd/system/

Should copy the system service timer that restarts the affinder-search service periodically

sudo cp myPeriodicRestarter.timer /etc/systemd/system/

Should copy the nginx file to the right place on the server, symlink it to the enabled sites, and restart ngnix to read the new configuration.

sudo cp affinder-search-nginx /etc/nginx/sites-available/affinder-search
sudo ln -s /etc/nginx/sites-available/myproject /etc/nginx/sites-enabled
sudo systemctl restart nginx

Contributing or Developing

Files of Interest

Production Code

search_yelp_places_server.py
search_yelp_places.py
yelp_academic_etl.py

Production Serving (e.g., hosting on a digital ocean droplet)

wsgi.py
affinder-search.ini
affinder-search.service

Development Code (e.g., creating the unlimited vocabulary from yelp review metadata)

yelp_academic_etl_training.py

Experimental Code

yelp_academic_etl_advanced.py
explore_embedding.py
affordance_language.py
affordance_language_advanced.py
lego_word_associations.py
oxford.py