peter/sound-recommender

Sound Recommender

Basic song metadata CRUD API with playlists and recommendations based on playlists. Implemented using FastAPI and Postgres + pgvector with OpenAI embeddings for recommendations.

Limitations/Scope/Discussion

• Using OpenAPI embeddings as the recommendation engine may not be ideal (the vectors are quite large and the recommendations not super strong) but at least it's an easy way to get basic recommendation support. As an alternative to using an LLM (Large Language Model) we could have matched just on words and/or have come up with our own algorithm for sound distance that may or may not have worked better. However, such an approach would have been limited in that it does not necessarily have the semantic knowledge/relationship of words that an LLM does (i.e. which genres/artists are related etc.). An even simpler approach to recommendations would be to first return other songs by the same artist and then return other songs in the same genre. Similarly I think the OpenAI recommendations probably would have been better if they had been based on only artist (credits) and genre (excluding song title and bpm as I believe they are less relevant).
• The original Postman collection only created a single sound and since my recommendation endpoint wont return the sounds that recommendation is based on it would return an empty response and the Postman collection would fail. I fixed this by making the Postman collection create two sounds.
• I did not have time to implement any unit or system/http level tests (other than the Postman test collection)
• We do not check that sound IDs in playlists actually exist (no referential integrity there)
• I did not have time to add linting or type checking or automatic code formatting
• It probably would have been cleaner to use a database library like SQLAlchemy instead of using psycopg2 directly. However, I had some psycopg2 wrapper code from a previous project that I was able to reuse and ended up never getting around to replacing it with SQLAlchemy.
• I did not particularly take performance/scalability concerns into account when designing this system. For example I did not investigate using asynchronous FastAPI code.

How to Evaluate this System without too much Installation

Since it can be tedious/difficult to install the dependencies for this system (you need Postgres + pgvector and an OpenAI API key) you can optionally evaulate it on Heroku:

export BASE_URL=https://sound-recommender-4853b1ecaf72.herokuapp.com
curl -s $BASE_URL/sounds?query=metallica | jq
curl -s "$BASE_URL/sounds/recommended?soundId=54&limit=20" | jq 

In addition, if you remove the openai_embedding parts from the sound model (i.e. lines 16 and 51) then you should be able to use all endpoints except the recommender endpoint (you will still need Postgres but you won't need pgvector or OpenAI).

Development Setup (Local Installation)

Dependencies:

• Python (tested with 3.11.6)
• Postgres (tested with PostgreSQL 15 using Postgres.app on Mac)
• The pgvector extension for embeddings
• An OpenAI API token in the environment variable OPENAI_API_KEY

Set up virtual environment and install Python libraries:

python -m venv venv
. venv/bin/activate
pip install -r requirements.txt

If you are using Postgres.app on Mac then the pgvector installation should look something like this:

git clone --branch v0.5.1 https://github.com/pgvector/pgvector.git
cd pgvector
make
export PG_CONFIG=/Applications/Postgres.app/Contents/Versions/latest/bin/pg_config
# NOTE: need to permit terminal in Mac to modify add/delete other apps
sudo --preserve-env=PG_CONFIG make install
# You can check the .so file is installed here:
ls -l /Applications/Postgres.app/Contents/Versions/15/lib/postgresql
# You should now be able to run 'CREATE EXTENSION vector;' in your Postgres database, see below

Make sure you have Postgres installed and running locally and create the database and the pgvector extension:

createdb -U postgres sound-recommender
psql -U postgres sound-recommender -c 'CREATE EXTENSION vector;'

Run the migration to create the database tables:

bin/schema-migrate

Start the server:

bin/start-dev

Running the Postman tests (requires first installing the Postman CLI):

bin/test-postman

OpenAPI docs:

open http://localhost:8080/docs

OpenAPI spec:

open http://localhost:8080/openapi.json

Test Data for Recommendations

To test the strength of recommendations I used a Spotify dataset from Kaggle that can be ingested with this script:

bin/ingest-test-data

Invoking the API with Curl

export BASE_URL=http://localhost:8080

# Admin sounds create - Stairway to Heaven / Led Zeppelin
curl -s -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Stairway to Heaven","genres":["rock"],"credits":[{"name":"Led Zeppelin","role":"ARTIST"}],"bpm":82}]}' $BASE_URL/admin/sounds | jq

# Admin sounds create - Halo / Beyonce
curl -s -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Halo","genres":["dance pop"],"credits":[{"name":"Beyonce","role":"ARTIST"}],"bpm":80}]}' $BASE_URL/admin/sounds | jq

# Admin sounds create - Blank Space / Taylor Swift
curl -s -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Blank Space","genres":["pop"],"credits":[{"name":"Taylor Swift","role":"ARTIST"}],"bpm":96}]}' $BASE_URL/admin/sounds | jq

# Admin sounds create - Hips Don't Lie / Shakira Featuring Wyclef Jean
curl -s -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Hips Dont Lie","genres":["latin pop","reggaeton"],"credits":[{"name":"Shakira Featuring Wyclef Jean","role":"ARTIST"}],"bpm":100}]}' $BASE_URL/admin/sounds | jq

# Admin sounds create - Wrecking Ball / Miley Cyrus
curl -s -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Wrecking Ball","genres":["pop"],"credits":[{"name":"Miley Cyrus","role":"ARTIST"}],"bpm":120}]}' $BASE_URL/admin/sounds | jq

# Admin sounds create - Livin' La Vida Loca / Ricky Martin
curl -s -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Livin La Vida Loca","genres":["latin pop","dance"],"credits":[{"name":"Ricky Martin","role":"ARTIST"}],"bpm":178}]}' $BASE_URL/admin/sounds | jq

# Admin sounds create - Single Ladies (Put A Ring On It) / Beyonce
curl -s -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Single Ladies (Put A Ring On It)","genres":["dance pop","r&b"],"credits":[{"name":"Beyonce","role":"ARTIST"}],"bpm":97}]}' $BASE_URL/admin/sounds | jq

# Admin sounds create - Master of Puppets / Metallica
curl -s -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Master of Puppets","genres":["thrash metal"],"credits":[{"name":"Metallica","role":"ARTIST"}],"bpm":220}]}' $BASE_URL/admin/sounds | jq

# Get sounds
curl -s $BASE_URL/sounds/1 | jq

# List sounds
curl -s $BASE_URL/sounds | jq

# List sounds by metallica
curl -s $BASE_URL/sounds?query=metallica | jq

# Admin sounds update
curl -i -H "Content-Type: application/json" -X PUT -d '{"title":"Stairway to Hell","genres":["death metal"],"credits":[{"name":"Jakob Marklund","role":"ARTIST"}]}' $BASE_URL/admin/sounds/1

# Create playlist
curl -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Greatest of all time", "sounds":[1]}]}' $BASE_URL/playlists

# Get playlist
curl -s $BASE_URL/playlists/1 | jq

# List playlist
curl -s $BASE_URL/playlists | jq

# Update playlist
curl -i -H "Content-Type: application/json" -X PUT -d '{"title":"Greatest of all time!!!", "sounds":[1]}' $BASE_URL/playlists/1

# Get playlist
curl -s $BASE_URL/playlists/1 | jq

# Get recommendation for playlist
# Note that you get similariy scores in the list of recommendations
curl -s $BASE_URL/sounds/recommended?playlistId=1 | jq

# Get recommendation for sound
curl -s $BASE_URL/sounds/recommended?soundId=1 | jq

# Get recommendation for sound without using pgvector (does similarity sort in memory based on all sounds)
curl -s "$BASE_URL/sounds/recommended?strategy=memory&soundId=1" | jq

# Delete playlist
curl -i -X DELETE $BASE_URL/playlists/1

# Admin sounds delete
curl -i -X DELETE $BASE_URL/admin/sounds/1

Heroku Deployment

The files runtime.txt and Procfile and they contain the Python version and the start command. The following commands were used to create and deploy the app with the Heroku CLI:

# Create app
heroku apps:create sound-recommender --region eu

# Add Postgres with pgvector support
heroku addons:create heroku-postgresql:standard-0

# See status of Postgres addon
heroku addons

# Enable pgvector extension
heroku pg:psql -c 'CREATE EXTENSION vector'

# Deploy
git push heroku main

# Create the Postgres schema
heroku run bin/schema-migrate

# Check the database on Heroku with psql
heroku pg:psql

# See info about your database
heroku pg:info

# Add the OPENAI_API_KEY environment variable
heroku config:set OPENAI_API_KEY=...
heroku config

# Open the docs
heroku open

# Create a sound
export BASE_URL=https://sound-recommender-4853b1ecaf72.herokuapp.com
curl -i -H "Content-Type: application/json" -X POST -d '{"data":[{"title":"Stairway to Heaven","genres":["pop"],"credits":[{"name":"Led Zeppelin","role":"ARTIST"}]}]}' $BASE_URL/admin/sounds

# List sounds
curl -s $BASE_URL/sounds | jq

# Run postman tests
BASE_URL=https://sound-recommender-4853b1ecaf72.herokuapp.com bin/test-postman

# Ingest test data
BASE_URL=https://sound-recommender-4853b1ecaf72.herokuapp.com bin/ingest-test-data

Sound Schema

{
    "title": "New song",
    "bpm": 120,
    "genres": [
        "pop"
    ],
    "duration_in_seconds": 120,
    "credits": [
        {
            "name": "King Sis",
            "role": "VOCALIST"
        },
        {
            "name": "Ooyy",
            "role": "PRODUCER"
        }
    ]
}

Playlist Schema

{
    "title": "New playlist",
    "sounds": [
        "{{soundId}}"
    ]
}

Resources

Python libraries, tools, and examples:

• FastAPI - Getting Started
• Example FastAPI CRUD API with Postgres
• SQLite in Python
• Poetry - Dependency Management
• Build a CRUD API using FastAPI, Python, and SQLite For New Coders
• SQLAlchemy

Heroku deployment:

• gunicorn vs uvicorn
• Heroku Python getting started app
• CREATE TABLE IF NOT EXISTS error - pg_class_relname_nsp_index

Recommendations:

• pgvector
• pgvector-python
• pgvector support on Heroku
• pgvector: Fewer dimensions are better
• Chroma - Vector Database
• sqlite-vss
• Cosine Similarity vs Euclidian Distance

Song Metadata:

• Spotify - All Time Top 2000s Mega Dataset
• Most Streamed Spotify Songs 2023
• Billboard top 100 Song/Artist
• FMA: A Dataset For Music Analysis

Alternatives to FastAPI for CRUD APIs with OpenAPI support in Python:

• Python Example CRUD API (with OpenAPI/Validation)