Picify is a Flask web application that converts your photos into Spotify playlists that can be saved for later listening, providing a uniquely personal way to explore new music. Try it here.
The main workflow for the app is as follows:
- The user uploads a photo to the Picify Flask server.
- The image is passed onto the Google Cloud Vision API, where labels and entities are predicted/extracted. This information then gets passed back to the Flask server.
- The labels and entities are filtered by a dynamic confidence threshold which is iteratively lowered until a large enough set of descriptors for the image can be formed.
- Each of the descriptors in the above set are then expanded into associated "moods" using the Datamuse API.
- All of the descriptors and associated moods are filtered against a whitelist of "musically-relevant" terms compiled from sources such as AllMusic and Every Noise at Once, excepting descriptors with extremely high confidence (for example, for a picture of Skrillex this might be "Skrillex").
- Finally, the processed words are matched against existing Spotify playlists, which are sampled to form the final playlist.
