A quick attempt at gathering music listening insights similar to Spotify Wrapped
Yesterday the Spotify Wrapped came out and I was very excited to check it!
However I wanted to have a few extra insights, so I requested my data export.
This export can take many days, or even weeks, and I was impatient, so I looked for data somwehere else in the meantime.
I have Spotify connected with LastFM, so every song I listen to is logged there. With this website I can export my LastFM data in JSON or CSV, so I just did that!
Python and Pandas sound like a better candidate for analysing this data, but I wanted to give a try to Kotlin dataframes and their Jupyter notebook integration.
So I created this notebook, which has just some basic aggregations on the music data for curiosity.
Then I manually made some poorly designed graphics mimicking the actual Wrapped.
Sorry to any designers looking at these!
Afterwards I compared the insights from the Spotify Wrapped with the LastFM data, but there's a few discrepancies, I suspect sometimes the Spotify-LastFM integration stops working and it skips some scrobbles, and also the cutoff date for the Spotify Wrapped is unknown so the best I can do is an estimate.
I suspect Spotify doesn't include December in the Wrapped because it'd be full of christmas music!
The data from LastFM is very limited, just names of songs, albums, artists, and timestamps, so only basic data aggregations can be performed.
With this data I tried extracting the following stuff for 2022 and 2023 to compare:
- A chart with the count of songs played every day for each year
- A chart with the count of songs played every hour for each year
- Top songs, albums, and artists in 2023, and their change in ranking compared to 2022
- Top songs, albums, and artists listened to repeatedly in a same day
- Top songs, albums, and artists in 2023 which I hadn't listened to in 2022
- Top songs, albums, and artists in 2022 which I didn't listen again in 2023
- The amount of "new" artists listened this year (listened to in 2023 but not 2022)
- The amount of unique songs I've listened to for each year
- The amount of unique artists I've listened to for each year
- The total amount of songs (including repeats) I've listened to for each year
With the full Spotify export I hope I'm also able to get data like:
- My playlists' content, to join it with streaming data to see how often, how long, and at what time I listen to my almost 200 playlists
- The amount time from each song actually listened to, to compute total listening time for songs, albums, artists, and playlists
I wish there was more data to play with but this should be enough for now!
This initial version expects that you have a LastFM account where you have scrobbled your songs' data. A future revision will include these operations with the Spotify exported data.
- Get your LastFM data export from https://mainstream.ghan.nl/export.html
- Make sure you either have Kotlin Notebook plugin for IDEA Ultimate or install Kotlin support for Jupyter
- IDEA Ultimate Kotlin Notebook plugin
- If you have Conda use:
$ conda install -c jetbrains kotlin-jupyter-kernel
- Open notebook and make sure the filename matches your export.
- LetsPlot Kotlin https://lets-plot.org/kotlin/
- Kotlin DataFrame documentation https://kotlin.github.io/dataframe/gettingstarted.html
- Kotlin libraries for Data Science https://kotlinlang.org/docs/data-science-overview.html
- Kotlin Data Frame Overview by Anatoly Nitikin https://www.youtube.com/watch?v=qGou8F2asNw
- Replacing SQL with Kotlin's 'dataframe' on the Las Vegas Strip by Andrew Goldberg https://www.youtube.com/watch?v=sDZWiu9nnuU
I hope this doesn't hurt the eyes of people who actually designs these things:
