MAL Heatmap

https://malheatmap.com

MAL Heatmap is a tool for tracking your anime/manga consumption based on your myanimelist.net recent history.
This app crawls the recent user history daily and generates a full visualization of all activities like Github contributions calendar.

It also generates an image of your activities calendar that you can embed into your myanimelist.net profile or forum signature extending your MAL experience.

Built with

Motivation

I've made this experiment to play around with some cool Rails features, like ActiveStorage and ActionCable. Also, I want to try web scrapping with ruby and learn a little bit more about it.

Also, this allowed me to test some gems, like ViewComponent.

How it works

The web app just displays the processed data and a set of jobs do the hard and dirty work of crawling, process, save data.

So, when the user subscribes to its profile to be tracked, a job will be triggered to check if the user exists on MAL and then crawl and process the data. While that, the UI will be connected to a channel awaiting a response from the job, once the job is completed, the user will be redirected to his page. After subscribing, a recurring job will repeat the crawling process daily to keep the user's data updated.

Demo

Developing

Requirements:

PostgreSQL 13.3+
Redis 6.2+
Ruby 3.0.2
Node 14.17.0
ImageMagick

Since this app is a classical Rails project, there is nothing special here, to setup project:

bin/setup

This script will create the database and install all dependencies.

To run the app:

bin/rails server

To run unit tests:

bin/rails test

Integration tests:

bin/rails test:system

To run all linters and test suites:

bin/qa

Deployment

This app is deployed with Dokku hosted on a VPS.

Auxiliary tools:

Rollbar for error tracking and uptime monitoring
Matomo for analytics
Amazon S3 as storage service
Sidekiq as background job adapter