IsThisYiff?! is a guessing game where you are shown a piece of artwork from e621.net cropped down to just a face and asked the question: Is This Yiff?
I needed a data set of anthropomorphic artwork, where each had a characters' face identified so that the image could be cropped-down.
To produce the dataset, I first had to train a model to detect faces in anthropomorphic artwork. There were several promising-looking pre-trained models but I'm no expert in AI and struggled to get any working well. I ended up training an AWS Rekognition Custom Labels model to do this. I took 250 JPG/PNG images with the s (Safe) rating on e621 matching this criteria:
solo anthro simple_background -human -absurd_res -monochrome -real -comic
Then a further 250 images with the e (Explicit) rating with the same criteria. I also took 250 random images from ThisFursonaDoesNotExist. After curating the dataset manually to remove any unsuitable images (faces overly obscured, too stylised to be meaningful, etc) I was left with 600 images, in which all faces were labelled using bounding boxes. Many thanks to Step for helping out with this particularly tedious job!
The default blacklist was applied throughout - the application was not trained on and will never display images covered by the default blacklist.
The trained model scored an average precision of 0.984 and an overall recall of 0.900, giving it an F1 score of 0.940, which was acceptable for this use case.
I then took images matching solo anthro simple_background -human -comic -real -monochrome -absurd_res for the e and s ratings and processed them with the model, producing a cropped version of each image around the face identified with the highest confidence. After any failures (due to large image sizes, unsuitable formats, failure to detect a face, etc), I still had a large number of cropped faces. The original filename, cropped filename, dimensions, and other metadata were recorded in a DynamoDB table for use in the application.
The data/ directory is where I did the ETL work for IsThisYiff?!
The frontend/ directory is a static Vue app that provides the UI for the game.
The lambda/ directory contains an AWS Lambda function that serves an API for the game that provides challenges and checks answers submitted by players.