Smohbot: a content aware image scaling Discord bot

Smohbot is a content aware image manipulation (seam carving) application written in Java, packaged into a Discord bot using JDA!

As opposed to scaling or cropping, seam carving identifies and removes the "least important" portions of an image first when reducing image size.

Features

Content aware image scaling (seam carving) for message image attachments
- Supports scaling along both axes
- Specify either flat pixel cuts or percentage scaling
- Implements multiple seam carving scaling strategies
Interacts with users within the Discord client via text
Detailed info and error messages to guide users on how to use commands
Features artwork of a friendly cat mascot to respond to commands!

Examples

- The Great Wave off Kanagawa, Hokusai

- Space Needle and Geese images are my own

Installation Guide

Create or use an existing bot application through Discord's Developer Portal and store the unique token and add it to the desired server. Note that Smohbot requires send and recieve messages with attachments.

Clone the repository using git

cd /projectpath
git clone https://github.com/qin-andy/Smohbot.git

cd into the newly created directory and use gradle install to collect dependencies
Run gradle shadow or gradlew shadow to generate the fat jar in build/libs/smohbot-shadowjar.jar
Launch Smohbot using java -jar <smohbot jar location> and enter the token to associate it with the Discord application created in step 1

Command Guide

Call Smohbot on any channel on the server!

!info: help command, details command usage
!carve x y: default seam carving command for dimensions x and y
!fcarve x y: experimental forwards energy seam carving command

Use Notes

Images are automatically compressed to fit within a 1000x1000 square to reduce seam carving time for larger images. This size can be changed or removed in the ModularCarver class's constant field MAX_SIZE.
Smohbot requires permission to download, read, and write image files
Picture assets found in the resources/assets use graphics assets folder (smoh_chop.gif, smoh_apology.jpg, smoh_help.jpg) are my own, do not reuse without express permission!

Algorithm Details

The implmentation of the "backward energy" seam carving algorithm (!carve) is built on the work of Shai Avidan and Ariel Shamir which was based on an inital 2007 paper introducing seamcarving as well as their incredibly informative video on the subject

This project was also inpsired by assignments by Princeton, Stanford, and University of Washington; However, my implementation of the algorithm is built from scratch and structured differently. For example, I found a 20 times speed increase in directly accessing raster data buffers for both reading and writing images as opposed to the using the Picture (or BufferedImage) class described in assignment specficiations.

The "forwards energy" algorithm (!fcarve) is built off of a follow up paper by Michael Rubinstein and the original authors, which reduces image artifacts by considering the energy of pixels made adjacent after a scene removal. Avik Das's incredible article summarizing the algorithm was a great help.

A comparison between backwards energy (left) and and forwards energy (right). Note the smoothness on the crest of the wave on the forwards energy image versus the rough choppiness on the left.

The general framework for the seam carving algorithm is as follows:

Extract RGB information from picture
Calculate an energy value for each pixel to construct an "energy map"
Modelling the energy map as a weighted directed acyclic graph which each pixel having an edge to 3 pixels on the row below it and with their energies as edge weights, find the shortest path from any pixels on the top row to any pixel on the bottom, the "lowest energy seam"
Remove every pixel along that path, i.e. seam carve
Repeat steps 2-4 until the image is the desired size, then reconstruct it based on the resulting RGB information

Forwards and backwards energy mapping are interchangeable strategies for steps 2 and 3.