A very slow movie player designed for the Raspberry Pi Zero
- Variable color depth dithering
- Optimized, pixel-packed data transfer
- On-device white-level adjustment
- So many dithering modes
- Supports all panels using IT8951 controller
- Supports hardware accelerated video decoding
- Supports ambient light sensing to pause display updates in darkness
This is my take on the concept of a very slow movie player, as previously done by Tom Whitwell
and, originally, Bryan Boyer.
As opposed to their implementations, this one is written in C to be close to the hardware, using libav and the IT8951 C library by waveshare to display frames from a video file on an ePaper display.
The code is designed for and tested with the 1872x1404 E-Ink panel from waveshare. Other panels using the same controller should work out of the box, for other panels you might have to customize the processFrame method.
This repo comes with a modified version of the IT8951 library, so all you need is the bcm2835 library as well as libavformat, libavcodec and libavutil.
To use hardware acceleration for video decoding, you'll have to use custom-built ffmpeg libraries (see https://maniaclander.blogspot.com/2017/08/ffmpeg-with-pi-hardware-acceleration.html).
vsmp-zero does not perform scaling, grayscale or framerate conversion of the video at runtime to save on computation and implementation complexity. You should pre-process your input accordingly on a faster machine using ffmpeg.
An example preprocessing command might look like this:
ffmpeg -i in.mkv -an -sn -c:v libx264 -vf "fps=fps=7,format=gray,pad=max(iw\,ih*(16/9)):ow/(16/9):(ow-iw)/2:(oh-ih)/2,scale=1872:-1" -aspect 16:9 -threads 4 out.mkv
This command removes any audio- or subtitle tracks, pads the input to a 16:9 aspect ratio with a width of 1872 px (the width of our panel), and reduces the framerate to 7fps such that playing the file frame-by-frame will be around 3x faster than playing at the original speed (assuming a 24fps input framerate). It also encodes the video in grayscale h264, for which the Raspberry Pi has hardware decoding support.
Configure vsmp.h to your liking, then run make vsmp. Because the libav libraries are linked statically (dynamic linking won't work for some reason), this might take a minute and produce a very large binary file. Also make sure you have enabled SPI on your Raspberry using raspi-config.
You can now run vsmp as follows:
sudo ./vsmp [video file] [start frame index]
Root rights are necessary to use SPI and the GPIO pins.
To continue running after you close the console, you might want to use nohup as follows:
sudo nohup ./vsmp [video file] [start frame index] &
The current frame index is backed up periodically to a file called vsmp-index. If the frame index argument is omitted on startup and an index file is found, playback is resumed at the last saved frame index.
If you'd like to have vsmp started automatically on boot, you might want to use this very bare-bone systemd service file:
[Unit]
Description=VSMP
[Service]
Type=simple
User=root
WorkingDirectory=[this is where the vsmp-index file will be]
ExecStart=[path to vsmp executable] [path to video file relative to working dir]
[Install]
WantedBy=multi-user.target
You can save the contents of this file to e.g. /etc/systemd/system/vsmp.service and then run sudo systemctl enable vsmp to start vsmp on boot.
Do you need nine different dithering algorithms? Probably not. The default should be fine, and for high-res displays with multiple grey-levels you probably won't be able to tell most of them apart. But it turns out image halftoning is a fascinating rabbit hole and there are loads of different ways to do it - so why not play with a bunch of them?
Currently, vsmp-zero supports the following dithering modes:
- Floyd-Steinberg (regular and serpentine, default)
- Interleaved Noise
- Blue Noise
- White Noise
- Sierra (full and two-row versions)
- Stucki
- Atkinson
If you're not inspired yet, here's another sample image from In the Mood for Love which I think came out really great. The first image is dithered to 1 bit per pixel, the second one to 2 bits per pixel.
As of now, only displays using the IT8951 controller are supported out of the box. It appears that many of the cheaper waveshare displays are different in very subtle ways, so there is likely no one comprehensive solution to support them all.
If you want to work with an unsupported display, you'll have to provide what I'll call a display driver file, which is included by vsmp.c.
All that this driver file has to do is provide implementations of a few functions used by vsmp-zero — you can have a look at displays/genericIT8951.c and displays/dryrun.c for some examples. Most of the time, you should get away with some clever copy/pasting from the waveshare example code for your display.
Generally speaking, your file should look something like this:
static int initDisplay() {
// Initialization code, only executed once on program start
// return non-zero integer to indicate failure
}
static void teardownDisplay() {
// Cleanup code executed on program termination
}
static void clearDisplay() {
// Clears whatever is on the display, displays a white screen
}
static void pixelPush(unsigned char *frameBuf, int linesize, int width, int height) {
/* Writes an image buffer to the display and displays it.
Each entry in the frame buffer is a single greyscale pixel at 8bpp.
You may need to perform some downconversion here depending on the display
(e.g. packing 8 pixels into a singly byte at 1bpp).
Note that the height / width are that of your video file, not necessarily your display.
You may want to perform some scaling or aligning here.
Also note that one line in the frame buffer (given by linesize)
can be longer than the actual line of pixels.
To put less strain on your display and ensure a long lifetime,
you should also set your display to sleep / standby at the end of this method
and wake it up (if necessary) at the beginning.
*/
}If you successfully go through all that and add support for a new display type, feel free to open a pull request with your driver file and share your work!