Sometimes STB-VMM uses too much memory and becomes unusable in some setups, especially when magnifying large resolution videos. To mitigate this problem, this code breaks the sequence into multiple slightly overlapping tiles that are then magnifyed individually and stitched together at the end, yielding acceptable results using much less memory.
STB-VMM Demo
STB_VMM_Video.mp4
git submodule update --init --recursivepip install -r requirements.txtDownload trained STB-VMM model
For this example we place the checkpoint in ./STB-VMM/ckpt/ckpt_e49.pth.tar. You may place this file anywhere you like.
To magnify x20 a sample_video.mp4 with a tile size of 512px use:
python magnify.py -i sample_video.mp4 -c STB-VMM/ckpt/ckpt_e49.pth.tar -m 20 -t 512 For more information regarding options and parameters read the output of magnify.py -h:
Tiled STB-VMM: Break large videos into tiles, magnify those tiles and stitch'em together. Makes large videos processable with
low amounts of RAM
options:
-h, --help show this help message and exit
-i PATH, --video_path PATH
path to video input frames
--temp PATH path to save temporal data (deleted on exit) (default: /dev/shm/temp_STB-VMM)
-c PATH, --load_ckpt PATH
path to load checkpoint
-o PATH, --output PATH
path to save generated frames (default: demo.avi)
-m N, --mag N magnification factor (default: 20.0)
--mode {static,dynamic}
magnification mode (static, dynamic)
-t T, --tile_size T size of the tiles to be processed. The bigger the tile the faster magnification runs, as long as the
tile fits in VRAM
--overlap O tile edge overlap in pixels (default: 30)
-j N, --workers N number of data loading workers (default: 16)
-b N, --batch_size N batch size (default: 1)
-p N, --print_freq N print frequency (default: 100)
--device {auto,cpu,cuda}
select device [auto/cpu/cuda] (default: auto)