Opentestbench : an open-source framework to assess the performances of image compression schemes Copyright (C) 2015 intoPIX s.a. , Sébastien Lugan Copyright (C) 2017 Universite Catholique de Louvain, Alexandre Willème
Opentestbench is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.
For any question please contact : Alexandre Willème (alexandre.willeme@uclouvain.be)
IMPORTANT NOTES :
- the boost library must be installed manually (for instance http://sourceforge.net/projects/boost/files/boost/1.60.0/boost_1_60_0.tar.gz/download) in order to make vc2-reference work
- opentestbench works only on Linux.
- If the target directory for decompressed frames already contain frames of the sequence under test, the compression/decompression will not happen
Experiment types:
- singlegen : simple encoding/decoding of a set of pictures and PSNR computations wrt the original picture (example runSG.sh)
- multigen : several consecutive encoding/decoding of a set of pictures and PSNR computations after each iteration wrt the original picture (example runMG.sh)
- errortest (to be fixed) : a single bit-flip at a random position is inserted in the bitstream before decoding and PSNR computations wrt the original picture
- overlay test (to be fixed)
- multi-generation test with pixel shift between each generation (to be fixed)
- VDP assessment (to be fixed)
Currently supported codecs :
- JP2K ULL (Kakadu)
- JP2K ULLC (Kakadu)
- VC-2 LD (vc2-reference)
- VC-2 HQCBR (vc2-reference)
- JPEG (with pseudo-CBR mode) (jpeg-xt reference software)
- JPEG LS (with pseudo-CBR mode) (jpeg-xt reference software)
- DSC
- HEVC HM (with pseudo-CBR mode)
- HEVC HM Screen Content Coding extension (with pseudo-CBR mode)
- XSM (version 1.4)
Currently supported image formats :
- RGB 4:4:4 8b, 10b and 12b in PPM files
- YCbCr 4:2:2 10b and 12b in RAW planar 16-bit little-endian files
Instructions :
- To install:
- Go to the opentestbench folder.
- If you have a version of DSC (e.g. DSC_model_201407151.zip) put the zip file in opentestbench/codecs/DSC
- To be able to make vc2-reference work, libboost must be installed (http://www.boost.org/)
- Run bash install.sh.
- To add a new codec:
- Modify the encapsulator benchmark/prototype/prototype.sh to add your encoder and decoder command lines for RGB 8b, 10b and 12b and YCbCr 4:2:2 10b and 12b
- For example, look at benchmark/vc2ld/vc2ld.sh if you want to know how to do format conversion using difftest_ng
- Do not hesitate to ask support if needed
- To run compression-decompression and compute PSNR
- Configure the experiment (see runSG.sh or runMG.sh for examples)
- Run the experiment configuration file (e.g. bash runSG.sh)
- Observe the results in the "results" folder.
- To generate rate-distorsion curves using Matlab:
- Graphs can be generated automatically using matlab by modifying and running run_graphics.sh (work in progress)
- Observe the results in the "graphics" folder.
- To generate flicker sequences:
- Flicker sequences (in the sense of the JPEG XS CfP assessment) can be generated (see run_subj_seqs.sh). This requires ffmpeg.
- The results are available in the "preprocessed_material" folder
Uninstall:
- opentestbench installs all required software locally. Deleting the opentestbench/build folder undo the effect of install.sh