Webm VP8 test suite

[ezequielgarcia]

I would like to add a VP8 test suite. Main motivation here is to run the conformance test on V4L2 VP8 hardware accelerators.

Currently, the test vector can be found in https://chromium.googlesource.com/webm/vp8-test-vectors/ or its github mirror https://github.com/webmproject/vp8-test-vectors/. The former is exposed thru a Gitiles server, which means it's not straightforward to download files via HTTP easily.

It seems to me, either we extend class TestSuite et al to support a Git repository test suite, so instead of HTTP-getting the vectors, we'd be just cloning the repository, or we use some github python API, to download via HTTP so we can treat this new test suite just like the current ones.

I'm inclined for the latter, but would like to ask here first.

Thanks!

[pamarcos]

First of all, thanks for the contribution!

The easiest way I can find of achieving this doesn't require any modification in the source at all and consists of creating a new test suite with sources of the form https://github.com/webmproject/vp8-test-vectors/raw/master/vp80-00-comprehensive-001.ivf

This is not ideal for this particular case since cloning the whole repo takes < 2 secs in my computer and presumably downloading the files separately is not as efficient. In any case, this is what I'd go for as of now. Let's see if this does not work well and we need to extend the download functions to work per test suite rather per test vector. For now, I'd prefer to keep the dependencies to a minimum, and it seems GitHub allows downloading raw files using HTTP, so this seems fine.

In order to support VP8, we're going to need to merge this one and do some minor development in fluster to check on a checksum-per-frame basis: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1320

There is already a script that we used to build the test suites that you may want to take a look at because it may ease creating the VP8 ones: https://github.com/fluendo/fluster/blob/master/scripts/gen_jct_vc.py

[ezequielgarcia]

@pamarcos So far so good for getting the sources via github's HTTP. Now, I'm wondering how we'll express the output checksums in the test suite JSON.

How about:

        {
            "name": "vp80-00-comprehensive-001",
            "source": "https://github.com/webmproject/vp8-test-vectors/raw/master/vp80-00-comprehensive-001.ivf",
            "source_checksum": "5db435f13b5c35004f51307aee1074eb",
            "input_file": "vp80-00-comprehensive-001.ivf",
            "output_format": "yuv420p",
            "frame_result_file": "vp80-00-comprehensive-001.ivf.md5sum"
        },

Where we use frame_result_file so it's clear it's per-frame.

[rgonzalezfluendo]

Please check the work I started in this PR: #58

(GitHub should have created a reference to the PR automatically)