Issue with Third party feature extraction

[kanlions]

I was attempting to use the STTRED feature by running as given in the Git instructions as below
matlab -nodisplay -nosplash -nodesktop -r "run_strred('../../python/test/resource/yuv/src01_hrc00_576x324.yuv', '../../python/test/resource/yuv/src01_hrc01_576x324.yuv', 576, 324); exit;"

But I find 1200 frames in STRRED as compared to 600 frames in the VMAF for same set of videos. What is the correct way of running STRRED ?
I can share test outputs if required?

[krasuluk]

Hi @kanlions, can you please share the outputs? I was under the impression that the src01_hrc00_576x324.yuv file should only have 48 frames...

[kanlions]

strred.txt
vmaf.txt

Apologies for confusion. I had run on a custom video. I am sharing the vmaf and strred generated from this github for the same reference-target pair, I am having difficulty in understanding how twice number of frames are coming for STRRED in comparison to VMAF.

[nilfm]

Just a guess since we don't have access to the source video, but is it possible that VMAF and STRRED are using different bit depths? I'm assuming this video has 600 frames, so it is STRRED that is wrong. If the video was 10-bit and STRRED was interpreting it as 8-bit, it would find 2x the frames, but its input would be nonsense.

I am not familiar with this Matlab code but looking here, it seems to be hardcoded to 8-bit, yuv420p inputs. If your video is 10b, maybe the easiest way forward is to downscale it to 8b with ffmpeg, although there will be information loss. You can choose to use dithering to mitigate some artifacts that can appear when downscaling bit depth, such as banding artifacts.

[kanlions]

Thanks for your reply. Yes the bit depth is indeed 10 bit. I will try changing to 8bit and rerun and see what happens

[nilfm]

Great, please let me know if that didn't work.

[kanlions]

@nilfm Thanks for all the support the 8 bit version works fine. But now I have run into challenges https://github.com/Netflix/vmaf/blob/master/python/vmaf/core/matlab_quality_runner.py it may be due to environment clash but I couldnot find any test case for matlab_quality_runner in the test folder. Was there any test case for these features?

[nilfm]

@kanlions I believe you're looking for this:

vmaf/python/test/extra/feature_extractor_extratest.py

Lines 25 to 44 in e698b4d

	def test_run_strred_fextractor(self):
	ref_path, dis_path, asset, asset_original = set_default_576_324_videos_for_testing()
	self.fextractor = StrredFeatureExtractor(
	[asset, asset_original],
	None, fifo_mode=True,
	result_store=None
	)
	self.fextractor.run(parallelize=False)
	results = self.fextractor.results
	self.assertAlmostEqual(results[0]['STRRED_feature_srred_score'], 3.0166328541666663, places=4)
	self.assertAlmostEqual(results[0]['STRRED_feature_trred_score'], 7.338665770833333, places=4)
	self.assertAlmostEqual(results[0]['STRRED_feature_strred_score'], 22.336452104611016, places=4)
	self.assertAlmostEqual(results[0]['STRRED_feature_strred_all_same_score'], 22.138060270044175, places=4)
	self.assertAlmostEqual(results[1]['STRRED_feature_srred_score'], 0.0, places=4)
	self.assertAlmostEqual(results[1]['STRRED_feature_trred_score'], 0.0, places=4)
	self.assertAlmostEqual(results[1]['STRRED_feature_strred_score'], 0.0, places=4)

This is not present in master because it was removed 1.5 years ago in this commit, @li-zhi do you know why those tests were removed? Note that they are mentioned in CONTRIBUTING.md here.

Hopefully that test helps you understand how to call the quality runner. I believe you can use your original 10b file and add "workfile_yuv_type": "yuv420p" to asset_dict to have an automatic conversion but I'm not sure the Matlab-related components are set up correctly for that.