Telecommunication-Telemedia-Assessment/bitstream_mode3_p1204_3

check.sh errors

FireCulex opened this issue · 1 comments

Everything is fresh, I assume check.sh is there to report that the values given by p1204_3 are accurate

culex@ubuntu:~/bitstream_mode3_p1204_3$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.2 LTS
Release:	20.04
Codename:	focal

culex@ubuntu:~/bitstream_mode3_p1204_3$ ./check.sh
Installing dependencies from lock file

No dependencies to install or update

Installing the current project: p1204_3 (0.1.2)
INFO:root:handle the following videos (# 3): 
  test_videos/test_video_h264.mkv
  test_videos/test_video_h265.mkv
  test_videos/test_video_vp9.mkv
INFO:root:check or install video parser
INFO:root:video parser is checked out
INFO:root:video parser is build
INFO:root:features are already cached, extraction skipped
INFO:root:features extracted
INFO:root:check or install video parser
INFO:root:video parser is checked out
INFO:root:video parser is build
INFO:root:features are already cached, extraction skipped
INFO:root:features extracted
INFO:root:check or install video parser
INFO:root:video parser is checked out
INFO:root:video parser is build
INFO:root:features are already cached, extraction skipped
INFO:root:features extracted
[
    {
        "date": "2021-08-04 11:32:34.443452",
        "debug": {
            "baseline": 4.16292374098929,
            "coding_deg": 32.43006645212775,
            "rf_pred": 4.010545689643026,
            "temporal_deg": 0.0,
            "upscaling_deg": 0.0
        },
        "features": {
            "0.0_quantil_StdDev_MotionX_non-i": 2.822123678091511,
            "1.0_quantil_FrameSize": 263605.0,
            "Bitrate": 6621.5419921875,
            "Framerate": 60.0,
            "Resolution": 8294400.0,
            "h264": 1.0,
            "h264_10bit": 0.0,
            "hevc": 0.0,
            "hevc_10bit": 0.0,
            "iqr_Av_QPBB_non-i": 4.828716049382718,
            "iqr_min_QP": 6.4,
            "kurtosis_Av_Motion": 1.4149027994195458,
            "kurtosis_Av_QPBB_non-i": -0.5480996174640883,
            "kurtosis_FrameSize_non-i": 0.21780539573460694,
            "mean_Av_QPBB_non_i": 32.67390905734393,
            "predicted_mos_mode3_baseline": 4.16292374098929,
            "quant": 0.6406648834773319,
            "std_FrameSize_non-i": 7841.7767315486035,
            "std_max_QP_non-i": 1.3921540581456298,
            "vp9": 0.0
        },
        "per_second": [
            4.132547886568417,
            3.931621503558015,
            4.328687969986304,
            4.3002566462931,
            4.1819097998511054,
            3.9471693619756834,
            3.935126424508243,
            3.9407565516571026,
            3.955412313745017,
            3.991729591232866
        ],
        "per_sequence": 4.086734715316158,
        "version": "0.1.2",
        "video_basename": "test_video_h264.mkv",
        "video_full_path": "test_videos/test_video_h264.mkv"
    },
    {
        "date": "2021-08-04 11:32:34.835867",
        "debug": {
            "baseline": 4.242288598950454,
            "coding_deg": 31.08324751733018,
            "rf_pred": 4.013317160671967,
            "temporal_deg": 0.0,
            "upscaling_deg": 0.0
        },
        "features": {
            "0.0_quantil_StdDev_MotionX_non-i": 0.0031622776601683794,
            "1.0_quantil_FrameSize": 223087.0,
            "Bitrate": 4045.6611328125,
            "Framerate": 60.0,
            "Resolution": 8294400.0,
            "h264": 0.0,
            "h264_10bit": 0.0,
            "hevc": 1.0,
            "hevc_10bit": 0.0,
            "iqr_Av_QPBB_non-i": 6.907654199786155,
            "iqr_min_QP": 8.0,
            "kurtosis_Av_Motion": 68.05316910522406,
            "kurtosis_Av_QPBB_non-i": -1.4577144394360826,
            "kurtosis_FrameSize_non-i": 0.7450933307203251,
            "mean_Av_QPBB_non_i": 34.80674794421678,
            "predicted_mos_mode3_baseline": 4.242288598950454,
            "quant": 0.6824852538081722,
            "std_FrameSize_non-i": 5663.799689161738,
            "std_max_QP_non-i": 1.6314642439608165,
            "vp9": 0.0
        },
        "per_second": [
            4.28077807773206,
            4.053632235922142,
            4.136342580097646,
            4.2571985844582665,
            4.286142690716187,
            4.066332673322398,
            4.061015016627177,
            4.047355487064874,
            4.080244792956893,
            4.054174969469214
        ],
        "per_sequence": 4.1278028798112105,
        "version": "0.1.2",
        "video_basename": "test_video_h265.mkv",
        "video_full_path": "test_videos/test_video_h265.mkv"
    },
    {
        "date": "2021-08-04 11:32:35.212758",
        "debug": {
            "baseline": 2.898352789601364,
            "coding_deg": 52.49078139374002,
            "rf_pred": 2.444248668622345,
            "temporal_deg": 0.0,
            "upscaling_deg": 0.0
        },
        "features": {
            "0.0_quantil_StdDev_MotionX_non-i": 0.315658943322485,
            "1.0_quantil_FrameSize": 20902.6,
            "Bitrate": 772.138671875,
            "Framerate": 60.0,
            "Resolution": 8294400.0,
            "h264": 0.0,
            "h264_10bit": 0.0,
            "hevc": 0.0,
            "hevc_10bit": 0.0,
            "iqr_Av_QPBB_non-i": 0.4,
            "iqr_min_QP": 0.6,
            "kurtosis_Av_Motion": 11.64902518023714,
            "kurtosis_Av_QPBB_non-i": 9.454349936497985,
            "kurtosis_FrameSize_non-i": 17.787335033398243,
            "mean_Av_QPBB_non_i": 252.64795004072766,
            "predicted_mos_mode3_baseline": 2.898352789601364,
            "quant": 0.9907762746695202,
            "std_FrameSize_non-i": 1333.1865240382524,
            "std_max_QP_non-i": 7.753703137279524,
            "vp9": 1.0
        },
        "per_second": [
            2.6717002506704413,
            2.6682867151518628,
            2.6666903653892486,
            2.6682867151518628,
            2.6686562922050974,
            2.6682867151518628,
            2.6686562922050974,
            2.6682867151518628,
            2.663123363830408,
            2.6934893700375597
        ],
        "per_sequence": 2.6713007291118545,
        "version": "0.1.2",
        "video_basename": "test_video_vp9.mkv",
        "video_full_path": "test_videos/test_video_vp9.mkv"
    }
]
INFO:root:store all results to test_videos/reports_new
[ERROR] difference in test_videos/reports/test_video_h264.json and test_videos/reports_new/test_video_h264.json 
2c2
---
9a10,31
>     "features": {
>         "0.0_quantil_StdDev_MotionX_non-i": 2.822123678091511,
>         "1.0_quantil_FrameSize": 263605.0,
>         "Bitrate": 6621.5419921875,
>         "Framerate": 60.0,
>         "Resolution": 8294400.0,
>         "h264": 1.0,
>         "h264_10bit": 0.0,
>         "hevc": 0.0,
>         "hevc_10bit": 0.0,
>         "iqr_Av_QPBB_non-i": 4.828716049382718,
>         "iqr_min_QP": 6.4,
>         "kurtosis_Av_Motion": 1.4149027994195458,
>         "kurtosis_Av_QPBB_non-i": -0.5480996174640883,
>         "kurtosis_FrameSize_non-i": 0.21780539573460694,
>         "mean_Av_QPBB_non_i": 32.67390905734393,
>         "predicted_mos_mode3_baseline": 4.16292374098929,
>         "quant": 0.6406648834773319,
>         "std_FrameSize_non-i": 7841.7767315486035,
>         "std_max_QP_non-i": 1.3921540581456298,
>         "vp9": 0.0
>     },
22a45
>     "version": "0.1.2",
[ERROR] difference in test_videos/reports/test_video_h265.json and test_videos/reports_new/test_video_h265.json 
2c2
---
9a10,31
>     "features": {
>         "0.0_quantil_StdDev_MotionX_non-i": 0.0031622776601683794,
>         "1.0_quantil_FrameSize": 223087.0,
>         "Bitrate": 4045.6611328125,
>         "Framerate": 60.0,
>         "Resolution": 8294400.0,
>         "h264": 0.0,
>         "h264_10bit": 0.0,
>         "hevc": 1.0,
>         "hevc_10bit": 0.0,
>         "iqr_Av_QPBB_non-i": 6.907654199786155,
>         "iqr_min_QP": 8.0,
>         "kurtosis_Av_Motion": 68.05316910522406,
>         "kurtosis_Av_QPBB_non-i": -1.4577144394360826,
>         "kurtosis_FrameSize_non-i": 0.7450933307203251,
>         "mean_Av_QPBB_non_i": 34.80674794421678,
>         "predicted_mos_mode3_baseline": 4.242288598950454,
>         "quant": 0.6824852538081722,
>         "std_FrameSize_non-i": 5663.799689161738,
>         "std_max_QP_non-i": 1.6314642439608165,
>         "vp9": 0.0
>     },
22a45
>     "version": "0.1.2",
[ERROR] difference in test_videos/reports/test_video_vp9.json and test_videos/reports_new/test_video_vp9.json 
2c2
---
9a10,31
>     "features": {
>         "0.0_quantil_StdDev_MotionX_non-i": 0.315658943322485,
>         "1.0_quantil_FrameSize": 20902.6,
>         "Bitrate": 772.138671875,
>         "Framerate": 60.0,
>         "Resolution": 8294400.0,
>         "h264": 0.0,
>         "h264_10bit": 0.0,
>         "hevc": 0.0,
>         "hevc_10bit": 0.0,
>         "iqr_Av_QPBB_non-i": 0.4,
>         "iqr_min_QP": 0.6,
>         "kurtosis_Av_Motion": 11.64902518023714,
>         "kurtosis_Av_QPBB_non-i": 9.454349936497985,
>         "kurtosis_FrameSize_non-i": 17.787335033398243,
>         "mean_Av_QPBB_non_i": 252.64795004072766,
>         "predicted_mos_mode3_baseline": 2.898352789601364,
>         "quant": 0.9907762746695202,
>         "std_FrameSize_non-i": 1333.1865240382524,
>         "std_max_QP_non-i": 7.753703137279524,
>         "vp9": 1.0
>     },
22a45
>     "version": "0.1.2",
stg7 commented

Thanks for pointing this out, we extended the output of the model to include also some feature values, and we forgot to update the corresponding report files for the check (done here: c699cbb) , this should be now solved.

best
Steve