OpenHEVC Decoder Segfault

[jlsantiago0]

This is really an issue for https://github.com/OpenHEVC/libav , but I cannot log an issue on that project for some reason. The backport of OpenHEVC codec to the libav project on the branch:hm10.0 demonstrates a segfault when decoding the following H.265 elementary streams:

wget http://208.51.85.91/hevc-openhevc-decoder-wrong-output-order.tar.bz2

Note the filename is misleading, these streams used to have frame ordering issues with the decoder. But not anymore. At least not in https://github.com/OpenHEVC/openHEVC/ .

[jlsantiago0]

This patch fixes the issue in https://github.com/OpenHEVC/libav . This patch also prevents dereference of NULL pointer if the sps is not available at this point during decoder initialization.

diff --git a/libavcodec/hevc_ps.c b/libavcodec/hevc_ps.c
index 64c99d9..08a3567 100644
--- a/libavcodec/hevc_ps.c
+++ b/libavcodec/hevc_ps.c
@@ -462,6 +462,8 @@ int ff_hevc_decode_nal_sps(HEVCContext *s)
     start = (sps->sps_sub_layer_ordering_info_present_flag ? 0 : (sps->sps_max_sub_layers-1));
     for (i = start; i < sps->sps_max_sub_layers; i++) {
         sps->temporal_layer[i].max_dec_pic_buffering = get_ue_golomb(gb);
+        // Fixes MC Streams.
+        sps->temporal_layer[i].max_dec_pic_buffering += 1;
         sps->temporal_layer[i].num_reorder_pics      = get_ue_golomb(gb);
         sps->temporal_layer[i].max_latency_increase  = get_ue_golomb(gb);
         if (sps->temporal_layer[i].max_dec_pic_buffering >= MAX_DPB_SIZE) {
@@ -470,8 +472,8 @@ int ff_hevc_decode_nal_sps(HEVCContext *s)
             goto err;
         }
         if (sps->temporal_layer[i].num_reorder_pics > sps->temporal_layer[i].max_dec_pic_buffering) {
-            av_log(s->avctx, AV_LOG_ERROR, "sps_max_num_reorder_pics out of range: %d\n",
-                   sps->temporal_layer[i].num_reorder_pics);
+            av_log(s->avctx, AV_LOG_ERROR, "sps_max_num_reorder_pics=%d out of range: %d\n",
+                   sps->temporal_layer[i].num_reorder_pics, sps->temporal_layer[i].num_reorder_pics);
             goto err;
         }
     }
diff --git a/libavcodec/hevc_refs.c b/libavcodec/hevc_refs.c
index 02d5096..38c070c 100644
--- a/libavcodec/hevc_refs.c
+++ b/libavcodec/hevc_refs.c
@@ -163,6 +163,7 @@ int ff_hevc_find_display(HEVCContext *s, AVFrame *out, int flush, int* poc_displ
         }
         /* wait for more frames before output */
         if (!flush && sc->seq_output == sc->seq_decode &&
+            sc->sps != NULL &&
             nb_output <= sc->sps->temporal_layer[0].num_reorder_pics+1)
             return 0;

[mraulet]

I will have a look still stick with other stuff, but I have an idea that I haven t been to check yet

[jlsantiago0]

Another stability patch for https://github.com/OpenHEVC/libav .

diff --git a/libavcodec/hevc_ps.c b/libavcodec/hevc_ps.c
index 08a3567..e574c5e 100644
--- a/libavcodec/hevc_ps.c
+++ b/libavcodec/hevc_ps.c
@@ -628,6 +628,11 @@ int ff_hevc_decode_nal_pps(HEVCContext *s)
         goto err;
     }
     sps = s->HEVCsc->sps_list[pps->sps_id];
+    if (sps == NULL)
+    {
+       av_log(s->avctx, AV_LOG_ERROR, "SPS is NULL\n");
+       goto err;
+    }

     pps->dependent_slice_segments_enabled_flag = get_bits1(gb);
     pps->output_flag_present_flag = get_bits1(gb);

[mraulet]

fixed.

Could you try with the latest version? It seems ok to me