hbiyik/FFmpeg

FFmpeg transcoding error on RK3588

Closed this issue · 24 comments

root@scensmart:~ ffmpeg -i source1.mp4 -an -v trace out.mp4
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.2)
configuration: --prefix=/usr/local --enable-version3 --enable-libdrm --enable-rkmpp --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-shared
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
Splitting the commandline.
Reading option '-i' ... matched as input url with argument 'source1.mp4'.
Reading option '-an' ... matched as option 'an' (disable audio) with argument '1'.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'trace'.
Reading option 'out.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument trace.
Successfully parsed a group of options.
Parsing a group of options: input url source1.mp4.
Successfully parsed a group of options.
Opening an input file: source1.mp4.
[NULL @ 0x5589640820] Opening 'source1.mp4' for reading
[file @ 0x5589640e10] Setting default whitelist 'file,crypto,data'
Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:2048
Successfully opened the file.
Parsing a group of options: output url out.mp4.
Applying option an (disable audio) with argument 1.
Successfully parsed a group of options.
Opening an output file: out.mp4.
File 'out.mp4' already exists. Overwrite? [y/N] y
[file @ 0x5589645090] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
[h264_rkmpp_decoder @ 0x5589648a10] Initializing RKMPP Codec.
[h264_rkmpp_decoder @ 0x5589648a10] Format nv12 chosen by get_format().
[h264_rkmpp_decoder @ 0x5589648a10] Picture format is nv12.
Stream mapping:
Stream #0:0 -> #0:0 (h264 (h264_rkmpp_decoder) -> h264 (h264_rkmpp_encoder))
Press [q] to stop, [?] for help
[vost#0:0/h264_rkmpp_encoder @ 0x55896be940] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp_decoder @ 0x5589648a10] ff_decode_get_packet return (code = 0)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 0, sample 1, dts 40000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 1, sample 1, dts 64000
[h264_rkmpp_decoder @ 0x5589648a10] Wrote 82631 bytes to decoder
[h264_rkmpp_decoder @ 0x5589648a10] ff_decode_get_packet return (code = -11)
[h264_rkmpp_decoder @ 0x5589648a10] Decoder Can't get packet retrying.
[vost#0:0/h264_rkmpp_encoder @ 0x55896be940] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
Last message repeated 1 times
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 0, sample 2, dts 80000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 1, sample 1, dts 64000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 0, sample 2, dts 80000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 1, sample 2, dts 128000
[h264_rkmpp_decoder @ 0x5589648a10] ff_decode_get_packet return (code = 0)
[h264_rkmpp_decoder @ 0x5589648a10] Wrote 27568 bytes to decoder
[h264_rkmpp_decoder @ 0x5589648a10] ff_decode_get_packet return (code = -11)
[h264_rkmpp_decoder @ 0x5589648a10] Decoder Can't get packet retrying.
[vost#0:0/h264_rkmpp_encoder @ 0x55896be940] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 0, sample 3, dts 120000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 1, sample 2, dts 128000
[h264_rkmpp_decoder @ 0x5589648a10] ff_decode_get_packet return (code = 0)
[h264_rkmpp_decoder @ 0x5589648a10] Wrote 25821 bytes to decoder
[h264_rkmpp_decoder @ 0x5589648a10] ff_decode_get_packet return (code = -11)
[h264_rkmpp_decoder @ 0x5589648a10] Decoder Can't get packet retrying.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 0, sample 4, dts 160000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 1, sample 2, dts 128000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 0, sample 4, dts 160000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 1, sample 3, dts 192000
[vost#0:0/h264_rkmpp_encoder @ 0x55896be940] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp_decoder @ 0x5589648a10] ff_decode_get_packet return (code = 0)
[h264_rkmpp_decoder @ 0x5589648a10] Wrote 25423 bytes to decoder
[h264_rkmpp_decoder @ 0x5589648a10] ff_decode_get_packet return (code = -11)
[h264_rkmpp_decoder @ 0x5589648a10] Decoder Can't get packet retrying.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 0, sample 5, dts 200000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 1, sample 3, dts 192000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 0, sample 5, dts 200000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 1, sample 4, dts 256000
[vost#0:0/h264_rkmpp_encoder @ 0x55896be940] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp_decoder @ 0x5589648a10] ff_decode_get_packet return (code = 0)
[h264_rkmpp_decoder @ 0x5589648a10] Decoder buffer full
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 0, sample 6, dts 240000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5589640820] stream 1, sample 4, dts 256000
[h264_rkmpp_decoder @ 0x5589648a10] Decoder noticed an info change
[h264_rkmpp_decoder @ 0x5589648a10] Bus error (core dumped)

hbiyik commented

How can i reproduce this?

I followed the links listed in Wiki docs and installed mpp, librga-multi, libyuv and ffmpeg 6.
However, I'm stuck by ff_decode_get_packet returning (code = -11), but have no idea about how to debug deeper.
Any ideas to help me move forward?

Here is some detail information.
Hardware:
SoC: Rockchip RK3588
CPU: Quad-core ARM Cortex-A76(up to 2.4GHz) and quad-core Cortex-A55 CPU (up to 1.8GHz)
GPU: Mali-G610 MP4, compatible with OpenGLES 1.1, 2.0, and 3.2, OpenCL up to 2.2 and Vulkan1.2
VPU: 8K@60fps H.265 and VP9 decoder, 8K@30fps H.264 decoder, 4K@60fps AV1 decoder, 8K@30fps H.264 and H.265 encoder
NPU: 6TOPs, supports INT4/INT8/INT16/FP16
RAM: 64-bit 4GB/8GB/16GB LPDDR4X at 2133MHz
Flash: 32GB/64GB/256GB eMMC, at HS400 mode
microSD: support up to SDR104 mode
Ethernet: 2x PCIe 2.5G Ethernet

OS:
Ubuntu 20.04.4 LTS (Focal Fossa)

gcc:
gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.2)

MPP:
Name: rockchip_mpp
Description: Rockchip Media Process Platform
Requires.private:
Version: 1.3.8

libdrm:
Name: libdrm
Description: Userspace interface to kernel DRM services
Version: 2.4.89

librga-multi:
Name: librga
Description: librga 2D library
Version: 1.10.0

libyuv:
Name: libyuv
URL: http://code.google.com/p/libyuv/
Version: 1419

hbiyik commented

Can you run

coredumpctl gdb

and then

bt full

You need gdb installed in your system

hbiyik commented

and is this hapenning with everyfile?

hbiyik commented

And additionally

ffmpeg -loglevel debug -i file.mp4 -an -sn -f null -

root@scensmart:~ ffmpeg -loglevel debug -i source.mp4 -an -sn -f null -
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.2)
configuration: --prefix=/usr/local --enable-version3 --enable-libdrm --enable-rkmpp --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-shared
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
Splitting the commandline.
...
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url source.mp4.
Successfully parsed a group of options.
Opening an input file: source.mp4.
[NULL @ 0x5596367850] Opening 'source.mp4' for reading
[file @ 0x5596367e40] Setting default whitelist 'file,crypto,data'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5596367850] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5596367850] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5596367850] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5596367850] Processing st: 0, edit list 0 - media time: 0, duration: 27105840
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5596367850] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5596367850] Processing st: 1, edit list 0 - media time: 0, duration: 2409208
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5596367850] Before avformat_find_stream_info() pos: 126671720 bytes read:112745 seeks:1 nb_streams:2
[h264 @ 0x55963686a0] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x55963686a0] Decoding VUI
[h264 @ 0x55963686a0] nal_unit_type: 8(PPS), nal_ref_idc: 3
...
[h264 @ 0x55963686a0] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x55963686a0] Decoding VUI
[h264 @ 0x55963686a0] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x55963686a0] Decoding VUI
[h264 @ 0x55963686a0] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x55963686a0] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x55963686a0] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x55963686a0] Decoding VUI
[h264 @ 0x55963686a0] Format yuv420p chosen by get_format().
[h264 @ 0x55963686a0] Reinit context to 1920x1088, pix_fmt: yuv420p
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5596367850] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5596367850] After avformat_find_stream_info() pos: 485465 bytes read:630672 seeks:2 frames:2
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'source.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf60.3.100
Duration: 00:05:01.18, start: 0.000000, bitrate: 3364 kb/s
Stream #0:00x1, 1, 1/90000: Video: h264 (Main), 1 reference frame (avc1 / 0x31637661), yuv420p(tv, progressive, left), 1920x1080 (1920x1088), 0/1, 3347 kb/s, 25 fps, 25 tbr, 90k tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream #0:10x2, 1, 1/8000: Audio: aac (LC) (mp4a / 0x6134706D), 8000 Hz, mono, fltp, 15 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
Successfully opened the file.
Parsing a group of options: output url -.
Applying option an (disable audio) with argument 1.
Applying option sn (disable subtitle) with argument 1.
Applying option f (force format) with argument null.
Successfully parsed a group of options.
Opening an output file: -.
Successfully opened the file.
[h264_rkmpp_decoder @ 0x55963ce0d0] Initializing RKMPP Codec.
[h264_rkmpp_decoder @ 0x55963ce0d0] Format nv12 chosen by get_format().
[h264_rkmpp_decoder @ 0x55963ce0d0] Picture format is nv12.
Stream mapping:
Stream #0:0 -> #0:0 (h264 (h264_rkmpp_decoder) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[vost#0:0/wrapped_avframe @ 0x55963d22b0] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp_decoder @ 0x55963ce0d0] Wrote 485158 bytes to decoder
[vost#0:0/wrapped_avframe @ 0x55963d22b0] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
Last message repeated 1 times
[h264_rkmpp_decoder @ 0x55963ce0d0] Wrote 4910 bytes to decoder
[vost#0:0/wrapped_avframe @ 0x55963d22b0] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp_decoder @ 0x55963ce0d0] Wrote 17824 bytes to decoder
[vost#0:0/wrapped_avframe @ 0x55963d22b0] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp_decoder @ 0x55963ce0d0] Wrote 15159 bytes to decoder
[vost#0:0/wrapped_avframe @ 0x55963d22b0] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[h264_rkmpp_decoder @ 0x55963ce0d0] Decoder noticed an info change
Bus error (core dumped)

hbiyik commented

can you share your source.mp4?

and

MPP:
Name: rockchip_mpp
Description: Rockchip Media Process Platform
Requires.private:
Version: 1.3.8

Where did you get this mpp from? there is no 1.3.8 mpp, latest is 1.0.2

Where did you get this mpp from? there is no 1.3.8 mpp, latest is 1.0.2

I got this info from /usr/local/lib/pkgconfig/rockchip_mpp.pc.
image

hbiyik commented

output of

ls -la /dev/mpp_service

and

groups

and

uname -a

?

hbiyik commented

i think you installed mpp from ppa? is it amazingfate`s ppa?

When I got this Nano T6, the rockchip_mpp had been installed. So I don't know if it was installed from ppa. And then I reinstalled mpp from source code.

hbiyik commented

we can verify this, can you run
journalctl -f

in a window and run

ffmpeg -loglevel debug -i file.mp4 -an -sn -f null - in another window

this will print the version in the journal ctl, please paste the journalctl output.

hbiyik commented

ok i see several problems here, first your mpp version is potato, please use the latest git, 2nd, you have some allocator problems which is sucpicious can you also send the output of

ls -la /dev/dma_heap/*

hbiyik commented

ok your dma devices are there, which means mpp can not allocate from dma, and when it tries to clean it segfaults.
i am guessing mpp can not allocate from dma, because it is a very old version. Please use this version.

Thank you for letting me know this problems. I am updating git and mpp right now.

I have updated mpp from source code compilation. But the errors are still there.
image

Update mpp from source code:
git clone https://github.com/rockchip-linux/mpp.git
cd mpp/build/linux/aarch64
./make-Makefiles.bash

...
Install the project...
-- Install configuration: "Release"
-- Up-to-date: /usr/local/include/rockchip
-- Up-to-date: /usr/local/include/rockchip/mpp_frame.h
-- Up-to-date: /usr/local/include/rockchip/mpp_packet.h
-- Up-to-date: /usr/local/include/rockchip/rk_hdr_meta_com.h
-- Up-to-date: /usr/local/include/rockchip/rk_venc_ref.h
-- Up-to-date: /usr/local/include/rockchip/mpp_log_def.h
-- Up-to-date: /usr/local/include/rockchip/vpu.h
-- Up-to-date: /usr/local/include/rockchip/mpp_meta.h
-- Up-to-date: /usr/local/include/rockchip/mpp_task.h
-- Up-to-date: /usr/local/include/rockchip/rk_venc_cmd.h
-- Up-to-date: /usr/local/include/rockchip/mpp_compat.h
-- Up-to-date: /usr/local/include/rockchip/vpu_api.h
-- Up-to-date: /usr/local/include/rockchip/mpp_log.h
-- Up-to-date: /usr/local/include/rockchip/rk_type.h
-- Up-to-date: /usr/local/include/rockchip/rk_venc_cfg.h
-- Up-to-date: /usr/local/include/rockchip/mpp_err.h
-- Up-to-date: /usr/local/include/rockchip/rk_vdec_cfg.h
-- Up-to-date: /usr/local/include/rockchip/mpp_rc_defs.h
-- Up-to-date: /usr/local/include/rockchip/rk_venc_rc.h
-- Up-to-date: /usr/local/include/rockchip/mpp_buffer.h
-- Up-to-date: /usr/local/include/rockchip/rk_vdec_cmd.h
-- Up-to-date: /usr/local/include/rockchip/mpp_rc_api.h
-- Up-to-date: /usr/local/include/rockchip/rk_mpi.h
-- Up-to-date: /usr/local/include/rockchip/rk_mpi_cmd.h
-- Up-to-date: /usr/local/lib/pkgconfig/rockchip_mpp.pc
-- Up-to-date: /usr/local/lib/pkgconfig/rockchip_vpu.pc
-- Installing: /usr/local/lib/librockchip_mpp.so.0
-- Up-to-date: /usr/local/lib/librockchip_mpp.so.1
-- Up-to-date: /usr/local/lib/librockchip_mpp.so
-- Installing: /usr/local/lib/librockchip_vpu.so.0
-- Up-to-date: /usr/local/lib/librockchip_vpu.so.1
-- Set runtime path of "/usr/local/lib/librockchip_vpu.so.0" to ""
-- Up-to-date: /usr/local/lib/librockchip_vpu.so
-- Installing: /usr/local/bin/mpp_info_test
-- Set runtime path of "/usr/local/bin/mpp_info_test" to ""
-- Installing: /usr/local/bin/mpi_dec_test
-- Set runtime path of "/usr/local/bin/mpi_dec_test" to ""
-- Installing: /usr/local/bin/mpi_dec_mt_test
-- Set runtime path of "/usr/local/bin/mpi_dec_mt_test" to ""
-- Installing: /usr/local/bin/mpi_dec_nt_test
-- Set runtime path of "/usr/local/bin/mpi_dec_nt_test" to ""
-- Installing: /usr/local/bin/mpi_enc_test
-- Set runtime path of "/usr/local/bin/mpi_enc_test" to ""
-- Installing: /usr/local/bin/mpi_enc_mt_test
-- Set runtime path of "/usr/local/bin/mpi_enc_mt_test" to ""
-- Installing: /usr/local/bin/mpi_rc2_test
-- Set runtime path of "/usr/local/bin/mpi_rc2_test" to ""
-- Installing: /usr/local/bin/mpi_dec_multi_test
-- Set runtime path of "/usr/local/bin/mpi_dec_multi_test" to ""
-- Installing: /usr/local/bin/vpu_api_test

make && make install

hbiyik commented

Thats weird i will check tomorrow

@hbiyik You are right. ffmpeg calls an old mpp lib. I installed new mpp to directory /usr/local/lib, but there is an old mpp lib file in /usr/lib. That is why ffmpeg always calls an old one. After replacing the old file librockchip_mpp.so.0 with new one, transcoding is working well. Thank you a lot.