Wrong FPS when recording MP4 to local storage

[NerdySouth]

Describe the bug
After upgrading from v2.4.8 to v.2.5.0 (and also v2.5.4), video's recorded (with audio) on our devices use the incorrect FPS value.

To Reproduce
Steps to reproduce the behavior:

1. Use GenericStream, set to 720x720, 550 kbps bitrate, 15 fps with MicrophoneSource and Camera2Source.
2. Record to mp4 on local 'external' storage
3. run ffprobe -i on final video to get average fps and tbr
4. See error

Expected behavior
That fps from ffprobe reflects the fps set on the device when calling 'prepareVideo'

Smartphone (please complete the following information):

• Library version 2.4.8 AND 2.5.4
• Device: Motorola Razr+ 2023
• OS: Android 13 and Android 14
• Media server Nginx-rtmp
• Class used: GenericStream, Camera2Source, MicrophoneSource

Additional context
We passed prepareVideo a value of 15 fps, which has always worked for us. It results in a video which has ~15 fps and ~15 tbr when running ffprobe -i on the resulting mp4 file. An example output from this ffprobe command from a video recorded with version 2.4.8 is:

Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt470bg/bt470bg/smpte170m, progressive), 720x720, 558 kb/s, **14.97 fps, 14.92 tbr**, 90k tbn (default)

Which we can see has a resolution of 720x720, with 14.97 fps and 14.92 tbr. All of this is correct.

After upgrading the library version to 2.5.4 (and later downgrading to 2.5.0), videos now have ~30 fps and ~30 tbr, despite setting the fps to 15 in prepareVideo. An example output on a video recorded with this version of the library would be:

Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt470bg/bt470bg/smpte170m, progressive), 720x720, 558 kb/s, **29.89 fps, 29.93 tbr,** 90k tbn (default)

Which looks correct, except the video SHOULD be 15 fps, not 30. This has an impact on the battery life for the devices and deployment we have.

All tests and code uses the GenericStream from StreamBase, a Camera2Source with preview stopped and Microphone source.

[pedroSG94]

Hello,

Try this gradle:

  implementation 'com.github.pedroSG94.RootEncoder:library:ec02c81361'

This is the last commit in master and should fix your problem. I have plan to upload a release after receive a confirmation of a fix.

[NerdySouth]

So, after using the above commit, when my app calls startRecord() i get this error and crash:

FATAL EXCEPTION: DefaultDispatcher-worker-16 Process: com.plix.plix_test_v2, PID: 19121 java.lang.NullPointerException: getInputSurface(...) must not be null at com.pedro.library.base.StreamBase.startSources(StreamBase.kt:502) at com.pedro.library.base.StreamBase.startRecord(StreamBase.kt:257) at com.plix.plix_test_v2.PlixStreamManager$startRecording$2.invokeSuspend(PlixStreamManager.kt:648) at com.plix.plix_test_v2.PlixStreamManager$startRecording$2.invoke(Unknown Source:8) at com.plix.plix_test_v2.PlixStreamManager$startRecording$2.invoke(Unknown Source:4) at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:42) at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:164) at kotlinx.coroutines.BuildersKt.withContext(Unknown Source:1) at com.plix.plix_test_v2.PlixStreamManager.startRecording(PlixStreamManager.kt:637) at com.plix.plix_test_v2.MainActivity$onStart$3$1.invokeSuspend(MainActivity.kt:1075) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:101) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:113) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:589) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:823) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:720) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:707) Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@6d4ea38, Dispatchers.IO]

Which i was never getting before. Before calling startRecord(), i call my own init() and prepareStreams() functions which do this:

fun init() {
        this.window = (context as? Activity)?.window
        recordCamSource.enableLantern()
        recordStream = GenericStream(
            this.context, this, recordCamSource, MicrophoneSource()
        ).apply {
            getGlInterface().autoHandleOrientation = false
            forceFpsLimit(true)
        }
        liveStream = GenericStream(this.context, this, NoVideoSource(), MicrophoneSource()).apply {
            getGlInterface().autoHandleOrientation = false
        }

        prepareStreams()
        CoroutineScope(networkDispatcher).launch {
            loadState()
        }

        retryEscalationTimer.start()
        retryUploadTimer.start()
        synchronized(initLock) {
            initialized = true
        }
    }

fun prepareStreams() {
        try {
            recordStream.forceFpsLimit(true)
            liveStream.prepareAudio(
                sampleRate,
                isStereo,
                aBitrate
            ) && liveStream.prepareVideo(
                width,
                height,
                vBitrate,
                0,
                2,
                90,
                -1,
                -1
            ) && recordStream.prepareVideo(
                width,
                height,
                vBitrate,
                fps,
                2,
                90,
                -1,
                -1
            ) && recordStream.prepareAudio(sampleRate, isStereo, aBitrate)
        } catch (e: IllegalArgumentException) {
            Log.e(TAG, "onStart: Failed to prepare streams.")
        } catch (e: IllegalStateException) {
            // already prepared
            Log.e(TAG, e.stackTraceToString())
            return
        }

        // turn off preview
        liveStream.stopPreview()
        recordStream.stopPreview()

        synchronized(prepareLock) {
            prepared = true
        }

You can ignore the liveStream object, since recordStream is the only one which handles recording locally. Any ideas why this error just started to surface?

[pedroSG94]

Hello,

I think that the problem is that you are receiving an error in prepareVideo here:

liveStream.prepareVideo(
                width,
                height,
                vBitrate,
                0,
                2,
                90,
                -1,
                -1
            )

The reason is because I added sanity checks in the parameters, Now set fps to 0 is not a valid value. You should set a valid value, the equivalent to previous versions is 30fps
I recommend you check again that method to avoid call startRecord/startStream if the prepare fail

[NerdySouth]

That seems to have been it! I had used that as a quick hack since that object only sometimes needs video, but mostly is audio-only. Not a huge deal to set it to 1 fps. Thanks for the help! FPS and TBR values reported by ffprobe are now consistent with what values i set again.

[pedroSG94]

Closing as solved. You can open other issue if you need it.