Problems with audio playback

[wenbingzhang]

11.mp4

[valbok]

Hi, thanks for your contribution, could you please explain a bit more,how you use and what is wrong?
What is Qt version?
There are 2 steps

1. Demuxing audio frames
2. Playing them using QAudioSink

For demuxing, it is important that it is sending the frames fast enough. It is easy to check on you callback audioFrame to see the fps. Or to print progress:

for (const auto &s : p.availableAudioStreams())
    qDebug() << s << p.progress(s);

You could also print frame.pts() and see if the samples or frames are skipped by mistake.

For QAVAudioOutput you could try to change buffer size: https://github.com/valbok/QtAVPlayer/blob/master/src/QtAVPlayer/qavaudiooutput.h

[wenbingzhang]

Hi, thanks for your contribution, could you please explain a bit more,how you use and what is wrong? What is Qt version? There are 2 steps

1. Demuxing audio frames
2. Playing them using QAudioSink

For demuxing, it is important that it is sending the frames fast enough. It is easy to check on you callback audioFrame to see the fps. Or to print progress:

for (const auto &s : p.availableAudioStreams())
    qDebug() << s << p.progress(s);

You could also print frame.pts() and see if the samples or frames are skipped by mistake.

For QAVAudioOutput you could try to change buffer size: https://github.com/valbok/QtAVPlayer/blob/master/src/QtAVPlayer/qavaudiooutput.h

I copied this audio from another video, and the sound freezes when played.

[wenbingzhang]

Hi, thanks for your contribution, could you please explain a bit more,how you use and what is wrong? What is Qt version? There are 2 steps

1. Demuxing audio frames
2. Playing them using QAudioSink

For demuxing, it is important that it is sending the frames fast enough. It is easy to check on you callback audioFrame to see the fps. Or to print progress:

for (const auto &s : p.availableAudioStreams())
    qDebug() << s << p.progress(s);

You could also print frame.pts() and see if the samples or frames are skipped by mistake.

For QAVAudioOutput you could try to change buffer size: https://github.com/valbok/QtAVPlayer/blob/master/src/QtAVPlayer/qavaudiooutput.h

Setting "BufferSize" does not improve the lag situation

[wenbingzhang]

for (const auto &s : p.availableAudioStreams())
    qDebug() << s << p.progress(s);

pts

[valbok]

I copied this audio from another video, and the sound freezes when played.

Playing only audio? Does it have video stream? It might try to synchronize audio and video frames and no video frames are sent.

[wenbingzhang]

I copied this audio from another video, and the sound freezes when played.

Playing only audio? Does it have video stream? It might try to synchronize audio and video frames and no video frames are sent.

Yes, audio only. But the same goes for playing audio and video.

[valbok]

Yes, audio only. But the same goes for playing audio and video.

I downloaded the file, built qml_video example
$ ./qml_video 289002226-f65252df-a02c-4877-b317-84c08993a4a2.mp4

And it works

[valbok]

Playing from the network also works
$ ./qml_video https://private-user-images.githubusercontent.com/13162606/289002226-f65252df-a02c-4877-b317-84c08993a4a2.mp4?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTEiLCJleHAiOjE3MDI4MTU3NDksIm5iZiI6MTcwMjgxNTQ0OSwicGF0aCI6Ii8xMzE2MjYwNi8yODkwMDIyMjYtZjY1MjUyZGYtYTAyYy00ODc3LWIzMTctODRjMDg5OTNhNGEyLm1wND9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFJV05KWUFYNENTVkVINTNBJTJGMjAyMzEyMTclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjMxMjE3VDEyMTcyOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWFlZWJlYTU2ZmM5ZDNkZDhjODg5OTU1YjIyNDBmNWQ4MTMxMzc1MDk0MGU5YzdjOTJiYjU4MTY4MDI1Njc2ODMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.-Yia1mwihJnX9XM1QvT6VVgslYsSrf35nmz3mi5c-48

[valbok]

Tested Qt5.15 and Qt6.6 both works

durationChanged 88880
mediaStatusChanged LoadedMedia StoppedState
Video streams: 0
Audio streams: 1
[ 0 ] QMap(("handler_name", "#Mainconcept MP4 Sound Media Handler")("language", "eng")("vendor_id", "[0][0][0][0]")) 4167 frames, 0 frame rate ---current
Subtitle streams: 0
stateChanged PlayingState LoadedMedia
mediaStatusChanged EndOfMedia StoppedState
QAVStream(0) Progress(88.8747/88.88 pts, 4167/4167 frames, 0.0213232/0 frame rate, 46 fps

[valbok]

Tested on Linux.
Do you use Windows?

[wenbingzhang]

Tested on Linux. Do you use Windows?

macos qt5

[wenbingzhang]

https://drive.google.com/file/d/1s7Pf95dHclYEcsB8_hXeOqJyk5r-fytR/view?usp=sharing

This is a complete file, the audio sound is noisy, and the sound and picture are not synchronized.

[wenbingzhang]

11.mp4

Can play, but there is a problem with the sound quality.

[valbok]

Ok, maybe I hear some noize on Mac, checked Qt 5 and 6,

1. Using audioOutput.setBufferSize(128*1024); can't hear it, you can play with the size of the buffer. But looks it is not enough bytes in the buffer.
2. Make sure that you use Qt::DirectConnection when receiving the audio frames.
3. Compare with QMediaPlayer:

QMediaPlayer mp;
mp.setSource(file);
auto audioOutput = new QAudioOutput;
mp.setAudioOutput(audioOutput); 
mp.setVideoOutput(videoSurface);
mp.play();

4. Don't see any issues with syncing audio and video frames.

[wenbingzhang]

Ok, maybe I hear some noize on Mac, checked Qt 5 and 6,

1. Using audioOutput.setBufferSize(128*1024); can't hear it, you can play with the size of the buffer. But looks it is not enough bytes in the buffer.
2. Make sure that you use Qt::DirectConnection when receiving the audio frames.
3. Compare with QMediaPlayer:

QMediaPlayer mp;
mp.setSource(file);
auto audioOutput = new QAudioOutput;
mp.setAudioOutput(audioOutput); 
mp.setVideoOutput(videoSurface);
mp.play();

4. Don't see any issues with syncing audio and video frames.

audioOutput.setBufferSize(128*1024);
Solved the problem I mentioned earlier

[valbok]

Hello, there is some fixes landed #464 to help to make more smooth audio playback, could you please confirm that it does not brake anything for you? thanks