HLS videos do not start on slow connections

Describe the bug

HLS videos do not play on a slow internet connection or with artificial throttling enabled.

To Reproduce

Steps to reproduce the behavior:

Go to https://paellaplayer.upv.es/demos/multi-quality-hls/
Open browser's developer tools
Enable throttling at "Fast 3G" (Chrome) or "Regular 3G" (Firefox)
Reload page
Click video

The loading animation plays endlessly.

Workaround

Comment out this line:

paella/plugins/es.upv.paella.hlsPlayer/hls-player.js

Line 208 in a34c2e1

setTimeout(() => this._hls.currentLevel = -1, 1000);

Change the following value to -1:

paella/plugins/es.upv.paella.hlsPlayer/hls-player.js

Line 15 in a34c2e1

initialQualityLevel: 1,

Environment Information

OS: Windows 10, macOS 11.5
Browser: Chrome, Firefox, Safari
Browser Version: (Chrome) 98.0.4758.82, (Firefox) 91.5.1, (Safari) 14.1.2
Paella version: 6.5.4, 6.5.5

Additional context

Since we upgraded Opencast to 10.9 (Paella 6.4.3 -> 6.5.5) we have a few users that can't play videos anymore. We couldn't reproduce their problems. We did notice that many of the users having this problem were abroad. So I tested with a VPN set in another continent and could reproduce the problem this way. Then I tested throttling the connection artificially. That way I could reproduce the problem, too.

I don't understand why the line that I commented out as a workaround is necessary to begin with. The comment in the source code only says:

paella/plugins/es.upv.paella.hlsPlayer/hls-player.js

Lines 206 to 208 in a34c2e1

    
           // Fixes hls.js problems when loading the initial quality level 
        
           this._hls.currentLevel = this._hls.levels.length>=initialQualityLevel ? initialQualityLevel : -1; 
        
           setTimeout(() => this._hls.currentLevel = -1, 1000);

And the commit where this got added (bd81d5ed1) doesn't explain that either.

@ferserc1 since you wrote this section of code, can you explain what it was for exactly? Is it safe to remove that one line? Do you know a better way to fix this?

Unfortunately when I comment out that one line Paella no longer changes quality levels automatically. So there has to be a better solution.

I tested this again today with the HLS video on your demo page and there I can no longer reproduce this bug. It does still occur with our setup (Paella 6.5.5 - build: 9888808 + Wowza streaming server) though.

Hi Tim

Thanks for your mail. For the demos we have a Wowza 4.8.15 with an VOD app for HLS, where we customized these parameters:

/Root/Application/LiveStreamPacketizer cupertinoChunkDurationTarget Integer 500
/Root/Application/LiveStreamPacketizer cupertinoMaxChunkCount Integer 20
/Root/Application/LiveStreamPacketizer cupertinoPlaylistChunkCount Integer 3

What is what you have in your setup?

Regarding the comment about the codehttps://github.com/polimediaupv/paella/blob/a34c2e1e46dab49bfee582dd2b61d9a766f926ae/plugins/es.upv.paella.hlsPlayer/hls-player.js#L206-L208

The this._hls.currentLevel = -1 sets the hls quality change to "auto". Valued levels set that level statically in the array loaded from the hls manifest

There were some cases in which the level was loaded before the manifest was ready, so there is a little delay and then it is set to auto.

Maybe we could provide our demo mp4 file to load in your wowza and see what happens

Forget all that I said. Could you try if changing the 1000 by 5000 this solves the problem?. My guess is that it is the same problem that we had but we need to wait longer

setTimeout(() => this._hls.currentLevel = -1, 5000);

Hi Carlos,

thanks for looking into this. I've sent you a link to a test video on our site via email. I tested increasing the delay to 5s but that doesn't work for me. Increasing it to 10s does work. But either way without throttling the video then always hangs because it reloads some chunks as soon as you reset this._hls.currentLevel to -1.

It seems like the problem occurs when you set this._hls.currentLevel = -1 while the first chunk is loading.

Hi,

here in Cologne (same hardware as Aachen and using still OC 9 with Paella Version 6.4.4 / 6.5.2 in production) after some time we had the same problem on our test-cluster with OC 10 (I think with 6.5.5). The same problem persists on our dev-cluster with OC 11 and Paella 7. Everything works quite well as long as I'm in my office at the university but not at home.

On the other side we have no problems with players like VideoJS or Plyr using the same playlist generated by Wowza. They start playing a video without any delay or other problems like stuttering.

A quick note on the recommended Wowza parameters:

/Root/Application/LiveStreamPacketizer cupertinoChunkDurationTarget Integer 500 
/Root/Application/LiveStreamPacketizer cupertinoMaxChunkCount Integer 20
/Root/Application/LiveStreamPacketizer cupertinoPlaylistChunkCount Integer 3

Besides the first parameter, all others only belong to live video configuration and do not affect VOD or can be set for VOD.
We have set this Parameter to 6,000 ms as recommended by Apple.

/Root/Application/HTTPStreamer cupertinoChunkDurationTarget Integer 6000

Hopefully this issue can be fixed as students watch their videos from home rather than on campus

Best
Ruth

We ( Ghent University ) are experiencing the same issue, we'll try setting the cupertinoChunkDurationTarget to 6000 as well to see if this helps.

I've been digging with @ferserc1 into the issue. The line
setTimeout(() => this._hls.currentLevel = -1, 1000);

It is related to a feature of paella in which the video starts in high quality. In a standard hls connection, you start with the quality level 0 (the lowest). As I said, -1 means auto

When you remove the line, it goes to manual, because there is no auto setting, but it can be defined in the config.json file

So, any of you could try to both remove the line and change the initialQualityLevel parameter of config.json to -1 ?

Hi Carlos,

i will test this. For thursday we will now set the following Options:

cupertinoChunkDurationTarget to 6000
remove the Line setTimeout(() => this._hls.currentLevel = -1, 1000);
set "initialQualityLevel": -1 in config.json

Hi Carlos,

I've tried your suggestion and that fixes the problem. The video starts even when throttling and it auto-adjusts the quality.

The only downside: On faster connections it now takes longer for the video to start playing after you click Play. Now it loads the first chunk in the lowest quality, switches quality automatically, loads the new chunklist and loads the first chunk again in a higher quality (= larger file) before it starts video playback. With 10s chunks this means waiting ~3.5s longer for the video to start. This could be solved by using smaller chunks. Ideally the video would consist of 500ms chunks in the beginning and the rest would be larger chunks. But I don't think Wowza can do that.

Well, at least this is an improvement. Does that work also when cupertinoChunkDurationTarget = 1000 with 3G settings?

Anyway, today we thought of a new idea to start faster; we are going to test and let's see

Fixed in 6.5.6

	// Fixes hls.js problems when loading the initial quality level
	this._hls.currentLevel = this._hls.levels.length>=initialQualityLevel ? initialQualityLevel : -1;
	setTimeout(() => this._hls.currentLevel = -1, 1000);