WebVTT: Text cues split over multiple segments with different timing causing duplicate subtitles

[Hyddan]

What version of Hls.js are you using?

Latest (1.5.15)

What browser (including version) are you using?

Latest Chrome (128.0.6613.114)

What OS (including version) are you using?

Windows 11

Test stream

https://storage.googleapis.com/shaka-demo-assets/angel-one-hls/hls.m3u8

Configuration

{}

Additional player setup steps

hls.once(Hls.Events.MANIFEST_LOADED, (e, data) => {
  hls.subtitleTrack = 2;
});

Checklist

• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
• The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

Select the french subtitles of this stream (id: 2) and at the 4 second mark the subtitle will be duplicated.

Minimal reproduction available here:
https://hyddan.github.io/hls.js-duplicate-cues-reproduction/

Expected behaviour

I'm not sure if this should be a feature request rather than a bug, and I'm also not entirely sure what is the correct behavior here so if this is already working as it should then feel free to close this.. But it seems like when this subtitle was split over two segments the start time of it was altered - presumably clamped to the start time of the segment as the manifest states that it has 4-second segments.
This however means that the duplicate is not caught by hls.js's current duplication-catcher method which relies on the ids generated from the start and end times.

In my own setup, I have 'solved' this by matching on text content and, if found, modifying the times on the existing cue instead of adding it.

const cueByTextContent = findCueByTextContent(track, cue.text);
if (cueByTextContent) { // Adjust times instead of adding
  cueByTextContent.startTime = Math.min(cueByTextContent.startTime, cue.startTime);
  cueByTextContent.endTime = Math.max(cueByTextContent.endTime, cue.endTime);
}
else {
  track.addCue(cue);
}

Though I am not sure this would work well enough in a general case.

What actually happened?

Subtitles are duplicated.

Console output

N/A

Chrome media internals output

N/A

[robwalch]

The VTT described, with overlapping times (please confirm if the overlap or this is a rounding error), are being rendered as expected. Adding a matching identifier to both VTT time range might work to deduplicate the cues but still would not result in merging of the time ranges.

The correct segmentation for these subs would be to either clip the overlapping time values (a should end at the time that b starts) or make both entries have the same start and end time (cues can extend beyond the start and end of a segment time range without being considered invalid).

This must be done in the WebVTT files. This is not something the client (hls.js) should do as it would result in incorrect rendering of properly segmented VTT where the repeating overlap is expected.

[Hyddan]

@robwalch in this case they don't overlap actually and it is not a rounding error.

In this particular case the the timing in the first segment is 00:00:03.837 --> 00:00:07.300 and in the second 00:00:04.000 --> 00:00:07.300 (presumably this start time is clamped to the start time of the segment), so here it would act more like deduplication instead of extending the range.

But the overlap case was what I mostly had in mind could be helped by merging the ranges. I can however agree that it is not hls.js' responsibility to handle that, especially since it is already possible to handle on the outside by disabling renderTextTracksNatively so we can close this.