Buffering strategies
ddfreiling opened this issue · 19 comments
Is your feature request related to a problem? Please describe.
Buffering can take a while before playback begins. I therefore suggest the setting of a buffering strategy.
Describe the solution you'd like
It should be possible to use different buffering strategies for just_audio. Examples:
Default: As normal AVAudioPlayer, which buffers quite a bit before it starts playback
OnPreferredBufferDuration: Playback starts once the buffer has filled to a certain duration that the client can set
Immediately: Playback begins whenever the buffer is non-empty
Describe alternatives you've considered
Additional context
I implemented this exact feature for the iOS library KDEAudioPlayer, and could probably add it here. I noted some TODOs in this library about "figuring out how to detect buffering", which I'm not quite sure what meant. In the PR I made for KDEAudioPlayer with this exact feature, I just used the loadedTimeRanges event to check the buffer. See https://github.com/delannoyk/AudioPlayer/pull/109/files
Related to #127
Note that just_audio already provides the setAutomaticallyWaitsToMinimizeStalling method, but would be happy to consider more options like preferredForwardBufferDuration.
Not everything has a direct counterpart in Android's ExoPlayer (or the reverse), so it probably makes sense to have separate platform-specific APIs to control buffering behaviour on both platforms.
I could deprecate setAutomaticallyWaitsToMinimizeStalling and provide a setBufferingOptions as a replacement which takes a bundle of various platform-specific options like the aforementioned. We don't need to place a further abstraction around it with an enum of strategies, I think we can just directly expose each of the native parameters that can be configured on both platforms.
Agree that we should allow access to the platform-specific buffering options and setAutomaticallyWaitsToMinimizeStalling already covers the Immediately strategy on iOS.
There is some common ground for a preferredForwardBufferDuration that works on both platforms.
For Android we could use the ExoPlayer DefaultLoadControl to set a min and max buffer.
Here is a first draft of the configuration object:
class AudioLoadConfiguration {
final bool darwinAutomaticallyWaitsToMinimizeStalling;
final Duration darwinPreferredForwardBufferDuration;
final bool darwinCanUseNetworkResourcesForLiveStreamingWhilePaused;
final double darwinPreferredPeakBitRate;
final Duration androidMinBufferDuration;
final Duration androidMaxBufferDuration;
final Duration androidBufferForPlaybackDuration;
final Duration androidBufferForPlaybackAfterRebufferDuration;
final int androidTargetBufferBytes;
final bool androidPrioritizeTimeOverSizeThresholds;
final Duration androidBackBufferDurationDuration;
}Would you be happy with the above design?
Yes I think so. It seems like a good idea to give the client all the options available. Any options not provided will just use the platform defaults right?
Btw. can probably shave off a Duration from androidBackBufferDurationDuration :)
Yes I think so. It seems like a good idea to give the client all the options available. Any options not provided will just use the platform defaults right?
That would be reasonable.
Btw. can probably shave off a Duration from
androidBackBufferDurationDuration:)
Hey, it worked for Duran Duran, but fair enough ;-)
Please include targetLiveOffsetUs and LivePlaybackSpeedControl for streaming uses if possible.
from here: google/ExoPlayer@ab4c92e
@MichealReed those options look like great candidate options to add as well, although it might be better to wait until those features make it into a stable release.
I have a fork going to scout ahead (and use this early), will submit a PR when it's all said and done if you'd like and report back on any stability issues I encounter. Breaking changes were minimal.
Ah, so 2.13.1 is released (as of about a week ago).
In that case, I'd be happy to include those options, and yes would welcome a pull request!
Might be worth including this as well -- google/ExoPlayer#7685 not positive it works with dash, so it may be best to calculate from the request as mentioned.
Just merged your ExoPlayer version bump PR, @MichealReed . Would you be interested in contributing a PR for the proposed config parameters sans one Duration suffix, or shall I proceed with it?
How about you kick one off and I can merge in the dash stuff? Have not learned the package enough to understand the buffer fully, much less know where to put this.
I've implemented this on a local branch but I'm just thinking about distinguishing between buffering options that should be set on player instantiation vs options that can be switched any time.
On Android, they are all set during player instantiation. On iOS, they can in theory all be set at any time, so if this is something worth distinguishing, then they would not be bundled together into the same configuration object. Here are the iOS-specific options:
final bool darwinAutomaticallyWaitsToMinimizeStalling;
final Duration? darwinPreferredForwardBufferDuration;
final bool darwinCanUseNetworkResourcesForLiveStreamingWhilePaused;
final double? darwinPreferredPeakBitRate;I think the first two are most similar to the Android buffering options and could still perhaps go into the same configuration object. But the last two you could argue are useful to be able to turn on and off dynamically at the instruction of a user's preference.
So perhaps the first 2 should go into the configuration object passed into the AudioPlayer constructor, while the last two should be settable via independent methods.
Thoughts?
This API design decision is taking me a surprisingly long time to settle on. I want to at least decide on something that will be backwards compatible. Here is the configuration object I have in my current version:
/// Configuration options to use when loading audio from a source.
class AudioLoadConfiguration {
/// (iOS/macOS) Whether the player will wait for sufficient data to be
/// buffered before starting playback to avoid the likelihood of stalling.
final bool darwinAutomaticallyWaitsToMinimizeStalling;
/// (iOS/macOS) The duration of audio that should be buffered ahead of the
/// current position. If not set or `null`, the system will try to set an
/// appropriate buffer duration.
final Duration? darwinPreferredForwardBufferDuration;
/// (iOS/macOS) Whether the player can continue downloading while paused to
/// keep the state up to date with the live stream.
final bool darwinCanUseNetworkResourcesForLiveStreamingWhilePaused;
/// (iOS/macOS) If specified, limits the download bandwidth in bits per
/// second.
final double? darwinPreferredPeakBitRate;
/// (Android) The minimum duration of audio that should be buffered ahead of
/// the current position.
final Duration androidMinBufferDuration;
/// (Android) The maximum duration of audio that should be buffered ahead of
/// the current position.
final Duration androidMaxBufferDuration;
/// (Android) The duration of audio that must be buffered before starting
/// playback after a user action.
final Duration androidBufferForPlaybackDuration;
/// (Android) The duration of audio that must be buffered before starting
/// playback after a buffer depletion.
final Duration androidBufferForPlaybackAfterRebufferDuration;
/// (Android) The target buffer size in bytes.
final int? androidTargetBufferBytes;
/// (Android) Whether to prioritize buffer time constraints over buffer size
/// constraints.
final bool androidPrioritizeTimeOverSizeThresholds;
/// (Android) The back buffer duration.
final Duration androidBackBufferDuration;
/// (Android) The minimum playback speed to use when adjusting playback speed
/// to approach the target live offset, if none is defined by the media.
final double androidLiveFallbackMinPlaybackSpeed;
/// (Android) The maximum playback speed to use when adjusting playback speed
/// to approach the target live offset, if none is defined by the media.
final double androidLiveFallbackMaxPlaybackSpeed;
/// (Android) The minimum interval between playback speed changes on a live
/// stream.
final Duration androidLiveMinUpdateInterval;
/// (Android) The proportional control factor used to adjust playback speed on
/// a live stream. The adjusted speed is calculated as: `1.0 +
/// proportionalControlFactor * (currentLiveOffsetSec - targetLiveOffsetSec)`.
final double androidLiveProportionalControlFactor;
/// (Android) The maximum difference between the current live offset and the
/// target live offset within which the speed 1.0 is used.
final Duration androidLiveMaxLiveOffsetErrorForUnitSpeed;
/// (Android) The increment applied to the target live offset whenever the
/// player rebuffers.
final Duration androidLiveTargetLiveOffsetIncrementOnRebuffer;
/// (Android) The factor for smoothing the minimum possible live offset
/// achievable during playback.
final double androidLiveMinPossibleLiveOffsetSmoothingFactor;
AudioLoadConfiguration({
this.darwinAutomaticallyWaitsToMinimizeStalling = true,
this.darwinPreferredForwardBufferDuration,
this.darwinCanUseNetworkResourcesForLiveStreamingWhilePaused = false,
this.darwinPreferredPeakBitRate,
this.androidMinBufferDuration = const Duration(seconds: 50),
this.androidMaxBufferDuration = const Duration(seconds: 50),
this.androidBufferForPlaybackDuration = const Duration(milliseconds: 2500),
this.androidBufferForPlaybackAfterRebufferDuration =
const Duration(seconds: 5),
this.androidTargetBufferBytes,
this.androidPrioritizeTimeOverSizeThresholds = false,
this.androidBackBufferDuration = Duration.zero,
this.androidLiveFallbackMinPlaybackSpeed = 0.97,
this.androidLiveFallbackMaxPlaybackSpeed = 1.03,
this.androidLiveMinUpdateInterval = const Duration(seconds: 1),
this.androidLiveProportionalControlFactor = 1.0,
this.androidLiveMaxLiveOffsetErrorForUnitSpeed =
const Duration(milliseconds: 20),
this.androidLiveTargetLiveOffsetIncrementOnRebuffer =
const Duration(milliseconds: 500),
this.androidLiveMinPossibleLiveOffsetSmoothingFactor = 0.999,
});
}A superficial improvement I could make is to make the Android LoadControl and LivePlaybackSpeedControl data structures as nested structures rather than the current flat structure but this is not the most important consideration here. The main issue is whether to separate out the options that the app may want to dynamically change during playback.
The two iOS options I think could be useful to change dynamically could either be treated separately and only set via their own independent set methods, OR they could be included in both places, so you could also pass them into the constructor. I don't like putting the same option in two places, but by function, they sort of seem like they are options for loading audio.
I've created a dev branch where new features will be developed and added my first implementation of this to it.
My approach: I've split the configuration object into some nested options:
/// Configuration options to use when loading audio from a source.
class AudioLoadConfiguration {
final DarwinLoadControl? darwinLoadControl;
final AndroidLoadControl? androidLoadControl;
final AndroidLivePlaybackSpeedControl? androidLivePlaybackSpeedControl;
}This is passed into the AudioPlayer constructor as an optional parameter. The DarwinLoadControl class contains all 4 parameters discussed above:
/// Buffering and loading options for iOS/macOS.
class DarwinLoadControl {
final bool automaticallyWaitsToMinimizeStalling;
final Duration? preferredForwardBufferDuration;
final bool canUseNetworkResourcesForLiveStreamingWhilePaused;
final double? preferredPeakBitRate;
}But I have also added setters for the last two values on the player so they can be later changed at any time. The question is whether this redundancy is OK. It could be considered convenient to have it bundled into the above object if an app just wants to set everything once and be done with it. But it is also useful to be able to change the last two at any time.
Would appreciate some feedback on this approach.
This feature is now published in release 0.8.0.
This feature is now published in release 0.8.0.
Hey Ryan, that's awesome to hear! Sorry that I never got back to you on the design discussion, real life got in the way.
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs, or use StackOverflow if you need help with just_audio.