[Feature Request] Let's settle the disagreement regarding "Disperse Siblings" with the community

Which module is related to your feature request?
Helper add-on

Is your feature request related to a problem? Please describe.
Right now, Disperse Siblings can bring siblings closer together in some cases, in other words, the distance between their due dates (in days) can decrease. This is undesirable for two reasons:

It confuses the users since they expect that siblings will only become spaced further away from each other once this feature is enabled.
It defeats the point of this feature, which is to amplify the spacing effect for all siblings.

Describe the solution you'd like
Ensure that Disperse Siblings can only keep the distance (in days) between two cards the same or increase it, but cannot decrease it.

Additional context
Me and @L-M-Sherlock have been arguing about this for quite some time now, so I decided to make a Google Form survey. I will post it on r/Anki, if Sherlock isn't against it. We will let all FSRS users (well, those of them who use Reddit, I guess) vote on how to handle this.
@user1823 @AnKingMed @dae https://forms.gle/a8CyrYK9EpZYFszSA

In my opinion, the only case where the current implementation decreases the gap is the case when this decrease is necessary to increase the gap from the last review date (which is equally important in my opinion).

You can fill out my survey and choose "Other" for the last question.

The way you have written about the issue, no one will select the second or third option (unless they know why the add-on can sometimes decrease the spacing).

Then, would you make a better write-up? Preferably, one that explains why the current implementation should remain unchanged in layman's terms.

The Disperse Siblings feature is designed to maximize

the gap between the due dates of the siblings; and
the gap between the latest review date of the siblings and the current due dates of the siblings.

The function also ensures that the due dates of all the siblings stay within their respective fuzz ranges, preventing any card from diverging too much from its optimal interval. This is crucial for maintaining the balance between sibling dispersion and retention.

Occasionally, the function can bring the due dates of siblings closer together rather than dispersing them. This occurs in an attempt to increase the gap from the latest review date, which is equally important.

The issue of siblings being brought closer could theoretically be solved by allowing dispersion beyond their fuzz ranges. However, this approach introduces several potential problems:

It's unclear how much beyond the fuzz range would be acceptable. Even if an arbitrary amount is chosen, the same issue of siblings being brought closer together could still arise.
Moving cards significantly beyond their fuzz range could negatively affect their retrievability.
If one sibling is being reviewed frequently at short intervals, allowing unlimited dispersion could result in other siblings being delayed multiple times. This could significantly affect their retrievability and may even cause them to be never shown to the user.

In the light of these considerations, changing the way the feature works can potentially introduce more problems than it solves. The current design strikes a careful balance between maximizing dispersion and maintaining optimal review intervals, which is crucial for effective learning. While it may occasionally bring siblings closer together, this is a necessary trade-off to ensure the overall effectiveness of the feature. Therefore, it is not prudent to change how it works.

The above comment can also address the concerns that @dae expressed in the discussion about adding the feature to Anki.

I think your argument makes sense @user1823. Sounds like this is just a few small cases as well

Ok, I changed the text of the final question. Everyone, please participate: https://forms.gle/a8CyrYK9EpZYFszSA

I suggest adding the following text to the bottom (just above the question):

The current design tries to strike a careful balance between maximizing dispersion and maintaining optimal review intervals.

It already says "This is crucial for maintaining the balance between sibling dispersion and retention.", I think your suggestion is redundant.

3. If one sibling is being reviewed frequently at short intervals, allowing unlimited dispersion could result in other siblings being delayed multiple times. This could significantly affect their retrievability and may even cause them to be never shown to the user.

Btw, this could be solved with a tagging system. The add-on could do the following:

Add a "Dispersed" tag to a card
Check if a card has that tag once it's about to be dispersed
If it has: don't disperse it
if it doesn't have the tag: disperse it
if it has a tag and is being reviewed: remove the tag

This ensures that a card cannot be dispersed twice without being reviewed inbetween those dispersals.

@L-M-Sherlock the latest response to my survey was submitted yesterday, so it seems that everyone who saw my reddit post/forum post and cared about FSRS has already participated, and there probably won't be more responses.
Here are the results:
25 people have voted for keeping the current implementation.
16 people have voted for changing the implementation.
1 person has offered a unique answer: "If siblings are dispersed once, they shouldn’t be dispersed again until hitting an arbitrsry threshold. Maybe this could work better? I personally cannot afford problem 3, and I have experienced the siblings being moved closer together multiple times. If a dispersed sibling is lapsed, there should be some sort of failsafe or other to prevent the non-lapsed sibling from getting ignored due to constant dispersal."

I got an idea to solve this issue, at least partially.

To ensure that this can be implemented easily and doesn't cause performance issues, we should restrict this to the cases where the number of siblings per note is small (say, less than 4). This should cover most cases, though.

The idea:
If the function is causing the gap between any two siblings to decrease, we can increase the fuzz for that note from 5% to 10%. Since we will now have a larger fuzz range, the chances of siblings being closer to one another decrease.

The chances that the siblings will still be closer to each other than before dispersing (issue 1 that I mentioned before) are low because we are enlarging the fuzz range only in the cases of small gap, not for all cards.

By the way, I noticed that Anki uses larger fuzz ranges. So, this issue may be caused, in part, by the use of shorter fuzz ranges in the add-on.

Fuzz ranges in Anki:

static FUZZ_RANGES: [FuzzRange; 3] = [
    FuzzRange {
        start: 2.5,
        end: 7.0,
        factor: 0.15,
    },
    FuzzRange {
        start: 7.0,
        end: 20.0,
        factor: 0.1,
    },
    FuzzRange {
        start: 20.0,
        end: f32::MAX,
        factor: 0.05,
    },
];

https://github.com/ankitects/anki/blob/77cb3220c5e83206725b1f3f2fbdb536dc180ec2/rslib/src/scheduler/states/fuzz.rs#L16-L32

Fuzz ranges in the add-on:

fsrs4anki-helper/utils.py

Lines 107 to 113 in d252035

    
           def get_fuzz_range(interval, elapsed_days, maximum_interval): 
        
               min_ivl = max(2, int(round(min(interval, maximum_interval) * 0.95 - 1))) 
        
               max_ivl = min(int(round(interval * 1.05 + 1)), maximum_interval) 
        
               if interval > elapsed_days: 
        
                   min_ivl = max(min_ivl, elapsed_days + 1) 
        
               min_ivl = min(min_ivl, max_ivl) 
        
               return min_ivl, max_ivl

If you decide to use Anki's fuzz ranges in the add-on, don't remove the lines 110-112 in the add-on code until ankitects/anki#2694 is done.

I like user1823's idea. Let's keep the fuzz range at 5%, but extend it to 10% when necessary:

If two siblings are to be dispersed, use 5% as the fuzz range
If it doesn't cause their due dates to become closer - good
If it does - try a 10% fuzz range instead
If the problem is solved - good
If the siblings have to be brought closer even when the fuzz range is 10%, then there is nothing we can do, let them become closer

@L-M-Sherlock please read the comments in this issue, especially the results of the survey and user1823's idea of extending the fuzz range only when necessary.

If it doesn't cause their due dates to become closer - good

How to determine the due dates to become closer? Does it include the last due dates?

I don't know what you mean by "last due dates".

To determine whether the due dates become closer, calculate the difference between the due dates of the cards before dispersing and after dispersing. Then, see if the difference increased or decreased.

I guess that by "last due date", you mean the "last review date".

If it is so, then no, we don't have to consider the last review date. By due date, I mean the date on which the card is currently scheduled to be reviewed again.

What if there are three siblings?

The original due date:
A: 2023-12-01
B: 2023-12-05
C: 2023-12-30

The new due date:
A: 2023-12-01
B: 2023-12-15
C: 2023-12-30

It's getting farther between A and B. But B and C becomes closer.

In this case, we should increase the fuzz range of B & C but not A.

However, there is not much problem if the fuzz range of all the three cards increases, if it is easier to implement it that way.

OK. I need to redesign the algorithm now. The current implementation is binary searching the maximum minimum interval of adjacent siblings and checking whether the interval could respect the fuzz range.

fsrs4anki-helper/schedule/disperse_siblings.py

Lines 318 to 328 in 247e94e

    
           while min_gap <= max_gap: 
        
               mid_gap = (min_gap + max_gap) // 2  # Compute the middle gap 
        
               can_place, temp_arrangement = can_place_points_with_arrangement(points, mid_gap) 
        
               if can_place: 
        
                   # If we can place all points with this gap, it means we can try to increase it 
        
                   best_gap = mid_gap 
        
                   arrangement = temp_arrangement  # Update the best arrangement 
        
                   min_gap = mid_gap + 1 
        
               else: 
        
                   # If we can't place all points with this gap, it means we need to try a smaller gap 
        
                   max_gap = mid_gap - 1

I find that the later-due siblings all should be delayed once the former-due sibling is delayed because we don't allow any siblings become closer to each other. So the new algorithm will be very different than the original one.

If the siblings are still dispersed within their (expanded) fuzz ranges, I think that it would be OK.

However, I think that the latest change (adopting Anki's fuzz ranges) should already make the situation much better.

So, in my opinion, @Expertium should try it for a few days before implementing my idea.

Ok

Note that the add-on is not updated on AnkiWeb. So, you need to download the latest code from the repo.

Nope, I still get cards spaced only by 1 day. Though it seems like they are cards that have been reviewed a long time ago, and not recently.

Suppose there are three siblings A, B, and C. Today, card A is due for review, which I recall it. Now, it's time to disperse the related cards. There are three possbile cases:

Card A is scheduled before B and C.
Card A is scheduled between B and C.
Card A is scheduled after B and C.

In the first case, both B and C need to be postponed. The number of days they are postponed is determined by the card with the shortest postponement window (determined by their fuzz range and current due date). For instance, if B can be delayed by 3 days and C by 2 days, then both B and C can be postponed by a maximum of 2 days.

In the second case, both A and C need to be postponed.

In the third case, only card A needs to be postponed.

Is it correct? What do you think of it?

You are still considering the latest review date as the pseudosibling. Right?

If so, in the second and the third case, all the three cards may need to be postponed depending on their fuzz ranges and current due dates.

@L-M-Sherlock how's the progress?

No progress. I have a paper deadline to catch up.

@L-M-Sherlock How is the paper doing?

I'm anxious with it because my experiments in reinforcement learning didn't going well.

I'm developing a notebook in https://github.com/open-spaced-repetition/disperse-siblings-simulator

@Expertium, I think that your issue can be explained by https://www.reddit.com/r/Anki/comments/19e0x2l/comment/kjapnru/.

You may be having the issue because you don't reschedule your cards after changing the parameters. I have never experienced this issue and that's probably because I always reschedule all cards after changing the parameters.

I have added a fix for this case. Update the add-on and try it for a few days to see if the issue has been solved or not.

@Expertium, I think that we can close this issue now.

	def get_fuzz_range(interval, elapsed_days, maximum_interval):
	min_ivl = max(2, int(round(min(interval, maximum_interval) * 0.95 - 1)))
	max_ivl = min(int(round(interval * 1.05 + 1)), maximum_interval)
	if interval > elapsed_days:
	min_ivl = max(min_ivl, elapsed_days + 1)
	min_ivl = min(min_ivl, max_ivl)
	return min_ivl, max_ivl

	while min_gap <= max_gap:
	mid_gap = (min_gap + max_gap) // 2 # Compute the middle gap
	can_place, temp_arrangement = can_place_points_with_arrangement(points, mid_gap)
	if can_place:
	# If we can place all points with this gap, it means we can try to increase it
	best_gap = mid_gap
	arrangement = temp_arrangement # Update the best arrangement
	min_gap = mid_gap + 1
	else:
	# If we can't place all points with this gap, it means we need to try a smaller gap
	max_gap = mid_gap - 1