[linked_scroll_controller] Possibility to utilize Key-mechanics within a dynamically created List

[Eerey]

From the pub.dev main page:
If you add controllers dynamically, the corresponding scrollables must be given unique keys to avoid the scroll offset going out of sync.
Within my Scrollable (ListView or SingleChildScrollView) I do not want to use a UniqueKey since it will ultimately rebuild all the subsequent Widgets within it.
If I use a TweenAnimationBuilder that relies on a ValueKey within the Scrollable, it will never work because the UniqueKey will force a redraw of all subsequent Widgets.

Note: Yes, I create the controllers dynamically.

Is there any possibility to utilize the Key-mechanics in Flutter within my Scrollable or is this the drawback of using the linked scroll controller package (https://pub.dev/packages/linked_scroll_controller)?

Here is a part of the code:

// Stateful widget class
  LinkedScrollControllerGroup group = LinkedScrollControllerGroup();

  List<ScrollController> ctrls = <ScrollController>[];

  @override
  void dispose() {
    for (final ctrl in ctrls) {
      ctrl.dispose();
    }
    super.dispose();
  }


// ...
ListView.builder(
  itemBuilder: (context, index) {
    final ctrl = group.addAndGet();
    ctrls.add(ctrl);
    return Column(
      crossAxisAlignment: CrossAxisAlignment.start,
      children: [
        widget.header.call(context, widget.entries[index]),
        SizedBox(height: widget.gap),
        SingleChildScrollView(
          key: ValueKey(index),
          scrollDirection: Axis.horizontal,
          controller: ctrl,
          child: IgnorePointer(
            child: widget.slider
                .call(context, widget.entries[index]),
          ),
        ),
      ],
    );
  },
// ...