combine "generating variations" and "unsampling"

[ganzinotti]

I use your nodes a lot. There are many applications for both of your example workflows "generating variations" and "unsampling". I really would like to combine both of these workflows to start with an image, unsample to get noise that relates to the original image (1x), duplicate the output of the unsampler (8x), generate different latent noise vectors (8x), slerp the two together, run them through the ksampler, get 8 different images that all relate to the original images with some variety, pick the best looking image.

If I put the slerp factor to 1.00 (discard the output of the unsampler) it correctly generates 8x images that have no relation with the original image:

Any other slerp factor that gives some positive weight to the output of the unsampler returns 8 identical noisy outputs:

If I remove the latent noise/slerp/noise inject the workflow functions as expected:

The reason I want to do this is to get a lot more control over inpainting, which is in my experience a weakness of comfyui. I would expect this to work. Is this idea even possible? Is the execution wrong?