sarafridov/K-Planes

Temporal difference (IST) strategy

Closed this issue · 4 comments

xuyaojian123 commented

image

could you explain why temporal difference (IST) strategy cannot be used for monocular videos?

sarafridov commented

The temporal difference strategy works like this: Given a fixed-position camera, compare pixel values over time to detect motion, and then prioritize training with those pixels that change value. If the camera is moving between different timesteps, then you can't discover and prioritize motion by doing pixel-wise temporal differencing because all the pixels will exhibit camera motion.

xuyaojian123 commented

Thanks your reply. You may understand me wrong, I understand how IST is calculated, I want to ask why IST does not apply to monocular videos (only one fixed camera)?

sarafridov commented

Ah, usually when people say "monocular videos" in the context of radiance field modeling, they mean a single camera that moves around over time. If there is only one camera and it doesn't move, then you don't get enough information for a volumetric reconstruction (at least, not without relying on very strong depth priors).

xuyaojian123 commented

Thank you very much! By the way, help me see see issue #38 (comment).