Question About The role of Optical Flow

[LittlefatChuan]

Hello. idk why i'm just unable to achieve the result of Figure 8 (b) red Line in your paper, i only got the blue curve with all loss function enabled.And I retrain the model without Optical flow input and disable the optical flow loss(i set noFlo = True in your code). Then i
noticed the rotation curve between with Optical Flow and the without one are very similar. Does the Optical Flow really matter?

[DEEPMINDing]

Hi, I would like to know how you changed the code to the version without optical flow, and what specific changes have been made? I only made changes to [no_flo=True], which resulted in this error：
size mismatch for rnns.0.LSTM.weight_ih: copying a param with shape torch.Size([2048, 156]) from checkpoint, the shape in current model is torch.Size([2048, 92]).

[LittlefatChuan]

Hi, I would like to know how you changed the code to the version without optical flow, and what specific changes have been made? I only made changes to [no_flo=True], which resulted in this error： size mismatch for rnns.0.LSTM.weight_ih: copying a param with shape torch.Size([2048, 156]) from checkpoint, the shape in current model is torch.Size([2048, 92]).

hello, you should also tuning the model.py Line 78, "self.no_flo = False" set self.no_flo = True
and disable the optical loss in "train.py" and "inference.py"： "loss_step = model.loss(out, vt_1, virtual_inputs, real_inputs_step, flo_step, flo_back_step, real_projections_t, real_projections_t_1, real_postion_anchor, follow = True, optical = True, undefine = True)", set optical = False.
then re-train the model to get a checkpoint file without optical loss.

[sabitarajbanshi]

@LittlefatChuan Hi, I have tried to run this according to the above code setup, but got error as;

how can I run the inference for a video without the flo values?

[LittlefatChuan]

@LittlefatChuan Hi, I have tried to run this according to the above code setup, but got error as; how can I run the inference for a video without the flo values?

sorry, there is another param about optical flow in dataset loader that i forget to mention：
Line 148 in inference.py "test_loader = get_inference_data_loader(cf, data_path, no_flo = False)" --> noflo = True
Line 220 in train.py "train_loader, test_loader = get_data_loader(cf, no_flo = False)" --> noflo = True

[sabitarajbanshi]

@LittlefatChuan have you successfully run this code if yes can we connect so that you can guide me through other dependencies issues? my email id; sabita.rajbanshi@naamii.org.np