How do you run evaluation with unseen images? Currently getting the training images in the reconstruction.

[ebartrum]

Hey, I am trained the model on my own training data. When I run it on the test dataset, what I get is reconstructions of the training data, not the images in the test dataset. I am using the following options for training:
python train.py --data_root data/cars/train --val_root data/cars/val
--logging_root logs_cars --batch_size 16

And for evaluation:
python test.py
--data_root data/cars/test
--logging_root logs_cars
--num_instances 6210 (number of images in training set)
--checkpoint logs_cars/checkpoints/
--specific_observation_idcs 0

The output I'm getting is reconstructions of images in the training set, in the poses specified in the test set. What I need to get is for the unseen objects in the test set, specific output views predicted from some specific input views. How should I do that?

[feemthan]

@ebartrum , have you tried few shot learning ? Not a 100% sure this would help but you might wanna give it a spin.

[ebartrum]

Hey @feemthan do you know why freeze_networks is True in cars_one_shot.yml? When I train with this on, it doesnt get anywhere.

[vsitzmann]

Dear Ed,

please refer to section 3.3 in the paper: "Generalizing across Scenes", specifically the paragraph "Finding latent codes z_j".

SRNs do not have an encoder to find the latent code - instead, you have to freeze the weights of the hypernetwork, raymarcher, and renderer (as you said, freeze_networks=True!). Then, you initialize a new latent code for each scene you want to reconstruct - that's the CLI parameter "overwrite_embeddings". Then, you "train" again, this time, only optimizing the latent codes! This is what I intended the cars_one_shot.yml script for.

Finally, with these new latent codes, you can then run the test script.

Hope that helps!

[ebartrum]

Thanks a lot! Makes sense now