what is the difference between gt_labels and robust_labels ?
kunlongsolid opened this issue · 3 comments
groundtruth labels are manually labeled and robust labels are labeled using a neural net. See page 3 of the paper: "These works use groundtruth
annotations for guiding the discriminator. While this
provides ground-truth semantic information, it restricts the
image collections available for training to densely annotated
datasets compatible with the synthetic label maps at hand.
In our work, we leverage a robust semantic segmentation
network [72] to provide label maps that are approximately
consistent for synthetic and real images. Previously,"
We use the gt_labels for separate processing streams in the G-buffer encoder. Each of the labels gets a separate branch in the encoder network. It's named gt_labels as we used the ground truth segmentation for the synthetic dataset to derive a segmentation into different materials. The key idea is to provide a segmentation of the synthetic data into different materials with the gt_labels. There is no need for any other semantic meaning or correspondence to classes in some dataset for these gt_labels.
The robust-labels are used to guide the discriminator. To that end, they need to be compatible and as consistent as possible for synthetic and real dataset. You could use ground truth semantic segmentation labels here, but this is costly to annotate for real data and most of the time requires considerable effort for the synthetic data as well. The alternative is a robust segmentation network, which works directly on images and produces approximately consistent semantic segmentations for synthetic and real data. We want the discriminator to judge realism conditioned on semantic classes. So a semantically meaningful segmentation is important for the robust_labels.
Hi, could you please explain why the variable gt_label_path in the file "pdf.py" is only used if if self.gbuffers == 'fake':?
What does self.gbuffers == 'fake' precisely mean?