how to supervise intermediate layers?
nabsabraham opened this issue · 6 comments
thanks for the great repo! I'm pretty new and can't seem to understand how your code is intermediately supervising layers. Would you be able to point that out to me? According to the paper, the are two losses being computed for each block - how do you implement this in keras?
Each hourglass block will have one head as output, the output will be compared with ground truth to calculate loss. This is intermediately supervision. In paper, each hourglass block only has one loss, not two.
In code, please pay attention to src/net/hg_blocks.py
head_next_stage, head_to_loss = hourglass_module(head_next_stage, num_classes, bottleneck, i)
outputs.append(head_to_loss)
I see! so you just keep a list of outputs that will be used later for the Model() method.
But each trainable output will only be compared to the same ground truth mask, correct?
in this case, yes. If your model have 5 outputs, your data generator need to have 5 ground truth mask. In this case, the ground truth masks are the same for all outputs.
In general, the ground truth can be different. Just make sure the ground truth from data generator is paired with your model's output.
thank you for all your help!
i just have a last follow-up question before I close this issue- if I want to use different ground truths I would have to set them in my inputs to the Model() method, correct?
example, training 3 intermediate layers with 3 different sized ground truths:
model = Model(inputs=[input_img, gt_1, gt_2, gt_3], outputs=[output, layer5, layer4, layer3]
but how would I tell keras which gt to use for which supervision?
No. you need set different ground truths in outputs, instead of inputs.
model = Model(inputs=[input_img], outputs=[output, layer5, layer4, layer3]
Your data generator is supposed to generate
yield input, [gt_1, gt_2, gt_3]
Just make sure the sequence of gt_1 gt_2 gt_3, matched with layer5, layer4, layer3.
I will not be using a generator because my data is sufficient so is my answer still correct?
Again, thank you for all your help.