akhilmathurs/orchestra

Incorrect semi-supervised evaluation?

Closed this issue · 2 comments

optimizer = torch.optim.SGD(classifier.parameters(), lr=0.1, momentum=0.9, weight_decay=0)

Hi,
According to the paper (Section 5, "Evaluation protocol" paragraph),

For semi-supervised evaluation, we fine-tune the entire model using limited labelled data (1% or 10% labels).

However, backbone weights are never fine-tuned. Only running statistics of BatchNorm are updated on the entire dataset.

Thanks for flagging this! There's a good chance that while releasing the code, we mistakenly copied a line from the linear eval file and the comment above is an artifact of that. I'll check this out in detail right away.

resolved by b6ae718