cluster_centers not updating

[seanyang38]

Hi vlukiyanov,

Thank you for your pytorch implement. It has been a big help. However, I noticed that in this version, the cluster_centers are not updating during training.

This is the code I used to test:

        a = list(model.parameters())[-2].clone()  # hidden layer
        b = list(model.parameters())[-1].clone()  # cluster centers
        optimizer.zero_grad()
        loss.backward()
        optimizer.step(closure=None)
        print (torch.equal(a, list(model.parameters())[-2]), torch.equal(b, list(model.parameters())[-1]))

It returns False, True

I am very new to pytorch, but I think this might be a bug from pytorch. I checked there's gradient
for cluster centers but somehow the parameters are not updating.

Maybe there's something I don't understand about pytorch, but I just wanted to point this out to you.

[vlukiyanov]

You're quite right (you could also get it out using model.state_dict()['assignment.cluster_centers'] I think); must be another issue when I converted everything to 1.0.0 from 0.4.0.

I think the cluster centres don't contribute as much to the gradient in practice, so hopefully that explains why the accuracy was still high.

I've pushed the commit above to a branch, but need to test on some real data and make sure its still reproduces the accuracy.

Thanks for spotting.

[vlukiyanov]

Ok merged that branch in should be fixed, let me know how it works for you and reopen if it doesn't or you think there's an error.

Thanks again for spotting.

[seanyang38]

Hi vlukiyanov

Thanks for the prompt response and resolving the issue so quickly! The parameters are updating now!

Thanks again for the implementation :)