Are bias terms also constrained when running the SFW algorithms?

[GeoffNN]

Hi! Really cool work, it's nice to see FW methods working on neural nets.

Quick question: from the code, I don't see any distinction when using bias terms; is everything constrained?

[zimmer-m]

Hi @GeoffNN , we probably discussed this already but yes, you're right. We do no distinguish between weights and biases. Each layer has a separate bias tensor which gets its own constraint. When using a global constraint (i.e. when concatenating all parameters to form a high-dim. vector which is then constrained), biases are treated the same way as weights.

I hope this answers your question!
Max