May I ask one question?

[Bhongcheng]

If we tune the global model for every client by training a few epochs with their own train dataset, instead of one global model, and then we apply knn algorithm to these different personalized models, I wonder will the results still outperform?
Thank you.

[omarfoq]

Hello @Bhongcheng,

Thank you for your question. If the number of local epochs is and learning rate used to train locally at each client is optimally tuned, then it is likely that the local tuning + local memorization will outperform local memorization. However, tuning both the number of local steps and the learning rate is a resource consuming task.

[Bhongcheng]

Hi, thanks for you warm reply, but maybe my description is confusing...
In your paper, the knn is applied to one global model(we name it g_model). If one client, for example client 1, after get the g_model, and then tune the model by his own dataset, then client 1 gets his own personalized model(we name it p_model, different from the g_model), I wonder in this case will p_model + knn still outperforms p_model?
Thank you

[omarfoq]

Hi, thanks for the clarification. I misunderstood your question initially; I taught that your question was about the comparison between p_model + knn and g_model + knn.

Regarding your actual question on the comparison between p_model + knn and p_model, I do not have a firm answer. However, I lean towards a negative answer, i.e., the performance of p_model + knn and p_model well be (essentially) the same. The reason is that if the quality of p_model is bad, e.g., p_model overfits the local dataset, then the quality of p_model + knn will be also bad. On the contrary if p_model performs well and is personalized to the local dataset, then there is no real advantage of the memorization mechanism.

[Bhongcheng]

Thanks for your warm reply!