unsupervised FL setting issues

[ZhiyuanDang]

Hi,

thanks for your paper! I am worked in the deep unsupervised field. Recently, I read your paper about the unsupervised FL method. Then, I have a question about the setting of unsupervised FL.

From the function get_U_sets_Multiclass in /utils/data_utils.py, I found that your method builds ``set index'' according to the ground-truth class label. Since the adoption of the ground-truth class label and corresponding sample index, this function may introduce some extra supervised information.

Then, a problem raise in my mind: is it reasonable to build the set index with this function? And after using such information, can this method still can be viewed as an unsupervised FL method?

In some deep unsupervised works, they hardly consider using such information. Please give me some information.

[lunanbit]

Hi, 

thanks for your interest in our work!

As the title of our paper suggests, the setting is FL from only unlabeled data with class-conditional-sharing clients.
The main contribution is showing the possibility of unsupervised FL whose model is still a classifier for predicting class labels, if

1. the class-prior probabilities are shifted 
2. while the class-conditional distributions are shared 
   among the unlabeled data owned by the clients.
   Given the above two conditions, the problem of interest is not exactly the unsupervised FL as you mentioned.

Note that our method did not use any ground-truth class labels.
As introduced in the paper, we consider the unlabeled data at a client to come in the form of separate data sets, and the data distribution of each unlabeled set follows Equation 5.
Therefore in the codes, we used the ground-truth class labels only for simulating our data generation process (see Equation 5 and our experimental setup), and the generated datasets for each client do not have any labels.
To make FL possible in such unlabeled cases, the proposed method uses indexes of the unlabeled datasets as surrogate labels, modifies the client model to be compatible with the surrogate task, and retrieves the wanted model from the surrogate scheme.

Hope this helps to clarify your question :)

[ZhiyuanDang]

Hi,

Thanks for your reply, but it may not address my problem.

In the test phase of the unsupervised method, since the classified results from softmax are not aligned with the ground-truth class, our solution is to use the hungarian_evaluate algorithm to map the classified results to the ground-truth class. After that, we compute ACC based on the correction class of samples. From your code, I do not notice any hungarian evaluate related method. The computation of ACC seems directly adopt the output of softmax, which is against common sense in the unsupervised method. I am confused about that, could you please give me more information?

[lunanbit]

Hi,

thanks for your interest in our work!

As discussed in the previous reply and also in the paper, our problem setting is not the unsupervised FL setting that you mentioned, but the FL from only unlabeled data with class-conditional-sharing clients.
Thanks to the class-conditional-sharing conditions, we propose a method that follows the empirical risk minimization framework and can predict classification labels (please carefully read Section 3 of the paper).
Such a treatment is intrinsically different from the unsupervised methods, please refer to any machine learning textbook for understanding the difference.

Hope this clarifies your question :)