can't reproduce the result

[challow0]

Hi ~
Thank you for your excellent work. I've been working on this recently, but I can't exactly reproduce the results in the paper through your open-source code. The dataset partitioning in my experiment refers to split files, and then all parameters are set by default. I would like to know if the results in your paper are averaged over the last few epochs or if the best results are obtained. And whether the experimental parameter settings are exactly the same for PACS and VLCS datasets?
Looking forward to your reply!

[xxgege]

Hi ~
Thank you for your interest in our work! The reported results are the average of the results of multiple runs, and each result is selected via validation accuracy.
As mentioned in readme, we find that many hyper-parameters can affect the performance (such as lrbl, epochb, lambdap), which can vary among different tasks/environments/software/hardware/random seeds, and thus carefully tunning is required. We find the best hyper-parameters are similar yet not exactly the same for PACS and VLCS datasets and the default ones show good performance in our environment.
We are sorry for the problem and trying to address it in future work.

[challow0]

@xxgege Thanks for your reply ~ There is one more thing. I am wondering if you can share the MNIST-M dataset split solution used in this experiment? Maybe the python file that made the MNIST-M dataset or a link to the made MNIST-M dataset? Thanks for your help!