Reproduce Table 1 of ICLR24 Paper

Question

Reproduce Table 1 of ICLR24 Paper

Closed this issue 5 months ago · 3 comments

Thanks for the great work of DEXML, the results are very promising.

Can you please provide the more details about how to reproduce DEXML for datasets shown in Table 1 of your ICLR paper?
For example, what's the input data format, the exact hyper-parameters, and command line usage, etc.

Very much appreciate!

Answer 1 · 2024-04-03T01:28:53.000Z

Hi @OctoberChang,

Thanks for your interest in the work. I have updated the readme to provide details on how to run the code on benchmark datasets. I am still in the process of documenting the code, so going ahead I will try to elaborate on the readme more and provide demo notebooks. We have also released the model checkpoints on huggingface (please see the updated readme for the links), hope this helps in reproducing results in the paper. Please feel free to reach out if you have any other questions/suggestions.

Thanks!
Nilesh

Answer 2 · 2024-04-04T18:31:57.000Z

Q4: Does Datasets/<dataset>/filter_labels_train.txt or Datasets/<dataset>/filter_labels_test.txt affect the training dynamics? In other words, with and without those files, does DEXML training end up with the same model parameters? And it is just the evaluation metric becoming different?

Answer 3 · 2024-04-04T19:26:27.000Z

Moved our further discussion to #2. Closing this issue.