user defined sampler

Question

user defined sampler

Closed this issue a year ago · 2 comments

Hi, the InfoBatch wraps the dataset and generates a sampler for dataloader. What if I have a pre-defined sampler? How to mitigate my sampler to the InfoBatch's sampler?

Answer 1 · 2024-01-22T02:21:42.000Z

The essence of this method is to achieve lossless acceleration by randomly discarding well-learned samples and rescaling the gradients of the discarded samples. Therefore, you can customize the implementation based on the ideas presented in the paper. Feel free to reopen the issue.

Answer 2 · 2024-01-22T08:24:23.000Z

PS: you can refer to the research/ directory for a more straightforward implementation of the code. If you are using distributed training, there are some more points to take care of. If you have further questions, you can post them in detail.