google/fedjax

CIFAR 100 Questions

HanGuo97 opened this issue · 5 comments

HanGuo97 commented

Hi, thanks for the awesome library! I want to ask a couple of questions related to CIFAR100 datasets.

  1. I noticed that while the dataset is available in the library, the model is not. Curious if a model for CIFAR100 is work-in-progress, or if there is no short-term plan for this?
  2. Looking at the CIFAR100 dataset, this seems to be inconsistent with Google's TFF. Notably, the cropping size and normalizing are done differently from TFF. Is this intentional? Would it be correct to say that we could expect this to mirror TFF's design eventually?

Thanks in advance for all the help!

jaehunro commented

Hi!

Thanks for reaching out to us about this.

  1. The model for CIFAR100 was a work in progress but we decided to put it on hold until there was an explicit request for it. Our initial plan was to add a simple linear model that provides a complete example of all the requirements (e.g. baseline accuracy, hyperparameter configs, etc.) needed to add a model so that other collaborators could follow it as a guideline for contributing. If you're interested, we can work on adding this example PR for a simple model so that you can follow it as a guideline for contributing.
  2. We actually are downloading the dataset from the same source as TFF. The only difference is that we apply additional preprocessing on top of the source data following pytorch-cifar. If you don't want to apply this preprocessing, you could just manually run fedjax.datasets.cifar100.load_split.
HanGuo97 commented

Got it, thanks for letting me know!

jaehunro commented

No problem!

I'm also working on some PRs to add an example model for CIFAR-100 and add an option to match the preprocessing done by TFF according to their baseline task tff.simulation.baselines.cifar100.create_image_classification_task

Hopefully those will help!

HanGuo97 commented

Oh nice, good to hear -- looking forward to it!

jaehunro commented

Just wanted to post an update.
I just merged in #266 that adds the option for matching the preprocessing logic done by TFF.
I also added an example linear model for CIFAR-100 in #265 to show what's needed to check in and contribute a model.

I've also filed a separate issue #268 to add the ResNet-18 model for CIFAR-100 that's actually used in the "Adaptive Federated Optimization" paper. The issue details steps, requirements, etc.
Contributions are very welcomed and encouraged!

  • 👍1