Make HiT-SR available on HF

Question

Make HiT-SR available on HF

Opened this issue 6 months ago · 7 comments

Niels here from the open-source team at Hugging Face. I discovered your work through the paper page: https://huggingface.co/papers/2407.05878 (congrats on the ECCV Oral!). I work together with AK on improving the visibility of researchers' work on the hub.

It'd be great to make the models available on the 🤗 hub, we can add tags so that people find them when filtering https://huggingface.co/models.

For instance, I contributed the Swin2-SR model to HF here: https://huggingface.co/caidas/swin2SR-realworld-sr-x4-64-bsrgan-psnr. We can add tags like "image-to-image" and "image super-resolution" so that people will find your work.

Uploading models

See here for a guide: https://huggingface.co/docs/hub/models-uploading. In case the models are custom PyTorch model, we could probably leverage the PyTorchModelHubMixin class which adds from_pretrained and push_to_hub to each model. Alternatively, one can leverages the hf_hub_download one-liner to download a checkpoint from the hub.

We encourage researchers to push each model checkpoint to a separate model repository, so that things like download stats also work. We can then also link the checkpoints to the paper page.

Let me know if you need any help regarding this!

Cheers,

Niels
ML Engineer @ HF 🤗

Answer 1 · 2024-09-14T07:19:53.000Z

Hi Niels, thank you for your interest in our work! It would be very cool to have HiT-SR on Hugging Face 🤗. I am pretty new to 🤗 hub but will try to upload HiT-SR next week. Please stay tuned!

Answer 2 · 2024-09-14T07:42:25.000Z

Great, happy to send a PR if required

Answer 3 · 2024-09-14T08:36:39.000Z

That would be very nice if it's convenient for you to send a PR :) In my understanding, we need to integrate PyTorchModelHubMixin class into the models and then run push_to_hub. Is it right?

Answer 4 · 2024-09-14T10:02:45.000Z

Yes that is right. Would it be possible for you to try this out?

Note that for the PyTorchModelHubMixin to work, your nn.Module class should only take arguments in its init which are JSON serializable (like integers, floats, lists, dicts,...). This way it will push a config.json (along with safetensors weights) to a model repo on the hub. Basically this:

from basicsr.models.hit_model import HITModel

# initialize model
model = HITModel(...)

# equip with weights
model.load_state_dict(...)

# push to the hub
model.push_to_hub("your-hf-org/hit-base")

# reload
model = HITModel.from_pretrained("your-hf-org/hit-base")

Answer 5 · 2024-09-15T10:29:13.000Z

Thanks for the guidance. I have uploaded all the models to Hugging Face 🤗, and HiT-SR is now available here! But I am not sure how to integrate HiT-SR into the Hugging Face documents (like what you did for Swin2SR here) 🤔, could you help me with it? Thanks a lot!

Answer 6 · 2024-09-20T18:52:51.000Z

Hi,

What I did for Swin2SR is making the model Transformers compatible as explained here: https://huggingface.co/docs/transformers/custom_models.

Regarding the models on HF, what did you use to upload the models? Cause it seems there's no from_pretrained method present in this code base. Update: I see you've pushed the code to a repo on the hub, great! Feel free to also include it in the Github repository :)

Looks like download numbers are working now, so we could perhaps update the README here so that people can find the HF models?

Answer 7 · 2024-09-21T13:20:54.000Z

Hi Niels,

Thanks a lot for the suggestions and help! I have updated the Github codebase to make it compatible with Huggingface models, and README is also updated. Will try to make HiT-SR compatible with transformers later. :)