apple/ml-cvnets

Questions about the file bytes length of ByteFormer

DanJun6737 opened this issue · 3 comments

DanJun6737 commented

Hello!

I read the ByteFormer paper and it's really an amazing piece of work! And I read the code about the encoding of file bytes: https://github.com/apple/ml-cvnets/blob/main/data/transforms/image_bytes.py
We apply the code to our own datasets.
However, we found that when using "png" or "jpeg" encoding mode, the file bytes length of each sample is not consistent. How to solve this problem of file bytes length change?

Thanks a lot!

mchorton commented

Hi @DanJun6737,

For JPEG: Our collate functions handle variable-length samples. See here: https://github.com/apple/ml-cvnets/blob/main/data/collate_fns/byteformer_collate_functions.py

For the PNG experiments, the inputs should be the same length. The reason yours aren't is due to a bug, where compress_level was not being passed as a kwarg. As mentioned in our paper, we deactivate zlib compression in our experiments - this bit of code was missing. Note, our experimental results in the paper were not affected (the bug was introduced later, when refactoring the code).

The fix is here: fc62a84 . Please make sure to apply it to your experiments. You can just pull the update that I pushed to the main branch.

DanJun6737 commented

Thank you very much for your answer.

  • 👍1