The image_mask in the below image becomes all Falses, when i use LoRA.

Question

The image_mask in the below image becomes all Falses, when i use LoRA.

Closed this issue 10 days ago · 4 comments

Answer 1 · 2024-09-12T14:12:55.000Z

Thanks for the issue.
Does the latest commit still has the issue? I've tested with the latest commit and it's okay for me. Can you test it with the latest commit?

Answer 2 · 2024-09-12T15:05:10.000Z

The problem is still existing:

Answer 3 · 2024-09-12T15:07:15.000Z

I have modified the data.py file:
data.zip
Please use the diff command to compare the differences.

Answer 4 · 2024-09-12T17:28:30.000Z

Maybe your dataset is a bit different from the format I've worte in the README. I have '\n' between the <image> token and the text. I use the dataset as LLaVA format.