01-ai/Yi

The training of yi-vl models is supported by SWIFT Framework of ModelScope community.

tastelikefeet opened this issue · 2 comments

tastelikefeet commented

Reminder

  • I have searched the Github Discussion and issues and have not found anything similar to this.

Motivation

We have supported the training of yi-vl models, any interested developer is welcome to use SWIFT from ModelScope community.

Solution

Please check: https://github.com/modelscope/swift/tree/main/examples/pytorch/llm/scripts/yi_vl_6b_chat/lora for details

Alternatives

No response

Anything Else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
Yimi81 commented

Thanks! 😊

babla9 commented

@tastelikefeet thanks for this!

Some additional questions:

  1. Do you know what the hardware requirements are for finetuning Yi VL 6B?
  2. Whats the amount of time it takes to process a single batch (size 1) with a single A100 40gb?
  3. The dataset thats being used in the finetuning script is a captioning dataset coco-mini-en-2. How can I use an multiturn instruction finetuning instead eg a dataset like
    { "conversations": [ { "from": "human", "value": "How many animals in this image?" }, { "from": "gpt", "value": "There are 5 animals in the picture provided." } ], "image": "abc123.jpg", "id": "abc123" }