chaiNNer-org/chaiNNer

Add more tiling options

Closed this issue · 2 comments

Kim2091 commented

The main reason for this is just so we can more finely tune the tile size to prevent VRAM overflow. Having to choose between 1024 and 2048 is not good, it can go from 11 GB allocated to over 24, with no in-between

There are two possible ways to do this that I see:

  • add more options to the dropdown
  • allow the user to type in the number
joeyballentine commented

We actually could just add a "custom" option to the dropdown super easily. We previously (a long time ago when this was first added) were restricted in not being able to do this, but now we definitely can. Good callout

  • 👍1
joeyballentine commented

Closed by #2828