Request: image vision dynamic resize

[dillfrescott]

Often times my users send huge images that cause the vision model to throw an error due to the sheer size of the images pixel wise.

A way for the bot to dynamically resize the image smaller before base 64 sending it to the vision model would be nice.

When the image is too large the bot can't process the image, and it sorta gets stuck until you type ignoreolder.

[kumitterer]

Oooh, good catch! I'll look into this tomorrow.

The API docs don't really include a lot of information on how to prepare images for upload, but according to this, it should be smaller than 2000x768: https://platform.openai.com/docs/guides/vision/managing-images

We already have Pillow as a dependency, so scaling down images should be really easy to do.

[dillfrescott]

Oh excellent! Sounds good!

[kumitterer]

Just tagged a new release, v0.3.13 (3f084ff), that will automatically resize images accordingly. It also comes with new config options in the OpenAI section that allow you to override the defaults of 2000x768 in case Gemini expects something different (I'll admit, I didn't look that up).

matrix-gptbot/config.dist.ini

Lines 142 to 150 in 3f084ff

	# Maximum width and height of images sent to the API if vision is enabled
	#
	# The OpenAI API has a limit of 2000 pixels for the long side of an image, and
	# 768 pixels for the short side. You may have to adjust these values if you're
	# using a self-hosted model that has different limits. You can also set these
	# to 0 to disable image resizing.
	#
	# MaxImageLongSide = 2000
	# MaxImageShortSide = 768

Please check if that fixes the issue for you.

[dillfrescott]

Works great! Thank you!

[dillfrescott]

I even gave it an image with a size 6916x4616 and it described it perfectly!