Getting http 422 error code

[stendu]

IntelliJ version:

IntelliJ IDEA 2023.2.5 (Community Edition)
Build #IC-232.10227.8, built on November 9, 2023
Runtime version: 17.0.9+7-b1000.46 aarch64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 13.6
GC: G1 Young Generation, G1 Old Generation
Memory: 2048M
Cores: 10
Metal Rendering is ON
Non-Bundled Plugins:
com.videogameaholic.intellij.starcoder (0.230912)
com.hotels.intellij.plugins.network (1.5)
TunnelliJ (1.0)
JOL (1.11.2)
org.sonarlint.idea (10.3.0.77475)

Kotlin: 232-1.9.0-IJ10227.8

[stendu]

No proxy, it works fine with following curl command:
curl -X POST
-d '{"inputs": "//Write a method to print odd numbers till 100"}'
-H "Authorization: Bearer xxxxxxxxxxxxx"
-H 'Content-Type: application/json'
'https://api-inference.huggingface.co/models/bigcode/starcoder'

But it giving http error code 422 for same token and model

[stendu]

This is resolved, there is limit to max new token (advanced parameter). It can take a value of less than equal to 192. If we give a value higher than this then it is returning 422 error code.

[videogameaholic]

I have also seen this when the request is too large (larger than the context window of the model). I'm adding a setting to adjust the max context window size. It's hard to do this automatically without a setting since not only do different models have different context windows (Starcoder2 is twice the size of Starcoder) but also there are tokenization differences between languages.