askthecode/documentation

Having issues with documents at approx. 30k characters

Opened this issue · 1 comments

ofk1www commented

Hey,

I believe 95k is the limit but I'm having issues with different files ranging from 27k-33k characters. 26k and below is fine.

dsomok commented

Hi @ofk1www,

I've also observed this behavior recently and have created a bug in our internal tracking system to take a closer look. Most probably, this happens because of the model updates OpenAI conducts. For large plugin responses, they have most likely always performed background summarization since their most capable model can still use no more than 32k tokens. However, it seems these summarizations have become either more intensive or the algorithm has changed.

I'll delve deeper into this issue, but unfortunately, it's probable that I'll have to adapt the plugin constraints for the response size.

Thank you for raising this issue. I'll ping you in this issue once I have the results of the investigation.

  • 👍1