Failed to value at token created because it is not preceeded by a :, prceeded by :1657872213

[DomerDev]

OpenAiJsonException: Failed to value at token created because it is not preceeded by a :, prceeded by :1657872213
For some reason json deserializer reads the "created" parameter of openai's response not as ["created"; ":"; "1657872213"] but ["created"; ":1657872213"; ","]. An example of openai's response:

{
"id": "cmpl-uqkvlQyYK7bGYrRHQ0eXlWi7",
"object": "text_completion",
"created": 1589478378,
"model": "text-davinci-002",
"choices": [
{
"text": "\n\nThis is a test",
"index": 0,
"logprobs": null,
"finish_reason": "length"
}
],
"usage": {
"prompt_tokens": 5,
"completion_tokens": 6,
"total_tokens": 11
}
}
As you can see, the spacing is the the same as in two previous lines in the original json file. Anyone knows how to deal with it?

[hexthedev]

I wrote the bare-minimum JSON parser a while a go now. It's possibly a bug, or just some new thing I never encountered when I wrote it.

Looking at the description, it looks like the JSON parser is acting strangely with JSON tokens that are inside strings. i.e ":" and "," are key token in JSON.

Sadly I don't have time to fix it myself any time soon.

I would suggest debugging the parsing code against this input. Inside the project there are unit tests testing the parser. You could write a unit test that sends this input to the parser and check the output. Then, you could try stepping through the process to figure out where the issue is.

If you find the problem, feel free to make a PR and will gladly review and merge.

[rainbowcreatures]

For me the issue was that "created":15320582 actually came in without any space.

That's why it parses the whole ":15320582" (for example) as a single token, instead of parsing ":" and "15320582" as two separate tokens. I had the same error, not sure how @DomerDev could have gotten the error with a space (I tested an input with space and it worked fine for me).

I created a fix/hack, because I don't understand the code properly and just tried something on a hunch....

In BaseAnalyser.cs, line 23, I tried adding the colon itself as one of the characters which separate a token (at the end of the line). Surprisingly, it works. Not sure if there are any side effects, really did it without any deeper knowledge about the how the json deserializer works.

if( c == '{' || c == '}' || c == '[' || c == ']' || c == ',' || c == ':')