Is there a restriction on json size

Question

Is there a restriction on json size

Closed this issue 6 years ago · 3 comments

Hi again, awesome project. I got it to work super easy with my project and I am still exploring the usecases.

One strange thing just happened. My application started to freeze on loading when I updated the json. Finally I deducted it had something to do with the size of the json so I started to add characters to a string inside the json until I discovered the limit where the json size caused the application to hang.

At exactly 65284 characters the document parses ok but at 65285 characters of json it crashes when running Deserialize. From what I can see in the callstack, it referes to a MergeMap operation and the error is: "e is not defined".

My code is this:
const config = await this.http.get('assets/program.json').toPromise();
this.program = Deserialize(config, Program);

and when I debug config, I see that it is showing the json object correctly even when the file size goes p to 65285 characters but then when it calls Deserialize it crashes.

It seems suspiciously close to the 65535 limit.

I realize this is a hoppy project but any ideas on the reason for this would be super helpful since all my code is now depending on cerialize and my json is rather big :-) I will continue to investigate as well. Hopefully can be reproduced with any large json.

Answer 1 · 2018-07-20T18:50:04.000Z

I haven't tried it with json that big, maybe it's a js limit? I also find the 65284 suspicious. Does everything work before you deserialize? johanlantz <notifications@github.com> schrieb am Fr., 20. Juli 2018, 17:15:

…

Hi again, awesome project. I got it to work super easy with my project and I am still exploring the usecases. One strange thing just happened. My application started to freeze on loading when I updated the json. Finally I deducted it had something to do with the size of the json so I started to add characters to a string inside the json until I discovered the limit where the json size caused the application to hang. At exactly characters the document parses ok but at 65285 characters of json it crashes when running Deserialize. From what I can see in the callstack, it referes to a MergeMap operation and the error is: "e is not defined". My code is this: const config = await this.http.get('assets/program.json').toPromise(); this.program = Deserialize(config, Program); and when I debug config, I see that it is showing the json object correctly even when the file size goes p to 65285 characters but then when it calls Deserialize it crashes. It seems suspiciously close to the 65535 limit. I realize this is a hoppy project but any ideas on the reason for this would be super helpful since all my code is now depending on cerialize and my json is rather big :-) I will continue to investigate as well. Hopefully can be reproduced with any large json. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#87>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABTXwQ6I8OLyotRSLmtA8cfP-QWMKmmcks5uIfP3gaJpZM4VYM7k> .

Answer 2 · 2018-07-20T19:47:29.000Z

I will test more but what I saw today was:

When I debug the first line:
const config = await this.http.get('assets/program.json').toPromise();
was working ok. So its not the size of the string.
Secondly, I did not add any new elements or anything to the json. I just noticed that something did not work and I started to remove elements etc finally narrowing it down to a single character difference. Maybe its 65535 but some bytes are taken by the system (exactly 250) so finally it leaves us the 65284. Either way, it seems very strange. I do not think its a pure js limitation but something else internal, maybe with the MergeMap usage.

Everything with Deserialize works just great, its simply this limit that causes issues. I can see if I can create a simple reproducible file to share if you are interested. Happy to help in any way I can.

Answer 3 · 2018-07-22T10:13:46.000Z

I found the issue. It was nothing with this project. There was a race condition in my app. I read the json from file in the service. Then in the component used to render the content, I asked for the deserialized object tree when the view was about to show.

The super crazy thing is that while the json was below the limit, the file read and deserialization finished before the view was about to show. Adding the extra character caused the viewAboutToEnter function to be called before the Deserialize had finished so nothing was rendered.

The amazing thing was that this was 100% reproducible, adding the extra character caused the view to execute before the service was done, everytime.

So this was a mistake by me in how the async execution was managed. Not trivial to spot but now I know the reason, sorry for bothering you about this.