Unable to parse json with boundary symbols in the response
ch1ffa opened this issue · 20 comments
I'm using meros with relay for the @defer feature and by a coincidence we got exactly --- symbols in a particular field of received json, which leads to a parse error, because part.json is false.
Of course we can just remove these symbols from the data, but maybe there is a better way to handle it in the library?
Hi there 👋,
Thank you so much for the issue!!
Are you able to provide me with a more tangible example? --- should only trigger a "boundary" if it's got a \n before and after.
eg
\r\n
---
\r\n
Those should all be encoded away by the json compiler on your server side.
That being said, I have had @michaelstaib have similar reports. So be great to understand this edge case a tad more.
Hi @maraisr
Indeed, it's from HC backend.
Our message contains something like that:
{"__typename":"tDO","desc":"---","cdc":"SPS"}
Small update, was able to reproduce the bug with your example! 🙏🏻 working on a solution.
@maraisr now I've got the first part filtered out as it starts just from --- without \r\n\ at the beginning
😅 can you flick me an example payload again, if its sensitive more than happy to have it sent privately.
Actually, I got an error in a different query just at the start of the app. That's what I see in the browser dev tools:
---
Content-Type: application/json; charset=utf-8
{"data":{"user":{"id":"VXNlcgpkN2FhNzFjMjctN2I0Yy00MzczLTkwZGItMzhjMjZlNjA4MzNh"}},"hasNext":true}
---
Content-Type: application/json; charset=utf-8
{"label":"WelcomeQuery$defer$ProjectList_projects_1qwc77","path":["user"],"data":{"id":"VXNlcgpkN2FhNzFjMjctN2I0Yy00MzczLTkwZGItMzhjMjZlNjA4MzNh","projects":{"edges":[{"node":{"id":"UHJvamVjdAppMQ==","name":"New project","desc":"","lastUpdate":"2021-12-22T12:57:45.488\u002B03:00","__typename":"Project"},"cursor":"MA=="}],"pageInfo":{"endCursor":"MA==","hasNextPage":false}}},"hasNext":false}
-----
And I get only the part with "label" from meros
hmmmm, that is interesting. It is passing my unit tests.
check; 612e187
What I'll do is I'll release this PR under a next npm tag, maybe something up with the custom build perhaps 🤔
Let me know how you go with meros@1.2.0-next.3 (sorry about the issue closure) opening that back up.
Unfortunately, the same behaviour as for the custom build. The first part I see in a loop part of parts is the second chunk that contains "label". Maybe I should create an example to reproduce both issues
@maraisr here is the example quite similar to our app https://github.com/ch1ffa/react-location-relay
There is one @defer fragment which works on the version 1.1.4, and if you change bio to --- and reload the page, you can see an error. For the version 1.2.0-next.3 no data is shown because the first part in parts is missing
Thank you so much!! 🙏🏻
Ill check this asap, it is Christmas and have a stack of life events to attend this next few days, but 100% will check/fix this before the new year.
Okay I know the problem,
HC's preamble boundary doesnt start with a newline delimieter
- ---\r\n
+ \r\n---\r\nsame with the tail has stuff after it.
- \r\n----
+ \r\n----\r\nthe tail boundary is who-cares in meros eyes. So main moment about the "first" boundary not having a CRLF token (aka \r\n).
Or am I misunderstanding the spec?
The requirement that the encapsulation boundary begins with
a CRLF implies that the body of a multipart entity must
itself begin with a CRLF before the first encapsulation line
-- that is, if the "preamble" area is not used, the entity
headers must be followed by TWO CRLFs. This is indeed how
such entities should be composed.
https://datatracker.ietf.org/doc/html/rfc1341
Perhaps the strategy is; if the payload starts with a boundary (w/o the CRLF) then yield aka there is NO preamble.
Coz i outright skip the first "payload"
cc @michaelstaib is this right.
They also say
A tolerant mail reading
program, however, may interpret a body of type multipart
that begins with an encapsulation line NOT initiated by a
CRLF as also being an encapsulation boundary, but a
compliant mail sending program must not generate such
entities.
I am wondering how to configure a unique boundary for my configuration to eliminate any possibility of getting boundary in data 🤔
You shouldnt worry about that, if you JSON encode you will never collide with a boundary coz \r\n---\r\n is encoded away with json, like '"\\r\\n---\\r\\n"' (note the double slashes)
@ch1ffa please check ch1ffa/react-location-relay#1 Believe its rectified for now.
I will liaise with @michaelstaib to further understand the multipart spec nuisances around that initial preamble boundary.
To that end; thank you so much for your patience with this one!
@maraisr @michaelstaib Version 1.2.0-next.4 works fine while "latest 1.1.4" fails for me.
What is the reason/status? (about preamble, right?) ... is it an HC problem or meros?
If it is just at meros, it might be helpful to release 1.2.
Hi @artola
yeah I think im ready to release it as stable. Think was mainly about confusion if preamble when omitted, should still have a presiding boundary, @next supports that avenue.
Ive released 1.2.0 as stable 🎉, that contains the fixes outlined in this chatter. Hopefully that lines it up with our MS/C# is composing messages.