[BUG] from_json and scan json do not replace \u escaped chars in nested data returned as a string.

Question

[BUG] from_json and scan json do not replace \u escaped chars in nested data returned as a string.

Opened this issue a month ago · 0 comments

Describe the bug
In nested data that is being returned as a string column in from_json or json scan if a string contains a \u escaped character sequence in it Spark will replace that \u sequence with the smallest replacement possible. It may be the regular characters or a smaller escape sequence like \r or \n. Our code does not do this.