Nondeterministic output with garbled strings
jgm opened this issue · 5 comments
jgm commented
I suspect this is a memory handling bug related to PR #13; see the discussion there.
@andersk can you have a look? I'm almost positive it's an issue with your PR, rather than with cmark.
@kivikakk this affects cmark-gfm too.
If a quick fix is not forthcoming, we should roll back PR #13 and make a new release.
Prelude CMarkGFM> nodeToCommonmark [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "^") []]]
"^\n"
Prelude CMarkGFM> nodeToCommonmark [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "^") []]]
"^Z\n"
Prelude CMarkGFM> nodeToCommonmark [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "^") []]]
"^\\*%\n"
Also reproducible with other strings:
Prelude CMarkGFM> nodeToCommonmark [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT " ") []]]
" p@R\n"
Prelude CMarkGFM> nodeToCommonmark [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT " ") []]]
" \n"
Prelude CMarkGFM> nodeToCommonmark [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "hi") []]]
"hiC\SOHB\n"
Affects other writers as well:
Prelude CMarkGFM> nodeToMan [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "hi") []]]
".PP\nhiz\v\SOH\n"
Prelude CMarkGFM> nodeToLaTeX [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "hi") []]]
"hiaR\n"
Prelude CMarkGFM> nodeToXml [] $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "hi") []]]
"*** Exception: Cannot decode byte '\xd2': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream
Prelude CMarkGFM> nodeToHtml [] [] $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "hi") []]]
"<p>hi[\aB</p>\n"
Prelude CMarkGFM> nodeToHtml [] [] $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "hi") []]]
"<p>hio\EOTB</p>\n"
jgm commented
We need that null terminator!
jgm commented
andersk commented
Good catch, that was silly of me. I’ve proposed the addition of withCString
to Data.Text in haskell/text#254.
jgm commented
Anders Kaseorg <notifications@github.com> writes:
Good catch, that was silly of me. I’ve proposed the addition of `withCString` to Data.Text in haskell/text#254.
That would be a good addition indeed!
kivikakk commented
Thanks for the update again!