purescript-contrib/purescript-parsing

CodePoints uncons? Deprecate drop?

Closed this issue · 8 comments

jamesdbrock commented

I just noticed that instance stringStringLike uses CodeUnits for uncons.

purescript-parsing/src/Text/Parsing/Parser/String.purs

Lines 27 to 29 in 3d7976e

instance stringLikeString :: StringLike String where
uncons = SCU.uncons
drop = S.drop

Doesn't that mean that anyChar will be wrong for astral characters?

purescript-parsing/src/Text/Parsing/Parser/String.purs

Lines 54 to 56 in 3d7976e

anyChar = do
input <- gets \(ParseState input _ _) -> input
case uncons input of

Also, the drop member of StringLike is now unused?

drathier commented

I just hit the anyChar issue you found above, but with the purerl backend.

jamesdbrock commented

I'm looking through the code for this package and basically the whole thing is completely oriented to UTF-16 codeunits...

drathier commented

The problem is that https://pursuit.purescript.org/packages/purescript-string-parsers/6.0.1/docs/Text.Parsing.StringParser.CodePoints#v:anyChar puts the result into a Char so it isn't parsing code points after all. we'll be writing our own anyCodePoint and anyGrapheme later today.

  • 👀1
jamesdbrock commented

@drathier I'd like to see your anyGrapheme parser. Link please? Do you think that this library should include anyGrapheme?

jamesdbrock commented

@drathier Would PR #119 solve your problem? That PR deletes the StringLike class. Would you like to rely on that class for purerl? Or have your implemented the Data.String API for purerl?