emancu/toml-rb

Whitespace is not removed after \ in a multi-line string

Closed this issue · 3 comments

rsim commented

I tested with the latest version from the master branch:

>> TOML.parse <<-EOS
# The following strings are byte-for-byte equivalent:
key1 = "The quick brown fox jumps over the lazy dog."

key2 = """
The quick brown \


  fox jumps over \
    the lazy dog."""

key3 = """\
       The quick brown \
       fox jumps over \
       the lazy dog.\
       """
EOS
=> {"key1"=>"The quick brown fox jumps over the lazy dog.",
 "key2"=>"The quick brown \n\n  fox jumps over     the lazy dog.",
 "key3"=>"       The quick brown        fox jumps over        the lazy dog.       "}

According to the specification

The \ will be trimmed along with all whitespace (including newlines) up to the next non-whitespace character or closing delimiter. If the first characters after the opening delimiter are a backslash and a newline, then they will both be trimmed along with all whitespace and newlines up to the next non-whitespace character or closing delimiter.

but currently in this example whitespace is not trimmed for key2 and key3.

Super weird. I'll take a look now.

I think the issue is using \ with heredoc.
If you put the same string into a file and load it, it works.

Lets call eos to the variable that contains your example, and file to the same string loaded from a file.

> eos
=> "# The following strings are byte-for-byte equivalent:\nkey1 = \"The quick brown fox jumps over the lazy dog.\"\n\nkey2 = \"\"\"\nThe quick brown \n\n  fox jumps over     the lazy dog.\"\"\"\n\nkey3 = \"\"\"       The quick brown        fox jumps over        the lazy dog.       \"\"\"\n"

> file
=> "# The following strings are byte-for-byte equivalent:\nkey1 = \"The quick brown fox jumps over the lazy dog.\"\n\nkey2 = \"\"\"\nThe quick brown \\\n\n\n  fox jumps over \\\n    the lazy dog.\"\"\"\n\nkey3 = \"\"\"\\\n       The quick brown \\\n       fox jumps over \\\n       the lazy dog.\\\n       \"\"\"\n"

They are different. I guess the heredoc doesn't escape the \ character.

I'll keep researching about this issue.

rsim commented

Yes, I should have used \\ in EOS block. Just rested that it is OK.