heredoc parsing fails if file has DOS/Windows line-endings (carriage returns)

[bgehman]

heredoc parsing fails if file has DOS/Windows line-endings (carriage returns)

Sample TF file, with heredoc notation (<<EOT):

$ cat test.tf 
resource "vault_generic_secret" "example" {
  path = "secret/foo"

  data_json = <<EOT
{
  "foo":   "bar",
  "pizza": "cheese"
}
EOT
}

Unix line-endings work:

$ dos2unix test.tf 
dos2unix: converting file test.tf to Unix format ...

$ echo -e "import hcl\nwith open('test.tf','r') as fp: obj = hcl.load(fp)" | python

DOS line-endings don't:

$ unix2dos test.tf
unix2dos: converting file test.tf to DOS format ...

$ echo -e "import hcl\nwith open('test.tf','r') as fp: obj = hcl.load(fp)" | python
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/lib/python2.7/site-packages/hcl/api.py", line 51, in load
    return loads(fp.read())
  File "/usr/lib/python2.7/site-packages/hcl/api.py", line 62, in loads
    return HclParser().parse(s)
  File "/usr/lib/python2.7/site-packages/hcl/parser.py", line 295, in parse
    return self.yacc.parse(s, lexer=Lexer())
  File "/usr/lib/python2.7/site-packages/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/usr/lib/python2.7/site-packages/ply/yacc.py", line 1061, in parseopt_notrack
    lookahead = get_token()     # Get the next token
  File "/usr/lib/python2.7/site-packages/hcl/lexer.py", line 225, in token
    return self.lex.token()
  File "/usr/lib/python2.7/site-packages/ply/lex.py", line 386, in token
    newtok = self.lexerrorf(tok)
  File "/usr/lib/python2.7/site-packages/hcl/lexer.py", line 211, in t_error
    _raise_error(t, "Heredoc must have a marker, e.g. '<<FOO'")
  File "/usr/lib/python2.7/site-packages/hcl/lexer.py", line 17, in _raise_error
    raise ValueError("Line %d, column %d, index %d: %s" % (lineno, column, lexpos, message))
ValueError: Line 4, column 14, index 84: Heredoc must have a marker, e.g. '<<FOO'

[scottbelden]

I'll take a look. I think I have an idea of what's going on.