virtuald/pyhcl

Failure to parse tab-indented heredoc

Closed this issue · 2 comments

Parsing fails when a tab-indented heredoc is encountered.

Example (the below should have tabs, but I cannot enter one directly and can't find a convenient place to copy/paste one from):

        someparam = <<-EOF
        some content
        some more content
        EOF

The error that I got:

Traceback (most recent call last):
  File "./generate_env_docs.py", line 81, in <module>
    main()
  File "./generate_env_docs.py", line 77, in main
    generate_env_doc(basedir, account, env)
  File "./generate_env_docs.py", line 61, in generate_env_doc
    terraform_config = load_terraform_config(os.path.join(basedir, account, env, 'main.tf'))
  File "./generate_env_docs.py", line 48, in load_terraform_config
    return hcl.load(f)
  File "/home/agaffney/.local/lib/python2.7/site-packages/hcl/api.py", line 51, in load
    return loads(fp.read())
  File "/home/agaffney/.local/lib/python2.7/site-packages/hcl/api.py", line 62, in loads
    return HclParser().parse(s)
  File "/home/agaffney/.local/lib/python2.7/site-packages/hcl/parser.py", line 307, in parse
    return self.yacc.parse(s, lexer=Lexer())
  File "/home/agaffney/.local/lib/python2.7/site-packages/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/home/agaffney/.local/lib/python2.7/site-packages/ply/yacc.py", line 1061, in parseopt_notrack
    lookahead = get_token()     # Get the next token
  File "/home/agaffney/.local/lib/python2.7/site-packages/hcl/lexer.py", line 230, in token
    return self.lex.token()
  File "/home/agaffney/.local/lib/python2.7/site-packages/ply/lex.py", line 406, in token
    newtok = self.lexeoff(tok)
  File "/home/agaffney/.local/lib/python2.7/site-packages/hcl/lexer.py", line 180, in t_heredoc_eof
    _raise_error(t, 'EOF before closing heredoc')
  File "/home/agaffney/.local/lib/python2.7/site-packages/hcl/lexer.py", line 17, in _raise_error
    raise ValueError("Line %d, column %d, index %d: %s" % (lineno, column, lexpos, message))
ValueError: Line 105, column 0, index 5102: EOF before closing heredoc

I think #32 should resolve this. There wasn't any support for <<- style heredocs.

Verified fixed in 0.3.8...thanks!