aws-ia/taskcat

TemplateURL parsing error: maximum recursion depth exceeded while calling a Python object

tlindsay42 opened this issue · 0 comments

Describe the bug
A clear and concise description of what the bug is.

TaskCat fails with max recursion depth error when using a trailing comma on the final element of a JSON format array in a YAML template. The CloudFormation service and cfn-lint can both handle this.

[ERROR  ] : TemplateURL parsing error: maximum recursion depth exceeded while calling a Python object 
[WARN   ] : Failed to discover path for {'Fn::Sub': ['https://${S3Bucket}.s3.${S3Region}.${AWS::URLSuffix}/${QSS3KeyPrefix}templates/util/copy.scripts.yaml', {'S3Region': {'Fn::If': ['UsingDefaultBucket', {'Ref': 'AWS::Region'}, {'Ref': 'QSS3BucketRegion'}]}, 'S3Bucket': {'Fn::If': ['UsingDefaultBucket', {'Fn::Sub': '${QSS3BucketName}-${AWS::Region}'}, {'Ref': 'QSS3BucketName'}]}}]}, path None does not exist
Fatal Python error: Cannot recover from stack overflow.
Python runtime state: initialized

To Reproduce
Steps to reproduce the behavior:

  1. Are you testing a QuickStart or Custom template? QuickStart
  2. Attach or link a copy of the template if possible (remove any sensitive info) https://github.com/aws-quickstart/quickstart-aws-utility-meter-data-analytics-platform-v2/tree/77a2d22baf1e660fd1c3d6990ba1d15fb28ccb52
  3. Provide the parameters that you passed. (remove any sensitive info) https://github.com/aws-quickstart/quickstart-aws-utility-meter-data-analytics-platform-v2/blob/77a2d22baf1e660fd1c3d6990ba1d15fb28ccb52/.taskcat.yml#L17-L32 (fails before overrided parameters are processed)
  4. How did you install taskcat? pip3
  5. Are you using a profile, an instance role or access keys to run taskcat? profile
  6. Is your AWS environment configured via aws configure? yes

Expected behavior
A clear and concise description of what you expected to happen.

TaskCat should be able to handle trailing commas in JSON format array in a YAML template like CloudFormation and cfn-lint.

Screenshots
If applicable, add screenshots to help explain your problem.

version 0.9.35
[DEBUG  ] : Traceback:
Traceback (most recent call last):
  File "[redacted]/taskcat/_cfn/template.py", line 111, in _template_url_to_path
    current_template_path=self.template_path, template_url=template_url
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 410, in template_url_to_path
    child_template_paths = self.flatten_template_url(template_url)
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 347, in flatten_template_url
    url_list = self._flatten_template_controller(template_url)
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 333, in _flatten_template_controller
    template_url_string
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 310, in evaluate_string
    template_url_temp, depth=(depth + 1)
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 310, in evaluate_string
    template_url_temp, depth=(depth + 1)
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 310, in evaluate_string
    template_url_temp, depth=(depth + 1)
  [Previous line repeated 4 more times]
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 300, in evaluate_string
    parts[0]
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 259, in evaluate_expression_controller
    results = self.evaluate_fn_sub(expression)
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 165, in evaluate_fn_sub
    values = self.values_to_dict("(" + values + ")")
  File "[redacted]/taskcat/_cfn/stack_url_helper.py", line 149, in values_to_dict
    values_dict = json.loads(values_dict_string)
  File "/usr/lib64/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
RecursionError: maximum recursion depth exceeded while calling a Python object
[ERROR  ] : TemplateURL parsing error: maximum recursion depth exceeded while calling a Python object 
[WARN   ] : Failed to discover path for {'Fn::Sub': ['https://${S3Bucket}.s3.${S3Region}.${AWS::URLSuffix}/${QSS3KeyPrefix}templates/dataflow/topology/topology.dataflow.ingestion.template.yaml', {'S3Region': {'Fn::If': ['UsingDefaultBucket', {'Ref': 'AWS::Region'}, {'Ref': 'QSS3BucketRegion'}]}, 'S3Bucket': {'Fn::If': ['UsingDefaultBucket', {'Fn::Sub': '${QSS3BucketName}-${AWS::Region}'}, {'Ref': 'QSS3BucketName'}]}}]}, path None does not exist
Fatal Python error: Cannot recover from stack overflow.

**Version (Please make sure you are running the latest version of taskcat)

  • Taskcat Version (ex: [2018.817.210357]) 0.9.35

Note: Python Version (python3 required) 3.7

To find versions:
Via taskcat: taskcat -V
Via pip3: pip3 show taskcat

Note: both version should match

To update taskcat run:
for docker : docker pull taskcat/taskcat
for pip3: pip3 install --upgrade taskcat

Additional context
Add any other context about the problem here.