`taskcat lint` uses all regions defined in `.taskcat.yaml` but doesn't output the region name in case of an error or warning
wombelix opened this issue · 0 comments
Describe the bug
I have multiple regions defined in .taskcat.yaml
, let's say us-east-1
on project
level and us-east-2
, us-west-1
and us-west-2
specifics tests. taskcat lint
seem to use all regions defined in any level but if an error or warning occurs, the region name is not part of the output. In my case, I have a list of allowed EC2 Instances, all of them are supported in us-east-1
, us-east-2
and us-west-2
, but some are not available in us-west-1
. In that case, taskcat lint
throws a lot of warnings but I can't see in the output, that it's about the region us-west-1
, which makes troubleshooting a little complicated.
To Reproduce
Steps to reproduce the behavior:
- Are you testing a QuickStart or Custom template? - Partner Solution (in development)
- Attach or link a copy of the template if possible (remove any sensitive info) - Not public yet
- Provide the parameters that you passed. (remove any sensitive info) - see end of the issue
- How did you install taskcat? (docker or pip3) - pip
- Are you using a profile, an instance role or access keys to run taskcat? - profile
- Is your AWS environment configured via
aws configure
? - no, isengard
Expected behavior
I would expect that taskcat
provides the name of the region a specific error or warning belongs to. Even better would be, if the region can be set by a cli parameter or only regions on project level / the default test is taken into account if not configured otherwise.
Screenshots
n/a
**Version (Please make sure you are running the latest version of taskcat)
- Taskcat Version: 0.9.41
- Python Version (python3 required): 3.9.16
Additional context
I can technically configure to ignore instance type related warnings, but in case of a typo or something, it feels valuable to run an actual test. But 100% coverage of all instance types across all regions is not realistic, so testing just against one specific region, as sanity check that in general the list is correct, feels like a good approach.
Passed parameters
This is an example of a instance type allowed values list that works for example with us-east-1
, because c6id
is available in that region, and will throw warnings with us-west-1
, which is expected and fine. But that I don't seem to have visibility in the taskcat lint
output and can't limit the regions that should be included is something that could be improved.
Parameters:
InstanceType:
AllowedValues:
- c6id.large
- c6id.xlarge
- c6id.2xlarge
- c6id.4xlarge
- c6id.8xlarge
- c6id.12xlarge
- c6id.16xlarge
- c6id.24xlarge
- c6id.32xlarge
- c6id.metal
Warning
[WARN ][WARN ] : line 62 [2030] [Check if parameters have a valid value] You must specify a valid allowed value for InstanceType
(c6id.large). Valid values are ["c1.medium", "c1.xlarge",
"c3.2xlarge", "c3.4xlarge", "c3.8xlarge", "c3.large", "c3.xlarge",
"c4.2xlarge", "c4.4xlarge", "c4.8xlarge", "c4.large", "c4.xlarge",
"c5.12xlarge", "c5.18xlarge", "c5.24xlarge", "c5.2xlarge",
"c5.4xlarge", "c5.9xlarge", "c5.large", "c5.metal", "c5.xlarge", ........... ]