newer botocore breaks `ecs deploy`
jperez3 opened this issue · 10 comments
seems like a an update to botocore has broken some functionality of the ecs deploy tool.
The version in question:
botocore-1.19.57
the Error:
botocore.exceptions.ParamValidationError: Parameter validation failed: 127 | Unknown parameter in input: "registeredAt", must be one of: family, taskRoleArn, executionRoleArn, networkMode, containerDefinitions, volumes, placementConstraints, requiresCompatibilities, cpu, memory, tags, pidMode, ipcMode, proxyConfiguration, inferenceAccelerators 128 | Unknown parameter in input: "registeredBy", must be one of: family, taskRoleArn, executionRoleArn, networkMode, containerDefinitions, volumes, placementConstraints, requiresCompatibilities, cpu, memory, tags, pidMode, ipcMode, proxyConfiguration, inferenceAccelerators 129
When I rolled back to an older versions of botocore ecs deploy started working again
valid versions:
botocore-1.19.55
both scenarios used: ecs-deploy-1.11.0
Our team also ran into this tonight. My local ecs-deploy version (1.10.1) works fine but our Bitbucket deployment failed on the latest versions reported by @jperez3 .
For those who are interested, I was able to quickly work around this by just specifying older dependency versions in our Bitbucket pipelines to match what I had locally. For example:
pip3 install -U awscli==1.18.187 ecs-deploy==1.10.1 botocore==1.19.27 boto3==1.16.27
Hi everyone, I will create a hotfix with the mentioned versions pinned until there is fixed version of botocore.
I have just released hotfix 1.11.1 (https://pypi.org/project/ecs-deploy/)
Can you confirm, that it works as expected and you can deploy your services again?
Am checking on my machine hotfix looks good @fabfuel
Thanks for the immediate fix.
This fixes the issue for us, however, the strict requirement causes pip to complain:
$ pip3 install ecs-deploy awscli
[...]
ERROR: awscli 1.18.217 has requirement botocore==1.19.57, but you'll have botocore 1.19.27 which is incompatible.
Everything is installed nonetheless, so for now we're good. I will implement a more permanent fix by pinning the versions of all components to something that works.
Thanks for the superfast reaction time on this issue!
@fabfuel thanks for the quick response!
I set the install back to the way it was by not pinning boto3 and ecs-deploy versions, but I'm still receiving the same error and seeing this dependency problem:
ecs-deploy 1.11.1 requires boto3<=1.16.27,>=1.14.47, but you'll have boto3 1.16.57 which is incompatible. ecs-deploy 1.11.1 requires botocore<=1.19.27,>=1.17.47, but you'll have botocore 1.19.57 which is incompatible.
@jperez3 Just an idea - maybe you need to uninstall
botocoreandboto3first?
thanks for the suggestion, but boto/ecs-deploy runs in a pipeline/workspace that starts fresh every time. Pinning the older version works for us right now, so we're not dead in the water. I don't want to pressure the maintainer and I'm grateful for this project :)
Hi everyone,
I just created another hotfix, which fixes the issue without the need to bin boto3 or botocore to older versions.
Version 1.11.2 has just been pushed to PyPI: https://pypi.org/project/ecs-deploy/1.11.2/
I had no time to dive into the details yesterday, but now I could figure out what happened. There was a similar situation back in 2017, when the parameter compatibilities was introduced. This and the new parameters registeredBy and registeredAt are returned from the ECS API (via DescribeTaskDefinition), but may not be send back to register a new task definition revision (via RegisterTaskDefinition).
The ECS API does not support this kind of round trip out of the box - that's one of the reasons for this project 🙂
From the current standpoint, it's not a bug in botocore (as assumed before), but a feature request for the ECS API to allow this kind of round trip natively, it would help a lot to avoid issues like that in the future.
Can you confirm, that the new version installs and works again as expected?
Best
Fabian