Native cloud options (e.g. ~/.aws/config) get used inconsistently
Closed this issue · 5 comments
When using a toml config file the standard cloud config (e.g. ~/.aws/config) is still used by cloud libraries like boto3, but those config options are ignored by pycloudlib native operations. For example a region can be specified in ~/.aws/config, and that will be used when creating instances, but it's ignored when searching for images here:
pycloudlib/pycloudlib/ec2/cloud.py
Line 316 in e52cc25
@paride , do you think it's better to point to the aws config instead? My goal in moving to pycloudlib.toml was to move as much config out of cloud-specific utilities and into a common configuration file. The [ec2] section has a region key. Do you think it would be better to remove region from pycloudlib.toml and just defer to the config?
@TheRealFalcon I see one issue in doing so: there are parts of pycloudlib that won't honor the region set in .aws/config, for example released_image() and daily_image().
In other words, a project using pycloudlib can request a pycloydlib.EC2() object without specifying a region. All calls using boto will fallback to the region specified in .aws/config, but not released_image() and daily_image(), and we'll end up with inconsistent behavior.
Maybe we can make pycloudlib query boto3 for the region it's going to use, and use it as a fallback if the region is not specified in the toml file?
As our goal is to use the config to make test runs more easily reproducible, I think the region should stay in the toml, as there are resources or configs which are region-specific.
In other words, a project using pycloudlib can request a pycloydlib.EC2() object without specifying a region
How can that happen? With https://github.com/canonical/pycloudlib/blob/main/pycloudlib/ec2/cloud.py#L48 , if the region isn't passed directly to the API or found in the config, we should get an exception.
I tried to reproduce the issue I filed this bug for, but I can't. I think I had a stray region = "" in my toml config, and I've got confused by errors looking like "region not set". I think we can close this!
I think we can archive this as user error, where the the user is me.