Credentials issue introduced in 0.27.0 when running on EC2 instance.
mjharvey opened this issue · 7 comments
mjharvey commented
Hi. I have been using awscurl on an EC2 instance to call an API endpoint that is secured via IAM permissions.
This worked fine under 0.26.0, but with the release of 0.27.0 I am now encountering this error
File "/usr/local/bin/awscurl", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/awscurl/awscurl.py", line 543, in main
inner_main(sys.argv[1:])
File "/usr/local/lib/python3.6/site-packages/awscurl/awscurl.py", line 508, in inner_main
args.profile)
File "/usr/local/lib/python3.6/site-packages/awscurl/awscurl.py", line 427, in load_aws_config
cred = session.get_credentials()
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 449, in get_credentials
'credential_provider').load_credentials()
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 897, in get_component
self._components[name] = factory()
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 176, in <lambda>
lambda: botocore.credentials.create_credential_resolver(self))
File "/usr/local/lib/python3.6/site-packages/botocore/credentials.py", line 55, in create_credential_resolver
metadata_timeout = session.get_config_variable('metadata_service_timeout')
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 265, in get_config_variable
elif self._found_in_config_file(methods, var_config):
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 286, in _found_in_config_file
return var_config[0] in self.get_scoped_config()
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 358, in get_scoped_config
raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (default) could not be found
The problem looks like it might have been caused by this change: 7b38c7f?diff=split?diff=split?diff=split%3Fdiff%3Dsplit
Is this a known issue, or this there a workaround for the problem I could apply?
Thanks.
okigan commented
Let me know how you invoke awscurl, and also how do you specify the
credentials (environment variable, command line, etc).
And thanks for reporting the issue.
…On Tue, May 2, 2023 at 3:35 PM Matt Harvey ***@***.***> wrote:
Hi. I have been using awscurl on an EC2 instance to call an API endpoint
that is secured via IAM permissions.
This worked fine under 0.26.0, but with the release of 0.27.0 I am now
encountering this error
File "/usr/local/bin/awscurl", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/awscurl/awscurl.py", line 543, in main
inner_main(sys.argv[1:])
File "/usr/local/lib/python3.6/site-packages/awscurl/awscurl.py", line 508, in inner_main
args.profile)
File "/usr/local/lib/python3.6/site-packages/awscurl/awscurl.py", line 427, in load_aws_config
cred = session.get_credentials()
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 449, in get_credentials
'credential_provider').load_credentials()
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 897, in get_component
self._components[name] = factory()
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 176, in <lambda>
lambda: botocore.credentials.create_credential_resolver(self))
File "/usr/local/lib/python3.6/site-packages/botocore/credentials.py", line 55, in create_credential_resolver
metadata_timeout = session.get_config_variable('metadata_service_timeout')
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 265, in get_config_variable
elif self._found_in_config_file(methods, var_config):
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 286, in _found_in_config_file
return var_config[0] in self.get_scoped_config()
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 358, in get_scoped_config
raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (default) could not be found
The problem looks like it might have been caused by this change: 7b38c7f
?diff=split?diff=split?diff=split%3Fdiff%3Dsplit
<7b38c7f?diff=split?diff=split?diff=split%3Fdiff%3Dsplit>
Is this a known issue, or this there a workaround for the problem I could
apply?
Thanks.
—
Reply to this email directly, view it on GitHub
<#163>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AADUYXVYESPMQBZQX2BWT3TXEFV2RANCNFSM6AAAAAAXTRIPQM>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
mjharvey commented
There are no environment variables or profile setup on the EC2 instance for authentication. What I am guessing was happening previously was the session = botocore.session.get_session() gets credentials for the execution role associated with the EC2 instance (I may be wrong though).
okigan commented
Alright I’ll recheck on ec2 instance. Which service are you accessing (ec2, lambda..)?On May 2, 2023, at 6:22 PM, Matt Harvey ***@***.***> wrote:
There are no environment variables or profile setup on the EC2 instance for authentication. What I am guessing was happening previously was the session = botocore.session.get_session() gets credentials for the execution role associated with the EC2 instance (I may be wrong though).
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>
mjharvey commented
Lambda via API Gateway endpoint.
The API gateway endpoint is secured via IAM policy that restricts access based on AWS Organisation Units.
…________________________________
From: Igor Okulist ***@***.***>
Sent: Wednesday, 3 May 2023 11:42 am
To: okigan/awscurl ***@***.***>
Cc: Matt Harvey ***@***.***>; Author ***@***.***>
Subject: Re: [okigan/awscurl] Credentials issue introduced in 0.27.0 when running on EC2 instance. (Issue #163)
Alright I’ll recheck on ec2 instance. Which service are you accessing (ec2, lambda..)?On May 2, 2023, at 6:22 PM, Matt Harvey ***@***.***> wrote:
There are no environment variables or profile setup on the EC2 instance for authentication. What I am guessing was happening previously was the session = botocore.session.get_session() gets credentials for the execution role associated with the EC2 instance (I may be wrong though).
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>
—
Reply to this email directly, view it on GitHub<#163 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABHSEFSDHKDNVKBKOIMUKVDXEGLUTANCNFSM6AAAAAAXTRIPQM>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
okigan commented
got pulled in with other items -- will review asap.
okigan commented
Capturing nice way to repro in aws cloud shell:
[cloudshell-user@ip-10-4-127-13 ~]$ pip3 install -q awscurl==0.26
[cloudshell-user@ip-10-4-127-13 ~]$ awscurl --service s3 https://awscurl-sample-bucket.s3.amazonaws.com
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>awscurl-sample-bucket</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Contents><Key>awscurl-sample-file.txt</Key><LastModified>2017-07-25T21:27:38.000Z</LastModified><ETag>"d41d8cd98f00b204e9800998ecf8427e"</ETag><Size>0</Size><Owner><ID>978427f78b00827efacb8fe2bd55ea30cbcb1d228cd3758972314cb67e763402</ID><DisplayName>okigan</DisplayName></Owner><StorageClass>STANDARD</StorageClass></Contents></ListBucketResult>
[cloudshell-user@ip-10-4-127-13 ~]$ pip3 install -q awscurl==0.27
[cloudshell-user@ip-10-4-127-13 ~]$ awscurl --service s3 https://awscurl-sample-bucket.s3.amazonaws.com
Traceback (most recent call last):
File "/home/cloudshell-user/.local/bin/awscurl", line 8, in <module>
sys.exit(main())
File "/home/cloudshell-user/.local/lib/python3.7/site-packages/awscurl/awscurl.py", line 543, in main
inner_main(sys.argv[1:])
File "/home/cloudshell-user/.local/lib/python3.7/site-packages/awscurl/awscurl.py", line 508, in inner_main
args.profile)
File "/home/cloudshell-user/.local/lib/python3.7/site-packages/awscurl/awscurl.py", line 427, in load_aws_config
cred = session.get_credentials()
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 508, in get_credentials
'credential_provider'
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 1108, in get_component
self._components[name] = factory()
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 187, in _create_credential_resolver
self, region_name=self._last_client_region_used
File "/usr/local/lib/python3.7/site-packages/botocore/credentials.py", line 74, in create_credential_resolver
metadata_timeout = session.get_config_variable('metadata_service_timeout')
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 318, in get_config_variable
logical_name
File "/usr/local/lib/python3.7/site-packages/botocore/configprovider.py", line 426, in get_config_variable
return provider.provide()
File "/usr/local/lib/python3.7/site-packages/botocore/configprovider.py", line 628, in provide
value = provider.provide()
File "/usr/local/lib/python3.7/site-packages/botocore/configprovider.py", line 718, in provide
scoped_config = self._session.get_scoped_config()
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 416, in get_scoped_config
raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (default) could not be found
[cloudshell-user@ip-10-4-127-13 ~]$
okigan commented
Output from the latest release:
[cloudshell-user@ip-10-2-12-122 ~]$ pip3 install -q awscurl==0.28
[cloudshell-user@ip-10-2-12-122 ~]$ awscurl --service s3 https://awscurl-sample-bucket.s3.amazonaws.com
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>awscurl-sample-bucket</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Contents><Key>awscurl-sample-file.txt</Key><LastModified>2017-07-25T21:27:38.000Z</LastModified><ETag>"d41d8cd98f00b204e9800998ecf8427e"</ETag><Size>0</Size><Owner><ID>978427f78b00827efacb8fe2bd55ea30cbcb1d228cd3758972314cb67e763402</ID><DisplayName>okigan</DisplayName></Owner><StorageClass>STANDARD</StorageClass></Contents><Contents><Key>awscurl-sample-file:.txt</Key><LastModified>2023-05-05T16:03:34.000Z</LastModified><ETag>"d41d8cd98f00b204e9800998ecf8427e"</ETag><Size>0</Size><Owner><ID>978427f78b00827efacb8fe2bd55ea30cbcb1d228cd3758972314cb67e763402</ID><DisplayName>okigan</DisplayName></Owner><StorageClass>STANDARD</StorageClass></Contents></ListBucketResult>