datasource "ami" failed to render template when using aws config with profile using "sso_session" parameter.

Question

datasource "ami" failed to render template when using aws config with profile using "sso_session" parameter.

steffakasid opened this issue a year ago · 14 comments

Hi,

I have a '~/aws' config file which looks like

[default]
sso_session = sso-session
sso_account_id = ACCOUNT_ID_TEST
region = eu-central-1
output = json

[profile test]
cli_pager=
retry_mode = adaptive
max_attempts = 20
output = json
region = eu-central-1
sso_session = sso-session
sso_account_id = ACCOUNT_ID_TEST

[profile prod]
cli_pager=
retry_mode = adaptive
max_attempts = 20
output = json
region = eu-central-1
sso_session = sso-session
sso_account_id = ACCOUNT_ID_PROD

[sso-session sso-session]
sso_region = eu-central-1
sso_start_url = https://my-sso.endpoint
sso_role_name = myRole

This works fine with aws cli and kubectl. Unfortunately when I use gomplate I get the following error:

12:11:25 ERR  error="failed to render template cluster.template.yaml: template: cluster.template.yaml:3:15: executing \"cluster.template.yaml\" at <datasource \"ami\">: error calling datasource: profile \"test\" is configured to use SSO but is missing required configuration: sso_region, sso_role_name, sso_start_url"

I'm using the ami datasource like {{- $amiID := (datasource "ami").Value -}}

gomplate version is gomplate version 3.11.5`.

This might be an issue of https://github.com/aws/aws-sdk-go. So not sure if this is here the right place. If not please let me know.

Kind regards
Steffen

Answer 1 · 2023-11-10T22:38:39.000Z

Hi @steffakasid - sorry for the delay. I presume you're setting AWS_PROFILE=test which is how it's picking up that profile?

I wonder if this is a bug with the SDK - unfortunately gomplate's using a fairly old version of it even in the main branch, due to a bug that I haven't had time to track down.

Perhaps you could build from main and see if that works?

Also - what is the full commandline? In particular, how is the ami datasource defined?

Answer 2 · 2023-11-13T08:31:16.000Z

HI @hairyhenderson,

I'll try to build the main branch with the latest version later and let you know if that work. I also think that this is a bug in the aws go sdk. I use it in one of my projects and there everything works fine...

The datasource and command run is the follwing:

export K8S_VERSION=$(kubectl version -o json | jq -r '.serverVersion.major+"."+.serverVersion.minor[:2]')

gomplate -f cluster.template.yaml -d config=config.yaml -d ami=aws+smp:/eks/$K8S_VERSION/image_id

Kind Regard
Steffen

Answer 3 · 2023-11-13T08:49:15.000Z

Do you think it would be easily possible to switch to aws-sdk-go-v2?

https://aws.github.io/aws-sdk-go-v2/docs/migrating/

looks like especially the handling of configuration changed.

Answer 4 · 2023-11-13T09:06:50.000Z

I upggraded the aws-sdk-go:

And it works. My template got rendered and I can see the AMI_ID is set.

Answer 5 · 2023-11-17T09:39:36.000Z

I was wrong. I missed to change my ~/.aws/config. I'll check if I can change gomplate to use aws-sdk-go-v2.

Answer 6 · 2023-11-22T14:45:45.000Z

We can't move to aws-sdk-go-v2 just yet in gomplate - but when v4 is released it'll move to using go-fsimpl, which uses aws-sdk-go-v2 (see #1336).

I've been able to figure out the broken tests that were blocking me from upgrading aws-sdk-go, and so as of #1913 we're now up-to-date there.

Answer 7 · 2024-01-22T02:17:12.000Z

This issue is stale because it has been open for 60 days with no
activity. If it is no longer relevant or necessary, please close it.
Given no action, it will be closed in 14 days.

If it's still relevant, one of the following will remove the stale
marking:

A maintainer can add this issue to a milestone to indicate that
it's been accepted and will be worked on
A maintainer can remove the stale label
Anyone can post an update or other comment

Answer 8 · 2024-01-22T07:50:49.000Z

Not stale.

Answer 9 · 2024-01-25T21:54:40.000Z

@steffakasid FYI there's a 4.0 prerelease available now - can you test it out and see if that solves this issue?

Answer 10 · 2024-01-25T23:05:28.000Z

Sure i'll take a look tomorrow at work.

Answer 11 · 2024-01-26T09:12:54.000Z

It looks like this is still an issue even with aws-sdk-go-v2. I'll create an issue there: aws/aws-sdk-go#5153

Answer 12 · 2024-01-26T14:34:19.000Z

Hrm... One other thing you could test, which might rule out something gomplate is doing, is try with the example fscli command in go-fsimpl. Install it with go install github.com/hairyhenderson/go-fsimpl/examples/fscli@latest and then try something like:

$ fscli -base-url=aws+smp:///eks/1.27/image_id ls

Answer 13 · 2024-01-26T14:49:25.000Z

the aws+smp stuff is working fine. When I change back my .aws/config I can run the template with gomplat v4

Answer 14 · 2024-03-27T04:45:12.000Z

This issue is stale because it has been open for 60 days with no
activity. If it is no longer relevant or necessary, please close it.
Given no action, it will be closed in 14 days.

If it's still relevant, one of the following will remove the stale
marking:

A maintainer can add this issue to a milestone to indicate that
it's been accepted and will be worked on
A maintainer can remove the stale label
Anyone can post an update or other comment