AWS SDK latency

If you were debugging tail latency in AWS Go SDK, you would probably try to trace the requests using httptrace and realize that at least one second is spent at Sign step.

Jinli Liang from Rokt wrote a great explanation of what's going on. In short, there are three issues:

by default all AWS STS requests go to a single endpoint at https://sts.amazonaws.com. AWS recommends using Regional AWS STS endpoints instead of the global endpoint to reduce latency, build in redundancy, and increase session token validity.
increased latency from AWS STS request made by an SDK client during application startup
increased latency when credentials expiry

This repository offers slightly refactored version of the code from the Rokt's post.

Swapper

package main

import (
	"context"
	"os"
	"os/signal"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
	"github.com/go-kit/log"
	"github.com/marselester/awscreds"
)

func main() {
	logger := log.NewJSONLogger(log.NewSyncWriter(os.Stderr))

	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
	defer stop()

	sess := session.Must(session.NewSession(&aws.Config{}))
	s3 := s3.New(sess)

	s, err := awscreds.NewSwapper(
		awscreds.New,
		awscreds.WithLogger(logger),
	)
	if err != nil {
		logger.Log("msg", "failed to get aws credentials", "err", err)
		return
	}
	s.Attach(s3.Client)
	s.Run(ctx)
}

There is also an option to refresh existing credentials.

Refresher

package main

import (
	"context"
	"os"
	"os/signal"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/go-kit/log"
	"github.com/marselester/awscreds"
)

func main() {
	logger := log.NewJSONLogger(log.NewSyncWriter(os.Stderr))

	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
	defer stop()

	sess := session.Must(session.NewSession(&aws.Config{}))

	r, err := awscreds.NewRefresher(
		sess.Config.Credentials,
		awscreds.WithLogger(logger),
	)
	if err != nil {
		logger.Log("msg", "failed to get aws credentials", "err", err)
		return
	}
	r.Run(ctx)
}