equinixmetal-archive/packngo

Packngo should offer broader client configuration options

displague opened this issue · 4 comments

displague commented

Tools that implement the Packet API using this wrapper invariably evolve the need to create a means to load the Packet API from the environment they are running in.

It is common in other cloud provider Go APIs to find functions like:

compute.NewService https://godoc.org/google.golang.org/api/compute/v1

Functions like this will follow a predetermined path to auto-configure the client:

  • examine the environment for one or more configuration file paths or configuration behavior modifiers (ignore metadata, for example)
  • examine the environment for raw configuration (including tokens)
  • examine the environment for alternate metadata locations
  • examine configuration made available through default service accounts, IAM, etc
  • examine all configurations to merge a client configuration

A function such as this may configure all of the existing Client configuration values:

  • token
  • debug
  • API URL
  • User-Agent / ConsumerID (I'm not certain that having this configurable outside of cod is valuable)

Once the client has been returned, it may offer interfaces to access default values for fields such as a default organization id, project id, device plan, and operating system.

With the metadata service available, the default facility and project id could be determined, for example.

Given only a token, this interface could determine the default organization and project id through the user profile of the token or (especially in the case of a project token) the organization or projects available (when there is only one available).

This interface could be used in:

What does it look like? (I'm looking for input here. Does this take some responsibilities away from NewClient? Does NewClient need to be refactored to support this?)

import "github.com/packethost/packngo/pkg/service"

func main() {
// What would "..." be here?
svc := service.NewSession(...) // what errors could this return?
client := svc.NewClient(...) // what errors could this return?

service.NewSession(..., service.ConfigFromEnv(), service.ConfigFromConfig(), service.ConfigFromMetadata())
}
displague commented

I've been thinking about the format of this sort of configuration file, with future considerations in mind.

Suppose we had the following format:

# ~/.config/equinix/global.yaml
# settings at this level apply to all clients, ignored by clients that don't understand the key + value
metal:
  # settings at this level apply to api.equinix.com/metal clients
  # Alternatively, ~/.config/equinix/metal.yaml can be used for this level
  v1:
    # settings at this level apply to api.equinix.com/metal/v1 clients
    project_id: ... # default project, available through the client. When unset, client will use the preferences stored in the API
    organization_id: ... # default org, ^
    os: ... # default os, clients can request this from the configuration service
    facility: ... # default facility, ^
    token: ...
    scope: project # alternative would be "user"
    rotate_after: 2021-03-17T10:10:10.50Z # Is this a worthwhile client feature? could it race other clients?
    rotate_interval: 1d
   
    clients:
    # settings at this level apply to api.equinix.com/metal/v1 clients with a matching user-agent
    - user_agent: "/Terraform/" # regex
      debug: true
      token: ... # alternate

A single configuration file could serve all Equinix-related client functions. This is helpful because the Equinix API is consumed behind a single base URL: api.equinix.com. Alternate base URLs could be provided globally. Perhaps this file should be called global.yaml.

When a "Metal" client is in use, the configuration search path could go as follows:

  • ~/.config/ (or the Windows equivalent) equinix/metal.yaml
  • ~/.config/ (or the Windows equivalent) equinix/metal.json # ?
  • ~/.config/ (or the Windows equivalent) equinix/global.yaml
  • ~/.config/ (or the Windows equivalent) equinix/global.json # ?

Client configuration would be merged. Clients would map service specific configuration names to the name of the API endpoint. (api.equinix.com/metal/v1)

The Global could serve as client defaults, regardless of service name. Features like debug, base_url, and ANY other configuration could be set at this level. These values will be unioned with other settings, giving preference to settings with greater specificity (metal:, metal: v1:).

global.yaml
|_ servicename section or servicename.yaml(*)
    |_ version or client(*)
       |_ client (-)
       
(*) asterisked path has higher specificity
(-) client can have no `version` or `client` descendants. version can only have `client` descendents. `client` must include `user_agent`.

What happens if the user-agent is changed after client initialization? Metal also uses Consumer tokens, can they be used instead of user_agent? The user-agent toggle is also a stand-in for language SDK client configurations, Go clients and Ruby clients could both use this config file and may have different settings configured.

It is imaginable to have projects: { "_id_": { settings } defined. We have to draw the line somewhere and I am drawing the line above that level.

Other examples:

  • Service specific files (metal.yaml) will have greater specificity than matching sections in global.yaml.
  • Client specific settings in global.yaml would have greater specificity than v1 settings in metal.yaml
displague commented

With token rotation, a client would:

  • check the rotation date of the token
  • attempt to update the configuration file
    • if it can not update the configuration file, it will log a warning and refuse to rotate the token
  • request a new token
    • if this fails, it will log a warning and refuse to rotate the token. the client can still be used.
  • delete the old token
    • if this fails, it will:
      • log a warning
      • verify the token still exists
      • attempt to delete the new token
        • log if this fails
      • the configuration file is unchanged
  • update the appropriate configuration file/section
  • if rotate_interval key exists
    • log warning and do nothing if invalid (client configuration should be prevalidated to prevent this)
    • include a new rotate_after timestamp based on the current datetime
displague commented

The role of scope was not mentioned thus far. I was thinking that this would be written during token rotation. This flag is a hint to the user about the token scope that serves a real purpose during rotation - this informs the type of token the rotation will attempt to generate.

That may be not be possible. Project tokens are not permitted to generate more project tokens. Project scoped token rotation can still be possible if a parent object contains a user scoped key.

Perhaps user and project scoped tokens should use a different field name. A User scoped key could exist at the same level as a project scoped key. When both exist the project scoped key would be used and rotated using the user scoped key.

Alternatively, the word "scope" has a specific OAuth meaning. While the EM API does not use OAuth, we could give client's a scope configuration.

metal:
  v1:
    token: ...
    scope:
    - projects
    - projects.bgp
    - devices
    - read:organization
    # alternatively support space separated values, the oauth2 scope standard

Clients would have to self-restrict based on the scopes provided. Clients would also have to agree on the meaning of the scopes. Baking this special behavior into the client may be ill-advised, especially if the API later introduces a similar concept.

displague commented

Clients may wish to take advantage of this configuration service and extend it with additional well-known files.

Packet CLI uses ~/.packet-cli and only knows about token:. Packet CLI could take advantage of this new client configuration and settings while supporting the legacy format.