Manage Istio ExtensionProvider's via charm

[valexby]

Context

There is currently no manageable way to configure Istio enable ExtensionProvider's. The useful business application of it is to enable the Istio External Authorization for the charmed istio.
Currently, the only way to enable External Auth is to manually update istio configmap, but charm becomes unable to manage istio after that.

What needs to get done

1. istio configmap seems to be generated by some of the istio charms, I suppose by istio-ingressgateway.
2. Add configuration key extension_providers to the istio-ingressgateway charm. This config key should accept yaml and charm should install the value into istio configmap into .data.mesh.extensionProviders array.

Definition of Done

1. update configuration of extensionProviders like

juju config istio-ingressgateway extension_providers="""
extensionProviders:
- name: "ext-authz-http-prd"
  envoyExtAuthzHttp:
    service: <>.svc.cluster.local
    port: 9700
    includeRequestHeadersInCheck: ["jwt-claim"]
    includeRequestBodyInCheck:
      maxRequestBytes: 2048
      allowPartialMessage: true
- name: "ext-authz-grpc-prd"
  envoyExtAuthzGrpc:
    service: <>.svc.cluster.local
    port: 9090
    includeRequestBodyInCheck:
      maxRequestBytes: 2048
      packAsBytes: true
      allowPartialMessage: true
"""

2. After charm is settled, istio config map get the value provided above populated into .data.mesh.extensionProviders.

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5461.

This message was autogenerated

[DnPlas]

Thanks @valexby for reporting this. As you correctly point out, there is no way in which our charm exposes any configuration option to enable ExtensionProviders. It is also true that these values can be directly modified in the istio ConfigMap that gets created when the Istio control plane gets deployed. This is entirely done when istio-pilot is deployed, though.

Also, question for you @valexby, are you proposing this enhancement for istio-operators 1.17 or edge?

The solution is not as trivial as providing a configuration option to which we can pass the yaml bits to edit the ConfigMap, this is because even though the istio-pilot charm owns the resource, it is the istioctl the ones that creates and manages it.

Instead we need to expose the --set meshConfig.ExtensionProvider extra argument so the istioctl that is wrapped by our charm knows what to do with that extra option. I think the tricky part is that we need to expose all the fields listed here and the fields of each field (i.e. MeshConfig.ExtensionProvider.EnvoyExternalAuthorizationHttpProvider has fields like service and port) to be able to properly configure the extension provider.

I'm thinking of something like an action that allows users to perform this operation:

$ juju run istio-pilot/0 set-extension-provider name=some-name provider=envoyExtAuthzHttp extra-fields={dictionary of extra fields?}

or

$ juju run istio-pilot/0 set-extension-provider name=some-name provider={json with all fields and sub-fields}

If we'd like to avoid actions and go with charm configs instead, we could pass all the values at once, maybe as a json(?)

$ juju config istio-pilot extension-provider-metadata=<json with all fields>

Potential changes in istio-pilot

1. Add an action/config option with all the required params
2. Observe the new action/config changed events and create methods to handle it
3. Parse the MeshConfig.ExtensionProvider fields provided via the action params or the config option
4. Every time the action runs or the config is changed, update the control plane with the new mesh configurations (e.g. --set meshConfig.<something>.<something>)

[ca-scribner]

To test this during implementation, we can use the example from istio that @valexby mentioned in the original post. It has an external authorizer that should be useful in our tests

[ca-scribner]

This feels like configuration, so I'd rather put this into config than an action if we can help it. That way it can be set via a bundle too.

But I do hate when multiple config values are needed for something. Juju doesn't give us a nice "set all of these or none of these" mechanism in config

[ca-scribner]

After reading through the linked references, I don't think implementing this will cause any issues on the Charmed Kubeflow side. These settings only come into play on AuthorizationPolicy objects that have {action: CUSTOM, provider: {name: NAME_OF_EXTENSION_PROVIDER}}.