SDP Operator is a Kubernetes operator to configure an Appgate SDP system.
The following tools are required to install the SDP Operator
- helm v3.8.0+ - https://helm.sh/docs/intro/install/
- kubectl - https://kubernetes.io/docs/tasks/tools/#kubectl
Browse available versions by navigating to the SDP Operator Releases
- Add
appgate/sdp-operatorto your Helm repositoryhelm repo add appgate https://appgate.github.io/sdp-operator/ helm repo update
- Install the SDP Operator CRD
helm install sdp-operator-crd appgate/sdp-operator-crd \ --namespace sdp-system \ --create-namespace
Standard Mode pushes entities on Kubernetes to an Appgate SDP system
-
Create a secret containing Admin API credentials
kubectl create secret generic sdp-operator-secret \ --from-literal=appgate-operator-user="<USERNAME>" \ --from-literal=appgate-operator-password="<PASSWORD>" \ --namespace sdp-system -
Install the SDP Operator in Standard Mode
sdp: operators: - sdp-operator sdpOperator: version: v18 host: "https://sdp.appgate.com:8443" deviceId: "00000000-1111-2222-3333-44444444" secret: sdp-operator-secret reverseMode: false
helm install normal-operator appgate/sdp-operator --values normal-operator.yaml --namespace sdp-system
Reverse Mode pulls SDP entities from an Appgate SDP system into Kubernetes
- Create a secret containing Admin API credentials
kubectl create secret generic sdp-operator-secret \ --from-literal=appgate-operator-user="<USERNAME>" \ --from-literal=appgate-operator-password="<PASSWORD>" \ --namespace sdp-system - Install the SDP Operator in Reverse Mode
sdp: operators: - sdp-operator sdpOperator: version: v18 host: "https://sdp.appgate.com:8443" deviceId: "00000000-1111-2222-3333-44444444" secret: sdp-operator-secret reverseMode: true
helm install reverse-operator appgate/sdp-operator --values reverse-operator.yaml --namespace sdp-system
Git Operator pushes SDP entities on Kubernetes to a Git repository and create pull requests on GitHub or GitLab
- Create a secret containing SSH key and GitHub token
kubectl create secret generic github-operator-secret \ --from-literal=github-token="<GITHUB_TOKEN>" \ --from-file=git-ssh-key="<SSH_KEY_PATH>" - Install the Git Operator for GitHub
sdp: operators: - git-operator gitOperator: version: v18 secret: github-operator-secret git: vendor: github mainBranch: main baseBranch: main repository: <ORGANIZATION_OR_USER/REPOSITORY>
helm install github-operator appgate/sdp-operator --values github-operator.yaml --namespace sdp-system
-
Create a secret containing SSH key and GitLab token
kubectl create secret generic gitlab-operator-secret \ --from-literal=gitlab-token="<GITLAB_TOKEN>" \ --from-file=git-ssh-key="<SSH_KEY_PATH>" -
Install the Git Operator for GitLab (SaaS)
sdp: operators: - git-operator gitOperator: version: v18 secret: gitlab-operator-secret git: vendor: gitlab mainBranch: main baseBranch: main repository: <ORGANIZATION_OR_USER/REPOSITORY>
helm install gitlab-operator appgate/sdp-operator --values gitlab-operator.yaml --namespace sdp-system
-
Generate a file containing the host key fingerprint of the self-hosted GitLab
ssh-keyscan <GITLAB_HOSTNAME> > <SSH_HOST_KEY_FINGERPRINT_PATH>
-
Create a secret containing SSH key and GitLab token
kubectl create secret generic gitlab-operator-secret \ --from-literal=gitlab-token="<GITLAB_TOKEN>" \ --from-file=git-ssh-key="<SSH_KEY_PATH>" \ --from-file=git-ssh-host-key-fingerprint="<SSH_HOST_KEY_FINGERPRINT_PATH>" -
Install the Git Operator for self-hosted GitLab
sdp: operators: - git-operator gitOperator: version: v18 secret: gitlab-operator-secret git: vendor: gitlab mainBranch: main baseBranch: main repository: <ORGANIZATION_OR_USER/REPOSITORY> hostname: <GITLAB_HOSTNAME>
helm install gitlab-operator appgate/sdp-operator --values gitlab-operator.yaml --namespace sdp-system
For the list of available Helm parameters, please go to Helm Values
The SDP Operator supports different of ways of specifying what to sync and how:
- Configuring what entity tags to sync :: To filter based on the tags of the actual entities
- Configuring what entity kinds to sync :: To only fetch entities of specific kinds
Note that both can be used together.
We can specify what tags the operator will work on via the environemnt variable APPGATE_OPERATOR_TARGET_TAGS. This variable supports a list of tags (separated by comma) to sync. This means that entities that have any of those tags will be managed by the operator.
Example:
APPGATE_OPERATOR_TARGET_TAGS="devops,dev"With that environemnt variable defined the operator will only create, modify or delete entities that any of the tags devops or dev (ot both at the same time).
We can also specify explicitly that we don't want to manage entities with specific tags. This can be done with the environemnt variable APPGATE_OPERATOR_EXCLUDE_TAGS. Entities with any of the tags listed there won't be deleted.
Note that if an entity has both tags it won't be deleted or if a tag is in both sets it will prevent entities with that tag to be deleted.
We saw how to use tags to decide what entities the operator should manage. In some occasions we don't want to load entities of some specific kind at all. To achive that we have 2 more environment variables:
APPGATE_OPERATOR_INCLUDE_ENTITIES:: list of entity kinds we want to manageAPPGATE_OPERATOR_EXCLUDE_ENTITIES:: list of entity kinds we don't want to manage
The list of entity kinds to use will be computed from the contents of those two environment variables. Note that APPGATE_OPERATOR_INCLUDE_ENTITIES has always precedence, this means that if we have the same entity kind in both sets then it will be included.
When the list of entity kinds is computed the operator will only create clients (SDP client, k8s watchers and git clients) for those entity kinds there. In practice that means that it won't load entities belonging to another kind.
Example:
APPGATE_OPERATOR_INCLUDE_ENTITIES="TrustedCertificate,Entitlement,Policy"If the operator runs with that environment variable it will only create 3 SDP clients (for TrustedCertificate, Entitlements and Policies) and 3 K8S watchers.
By design, the Admin API does not return sensitive information and binary data in its response. That means, you cannot sync any entity from Collective A to Collective B if it contains any secret or file fields.
Suppose you want to sync a LocalUser from Collective A to Collective B. The reverse operator will sync the LocalUser entity from Collective A into Kubernetes without the password field (because the response to GET /localuser does not contain such field). If this entity is pushed to Collective B as-is, it would fail because password is a required value. To fix this issue, you can configure the normal operator to fetch and load the password value in memory before pushing it to Collective B.
The same logic applies to binary data. DeviceScript is an entity that contains a binary data in field file. By configuring an external source for files, you can load the binary data before pushing the entity to target collective.
SDP Operator expects secrets and files to be uploaded to the store in a specific format:
{Entity Type}-{API Version}/{Name}/{Key}
The following secret source are supported:
- Hashicorp Vault
Create a secret containing the Vault token
kubectl create secret generic sdp-operator-vault-secret \
--from-literal=vault-token="<VAULT_TOKEN>"Install the SDP Operator with Vault
sdp:
externalSecret:
enabled: true
type: vault
source:
vault:
address: "https://vault.example.com:8200"
tokenSecret: sdp-operator-vault-secret
# Note: Other required helm values are omitted for simplicitySDP Operator expects secrets to be stored under /data/secret/sdp
Initialize the sdp path by uploading data for v18 LocalUser (username=john.doe password=password123)
vault kv put secret/sdp localuser-v18/john.doe/password="password123"On subsequent uploads, you can use vault kv patch to update the secret. For example, to upload another v18 LocalUser (username=jane.doe, password=password123), run the following:
vault kv patch secret/sdp localuser-v18/jane.doe/password="password123"The following external file source are supported:
- HTTP
- S3
sdp:
externalFile:
enabled: true
type: http
source:
http:
address: "https://example.com:8000"
# Note: Other required helm values are omitted for simplicitySDP Operator expects the files to be stored at the root of the filesystem.
Upload the file example.sh for v18 DeviceScript under the following path:
/devicescript-v18/example/file
Create secret containing the access key and secret key for the S3 storage
kubectl create secret generic sdp-operator-s3-secret \
--from-literal=access-key="<S3_ACCESS_KEY>"
--from-literal=secret-key="<S3_SECRET_KEY>"Install the SDP Operator with S3
sdp:
externalFile:
enabled: true
type: s3
source:
s3:
address: "http://example.com:8000"
tokenSecret: sdp-operator-http-secret
# Note: Other required helm values are omitted for simplicitySDP Operator expects the files to be stored under bucket sdp
Create a bucket /sdp on the root of the object store
Upload the file example.sh for v18 DeviceScript under the following path:
/sdp/devicescript-v18/example/file
Sensitive information can be stored in the YAML as encrypted secrets using fernet (symmetric encryption).
To generate a new fernet key, run:
python3 -c 'from cryptography.fernet import Fernet;print(Fernet.generate_key().decode())'To generate a secret with the key, run:
export SECRET="super-sensitive-information"
export KEY="dFVzzjKCa9mWbeig8dprliGLCXwnwE5Fbycz4Xe2ptk="
python3 -c 'from cryptography.fernet import Fernet;import os;print(Fernet(os.getenv("KEY")).encrypt(bytes(os.getenv("SECRET").encode())))'After generating the secret, the encrypted value can be safely stored inside the YAML file as plain-text. When the operator encounters such field, it will read the value of environment variable APPGATE_OPERATOR_FERNET_KET to decrypt and read secrets in entities.
sdp:
sdpOperator:
fernetKey: "dFVzzjKCa9mWbeig8dprliGLCXwnwE5Fbycz4Xe2ptk="CA certificate in PEM format can be stored in env APPGATE_OPERATOR_CACERT. We recommend storing the contents of the PEM as a base64 encoded string.
sdp:
sdpOperator:
caCert: "$(cat cert.ca)"dump-entities command will read the entities from an existing SDP system and dump them into YAML.
$ python3 -m appgate --spec-directory /appgate/appgate/api_specs/v15 dump-entitiesThe above command will generate a directory that contains the YAML
$ ls -l example-v15/
total 52
-rw-r--r-- 1 root root 1756 Jul 20 23:03 administrativerole.yaml
-rw-r--r-- 1 root root 3537 Jul 20 23:03 appliance.yaml
-rw-r--r-- 1 root root 143 Jul 20 23:03 clientconnection.yaml
-rw-r--r-- 1 root root 184 Jul 20 23:03 condition.yaml
-rw-r--r-- 1 root root 194 Jul 20 23:03 criteriascripts.yaml
-rw-r--r-- 1 root root 332 Jul 20 23:03 globalsettings.yaml
-rw-r--r-- 1 root root 1405 Jul 20 23:03 identityprovider.yaml
-rw-r--r-- 1 root root 423 Jul 20 23:03 ippool.yaml
-rw-r--r-- 1 root root 251 Jul 20 23:03 localuser.yaml
-rw-r--r-- 1 root root 479 Jul 20 23:03 mfaprovider.yaml
-rw-r--r-- 1 root root 1054 Jul 20 23:03 policy.yaml
-rw-r--r-- 1 root root 695 Jul 20 23:03 ringfencerule.yaml
-rw-r--r-- 1 root root 602 Jul 20 23:03 site.yamlvalidate-entities command will validate the compatibility of entities against a version of the OpenAPI specification. This is useful for verifying if entities dumped from one SDP system is compatible with another SDP system.
$ python3 -m appgate --spec-directory /appgate/appgate/api_specs/v17 validate-entities examples-v15/In the example above, we validated the v15 entities (generated by dump-entities command) to a v17 OpenAPI specification. The command will attempt to load all entities defined in examples-v15-entities/ as a v17 entities, reporting errors if encountered any.
To support a new version of the API, you need to update bin/get-open-spec.sh and bin/unzip-open-spec.sh. Once it is updated, run
$ make api-specs
to download the new API specs. Then, run
$ VERSION=<new API version> make dump-crd
which will read the new specification, generate the CRDs, and dump them into k8s/crd/templates/<new API version>.yaml
Custom Resource Definitions (CRD) on Kubernetes allow the operator to represent Appgate SDP entities as YAMLs. Each instance of an entity is stored as a Custom Resource. The operator reads each instance's spec and syncs the entity with the controller using the Admin API. SDP Operator consumes a version from Appgate SDP OpenAPI Spec to generate entities for a given version of the controller.
For each entity configured, the operator begins a timer and an event loop to listen for any changes happening on the cluster. Every time an event is received, the timer resets. After the timeout period has expired (in other words, when no event has newly arrived), the operator proceeds to compute a Plan. A Plan represents the difference between the current state on the controller vs the desired state defined in Kubernetes - it outlines what entities will be created/updated/deleted on the controller. After the plan is computed, the operator to execute the Plan on the controller using the API in order to produce the desired state.
It is important to note that, by design, any state defined in Kubernetes wins over state in the SDP system - any external changes made outside the operator will be overwritten. For example, if an administrator makes a change to the Policy via admin UI, the operator will determine the change as 'out-of-sync- and undoes the change.