OpenShift Cluster Console

External Service Integration Criteria

Any external service that integrates with console should satisfy the following requirements:

The service can be installed on a 3 node cluster with 4GB of RAM per node.
The backend will refuse to create invalid resources. (eg: CPU/RAM requests exceeding limits will result in an HTTP 400 code)
API response time is less than 2 seconds for synchronous actions.
Errors and statuses are propagated to the correct k8s objects for async actions.
The service has basic documentation on debugging. (eg: How do we know if the service is working or not?)
Nice to have: Updates to objects finish on the order of seconds, not minutes.
Nice to have: Owned resources have ownerReferences.
There is an agreed-upon API before any console work is started.

Codename: "Bridge"

The console is a more friendly kubectl in the form of a single page webapp. It also integrates with other services like monitoring, chargeback, ALM, and identity. Some things that go on behind the scenes include:

Proxying the Kubernetes API under /api/kubernetes
Providing additional non-Kubernetes APIs for interacting with the cluster
Serving all frontend static assets
User Authentication
Some additional proxying to the Dex API

Quickstart

Dependencies:

node.js >= 8 & yarn >= 1.3.2
go >= 1.8 & glide >= 0.12.0 (go get github.com/Masterminds/glide) & glide-vc
kubectl and a k8s cluster
jq (for contrib/environment.sh)
Google Chrome/Chromium >= 60 (needs --headless flag) for integration tests

Build everything:

./build

Backend binaries are output to /bin.

Configure the application

Tectonic

If you have a working kubectl on your path, you can run the application with:

export KUBECONFIG=/path/to/kubeconfig
source ./contrib/environment.sh
./bin/bridge

The script in contrib/environment.sh sets sensible defaults in the environment, and uses kubectl to query your cluster for endpoint and authentication information.

To configure the application to run by hand, (or if environment.sh doesn't work for some reason) you can manually provide a Kubernetes bearer token with the following steps.

First get the secret ID that has a type of kubernetes.io/service-account-token by running:

kubectl get secrets

then get the secret contents:

kubectl describe secrets/<secret-id-obtained-previously>

Use this token value to set the BRIDGE_K8S_BEARER_TOKEN environment variable when running Bridge.

OpenShift

Registering an OpenShift OAuth client requires administrative privileges for the entire cluster not just a local project. If you've got a working kubectl and oc on your path, but aren't a system administrator, run the following command to attempt to elevate privileges:

oc login -u system:admin
oc adm policy  --as system:admin add-cluster-role-to-user cluster-admin admin
oc login -u admin

To run bridge locally connected to a remote OpenShift cluster, create an OAuthClient resource with a generated secret and read that secret:

oc process -f examples/tectonic-console-oauth-client.yaml | oc apply -f -
export OAUTH_SECRET=$( oc get oauthclient tectonic-console -o jsonpath='{.secret}' )

If the CA bundle of the OpenShift API server is unavailable, fetch the CA certificates from a service account secret. Otherwise copy the CA bundle to examples/ca.crt:

oc get secrets -n default --field-selector type=kubernetes.io/service-account-token -o json | \
    jq '.items[0].data."service-ca.crt"' -r | openssl base64 -d > examples/ca.crt
# Note: use "openssl base64" because the "base64" tool is different between mac and linux

Set the OPENSHIFT_API environment variable to tell the script the API endpoint:

export OPENSHIFT_API="https://127.0.0.1:8443"

Finally run the Console and visit localhost:9000:

./examples/run-bridge.sh

Docker

The builder-run script will run any command from a docker container to ensure a consistent build environment. For example to build with docker run:

./builder-run ./build

The docker image used by builder-run is itself built and pushed by the script push-builder, which uses the file Dockerfile-builder to define an image. To update the builder-run build environment, first make your changes to Dockerfile-builder, then run push-builder, and then update the BUILDER_VERSION variable in builder-run to point to your new image. Our practice is to manually tag images builder images in the form Builder-v$SEMVER once we're happy with the state of the push.

Compile, Build, & Push Docker Image

(Almost no reason to ever do this manually, Jenkins handles this automation)

Build a docker image, tag it with the current git sha, and pushes it to the quay.io/coreos/tectonic-console repo.

Must set env vars DOCKER_USER and DOCKER_PASSWORD or have a valid .dockercfg file.

./build-docker-push

Jenkins automation

Master branch:

Runs a build, pushes an image to Quay tagged with the commit sha

Pull requests:

Runs a build when PRs are created or PR commits are pushed
Comment with Jenkins rebuild to manually trigger a re-build
Comment with Jenkins push to push an image to Quay, tagged with: pr_[pr #]_build_[jenkins build #]

If changes are ever required for the Jenkins job configuration, apply them to both the regular console job and PR image job.

Hacking

See CONTRIBUTING for workflow & convention details.

See STYLEGUIDE for file format and coding style guide.

Dev Dependencies

go, glide, glide-vc, nodejs/yarn, kubectl

Frontend Development

All frontend code lives in the frontend/ directory. The frontend uses node, yarn, and webpack to compile dependencies into self contained bundles which are loaded dynamically at run time in the browser. These bundles are not commited to git. Tasks are defined in package.json in the scripts section and are aliased to yarn run <cmd> (in the frontend directory).

Install Dependencies

To install the build tools and dependencies:

yarn install

You must run this command once, and every time the dependencies change. node_modules are not commited to git.

Interactive Development

The following build task will watch the source code for changes and compile automatically. You must reload the page in your browser!

yarn run dev

Tests

Run all unit tests:

./test

Run backend tests:

./test-backend

Run frontend tests:

./test-frontend

Integration Tests

Integration tests are run in a headless Chrome driven by protractor. Requirements include Chrome, a working cluster, kubectl, and bridge itself (see building above).

Setup (or any time you change node_modules - yarn add or yarn install)

cd frontend && yarn run webdriver-update

Run integration tests:

yarn run test-gui

Run integration tests on an OpenShift cluster:

yarn run test-gui-openshift

This will include the normal k8s CRUD tests and CRUD tests for OpenShift resources. It doesn't include ALM tests since it assumes ALM is not set up on an OpenShift cluster.

Hacking Integration Tests

Remove the --headless flag to Chrome (chromeOptions) in frontend/integration-tests/protractor.conf.ts to see what the tests are actually doing.

Local Dex

Checkout and build dex.

./bin/dex serve ../../openshift/console/contrib/dex-config-dev.yaml

Run bridge with the following options:

./bin/bridge \
  --user-auth=oidc \
  --user-auth-oidc-issuer-url='http://127.0.0.1:5556' \
  --user-auth-oidc-client-id='example-app' \
  --user-auth-oidc-client-secret='ZXhhbXBsZS1hcHAtc2VjcmV0' \
  --base-address='http://localhost:9000/' \
  --kubectl-client-id='example-app' \
  --kubectl-client-secret='ZXhhbXBsZS1hcHAtc2VjcmV0'

Dependency Management

Dependencies should be pinned to an exact semver, sha, or git tag (eg, no ^).

Backend

Whenever making vendor changes:

Finish updating dependencies & writing changes
Commit everything except vendor/ (eg, server: add x feature)
Make a second commit with only vendor/ (eg, vendor: revendor)

Add new backend dependencies:

Edit glide.yaml
./revendor

Update existing backend dependencies:

Edit the glide.yaml file to the desired verison (most likely a git hash)
Run ./revendor
Verify update was successful. glide.lock will have been updated to reflect the changes to glide.yaml and the package will have been updated in vendor.

Frontend

Add new frontend dependencies:

yarn add <package@version>

Update existing frontend dependencies:

yarn upgrade <package@version>

Supported Browsers

We support the latest versions of the following browsers:

Edge
Chrome
Safari
Firefox
TODO: IE 11. Needs polyfills. Also, IE 11 can't open more than 6 websockets.

kans/console