kpush - push signed messages from PubSub to Knative service

The use of GCP Cloud PubSub is common component in data flow pipelines. It creates global and elastic separation between the data provider and data consumers. One of its features is the ability to push subscription which sends messages to the configured webhook. Consider the following solution:

In this case, the kpush client publishes messages to PubSub topic which the subscription subsequently pushes to target processing service, an app hosted on an instance of Knative. PubSub provides instructions on how to include token in POST to ensure that only submissions from valid senders are processed by target service.

Since this token can leak (e.g. logs), kpush demonstrates how to add an additional level of validation by signing each message on the client and validating that signature on the service side. The below PubSub message sample illustrates the sig attribute holding SHA-1 of the message.data payload which is appended to each message.

{
    "message": {
        "attributes": {
            "sig": "sha1=22c477fd1269c9d3bab8591b371a66976f10006e"
        },
        "data": "eyJpZC...",
        "messageId": "333651121184341",
        "publishTime": "2018-12-22T19:05:01.067Z",
    }
}

Setup

The kpush flow includes two components: client, generating, signing, and publishing mocked up messages, and server which receives PubSub pushed messages and validates their signature.

Assuming gcloud SDK already configured. See gcloud docs for instructions if you need assistance

PubSub topic

Let's start by creating the PubSub topic named kpush to which our client will be publishing messages.

gcloud pubsub topics create kpush

The response should be

Created topic [projects/YOUR_PROJECT_ID/topics/kpush].

Knative Service

The installation and configuration of Knative are beyond the scope of this readme, but, you can find detailed instructions on how to configure it on Kubernetes service offered by most Cloud Service Providers here. In this example I will be using Google's Kubernetes Engine (GKE) which now can be easily configured with the validated version of Knative with a single checkbox.

Quickest way to build your service image is through GCP Build. Just submit the build request from within the kpush directory:

gcloud builds submit \
    --project ${GCP_PROJECT_ID} \
	--tag gcr.io/${GCP_PROJECT_ID}/kpush-server:latest

The build service is pretty verbose in output but eventually you should see something like this

ID           CREATE_TIME          DURATION  SOURCE                                   IMAGES                      STATUS
6905dd3a...  2018-12-23T03:48...  1M43S     gs://PROJECT_cloudbuild/source/15...tgz  gcr.io/PROJECT/kpush-server SUCCESS

The IMAGE column will have the repository URI for your new image (e.g. gcr.io/PROJECT/kpush-server)

Before we can deploy that service to Knative, we just need to update the deploy/server.yaml file

First, update the container image to the URI from the build IMAGE column value. Then update the two environment variables values:

KNOWN_PUBLISHER_TOKENS which holds the token we will use in PubSub URL
MSG_SIG_KEY which should be the key used by your clients to sign published messages

Note, KNOWN_PUBLISHER_TOKENS may include many tokens separated by comma

Finally, to deploy kpush service to Knative apply the updated deployment using following command:

kubectl apply -f deploy/server.yaml

The response should be

service.serving.knative.dev "pushme" configured

To check if the service was deployed successfully you can check the status using kubectl get pods command. The response should look something like this (e.g. Ready 3/3 and Status Running).

NAME                                          READY     STATUS    RESTARTS   AGE
pushme-00002-deployment-5645f48b4d-mb24j      3/3       Running   0          4h

Knative uses convention to build serving URL by combining the deployment name (e.g. pushme), namespace name (e.g. demo), and the pre-configured domain name (e.g. knative.tech). The resulting URL should look something like this

https://pushme.default.knative.tech

Go ahead, test it in browser, you should following JSON response:

{ "handlers": [ "POST: /post" ] }

Target of PubSub push must also be an HTTPS server with non-self-signed certificate. The instructions on how to configure Knative with SSL certificate are located here.

Note, PubSub will publish only to registering endpoints which will require you to A) Verify you have access to the domain, and B) Register your domain in APIs & services. Instructions for both can be set up here

PubSub Subscription

The final component we need to set up is the PubSub subscription which will push messages to our Knative hosted service. the MYTOKEN should be set to one of the tokens you defined in the Knative service manifest environment variables (KNOWN_PUBLISHER_TOKENS).

gcloud pubsub subscriptions create kpush-sub \
    --topic kpush \
    --push-endpoint https://pushme.demo.knative.tech/push?publisherToken=${MYTOKEN} \
    --ack-deadline 30

Note, cold-starts on Knative can be sometimes slow so we added the ack-deadline=30 parameter. For additional info on PubSub subscription options see this [doc](* https://cloud.google.com/pubsub/docs/subscriber)

Run

To demo the entire pipeline, kpush includes a client which will mock up a few messages, sign them, and submit them to the configured topic.

Navigate to the client directory...

cd cmd/client

...and run the following command:

go run main.go --project ${PROJECT_ID} \
               --key ${MSG_SIG_KEY} \
               --topic kpush

The response should look something like this

2018/12/23 13:58:44 Using topic publisher: YOUR-GCP-PROJECT-NAME:kpush
   status: published msg[0] eacc9be0-06fd-11e9-868f-acde48001122
   status: published msg[1] eb2e0cae-06fd-11e9-868f-acde48001122
   status: published msg[2] eb39aaa0-06fd-11e9-868f-acde48001122
sent 3 messages

Monitoring

A successful end-to-end run of the pipeline will result in 202 status code from the Knative hosted service. To validate you can either query the Knative logs or navigate to the GCP Stackdriver console to review the PubSub subscription panel (Resources > PubSub > Subscriptions)

Direct Push

kpush also includes an HTTPPublisher which can skip the PubSub entirely and submit signed messages directly to the Knative service. To do that include url parameter in the kpush client command like this

go run main.go --project ${PROJECT_ID} \
               --key ${MSG_SIG_KEY} \
               --url https://pushme.demo.knative.tech/push?publisherToken=${MYTOKEN} \
               --messages 10

How to use kpush in your project

You can import kpush into your project as a package

go get -u github.com/mchmarny/kpush

For demonstration how you can use kpush in your project see provided example.

Disclaimer

This is my personal project and it does not represent my employer. I take no responsibility for issues caused by this code. I do my best to ensure that everything works, but if something goes wrong, my apologies is all you will get.