timmyyuan/function-mesh-worker-service

Mesh Worker Service

This is a proxy that is used to forward requests to the k8s.

Run with pulsar broker

Prerequisite

• Pulsar 2.8.0 or later

Update configuration file

Add the following configuration to the functions_worker.yml configuration file:

functionsWorkerServiceNarPackage: /YOUR-NAR-PATH/mesh-worker-service-1.0-SNAPSHOT.nar

Replace the YOUR-NAR-PATH variable with your real path.

Configuring the development environment

Start service

Importing this project into the idea development environment.

Configuration environment variable KUBECONFIG for idea development environment.

Test Interface

getFunctionStatus

./bin/pulsar-admin --admin-url http://localhost:6750 functions status --tenant public --namespace default --name functionmesh-sample-ex1

or

curl http://localhost:6750/admin/v3/functions/test/default/functionmesh-sample-ex1/status

registerFunction

./bin/pulsar-admin --admin-url http://localhost:6750 functions create \
  --jar target/my-jar-with-dependencies.jar \
  --classname org.example.functions.WordCountFunction \
  --tenant public \
  --namespace default \
  --name word-count \
  --inputs persistent://public/default/sentences \
  --output persistent://public/default/count \
  --cpu 0.1 \
  --ram 1 \
  --custom-runtime-options \
  "{"clusterName": "test-pulsar", "inputTypeClassName": "java.lang.String", "outputTypeClassName": "java.lang.String"}"

sink connector

./bin/pulsar-admin --admin-url http://localhost:6750 sinks create \
  --sink-type data-generator \
  --classname org.apache.pulsar.io.datagenerator.DataGeneratorPrintSink \
  --tenant public \
  --namespace default \
  --name data-generator-sink \
  --inputs persistent://public/default/random-data-topic \
  --auto-ack true \
  --custom-runtime-options \
  "{"clusterName": "test-pulsar", "inputTypeClassName": "org.apache.pulsar.io.datagenerator.Person"}"

source connector

./bin/pulsar-admin --admin-url http://localhost:6750 sources create \
  --source-type data-generator \
  --tenant public \
  --namespace default \
  --name data-generator-source \
  --destination-topic-name persistent://public/default/random-data-topic \
  --source-config "{"sleepBetweenMessages": "5000"}"
  --custom-runtime-options \
  "{"clusterName": "test-pulsar", "outputTypeClassName": "org.apache.pulsar.io.datagenerator.Person"}"

updateFunction

./bin/pulsar-admin --admin-url http://localhost:6750 functions update \
  --jar target/my-jar-with-dependencies.jar \
  --classname org.example.functions.WordCountFunction \
  --tenant public \
  --namespace default \
  --name word-count \
  --inputs persistent://public/default/sentences \
  --output persistent://public/default/count \
  --input-specs "{"source": {"serdeClassName": "java.lang.String"}}" \
  --output-serde-classname java.lang.String \
  --cpu 0.2 \
  --ram 1 \
  --user-config "{"clusterName": "test-pulsar", "typeClassName": "java.lang.String"}"

getFunctionInfo

./bin/pulsar-admin --admin-url http://localhost:6750 functions get \
  --tenant public \
  --namespace default \
  --name word-count

deregisterFunction

./bin/pulsar-admin --admin-url http://localhost:6750 functions delete \
  --tenant public \
  --namespace default \
  --name word-count

More tools

Automatic generation java crd model

crd yaml file

Note: add the field preserveUnknownFields: false to spec for avoid this issue

CRD_FILE=compute.functionmesh.io_sources.yaml # Target CRD file

GEN_DIR=/tmp/functions-mesh/crd
mkdir -p $GEN_DIR
cp ../config/crd/bases/* $GEN_DIR
cd $GEN_DIR

LOCAL_MANIFEST_FILE=$GEN_DIR/$CRD_FILE

# yq site: https://mikefarah.gitbook.io/yq/
yq e ".spec.preserveUnknownFields = false" -i $CRD_FILE 

docker rm -f kind-control-plane
docker run \
  --rm \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$(pwd)":"$(pwd)" \
  -ti \
  --network host \
  docker.pkg.github.com/kubernetes-client/java/crd-model-gen:v1.0.3 \
  /generate.sh \
  -u $LOCAL_MANIFEST_FILE \
  -n io.functionmesh.compute \
  -p io.functionmesh.compute \
  -o "$(pwd)"

open $GEN_DIR

A auto tool for generated crd

./tool/generate-crd.sh

Then add license for crd model file

mvn license:format