This repo contains a step-by-step tutorial on implementing a simple service broker that conforms to the Open Service Broker API specification and deploys applications and backing services to Cloud Foundry using the Spring Cloud App Broker.
See Git tags for step-by-step progress of configuring the broker using Spring Cloud App broker. Clone this repository and run the below command from the root of the project folder:
$ git tag -ln
v1 Build a simple Service Broker
v2 Configure app deployment properties through App Broker configuration
v3 Configure service instance/binding lifecycle using Workflows
v4 Configure Parameter Transformer for backing application deployment
v5 Persisting Service Instance/Binding State
v6 Configure custom Target locations for backing applications
Previously, in order to build a Spring Boot based service broker application, you would add the Spring Cloud Open Service Broker starter to the the project, specify the configuration and implement the required interfaces such as ServiceInstanceService and ServiceInstanceBindingService. Spring Cloud Open Service broker is less opinionated about server broker implementation and leaves many of the decisions to the developer, requiring the developer to implement all of the broker application logic themselves like managing service instances, managing state, backing app deployment etc. A second project Spring Cloud App Broker is available which is an abstraction on top of Spring Cloud Open Service broker and provides opinionated implementations of the corresponding interfaces and also deploys applications and backing services to a platform, such as Cloud Foundry or Kubernetes.
Spring Cloud App Broker builds on Spring Cloud Open Service broker. You must provide Spring Cloud Open Service Broker configuration in order to use Spring Cloud App Broker.
The following are some features of Spring Cloud App Broker in comparison to the Spring Cloud Open Service Broker:
With spring-cloud-app-broker, you can declare the details of services, including applications to deploy, application deployment details and their dependent backing services in the App Broker configuration (using properties under spring.cloud.appbroker.services). For instance, you can specify the number of service instances to be deployed, the memory and disk resource requirements , etc for specific service instance deployments. App broker will manage the deployment and provision of dependent apps and services and bind those services and apps where appropriate. Conversely when a request is received to delete a service instance, App Broker will unbind and delete dependent services and applications that were previously provisioned
spring-cloud-app-broker provides the AppDeploymentCreateServiceInstanceWorkflow which handles deploying the configured backing applications and services. Similarly there is AppDeploymentUpdateServiceInstanceWorkflow and AppDeploymentDeleteServiceInstanceWorkflow for updating and deleting the deployment of configured backing applications and services respectively.
You can specify default values for all application deployment( using properties under spring.cloud.appbroker.deployer.cloudfoundry.*) or override specific deployment ( using properties under spring.cloud.appbroker.services.*).
Currently, Spring Cloud App Broker supports only Cloud Foundry as a deployment platform.
App broker allows you to create multiple workflows for the various stages of creating, updating and deleting service instances. For instance, app broker authors can implement
CreateServiceInstanceWorkflow and configure it as a Spring bean within the application to hook additional functionality into the request to create a service instance. The WorkflowServiceInstanceService will then pick up this implementation and execute it as part of the service instance creation process. Multiple workflows can be annotated with @Order so as to process the workflows in a specific order.
Similarly workflows can be configured for creating and deleting service instance bindings.
spring-cloud-app-broker provides default implementations of most of the components needed to implement a service broker. In Spring Boot fashion, you can override the default behaviour by providing your own implementation of Spring beans, and spring-boot-app-broker will back away from its defaults. This reduces lot of boiler plate code and enables to quickly build your own service broker.
For instance, in service instance provisioning / de-provisioning, spring-cloud-open-service-broker required a Spring bean that implements the ServiceInstanceService interface. spring-cloud-app-broker handles this by auto configuring an implementation in AppDeploymentCreateServiceInstanceWorkflow.
Similarly if no implementation is provided for persisting service instance/service binding, then the app broker provides an InMemoryServiceInstanceStateRepository which provides an in memory Map to save state and offers an easy getting-started experience.
The
InMemoryServiceInstanceStateRepositoryis provided for demonstration and testing purpose only. It is not suitable for production grade applications!
There are additional auto configuration performed by App Broker. Please look at AppBrokerAutoConfiguration class for further details.
The following are the tags to specific commits in the repository that showcase the various features of the Spring Cloud App Broker.
Build a simple service broker application using Spring Initializr and include the spring-boot-starter and spring-cloud-starter-app-broker-cloudfoundry dependency
- Define service broker with Spring Boot externalised configuration supplied by
application.yml - Specify the Spring Cloud Open Service Broker configuration( using properties under
spring.cloud.openservicebroker) and Spring Cloud App Broker configuration(using properties underspring.cloud.appbroker) - Specify the details of the Cloud Platform deployment (using properties under
spring.cloud.appbroker.deployer)
Update the Spring Cloud App Broker configuration(using properties under spring.cloud.appbroker):
- Configure the memory requirements for the app instance using the property
spring.cloud.appbroker.services[0].apps.properties.memory - Configure the number of service instances to be deployed using the property
spring.cloud.appbroker.services[0].apps.properties.count
The above configuration of
countwill result in two instances of the application to be deployed into CloudFoundry
For a complete list of properties please see Properties Configuration
Add workflows to configure perform actions before or after create, update, delete, bind and unbind.
- Create
CustomCreateServiceInstanceBindingServiceWorkflowclass by implementingCreateServiceInstanceAppBindingWorkflowinterface to generate the connection URI of the deployed service instance. Binding the service to an app will add the URI of the deployed instance toVCAP_SERVICESenvironment variable of the app. The URI is generated using the app broker configuration propertiesspring.cloud.appbroker.services[0].apps[0].properties.hostandspring.cloud.appbroker.services[0].apps[0].properties.domain. These two configuration properties are also used to map routes for the deployed application. - Create
ServiceInstanceParametersValidatorWorkflowby implementingCreateServiceInstanceWorkflowinterface to validate the parameters before the service instance is created. The classServiceInstanceServiceOrderis used to specify the order of the workflows.
Update the Spring Cloud App Broker Parameters Transformers configuration(using properties under spring.cloud.appbroker.services.apps.parameters-transformers):
-
Configure
PropertyMappingparameter transformer for the service instance using the propertyspring.cloud.appbroker.services[0].apps.parameters-transformers.nameand include thecountandmemoryproperties to set deployment properties of the backing application from parameters provided when a service instance is created or updated. -
Configure
EnvironmentMappingparameter transformer for the app instance using the propertyspring.cloud.appbroker.services[0].apps.parameters-transformers.nameand include thecount,memoryandlangproperties to populate environment variables on the backing application from parameters provided when a service instance is created or updated -
Implement a custom
ParameterTransformerto include business logic to handle user input parameters and make it available to the service instance. Create aRequestTimeoutParameterTransformerwhere we are going to map a parameter fromrequest-timeout-msto an environment variablesample-app.httpclient.connect-timeout.
-
Persisting Service Instance State
- Create a model class
ServiceInstanceto store the service instance data. - Create a repository interface definition by using Spring Data's
CrudRepository. In this example we have used theReactiveCrudRepositoryimplementation by using Spring Data R2DBC project. - Finally register
BeanobjectDefaultServiceInstanceStateRepositorythat implements theServiceInstanceStateRepositoryto persists the service instance state using the above created repository.
- Create a model class
-
Persisting Service Instance Binding State
- Create a model class
ServiceInstanceBindingto store the service instance data. - Create a repository interface definition
ServiceInstanceBindingRepositoryby using Spring Data'sCrudRepository. In this example we have used theReactiveCrudRepositoryimplementation by using Spring Data R2DBC project. - Finally register
BeanobjectDefaultServiceInstanceBindingStateRepositorythat implements theServiceInstanceBindingStateRepositoryto persists the service instance binding using the above created repository.
- Create a model class
-
Create required configuration classes for MySQL connection using the r2dbc-mysql library. Parse the Cloud Foundry environment variable
VCAP_SERVICESto get the connection parameters. The folder 'src/main/resources' includesschema.sqlwhich specifies the table definition and is executed during the initialization process.
You will need to create a MYSQL database instance in Cloud Foundry prior to using
cf push. You will need to use the same MYSQL instance name provided in the manifest file when creating the instance usingcf create-service...
-
Specify a new Target for deploying the backing applications
- Create a
CustomSpaceTargetthat extends TargetFactory - Specify the custom target in the app broker configuration
- Create a
By default, for Cloud Foundry the backing application is deployed to the org named by spring.cloud.appbroker.deployer.cloudfoundry.default-org and the space named by spring.cloud.appbroker.deployer.cloudfoundry.default-space. However, in the above case the application is deployed to custom-space space in Cloud Foundry.
Follow the next section for step-by-step tutorial to deploy the service broker to Cloud Foundry
This section will show you how to deploy the broker from this repository to a space in Cloud Foundry and make it available for use via the marketplace.
In order to complete the tutorial, please ensure you have:
- Installed Cloud Foundry CLI in your local workstation to push apps and creating/binding service instances.
- A Cloud Foundry account and a space to deploy apps. This can be public hosted Cloud Foundry or a private Cloud Foundry.
Before you begin, please be sure you are logged into a Cloud Foundry instance and targeted to an org and space.
Update the application.yml in src/main/resources of sample-app-broker to specify the connection details for your cloudfoundry instance.
Also, need to update the domain property under appbroker.services[0].apps.properties to provide the domain name of your cloud foundry instance.
For the steps below, replace the correct domain of your cloud foundry instance when executing the command. The steps below use example.io as a domain.
This project requires Java 8 at a minimum.
This project is a multi-module Gradle project.
sample-app-broker module is a Spring Boot application and contains the code which implements the Spring Cloud App Broker.
sample-app-service module is a Spring Boot application that acts as the service instance which is deployed by the sample-app-broker. It has a single endpoint at '/' which returns back a sample string response.
-
Run the following command from the root of the project folder to compile the code and run tests
$ ./gradlew clean build
The gradle build for sample-app-broker configures the bootJar task to copies the JAR of sample-app-service into its classpath. The app broker configuration provided in sample-app-broker YAML file deploys the sample-app-service JAR as a service instance.
-
Prior to deploying the broker create a mysql instance to store the service instance and binding state:
$ cf create-service p.mysql db-small mysql Creating service instance mysql in org sample / space apps as admin... OK Create in progress. Use 'cf services' or 'cf service mysql' to check operation status. -
After the mysql instance is created, from the root of the repository use the supplied manifest to deploy the
sample-app-brokerapplication.$ cf push -f deploy/cloudfoundry/manifest.ymlA sample output is shown below where the app is deployed to Cloud Foundry and is started successfully.
name: sample-app-broker requested state: started isolation segment: iso-01 routes: sample-app-broker.apps.pcfone.io last uploaded: Thu 27 Aug 20:58:36 IST 2020 stack: cflinuxfs3 buildpacks: client-certificate-mapper=1.11.0_RELEASE container-security-provider=1.18.0_RELEASE java-buildpack=v4.31.1-offline-<https://github.com/cloudfoundry/java-buildpack.git#03e8eec> java-main java-opts java-security jvmkill-agent... (no decorators apply) type: web instances: 1/1 memory usage: 1024M start command: JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.16.0_RELEASE=printHeapHistogram=1 -Djava.io.tmpdir=$TMPDIR -XX:ActiveProcessorCount=$(nproc) -Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security $JAVA_OPTS" && CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.13.0_RELEASE -totMemory=$MEMORY_LIMIT -loadedClasses=17441 -poolType=metaspace -stackThreads=250 -vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY && JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && MALLOC_ARENA_MAX=2 SERVER_PORT=$PORT eval exec $PWD/.java-buildpack/open_jdk_jre/bin/java $JAVA_OPTS -cp $PWD/. org.springframework.boot.loader.JarLauncher state since cpu memory disk details #0 running 2020-08-27T15:28:55Z 0.0% 154.1M of 1G 162.9M of 1G -
Check the status of the application
$ cf appsYou should see output similar to:
Getting apps in org sample / space apps as admin... OK name requested state instances memory disk urls sample-app-broker started 1/1 1G 1G sample-app-broker.example.io -
Accessing the catalog
Use
curlor any other REST client to access thesample-app-brokercatalog via the/v2/catalogendpoint. This is the same endpoint used to populate the marketplace. Use the url of the broker application from previous step.$ curl http://sample-app-broker.example.io/v2/catalog { "services": [ { "id": "3101b971-1044-4816-a7ac-9ded2e028079", "name": "sample", "description": "A sample service", "bindable": true, "plans": [ { "id": "2451fa22-df16-4c10-ba6e-1f682d3dcdc9", "name": "standard", "description": "A sample plan", "free": true, "bindable": true } ], "tags": [ "sample" ], "metadata": { }, "requires": [ ] } ] }As you can see above the broker exposes a service called 'sample' which offers a single standard plan.
Now that the application has been deployed and verified, it can be registered to the Cloud Foundry services marketplace.
-
With administrator privileges
If you have administrator privileges on Cloud Foundry, you can make the service broker available in all organisations and spaces.
The Open Service Broker API endpoints in the service broker application are secured with a basic auth username and password. Register the service broker using the URL from above and the credentials:
cf create-service-broker sample-broker admin admin http://sample-app-broker.example.ioMake the service offerings from the service broker visible in the service marketplace
$ cf enable-service-access sample-broker -
Without administrator privileges
If you do not have administrator privileges on Cloud Foundry, you can make the service broker available in a single organization and space that you have privileges in:
$ cf create-service-broker sample-broker admin admin http://sample-app-broker.example.io --space-scopedYou should see output similar to:
Creating service broker sample-broker in org sample / space apps as admin... OK -
Check the registered Service broker
$ cf service-brokers Getting service brokers as admin... name url sample-broker http://sample-app-broker.example.io
The new service sample-service should be now visible in the marketplace along with the other services
If in the previous step you have registered the broker as space-scoped then the new service will only be show up in the marketplace in the space in which it is registered
-
Use the below command to view the marketplace
$ cf marketplace Getting services from marketplace in org sample / space apps as admin... OK service plans description broker sample standard A sample service sample-broker TIP: Use 'cf marketplace -s SERVICE' to view descriptions of individual plans of a given service.
-
Use
cf create-serviceto create a service instance and pass additional configuration parameters as configured above in V4 tag:$ cf create-service sample standard my-sample -c '{"count":1,"memory":"1G","lang":"en","request-timeout-ms":60}' Creating service instance my-sample in org sample / space apps as admin... OK Create in progress. Use 'cf services' or 'cf service my-sample' to check operation status.This will take some amount of time and in the background will initiate the deployment of the service instance (
sample-app-service). -
Use
cf servicesto verify if the instance is created:$ cf services Getting services in org sample / space apps as admin... name service plan bound apps last operation broker upgrade available my-sample sample standard create succeeded sample-broker -
Check if the service instance has been deployed:
$ cf target -s my-custom-space $ cf apps Getting apps in org sample / space my-custom-space as admin... OK name requested state instances memory disk urls sample-service-app1 started 1/1 1G 1G sample-service-app1.example.ioYou should see a new app
sample-service-app1deployed into the samemy-custom-spacespace once the service has been successfully created. Notice the number of instances is 1, and the memory of 1G has been allocated overriding the static configuration provided in the YAML file. -
Check the environment of the service instance
$ cf env sample-service-app1 Getting env variables for app sample-service-app1 in org sample / space apps as admin... OK ... User-Provided: SPRING_APPLICATION_JSON: {"count":"1","memory":"1G","lang":"en","sample-app.httpclient.connect-timeout":60,"spring.cloud.appbroker.service-instance-id":"ca203ddf-591d-44e4-9e02-56cba3847797"}The environment parameter transformer configuration is applied and the
count,memorylangproperties are available in the environment. Environment variablesample-app.httpclient.connect-timeoutis also set by the custom Parameter Transformer implementation. -
Verify if the service instance state is stored in the database. You can either use
cf mysqlto remotelysshinto the Cloud Foundry mysql instance or follow instructions here. Below we have usedcf mysql$ cf mysql mysql mysql> select * from service_instance; +----+--------------------------------------+-----------------------------------+-----------------+--------------+ | id | service_instance_id | description | operation_state | last_updated | +----+--------------------------------------+-----------------------------------+-----------------+--------------+ | 1 | f966780f-2027-435c-b099-2f464937bab0 | create service instance completed | SUCCEEDED | NULL | +----+--------------------------------------+-----------------------------------+-----------------+--------------+ 1 row in set (1.00 sec)
Let us know try to bind the service created above. We do not intend to use the service and so for the sake of simplicity let us bind the service to the the sample-app-broker app. Ideally, we would have a separate application that will bind to our new service
-
Use
cf bind-serviceto bind the instance. Change to the correct cloud foundry space before running the below command:$ cf target -s apps $ cf bind-service sample-app-broker my-sample -
You can then see the URI returned by inspecting the
VCAP_SERVICESenvironment variable$ cf env sample-app-broker Getting env variables for app sample-app-broker in org sample / space apps as admin... OK System-Provided: { "VCAP_SERVICES": { "sample": [ { "binding_name": null, "credentials": { "uri": "sample-service-demo.example.io" }, "instance_name": "my-sample", "label": "sample", "name": "my-sample", "plan": "standard", "provider": null, "syslog_drain_url": null, "tags": [ "sample" ], "volume_mounts": [] } ] } } -
Verify if the service instance binding state is stored in the database. You can either use
cf mysqlto remotelysshinto the Cloud Foundry mysql instance or follow instructions here. Below we have usedcf mysql$ cf mysql mysql mysql> select * from service_instance_binding; +----+--------------------------------------+--------------------------------------+-------------------------------------------+-----------------+--------------+ | id | service_instance_id | binding_id | description | operation_state | last_updated | +----+--------------------------------------+--------------------------------------+-------------------------------------------+-----------------+--------------+ | 1 | f966780f-2027-435c-b099-2f464937bab0 | ae217d8e-a97a-401f-87bd-ff9f1d70c0ea | create service instance binding completed | SUCCEEDED | NULL | +----+--------------------------------------+--------------------------------------+-------------------------------------------+-----------------+--------------+ 1 row in set (0.29 sec)
-
Using the URI from the
cf appsoutput you can access the service instance endpoint. Change to correct space in Cloud Foundry first# Check the endpoint $ cf target -s my-custom-space $ curl http://sample-service-app1.example.io/ Hello from brokered service...
- Unbind the service instance from the app
$ cf target -s apps $ cf unbind-service sample-app-broker my-sample Unbinding app sample-app-broker from service my-sample in org sample / space apps as admin... OK
We can now begin to clean up the environment after completing the previous step.
-
Delete the service:
$ cf delete-service my-sample Really delete the service my-sample?> y Deleting service my-sample in org sample / space apps as admin... OK Delete in progress. Use 'cf services' or 'cf service my-sample' to check operation status.This should take some minutes to complete. Deleting the service automatically deletes the service instance as well.
-
Check the apps
$ cf apps Getting apps in org sample / space apps as admin... OK name requested state instances memory disk urls sample-app-broker started 1/1 1G 1G sample-app-broker.example.io
After completing the above steps, you can clean up by deleting the service broker and the application
-
Delete the service broker:
$ cf delete-service-broker -f sample-brokerAfter confirming the deletion, you should see output similar to:
Deleting service broker sample-broker as admin... OKIf you re-run
cf service-brokersyou will not see thesample-brokerlisted -
Delete the app broker application:
$ cf delete -f sample-app-broker -
Delete the mysql instance
Use the below command to delete the service keys associated with the database:
$ cf delete-service-key mysql cf-mysqlFinally, delete the mysql instance using:
$ cf delete-service mysql