User Defined Policies (UDP) in API Connect 2018.
This tutorial target UDP for API Gateway Native or new implementation.
The purpose of this asset is to help deploying an UDP on APIC 2018. To illustrate this a mq-invoke policy is used as UDP.
The mq-invoke policy sends a message to MQ.
UDP Principle
Introduction
UDP is a user defined policy that is made available as a new policy in the manger UI and made available on the gateway.
The new API Gateway is made of native datapower objects. The configuration and the creation of these objects are made by the DataPower configuration sequence service using configuration file (by default it uses cfg extension) files available on the DataPower.
From KC:
A configuration sequence is a script-based way to create, modify, and delete configurations on the DataPower® Gateway.
Configuration file consists of DataPower commands. Those configuration files can be found in config, local and temporary/config.
The API Connect Gateway Service object in DataPower is retrieving from API Connect configuration files.
All commands can be found at the following link: KC commands
API Gateway references API Objects that has the following hierarchical structure:
|- API Object
|- API Assembly
|- API Rule
|- API Assembly Action
|- API Assembly function call
|- API Assembly function
|- API Assembly
A UDP is actually an API Assembly function that are created using gateway extension feature. The UDP is created using configuration files that contain all the commands to create the assembly, actions, ... and any DataPower objects that would be required to be used from your API Assembly action.
When an UDP is deployed, it will be added as user-defined policy on API Connect Gateway Service. The API Gateway Service will tell API Connect that there are new policies available and it will be made available in the API Manager UI.
UDP configurations are stored as part of gateway extensions at temporary:///config/filestores/extensions/gateway-extension More information is provided in the packaging session.
The following provides an example of configuration commands used to create a UDP: KC UDP example The hierarchical structure of the UDP is as follow:
|- API Assembly function
|- API Assembly
|- API Rule
|- API Assembly Action
The assembly actions are "Assembly xxx Actions" such set-variable, Redact, gatewayscript, ... Each action has a set of commands that can be executed to configure the Action. For example the Assembly GatewayScript Action can be created using:
assembly-gatewayscript "mq-invoke_1.0.0_gws_0"
reset
title "MQ Invoke"
gatewayscript-location "temporary:///filestores/extensions/gateway-extension/mq-invoke/mq-invoke-code.js"
exit
- reset is a command to set the value to default
- title command creates the title
- gatewayscript-location command defines where the gws file is located
For example the gatewayscript action is available at the following link: KC gws action command Viewing running configuration can be found by: Troubleshooting -> Advanced -> View running config
Capabilities
Any DataPower capabilities can be used from a UDP:
- XSLT
- GatewayScript
- Processing rules called from an XSLT or Gatewayscript
It is good to know that all of these capabilities can be used directly in an assembly as well, the UDP just made a wrapper arround the actions that you have defined.
DataPower objects can be created along with your assembly UDP when the extension is created. For example in this tutorial, we have created a mq-qm object to connect to a remote queue manager. This queue manager object is then used in the gatewayscript using an url-open.
Processing rules
Those can be called from a gatewayscript using the following approach: Multi step documentation available on KC: KC multistep
var ms = require ('multistep');
var inputMessage = context.createMessage('inputCtx');
inputMessage.body.write('{"myJSONPayload":"MyPayload"}');
// Input of multistep is the name of a message object. And the messageObj.body is the payload
// outputCtx will be a message object containing the response
try {
ms.callRule ('myRuleName', 'inputCtx', 'outputCtx',
function(error) {
var result = null;
if (error) {
//check if output context variable is defined
if (outputCtx != '') {
console.info("output context %s", JSON.stringify(outputCtx));
}
console.error(error);
session.output.write(error);
} else if (outputCtx != '') {
console.info("writing result %s", result);
session.output.write(context.get(outputCtx.body));
}
}
);
} catch (error) {
console.error(error);
session.output.write(error);
}
var ms = require('multistep');
var fooObj = context.createMessage('foo');
var barObj = context.createMessage('bar');
fooObj.body.write('HelloWorld');
fooObj.header.set('Header1', 'myHeader');
fooObj.setVariable('Var1', 'myVar');
ms.callRule('rule1', 'foo', 'bar', function(error) {
barObj.body.readAsBuffer(function(error, buf) {
var OutputPayload = buf.toString();
var OutputHeaders = barObj.header.get();
var OutputVar1 = barObj.getVariable('Var2');
});
});
Building UDP
The following part explains the contents of the UDP.
The asset contains
- configuration file to build the API assembly -
mq-invoke-policy.cfg
. - configuration file to build the queue manager object -
mq-invoke-dpobj.cfg
- the gatewayscript used by the gatewayscript action to call mq using url-open and made use of the dp queue manager object.
The hierarchy call structure of a UDP assembly is:
|- API Assembly function : assembly-function "mq-invoke_1.0.0"
|- API Assembly: assembly "mq-invoke_1.0.0"
|- API Rule: api-rule "mq-invoke_1.0.0_main"
|- API Assembly Action: assembly-gatewayscript "mq-invoke_1.0.0_gws_0"
The configuration file used to create the UDP assembly contains all the commands to created the API objects. The different API objects are described here after.
Assembly action
The example has only one Assembly Action: a gatewayscript file.
assembly-gatewayscript "mq-invoke_1.0.0_gws_0"
reset
title "MQ Invoke"
correlation-path "$.x-ibm-configuration.assembly.execute[0]"
gatewayscript-location "temporary:///filestores/extensions/gateway-extension/mq-invoke/mq-invoke-code.js"
exit
The correlation-path
command is only used for debugging purpose. And the x-ibm-configuration
object has a specific structure that has different elements:
enforced, phase, testable, cors, activity-log, assembly.execute, gateway …
The UDP pakaged zip file (see packaging UDP below), that contains the gatewayscript file, is unzipped on DataPower under the location temporary:///filestores/extensions/gateway-extension
. The folder mq-invoke
is a standard folder in the zip containing the gatewayscript file.
It is possible to have multiple actions and data can be passed along using standard variable using context object: context.set('myvar')
or context.get('myVar')
.
Assembly rule
There is only one rule. The rule defines the Assembly actions.
api-rule "mq-invoke_1.0.0_main"
reset
action mq-invoke_1.0.0_gws_0
exit
You may have multiple actions defined as in this example:
api-rule udp-basic_1.0.0_main
reset
action udp-basic_1.0.0_set-variable_0
action udp-basic_1.0.0_gatewayscript_1
exit
Assembly
The assembly defines the rule to be called:
assembly udp-basic_1.0.0
reset
rule udp-basic_1.0.0_main
exit
Assembly-function
The assembly function is the "API Manager UI interface" and it is used to
- define the name of the policy that will be displayed in the manager UI assembly
- the parameters that will be displayed as user parameter of the policy assembly
- to define what assembly has to be called
assembly-function "mq-invoke_1.0.0"
reset
title "MQ put"
parameter
name "qmgrObj"
label "QMgr Object"
description "gateway qmgr object"
value-type string
value "mainmqm"
exit
parameter
name "variableName"
label "variableName"
description "name of the variable that contains the payload"
value-type string
value "mqmsgdata"
exit
parameter
name "varOutputName"
label "varOutputName"
description "name of the variable for the response"
value-type string
value "message.body"
exit
parameter
name "queue"
label "Queue"
description "The Queue to be used"
value-type string
exit
parameter
name "format"
label "Format"
description "Format"
value-type string
value "MQSTR"
exit
assembly mq-invoke_1.0.0
exit
Parameters defined by this assembly function call can be accessed in the different API assembly Action using the local object: local.parameter.MyParam
.
For instance the gatewayscript can access the queue parameter value using: context.get('local.parameter.queue')
.
OpenAPI file example
The yaml file mq-policy_1.0.0.yaml can be used as example on how to use the UDP policy in a API definition. You can import the OpenAPI in APIC once the
Packaging the UDP
structure
The UDP is deployed on the gateway as gateway extension. Gateway extensions are deployed to a gateway as one package that contains all the extensions. To add a new extension you need to
- Get the old extension package (from a source repo or by getting it from the manager)
- Add the new extension to the existing package (it's a zip file)
- Delete the old extension from the gateway (using the cli - see further): apic gateway-extensions:delete
- Create again the gateway extension using the cli and the new package: apic gateway-extensions:create
It is therefore a good idea to place the UDP files into a folder and add this folder to the package. The folder will be created on the DataPower under temporary:///filestores/extensions/gateway-extension/
.
In this example, the files have been placed under the folder "mq-invoke".
The folder mq-invoke contains the .cfg files (API assembly configuration and queue manager object configuration) and the gatewayscript.
Command to publish the UDP assembly
Here are the steps that can be performed to package and publish the UDP:
- Create the zip:
zip mq-invoke-ext.zip mq-invoke/*.cfg mq-invoke/*.js
- Login to the manager using the scope admin:
apic login --server mgmtHost --realm admin/myIdp -u myAdminUser
myIdp is the identity provider where the user myAdminUser is defined.
To get the list of identity provider in the admin scope, you can use the following cli:
apic identity-providers:list --scope admin -s mgmtHost
- Deploy the package with the command
apic gateway-extensions:create myPackage.zip --scope org --org admin --gateway-service myGatewayService --availability-zone availabilityZoneOfGatewaySrv -s mgmtHost
The organisation has to be admin. To get the list of availability zone, the following cli can be used:
apic availability-zones:list --org admin -s $mgmt
And the list of gateway service in a particular availability zone:
apic gateway-services:list --org admin --availability-zones availabilityZoneOfGatewaySrv -s $mgmt
To redeploy your UDP (if you make a change) follow the steps:
- delete the UDP:
apic gateway-extensions:delete --scope org --org admin --gateway-service myGatewayService --availability-zone availabilityZoneOfGatewaySrv -s mgmtHost
- create the UDP
The deployment takes a while to get the extension downloaded from the gateway. The API GatewayService on the DataPower has a configuration to tells the interval time to get information.
Once the extension files has been downloaded by the Gateway on the DataPower, the objects are created by the configuration sequence service.
The extension is not add automatically to the Gateway Gateway Service. This is because it requires a restart of the service to take the new configuration into account which implies a service disruption. For the Manual step:
- Login in the DataPower UI
- Search for the API Gateway Service
- Add the user-defined policy
- Disable the gw service
- Enable the gw service
In a kubernetes deployment, the configruation of the user defined policy can be automated using a config map.
After this configuration, the gateway will notify the manager that a user-defined policy is available and it will then appears in the assembly. This can take one to a couple of minutes.
Resources
Building User-Defined Policies on the API Gateway - API Connect
User Defined Policies / Custom Policies in API Connect 2018 (MQ invoke Example) GitHub - ozairs/apiconnect-policies