apache/camel-k-runtime

Optional parameters in Kamelet route templates

nicolaferraro opened this issue · 10 comments

nicolaferraro commented

Opening it here to discuss it better, but maybe now it's also a core Camel concern.

I need to define a Kamelet with an optional parameter, so;

# ...
spec:
  definition:
    properties:
      clientId: # this is optional, no default value
        type: string
  flow:
    from:
      uri: paho:the-topic
      parameters:
        clientId: "{{clientId}}"
        brokerUrl: "the-url"

This is how I'd define such Kamelet.

If I don't set any value for clientId, the operator does not pass any value in application properties for camel.kamelet.mqtt-source.clientId.

But, even if the clientId is not marked as required, the route template engine fails with:

2021-02-16 09:02:59,624 ERROR [io.qua.run.Application] (main) Failed to start application (with profile prod): java.lang.IllegalArgumentException: Route template mqtt-source the following mandatory parameters must be provided: clientId
        at org.apache.camel.quarkus.core.BaseModel.addRouteFromTemplate(BaseModel.java:225)
        at org.apache.camel.quarkus.core.FastCamelContext.addRouteFromTemplate(FastCamelContext.java:199)
        at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:370)
        at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:339)
        at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2646)
        at org.apache.camel.quarkus.core.FastCamelContext.doInit(FastCamelContext.java:505)
        at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
        at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2414)
        at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
        at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2431)
        at org.apache.camel.quarkus.main.CamelMain.doStart(CamelMain.java:75)
        at org.apache.camel.support.service.BaseService.start(BaseService.java:115)
        at org.apache.camel.quarkus.main.CamelMain.startEngine(CamelMain.java:120)
        at org.apache.camel.quarkus.main.CamelMainRuntime.start(CamelMainRuntime.java:49)
        at org.apache.camel.quarkus.core.CamelBootstrapRecorder.start(CamelBootstrapRecorder.java:45)
        at io.quarkus.deployment.steps.CamelBootstrapProcessor$boot-173480958.deploy_0(CamelBootstrapProcessor$boot-173480958.zig:101)
        at io.quarkus.deployment.steps.CamelBootstrapProcessor$boot-173480958.deploy(CamelBootstrapProcessor$boot-173480958.zig:40)
        at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:534)
        at io.quarkus.runtime.Application.start(Application.java:90)
        at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:97)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:62)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:38)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:104)
        at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)

A related problem is to ensure that once the issue is fixed, the clientId in the paho endpoint is not passed at all (i.e. not passed as "" empty string).

cc: @lburgazzoli

lburgazzoli commented

This is not a route template issue in reality but a generic camel one, as in fact the from is translated to:

from: "paho:the-topic?clientId={{clientId}}&brokerUrl=the-url"

So when the endpoint get resolved, then the property must be there. An option is to use default values, as example:

from:
  uri: paho:the-topic
  parameters:
    clientId: "{{clientId:defaultId}}"
    brokerUrl: "the-url"

but maybe we also need to introduce:

  • a way to explicit mark some properties as optional, like with a property function {{optional:nameOfTheProperty}}
  • a way to use simple language to set defaults, like {{nameOfTheProperty:${uuid}}}

/cc @davsclaus @valdar

davsclaus commented

a)
Ah yeah we can likely in camel-core skip adding the uri parameter if it was marked as optional and it has null value.

b)
The idea of having {{xxx:default}} is also something we can look at adding to camel core in general.

  • 👍1
davsclaus commented

Ad b)
Okay so we already have default values, you can do that

davsclaus commented

@nicolaferraro the route templates in camel-core has parameters that always must be provided a value for:
(a) by providing the value
(b) by setting a default value where the parameter is declared

There is no notion of optional parameter. You cannot use Lucas {{clientId:myDefaultValue}} in the flow as the route template validates that all parameters have values.

For optional parameters then we need to add support for this, so we know that for endpoint uri's we know that if the parameter is used in the uri, on either the key or value side (key=value), then that combo should not be included when the route is created. This is maybe a little bit tricker but lets create a JIRA ticket in camel-core

davsclaus commented 
  clientId: # this is optional, no default value
    type: string

So what you attempt to do, and say above is "not supported and a bit wrong according to current impl". Its actually a mandatory parameter as there is no default value assigned

davsclaus commented

Okay you can now use {{?myKey}} to mark a parameter as optional in Camel 3.9 onwards.

  • 🎉2
davsclaus commented

I wonder if we need something like this with route template DSL

                routeTemplate("myTemplate").templateParameter("foo").templateOptionalParameter("maxKeep")
                        .from("direct:{{foo}}")
                        .to("mock:result?retainFirst={{maxKeep}}");

As then you make it standout in the parameter definitions in the top which are required, and which are optional.

Then in the route DSL you just use {{key}} where its in use. Then you dont need to remember to use ? for optional. However this requires camel-core to transform the DSL by automatic replacing {{maxKeep}} to {{?maxKeep}} as it was declared as optional. And for this I need to add a new hook into the properties component

nicolaferraro commented

I wonder if we need something like this with route template DSL

                routeTemplate("myTemplate").templateParameter("foo").templateOptionalParameter("maxKeep")
                        .from("direct:{{foo}}")
                        .to("mock:result?retainFirst={{maxKeep}}");

As then you make it standout in the parameter definitions in the top which are required, and which are optional.

Then in the route DSL you just use {{key}} where its in use. Then you dont need to remember to use ? for optional. However this requires camel-core to transform the DSL by automatic replacing {{maxKeep}} to {{?maxKeep}} as it was declared as optional. And for this I need to add a new hook into the properties component

I think there's in place a mechanism by which all required parameters are checked in the Kamelet runtime, so having parameters marked as optional could help implementing such check correctly. Otherwise the runtime will still reject an instance if we don't provide a value for a param.

lburgazzoli commented

Implemented in camel 3.10, @nicolaferraro closing this