knative-extensions/eventing-kafka-broker

Add UpdateScale to KafkaSourceInterface

Zurvarian opened this issue · 2 comments

Problem
We have a need to handle the number of consumers of our KafkaSources programmatically, and checking the KafkaSourceInterface I can see how there is a GetScale method but there is no an UpdateScale counterpart.
This is the same pattern seen in other Scalable objects like Deployments.

I've tried to explore the option of scaling the ConsumerGroup but it does not seem to make any effect when trying it from command line and I could not find any API to work with them.

We are using v1.3.0 of the Client library, but checking the GoDoc of the latest version I cannot see the method in the API.

Persona:
Which persona is this feature for?
Event consumer (developer)

Exit Criteria
An UpdateScale method in the KafkaSourceInterface to handle the scaling of the KafkaSource object.

Something on the lines of:

// UpdateScale takes the Kafka Source name and the representation of a scale and updates it. Returns the server's representation of the scale, and an error, if there is any.
func (c *kafkaSources) UpdateScale(ctx context.Context, name string, scale *autoscalingv1.Scale, opts v1.UpdateOptions) (result *autoscalingv1.Scale, err error) {
	result = &autoscalingv1.Scale{}
	err = c.client.Put().
		Namespace(c.ns).
		Resource("kafkasources").
		Name(kafkaSourceName).
		SubResource("scale").
		VersionedParams(&opts, scheme.ParameterCodec).
		Body(scale).
		Do(ctx).
		Into(result)
	return
}

Time Estimate (optional):
How many developer-days do you think this may take to resolve?
Cannot say, implementation is easy, but testing I have no idea.

Additional context (optional)
I know Scaling can be handled too with Keda, but the problem we face cannot be handled with automated scaling, it must be manual to ensure that the KafkaSource remains with 0 consumers when required.

/triage-accepted
/assign

Awesome, thanks!