Back-filler deletes existing subscription

[kbadk]

When installing cloudwatch2humio, HumioCloudWatchLogsBackfiller deletes existing subscriptions:

cloudwatch2humio/src/logs_backfiller.py

Lines 76 to 88 in 93eb520

	# If our function is not subscribed, delete subscription and create ours.
	if all_subscription_filters["subscriptionFilters"][0]["destinationArn"] != humio_log_ingester_arn:
	helpers.delete_subscription(
	log_client,
	log_group["logGroupName"],
	all_subscription_filters["subscriptionFilters"][0]["filterName"]
	)
	helpers.create_subscription(
	log_client,
	log_group["logGroupName"],
	humio_log_ingester_arn,
	context
	)

This is not mentioned in the documentation. According to the documentation, HumioCloudWatchLogsBackfiller innocently subscribes to existing log groups – nothing more:

This function will paginate through your existing CloudWatch log groups and subscribe the HumioCloudWatchLogsIngester to every single one.

If this really is desired behavior – which I don't think it is – it should be made very clear in the documentation that installing cloudwatch2humio is desctructive to existing infrastructure!

[squatched]

This is dangerous and a real problem.

I independently observed this problem in my own analysis of the code for usage and determined that this keeps the backfiller from being usable and I recommend nobody use the backfiller until this is fixed. I understand the intent but the current execution is dangerous.

[Suzanna-Volkov]

Thanks for reporting this! I will update the docs for a starter, and then I will take a look into how this can be improved. CloudWatch should support two subscriptions per log group, and that hasn't been taken advantage of. I am thinking that the behaviour should be that the backfiller subscribes and only removes one other subscription if that is necessary, but this will be explicitly stated in the docs. However, I will start with updating the docs to reflect the current behaviour and its dangers.

[kbadk]

I don't think you should remove any subscriptions, ever. If the backfiller can't subscribe, it should fail. Killing other logging for any reason without explicit permission from the user is not acceptable behaviour, in my opinion.

[Suzanna-Volkov]

@kbadk yes, thinking about it, I think you are completely right. I will just have to make it clear why the backfiller fails in this case.