open-telemetry/opentelemetry-collector-contrib

Omit multiline pattern from log

Closed this issue · 6 comments

haimrubinstein commented

Component(s)

pkg/stanza, receiver/udplog

Is your feature request related to a problem? Please describe.

We have a scenario where we use OTEL in a very high-scale environment. We have solid concerns that we will hit our network limits. TO reduce network traffic we tried sending multiple logs over 1 packet using UDP receiver and multiline configuration. During the tests, we've noticed that the multiline delimiter is included in the log record.
For example, if we send set line_end_pattern to be 'PATTERN' and we send a UDP packet: log1 PATTERN log2, the final result is an array of 2 records [log1 PATTERN , log2], when what we actually need is [log1, log2].

Describe the solution you'd like

I would like to add a flag to the multiline config that if set to true will omit the pattern from the logs.
If the solution is OK with you I can open a pull request.

Describe alternatives you've considered

No response

Additional context

No response

haimrubinstein commented

@djaglowski I'd be glad to hear your opinion.

github-actions commented

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

djaglowski commented

@haim6678, makes sense to me. I'd be happy to review a PR.

haimrubinstein commented

@haim6678, makes sense to me. I'd be happy to review a PR.

Published my PR :)
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/26460/files

github-actions commented

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

djaglowski commented

resolved by #26460