PHP Stacktrace omits newlines
Closed this issue · 5 comments
Hello. I have PHP stacktrace combined, but the plugin breaks newlines, so the stack output is on one line always. How to fix?
<source>
@type forward
@label @RAW
</source>
<label @RAW>
<match **>
@type detect_exceptions
@label @STACKED
message log
multiline_flush_interval 1
</match>
</label>
<label @STACKED>
<match **>
@type stdout
</match>
</label>
This plugin was designed to work with in_tail, which retains newlines in each record. What's on the other side of your in_forward? You should make sure that the forwarder also retains newlines.
What's on the other side of your in_forward
It's docker fluentd driver stdout of container running PHP. The latter sends newlines, but the driver send an event for each line without break.
Docker version 19.03.5, build 633a0ea838
I am facing the same issue with kubernetes containerd runtime. Container stacktrace logs' new lines are omitted.
I'm using in_tail to monitor kubernetes container logs and I'm having the same issue. New lines are not preserved in the stacktrace.
The following python output with multiple lines
Traceback (most recent call last):
File "/force-exception.py", line 14, in <module>
func1()
File "/force-exception.py", line 4, in func1
func2()
File "/force-exception.py", line 8, in func2
func3()
File "/force-exception.py", line 12, in func3
raise Exception("meu erro")
Exception: meu erro
... results in the following log message collected by fluentd:
Traceback (most recent call last): File "/force-exception.py", line 14, in <module> func1() File "/force-exception.py", line 4, in func1 func2() File "/force-exception.py", line 8, in func2 func3() File "/force-exception.py", line 12, in func3 raise Exception("meu erro")Exception: meu erro
This is my fluentd.conf.
fluentd.conf: |
<system>
# suppress_repeated_stacktrace true
# ignore_repeated_log_interval 5m
</system>
<match fluentd.**>
@type null
</match>
<source>
@type tail
@id in_tail_container_logs
path /var/log/containers/*.log
pos_file /var/log/fluentd-containers.log.pos
tag kubernetes.*
read_from_head true
follow_inodes true
refresh_interval 5
<parse>
@type cri
</parse>
</source>
# The gelf plugin assumes input in utf-8
<filter **>
@type record_modifier
char_encoding utf-8
</filter>
# Ignora linhas em branco
<filter **>
@type grep
<exclude>
key message
pattern ^\n$
</exclude>
</filter>
## Adiciona metadados do Kubernetes nos logs
<filter kubernetes.var.log.containers.**>
@type kubernetes_metadata
annotation_match '[ "fluentd.+" ]'
</filter>
## Nginx Ingress Controller
<match kubernetes.var.log.containers.frontend-ingress-nginx-controller-**>
@type gelf
host "graylog-deploy-udp.logging"
port "12401"
protocol "udp"
include_tag_key true
<buffer>
flush_thread_count 10
flush_interval 5s
total_limit_size 1024MB
retry_max_interval 30
retry_forever true
</buffer>
</match>
## Plugin: concat
## Funciona bem para Glassfish
<filter kubernetes.var.log.containers.**>
@type concat
key message
multiline_start_regexp /^\[/
multiline_end_regexp /\]$/
</filter>
## Plugin: detect_exceptions
## Funciona bem para Wildfly e várias aplicações Java
## Não funciona bem para o Glassfish.
<match kubernetes.var.log.containers.**>
@type detect_exceptions
message message
multiline_flush_interval 5s
remove_tag_prefix kubernetes
</match>
<match **>
@type gelf
@id out_graylog
@log_level info
include_tag_key true
host "graylog-deploy-udp.logging"
port "12400"
protocol "udp"
<buffer>
flush_thread_count 10
flush_interval 5s
total_limit_size 1024MB
retry_max_interval 30
retry_forever true
</buffer>
</match>
Could this be related to the order in which the filters are placed?
Nevermind!
Just found the new option: force_line_breaks. This fixed the issue for me :D
<match kubernetes.var.log.containers.**>
@type detect_exceptions
message message
force_line_breaks true
multiline_flush_interval 5s
remove_tag_prefix kubernetes
</match>