Ignore HTTP Response Codes
Closed this issue · 5 comments
Hi. I'm using the http output plugin to execute a post request which can sometimes return non 2XX http response codes. The service sometimes returns a 409, which is safe to ignore.
The problem is that there is no way to ignore those 409 responses and the http output plugin logs every error request to the log, creating very large log files and a lot of non-useful debug.
The potential solution is to create an ignore_codes array that can be passed as an option into the plugin, so that it can safely ignore any required non-2xx codes.
Example:
output {
http {
url => "http://localhost:3000"
http_method => "post"
ignore_codes => [409]
[...]
}
}
The change is really simple. The diff can be found at alfmatos@d9eeed7. If this is an acceptable approach, I can issue a PR.
Is the problem only "creating very large log files" ? Or is there something else?
Hi @jordansissel. There are a few issues that break our workflow.
- We're looking at some files with hundreds of megabytes
- It's triggering our monitoring alarms
- It's messing up our log analysis since we're using grep through very large files just to exclude the false positives (409)
@jordansissel just faced a major side effect which I think is caused by this: the failed posts seem to trigger retries that are kept in memory, eventually leading to a lot of memory consumption and cause an OOM. Since I was using http output to do a direct elasticsearch post, it would block all of the logstash pipeline, because elasticsearch began to stall.
Had to change the approach completely. Right now I'm in the process of debugging to see if it was really an HTTP Output + post to elasticsearch combo that causes recurrent OOM.