logstash-plugins/logstash-patterns-core

Apache error log with referer

jbelien opened this issue · 3 comments

jbelien commented

Hello everyone,

I have this line in my Apache error log :

[Fri Oct 20 16:39:12.472720 2017] [proxy_fcgi:error] [pid 5727:tid 124259626391296] [client 12.34.56.78:12345] AH01071: Got error 'PHP message: PHP Notice: Undefined index: nis5 in /var/www/fn.php on line 305\n', referer: https://mywebsite/

Every field is correctly extracted with HTTPD24_ERRORLOG as defined in https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/httpd ;
But I would like to also extract the referer from the end of the error message.

This referer is of course not always there so I tried this :

HTTPD24_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{WORD:module}:%{LOGLEVEL:loglevel}\] \[pid %{POSINT:pid}(:tid %{NUMBER:tid})?\]( \(%{POSINT:proxy_errorcode}\)%{DATA:proxy_message}:)?( \[client %{IPORHOST:clientip}:%{POSINT:clientport}\])?( %{DATA:errorcode}:)? %{GREEDYDATA:message}(, referer: %{QS:referrer})?

But it doesn't work, the referer is not extracted and is still in message !
How can I extract this referer ?

Thanks for your help !

jsvd commented

I see you're trying to match , referer: https://mywebsite/ with , referer: %{QS:referrer}, but referrer isn't a quote string. probably %{GREEDYDATA:referrer} will work

jsvd commented

Also, for further usage questions, please ask them in https://discuss.elastic.co.

GitHub is for coding issues and error reporting.

jbelien commented

Discussion continues here : https://discuss.elastic.co/t/apache-error-log-with-referer/105770

(sorry for the noise here)