Grep, but for log files of various shapes and sizes
This project was created in slightly more than an evening with the intention that I'll add more log formats to it as I go along and feel like they would be useful. There's no guarantee I'll actually do that though.
Download the latest release from the releases page and add it somewhere on your path (e.g. ~/.bin/local
).
Windows users will currently need to build from source using cargo build --release
, sorry!
There is also a docker image available on Docker Hub
Grep, but for log files of various shapes and sizes
Usage: loggrep [OPTIONS] [FIELD] [PATTERN]
Arguments:
[FIELD] Field to filter on
[PATTERN] Pattern to filter the field on
Options:
-f, --format <FORMAT> Specify the expected format of the logs
-i, --ignore-case Ignore case
-v, --invert-match Invert the sense of matching, to select non-matching lines
-F, --fixed Set the pattern to compare fixed strings rather than a regex
-h, --help Print help
-V, --version Print version
For example, for filter for nginx requests from 66.249.65.159
, you could do:
cat logfile.txt | loggrep -F remote_addr '66.249.65.159'
Or to filter for GET requests:
cat logfile.txt | loggrep request '^GET'
With docker:
cat logfile.txt | docker run -i jamoyjamie/loggrep:<version> request '^GET'
Format identifier: nginx
The default format of nginx logs. The following fields map to the nginx variables of the same name.
Fields:
remote_addr
remote_user
request
status
body_bytes_sent
http_referer
user_agent
Format identifier: syslog-bsd
Original BSD syslog format defined in RFC-3164
Fields:
priority
timestamp
hostname
message
Format identifier: python
Default log format for Python as per the source
Fields:
levelname
name
message
Format identifier: postgresql
Default PostgreSQL database log format as per the documentation
timestamp
pid
user
database
type
message
Format identifier: update-alternatives
Simple format for the update-alternatives log in /var/log/alternatives.log
.
Fields:
timestamp
message
Format identifier: dpkg
Format for dpkg logs in /var/log/dpkg.log
as per the man page. Note that this log format has three different forms which makes it tricky to parse. Due to internal limitations, currently some fields need to be repeated with a numeric on the end.
Fields:
timestamp
type
command
state
pkg
installed_version
action
pkg_2
installed_version_2
available_version
filename
decision
Format identifier: clf
Common Log Format as per Wikipedia and Microsoft also known as NCSA HTTPd used by web servers. This also implements to Combined Log Format as per the httpd docs which is the same but with some additional fields.
Fields:
host
ident
auth_user
timestamp
request
status
bytes
referer
(combined log format)user_agent
(combined log format)
- More formats
- CEF
- ELF
- GELF (JSON based)
- W3C Extended Log Format
- IIS Server
- redis
- Syslog IETF format
- Syslog Extended IETF format
- Add argument to draw logs from file
- Add colour to matched field if outputting to a terminal
- Support matching on multiple fields? (You can always just pipe loggrep into loggrep to do this...)
- Tests, always tests
- Support custom log formats from config file