Log Util is a Rust-based tool designed to analyze Nginx access logs. It reads log files, extracts and counts requests by IP addresses and URLs, and displays the top IP addresses and URLs. The tool can handle large log files by reading only new data as it is appended, or by processing the entire file from the beginning.
- Reads Nginx access logs and extracts IP addresses and URLs.
- Counts and displays the top IP addresses and URLs.
- Supports specifying the number of top entries to display.
- Allows customizing the regular expression used for log parsing.
- Supports loading the regular expression from a file.
- Automatic cleanup of outdated entries if there are more than 10,000 records (can be disabled with
--no-clear
). - Displays the last requests for top IPs.
- Allows filtering results by IP address.
file
: Path to the log file.--count
: Number of lines to read from the end of the file (0
to start from the end,-1
to read the entire file; default:0
).--regex
: Regular expression to parse the log entries or path to a file containing the regex (default:^(\S+) - ".+" \[(.*?)\] \d+\.\d+ "\S+" "\S+ (\S+?)(?:\?.*?)? HTTP/.*
).--date-format
: Date format to parse the log entries (default:%d/%b/%Y:%H:%M:%S %z
).--top
: Number of top entries to display (default:10
).--no-clear
: Disable automatic cleanup of outdated entries.
To read the entire log file:
cargo run -- "./access.log" --count=-1
To read the last 100 lines from the log file:
cargo run -- "./access.log" --count=100
To read new data from the end of the log file as it is appended:
cargo run -- "./access.log" --count=0
If the --regex
parameter points to a file, the regular expression will be read from that file.
cargo run -- "/path/to/access.log" --regex "/path/to/regex.txt" --top 20
q
: Quit the application.t
: Toggle between different tabs (Overview, Last Requests, Detailed Requests, Requests Chart).Up Arrow
: Scroll up.Down Arrow
: Scroll down.Left Arrow
: Scroll left.Right Arrow
: Scroll right.Ctrl+C
: Quit the application.
You can download the pre-built binaries from the latest GitHub release using curl
. Here is an example of how to download, make the file executable, and move it to a directory in your PATH
:
For x86_64
architecture:
curl -L -o /usr/local/bin/logutil https://github.com/s00d/logutil/releases/latest/download/logutil-x86_64-unknown-linux-gnu
chmod +x /usr/local/bin/logutil
For aarch64
architecture:
curl -L -o /usr/local/bin/logutil https://github.com/s00d/logutil/releases/latest/download/logutil-aarch64-unknown-linux-gnu
chmod +x /usr/local/bin/logutil
After downloading and making the file executable, you can run it from anywhere with:
logutil "/path/to/access.log" --count=-1 --top 20
Ensure you have Rust installed on your system. You can install Rust using the following command:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rust-lang.org | sh
- Clone the repository:
git clone https://github.com/yourusername/log-analyzer.git
cd log-analyzer
- Build the project:
cargo build --release
- Run the project:
cargo run -- "/path/to/access.log" --count=-1 --top 20
# or nginx
cargo run -- ./example.log --regex='^(\S+) - - \[(.*?)\]( )\"(\S+) (\S+?)(?:\?.*?)? .+' --count="-1"
The project includes a GitHub Actions workflow that automatically builds the project for x86_64-unknown-linux-gnu
and aarch64-unknown-linux-gnu
targets on each push or pull request to the main
branch.
The workflow file is located at .github/workflows/build.yml
. It performs the following steps:
- Checks out the code.
- Installs the Rust toolchain and necessary targets.
- Installs dependencies (e.g.,
gcc
). - Builds the project in release mode for the specified targets.
- Uploads the build artifacts.
Contributions are welcome! Please open an issue or submit a pull request for any improvements or bug fixes.
- Fork the repository.
- Create a new branch (
git checkout -b feature-branch
). - Make your changes.
- Commit your changes (
git commit -am 'Add some feature'
). - Push to the branch (
git push origin feature-branch
). - Open a pull request.
- Follow Rust's official style guidelines.
- Ensure the code passes all tests and lints.
- Write or update tests as necessary.
This project is licensed under the MIT License - see the LICENSE file for details.
- Regex for regular expression parsing.
- Structopt for command-line argument parsing.
- Tokio for asynchronous runtime support.
- Crossterm for terminal handling.
- Nginx Access Log (default)
^(\S+) - - \[(.*?)\]( )\"(\S+) (\S+?)(?:\?.*?)? .+
- Date format:
%Y-%m-%d %H:%M:%S %z
- Apache Access Log
^(\S+) \S+ \S+ \[.*?\] "\S+ (\S+?)(?:\?.*?)? \S+" \d+ \d+
- Date format:
%d/%b/%Y:%H:%M:%S %z
- Nginx Error Log
^(\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[error\] \d+#\d+: \*.*?, client: (\S+), server: \S+, request: "\S+ \S+ \S+", host: "\S+"
- Date format:
%Y/%m/%d %H:%M:%S
- Apache Error Log
^\[\w+ \w+ \d+ \d{2}:\d{2}:\d{2} \d{4}\] \[error\] \[client (\S+)\] \S+:\s(\S+)
- Date format:
%a %b %d %H:%M:%S %Y
- Kubernetes Log
^(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d+Z) \S+ \S+ (\S+)\[\d+\]: \S+ \S+ (\S+)$
- Date format:
%Y-%m-%dT%H:%M:%S%.fZ