A simple console program that monitors HTTP traffic on your machine. Listens to an actively written log file and displays statistics.
- The console app displays stats every 10s about the traffic during those 10s: the sections of the web site with the most hits, as well as interesting summary statistics on the traffic as a whole.
- Whenever total traffic for the past 2 minutes exceeds a certain number on average, the app displays a message saying that “High traffic generated an alert - hits = {value}, triggered at {time}”. The default threshold is 10 requests per second, and is overridable.
- Whenever the total traffic drops again below that value on average for the past 2 minutes, the app displays another message detailing when the alert recovered
The project is written in Go and uses go modules. Developement was made on go1.14.1 but was also tested on go1.13.
To build the project, run:
go build
Once you built the project run:
./log-monitor
To exit the app, simply press Q.
The options of the program are the following:
Usage of ./log-monitor:
-demo
demo or not, if demo the log file will be concurrently written with fake logs
-logfile string
logfile path (default "/tmp/access.log")
-threshold int
threshold for alerting in requests per second (default 10)
-timewindow int
time window for alerting in seconds (default 120)
-updateInterval int
number of seconds between each statistic update (default 10)
type ./log-monitor -help to display this message.
Example:
./log-monitor -logfile /tmp/access.log -threshold 10 -timewindow 60 -updateInterval 5
If the demo
flag is set to true, a separate log_generator
goroutine writes the log file to simulate logging.
The evolution of the number of logs written follows a triangle pattern. With the default threshold (10 per second), the
evolution of the traffic should trigger alerts approximately every 1-2 minute.
The architecture of the log-monitor has two main components:
- A monitor
- A display
The monitor communicates with the display by using two channels: one for statistics, one for alerts
The monitor listens to the log file and continuously checks for new logs. It keeps trace of the logs
written during the last updateInterval
seconds. Every updateInterval
it computes statistics of the current
logs and sends the computed statistics to the display by using the statistics channel. The statistics sent are:
- The 5 most requested sections
- The 5 most used HTTP methods
- The 5 most frequent HTTP status codes returned
- The number of requests
- The number of bytes transferred
The monitor also checks for alerts,
if the average traffic during the last timewindow
exceeds the threshold per second, an alert is sent to the display.
Alerts are sent by using the alert channel
The display uses termdash which is a terminal based dashboard to display the important information. It contains 4 panels:
- The uptime of the app
- The information panel on which statistics are displayed
- The alert panel on which alerts are displayed
- An histogram of the traffic evolution. The purpose of the histogram is just to give an intuition of the traffic evolution
The app could be improved in many ways:
- The monitor continuously checks for file modification, it would be better that file modifications trigger events. This could be maybe done by some package like fsnotify or tail
- Improve the user interface. If I had more time, I think I would have implemented an interface available on a web browser for conviency
- Add performance benchmarks and do some pprof