Nginx logs not being rotated
Closed this issue · 11 comments
The Nginx logs stored under /usr/share/nginx/html/tripleawarclub.org/logs/ are not being rotated:
ssoloff@tripleawarclub:~$ ls -l /usr/share/nginx/html/tripleawarclub.org/logs/
total 1000372
-rw-r--r-- 1 www-data bung 957067724 Dec 9 13:40 access.log
-rw-r----- 1 www-data adm 66063409 Dec 9 13:39 dice.access.log
-rw-r----- 1 www-data adm 488707 Dec 8 10:16 dice.error.log
-rw-r--r-- 1 www-data bung 725413 Dec 9 02:37 error.log
-rw-r----- 1 www-data adm 1 Feb 11 2017 stats.access.log
-rw-r----- 1 www-data adm 1 Feb 11 2017 stats.error.logThe access log for www.tripleawarclub.org is currently over 900 MiB, while the access log for dice.tripleawarclub.org is ~63 MiB.
This article describes how to use logrotate on Ubuntu, which is pre-configured to support Nginx.
It looks like logrotate will automatically rotate logs in the /var/log/nginx/ folder. We probably just need to change the Nginx configurations to write the logs into this folder instead of under /usr/share/nginx/html/. /var/log/ is where I'd expect to find logs anyway; not sure what the reason was behind storing them parallel to the served web content.
I'll try this first with the new MARTI staging server and see how it works.
This has been implemented for the MARTI staging server. I'll monitor it over the next few days to ensure the logs get rotated.
One thing I'm not sure about is that the new logs under /var/log/nginx were created with ownership root:root, whereas the existing logs in that folder have ownership www-data:adm. I'm guessing that when the configuration was reloaded, the Nginx master process (running as root) created the log files. This post suggests that when Nginx receives a USR1 signal to flush the logs, the ownership will be changed (at least the user; not sure about the group) to allow worker processes (running as www-data) to write to the logs. logrotate sends USR1 as part of its log rotation process, so we'll see.
@RoiEXLab Agreed there's no point in moving the logs for the www subdomain.
I was going to close this issue once I confirmed the logs are actually being rotated (and that the ownership issue won't be a problem down the road). #16 will be closed once I deploy the new configurations that were just merged (hopefully later this evening).
This has been implemented for the MARTI production server.
As I mentioned previously, I'm just waiting to confirm logrotate is working as expected before closing this issue.
Looks like logrotate is working as expected:
ssoloff@tripleawarclub:~$ ll /var/log/nginx/dice*.log*
-rw-r----- 1 www-data adm 1724 Dec 16 13:31 /var/log/nginx/dice-staging.tripleawarclub.org-access.log
-rw-r--r-- 1 www-data root 23193 Dec 16 06:02 /var/log/nginx/dice-staging.tripleawarclub.org-access.log.1
-rw-r----- 1 www-data adm 281 Dec 16 11:17 /var/log/nginx/dice-staging.tripleawarclub.org-error.log
-rw-r--r-- 1 www-data root 9756 Dec 16 02:20 /var/log/nginx/dice-staging.tripleawarclub.org-error.log.1
-rw-r----- 1 www-data adm 55104 Dec 16 13:44 /var/log/nginx/dice.tripleawarclub.org-access.log
-rw-r--r-- 1 www-data root 179811 Dec 16 06:04 /var/log/nginx/dice.tripleawarclub.org-access.log.1
-rw-r----- 1 www-data adm 2389 Dec 16 12:13 /var/log/nginx/dice.tripleawarclub.org-error.log
-rw-r--r-- 1 www-data root 208 Dec 15 13:57 /var/log/nginx/dice.tripleawarclub.org-error.log.1
I verified the logrotate configuration for Nginx is responsible for changing ownership of new log files to www-data:adm and for setting the permissions to 640. So the original log files that had ownership of root:root and permissions 644 will be rotated off at some point and all log files will eventually have the same ownership/permissions.
NB: In case anyone's wondering, the original log file ownership was changed from root:root to www-data:root by Nginx itself, as discussed above, to allow worker processes to write logs. logrotate doesn't change the ownership/permissions of files it does not create.
BTW, it's pretty interesting looking at the MARTI access logs. I didn't realize how many attempts are made to attack this service. 😟
@ssoloff I need to have a look now ^^
But I'm not suprised, I'm regularily looking at the collected 404 errors of the forums where I always clear out all the GET requests to /wp-login.php or /phpmyadmin etc.
Here are some examples, I just checked:
