Master Logrotate: Automate Linux Log Rotation and Ditch Custom Scripts

Why Logrotate Matters

Log files are essential for Linux system security and troubleshooting, yet many operators create custom cron jobs instead of using the standard logrotate utility, which leads to duplicated effort and missed rotations.

Basic Configuration Files

The main configuration resides in /etc/logrotate.conf, while per‑service files are placed in /etc/logrotate.d/. The global file defines defaults such as weekly rotation, rotate 4 backups, create new files, and dateext for date‑based filenames.

# cat /etc/logrotate.conf
weekly
rotate 4
create
dateext
compress
include /etc/logrotate.d

Running Logrotate Manually

Logrotate runs daily via /etc/cron.daily/logrotate. You can test configurations with: # /usr/sbin/logrotate -d -f /etc/logrotate.conf Use -v for verbose output, -d for debug (no changes), and -f to force rotation.

Typical Service Configurations

Nginx example rotates daily, keeps seven backups, and reloads Nginx after rotation:

/usr/local/nginx/logs/*.log {
    daily
    rotate 7
    missingok
    notifempty
    dateext
    sharedscripts
    postrotate
        if [ -f /usr/local/nginx/logs/nginx.pid ]; then
            kill -USR1 `cat /usr/local/nginx/logs/nginx.pid`
        fi
    endscript
}

PHP‑FPM example keeps 365 daily backups and restarts the service after rotation:

/Data/logs/php/*log {
    daily
    rotate 365
    missingok
    notifempty
    compress
    dateext
    sharedscripts
    postrotate
        if [ -f /Data/app/php5.6.26/var/run/php-fpm.pid ]; then
            kill -USR1 `cat /Data/app/php5.6.26/var/run/php-fpm.pid`
        fi
    endscript
}

System logs (cron, maillog, messages, secure) share a common block with sharedscripts and a postrotate that sends HUP to syslogd.

/var/log/cron /var/log/maillog /var/log/messages /var/log/secure {
    sharedscripts
    compress
    rotate 30
    daily
    dateext
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid` || true
    endscript
}

Troubleshooting Logrotate

If a log (e.g., Nginx) does not rotate, verify that the daily cron job runs (check /var/log/cron for run-parts /etc/cron.daily) and inspect /etc/cron.daily/logrotate. Replace the default call with -f to force rotation when size thresholds prevent it.

# cat /etc/cron.daily/logrotate
#!/bin/sh
/usr/sbin/logrotate /etc/logrotate.conf > /dev/null 2>&1
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
    /usr/bin/logger -f logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit 0

After editing, restart the cron daemon:

# /etc/init.d/crond restart

Adjusting Rotation Schedule

Logrotate’s execution time is controlled by /etc/anacrontab. To rotate at midnight instead of the default early‑morning window, modify the START_HOURS_RANGE or replace the anacron entry with a custom cron line, e.g.:

59 23 * * * /usr/sbin/logrotate -f /etc/logrotate.d/syslog > /dev/null 2>&1

Custom Rotation Scripts

When Logrotate is insufficient, you can write Python or shell scripts. Example Python script moves a log to a dated directory and restarts the service:

#!/usr/bin/env python
import datetime, os, shutil, sys
log_path = '/opt/jumpserver/logs/'
log_file = 'jumpserver.log'
yesterday = datetime.datetime.now() - datetime.timedelta(days=1)
dest = os.path.join(log_path, yesterday.strftime('%Y'), yesterday.strftime('%m'))
os.makedirs(dest, exist_ok=True)
shutil.move(log_path + log_file, os.path.join(dest, f"{log_file}_{yesterday.strftime('%Y%m%d')}.log"))
os.system('sudo /opt/jumpserver/service.sh restart')

Shell script example for generic log rotation:

#!/bin/sh
rotate() {
    logs_path=$1
    echo "Rotating Log: $logs_path"
    cp "$logs_path" "${logs_path}.$(date -d "yesterday" +%Y%m%d)"
    > "$logs_path"
    rm -f "${logs_path}.$(date -d "7 days ago" +%Y%m%d)"
}
for i in "$@"; do
    rotate "$i"
done

Compression Helpers

A simple Bash script compresses logs older than a given number of days:

#!/usr/bin/sh
sysName=$1
appName=$2
keepDay=$3
logDir=/var/log/${sysName}/${appName}
cd "$logDir"
find . -name "${appName}.*[0-9][0-9].log" -mtime +$keepDay -exec gzip {} \;

Recommended Nginx Rotation Script

This script moves yesterday’s logs to a bak directory, reloads Nginx, and deletes backups older than seven days:

#!/bin/bash
yesterday=$(date -d "-1 days" +%Y%m%d)
logdir=$(dirname "$0")/bak
mkdir -p "$logdir"
for log in *.log; do
    mv "$log" "$logdir/${log}.${yesterday}.bak"
    # gzip "$logdir/${log}.${yesterday}.bak"
done
/usr/local/sbin/nginx -s reload
cd "$logdir"
find . -type f -name "*.bak" -mtime +7 -delete

By following these configurations and scripts, administrators can reliably rotate logs, conserve disk space, and avoid the pitfalls of ad‑hoc cron jobs.