Master Linux Log Management: From Crontab Cleanup to Logrotate Automation

Log Management Goals

Log management generally consists of two parts: the log content (anchors, format, etc.) that provides execution records and troubleshooting help, and the log archiving rules (splitting by date or size, retention period, etc.).

For self‑developed services, developers can customize logging via components such as Logback or Log4j. Third‑party services like MySQL, Nginx, Redis, or libraries such as Nacos and Sentinel often lack flexible log configuration, making centralized management difficult.

Special Log Scenarios

Some services produce uncontrolled log files if not specially configured:

Applications started with

nohup

without log redirection keep writing to

nohup.out

or a single file.

MySQL can set a log file path but does not clean logs automatically.

Nginx can set log templates and paths (default

access.log

,

error.log

) but also lacks automatic cleanup.

These single‑file logs can grow without bound and eventually exhaust disk space.

Special Tool – Scheduled Cleanup

Linux’s

crontab

combined with a cleanup script can delete logs older than a certain number of days. Example:

crontab -e
# Delete logs older than 7 days, keep recent 7 days
1 0 * * * find /logs.dir/ -mtime +7 | xargs rm -rf

When a log file is renamed or deleted while still held open by a process, the process continues writing to the old file descriptor, so simple removal does not free space. In such cases, either truncate the file (e.g.,

echo > log.log

) or restart the service after checking with

lsof | grep -i deleted

. Using

logrotate

with copy‑truncate avoids this problem.

Special Tool – logrotate

logrotate

is a Linux‑integrated log management utility scheduled by

crontab

. It supports custom rotation policies but can only rotate logs that the application writes to.

Common logrotate options include rotation period, file extension, rotation method (copy‑truncate or move‑delete), compression, and retention count.

logrotate [OPTION...] <configfile>
-d, --debug      # test configuration
-f, --force      # force rotation
-m, --mail=cmd   # mail compressed logs
-s, --state=file # state file
-v, --verbose    # verbose output

Example Nginx configuration (

/etc/logrotate.d/nginx

)

/usr/share/nginx/log/*.log{
    daily
    missingok
    rotate 7
    compress
    delaycompress
    notifempty
    create 644 root root
    sharedscripts
    postrotate
        [ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
    endscript
}

Typical parameters:

compress

,

copytruncate

,

size

,

daily

,

rotate

, etc.

Appendix – Simple Configurations

# MySQL logs
/data/mysql/log/mysqld.log{
    daily
    dateext
    copytruncate
    notifempty
    missingok
    olddir backup
    rotate 60
    compress
}

/usr/local/nginx/logs/access.log
/usr/local/nginx/logs/error.log{
    daily
    dateext
    copytruncate
    notifempty
    missingok
    olddir backup
    rotate 30
    compress
}

Appendix – Runtime Log Management for Common Components

Nginx does not auto‑clean; logs keep growing in a single file.

MySQL does not auto‑clean; logs keep growing in a single file.

Zookeeper supports automatic cleanup via size and count limits in its Log4j configuration.

Redis records only minimal core logs; cleanup is usually unnecessary.

Kafka data logs (topic, offset) support automatic cleanup via configuration.

Kafka operation logs rotate automatically but are not auto‑deleted; Log4j configuration is required.