Use logrotate for log rotation and automatic archiving.

logrotate can rotate, compress, and delete old log files. It is installed by default in most Linux distributions and automatically runs every day.

logrotate configuration file#

Create and edit the file /etc/logrotate.d/app. Generally, the configuration in /etc/logrotate.conf will include include /etc/logrotate.d, which will automatically load the configuration files in the /etc/logrotate.d directory.

/root/app/log/app.log {
    daily          # rotate every day
    missingok      # do not report error if log file is missing
    rotate 100     # keep 100 log files
    compress       # compress old log files
    delaycompress  # delay compression until next rotation
    notifempty     # only rotate if log file is not empty
    dateext             # use current date as log file suffix
    dateformat -%Y%m%d  # set date format
    olddir /root/app/log/old # set directory to store old log files (must be created beforehand)
    sharedscripts # run postrotate script once after all logs are rotated, instead of running it for each file
    postrotate    # script to run after log rotation
        /usr/bin/docker restart app
    endscript     # end of script
}

Note: Restarting the service may cause a network disconnect, so this should be taken into consideration. You can use the following command to test if the network will disconnect during the restart process:

for i in {1..1000}; do { curl -I https://example.com/; sleep 0.3; } ; done

Testing#

You can directly use the logrotate command to test. Use the -d option for debug mode, -f option to force execution, and -v option for verbose mode to see the execution process. In debug mode, the command will only print the execution process without actually executing it, which can be used for testing.

logrotate -dfv /etc/logrotate.d/app

After disabling debug mode, the result will be as follows:

$ tree
.
├── app.log
└── old
    ├── app.log-20231116.gz
    └── app.log-old

1 directory, 3 files

Potential Issues#

Permissions: I used the root user directly, so I did not encounter any permission issues. If you are using another user, you may encounter permission issues and need to be aware of them.
Restart: The script in the postrotate section is executed after log rotation, so if it is used to restart a service, it may cause a network disconnect. This should be taken into consideration.
Rotation based on file size: If log rotation is based on file size, it may result in log files being truncated. This should be taken into consideration. I did not encounter this issue because I am not using rotation based on file size.

References#

Linux Log Rotation Tool logrotate: Introduction and Configuration - A well-written article that explains the principles and configuration of logrotate.
logrotate(8) - Linux man page - Official documentation for logrotate, providing detailed configuration information.