I noticed 'acct' generates log files under /var/log/account/ which is on a 2GB partition. It is log rotated via a daily cron /etc/cron.daily/acct , keeping the last day uncompressed.
On deployment-bastion.eqiad.wmflabs that is 542MB currently, or a quarter of /var/ available disk space.
/etc/default/acct has an option to change the number of day to be kept:
Maybe we can reduce it to 15 days?
Version: unspecified
Severity: normal
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| base: Make number of days acct logs are kept customizable | operations/puppet | production | +17 -8 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | hashar | T71590 rsync errors to beta cluster, inconsistent state after scap | |||
| Resolved | hashar | T71601 Log files on labs instance fill up disk (/var is only 2GB) (tracking) | |||
| Resolved | hashar | T71604 acct (process and login accounting) fill up instances /var/ partition |
The package is acct which is part of of our standard packages installed via puppet. It provides data for commands such as 'ac' or 'lastcomm'.
Change 164520 had a related patch set uploaded by Hashar:
labs: reduce acct archiving retention
Change 177427 had a related patch set uploaded (by Yuvipanda):
base: Make number of days acct logs are kept customizable
Change 177427 abandoned by Yuvipanda:
base: Make number of days acct logs are kept customizable
/var is still filling up on deployment-bastion on a fairly regular basis because of these logs with the current retention on labs instances set to only 7 days.