Version: unspecified
Severity: normal
Whiteboard: u=Community c=General/Unknown p=0 s=2014-08-07
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=69663
Description
Description
Details
Details
- Reference
- bz69661
Event Timeline
Comment Actions
Change 154457 had a related patch set uploaded by QChris:
Disable udp2log forward to University of Minnesota
Comment Actions
Since noon 2014-08-16, the disk space on root of oxygen started to
fill up (see attachment Maximum-disk-space-used-oxygen-2014-08-07):
qchris@oxygen:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/md126 111G 105G 0 100% / [...]
It seems udp2log's log is responsible:
qchris@oxygen:~$ ls -lh /var/log/udp2log/udp2log.log -rw-r--r-- 1 udp2log udp2log 101G Aug 17 11:57 /var/log/udp2log/udp2log.log
. This file contains many lines like
/bin/grep: writing output: Broken pipe
. Since bento.cs.umn.edu (udp2log target for Univesity of Minnesota
stream) currently does not resolve, it seems this filter might be
responsible. The change from comment #1 tries to turn this filter off
for now.
Comment Actions
ottomata also cleaned up the udp2log log file on oxygen, and oxygen is all
green again in icinga.
Thanks ottomata!