Page MenuHomePhabricator

Make squid logger (udp2log) more scalable
Closed, ResolvedPublic

Description

udp2log-domas-multithread-2010-10-05.patch

Currently, our squid logging solution (udp2log) will start dropping packets when the configuration gets slightly complicated. This has limited our ability to log many things we'd like to log.

Given that the machine running udp2log (locke) isn't CPU-bound yet, we believe we can gain some headway by making udp2log multithreaded.

More details about the logger itself can be found here:
http://wikitech.wikimedia.org/view/Squid_logging

More details about this task can be found here:
http://lists.wikimedia.org/pipermail/wikitech-l/2010-August/048844.html

Additionally, Domas has already worked up an initial patch for this, which is attached.


Version: unspecified
Severity: enhancement

Attached:

Details

Reference
bz25427

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 11:12 PM
bzimport set Reference to bz25427.

Nimish has a version of this that uses multicast rather than changing to using threading.

Multicast UDP logging can be built on top frameworks like Spread (http://www.spread.org/docspread.html). Just FYI.

*Bulk BZ Change: +Patch to open bugs with patches attached that are missing the keyword*