Inherent dropping of messages during deployment of webstatscollector on 2014-10-15
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	QChris
	Oct 15 2014, 10:53 PM

Description

In order to redeploy webstatscollector, both the filter and the
collector process need to be restarted, which causes the messages that
have already been collected during this hour to get dropped.

To minimize message loss around the deployment of the fixes for bug
66352 and bug 71790, we waited until

http://dumps.wikimedia.org/other/pagecounts-raw/2014/2014-10/pagecounts-20141015-190000.gz
http://dumps.wikimedia.org/other/pagecounts-raw/2014/2014-10/projectcounts-20141015-190000

got written to disk (19:00:30) and then restarted webstatscollector's
filter and collector. On 19:02:30 both filter and collector were
running again.

So total loss is <2 minutes.

Version: unspecified
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=66352
https://bugzilla.wikimedia.org/show_bug.cgi?id=71790
https://bugzilla.wikimedia.org/show_bug.cgi?id=72107

Details

Reference: bz72102

Event Timeline

• bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:57 AM

• bzimport added a project: Analytics-General-or-Unknown.

• bzimport set Reference to bz72102.

• bzimport added a subscriber: Unknown Object (MLST).

QChris created this task.Oct 15 2014, 10:53 PM

Note that this only affects the files in

http://dumps.wikimedia.org/other/pagecounts-raw

. The files in

http://dumps.wikimedia.org/other/pagecounts-all-sites

are not affected.

Inherent dropping of messages during deployment of webstatscollector on 2014-10-15Closed, DeclinedPublicActions

Description

Details

Event Timeline

Inherent dropping of messages during deployment of webstatscollector on 2014-10-15
Closed, DeclinedPublic
Actions