Page MenuHomePhabricator

Run collectstats.pl --regenerate to populate statistics about bug resolution
Closed, ResolvedPublic

Description

Currently "old charts" reports show statistics only since 2010-01-20, e.g.
https://bugzilla.wikimedia.org/reports.cgi?product=Wikimedia&datasets=UNCONFIRMED&datasets=NEW&datasets=ASSIGNED&datasets=REOPENED&datasets=VERIFIED&datasets=LATER&datasets=REMIND .
Bugzilla was upgraded the day before: https://wikitech.wikimedia.org/view/Server_admin_log/Archive_15#January_19
collectstats.pl populates such statistics: http://www.bugzilla.org/docs/tip/en/html/extraconfig.html
collectstats.pl --regenerate is supposed to populate them for all time and should be fast enough in currently installed version (4.0): http://www.bugzilla.org/releases/4.0/release-notes.html#v40_feat_other


Version: unspecified
Severity: normal

Details

Reference
bz29203

Event Timeline

bzimport raised the priority of this task from to High.Nov 21 2014, 11:34 PM
bzimport set Reference to bz29203.

pdhanda wrote:

Bugmeister is the new Bugzilla maintainer and default assignee.

Removing "shell" keyword for things that aren't directly doable by shell users etc

Removing shell keyword if exists

It looks like someone already ran the regeneration script since the charts are populated.

I have asked the ops to setup two cronjobs:
https://gerrit.wikimedia.org/r/#change,915

cron reviewed by Ryan Lane yesterday. Puppet must have deployed that cronjob by now :-)

Sunday has passed but the charts don't seem to have been regenerated: the first day is still the same (day of bugzilla update, 2010-01-20), but it should be the first day of bugzilla since there's no temporal limit.
Either the cronjob didn't run, or this note in https://gerrit.wikimedia.org/r/#change,915 is the reason: «Might need to verify the script collectstats.pl does exist in that directory». Could someone verify that it's there?

The collect script does not seem to work properly. Apergos gave me the cronjob output:

DBD::mysql::st execute failed: Duplicate entry '118-2011-11-24 00:00:00' for key 1 [for Statement "INSERT INTO series_data (series_id, series_date, series_value) VALUES (?, '2011-11-24', ?)" with ParamValues: 0='118', 1=244] at ./collectstats.pl line 526
main::CollectSeriesData() called at ./collectstats.pl line 162

Antoine can't finish this since he'd need root.

I don't know when, but the script has run in the end and the charts are populated at last. No idea what changed.

we ran into a problem with this one when testing on the new server

https://gerrit.wikimedia.org/r/#/c/109086/

we'll debug this soon before the switch-over of BZ (the change influences the new module, not the current/old kaulen server where it works and we won't touch it anymore)

it fails to execute: /bin/sh: 1: /srv/org/wikimedia/bugzilla/collectstats.pl: not found happens only on new server

06:58 < mutante> /bin/sh: 1: /srv/org/wikimedia/bugzilla/collectstats.pl: not found

06:58 < mutante> /bin/sh: symbolic link to `dash'

06:59 < mutante> /srv/org/wikimedia/bugzilla/collectstats.pl: a /usr/bin/perl -w script, ASCII text executable

Daniel: Did breaking it into two commands completely fix this?