Page MenuHomePhabricator

New reports not working; New cohorts all invalid
Closed, ResolvedPublic

Description

I was running an edit report for an existing cohort for a monthly breakdown of user stats. The report has been pending for about 12 hours. I tried a new report this morning and it is still pending as well.

Report is "Edits - WEP_JO_ALL_01sept13"

Old reports can be downloaded, but new reports are not being generated it seems.

I also tested the cohorts by uploading a new version. All usernames were declared invalid even though I've uploaded a duplicate cohort and they are in fact all valid. It gives the error:

"reason_invalid": "invalid project: arwiki"

Something is up with cohorts and reports -- neither are working!


Version: unspecified
Severity: critical
Platform: Macintosh

Details

Reference
bz62830

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:02 AM
bzimport set Reference to bz62830.

Thank you for the report Tighe, we just migrated the server to a new datacenter, and it's most likely an environment problem. Sorry for the inconvenience, I'm looking at it now.

I found wikimetrics --mode queue running in the background as a ghost queue. This might have been an artifact of a bad deployment or a more systemic issue with upstart not being able to kill that process. I killed them manually but I needed -9. I was able to run through the scenarios that were reported failing in this bug without trouble after that. Please re-test. And thank you for the bug report!

Thank you, Dan! Yes, I just tested cohort validation and ran an edit report and both worked as they should! I think we're back in business. Thanks!

(In reply to Dan Andreescu from comment #2)

I found wikimetrics --mode queue running in the background as a ghost queue.
This might have been an artifact of a bad deployment or a more systemic
issue with upstart not being able to kill that process. I killed them
manually but I needed -9. I was able to run through the scenarios that were
reported failing in this bug without trouble after that. Please re-test.
And thank you for the bug report!

Please re-open if issues recur, and thank you for your patience while we figure these things out.