Page MenuHomePhabricator

Generate alerts if theoretically impossible or unwanted logging occurs
Closed, DeclinedPublic

Description

Author: swalling

Description:
In E3, we recently ran in to a situation where post-hoc analysis revealed logging that should in theory have been impossible given the user experience and bucketing logic (bug 47590). It would be relatively easy to set up alerts if actions like logging a test condition impression for a user id bucketed as control occurs, letting us know about errors as they happen. Since each experimental design is different, this may require setting up custom alerts per schema/log.


Version: unspecified
Severity: enhancement

Details

Reference
bz47591

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 1:32 AM
bzimport set Reference to bz47591.
bzimport added a subscriber: Unknown Object (MLST).

Here's one:

count(page-impression) >= count(page-save-attempt) >= count(page-save-success)

Super-simple approach: a repository which consists of a single script and a 'sql' subdirectory. The subdirectory contains SQL queries, one to a file. Each query should return a boolean. When the script is launched, it iterates through these queries and executes each in turn against the database. If any query returns 0, an e-mail alert is sent. A cron job ensures the script is invoked at regular intervals.

If we had this, Steven and Dario would be able to add and remove assertions very easily, using git rm / git add and git review. I'd use puppet to configure a cron job to keep the repository state up-to-date.

[moving from MediaWiki extensions to Analytics product - see bug 61946]

kevinator lowered the priority of this task from Medium to Low.Jan 23 2015, 5:46 PM
kevinator set Security to None.
kevinator subscribed.

This should happen on beta labs while product testing.