Page MenuHomePhabricator

Beta setup of event logging not working
Closed, ResolvedPublic

Description

There are no events being logged for event logging in beta since August 22, also looks like EL was restarted about then.

host: deployment-eventlogging02.eqiad.wmflabs


Version: unspecified
Severity: normal

Details

Reference
bz73388

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:56 AM
bzimport set Reference to bz73388.
bzimport added a subscriber: Unknown Object (MLST).

Looks like code deployed there is pretty old:

commit 395a1b1a9034ba413b7f9886923e08d734cb2ac7
Author: Ori Livneh <ori@wikimedia.org>
Date: Thu May 15 14:59:25 2014 -0700

Check that $_SERVER[ 'HTTP_USER_AGENT' ] is nonempty

$_SERVER[ 'HTTP_USER_AGENT' ] can be set to null, in which case the resulting
event fails validation, since null does not have type string. This doesn't
happen often, but we should catch it.

Change-Id: I89819bd943ddbbb959c12b768b76f02d3d1140c5

Last changeset deployed to vanadium:

nuria@vanadium:/srv/deployment/eventlogging/EventLogging$ git log -l1
commit b4b42effc61567407ff128ccab89205ec55880e0
Merge: e4b68b1 672a423
Author: jenkins-bot <jenkins-bot@gerrit.wikimedia.org>
Date: Mon Jun 16 19:32:53 2014 +0000

Merge "Harmonize source files coding tag to "utf-8""

I updated the code but I think the machine needs a reboot or at least, a re-start of all processes of EL for it to work properly. Looks like harshar is the owner, so it will let him know.

Rebooted machine & manually updated code, looks like there are events appearing now. Not sure if any other changes took place.

Additional changes that help fix this issue were done by Yuvi: https://gerrit.wikimedia.org/r/#/c/173758/

Eventlogging puppet role was "coupled" with ganglia and recent changes to labs (in which gmond was removed) made the EL role not work in labs.

In order to verify that events are coming in into EL beta-labs QA can tail
/var/log/eventlogging/all-events.log
where all events are stored.

Also:

https://gerrit.wikimedia.org/r/#/c/173352/

Then I had to manually reset the root MySQL password on the labs instance (I set it to 'secret') and create the 'eventlogging' account.

Closing, can see event logging events in db and log stream w/o issues.

Tables on db:

CentralAuth_5690875
Echo_7731316
ImageMetricsLoadingTime_10078363
MediaViewer_10308479
MobileOptionsTracking_8101982
MobileWebClickTracking_5929948
MobileWebEditing_8599025
MultimediaViewerDimensions_10014238
MultimediaViewerVersusPageFilePerformance_7907636
PageContentSaveComplete_5588433
PageCreation_7481635
PrefUpdate_5563398
TimingData_7254808
UploadWizardErrorFlowEvent_9924376
UploadWizardStep_8851805
UploadWizardTutorialActions_5803466
UploadWizardUploadFlowEvent_9651951