Page MenuHomePhabricator

Unable to connect to redis server
Closed, ResolvedPublic

Description

I tried to edit http://es.wikipedia.beta.wmflabs.org/w/index.php?title=P%C3%A1gina_principal , and got an error:

[d2001cbd] /w/index.php?title=P%C3%A1gina_principal&action=submit Exception from line 827 of /srv/mediawiki/php-master/includes/jobqueue/JobQueueRedis.php: Unable to connect to redis server.

Backtrace:

#0 /srv/mediawiki/php-master/includes/jobqueue/JobQueueRedis.php(209): JobQueueRedis->getConnection()
#1 /srv/mediawiki/php-master/includes/jobqueue/JobQueue.php(340): JobQueueRedis->doBatchPush(array, integer)
#2 /srv/mediawiki/php-master/includes/jobqueue/JobQueue.php(311): JobQueue->batchPush(array, integer)
#3 /srv/mediawiki/php-master/includes/jobqueue/JobQueueGroup.php(127): JobQueue->push(array)
#4 /srv/mediawiki/php-master/extensions/CirrusSearch/includes/Hooks.php(327): JobQueueGroup->push(array)
#5 /srv/mediawiki/php-master/includes/Hooks.php(206): CirrusSearch\Hooks::onLinksUpdateCompleted(LinksUpdate)
#6 /srv/mediawiki/php-master/includes/GlobalFunctions.php(3991): Hooks::run(string, array, NULL)
#7 /srv/mediawiki/php-master/includes/deferred/LinksUpdate.php(152): wfRunHooks(string, array)
#8 /srv/mediawiki/php-master/includes/deferred/DataUpdate.php(101): LinksUpdate->doUpdate()
#9 /srv/mediawiki/php-master/includes/page/WikiPage.php(2208): DataUpdate::runUpdates(array)
#10 /srv/mediawiki/php-master/includes/page/WikiPage.php(1935): WikiPage->doEditUpdates(Revision, User, array)
#11 /srv/mediawiki/php-master/includes/page/Article.php(2002): WikiPage->doEditContent(WikitextContent, string, integer, boolean, User, string)
#12 /srv/mediawiki/php-master/includes/EditPage.php(1902): Article->__call(string, array)
#13 /srv/mediawiki/php-master/includes/EditPage.php(1286): EditPage->internalAttemptSave(array, boolean)
#14 /srv/mediawiki/php-master/includes/EditPage.php(522): EditPage->attemptSave()
#15 /srv/mediawiki/php-master/includes/actions/EditAction.php(56): EditPage->edit()
#16 /srv/mediawiki/php-master/includes/actions/SubmitAction.php(40): EditAction->show()
#17 /srv/mediawiki/php-master/includes/MediaWiki.php(414): SubmitAction->show()
#18 /srv/mediawiki/php-master/includes/MediaWiki.php(282): MediaWiki->performAction(Article, Title)
#19 /srv/mediawiki/php-master/includes/MediaWiki.php(584): MediaWiki->performRequest()
#20 /srv/mediawiki/php-master/includes/MediaWiki.php(435): MediaWiki->main()
#21 /srv/mediawiki/php-master/index.php(46): MediaWiki->run()
#22 /srv/mediawiki/w/index.php(3): include(string)
#23 {main}

Similar errors were reported from Jenkins jobs by Zeljko, Chris and Hashar.


Version: unspecified
Severity: blocker

Details

Reference
bz71415

Event Timeline

bzimport raised the priority of this task from to Unbreak Now!.Nov 22 2014, 3:47 AM
bzimport set Reference to bz71415.

Created attachment 16622
redis error

Attached:

Screen_Shot_2014-09-29_at_8.24.00_AM.png (974×1 px, 270 KB)

gerritadmin wrote:

Change 163595 had a related patch set uploaded by Ori.livneh:
Labs: Update IP of jobqueue's redis

https://gerrit.wikimedia.org/r/163595

gerritadmin wrote:

Change 163595 merged by jenkins-bot:
Labs: Update IP of jobqueue's redis

https://gerrit.wikimedia.org/r/163595

I'm still seeing errors about redis when trying to save an edit

for example, clicking Preferences:

[fe58754d] /wiki/Special:Preferences Exception from line 827 of /srv/mediawiki/php-master/includes/jobqueue/JobQueueRedis.php: Unable to connect to redis server.

Backtrace:

#0 /srv/mediawiki/php-master/includes/jobqueue/JobQueueRedis.php(123): JobQueueRedis->getConnection()
#1 /srv/mediawiki/php-master/includes/jobqueue/JobQueueRedis.php(114): JobQueueRedis->doGetSize()
#2 /srv/mediawiki/php-master/includes/jobqueue/JobQueue.php(191): JobQueueRedis->doIsEmpty()
#3 /srv/mediawiki/php-master/extensions/BetaFeatures/BetaFeaturesHooks.php(53): JobQueue->isEmpty()
#4 /srv/mediawiki/php-master/extensions/BetaFeatures/BetaFeaturesHooks.php(100): BetaFeaturesHooks::getUserCountsFromDb(array)
#5 /srv/mediawiki/php-master/extensions/BetaFeatures/BetaFeaturesHooks.php(182): BetaFeaturesHooks::getUserCounts(array)
#6 /srv/mediawiki/php-master/includes/Hooks.php(206): BetaFeaturesHooks::getPreferences(User, array)
#7 /srv/mediawiki/php-master/includes/GlobalFunctions.php(3991): Hooks::run(string, array, NULL)
#8 /srv/mediawiki/php-master/includes/Preferences.php(99): wfRunHooks(string, array)
#9 /srv/mediawiki/php-master/includes/Preferences.php(1236): Preferences::getPreferences(User, RequestContext)
#10 /srv/mediawiki/php-master/includes/specials/SpecialPreferences.php(58): Preferences::getFormObject(User, RequestContext)
#11 /srv/mediawiki/php-master/includes/specialpage/SpecialPage.php(363): SpecialPreferences->execute(NULL)
#12 /srv/mediawiki/php-master/includes/specialpage/SpecialPageFactory.php(584): SpecialPage->run(NULL)
#13 /srv/mediawiki/php-master/includes/MediaWiki.php(275): SpecialPageFactory::executePath(Title, RequestContext)
#14 /srv/mediawiki/php-master/includes/MediaWiki.php(584): MediaWiki->performRequest()
#15 /srv/mediawiki/php-master/includes/MediaWiki.php(435): MediaWiki->main()
#16 /srv/mediawiki/php-master/index.php(46): MediaWiki->run()
#17 /srv/mediawiki/w/index.php(3): include(string)
#18 {main}

Ori: can you please take a look at this ASAP? Redis dependency is breaking Beta/it's unasable for testing now.

Can't save edits:

[29a22380] /w/index.php?title=0.01050393981591069&action=submit Exception from line 827 of /srv/mediawiki/php-master/includes/jobqueue/JobQueueRedis.php: Unable to connect to redis server.

Backtrace:

#0 /srv/mediawiki/php-master/includes/jobqueue/JobQueueRedis.php(209): JobQueueRedis->getConnection()
#1 /srv/mediawiki/php-master/includes/jobqueue/JobQueue.php(340): JobQueueRedis->doBatchPush(array, integer)
#2 /srv/mediawiki/php-master/includes/jobqueue/JobQueue.php(311): JobQueue->batchPush(array, integer)
#3 /srv/mediawiki/php-master/includes/jobqueue/JobQueueGroup.php(127): JobQueue->push(array)
#4 /srv/mediawiki/php-master/includes/mail/EmailNotification.php(171): JobQueueGroup->push(array)
#5 /srv/mediawiki/php-master/includes/changes/RecentChange.php(332): EmailNotification->notifyOnPageChange(User, Title, string, string, integer, integer, string)
#6 /srv/mediawiki/php-master/includes/changes/RecentChange.php(553): RecentChange->save()
#7 /srv/mediawiki/php-master/includes/page/WikiPage.php(1906): RecentChange::notifyEdit(string, Title, boolean, User, string, integer, string, integer, string, integer, integer, integer, boolean)
#8 /srv/mediawiki/php-master/includes/page/Article.php(2010): WikiPage->doEditContent(WikitextContent, string, integer, boolean, User, string)
#9 /srv/mediawiki/php-master/includes/EditPage.php(1902): Article->__call(string, array)
#10 /srv/mediawiki/php-master/includes/EditPage.php(1286): EditPage->internalAttemptSave(array, boolean)
#11 /srv/mediawiki/php-master/includes/EditPage.php(522): EditPage->attemptSave()
#12 /srv/mediawiki/php-master/includes/actions/EditAction.php(56): EditPage->edit()
#13 /srv/mediawiki/php-master/includes/actions/SubmitAction.php(40): EditAction->show()
#14 /srv/mediawiki/php-master/includes/MediaWiki.php(414): SubmitAction->show()
#15 /srv/mediawiki/php-master/includes/MediaWiki.php(282): MediaWiki->performAction(Article, Title)
#16 /srv/mediawiki/php-master/includes/MediaWiki.php(584): MediaWiki->performRequest()
#17 /srv/mediawiki/php-master/includes/MediaWiki.php(435): MediaWiki->main()
#18 /srv/mediawiki/php-master/index.php(46): MediaWiki->run()
#19 /srv/mediawiki/w/index.php(3): include(string)
#20 {main}

(Actually, I might just ask out on [Ops] for some (SWAT) deployer to help out.)

<insert something about how Labs was weird and let Ori mistakeningly make two instances with the name deployment-redis01 and something something DNS and ghosts in the machine and such>

Special:Preferences works! Saving an edit works!

Thanks Ori!