Page MenuHomePhabricator

wm-bot has been completely off-line since netsplit 2.5-3 hours ago
Closed, ResolvedPublic

Description

netsplit 2.5-3 hours ago took out chanserv?, nickserv, and wm-bot of which only wm-bot is yet to return.


Version: unspecified
Severity: critical

Details

Reference
bz50786

Event Timeline

bzimport raised the priority of this task from to Unbreak Now!.Nov 22 2014, 1:46 AM
bzimport added a project: WM-Bot.
bzimport set Reference to bz50786.

[21:39] Quit: wm-bot [*.net *.split]
[01:03] <T13|sleeps> Is there anyone here that can restart wm-bot now that the freenode netsplitting fiasco seems to be over?
[01:13] <T13|sleeps> https://bugzilla.wikimedia.org/show_bug.cgi?id=50786
[01:14] <Reedy> Apparently I don't have root on bots..
[01:15] <T13|sleeps> It may just have to wait for petan to wake up. (still not sure who gave him sleep permission :p)
[01:16] Quit: lbenedix [Quit: Leaving.]
[01:16] <apergos> I don't either
[01:16] <Reedy> 3 SF based staff have project admin..
[01:16] <apergos> I guess I can give it to myself
[01:16] <Reedy> It's a holiday, sooo
[01:16] <apergos> meh
[01:17] <T13|sleeps> Just thinking there are a lot of channels it logs...
[01:20] <Reedy> "This incident will be reported."
[01:20] <Reedy> That reminds me of the xkcd..
[01:20] <apergos> or not.

Gave myself project admin but /home on bots-labs is read-only and it couldn't create my hme dir so no login possible. Home appears to be from gluster, shouldn't it be moved to nfs??

Gave Reedy project admin since he already has a login, to see if he can restart.

Added Reedy and me to sudoers all hosts for bots-labs project also.

It's back after being restarted!