Page MenuHomePhabricator

[PoolCounter] Reproducible "pool-timeout" error on main pages of ja.wp, en.wp (and others)
Closed, ResolvedPublic

Description

When I try vieweing http://ja.wikipedia.org/wiki/%E3%83%A1%E3%82%A4%E3%83%B3%E3%83%9A%E3%83%BC%E3%82%B8 when logged out, I cannot see the page content, but see an error box:

"申し訳ありませんが、現在サーバーに過大な負荷がかかっています。
このページを閲覧しようとする利用者が多すぎます。 しばらく時間を置いてから、もう一度このページにアクセスしてみてください。

ロック待ちタイムアウト

"

which is in English:
"Sorry, the servers are overloaded at the moment. Too many users are trying to view this page. Please wait a while before you try to access this page again.

Timeout waiting for the lock

"


Version: wmf-deployment
Severity: critical
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=70485

Details

Reference
bz59993

Event Timeline

bzimport raised the priority of this task from to Unbreak Now!.Nov 22 2014, 2:42 AM
bzimport set Reference to bz59993.
bzimport added a subscriber: Unknown Object (MLST).

This is currently being discussed in #wikimedia-operations, as more sites seem to be affected:

<MaxSem> awww
now other wikis also report problems
2014-01-13 13:28:02 mw1201 enwiki: Pool queue is full
<mark> I see it even back in october
2013-10-19 07:53:22 mw1144 ruwiki: Накопитель запросов полон
2013-10-19 07:53:22 mw1201 enwiki: Pool queue is full
2013-10-19 07:53:22 mw1199 jawiki: プールキューがいっぱいです
2013-10-19 07:53:22 mw1130 dewiki: Poolwarteschlange ist voll
2013-10-19 07:53:22 mw1208 enwiki: Pool queue is full
<mark> we can perhaps increase the queue size a bit, see what that does
<MaxSem> however, looking in archive, yesterday's log was more than twice as long as the day before it

https://ja.wikipedia.org/ main page seems to work for me now.

See https://wikitech.wikimedia.org/wiki/Server_admin_log for Jan 13, 2014:
14:00 akosiaris: powering off hooper
13:47 logmsgbot: mark synchronized wmf-config/PoolCounterSettings-eqiad.php 'Raise ArticleView pool queue size by 50%'
13:46 logmsgbot: mark updated /a/common to I0442878ea: Raise ArticleView pool size by 50%
12:47 akosiaris: started poolcounter on potassium
12:46 mutante: starting poolcounter on heloum
12:45 MaxSem: that was https://bugzilla.wikimedia.org/show_bug.cgi?id=59993
12:44 akosiaris: restarted poolcounter on potassium, helium after MaxSem's request