Page MenuHomePhabricator

Search index not updating on en.wikipedia
Closed, ResolvedPublic

Description

Author: SpontaneousGrumbler

Description:
Looks like bugzilla 59979 all over again. Search for "~2014" and the date shows 15 January 2014. That page was edited on 16, 17, 18 and 19 January.


Version: wmf-deployment
Severity: normal

Details

Reference
bz60230

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 3:06 AM
bzimport set Reference to bz60230.

I'm unsure what was going on but I've restarted the incremental indexer as documented here: https://wikitech.wikimedia.org/wiki/Search#Main_indexer_on_searchidx2.2Fsearchidx1001_is_stuck

Hopefully it'll recover.

SpontaneousGrumbler wrote:

It ran OK on or about 19 January, but the index updating did not get done on 21 January.

The indexing machine is currently thrashing its disk something fierce. I'd dig into what is causing it but the service is end of life.... At this point I'm going to advise waiting to see if it gets through this rough patch.

Thrashing:
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 11.00 196.00 177.67 12.84 13.04 141.82 172.99 504.55 70.14 983.78 2.68 100.00

Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 7.00 285.00 198.67 20.93 15.47 154.11 173.22 436.21 49.78 990.56 2.07 100.00

Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 8.00 234.67 180.33 15.13 14.01 143.83 172.03 390.09 66.87 810.71 2.41 100.00

Adding Andrew Otto as a CC in case this could use some love from ops.

Index looks like it's been updating fairly regularly now. Closing this as I think we're done here.

Please reopen if we find things failing to update again.