Page MenuHomePhabricator

update search index and cached data on de.ws
Closed, ResolvedPublic

Description

Author: enomil

Description:
The search index on de.ws is from march 2010. Whit that nobody can really work (since march we have ~20.000 new pages and a few moved pages whit deleted redirects).

So update the Special:Search.

Also Special: pages with the last update from Ocotber 2009:

Special:AncientPages
Special:CrossNamespaceLinks
Special:DeadendPages
Special:FewestRevisions
Special:MostLinkedPages
Special:MostRevisions
Special:WantedPages


Version: unspecified
Severity: enhancement

Details

Reference
bz26203

Event Timeline

bzimport raised the priority of this task from to High.Nov 21 2014, 11:13 PM
bzimport set Reference to bz26203.

(In reply to comment #0)

The search index on de.ws is from march 2010. Whit that nobody can really work
(since march we have ~20.000 new pages and a few moved pages whit deleted
redirects).

So update the Special:Search.

Isn't the search index supposed to update itself?

rainman wrote:

The indexing seems to be stuck because Special:OAIRepository returns 500 on this request:

http://de.wikisource.org/w/index.php?title=Special:OAIRepository&verb=ListRecords&metadataPrefix=mediawiki&from=2010-03-17T21:09:10Z

I tried other dates, but they seem to work, so maybe it's some kind of strange database problem? I will bump the refresh date forward to continue indexing, but if someone has time could you please check what is wrong?

You can find the user/pass for OAIRepository in /etc/lsearch.conf on searchidx1

r.

rainman wrote:

Roan looked at the error logs, and I submitted his findings as bug 26304.

Skipping forward in time didn't work because the error reoccurred. Instead, I added another cronjob to searchidx1 to specifically handle dewikisource by rebuilding it completely from XML dumps on daily basis. The complete rebuild takes about 1h.

Thus, marking this as fixed.

enomil wrote:

It seems like the cronjob does not work anymore since middle/late september, for example "Julius Ailio" from 13:28 (CEST), 29. Sep. 2011 and later new entries cannot be find.

The search index is outdated now since over 3 month. Please rebuild (or whatever you have to do) the search index or fix the bug that the rebuild is not succesfull. All new texts on de-Wikisource since september can't find by anybody.

rainman wrote:

Probably same issue as Bug 32947

rainman wrote:

Seems to be fixed now, can you verify?

Mentioned in SAL (#wikimedia-operations) [2021-07-13T16:25:35Z] <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on mw1281.eqiad.wmnet with reason: decom T28203

Mentioned in SAL (#wikimedia-operations) [2021-07-13T16:25:40Z] <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on mw1281.eqiad.wmnet with reason: decom T28203

Mentioned in SAL (#wikimedia-operations) [2021-07-13T16:26:01Z] <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on mw[1282-1283].eqiad.wmnet with reason: decom T28203

Mentioned in SAL (#wikimedia-operations) [2021-07-13T16:26:07Z] <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on mw[1282-1283].eqiad.wmnet with reason: decom T28203