Page MenuHomePhabricator

CirrusSearch: updateOneSearchIndexConfig.php needs to be super careful to catch errors from Elasticsearch
Closed, ResolvedPublic

Description

When I was trying to recover from our bad redundancy I noticed that CirrusSearch would sometimes continue when Elasticsearch failed to apply some change. I know this error occurred for applying mappings. Here is an example:

Validating mapping for page type...different...failed!

Couldn't update mappings. Here is elasticsearch's error message: RemoteTransportException[[testsearch1001][inet[/10.64.32.138:9300]][indices/mapping/put]]; nested: ProcessClusterEventTimeoutException[failed to process cluster event (put-mapping [page]) within 30s];

Validating aliases...
        Validating content alias...is taken...
        Reindexing...

So far as I can tell the Elasticsearch error was caused by the mixed versions on the Elasticsearch cluster bug but it should have caused Cirrus to stop cold rather than proceed.


Version: unspecified
Severity: normal
Whiteboard: cirrus_reenable

Details

Reference
bz57250

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 2:23 AM
bzimport added a project: CirrusSearch.
bzimport set Reference to bz57250.

Change 96383 had a related patch set uploaded by Manybubbles:
Tighter error handling in updateSearchIndex

https://gerrit.wikimedia.org/r/96383

Change 96383 merged by jenkins-bot:
Tighter error handling in updateSearchIndex

https://gerrit.wikimedia.org/r/96383