Page MenuHomePhabricator

CirrusSearch: add timeouts for searches and for searches
Closed, ResolvedPublic

Description

We currently only have timeouts for updates but we'll need to add them for searches because bad nodes can take out the whole clusters if they sit on the request, say, because they have bad heap settings.

We'll need to set the shard timeout as well so Elasticsearch can timeout those requests. It is much better for Elasticsearch to timeout the requests because we'll get partial data and, if we timeout and just hit it again, then we can fill its request queues.


Version: unspecified
Severity: normal

Details

Reference
bz62079

Event Timeline

bzimport raised the priority of this task from to Unbreak Now!.Nov 22 2014, 3:06 AM
bzimport added a project: CirrusSearch.
bzimport set Reference to bz62079.

Looks like this is what we need on the Elasticsearch side:
timeout

A search timeout, bounding the search request to be executed within the specified time value and bail with the hits accumulated up to that point when expired. Defaults to no timeout.

Change 116481 had a related patch set uploaded by Manybubbles:
Introduce search timeouts

https://gerrit.wikimedia.org/r/116481

Change 116481 merged by jenkins-bot:
Introduce search timeouts

https://gerrit.wikimedia.org/r/116481

Change 116757 had a related patch set uploaded by Manybubbles:
Introduce search timeouts

https://gerrit.wikimedia.org/r/116757

Change 116757 merged by jenkins-bot:
Introduce search timeouts

https://gerrit.wikimedia.org/r/116757