Page MenuHomePhabricator

list=allcampaigns API can run slowly or timeout if data not cached
Closed, ResolvedPublic

Description

Currently the upload campaigns branch of the Android Commons app uses UploadWizard's list=allcampaigns API to periodically refresh the list of campaigns.

Unfortunately this can run really slowly on Commons, especially if the parsed titles/descriptions/etc aren't cached due to a software update, massive cache clear, or selection of a new language. The worst case is a timeout or outright HTTP 500 error.

A few possibilities:

  • (probably easy) could aggressively limit the number of items returned when not cached, so you at least get a fast result with fewer items instead of a timeout. This allows clients to continue fetching to get the rest of the results as they are built.
  • (a bit more work) could do more lightweight rendering -- at fetch time we really only need the title; we could fetch the description and other fields when we go to upload something. Separately caching/rendering 'just titles' and 'whole field sets' could allow the list to run *much* faster.
  • (probably not feasible?) could pre-cache items in the JobQueue after their dependencies change?

Version: unspecified
Severity: normal
URL: commons.wikimedia.org/w/api.php?action=query&list=allcampaigns&uclimit=500

Details

Reference
bz55754

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:37 AM
bzimport set Reference to bz55754.

For 1, I got a 500 even with 50, and limiting it to below 50 sounds terrible to me. Complete hack!

2 is the right thing to do, perhaps. We could do a 'titles and description only' rendering / cache cycle, yeah. prop=titles|description can also be the default. Much less things to parse, so should be faster. Will split the cache, but that should not be an issue, I think.

3 sounds terrible :P For all the languages?!

Can't see the difference to bug 54465 nor decide which of the two is more helpful, I'll just dupe.

  • This bug has been marked as a duplicate of bug 54465 ***
Gilles raised the priority of this task from Medium to Unbreak Now!.Dec 4 2014, 10:24 AM
Gilles moved this task from Untriaged to Done on the Multimedia board.
Gilles lowered the priority of this task from Unbreak Now! to Medium.Dec 4 2014, 11:20 AM