Page MenuHomePhabricator

Campaigns API call is very, very slow
Closed, ResolvedPublic

Description

Hitting https://commons.wikimedia.org/w/api.php/?action=query&list=allcampaigns takes me about 10s or so before it returns. Eeeek!


Version: unspecified
Severity: normal

Details

Reference
bz54465

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:11 AM
bzimport set Reference to bz54465.

Did some more investigation, and looks like DB hits aren't part of the problem (apergos ran EXPLAIN on the produced queries on production DB).

Asking for all campaigns (~140 at this point) takes about 17s, 50 takes 7s, and 5 takes 0.7s. Seems to be sortof linear. Current suspicion is that all the wikitext parsing is what is causing the problem...

Jotting down a quick note from IRC discussion:

Making sure the parsed output is cached, and invalidating via standard refreshlinks jobqueue etc may help...

I worry though that the multilanguage & translation support will make this tricky; after a large invalidation the worst-case rendering time for 140+ campaigns would still be really terrible.

A cheap hack to help might be to restrict the default paging limit on the API call, like some of the more expensive API calls do. This'd break up the worst-case rendering time into smaller individual requests which can return in decent time.

Another solution might be to use a cheaper bulk call that doesn't translate anything but the campaign names, then fetch the details on a subsequent request if you need them.

Yuvi, can you let us know if your latest patch has fixed this issue?

Should be, yeah. I can confirm once this rolls out to Commons on Monday. Testwiki perf testing makes the calls complete in about 200ms, while it was close to a second before. A similar reduction on commons would be great.

  • Bug 55754 has been marked as a duplicate of this bug. ***

Is Campaigns using pasercache? That seems like the easiest way to make it faster...

(In reply to Yuvi Panda from comment #4)

I can confirm once this rolls out to Commons on Monday.

Yuvi? Also, still working on this (assignee field)?

Nope, not anymore, sorry.

Cluster didn't melt, ops didn't scream, so I presumed it was ok.

matmarex assigned this task to yuvipanda.
matmarex set Security to None.