Page MenuHomePhabricator

Special:External links lets you search 5000
Closed, DeclinedPublic

Description

Author: M8R-udfkkf

Description:
Using Special:External links, you can set the limit to 5000 manually and it will work. However, the API seems to enforce that users are allowed only 500 per query.

Is this a bug or a feature?

Example:Special:External links for youtube
https://secure.wikimedia.org/wikipedia/en/w/index.php?title=Special:LinkSearch&target=http%3A%2F%2Fyoutube.com&limit=5000&offset=0

Example:api
http://en.wikipedia.org/w/api.php?action=query&list=exturlusage&euquery=youtube.com&eulimit=5000


Version: unspecified
Severity: trivial

Details

Reference
bz22611

Event Timeline

bzimport raised the priority of this task from to Lowest.Nov 21 2014, 11:04 PM
bzimport set Reference to bz22611.
bzimport added a subscriber: Unknown Object (MLST).

matthew.britton wrote:

If a user has the right 'apihighlimits', API queries are limited to 5,000 items, otherwise they are limited to 500 items. UI queries are limited to 5,000 items regardless.

This is the case with most queries; for example, user contributions, page history, backlinks. The main exception is category members, the UI for which has a fixed size of 200 items (c.f. bug 11281).

The rationale for this is that the API allows multiple list modules to be invoked in a single request, so the per-request limit is actually at least 17,000 without higher limits and 170,000 with them -- more if extensions are installed (c.f. bug 16781).

Addressing this inconsistency would either mean restricting UI queries to 500 items (for users without 'apihighlimits') or limiting the sum of all API module limits in a single query. Both of these constitute a loss of functionality and break backwards compatibility, so unless a performance issue can be demonstrated things will likely remain as they are.

I agree that this is a little odd -- if one requires 5000 items of some query it is often quicker to scrape the UI than send 10 API requests.

I would suggest the best method of addressing this should one wish to do so would be to ask for 'apihighlimits' to be assigned to all user accounts on Wikimedia wikis. I don't know if the developers would be willing to do that, but I can't imagine the performance implications would be that great. Very few Wikimedia wiki users ever make custom API requests, and while 'apihighlimits' is granted as part of the administrator group, any concerns about giving it to non-privileged users are moot as the bot group also has it, and obtaining a bot account is trivial.