Page MenuHomePhabricator

Allow sorting of Special:Whatlinkshere lists by page title
Open, LowPublicFeature

Description

Author: gangleri

Special:Whatlinkshere sorts its output by page namespace, then page ID. This means that the oldest pages are listed first, a sort order which is unexpected and unfamiliar to wiki users, as it is not used anywhere else in the UI.

It should be possible to sort Special:Whatlinkshere by page title. In fact, this should probably be the default sort order, to match other "list"-type special pages.

Event Timeline

bzimport raised the priority of this task from to Low.Nov 21 2014, 8:34 PM
bzimport set Reference to bz2306.
bzimport added a subscriber: Unknown Object (MLST).

gangleri wrote:

*** Bug 3780 has been marked as a duplicate of this bug. ***

gangleri wrote:

Hallo!

Exchanged url because Betawiki has moved.

Bug 4588: Group pages by templates in list of file links on Image page

suggests that the code from "special:Whatlinkshere" should be reused in the file
links section in Image pages. It would be great to fix this before. Added Bug
4588 to the "blocks" list.

[[special:Watchlist/edit]] is not using the same code but there pages are
grouped by namespaces.

Filtering by namespace is possible at Special:Contributions, Special:Allpages
and Special:Recentchanges.
A common code is planed at
Bug 2468: The code that generates the namespace form for Special:Contributions,
Special:Allpages and Special:Recentchanges should be merged

See also: http://bugzilla.wikimedia.org/showdependencytree.cgi?id=2468

best regards reinhardt [[user:gangleri]]

robchur wrote:

With our current schema, it would be impossible to do this in any kind of
efficient manner. I'm also inclined to question the utility of such a function,
even if it were implemented.

yonatanh wrote:

*** Bug 9423 has been marked as a duplicate of this bug. ***

They are listed by page_id, which generally correlates with creation time, except for restorations. A namespace filter seems to have been added for some time.

spacebirdy wrote:

Reopened: it is possible to sort by namespace, but _not_ alphabetically, thus reopening bug again, since bug 3780 also links here and also other filters had been requested, thanks.

One visits [[Substance abuse]] and checking
[[Special:WhatLinksHere/Substance abuse]] one is convinced that [[Drug
abuse]] does not link to [[Substance abuse]]... it seems alphabetically
ordered, but that is just a coincidence for this page.

If MediaWiki insists on not making these lists alphabetically ordered,
then it should make it more obvious, so the user will remember to try
the 500 button. And while you are at it, make Category lists etc. etc.
also not alphabetically ordered, so the user doesn't get used to
thinking that all lists on MediaWiki are alphabetically ordered.

(In reply to comment #8)

then it should make it more obvious

E.g., have at the top of the list some mention of what the ordering is by,
as there is no way the user could guess. That way the user would not get so despondent that there is apparently no logic.

  • Bug 23763 has been marked as a duplicate of this bug. ***

If the database can't produce results in alphabetical order, pump the results into a sortable table and let JavaScript do the work.

Table sorting code is ten a penny. Here's some, MIT-licensed:
http://yoast.com/articles/sortable-table/

(In reply to comment #4)

I'm also inclined to question the utility of such a function,
even if it were implemented.

People work through lists in order while fixing things. It's very hard to know how far you are through a task when the list is in apparently* completely random order.

  • A normal user will have no idea what's going on inside the database or how it's ordering the results of this query.

(In reply to comment #11)

If the database can't produce results in alphabetical order, pump the results
into a sortable table and let JavaScript do the work.

Table sorting code is ten a penny. Here's some, MIT-licensed:
http://yoast.com/articles/sortable-table/

MediaWiki already has JavaScript table sorting. The issue is that paginated results will make the sorting useless, isn't it?

(In reply to comment #12)

MediaWiki already has JavaScript table sorting.

Ah. I had been under the misapprehension it was something template-based. Okay.

The issue is that paginated results will make the sorting useless, isn't it?

Well, in many cases it's possible to see all the results of the query by bumping up the limit parameter, at which point sorting becomes valuable.

It should be made clear that sorting applies only to the visible results.

"The following 50 pages link to Example.
Sorted in alphabetical order. [change sort direction]
(previous 50 | next 50) (20 | 50 | 100 | 250 | 500)"

Aklapper changed the subtype of this task from "Task" to "Feature Request".Feb 4 2022, 11:02 AM
Aklapper removed a subscriber: wikibugs-l-list.
TTO added a subscriber: TTO.

This was wish #23 in this year's Community Wishlist Survey. In Community-Tech 's prioritised list of wishes, it came in at #2.

In T4306#67806, robchur wrote:

With our current schema, it would be impossible to do this in any kind of
efficient manner.

Is this comment from 2007 still accurate?

TTO renamed this task from Special:Whatlinkshere lists ordered by creation timestamp - ordering and filtering should be possible to Allow sorting of Special:Whatlinkshere lists by page title.May 28 2022, 4:37 AM
TTO updated the task description. (Show Details)

Change 837213 had a related patch set uploaded (by Lens0021; author: Lens0021):

[mediawiki/core@master] (WIP) SpecialWhatLinksHere: Add support for sorting by title

https://gerrit.wikimedia.org/r/837213

Thanks a million for this @Lens0021.

(Yes I know that this comment is bugspam, but I've been watching this issue for literally a decade, so... yolo)

Any review or advice for my patch are welcome. :)

The patch needs a look from someone who knows about MediaWiki DB performance, as this query joins some very large tables.

Marostegui added subscribers: Ladsgroup, Marostegui.

@Ladsgroup can you take a look once you are back from vacation? Thanks.

@Ladsgroup can you take a look once you are back from vacation? Thanks.

Thanks. Now I'm around.

Currently, you can't really merge this patch at this state. The problem is not and never was the code, it's the infrastructure to support it. Ordering based on page title will make the condition of query on pagelinks table but the order on another table (page table) and you can have millions of rows in the condition and that practically means DDoS-vector as a service.

Some alternative solutions:

  • A service in toolforge, keep it in mind that will be really slow in some cases (you don't fix the problem, you just move the slowness from production to the cloud)
  • Building some derivative data tables, it'll take a while, possibly years considering that it's a lot of work for supporting something very narrow.
  • Possibly dependency engine might be useful, if not a graph db, maybe using a sharded noSQL might help in this case. But again, it's a lot of work for too little gain unless you can bundle this with some other benefits.

I believe the question was answered. I'm untagging performance, but feel free to reach out to us again if needed

I'm looking at this as part of the Community Wishlist for 2022. I was wondering about the following solution:

  1. query the Backlinks API (https://www.mediawiki.org/wiki/API:Backlinks)
  2. use the results to populate a sortable table (https://www.mediawiki.org/wiki/Help:Sorting#Sorting_rows_of_a_table)

Some foreseen issues:

  1. pagination: we'd likely remove the pagination in this view, this could cause issues for lower end devices
  2. multiple API calls: the Backlinks api has a maximum return of 500 links, but allows API pagination, so multiple calls could be used to get the complete list. This might mean the view takes a while to load, especially on slower connections

Suggested mitigation:

  1. display the current, unsorted, paginated list view by default
  2. add a "view as sortable table" button/link, which when clicked:
    1. shows a loading spinner while the API call collects the Backlinks, after load completes
    2. hide the old list view
    3. populate and show a sortable table with the return from the Backlinks
    4. show a link/button to "view a list" which hides the table and shows the original view

I'm eager to hear thoughts and feedback. Thanks!

Thank you for your prompt response. I read that and all other comments before posting my idea as an alternate solution.

The disadvantages I see from either off-loading to toolforge or using a dependency engine is, as you stated, it would take a lot of effort for relatively little gain, and would either be very Wikipedia specific or complicate the deployment of Mediawiki. Doing the work client-side mitigates the server load concern while being generalizable to any Mediawiki installation.

I'm sorry if I wasn't clear. To emphasize, we can't really do this ticket altogether without a massive work on the database. There is no database table to support it.

I see. But we already have the Backlinks API correct? What prevents us from populating an (unsorted) list from the API and sorting it clientside?

We have pages that are linked by millions of page, sometimes 100 million pages. So you have two options:

  • Load 100 million items by the API, then order them client side (if their systems don't OOM), then show the first 50 or 500 and then throw the rest away
    • That is not feasible
  • Load first 500 ordered by page_id and then order them based on page_title then show them to the user.
    • It's not really solving the problem users have here, it basically not ordering as you can expect another "A" in the next batch and it's confusing to users, it won't be deterministic to order and paginate.

Thank you for that context. How common is it that a page is linked to by more than, say 5000 pages? I'm thinking that maybe we could have the "view as sorted table button," but grey it out if the number of pages is greater than some threshold (with some hover-text that says there are too many results for a sorted table.) If this provided user value for most pages, that would be a win.

It sounds like a bad experience. I am waiting for someone who builds the new extension.

I see. But we already have the Backlinks API correct? What prevents us from populating an (unsorted) list from the API and sorting it clientside?

Doing this client-side is somewhat contrary to our software principles, but, a sorting feature is a relatively minor feature so we can probably accept that it isn't going to be as accessible, performant, or universal as reading, search and editing. That aside, client-side solutions tend to still have further UX expectations e.g. will the data on screen correspond to the permalink? Is paginating to "page 2" stable? Will the first render remain misordered while we serially make upto N API calls before reordering the list or showing an error if timed out/exceeded limit? FIle pages and global links? Wikidata Q-pages and P-pages?

I believe at our scale edge cases generally should be blocking implementation of core solutions that we have to support globally on our projects and in stable and localised versions of the released software. Having said that, I also agree that there is space here for something that brings the absolute benefit and value to power users that need only a simple best-effort as-is limited-to-N-items client-side solution that e.g. could initially only have been tested on popular English Wikipedia pages only and be expanded if and when issues and interested engineers align.

That space, however, is not in core and extensions; but in gadgets, user scripts, and browser extensions. We manage thousands of such gadgets in the community, including some that the Foundation helps maintain (CommTech team). I maintain a few of my own as well, including WhatLeavesHere which is somewhat similar to this. Important limitations are that these are opt-in and not expected to integrate with or meet expectations from the platform as a whole.

Thank you so much for your thoughts and guidance. I've been thinking about attempting this as a toolforge tool as @Ladsgroup suggested a while back. If it's performant/good enough, perhaps it can be linked as an "External Tool" on the What Links Here page, similar to "Link Count" and "Transclusion Count".

If doing more work on the client side is an acceptable approach for this tool, we could consider updating the UI of this Special Page to use Vue and Codex.

Since this feature exists on a dedicated special page, I think that loading Vue and Codex here is ok in terms of performance impact. And we could keep the existing server-rendered UI for clients that aren't running (modern) JavaScript.

The advantage of using Vue here is that users could benefit from a much more dynamic display of results – in addition to sorting / filtering abilities on the client, you could also improve the pagination experience where additional results are just loaded via AJAX requests without requiring a new page load.

What do folks think about developing a more interactive UI for this feature?

As a small update, we have made an external tool on tool forge https://wlh.toolforge.org/ and had it added to English wikipedia to the Special:WhatLinksHere pages as an external tool called "Sorted list". We'd like to iterate on it, and roll it out to other wikis soon.

Should the UI at https://wlh.toolforge.org/ be considered somewhat final (I mean, is the Special:WhatLinksHere going to look similarly)? I think that four-five records per screen on a laptop is way too low for this page to be useful. Is the empty line necessary? Can't the stats (edit count and last edit) be combined into a single line? Or, from another side, what's wrong with the current layout?

I believe that 10 links per page, with no option to cutomize it, will often render the special page useless. There are cases, like [[Wikipedia]], when there are dozens of thousands of pages. The list cannot be easily expanded then. The "View all" button causes only a "SBOX_FATAL_MEMORY_EXCEEDED" error in browser (ona 16GB RAM laptop).

This is how I see it (15.6" laptop):

image.png (1×1 px, 203 KB)

Should the UI at https://wlh.toolforge.org/ be considered somewhat final (I mean, is the Special:WhatLinksHere going to look similarly)? I think that four-five records per screen on a laptop is way too low for this page to be useful. Is the empty line necessary? Can't the stats (edit count and last edit) be combined into a single line? Or, from another side, what's wrong with the current layout?

I believe that 10 links per page, with no option to cutomize it, will often render the special page useless. There are cases, like [[Wikipedia]], when there are dozens of thousands of pages. The list cannot be easily expanded then. The "View all" button causes only a "SBOX_FATAL_MEMORY_EXCEEDED" error in browser (ona 16GB RAM laptop).

This is how I see it (15.6" laptop):

image.png (1×1 px, 203 KB)

The UI is definitely not final – the opposite right now I would say. Happy to consider tickets for specific features or design suggestions.

In terms of dealing with large numbers of results, there are a couple of things we could do. Having the entire dataset sent to the client is useful, because we could allow the user to search within the results or filter them and things would update near-immediately. But we should probably remove the "view all" button in that case (maybe replacing it with the ability to select how many links are shown at once – 10, 20, 100, etc).

Alternatively we could change the API to support pagination on the server, but then we'd also need to handle any "search within these results" feature the same way, and sorting/filtering on the client becomes less useful because you are only working with a partial set.

Change 837213 abandoned by Lens0021:

[mediawiki/core@master] (WIP) SpecialWhatLinksHere: Add support for sorting by title

Reason:

https://phabricator.wikimedia.org/T4306#8305441

https://gerrit.wikimedia.org/r/837213

And will this fix work for all wikis or only Wikipedia?

The tool can be generalised to any wiki. We would like to roll it out to other wikis in the foundation with their blessing. For wiki's outside the foundation, the tool can be self-hosted and added to any given wiki. I'd be happy to help with that.

Heads up, we are in another "wishathon week" here so I intend to spend some time improving the UI (by adding at least some client-side pagination and ideally filtering). Will share new improvements when they are ready.