Page MenuHomePhabricator

Opening Special:EditWatchlist with a large watchlist hits server timeout (Create watchlist pager)
Open, LowPublic

Description

From T220245#5105063 (April 2019) where @kostajh wrote:

Plan as discussed with @Catrope and @SBisson:

  1. Implement a pager that respects the groupings displayed on EditWatchlist. Max number of items rendered per page would be 500.
  2. Add a namespace filter at the top of the page which defaults to "(all)"
Original 2012 report by @Nemo_bis

A user on translatewiki.net: "And I cannot now edit my watchlist due to its huge volume. The server does not allow me to do that (HTTP error 500)". From the log:
I don't know if he tried Special:EditWatchlist or Special:EditWatchlist/raw, he also mentioned having 15.000 items in watchlist but that may be an hyperbole (works for me with only 3000 watchlisted pages).

Examples of (possibly?) caused errors in the logs:

-rakkaus:#mediawiki-i18n- [20-Aug-2012 14:52:34] PHP Fatal error: Allowed memory size of 204472320 bytes exhausted (tried to allocate 32 bytes) in /www/w/includes/db/Database.php on line 1746

-rakkaus:#mediawiki-i18n- [20-Aug-2012 14:53:06] PHP Fatal error: Allowed memory size of 204472320 bytes exhausted (tried to allocate 32 bytes) in /www/w/includes/db/Database.php on line 1733

-rakkaus:#mediawiki-i18n- [20-Aug-2012 14:56:14] PHP Fatal error: Allowed memory size of 204472320 bytes exhausted (tried to allocate 83 bytes) in /www/w/includes/db/DatabaseMysql.php on line 210


See Also:

SeverityUsers with large watchlists (anything that requires more than 60 seconds for Special:EditWatchlist to load, anecdotally seems to be 10k items and above) experience request timeout errors, and are unable to use Special:EditWatchlist
Frequency648 instances in last 90 days
Level of effort to fixMedium/High . There's a patch that needs to be rebased, and substantially reworked for the namespace/title ordering proposed by Platform Engineering
Consequences if we don't fixEditors with large watchlists are frustrated that key functionality (Special:EditWatchlist) is not usable.

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Krinkle subscribed.

Plan as discussed with @Catrope and @SBisson:

  1. Implement a pager that respects the groupings displayed on EditWatchlist. Max number of items rendered per page would be 500.
  2. Add a namespace filter at the top of the page which defaults to "(all)"

The solution proposed in the patch for this task is to implement an alphabetic pager for Special:EditWatchlist. The pager will respect the namespace groupings currently on EditWatchlist, but the maximum number of items rendered per page will be 500 (as opposed to the unlimited number currently). Also, the TOC at the top of EditWatchlist is replaced with a namespace filter, which defaults to (all).

@eprodromou can you please weigh in on this comment before I rework the patch?

kostajh changed the task status from Open to Stalled.Oct 15 2019, 12:35 PM
kostajh removed kostajh as the assignee of this task.

Marking as stalled pending T41510#5506021, and moving out of Growth-Team's current sprint as we are working on other things at the moment. I've also unassigned myself, if someone else wants to pick this up please do, otherwise I will come back to it later this year.

Krinkle edited projects, added Platform Engineering; removed Patch-For-Review.

Re-triaging in CPT inbox per the below question:

@eprodromou can you please weigh in on this comment before I rework the patch?

@eprodromou can you please weigh in on this comment before I rework the patch?

Sorry I missed this. I'll review tomorrow and get a response.

OK, I had a talk with Brad about this. I think from the CPT side, it's best to order the pages by namespace then title, because our database indexes are set up to do that query efficiently, and the page won't work very well without the indexes.

@kostajh if that works for you, let me know. Otherwise, I'll see if we can be helpful in other ways.

kostajh changed the task status from Stalled to Open.Nov 19 2019, 8:03 PM

sounds good, thanks @eprodromou

CCicalese_WMF subscribed.

Let us know if you need any future review from CPT.

Krinkle renamed this task from Editing large watchlists via Special:EditWatchlist fails due to a server timeout to Editing large watchlists via Special:EditWatchlist fails due to server timeout (Create watchlist pager).Jan 11 2020, 6:58 PM
Krinkle renamed this task from Editing large watchlists via Special:EditWatchlist fails due to server timeout (Create watchlist pager) to Opening Special:EditWatchlist with a large watchlist hits server timeout (Create watchlist pager).Jan 15 2020, 3:11 AM

If this bug is persistent, can we add a suggestion line "Consider maintaining your watchlist at less than 5k pages" or a warning line "Your watchlist exceeded 5k pages and Edit button might not work" somewhere? I mean, being honest is appreciated.

In T41510#5859986, @Ata wrote:

If this bug is persistent, can we add a suggestion line "Consider maintaining your watchlist at less than 5k pages" or a warning line "Your watchlist exceeded 5k pages and Edit button might not work" somewhere? I mean, being honest is appreciated.

T222533: Add message to Special:Watchlist /raw and /clear to indicate slower speed

@Krinkle do I understand correctly that T222533 suggests a message to be shown to users who try emptying/editing their watchlists, notifying them that it might take a while?
My suggestion is to show something that will prevent users from adding too many pages to their watchlists in the first place, before they face T41510 bug. Let users know about the limitations in place.

I've been working on the patch for this again in limited spare time, and hope to have it up for review again soon.

Large watchlist - I have been unable to edit in any way - have tried many times and ways

[XnsCjwpAIHwAA8yh7zsAAAAP] 2020-03-25 07:05:47: Fatal exception of type "WMFTimeoutException"

a simple courtesy of indicating that if watchlists go over a certain size that they are not possible to manage, it would really be very helpful..

Large watchlist - I have been unable to edit in any way - have tried many times and ways

[XnsCjwpAIHwAA8yh7zsAAAAP] 2020-03-25 07:05:47: Fatal exception of type "WMFTimeoutException"

a simple courtesy of indicating that if watchlists go over a certain size that they are not possible to manage, it would really be very helpful..

Maybe that could be squashed into T222533: Add message to Special:Watchlist /raw and /clear to indicate slower speed

I just hit (what I think is) this issue with a ~30k page watchlist on enWS. The error message now (9 years after first report) looks like:

[fa6c503d-725f-4305-9a63-86a809510979] 2021-04-29 18:06:52: Fatal exception of type "Wikimedia\RequestTimeout\RequestTimeoutException"

And just for reference, a 30kp watchlist isn't particularly busy on Special:Watchlist, so it was pretty random circumstance that led me to even go try to prune it.

But since the actual fix for this is taking a long time to get done, and since we clearly already have the total count for the watchlist available (as it's displayed at the top of Special:Watchlist), perhaps we could at least make the failure mode a little nicer? How about picking a number (5k? 10k? 15k?) after which the "Edit watchlist" button shows you a message that tells you this won't work and refer to this task? And maybe at around 5k the text on Special:Watchlist turns red and has a popup warning you that you are about to hit the size limit? Not a fix in any sense, of course, but at least we won't be throwing what is essentially an uncaught exception at end users.

kostajh lowered the priority of this task from Medium to Low.Sep 14 2021, 9:02 AM
kostajh moved this task from Upcoming Work to Triaged on the Growth-Team board.

I'm changing the priority to Low, not because I don't think this is important, but because it's an indicator to Release-Engineering-Team that if this error is spotted during train deployments, it's not a cause for undue alarm.

This is also something unlikely for the Growth team to be able to prioritize. I'll move this task back into Platform's inbox in case they have capacity to look at this in an upcoming quarter.

15K, 10K, 32K, 30K, I say peanuts.

I have 128,281 pages on my Commons (production) watchlist, leave a message if you want me to test if you change something. (a ping here I'm more likely to miss)

As this does not seem to be solved in the near future it might be useful to add a link to Special:EditWatchlist/raw to the GUI of the watchlist page.

For a warning message to be timely and not-too-intrusive it would need to be at Special:Preferences#mw-prefsection-watchlist "Watched pages". The six checkboxes there provide for automatic additions to the watchlist. But a real solution would be much better, such as the possibility of downloading a complete watchlist in editable form and allowing uploading of a properly formatted list, possibly only to an empty watchlist.

Change 505784 abandoned by Kosta Harlan:

[mediawiki/core@master] (wip) Introduce alphabetic pager for Special:EditWatchlist

Reason:

https://gerrit.wikimedia.org/r/505784