Example:
https://beta.wikiversity.org/wiki/USA:s_presidenter is deleted, but See https://commons.wikimedia.org/w/index.php?title=Special%3AGlobalUsage&limit=500&target=Harry-truman.jpg says that it is "IN USE".
Pleas purge the Database.
Thank you.
Version: wmf-deployment
Severity: major
Usually this is not an issue for other link tables, since we inner join on page, so its not noticeable.
As it stands, the only way to address the issue is to recreate the page in question, and then delete it.
We should probably
This was a example. All databases are affected. This is a problem and should be fixed.
A other exaple:
https://commons.wikimedia.org/w/index.php?title=Special%3AGlobalUsage&limit=50&target=Flag+of+Veneto.png
But the page on incubator.wikimedia.org is deleted.
A lotos of files are affected...
MariaDB [commonswiki_p]> SELECT * FROM globalimagelinks WHERE gil_page_title = "USA:s_presidenter" AND gil_wiki = "betawikiversity";
I see only on this page ~50 affected files...
Global are thousands of files affected
Please see https://www.mediawiki.org/wiki/Bugzilla/Fields#Severity
If this is obstructing work in a major way, it is more helpful to give an explanation of how that happens.
How long time do we need to wait @Tisza Gergő ...?
Restored prio, CC Erik. Major loss of function in an important area.
As I said, explaining how this is hindering your work or causing you problems is more helpful than playing around with severity flags (which does not cause bugs to magically fix themselves).
It is a problem in DeletionRequests if nonexisting usage is schown, it is a problem for the GlobalBot Commons Delinker (flooding the error log)...
etc etc. A experienced wikimedian schold know this.
Change 133351 had a related patch set uploaded by Aaron Schulz:
Added a links refresh mode to fix links to deleted pages
Change 133351 merged by jenkins-bot:
Added a links refresh mode to fix links to deleted pages
Will run this on Monday: foreachwiki extensions/GlobalUsage/refreshGlobalimagelinks.php --pages=nonexisting
tgr@terbium:~$ mwscript extensions/GlobalUsage/refreshGlobalimagelinks.php --wiki=testwiki --pages=nonexisting
Querying for broken links after (page_id) = (0)
Querying for broken links after (page_id) = (80125)
[e4d6d611] [no req] MWException from line 2206 of /srv/mediawiki/php-1.25wmf20/includes/db/Database.php: DatabaseBase::makeList: empty input for field page_id
Backtrace:
#0 /srv/mediawiki/php-1.25wmf20/includes/db/Database.php(1779): DatabaseBase->makeList(array, integer)
#1 /srv/mediawiki/php-1.25wmf20/includes/db/Database.php(1727): DatabaseBase->selectSQLText(string, string, array, string, array, array)
#2 /srv/mediawiki/php-1.25wmf20/extensions/GlobalUsage/refreshGlobalimagelinks.php(126): DatabaseBase->select(string, string, array, string)
#3 /srv/mediawiki/php-1.25wmf20/maintenance/doMaintenance.php(101): RefreshGlobalImageLinks->execute()
#4 /srv/mediawiki/php-1.25wmf20/extensions/GlobalUsage/refreshGlobalimagelinks.php(149): require_once(string)
#5 /srv/mediawiki/multiversion/MWScript.php(97): require_once(string)
#6 {main}Change 195479 had a related patch set uploaded (by Gergő Tisza):
Fix deleted page handling in update script
Change 195812 had a related patch set uploaded (by Gergő Tisza):
Fix deleted page handling in update script
Change 195813 had a related patch set uploaded (by Gergő Tisza):
Update GlobalUsage submodule
Finished running the script, but the output looked suspicious - it seems most of it went to stderr (which I stupidly did not log, only stdout). @Steinsplitter, can you verify if it worked correctly?
I dug up a bunch of deleted pages with images from enwiki AfD and those have correct globalusage lists. Not sure if they would have been affected by this bug at all, though - GlobalUsage had an article deletion hook from the start, so in theory no maintenance script would be needed. Is there a definite period in which that hook failed? Or does anyone have concrete reproduction steps?
Re-run on enwiki with proper logging:
Querying for broken links after (page_id) = (0)
A database query error has occurred.
Query: SELECT page_id FROM `page` WHERE page_id IN ('10','12','25','39','42',
'128','270','278','290','303','305','307','308','309','316','324','330','334',
...
...
...
'45657111','45657112','45657113','45657117','45657118','45657121','45657122',
'45657128','45657130','45657134','45657139','45657142','45657151','45657153',
'45657154','45657155','45657165','45657169','45657170','45657173','45657180',
'45657183','45657191')
Function: RefreshGlobalImageLinks::execute
Error: 2006 MySQL server has gone away (10.64.48.28)The problem is that for existing pages the script uses a hardcoded batch size of 500, for non-existing it uses mBatchSize but never calls setBatchSize().
Change 196501 had a related patch set uploaded (by Gergő Tisza):
Add setBatchSize() call for refreshGlobalimagelinks.php
Change 196501 merged by jenkins-bot:
Add setBatchSize() call for refreshGlobalimagelinks.php
It seems to have died halfway though. I'll put the fix for SWAT and re-run it on Monday.
Change 196991 had a related patch set uploaded (by Gergő Tisza):
Add setBatchSize() call for refreshGlobalimagelinks.php
Change 196992 had a related patch set uploaded (by Gergő Tisza):
Add setBatchSize() call for refreshGlobalimagelinks.php
Change 196991 merged by jenkins-bot:
Add setBatchSize() call for refreshGlobalimagelinks.php
Change 196992 merged by jenkins-bot:
Add setBatchSize() call for refreshGlobalimagelinks.php
Change 196993 had a related patch set uploaded (by Gergő Tisza):
Update GlobalUsage submodule
Change 196994 had a related patch set uploaded (by Gergő Tisza):
Update GlobalUsage submodule
The patch is now deployed on the cluster... Next item here is for it to be run again.
I don't have a good test case (apart from running the script on all wikis and observing that it does not give any error, which I did). All the examples of phantom pages on global usage lists that I knew of have been fixed by the first, partial run already. Steinsplitter said the same, so I think we can just assume that inconsistencies between the GU table and actual file usage have been fixed.
Whether there is a bug causing new inconsistencies is another matter. But at a glance the code seems good. I'll just close this and if someone sees new instances of the GU list being out of sync / finds reproduction steps, they can reopen it.