Marking 1.19 until I find out I'm wrong.

Created attachment 10117
Screenshot

<zscout370> that is all the versions that are at Flag of Kazakhstan.svg (I was

trying to fix the image since someone uploaded so many images in
about 3 minutes, with many reverts) so I was trying to fix that
and that was how I encountered the bug.

<zscout370> now

http://upload.wikimedia.org/wikipedia/commons/d/d3/Flag_of_Kazakhstan.svg
is showing as 404, but the image is still there?

<hexmode> you mean the thumbs are there?
<zscout370> yes
<zscout370> it looks like an image is there, thumbs are there, but when you

click on it it goes 404

<zscout370> and if you need it, I was on Firefox 10.0.2, Win7 Ultimate

Attached:

zscout370 wrote:

Comment on attachment 10117
Screenshot

The screenshot can be seen at http://www.flickr.com/photos/zscout370/6789632196/

zscout370 wrote:

The file was uploaded several times by a user today, and while in the process of fixing the image, I encountered that error. It will not let me delete the old versions and deleting the top version will only delete the physical page (but not the files themselves). The image also shows as 404 but all thumbnails and data are being generated in the file history.

Dumping the DB sha1 and the actual sha1 via script, I see:
string(31) "hmag9lxxuekjy8xsj3e133qmkbpryfb"
string(31) "hmag9lxxuekjy8xsj3e133qmkbpryfb"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"

string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"
string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "6lo5nwy70mn4g5w1er8ys8syfi2hk67"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"

string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"
string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"

Notice the two pairs in the middle that due not match. This is a side-effect of bug 17057.

Even if we don't solve bug 17057, there should be a better failure mode. This looks pretty rare, though, so setting milestone appropriately.

• Bug 34959 has been marked as a duplicate of this bug. ***

Aaron and I discussed this, and nearly marked this as a duplicate of bug 17057. However, Aaron doesn't feel comfortable closing bug 17057 until we actually know that new images with the wrong SHA1 are no longer being created. So, we're reappropriating this bug to be "clean everything up to this point", which basically will involve writing and running a cleanup script. After that script is run, this bug will be closed, but bug 17057 will remain open.

Using the 1.19wmf1 deployment milestone, since this isn't waiting for 1.20 in any meaningful way.

*** Bug 35656 has been marked as a duplicate of this bug. ***

zscout370 wrote:

I am now seeing this again in another file: http://commons.wikimedia.org/w/index.php?title=Special:Undelete&action=submit

"Undelete failed; someone else may have undeleted the page first.

Undeletion will not be performed if it will result in the top page or file revision being partially deleted. In such cases, you must uncheck or unhide the newest deleted revision.

Error undeleting file: A non-identical file already exists at mwstore://local-backend/local-public/8/85/Flag_of_Belarus.svg."

When trying to upload: "Unexpected non-MediaWiki exception encountered, of type "SyntaxException" exception 'SyntaxException' with message 'Object name not set.' in /usr/local/apache/common-local/php-1.19/extensions/SwiftCloudFiles/php-cloudfiles-1.7.10/cloudfiles.php:1767 Stack trace: #0 /usr/local/apache/common-local/wmf-config/swift.php(48): CF_Container->delete_object('') #1 [internal function]: wmfPurgeBackendThumbCache(Object(LocalFile), false) #2 /usr/local/apache/common-local/php-1.19/includes/Hooks.php(216): call_user_func_array('wmfPurgeBackend...', Array) #3 /usr/local/apache/common-local/php-1.19/includes/GlobalFunctions.php(3823): Hooks::run('LocalFilePurgeT...', Array) #4 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(732): wfRunHooks('LocalFilePurgeT...', Array) #5 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(972): LocalFile->purgeThumbnails() #6 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(914): LocalFile->recordUpload2('20120410003419!...', 'Updated flag ht...', false, Array, false, Object(User)) #7 /usr/local/apache/common-local/php-1.19/includes/upload/UploadBase.php(573): LocalFile->upload('/tmp/phpb0HOKQ', 'Updated flag ht...', false, 1, Array, false, Object(User)) #8 /usr/local/apache/common-local/php-1.19/includes/specials/SpecialUpload.php(439): UploadBase->performUpload('Updated flag ht...', false, true, Object(User)) #9 /usr/local/apache/common-local/php-1.19/includes/specials/SpecialUpload.php(174): SpecialUpload->processUpload() #10 /usr/local/apache/common-local/php-1.19/includes/SpecialPageFactory.php(476): SpecialUpload->execute(NULL) #11 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(263): SpecialPageFactory::executePath(Object(Title), Object(RequestContext)) #12 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(593): MediaWiki->performRequest() #13 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(503): MediaWiki->main() #14 /usr/local/apache/common-local/php-1.19/index.php(58): MediaWiki->run() #15 /usr/local/apache/common-local/live-1.5/index.php(3): require('/usr/local/apac...') #16 {main} #0 /usr/local/apache/common-local/wmf-config/swift.php(48): CF_Container->delete_object('') #1 [internal function]: wmfPurgeBackendThumbCache(Object(LocalFile), false) #2 /usr/local/apache/common-local/php-1.19/includes/Hooks.php(216): call_user_func_array('wmfPurgeBackend...', Array) #3 /usr/local/apache/common-local/php-1.19/includes/GlobalFunctions.php(3823): Hooks::run('LocalFilePurgeT...', Array) #4 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(732): wfRunHooks('LocalFilePurgeT...', Array) #5 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(972): LocalFile->purgeThumbnails() #6 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(914): LocalFile->recordUpload2('20120410003419!...', 'Updated flag ht...', false, Array, false, Object(User)) #7 /usr/local/apache/common-local/php-1.19/includes/upload/UploadBase.php(573): LocalFile->upload('/tmp/phpb0HOKQ', 'Updated flag ht...', false, 1, Array, false, Object(User)) #8 /usr/local/apache/common-local/php-1.19/includes/specials/SpecialUpload.php(439): UploadBase->performUpload('Updated flag ht...', false, true, Object(User)) #9 /usr/local/apache/common-local/php-1.19/includes/specials/SpecialUpload.php(174): SpecialUpload->processUpload() #10 /usr/local/apache/common-local/php-1.19/includes/SpecialPageFactory.php(476): SpecialUpload->execute(NULL) #11 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(263): SpecialPageFactory::executePath(Object(Title), Object(RequestContext)) #12 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(593): MediaWiki->performRequest() #13 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(503): MediaWiki->main() #14 /usr/local/apache/common-local/php-1.19/index.php(58): MediaWiki->run() #15 /usr/local/apache/common-local/live-1.5/index.php(3): require('/usr/local/apac...') #16 {main}"

(In reply to comment #10)

[[commons:File:Flag of Belarus.svg]] now disappeared, the file description page stating “No file by this name exists, but you can upload it (even though there still is https://upload.wikimedia.org/wikipedia/commons/8/85/Flag_of_Belarus.svg). If a file used to exist, try to purge this page's cache”

And trying to purge the page leads to:

Unexpected non-MediaWiki exception encountered, of type "SyntaxException"
exception 'SyntaxException' with message 'Object name not set.' in /usr/local/apache/common-local/php-1.19/extensions/SwiftCloudFiles/php-cloudfiles-1.7.10/cloudfiles.php:1767
Stack trace:
#0 /usr/local/apache/common-local/wmf-config/swift.php(48): CF_Container->delete_object('')
#1 [internal function]: wmfPurgeBackendThumbCache(Object(LocalFile), false)
#2 /usr/local/apache/common-local/php-1.19/includes/Hooks.php(216): call_user_func_array('wmfPurgeBackend...', Array)
#3 /usr/local/apache/common-local/php-1.19/includes/GlobalFunctions.php(3823): Hooks::run('LocalFilePurgeT...', Array)
#4 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(732): wfRunHooks('LocalFilePurgeT...', Array)
#5 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(681): LocalFile->purgeThumbnails(Array)
#6 /usr/local/apache/common-local/php-1.19/includes/WikiFilePage.php(164): LocalFile->purgeCache(Array)
#7 [internal function]: WikiFilePage->doPurge()
#8 /usr/local/apache/common-local/php-1.19/includes/Article.php(1786): call_user_func_array(Array, Array)
#9 [internal function]: Article->__call('doPurge', Array)
#10 /usr/local/apache/common-local/php-1.19/includes/actions/PurgeAction.php(51): ImagePage->doPurge()
#11 /usr/local/apache/common-local/php-1.19/includes/actions/PurgeAction.php(69): PurgeAction->onSubmit(Array)
#12 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(484): PurgeAction->show()
#13 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(278): MediaWiki->performAction(Object(ImagePage))
#14 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(593): MediaWiki->performRequest()
#15 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(503): MediaWiki->main()
#16 /usr/local/apache/common-local/php-1.19/index.php(58): MediaWiki->run()
#17 /usr/local/apache/common-local/live-1.5/index.php(3): require('/usr/local/apac...')
#18 {main}

#0 /usr/local/apache/common-local/wmf-config/swift.php(48): CF_Container->delete_object('')
#1 [internal function]: wmfPurgeBackendThumbCache(Object(LocalFile), false)
#2 /usr/local/apache/common-local/php-1.19/includes/Hooks.php(216): call_user_func_array('wmfPurgeBackend...', Array)
#3 /usr/local/apache/common-local/php-1.19/includes/GlobalFunctions.php(3823): Hooks::run('LocalFilePurgeT...', Array)
#4 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(732): wfRunHooks('LocalFilePurgeT...', Array)
#5 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(681): LocalFile->purgeThumbnails(Array)
#6 /usr/local/apache/common-local/php-1.19/includes/WikiFilePage.php(164): LocalFile->purgeCache(Array)
#7 [internal function]: WikiFilePage->doPurge()
#8 /usr/local/apache/common-local/php-1.19/includes/Article.php(1786): call_user_func_array(Array, Array)
#9 [internal function]: Article->__call('doPurge', Array)
#10 /usr/local/apache/common-local/php-1.19/includes/actions/PurgeAction.php(51): ImagePage->doPurge()
#11 /usr/local/apache/common-local/php-1.19/includes/actions/PurgeAction.php(69): PurgeAction->onSubmit(Array)
#12 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(484): PurgeAction->show()
#13 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(278): MediaWiki->performAction(Object(ImagePage))
#14 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(593): MediaWiki->performRequest()
#15 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(503): MediaWiki->main()
#16 /usr/local/apache/common-local/php-1.19/index.php(58): MediaWiki->run()
#17 /usr/local/apache/common-local/live-1.5/index.php(3): require('/usr/local/apac...')
#18 {main}

I would say this is quite a severe problem, given e.g. [[Special:WhatLinksHere/File:Flag_of_Belarus.svg]]

I've changed swift.php to skip "" files in the listings. I have no idea why the swift listings have empty files.

Moving this down in priority only because Aaron is working on what he believes is the root cause (bug 36132), and I want to move this down below that on his list of stuff to do. With any luck, this will automatically get fixed as a result of that.

Another example that might be related:
http://commons.wikimedia.org/wiki/File:Terracotta_footman_seiza2.jpg

(removing 1.19wmf1 deployment blocker, 1.19wmf1 is already deployed)
(moving from 1.19.0 release to 1.19.x, 1.19.0 has been released)

Is this a MediaWiki bug or something to be fixed in Wikimedia databases?

I think it is in MediaWiki, but just asking to be sure.

So this would be in the form of a maintenance script? Or part of update.php? (both?)

This bug is for running a script.

I've starting run this on commons and a few large wikis. It may take a few weeks to finish.

(In reply to comment #17)

I've starting run this on commons and a few large wikis. It may take a few
weeks to finish.

Finally finished on commons after the second attempt (other wikis done already).