Page MenuHomePhabricator

Upload doesn't apply mime type check
Closed, ResolvedPublic

Description

There is a bug in the file upload, which allow uploading wrong mime types to commons.

I have found the following images in the images.sql dump of commonswiki from 2011-12-30

  • [[:commons:File:20110420-RD-LSC-1074 - Flickr - USDAgov.jpg]] (video/mp4; already deleted)
  • [[:commons:File:Harvest.gif]] (video/3gpp; deletion request started)
  • [[:commons:File:Harvest.tiff]] (video/mp4; deletion request started)
  • [[:commons:File:Pétard.ogg]] (audio/wav; see old version)

Please have a look at that files and maybe found that bug and fix it. Thanks.


Version: 1.18.x
Severity: enhancement

Details

Reference
bz33549

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 11:59 PM
bzimport set Reference to bz33549.
bzimport added a subscriber: Unknown Object (MLST).

I know the first file was uploaded using the api (that's my bot). I assume the other files also used the api based on the UploadWizard category.

Maybe some mime type checking code is missing when using the api? This could be confirmed easily if you try to upload the same file using special:upload and see if you get a denied message.

These are UploadWizard and Bot uploads. So this probably indicates that the MIME verification on these uploads is currently not being applied.

Happening on all filetypes that use an incorrect file extension, also those uploaded with the old Special:Upload

https://commons.wikimedia.org/wiki/File:Mime_test_upload.tiff

This is not the first time this issue has surfaced. We probably need a regression test for this.

renaming bug back to original per comment 3 and my own testing.

Log of local upload:

MimeMagic::construct: loading mime types from /Users/hartman/Development/phase3/includes/mime.types
MimeMagic::
construct: loading mime info from /Users/hartman/Development/phase3/includes/mime.info
MimeMagic::doGuessMimeType: analyzing head and tail of /private/var/tmp/phpWLR4ey for magic numbers.
DjVuImage::getInfo: not a DjVu file
MimeMagic::guessMimeType: internal type detection failed for /private/var/tmp/phpWLR4ey (.)...
MimeMagic::detectMimeType: magic mime type of /private/var/tmp/phpWLR4ey: video/mp4
MimeMagic::guessMimeType: guessed mime type of /private/var/tmp/phpWLR4ey: video/mp4
MimeMagic::improveTypeFromExtension: improved mime type for .tiff: video/mp4
MediaHandler::getHandler: no handler found for video/mp4.
FSFile::getProps: /private/var/tmp/phpWLR4ey loaded, 230482 bytes, video/mp4.

mime: <video/mp4> extension: <tiff>

file extension not on blacklist
UploadBase::verifyExtension: no file extension known for mime type video/mp4, passing file
mime mismatch not detected
EXTMIME guessed:
IETYPES guessed: image/tiff, image/tiff, image/tiff, image/tiff, image/tiff
MIME type verified and confirmed
UploadBase::detectScript: checking for embedded scripts and HTML stuff
UploadBase::detectScript: no scripts found
ZipDirectoryReader: Fatal error: zip file lacks EOCDR signature. It probably isn't a zip file.
UploadBase::detectVirus: virus scanner disabled

So what happens is this:
We are able to detect (through finfo or mime_content_type of detectMimeType), that this is likely a video/mp4 file
.tiff is on the whitelist
.tiff is not on the blacklist
The guessed mime type is not checked to match a specific file extension (also would break uploads, if upload restrictions are very loose)

Verification:
mimeinfo and mime.types don't contain entries for video/mp4
In that case verifyExtension passes the file by default. (UploadBase::verifyExtension: no file extension known for mime type video/mp4, passing file)
No other bad aspects are found (IE/virus checks)

Solutions ?:

  • Change the default behavior for verifyExtension to reject files that we do not know in mime.info and/or mime.types (I remember this was a dubious case when we rewrote it in the past)
  • Add mp4 extensions and friends to mime.info mime.types (can't really hurt but also somewhat useless I think, since we have no internal handling of them at all right now).

It's always been possible to upload arbitrary files with certain extensions that don't have special MIME type verification. Unless there is some aspect of this that I'm missing, it's not a regression or a security vulnerability. Reducing priority/severity.

[[commons:File:Deamado ko.png.bmp]] is MIME type: image/x-bmp.

Looking at MediaWiki core's DefaultSettings.php and Wikimedia's CommonSettings.php, I can't figure out how this file type is allowed. Don't we strictly valid file extensions at least? I was able to reproduce an upload of this file type on Commons via [[commons:Special:Upload]] a few minutes ago by simply disabling JavaScript in my browser (the file selection input has some associated JavaScript validation logic).

(In reply to MZMcBride from comment #8)

Don't we strictly valid file extensions at least?

Err, validate. Referring to [[mw:Manual:$wgStrictFileExtensions]], I suppose.

(In reply to MZMcBride from comment #8)

[[commons:File:Deamado ko.png.bmp]] is MIME type: image/x-bmp.

Looking at MediaWiki core's DefaultSettings.php and Wikimedia's
CommonSettings.php, I can't figure out how this file type is allowed. Don't
we strictly valid file extensions at least? I was able to reproduce an
upload of this file type on Commons via [[commons:Special:Upload]] a few
minutes ago by simply disabling JavaScript in my browser (the file selection
input has some associated JavaScript validation logic).

Umm yeah, that shouldnt be allowed.

Also i thought i fixed this bug (the comment 0 bug, not most recent comment) a while back...

(In reply to Bawolff (Brian Wolff) from comment #10)

Also i thought i fixed this bug (the comment 0 bug, not most recent comment)
a while back...

I split this out to bug 62451. I almost initially filed comment 8 as a separate bug report, but I was concerned it would be marked as a duplicate of this bug. It seems this concern was unfounded.

Also i thought i fixed this bug (the comment 0 bug, not most recent comment)
a while back...

This is possibly a dupe of bug 39012, or other issues from the past (Hard to say). I'm duping this to bug 39012, if anyone can manage to upload files of wrong types currently (or have recently uploaded examples, please reopen or file a new bug).

  • This bug has been marked as a duplicate of bug 39012 ***
Gilles raised the priority of this task from Medium to Unbreak Now!.Dec 4 2014, 10:25 AM
Gilles moved this task from Untriaged to Done on the Multimedia board.
Gilles lowered the priority of this task from Unbreak Now! to Medium.Dec 4 2014, 11:22 AM