Page MenuHomePhabricator

Broken thumbnails not being generated
Closed, ResolvedPublic

Description

Author: dbenbenn

Description:
See
http://commons.wikimedia.org/wiki/Category:Colorado_county_locator_maps
for example. At the moment, four of the images in the category
have broken thumbnails, though the full-size image displays fine.


Version: 1.6.x
Severity: major
OS: Windows XP
Platform: PC

Details

Reference
bz2888

Related Objects

Event Timeline

bzimport raised the priority of this task from to Unbreak Now!.Nov 21 2014, 8:40 PM
bzimport set Reference to bz2888.
bzimport added a subscriber: Unknown Object (MLST).

dbenbenn wrote:

The problem seems to have been fixed. I re-uploaded all of the images I
was aware of with broken thumbnails. The new thumbnails that were
generated are fine now.

dbenbenn wrote:

I'm reopening this bug. I've been seeing the same error a lot lately, and I'm
getting tired of re-uploading images.

It would be okay if MediaWiki failed to generate thumbnails when the servers
are overloaded. It could be smart about it: don't try to generate the same
thumbnail for a half hour, say. But at the moment, MediaWiki produces
incorrect thumbnails, and doesn't detect the error. The only way to get it to
re-generate the broken thumbnails is to re-upload the image, which is not a
good solution.

See, for example,

http://commons.wikimedia.org/wiki/Category:Colorado_wilderness_area_maps

At the moment, 3 of 9 entries in the category are just vertical lines.

robchur wrote:

Is this still an issue, given the recent upgrades and performance tweaks
regarding images and image servers, etc?

dbenbenn wrote:

I haven't seen the problem lately. I went ahead and set this to FIXED; I'll
reopen it if the problem pops up again.

dodgy wrote:

We seem to be suffering from this bug at the yoyowiki:
http://www.yoyowiki.org/index.php/Special:Newimages

It seems to be limited to .jpg files, or at least .gif files are unaffected. We
can post images in pages with a frame, just not with a thumbnail or you simply
end up with a vertical line. Help would be appreciated.

dodgy wrote:

We seem to be suffering from this bug at the yoyowiki:
http://www.yoyowiki.org/index.php/Special:Newimages

It seems to be limited to .jpg files, or at least .gif files are unaffected. We
can post images in pages with a frame, just not with a thumbnail or you simply
end up with a vertical line. Help would be appreciated.

dbenbenn wrote:

I'm pushing up the severity on this bug, since the problems have been happening
more and more lately. For example, the 94 pixel thumbnail of
[[Image:Schwingende Saiten.png]] is blank, and the 761px thumbnail of
[[Image:Map of Eagle County, Colorado.png]] is a green line, though the map
displays fine at [[Eagle County, Colorado]].

Presumably MediaWiki's installation of ImageMagick is broken.

Let me guess, they're very large images, especially PNGs or progressive JPEGs.
This hits the memory limit we've set to keep them from bringing down the servers.

dbenbenn wrote:

Well, there's no need to guess. You can see for yourself that the first example
is 38KB, and the second is 55KB. The problem isn't that MediaWiki can't
thumbnail these images at all. For example, the 95px thumbnail of
[[Image:Schwingende Saiten.png]], and the 760px thumbnail of
[[Image:Map of Eagle County, Colorado.png]], are fine. The problem seems to be
that from time to time, randomly, MediaWiki produces broken thumbnails. And the
only way to make MediaWiki throw away a broken thumbnail and try again is to
re-upload the image.

It's not the compressed size, it's the *uncompressed* size.

X pixels times Y pixels times Z color depth == lots of memory

Oh, and you don't have to reupload the image. Use ?action=purge on the image page.
(If it's a Commons image, do that *ON COMMONS*.)

dbenbenn wrote:

(In reply to comment #11)

It's not the compressed size, it's the *uncompressed* size.

X pixels times Y pixels times Z color depth == lots of memory

Well, the first example is 9.1 megapixels, and the second is 8.4 Mpixels. There
is, of course, the 12.5 Mpixel hard limit for PNG images, but neither of these
images is affected by that.

Thanks for the hint about action=purge. After using it about 10 times, I
managed to get it so that all of the relevant thumbnails of [[Image:Map of Eagle
County, Colorado.png]] were non-broken. (The relevant thumbnail sizes being the
ones selectable in the preferences.) Of course, that situation will only last
until someone else purges the thumbnails, and MediaWiki produces more broken ones.

So action=purge isn't much of a workaround. A much better solution would
involve first figuring out why MediaWiki is producing broken thumbnails.
Perhaps the convert command is getting killed before it finishes? If so, then
it should be possible to just delete broken thumbnails as they are produced, so
they don't hang around and cause trouble.

We set a runtime memory limit, as I already stated.

dbenbenn wrote:

Okay, perhaps that runtime memory limit is what is causing the broken thumbnails
to be generated. Any thoughts on how to fix the problem?

brianna.laugher wrote:

Is this what is causing so many SVGs, especially produced by Adobe Illustrator,
to have broken thumbnails?

http://commons.wikimedia.org/wiki/Commons:Help_desk#SVG_problem

Is there anything at all that can be done to avoid this problem?

There seems to be something more to it then just a memory limit. And the image
below is not a progressive JPG so thats not it either.

Have a look at:
http://commons.wikimedia.org/wiki/Image_talk:FordFiesta6.jpg
You may see that every thumb of Image:FordFiesta6.jpg is fine (including 179px)
except for the one that is 180px (which is also a default one).

I've tried purging few times and CTRL+R on the thumb but none if this works. I
thought it will get generated but a day have passed and the thumb is still
missing so it looks like something more permanent.

It looks like the thumb got broken on upload of the image and it just stays that
way.

jdujic wrote:

Same thing for http://commons.wikimedia.org/wiki/Image_talk:Serbia mountain
ranges.png. It appears on 201 and 199, but not on 200 px.

[[en:User:Duja]]

It seams like it's happening more often and happens not only on loading the
first version of an image. See:
http://commons.wikimedia.org/wiki/Image_talk:Bl%C3%BCcher.jpg
were you can find _previous_ version of the current (colored) image. I've
already tried purging and even reuploading but the thumbs remain black&white.

Could it be that this is related to creation of the Wikimedia logo mosaic (about
400 unique images now)?

There is a workaround for the issue which seems to work every time. Just use
link like this:
http://commons.wikimedia.org/w/thumb.php?action=purge&f=file_name.ext&w=250
where file_name.ext is file name with extension
and 250 is a thumb size to refresh

So I guess it WORKSFORME, but not sure if all would agree.

(In reply to comment #20)

There is a workaround for the issue which seems to work every time. Just use
link like this:
http://commons.wikimedia.org/w/thumb.php?action=purge&f=file_name.ext&w=250
where file_name.ext is file name with extension
and 250 is a thumb size to refresh

So I guess it WORKSFORME, but not sure if all would agree.

This doesn't seems to work:
http://commons.wikimedia.org/w/thumb.php?action=purge&f=Foucault-rotz.gif&w=300
will give you the answer

Although this PHP script (/w/thumb.php) exists, the file requested for output
(/mnt/upload3/wikipedia/commons/thumb/8/82/Foucault-rotz.gif/300px-Foucault-rotz.gif)
does not.

Changing 'rotz' with 'anim' will work. It is true that the convert uses more
memory and time with 'rotz' where the space is rotated than with 'anim'. But on
my machine, the command
<pre>
/usr/bin/convert -background white -size 300
/var/www/html/foo/w/images/8/82/Foucault-rotz.gif -coalesce -thumbnail 300x225!
-depth 8
/var/www/html/foo/w/images/thumb/8/82/Foucault-rotz.gif/300px-Foucault-rotz.gif
</pre>
works and needs 2 minutes to achieve. The size of the thumb
300px-Foucault-rotz.gif is 374364.

How can I upload it whith the already uploaded generic name Foucault-rotz.gif?

See [[wp:fr:Pendule de Foucault]] if you are interested in knowing why 2 graphs
are needed for better comprehension.

andy wrote:

I'm having this same problem with the images on the [[Maps of Japan]] page. In
addition some of the full size images are not available because of a server
cache error. See my post on the
[[Commons:Village_pump#Thumbnails_and_large_version_problem|Village pump]] for a
full description.

wikibug wrote:

Not sure whether I am reporting the same problem, but my Finnish friend Para says that it
is. Sometimes an SVG produces in some sizes an incorrect PNG and a correct one in other
sizes. The incorrect PNG may be an obsolete version (for example
http://commons.wikimedia.org/wiki/Image:BSicon_MWEXlfrfu.svg) but it can also be a
completely different picture which never existed as a previous version (for example
http://commons.wikimedia.org/wiki/Image:BSicon dSTRe.svg.
The problem cannot be solved with action=purge. Uploading the picture again does not solve
the problem either. Para suggested to use:
http://upload.wikimedia.org/wikipedia/commons/thumb/e/e7/BSicon_MWEXlfrfu.svg/500px-
BSicon_MWEXlfrfu.svg.pngsomearbitrarycharacters and this solves the problem, but (in this
case) for 500px only. Handige Harry ~~~~

I have also noticed that SVGs do not always rerender when you upload a new version. The problem is difficult to reproduce, but purging does not appear to fix the issue.

Bryan.TongMinh wrote:

If convert processes get killed, do they produce 0-byte files? If they do, the bad files should be removed by the thumbnailing code.

Closing this since we think it has been fixed (re bug triage meeting). Please re-open with an example if you think it isn't fixed.

Restricted Application added subscribers: Steinsplitter, Matanya. · View Herald Transcript