Taking this bug.

See also bug 16739 for similar request.

Gerrit change 31580

(In reply to comment #2)

Gerrit change #31580

this does not seems right - the change talks about 250x250 for gallery size default. the reques is for 200x200 for gallery (the 250 for thumb width is correct).

peace.

Oh, thank you to have spotted that.

I fixed the gallery values to 200x200.

Cant we use the same value for both default thumb sizes and thumb size in gallery? That will avoid the cluster having to generate a thumb in two size and save a bit of processing power.

(In reply to comment #5)

Cant we use the same value for both default thumb sizes and thumb size in
gallery? That will avoid the cluster having to generate a thumb in two size and
save a bit of processing power.

galleries and thumbs are two different things, and they do not use the same size by default: on a default media wiki installation, default thumb width is 180, while on the mediawiki wikis, the default for thumb is 220. for gallery, the media wiki default is 120x120, and wikimedia wikis do not change it.

so we do not use the same size anywhere, and it's hard to see why we would need to do this on hewiki, just because hewiki asks to use different defaults.

currently, many (most?) of the thumbs and galleries on hewiki define image sizes explicitly, because most editors were not happy with the defaults. adjusting the defaults will allow us to stop doing that, but the effect on the image servers should be nill, b/c either explicit or default, we would not use same size for gallery and thumbs.

peace.

any updates on this request?

was it denied? accepted? is there ETA?

it seems there was one code upgrade since the Gerrit change #31580 was submitted, but the defaults in hewiki seems to be unchanged.

peace.

any updates on this request?

was it denied? accepted? is there ETA?

we are now on 1.21wmf4, but the defaults in hewiki is still unchanged.

peace.

Configuration deployment is different than MediaWiki deployment.

We're still in a situation where Hashar has made a comment on your bug, not mitigated by other advices. I sent November 23 a mail to wikitech to gather more opinion and know if we've a way to measure server load impact -> no reply.

Could you tell us your feeling from your side: is this request urgent or could it wait we find a proper time to discuss it more in detail?

Wikitech-l thread : http://lists.wikimedia.org/pipermail/wikitech-l/2012-November/064631.html

I have sent an email to the ops team to find out if this change can be merged without killing imagescalers / swift (I am maybe too paranoid).

sumanah wrote:

kipod, as Dereckson asked, is this request urgent?

Antoine, any update from Ops?

Question (or statement?) from Tim Starling asked on Gerrit:

I don't understand why different wikis need different thumbnail sizes. It's not like they use different kinds of images.

As to urgency:
This was a consensus of the community on hewiki. personally, i do not feel that strongly about it, but some (majority?) in the community do.

what happens ATM is that practically all[1] thumbnails in hewiki are placed with explicit size of 250px. we would like very much to stop doing that, but the community will not cease unless the default would change.

as to the effect on the image servers: since we hardcode 250px anyway, i do not see what difference does it make.

As to Tim sterling's question: it's not essential that Tim understand the answer "why". i do not understand the answer to this question myself, but the fact is that different communities seem to prefer different "default" thumbnail size. specifically, i think svwiki moved to 250 some years ago, no? all we ask is the same for hewiki, and while you are at it, also change default gallery size to 200x200.

this is not personal - i serve here as an emissary for hewiki community wish, as expressed by the consensus (in the meantime, this was archived: here is a link to the archived discussion: https://he.wikipedia.org/wiki/%D7%95%D7%99%D7%A7%D7%99%D7%A4%D7%93%D7%99%D7%94:%D7%9E%D7%96%D7%A0%D7%95%D7%9F/%D7%90%D7%A8%D7%9B%D7%99%D7%95%D7%9F_302#.D7.9C.D7.A4.D7.A0.D7.99_.D7.91.D7.A7.D7.A9.D7.94_.D7.91.D7.91.D7.92.D7.96.D7.99.D7.9C.D7.94
)

peace.

[1] - not really "all", but i estimate > 90%.

Sorry for not having followed up on that issue. Here is a summary I have posted on Gerrit change #31580 which is based off a discussion we had in a restricted mailing list:

It is not reasonable for us to handle different thumbnail sizes per wiki for the following reasons:

• we keep thumbnails forever currently, the more we have the more disk space it takes
• different sizes lower the cache hit rate which in turns cause...
• ... a CPU cost on the cluster to generate a thumbnail, varying the sizes cause more and more thumbnails generations
• whenever a file is updated, we have to purge each thumbnails ever generated

The consensus among the engineering team is to standardize the thumbnails to a few well known formats and have the scaling being done by the web browser. We could let the mobile clients scale up small thumbnails (to save bandwidth) and desktop scale down larger thumbnails (to get nicer pictures).

So I am sorry to say it, but that changes will not land on the live cluster.

the conclusion from the above should really be a removal of "per user" thumbnail size from preferences: as it is now, where every user can define a different default thumbsize for themselves, any thumbnail where size is not defined explicitly, will cause exactly the effect you cite as the reason to reject hewiki request (multiple thumb sizes maintained on servers).

on the other hand, changing the default for hewiki will have almost no effect: in the vast majority of cases where hewiki is using the same image as some other wiki, we do define size of 250 anyway, so the servers are *forced* to maintain the "250" size of the thumb. making it the hewiki default will allow us to remove the explicit setting of size, but will not cause creation of yet-another-thumb-on-the-server: this thumb is already there due to the explicit setting in hewiki...

oh well....

peace.

Related request by frwiki was bug 27839

@Gilles Would you have an opinion on this task, so we can have a coherent response for T113877 and this one?

Since 250px is much more common than 220px (as per Swift figures recently compiled in P4577, CountPerSize section)? 200 is even more common. This change should actually improve performance.

Change 31580 had a related patch set uploaded (by Reedy):
he.wiki images size configuration

https://gerrit.wikimedia.org/r/31580

@Gilles @hashar Do we have some metrics capabilities to see how the servers load evolve when we deploy https://gerrit.wikimedia.org/r/31580 change?

https://grafana.wikimedia.org/dashboard/db/media and https://grafana.wikimedia.org/dashboard/file/swift.json for storage/cache hit changes

And https://grafana.wikimedia.org/dashboard/db/prometheus-machine-stats for machine load. That one requires looking machine per machine, and unfortunately image scalers don't have recognizable names, you have to look at the list of servers assigned to image scaling in puppet to know which ones to look at.

In T43712#3024196, @Gilles wrote:

And https://grafana.wikimedia.org/dashboard/db/prometheus-machine-stats for machine load. That one requires looking machine per machine, and unfortunately image scalers don't have recognizable names, you have to look at the list of servers assigned to image scaling in puppet to know which ones to look at.

$ grep -B 1 "mediawiki::imagescaler" manifests/site.pp
node /^mw129[3-8]\.eqiad\.wmnet$/ {
    role(mediawiki::imagescaler)
--
node /^mw208[6-9]\.codfw\.wmnet$/ {
    role(mediawiki::imagescaler)
--
node /^mw21(4[89]|5[01])\.codfw\.wmnet$/ {
    role(mediawiki::imagescaler)

So e.g. https://ganglia.wikimedia.org/latest/graph_all_periods.php?title=Network+output+of+all+imagescalers&vl=&x=&n=&hreg%5B%5D=(mw129%5B3-8%5D%7Cmw208%5B6-9%5D%7Cmw21(4%5B89%5D%7C5%5B01%5D))&mreg%5B%5D=bytes_in&gtype=stack&glegend=show&aggregate=1 ?

No idea how to (quickly) make such a link in grafana.

Change 31580 abandoned by Dereckson:
Configure he.wiki images size

Reason:
Still not consensual, even if approved by performance.

https://gerrit.wikimedia.org/r/31580