| Legoktm | |
| Aug 8 2014, 1:30 AM |
| F14252: skins.common.access.tsv | |
| Nov 22 2014, 3:30 AM |
| F14251: skins.common.access.2014.09.16.tsv | |
| Nov 22 2014, 3:30 AM |
| F14250: raw[1] | |
| Nov 22 2014, 3:29 AM |
| F14249: august.1.2014.bits.skins.common.tsv | |
| Nov 22 2014, 3:29 AM |
Right now the skins/ directory in core has a folder named "common" in it, which is preventing people from cloning the mediawiki/skins meta-repo over it (like is done with extensions).
I don't really have an opinion on where to move it, but if we could not have it in skins/, that would be nice.
Version: 1.24rc
Severity: enhancement
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| OutputPage: Remove support for non-existent /w/skins/common directory | mediawiki/core | master | +10 -8 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Jdlrobson | T29292 Skinning system improvement (tracking) | |||
| Resolved | matmarex | T71277 Kill the skins/common/ directory in mediawiki/core, moving the contents somewhere else | |||
| Resolved | matmarex | T72412 LQT: [Regression 1.24wmf20] Pagination arrows have been moved | |||
| Resolved | matmarex | T72615 Fix extensions depending on skins/common/ | |||
| Resolved | matmarex | T71672 Use jquery.spinner module instead of skins/common/images/spinner.gif | |||
| Resolved | matmarex | T71673 Stop using inline <img> for thumbnail magnify icon | |||
| Resolved | matmarex | T71675 EnhancedChangesList::arrow(), sideArrow(), downArrow(), spacerArrow() are silly methods and should be removed | |||
| Resolved | matmarex | T71676 CleanChanges should provide its own arrow images | |||
| Resolved | matmarex | T71678 Track down popular gadgets that refer to files in skins/common/ and fix them not to |
Many of the requests for images @embed-ded in CSS are probably from IE 6 and should disappear after we move the style files (I have already submitted patches for that above).
images/magnify-clip.png 87619775
images/magnify-clip-rtl.png 1163918
Filed bug 69277 to track these. Gonna be fun.
IEFixes.js 153079
No longer exists after a68ce00d.
images/bullet.gif 28780
This is surprising, none of our code seems to refer to it. Needs investigating.
images/spinner.gif 11633
Some of these is ApiSandbox, see bug 69672.
(In reply to Bartosz Dziewoński from comment #12)
images/magnify-clip.png 87619775
images/magnify-clip-rtl.png 1163918Filed bug 69277 to track these. Gonna be fun.
bug 69673*
Change 154525 had a related patch set uploaded by Bartosz Dziewoński:
Remove some unused files from skins/common/
Interesting findings Bartosz, please send an email to Analytics-l when you want to do this again. Tools built on usage data that could be used for ongoing cleanup sound like a good idea.
Change 153368 merged by jenkins-bot:
Move installer files from skins/common/ to mw-config/
(In reply to Bartosz Dziewoński from comment #12)
images/bullet.gif 28780
This is surprising, none of our code seems to refer to it. Needs
investigating.
[[MediaWiki:Common.css]] uses this (and perhaps other wiki styles, too).
Change 155442 had a related patch set uploaded by Bartosz Dziewoński:
Revamp classic edit toolbar not to hardcode paths in HTML
Change 155631 had a related patch set uploaded by Bartosz Dziewoński:
Delete skins/common/images/button_template.png
Where are the images going to end up from Wikipedia's viewpoint? Many userscripts and gadgets also depend on it so I'd like a heads up when the location is known.
(In reply to Erwin Dokter from comment #22)
Where are the images going to end up from Wikipedia's viewpoint? Many
userscripts and gadgets also depend on it so I'd like a heads up when the
location is known.
Commons usually. Bartosz has been using mwgrep to check if the images are used in sitewide CSS/JS, and updating the urls of those that are.
Change 155771 had a related patch set uploaded by Bartosz Dziewoński:
Move feed.css to resources/
(In reply to Kunal Mehta (Legoktm) from comment #23)
Commons usually. Bartosz has been using mwgrep to check if the images are
used in sitewide CSS/JS, and updating the urls of those that are.
That is not what I ment. I can't rely on Common as it does not have all the skin images. Right now, I can call the images locally in /w/skins/common/images/. Where are they going to be moved to from the viewpoint of the local installation? Will it be /w/resources/common/images/? Or perhaps /w/resources/skins/common/images/? Or may we perhaps have symbolic links mapping to their new location?
Just a very simple question.
(In reply to Erwin Dokter from comment #25)
Where are they going to be moved to from the
viewpoint of the local installation? Will it be /w/resources/common/images/?
Or perhaps /w/resources/skins/common/images/?
Nowhere. It's irrelevant. The paths to files are *not* stable and should not be relied on. (The old ones in skins/common/ were an exception.)
So far I've been able to find all of the files I removed on Wikimedia Commons (https://commons.wikimedia.org/), under the same filenames too.
If there is anything that's used by on-wiki scripts and isn't there, it should be uploaded there (and I will upload it if I run into anything).
Or may we perhaps have
symbolic links mapping to their new location?
MediaWiki supports Windows hosts too, which don't support UNIX-style symbolic links, so we can't do that in the main repository. Perhaps you could convince ops to keep the old URLs valid in some way, but I'm not going to try convincing anyone it's a good idea :)
(In reply to Bartosz Dziewoński from comment #26)
Nowhere. It's irrelevant. The paths to files are *not* stable and should not
be relied on. (The old ones in skins/common/ were an exception.)
Commons files are no more stable; the ones without protection are vulnerable to vandalism. I do consider the local paths to be safer then Commons. So please... where are they going?
(In reply to Erwin Dokter from comment #27)
Commons files are no more stable; the ones without protection are vulnerable
to vandalism. I do consider the local paths to be safer then Commons. So
please... where are they going?
It depends on the image; I'm trying to move them somewhere that makes sense. Generally somewhere under resources/. Some, which were unused (both in extensions and on Wikimedia wikis, as far as I could tell), were also removed entirely.
I intend to make a list and add it to release notes; I was planning to do that later (would make it easier to have several independent patches pending review at the same time), but I guess I can do that now (later today).
(In reply to Erwin Dokter from comment #27)
Commons files are no more stable; the ones without protection are vulnerable
to vandalism. I do consider the local paths to be safer then Commons. So
please... where are they going?
If the gadget doesn't have to support IE 7 and the image is small enough, using data URIs is also an option.
Change 155910 had a related patch set uploaded by Bartosz Dziewoński:
TablePager: Load images via CSS backgrounds rather than HTML <img>s
Change 155914 had a related patch set uploaded by Bartosz Dziewoński:
TablePager: Redo arrow icons from scratch as CSS backgrounds
Change 156292 had a related patch set uploaded by Bartosz Dziewoński:
Delete feed.css
Change 155771 abandoned by Bartosz Dziewoński:
Move feed.css to resources/, introduce resources/src/assets/
Reason:
I'll revisit the 'assets' directory idea later.
Not overwriting this patch so that I can refer to the discussion from elsewhere.
Seems like a slight botched preparation of the mediawiki/skins project :(
Frankly I think it's a wasted effort here. Some files have been here nearly forever and I am not sure it is worth chasing links to those resources.
Cool URIs do not change :) - just provide 302 redirects instead, if you really need to move them.
(In reply to Marcin Cieślak from comment #35)
Seems like a slight botched preparation of the mediawiki/skins project :(
Indeed :( There are limits to what one person can do.
Frankly I think it's a wasted effort here. Some files have been here nearly
forever and I am not sure it is worth chasing links to those resources.
This bug, and the patches, are not really about chasing links (although I fix the important ones) but about just moving the files elsewhere. As you can see, it's taking surprisingly long, because, again, I'm the only person working on it and no one wants to review patches like that.
Cool URIs do not change :) - just provide 302 redirects instead, if you
really need to move them.
From comment 26:
Perhaps you
could convince ops to keep the old URLs valid in some way, but I'm not going
to try convincing anyone it's a good idea :)
(In reply to Bartosz Dziewoński from comment #36)
(In reply to Marcin Cieślak from comment #35)
Seems like a slight botched preparation of the mediawiki/skins project :(
Indeed :( There are limits to what one person can do.
Frankly I think it's a wasted effort here. Some files have been here nearly
forever and I am not sure it is worth chasing links to those resources.This bug, and the patches, are not really about chasing links (although I
fix the important ones) but about just moving the files elsewhere. As you
can see, it's taking surprisingly long, because, again, I'm the only person
working on it and no one wants to review patches like that.
That's why I think bug 69678 is not worth your time, really (but I'm not the one to judge it).
Cool URIs do not change :) - just provide 302 redirects instead, if you
really need to move them.From comment 26:
Perhaps you
could convince ops to keep the old URLs valid in some way, but I'm not going
to try convincing anyone it's a good idea :)
Personally I am fine with the symlink solution and I'd close all of it as WONTFIX.
Or maybe why not MediaWiki search for skins also in some other directory, say /w/skins-ext (or whatever the name) and leave /w/skins/common in peace.
What do you think?
Change 153370 merged by jenkins-bot:
Move mediawiki.legacy.* modules from skins/common/ to resources/
Change 155910 merged by jenkins-bot:
TablePager: Load images via CSS backgrounds rather than HTML <img>s
Change 155914 merged by Bartosz Dziewoński:
TablePager: Redo arrow icons from scratch as CSS backgrounds
Change 157490 had a related patch set uploaded by Bartosz Dziewoński:
Move mediawiki.skinning.* modules from skins/common/ to resources/
Change 158091 had a related patch set uploaded by Bartosz Dziewoński:
Delete closewindow19x19.png
Change 157490 merged by jenkins-bot:
Move mediawiki.skinning.* modules from skins/common/ to resources/
Change 158627 had a related patch set uploaded by Bartosz Dziewoński:
Article: Don't hardcode <img> tags on redirect page
Change 158648 had a related patch set uploaded by Bartosz Dziewoński:
Move file type icons to new assets/ directory
Change 158649 had a related patch set uploaded by Bartosz Dziewoński:
Move mediawiki.png to assets/ directory
Change 158650 had a related patch set uploaded by Bartosz Dziewoński:
Move remaining skin files to assets/ directory
And with this, I have patches pending or merged to make MediaWiki stop using skins/common/ entirely.
What remains is getting the remaining copies of files actually deleted, and probably fixing up some unavoidable bugs.
Patches relevant to this bug that are still pending:
Change 158627 merged by jenkins-bot:
Article: Don't hardcode <img> tags on redirect page
I put up a note on the release page: [[mw:MediaWiki 1.24#Directory changes]].
Haven't updated RELEASE-NOTES-1.24 yet.
Change 159290 had a related patch set uploaded by Bartosz Dziewoński:
Move default logo to assets/ directory
Change 155442 merged by jenkins-bot:
Revamp classic edit toolbar not to hardcode paths in HTML
Last remaining item in gerrit is https://gerrit.wikimedia.org/r/#/c/159290/ per https://gerrit.wikimedia.org/r/#/q/bug:69277,n,z – then "just" gadgets.
(In reply to James Forrester from comment #58)
Last remaining item in gerrit is https://gerrit.wikimedia.org/r/#/c/159290/
Yup, also remaining to do:
then "just" gadgets.
I've been trying to mwgrep as I was moving/deleting files and this seems to be a much lesser issue than I anticipated. Unless I missed something, the only files that still need action are bullet.gif and possibly wikibits.js.
It's been about a month, so I asked Dan to re-run the query from comment 8.
I wonder how the request counts have changed with the patches that have already been deployed. There are probably still many requests now, but hopefully when we re-run it again next month the numbers will hover around zero. :)
I ran the query again for September 16th, but I got some fairly detailed URLs that I'm not sure I should post. I'm checking with some people and I'll post if they think it's ok.
I'm posting the file. I just redacted all results that had <= 20 hits. There were some weird, overly-specific URLs in there. I left everything as it came back from my query: https://bugzilla.wikimedia.org/show_bug.cgi?id=69277#c8
Change 161770 had a related patch set uploaded by Legoktm:
Move default logo to resources/assets/ directory
Change 159290 merged by jenkins-bot:
Move default logo to resources/assets/ directory
Change 161770 merged by jenkins-bot:
Move default logo to resources/assets/ directory
(In reply to James Forrester from comment #67)
Still a few files left to go…
Link for convenience: http://git.wikimedia.org/tree/mediawiki%2Fcore.git/HEAD/skins%2Fcommon
Change 163168 had a related patch set uploaded by Bartosz Dziewoński:
Delete skins/common/
Change 163169 had a related patch set uploaded by Bartosz Dziewoński:
Add release notes for skins/common/ removal
Change 163171 had a related patch set uploaded by Bartosz Dziewoński:
Add release notes for skins/common/ removal
(In reply to Bartosz Dziewoński from comment #59)
Release notes listing what exactly was removed
Done for 1.24 and synchronised on master, see above.
Remove the stray files in skins/common/ that are waiting for caches to expire
Removed for 1.24, see above. Will do so on master, too, but only after finishing with bug 69678. I think this is the only thing remaining before we can finally close this bug.
Where will bullet.gif end up?
Please don't tell me "it depends" or anything like that... Monobook (as does Common.css) still needs this and I cannot trace where it has been/will be moved.
I absolutely need to know.
(In reply to Dan Andreescu from comment #62)
Created attachment 16507 [details]
hits to */skins/common/** on September 16th
(I did look at this, but it just so happened that the changes I was specifically interested in evaluating didn't go into production soon enough for this to contain data useful for me :( )
Attached:
Bartosz, let me know if you need another run at some point. If I don't answer just ping me on IRC (milimetric in #wikimedia-analytics).
Be sure to account for:
bits.wikimedia.org
{wiki domain}.org
paths:
static-{mwversion|"current"}/skins
e.g.
http://en.wikipedia.org/w/skins-1.5/
http://en.wikipedia.org/w/skins/
http://en.wikipedia.org/w/static-1.25wmf1/skins/
http://bits.wikimedia.org/skins-1.5/
http://bits.wikimedia.org/skins/
http://bits.wikimedia.org/static-current/skins/
wtf
/static-1.21wmf10/skins/common/images/magnify-clip.png 14926
/static-1.21wmf8/skins/common/images/magnify-clip.png 11160
All these links to ancient mediawiki dirs...
(In reply to Dan Andreescu from comment #83)
Created attachment 16942 [details]
hits to */skins/common/** on October 27th
Are you able to tell us where these requests are coming from? A lot of those are long long dead urls... I presume they're 404-ing
Attached:
Well, you can query the logs with SQL on our hadoop cluster if you'd like more info (you have rights to stat1002.eqiad.wmnet right Sam?) I mean, I could run queries for you but it sounds like you just want to investigate. For example, to get a breakdown by status, you could do:
sta1002> hive
hive (default)> use wmf_raw;
hive (wmf_raw)>
select uri_path,
http_status,
count(*)
from webrequest
where webrequest_source='bits'
and year=2014
and month=10
and day=28
and uri_path like '%skins/common/%'
group by uri_path,
http_statushaving count(*) > 20
;
You can see what else is available (like referrer) with:
hive (wmf_raw)> describe webrequest;
If you just want to poke around smaller bits of data, you can add something like "and hour=12". Without that, the query above could take over an hour.
reedy@tin:/srv/mediawiki-staging$ ssh -A stat1002
Permission denied (publickey).
reedy@tin:/srv/mediawiki-staging$
Nope :(
(In reply to Dan Andreescu from comment #86)
Well, you can query the logs with SQL on our hadoop cluster if you'd like
more info (you have rights to stat1002.eqiad.wmnet right Sam?) I mean, I
could run queries for you
I guess either Reedy needs to get access, or Dan could run those queries?
Sorry to not update the bug - RT ticket has been filed to get Sam access: https://rt.wikimedia.org/Ticket/Display.html?id=8773
I don't have access to see it, but I can follow up if it hasn't been resolved.
(In reply to Dan Andreescu from comment #83)
Created attachment 16942 [details]
hits to */skins/common/** on October 27th
Limiting this to data from 1.25wmfX and grouping, we get:
images/Checker-16x16.png 14826
images/magnify-clip.png 13622
images/bullet.gif 1407
images/closewindow19x19.png 700
shared.css 250
wikibits.js 152
images/spinner.gif 95
images/ajax-loader.gif 54
images/redirectltr.png 37
images/button_template.png 29
No idea what the deal with Checker-16x16.png is now, let's ignore it, it can't be a very important asset anywhere.
Some more of these files are already deleted and get neglibigle number of requests. Getting rid of that, we have these stats for files that still exist:
images/magnify-clip.png 13622
images/bullet.gif 1407
images/redirectltr.png 37
images/magnify-clip-rtl.png 0
images/redirectrtl.png 0
I'll spend some more time hunting down magnify-clip.png and bullet.gif, let's remove the rest already.
Attached:
Correction:
images/magnify-clip.png 13622
images/bullet.gif 1407
wikibits.js 152
images/redirectltr.png 37
images/magnify-clip-rtl.png 0
images/redirectrtl.png 0
ajax.js 0
Change 172256 had a related patch set uploaded by Bartosz Dziewoński:
Delete skins/common/{ajax.js, wikibits.js, images/{magnify-clip-rtl.png, redirectltr.png, redirectrtl.png}}
(In reply to Bartosz Dziewoński from comment #90)
No idea what the deal with Checker-16x16.png is now, let's ignore it, it
can't be a very important asset anywhere.
That is the backdrop for transparent images on file description pages, and often called from Common.css.
Limiting this to data from 1.25wmfX and grouping, we get...
Is this the entire WMF cluster, or just one wiki? If the first, the request can come from any project's Common.css.
(In reply to Erwin Dokter from comment #93)
(In reply to Bartosz Dziewoński from comment #90)
No idea what the deal with Checker-16x16.png is now, let's ignore it, it
can't be a very important asset anywhere.That is the backdrop for transparent images on file description pages, and
often called from Common.css.
I know what it is and where it used to be called. I only have no idea what the deal is with the thousands requests per day, as I went through the wikis a few weeks ago and replaced all of the references. Perhaps some bright soul decided to revert my changes somewhere.
Limiting this to data from 1.25wmfX and grouping, we get...
Is this the entire WMF cluster, or just one wiki? If the first, the request
can come from any project's Common.css.
The entire cluster. I am aware that the request can come from anywhere, and I've been tracking down and replacing these references in the past months.
This isn't really relevant to the release of 1.24.0 or the tarball in general as other bugs exist for dealing with it.
Change 173417 had a related patch set uploaded by Bartosz Dziewoński:
Delete skins/common/images/magnify-clip.png
Change 173434 had a related patch set uploaded by Bartosz Dziewoński:
Delete skins/common/images/bullet.gif
Change 172256 merged by jenkins-bot:
Delete skins/common/{ajax.js, wikibits.js, images/{magnify-clip-rtl.png, redirectltr.png, redirectrtl.png}}
And the directory is gone from master too! At last! Thank you, everyone who helped wit this boring cleanup work :D
(MW 1.24 will also be released without it.)
Rats. And I just sent out a note about not seeing any need to send out an RC this week. I'm sorry I missed this.
Bartosz, could you cherry pick these to REL1_24 so they'll be ready next week?
It has been done a long time ago, see comment 72 and comment 77 and https://gerrit.wikimedia.org/r/#/c/163168/ and such. We couldn't delete them on master at the time because of fear of breaking on-wiki customizations.
Change 434421 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] OutputPage: Remove support for non-existent /w/skins/common directory
Change 434421 merged by jenkins-bot:
[mediawiki/core@master] OutputPage: Remove support for non-existent /w/skins/common directory