Page MenuHomePhabricator

Design new on-disk layout for MediaWiki install on tin/eqiad Apaches
Closed, ResolvedPublic

Description

One thing that will be different in from pmtpa to eqiad are file paths. This task is about making sure the configuration files and other places can deal with different paths in different data centers.


Version: unspecified
Severity: enhancement

Details

Reference
bz43340

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 1:02 AM
bzimport set Reference to bz43340.

First, let's list "the configuration files and other places".

As far as the MediaWiki config files go, "can deal with" is already done: I52a4b0ed added $wmfDatacenter which is filled in from /etc/wikimedia-site, which is being created by puppet as "pmtpa" currently and should be "eqiad" in the eqiad configuration. Anything needing different paths can switch based on this variable just like it does now with $wmfRealm (and formerly did with $cluster) if it needs something different for production vs labs. And I would hope the rest of MediaWiki goes by what's in the config files or uses paths relative to DIR.

There may be some maintenance scripts that hard-code paths rather than using relative paths or reading the config files; if nothing else, those too can be modified to read /etc/wikimedia-site and switch based on the value. They could even use MWRealm.php or MWRealm.sh out of operations/mediawiki-config, if those can be located.

And then there's Apache, Varnish, and so on that I would assume are dealt with directly by puppet. And I expect puppet already can easily deal with different paths in different configurations.

Anything else?

Sam and Tim will be working together on a layout on fenari, building out a symlink farm, and migrating us to the new layout. This will hopefully happen before the switchover.

[23:55:49] <TimStarling> Reedy: still awake?
[23:56:06] <Reedy> Yup
[23:56:26] <Reedy> Just had one of my internet connections die, just been trying to work out why
[23:59:57] <TimStarling> Reedy: so I see 366 references to /usr/local in the apache configuration
[00:01:04] <TimStarling> plus about 339 symlinks in mediawiki-config
[00:01:04] <Reedy> TimStarling: Is that the primary path being used?
[00:01:19] <TimStarling> that is the primary path in the apache configuration
[00:01:29] <TimStarling> in mediawiki-config I am including links to /apache since that is more common
[00:01:49] <Reedy> yay, consistency
[00:02:00] <TimStarling> not my fault
[00:02:09] <Reedy> wasn't blaming you
[00:02:19] <TimStarling> blame brion ;)
[00:02:22] <Reedy> heh
[00:02:35] <Reedy> Who do we have to <s>thank</s>blame for /h/w/c/p and stuff?
[00:02:47] <TimStarling> jeronim
[00:03:07] <TimStarling> do you know him?
[00:03:20] <TimStarling> I think /apache was either brion or lee
[00:03:20] <Reedy> don't think I do..
[00:03:36] <TimStarling> but /h/w was jeronim
[00:04:09] <TimStarling> he's australian, he did some sysadmin work for us in maybe 2004-2005
[00:05:39] <TimStarling> anyway, Ryan's proposed layout has various git-deploy managed repos in /srv/deployment
[00:07:31] <Ryan_Lane> if it's far too much of a pain in the ass, I can make the deployment system deploy to different destination locations
[00:07:37] <TimStarling> maybe we should try to avoid updating these 700 /usr/local/apache references more than once
[00:08:04] <TimStarling> we could change scap to push out to a new location, say /srv/old
[00:08:21] <TimStarling> then replace the old /usr/local/apache with a symlink farm
[00:08:28] * Ryan_Lane nods\x01
[00:08:58] <TimStarling> then once git-deploy is stable, update the apache configuration and document root symlinks so that we can remove /usr/local/apache
[00:23:09] <TimStarling> Reedy: mediawiki-config needs to be in a separate directory to the MW checkouts
[00:23:28] <TimStarling> let's say /srv/old/mediawiki-config
[00:23:43] <TimStarling> also, can we change the php- prefix now?
[00:24:06] <TimStarling> maybe just remove it? 1.21wmf6 seems like a good directory name to me
[00:24:40] <TimStarling> the php- prefix predates the name "mediawiki", it annoys me that it was kept through hetdeploy
[00:25:08] <Reedy> can't see why not
[00:25:09] <TimStarling> i.e. /srv/old/1.21wmf6
[00:26:45] <TimStarling> so the idea is to commit relevant changes to mediawiki-config, puppet, etc. to allow these directory structure changes
[00:27:34] <TimStarling> then swap the existing /usr/local/apache with a symlink farm going to /srv/old
[00:28:18] <TimStarling> before the swap, /usr/local/apache will have the old code in it, referring to the old directory structure
[00:28:59] <TimStarling> and people will have to be somehow temporarily prevented from doing a git pull on the old code location
[00:29:07] <TimStarling> maybe it will have to be in a separate branch of mediawiki-config
[00:29:17] <TimStarling> to make git-pull harmless