Page MenuHomePhabricator

[DO NOT USE] Wikis waiting to be renamed (tracking) [superseded by #Wiki-Setup (Rename)]
Closed, ResolvedPublic

Description

WARNING: 1. This project is replaced by the Rename column of Wiki-Setup, please add this tag and move to that column for new related tasks. 2. Discussions of technical blockers of wiki renaming process are continued at T172035. 3. Discussions of renaming database name are continued at T83609.

Tracking bug for wikis that have requests to be renamed/moved to other subdomains.

Also See:


I believe this requires a script to be written to automatically change "alot" of things for it to work.


Version: unspecified
Severity: enhancement
See Also:
T44396: duplicate/invalid language codes

Details

Reference
bz19986

Related Objects

Mentioned In
T356849: Featured articles in als:wp
T177471: Discussing #Wikimedia-Site-requests after #Wiki-Setup creation
Wiki-Setup
T173602: Revert "norsk bokmål" -> "norsk" change
T172035: Blockers for Wikimedia wiki domain renaming
T36217: Rename emlwiki -> eglwiki
T31186: Rename Võro Wikipedia, fiu-vro -> vro
T30443: Rename zh-classical -> lzh (invalid lang tag format)
T10217: Wikipedias with zh-* language codes waiting to be renamed (zh-min-nan -> nan, zh-yue -> yue, zh-classical -> lzh)
T17988: Rename 'roa-rup' wikis to 'rup'
T30442: Rename zh-min-nan -> nan
T28725: Renaming the Aramaic (arc) Wikipedia to the Syriac (syc) Wikipedia
T102509: Replace tracking bugs for wiki setup (create/rename/close) with project tags
T106687: Flow does not support varying language of parts of content based on user interface language (e.g. {{int:}})
T102510: Replace tracking bug T21986 by new project tag "wikis-to-rename"
T147164: Site identifier and domain prefix for nowiki should be changed
T83609: create script & docs to rename wiki databases
Wikimedia-Site-requests
T114772: Allow entering Wikidata sitelinks to wikis that have non-typical wiki ID (not matching the database name)
T112647: [Task] Investigation: how to handle the rename of a site id in Wikidata
T112285: ContentTranslation link adaptation is mostly broken when translating to be-tarask
T112426: [Bug] Querying Wikipedia for langlinks doesn't work for be-tarask, but works for be-x-old
T121259: Q3 Goal: help teams without dedicated CL support
T44396: duplicate/invalid language codes
T15: Migrate Bugzilla to Phabricator
T30441: Rename zh-yue -> yue
T99888: Link adaptation does not happen when the language code involved has a different domain name
T18976: [DO NOT USE] Wikis waiting for creation (tracking) [superseded by #Wiki-Setup (Create)]
T92565: Release/QA tasks at the Wikimedia Hackathon 2015
T90468: Sprint: Wikimedia Site Requests triage/cleanup/process
Mentioned Here
T83609: create script & docs to rename wiki databases
T172035: Blockers for Wikimedia wiki domain renaming
T102509: Replace tracking bugs for wiki setup (create/rename/close) with project tags
T30441: Rename zh-yue -> yue
T111853: The href of be-tarask: interlanguage link points to the be-x-old domain
T111876: Figure out how renaming languages without database names is supposed to work with SiteMatrix, and implement for be-x-old -> be-tarask
T134348: English names of azb: and be-x-old: in SiteMatrix are not in English
T112426: [Bug] Querying Wikipedia for langlinks doesn't work for be-tarask, but works for be-x-old
T102510: Replace tracking bug T21986 by new project tag "wikis-to-rename"
T44396: duplicate/invalid language codes
T18976: [DO NOT USE] Wikis waiting for creation (tracking) [superseded by #Wiki-Setup (Create)]

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

This ticket opened in 2009 and now it is 2016. Why does it take 7 years without anything done? While it is not urgent and not trivial, there is a date it must be done. Please raise its priority and start a project as soon as possible to streamline the renaming process. It affects the consistence of user interface and the alignment to language code standard.

This is a tracking task, which was inherited from previous bug tracking software - bugzilla. Tracking tasks are a list of bugs about specific stuff (in this case, renaming a wiki), not actual tasks to do. People are migrating such tracking tasks to dedicated projects iirc.

This ticket opened in 2009 and now it is 2016. Why does it take 7 years without anything done? While it is not urgent and not trivial, there is a date it must be done. Please raise its priority and start a project as soon as possible to streamline the renaming process. It affects the consistence of user interface and the alignment to language code standard.

Good question, and as far I know the answer is this:

For a long time everybody thought that it's too complicated technically. Then about a year ago somebody decided to take a chance, and things were actually done: be-x-old.wikipedia.org was renamed to be-tarask.wikipedia.org. It was mostly successful, but it uncovered several other complex issues, most notably T112426 (again, that's as far as I know, and there may be more). This is the tip of the iceberg of the blockers for further renaming work.

@daniel may have a better reply.

This is a tracking task, which was inherited from previous bug tracking software - bugzilla. Tracking tasks are a list of bugs about specific stuff (in this case, renaming a wiki), not actual tasks to do. People are migrating such tracking tasks to dedicated projects iirc.

Well, not many of the tracked tasks were done either, so @HenryLi's complaint is true.

Once technical blockers such as T112426 are resolved, some of the less controversial renaming actials can be done fairly quickly, such as "nrm -> nrf" and "bat-smg -> sgs". (ku -> kmr, for example, is controversial and not for technical reasons, and there's not much point in discussing as long as the technical work is blocked.)

So amire80, what's the reason we can't do T30441 as the second kiseki (or "miracle" if you really don't know romaji) here? The only problems after doing it, IMHO are:

The href of interwiki links of yuewiki ([[yue:xxx]]) and on SiteMatrix are still zh-yue.wikipedia.org/wiki/xxx, and to fix them, a sync fixing in operations/mediawiki-config must also be happen.

T111853, T111876

Querying this wiki, either on Wikidata or via DBA, or elsewhere, should still use zh_yuewiki, as database renaming may only possible in several decades later (maybe wait for 2050?)

as you said

Language names of that (if applicable) maybe autonym only, no longer having any translations, even in English.

T134348

But I should say that:
I don't be afraid of seeing a broken database, I'm only afraid, something is possible in several servers of the world but not in WMF servers

I am glad to hear some good progress. I read through all the related issues and come across with this document.

Rename a wiki domain

It is almost done and some little glitches come out. Do those blocking issues have someone to follow and work on? They are some batch processing and configuration issues.

It is better to complete as soon as possible. If it keeps unfinished, there would be more and more issues as new components are coming. ( Like the issues on the introduction of Wikidata.)

Please raise its priority and start a project as soon as possible to streamline the renaming process.

Well, give me a very fairy reason before that, as changing priority is actually, actually, and actually, a strong influential action.

As someone without developer access, I gather from the discussion so far (over 7 years) that we're solving four different problems of various importance:

  1. Change the domain name (mission-critical)
  2. Change the Wikidata interwiki prefix so language links can be added using correct language codes (important where applicable)
  3. Change the "traditional" MediaWiki interwiki prefix (not so important because Wikidata has made that mostly obsolete) [Edit: it will be crucial to have both old and new interwiki prefixes as aliases, though switching the default to ISO 639 prefixes is less important]
  4. Change the database name (not important at all because it's largely invisible except to developers)

I feel that we've been fussing about (4) for years but actually we want to solve (1) and (2)...

"3. Change the "traditional" MediaWiki interwiki prefix (not so important because Wikidata has made that mostly obsolete)"

That's wrong, we very frequently need interwiki links within articles, or in many discussions, translation requests when we cannot link to an article on the local wiki.

It is not just for linking a local article to other versions (not really translations) for the same topic. Note that many articles may be splitted with subtopics in one wiki but not in another, so interwiki links from Wikdiata will not match exactly (and there's not always an exact matching anchor in the target article in another language). Wikidata only works for 1:1 relations between topics, not for 1:N or N:1. The organization of articles do not follow the same schemes between wikis that have different conventions, even if there's a common base of topics that are easily identifiable by a name.

Project pages on each wiki rarely connect well across wikis because each wikiproject has different teams working on them. Still we need to cross references "similar" or "related project pages acrtoss wikis, and cite many external links to other wikis.

As well, user pages (and their talk pages) are not found in Wikidata, but users want to link severtal accounts not always unified with SUL across wikis. We need interwiki prefixes to specify the target wiki or to inform other people to take contact on a specific wiki.

But in fact adding the support for new interwiki prefixes (or to alias them so tat a new one is prefered) requires no big database maintenance: it's very basic configuration (simpler than the domain name on Wikimedia DNS servers, as it requires also updating certificates for HTTPS). It can be the first thing to do

Then there are some essential tools to check or reconfigure (e.g. the Universal Language Selector, the BCP47 code to use for HTML conformance, which is generated in HTML lang="" attributes), some modules to update (list of supported languages, directionality).

For MediaWiki itself, there's the need to alias its linguisitic resources (MediaWiki translation on translate.net must be informed, existing translations for Wikimedia must be temporarily blocked, then renamed or copied there with the new code, existing translators should still be able to translate for the new code.

There are several PHP config files to update, and all other wikis should be informed with a technical notifice. On each wiki there will be work to check in templates or Lua modules; as the transitionj will take time (months, but most probably years), this requires keeping aliases for the older code so that they continue working. Some wikis will require admins to check codes used within pagenames or in protected pages, so that pages can be renamed.

Renaming the internal database is largely invisible except for those that download snapshots to create mirrors, if they expect a specific filename matching the wiki code.

In summary, the simplest thing to update is:

  • adding an interwiki and aliasing the former one, and checking that "#language:" correctly resolves both codes.
  • makign sure that Wikidata will accept the new code
  • updating the certificates for HTTPS so they support two domains (the former one and the new one) and installing them.

Then only we can:

  • add support for a new subdomain and aliasing (CNAME) the former one.
  • update the map of virtual hosts on webservers. Check the security issues which may exist when the same site will be available under two equivalent domain names simultaneously.
  • reconfigure the front proxies and make sure they correctly resolve domain names to an appropriate webserver and that its virtual hosts will also accept connections with the new domain. These two steps may require putting a domain offline to run a test of connectivity, making sure that new certificates are also delivered.

Then update Wikidata with some bot (Wikidata should already only contain BCP47 codes, not old non conforming codes, but it uses non-conforming codes explicitly for the list of Wikipedia or Wiktionnary pages)

Ideally, before aliasing the old code, the new code should first be created as an alias, and then the aliasing reversed later: this will help updating and testing various utilities, templates and modules when they generate links. Some bots may incorrectly revert changes in pages/templates/modules by detecting errors if we don't use the canonical names: if we first create the new domain and alias the old one, these bots may start doing massive jobs, but it will interfere with other tests to perform first. It will be easier to detect these bots and block them/inform their maintainers, that the new code is valid and still being tested.. When we'll reverse the aliasing, there could be tons of reports in logs about usage of non-canonical codes: some tuneup may be necessary to filter these false alarms: cleanup of wikis is the last thing to do and should first start manually.

@Verdy_p I agree with you.

  • adding an interwiki and aliasing the former one, and checking that "#language:" correctly resolves both codes.
  • makign sure that Wikidata will accept the new code

If we can do these two items first, that'll already solve half the problem as our users know it (points 2 and 3 of my previous comment)

As someone without developer access, I gather from the discussion so far (over 7 years) that we're solving four different problems of various importance:

  1. Change the domain name (mission-critical)
  2. Change the Wikidata interwiki prefix so language links can be added using correct language codes (important where applicable)
  3. Change the "traditional" MediaWiki interwiki prefix (not so important because Wikidata has made that mostly obsolete) [Edit: it will be crucial to have both old and new interwiki prefixes as aliases, though switching the default to ISO 639 prefixes is less important]
  4. Change the database name (not important at all because it's largely invisible except to developers)

I feel that we've been fussing about (4) for years but actually we want to solve (1) and (2)...

This is mostly correct and is pretty much what I was saying last year, except for, as @Verdy_p points out, #3 is important.

But in fact adding the support for new interwiki prefixes (or to alias them so tat a new one is prefered) requires no big database maintenance: it's very basic configuration (simpler than the domain name on Wikimedia DNS servers, as it requires also updating certificates for HTTPS). It can be the first thing to do

It'd be very basic configuration, except we have a script that generates them and it handles this wrong. Also, we generally don't need to update HTTPS certificates because they use wildcards and we only change subdomains.

For MediaWiki itself, there's the need to alias its linguisitic resources (MediaWiki translation on translate.net must be informed, existing translations for Wikimedia must be temporarily blocked, then renamed or copied there with the new code, existing translators should still be able to translate for the new code.

I believe renaming of MediaWiki languages is considered a separate (and solved?) issue, certainly not something I'm worried about when it comes to domain names. We already have proper separation between language codes and subdomains (e.g. meta, commons, mediawiki, etc. etc.)

Renaming the internal database is largely invisible except for those that download snapshots to create mirrors, if they expect a specific filename matching the wiki code.

Also except labs database replicas.

  • updating the certificates for HTTPS so they support two domains (the former one and the new one) and installing them.

As I wrote above, not usually.

  • add support for a new subdomain and aliasing (CNAME) the former one.

I think we want an HTTP redirect, not DNS CNAMEs.

  • reconfigure the front proxies and make sure they correctly resolve domain names to an appropriate webserver and that its virtual hosts will also accept connections with the new domain. These two steps may require putting a domain offline to run a test of connectivity, making sure that new certificates are also delivered.

Nothing needs to be done here. If we ignore (labtest)wikitech (which have different servers and don't sit behind varnish anyway) for now, all wikis are served by the same set of backend MW servers.

  1. Change the "traditional" MediaWiki interwiki prefix (not so important because Wikidata has made that mostly obsolete) [Edit: it will be crucial to have both old and new interwiki prefixes as aliases, though switching the default to ISO 639 prefixes is less important]

It's still incorrect even you added these notations, see tasks that I added you to follow

Krinkle changed the status of subtask T30442: Rename zh-min-nan -> nan from Open to Stalled.
Krinkle changed the status of subtask T17988: Rename 'roa-rup' wikis to 'rup' from Open to Stalled.
Krinkle changed the status of subtask T30441: Rename zh-yue -> yue from Open to Stalled.
Krinkle changed the status of subtask T30443: Rename zh-classical -> lzh (invalid lang tag format) from Open to Stalled.
Krinkle changed the status of subtask T31186: Rename Võro Wikipedia, fiu-vro -> vro from Open to Stalled.
Krinkle changed the status of subtask T36217: Rename emlwiki -> eglwiki from Open to Stalled.
Krinkle removed subtasks: T147164: Site identifier and domain prefix for nowiki should be changed, T127680: Rename Serbo-Croatian Wikipedia and Wiktionary from sh.wiki* to hbs.wiki*, T127679: Rename sh.wikipedia to hbs.wikipedia, T124657: Rename cbk-zamwiki to cbkwiki, T110190: Rename "simple" wikis to "en-simple", T84537: Changing address of Võro Vikipeediä, T71670: Change domain chapcom.wikimedia.org to affcom.wikimedia.org, T64717: Move oldwikisource on www.wikisource.org to mul.wikisource.org, T41968: Bhojpuri wikipedia should start with 'bho' instead of 'bh' to avoid confusion with Bihari, T36866: Change wgLanguageCode of several wikis to be renamed, T41482: Rename "chapcomwiki" to "affcomwiki", T36217: Rename emlwiki -> eglwiki, T31186: Rename Võro Wikipedia, fiu-vro -> vro, T40763: Our *.wikimedia.org cert doesn't properly cover https://pa.us.wikimedia.org/, T27522: language code change for Samogitian: "bat-smg" to "sgs", T32376: Rename ku.wikipedia to kmr.wikipedia, T25537: Migration of pt.wikimedia.org, T30441: Rename zh-yue -> yue, T30442: Rename zh-min-nan -> nan, T30443: Rename zh-classical -> lzh (invalid lang tag format), T28725: Renaming the Aramaic (arc) Wikipedia to the Syriac (syc) Wikipedia, T25215: Rename the als.*.org projects to gsw.*.org, T25217: Move the Moldovan Wikipedia, T25216: Move the Nourmande Wikipedia from nrm to nrf, T31919: Move the wiki of WMEE, T33335: Rename wikis with multiple subdomains, T17988: Rename 'roa-rup' wikis to 'rup', T19592: move nds to nds-de, newly create nds locale, T10540: Rename nds projects to nds-de, T17357: Redirect dk.wiktionary and dk.wikibooks to da.wiktionary and da.wikibooks respectively., T10217: Wikipedias with zh-* language codes waiting to be renamed (zh-min-nan -> nan, zh-yue -> yue, zh-classical -> lzh), T19047: Change no.wikipedia.org to nb.wikipedia.org, T6793: Rename Alemannic Wikipedia from als.wikipedia to gsw.wikipedia, T11823: Rename "be-x-old" to "be-tarask".
Krinkle added a subscriber: Krinkle.

Per T102509, closed in favour of Wiki-Setup (Rename)

@Krinkle Thanks. But creating a subproject on Wikimedia-Site-requests meant that now subscriptors are broken. I mean, parent projects cannot have direct membership, but they inherit them from their subprojects and milestones, as appears in the MediaWiki docs. That is, now people can not be a member of Wikimedia-Site-requests anymore but if they subscribe to Wiki-Setup, which they might not be interested in. That's why I proposed at T102509#3537380 a separate independent project.

This has to be fixed. Wikimedia-Site-requests was probably the most popular project here and this is a breaking change. Maybe @mmodell can do some CLI magic and "desubproject" Wiki-Setup from Wikimedia-Site-requests, making both of them again independent projects so people can subscribe to whatever they want?

@MarcoAurelio I don't think it's possible to move a subproject once it's been migrated, at least not without directly screwing with the SQL. I will nonetheless see what I can do.

Liuxinyu970226 renamed this task from Wikis waiting to be renamed (tracking) to [DO NOT USE] Wikis waiting to be renamed (tracking) [superseded by #Wiki-Setup (Rename)].Aug 24 2017, 1:25 PM
Liuxinyu970226 updated the task description. (Show Details)

@MarcoAurelio What about a second subproject specifically for people to subscribe to? Essentially parent projects are only useful as "folders' to group together multiple subprojects into a hierarchy and there could be a "general" subproject that is used as a catch-all for things that don't fit into other specific subprojects. That would be easier to implement than writing the script to undo subproject migrations.

@mmodell I think we should discuss that in Z398. Can you raise the issue there, please? Thank you.