Page MenuHomePhabricator
Create Task
Maniphest T22741

Run cleanupTitles.php and cleanupImages.php on any wikis with unicode whitespace in page/file names
Closed, ResolvedPublic
Actions

Assigned To
None
Authored By
• bzimport
Sep 19 2009, 3:57 PM
Tags
Referenced Files
F6414: bad_titles.txt
Nov 21 2014, 10:57 PM
F6415: bad_images.txt
Nov 21 2014, 10:57 PM
F6411: results.txt
Nov 21 2014, 10:57 PM
F6413: imageresults.txt
Nov 21 2014, 10:57 PM
Subscribers
• brion
Catrope
• demon
Grondin
MarkAHershberger
• MZMcBride
vvv
View All 10 Subscribers

Description

Author: herd

Description:
Per r55382 (ref bug 15248 that introduced the change) there are now many pages and files inaccessable on some projects that contain these characters.

Example: http://toolserver.org/~nikola/grep.php?pattern=%E3%80%80&lang=ja&wiki=wikipedia&ns=6

http://ja.wikipedia.org/wiki/%E3%83%95%E3%82%A1%E3%82%A4%E3%83%AB:110%E3%80%80PICT0001.JPG
becomes
http://ja.wikipedia.org/wiki/%E3%83%95%E3%82%A1%E3%82%A4%E3%83%AB:110_PICT0001.JPG

Version: unspecified
Severity: critical
URL: http://svn.wikimedia.org/viewvc/mediawiki?view=rev&revision=55382

Details

Reference
bz20741

Event Timeline

• bzimport raised the priority of this task from to Medium.Nov 21 2014, 10:56 PM
• bzimport added projects: Wikimedia-Site-requests, Shell.
• bzimport set Reference to bz20741.
• bzimport created this task.Sep 19 2009, 3:57 PM
• bzimport added a comment.Sep 19 2009, 4:05 PM

herd wrote:

more examples, NBSP in en.wp titles (12 at this time):
http://toolserver.org/~nikola/grep.php?pattern=%C2%A0&lang=en&wiki=wikipedia&ns=0

• bzimport added a comment.Sep 20 2009, 8:42 AM

herd wrote:

*** Bug 20738 has been marked as a duplicate of this bug. ***

vvv added a comment.Sep 20 2009, 8:56 AM

Change to critical since it's related to several data made inaccessible.

• demon added a comment.Sep 20 2009, 2:09 PM
  • Bug 20746 has been marked as a duplicate of this bug. ***
• demon added a comment.Sep 20 2009, 2:10 PM

Upping to BLOCKER. Lots of pages on frwikinews are unavailable: http://toolserver.org/~nikola/grep.php?pattern=%C2%A0&lang=fr&wiki=wikinews&ns=0

Adding Brion as CC so he can delegate someone for this.

• bzimport added a comment.Sep 20 2009, 2:31 PM

herd wrote:

*** Bug 20747 has been marked as a duplicate of this bug. ***

Grondin added a comment.Sep 20 2009, 2:32 PM

See :
http://fr.wikinews.org/w/index.php?title=Tri-nations_2009%C2%A0:_les_Springbox_s%27imposent_en_Nouvelle-Z%C3%A9lande_et_remportent_le_tournoi&action=history

Mediawiki says : no history

But if you see that :
http://fr.wikinews.org/w/index.php?title=Tri-nations_2009%C2%A0:_les_Springbox_s%27imposent_en_Nouvelle-Z%C3%A9lande_et_remportent_le_tournoi&diff=next&oldid=153570

The versions really exist !

• bzimport added a comment.Sep 21 2009, 8:51 AM

Wiki.Melancholie wrote:

*** Bug 20703 has been marked as a duplicate of this bug. ***

• werdna added a comment.Sep 21 2009, 11:37 AM

The script is running in a screen on zwinger.

• werdna added a comment.Sep 21 2009, 9:02 PM

Done, invalid titles can be found with Special:PrefixIndex/Broken/

Grondin added a comment.Sep 21 2009, 10:17 PM

(In reply to comment #10)

Done, invalid titles can be found with Special:PrefixIndex/Broken/

No, invalid titles can'be be found with Special:PrefixIndex/Broken/

See fr-wikinews : five hundred page are still unavailable : 10 % of the project is canceled.

Histories are canceled, too ! It's not a bug, but a disaster.

Grondin added a comment.Sep 21 2009, 10:22 PM

This is the list of broken pages: http://toolserver.org/~nikola/grep.php?pattern=%C2%A0&lang=en&wiki=wikipedia&ns=0

How we'll do to restore them ?

• werdna added a comment.Sep 21 2009, 10:30 PM

Looks like the script is stopping after fixing one title. Looking...

• demon added a comment.Sep 22 2009, 12:49 AM

(In reply to comment #13)

Looks like the script is stopping after fixing one title. Looking...

Cf bug 17479. Maybe a problem with TableCleanup itself?

• brion added a comment.Sep 25 2009, 4:49 PM

Pending on fixes to cleanupTitles/namespaceDupes per Tim.

Zyephyrus added a comment.Sep 29 2009, 6:04 PM

On French wikisource this page [[Discussion:auteur:Charles Baudelaire]] has disappeared: [http://fr.wikisource.org/w/index.php?title=Discussion:Charles_Baudelaire&action=history the history gives this].

The cache in Google is http://209.85.229.132/search?q=cache:92baBBRt9lsJ:fr.wikisource.org/wiki/Discussion:Broken/Auteur%255Cx3aCharles_Baudelaire+wikisource+discussion+auteur+baudelaire&cd=1&hl=en&ct=clnk&client=firefox-a

When I ask for this: http://toolserver.org/~nikola/grep.php?pattern=Charles+Baudelaire&lang=fr&wiki=wikisource&ns=1

I get an answer

Auteur:Charles Baudelaire

but the link to Talk:Auteur:Charles Baudelaire is not good, I have this message:

Mauvais titre

Le titre de la page demandée est invalide, vide, ou il s’agit d’un titre inter-langue ou inter-projet mal lié. Il contient peut-être un ou plusieurs caractères qui ne peuvent pas être utilisés dans les titres.

(Bad title)

This page represented many hours of work, is it possible to have it back?

Zeph

Zyephyrus added a comment.Oct 15 2009, 1:01 AM

The same thing happened to another author talk page: Émile Zola but I was more lucky this time because I renamed the broken page like this:

15 octobre 2009 à 00:20 Zyephyrus (discuter | contributions | bloquer) m (5 265 octets) (Discussion:Broken/Auteur\x3a\xc3\x89mile Zola renommé en Discussion Auteur:Émile Zola) (révoquer | défaire)

Unfortunately I can't find again the previous broken page: Discussion auteur Charles Baudelaire. Is there some way to find it?

Zeph

Yann added a comment.Oct 23 2009, 6:32 PM

3 pages to restored: http://fr.wikisource.org/w/index.php?title=Sp%C3%A9cial%3AIndex&prefix=Auteur&namespace=1

Catrope added a comment.Dec 13 2009, 7:10 PM

I just ran cleanupTitles on frwikisource and it fixed nothing, and the linked list is empty. Please REOPEN if you still find issues.

Catrope added a comment.Dec 13 2009, 7:17 PM

As Natalie pointed out on IRC, this bug requests that cleanupTitles be run against all wikis, which I obviously didn't do yet. This should probably happen during office hours.

MarkAHershberger added a comment.Apr 4 2011, 10:31 PM

Pdhanda volunteered to handle this one.

• bzimport added a comment.Apr 12 2011, 6:34 PM

pdhanda wrote:

Dry run results from maintenance/cleanupTitles.php

This took a few hours to run. So I'll start the actual run early morning PDT on 04/14.

Attached:

results.txt172 KBDownload

• bzimport added a comment.Apr 12 2011, 9:21 PM

pdhanda wrote:

Dry run results from maintenance/cleanupImages.php

Attached:

imageresults.txt2 KBDownload

• bzimport added a comment.Apr 19 2011, 8:13 PM

pdhanda wrote:

Results after running maintenance/cleanupTitiles on all wikis in all.dblist

Attached:

bad_titles.txt159 KBDownload

• bzimport added a comment.Apr 19 2011, 8:14 PM

pdhanda wrote:

Results after running maintenance/cleanupImages on all wikis in all.dblist

Attached:

bad_images.txt2 KBDownload

• bzimport added a comment.Apr 19 2011, 8:14 PM

pdhanda wrote:

Done for all wikis in all.dblist

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL