Page MenuHomePhabricator

Add entries to robots.txt to block AFD/VFD/MFD
Closed, ResolvedPublic

Description

Author: en.ABCD

Description:
please add the following lines to http://en.wikipedia.org/robots.txt under
"User-agent: *"

Disallow: /wiki/Wikipedia:Articles_for_deletion/
Disallow: /wiki/Wikipedia%3AArticles_for_deletion/
Disallow: /wiki/Wikipedia:Votes_for_deletion/
Disallow: /wiki/Wikipedia%3AVotes_for_deletion/
Disallow: /wiki/Wikipedia:Pages_for_deletion/
Disallow: /wiki/Wikipedia%3APages_for_deletion/
Disallow: /wiki/Wikipedia:Miscellany_for_deletion/
Disallow: /wiki/Wikipedia%3AMiscellany_for_deletion/
Disallow: /wiki/Wikipedia:Miscellaneous_deletion/
Disallow: /wiki/Wikipedia%3AMiscellaneous_deletion/


Version: unspecified
Severity: normal
URL: http://fr.wikipedia.org/robots.txt

Details

Reference
bz4776

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 9:04 PM
bzimport set Reference to bz4776.
bzimport added a subscriber: Unknown Object (MLST).

Added to the master robots.txt.

le.korrigan wrote:

Would it be possible to do the same on fr.wikipedia ? The line to add would be

Disallow: /wiki/Wikip%C3%A9dia:Pages_%C3%A0_supprimer/

for "User-agent: *"

arnomane wrote:

Please add the following for de.wikipedia robots.txt under "User-agent: *" The disallow for talk
pages was voted in a large vote (see URL in comment) and would help very much with ongoing highly
problematic discussions about topics that are currently in the media and in court. I don't know
if the "talk"-equivalent is necessary. I added it as you can reach inside every local wiki the
namespaces via their english name and you can ignore it if it is not needed in order to make the
"noindex" perfect.

de

special pages

Disallow: /wiki/Spezial:*

sensible deletion and meta user discussion pages:

Disallow: /wiki/Wikipedia:Löschkandidaten*
Disallow: /wiki/Wikipedia:Vandalensperrung*
Disallow: /wiki/Wikipedia:Benutzersperrung*
Disallow: /wiki/Wikipedia:Vermittlungsausschuss*
Disallow: /wiki/Wikipedia:Administratoren/Probleme*
Disallow: /wiki/Wikipedia:Adminkandidaturen*

  1. all talk pages get excluded, as wanted by de.wikipedia community
  2. see:

http://de.wikipedia.org/wiki/Wikipedia:Meinungsbilder/Unterschriftensammlung_für_Nichtindizierung_von_Diskussionsseiten_durch_Suchmaschinen

english variant and german translation

Disallow: /wiki/Talk:*
Disallow: /wiki/Diskussion:*
Disallow: /wiki/User_Talk:*
Disallow: /wiki/Benutzer_Diskussion:*
Disallow: /wiki/Wikipedia_Talk:*
Disallow: /wiki/Wikipedia_Diskussion:*
Disallow: /wiki/MediaWiki_Talk:*
Disallow: /wiki/MediaWiki_Diskussion:*
Disallow: /wiki/Image_Talk:*
Disallow: /wiki/Bild_Diskussion:*
Disallow: /wiki/Template_Talk:*
Disallow: /wiki/Vorlage_Diskussion:*
Disallow: /wiki/Help_Talk:*
Disallow: /wiki/Hilfe_Diskussion:*
Disallow: /wiki/Category_Talk:*
Disallow: /wiki/Kategorie_Diskussion:*
Disallow: /wiki/Portal_Talk:*
Disallow: /wiki/Portal_Diskussion:*

Please add the following for zh.wikipedia robots.txt under "User-agent: *" The disallow for AFD and Copyvio
pages was voted.

zh

Disallow: /wiki/Wikipedia:删除投票/侵权
Disallow: /wiki/Wikipedia:%E5%88%A0%E9%99%A4%E6%8A%95%E7%A5%A8/%E4%BE%B5%E6%9D%83
Disallow: /wiki/Wikipedia:删除投票和请求
Disallow: /wiki/Wikipedia:删除投票和请求*
Disallow: /wiki/Wikipedia:%E5%88%A0%E9%99%A4%E6%8A%95%E7%A5%A8%E5%92%8C%E8%AF%B7%E6%B1%82
Disallow: /wiki/Wikipedia:%E5%88%A0%E9%99%A4%E6%8A%95%E7%A5%A8%E5%92%8C%E8%AF%B7%E6%B1%82*
Disallow: /wiki/Category:快速删除候选
Disallow: /wiki/Category:%E5%BF%AB%E9%80%9F%E5%88%A0%E9%99%A4%E5%80%99%E9%80%89
Disallow: /wiki/Category:维基百科需要翻译的文章
Disallow: /wiki/Category:%E7%BB%B4%E5%9F%BA%E7%99%BE%E7%A7%91%E9%9C%80%E8%A6%81%E7%BF%BB%E8%AF%91%E7%9A%84%E6%96%87%E7%AB%A0

robchur wrote:

Please don't add additional requests here, we won't see them - the bug is marked
fixed. Separate request, separate ticket...

  • Bug 3081 has been marked as a duplicate of this bug. ***

manolovarg wrote:

Please add the following for es.wikipedia robots.txt. We have had problems with google page rank that amplifies these nonencyclopedic pages
of WP and we have had usuary that have complained with us. Thanks

Disallow: /wiki/Usuario_Discusión:*
Disallow: /wiki/Usuario_Discusi%C3%B3n:*
Disallow: /wiki/Usuario:*
Disallow: /wiki/Wikipedia:Caf%C3%A9/Portal/Archivo/
Disallow: /wiki/Wikipedia:Caf%C3%A9/Portal/Archivo/Miscel%C3%A1nea/
Disallow: /wiki/Wikipedia:Café/Portal/Archivo/Miscelánea/

(In reply to comment #7)
This is now bug 6746.