Page MenuHomePhabricator

Add talk pages of deletion discussions to robots.txt
Closed, ResolvedPublic

Description

While subpages of [[Wikipedia:Articles for deletion]] are marked as disallowed in robots.txt per bug 4776, their talk pages, if they exist, still get indexed. I'd suggest at least the following lines should be added to robots.txt under "User-agent: *":

Disallow: /wiki/Wikipedia_talk:Articles_for_deletion/
Disallow: /wiki/Wikipedia_talk%3AArticles_for_deletion/
Disallow: /wiki/Wikipedia_talk:Votes_for_deletion/
Disallow: /wiki/Wikipedia_talk%3AVotes_for_deletion/
Disallow: /wiki/Wikipedia_talk:Pages_for_deletion/
Disallow: /wiki/Wikipedia_talk%3APages_for_deletion/
Disallow: /wiki/Wikipedia_talk:Miscellany_for_deletion/
Disallow: /wiki/Wikipedia_talk%3AMiscellany_for_deletion/
Disallow: /wiki/Wikipedia_talk:Miscellaneous_deletion/
Disallow: /wiki/Wikipedia_talk%3AMiscellaneous_deletion/


Version: unspecified
Severity: enhancement
URL: http://en.wikipedia.org/robots.txt

Details

Reference
bz10288