Page MenuHomePhabricator

add user talk namespace of En.Wikipedia to Robots.txt
Closed, ResolvedPublic

Description

As discussed in the link above, the user talk namespace should not appear in Google searches for privacy reasons. Also see this Wikimedia blog entry: http://blog.wikimedia.org/2008/04/29/robotstxt/


Version: unspecified
Severity: enhancement
URL: http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28proposals%29/Archive_25#Stopping_search_engines_from_indexing_the_user_talk_namespace.3F

Details

Reference
bz13890

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 10:07 PM
bzimport set Reference to bz13890.
bzimport added a subscriber: Unknown Object (MLST).

[Note that if implemented, they should not be added to robots.txt, but rather the $wgNamespaceRobotPolicies setting changed.]

  • Bug 14333 has been marked as a duplicate of this bug. ***

jeluf wrote:

Done.

'enwiki' => array(
    NS_USER_TALK => 'noindex,follow',
),

ned wrote:

Where the hell was the discussion for this? The "cited" url break off on this page, and was originally posted four months ago. The NOINDEX magic word was supposed to solve this, anyways. When you guys see a major gap in time like this, someone needs to go back and probe the community for comment to see if there really was a consensus. This should not have been done.

The cited URL works for me: go to Wikipedia:Village pump (proposals)/Archive 25 in the English Wikipedia and check the section titled "Stopping search engines from indexing the user talk namespace?" Also see my message at Wikipedia talk:NOINDEX of noticeboards in the section entitled "Noindexing user talk space and other thoughts", where I demonstrated a real case where this could be a problem.

ned wrote:

Yes, the url works for me now (it only cut off when I wasn't logged in). While I understand and respect the opinion that user talk shouldn't be indexed, there certainly wasn't any community support for the change. Now seeing that discussion I'm even more surprised that this change occurred. I've now gone ahead and started a formal request to re-include user talk pages at http://en.wikipedia.org/wiki/Wikipedia:VPR#Re-enable_searches_in_the_user_talk_space

happy_melon wrote:

Why was this changed to "noindex *follow*"?? Forgive me if I'm talking completely out of my arse, but shouldn't that allow external links on user talk pages to affect the rankings of other sites? Zomg spam, anyone??

matthew.britton wrote:

It is my personal opinion that anyone in favour of changes like these does not understand the way in which today's World Wide Web functions.