See bug 56893 for instructions.
Version: unspecified
Severity: enhancement
URL: http://tools.wmflabs.org/robots.txt
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=61132
See bug 56893 for instructions.
Version: unspecified
Severity: enhancement
URL: http://tools.wmflabs.org/robots.txt
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=61132
What does "archive tools" mean? Tools (for the most part) process input and generate output from that. A spider doesn't provide input and thus doesn't get output.
For "archiving tools" (i. e. the interesting bit, the processor), their source code needs to be put in a repository. But neither Internet Archive nor any other spider can access private source code from the web.
(In reply to comment #1)
What does "archive tools" mean? Tools (for the most part) process input and
generate output from that. A spider doesn't provide input and thus doesn't
get
output.
Which is why this operation is inexpensive but will allow Wayback to archive URLs referenced from the web or by users.
Pages with dynamically generated content make no semantic sense to archive, and the cost in resources of allowing spidering of tool URLs is prohibitive.
Tool Labs is not intended for long-lived mostly static content (which is what archiving makes sense for); that data belongs on a wiki -- possibly generated and put there /by/ tools.