Track changes to robots.txt

Author: kornelski

src/doc/robots.txt

@@ -0,0 +1,19 @@
+# NB: This file is not automatically deployed. After changes, it needs to be uploaded manually to doc.rust-lang.org
+User-agent: *
+Disallow: /0.3/
+Disallow: /0.4/
+Disallow: /0.5/
+Disallow: /0.6/
+Disallow: /0.7/
+Disallow: /0.8/
+Disallow: /0.9/
+Disallow: /0.10/
+Disallow: /0.11.0/
+Disallow: /0.12.0/
+Disallow: /1.0.0-alpha/
+Disallow: /1.0.0-alpha.2/
+Disallow: /1.0.0-beta/
+Disallow: /1.0.0-beta.2/
+Disallow: /1.0.0-beta.3/
+Disallow: /1.0.0-beta.4/
+Disallow: /1.0.0-beta.5/