#robots.txt (7 links)

• Humans TXT: We Are People, Not Machines. humanstxt.org

  "An initiative for knowing the people behind a website." Cute.

  26 Jan 2011 #web #robots.txt#accountability
• SEOmoz | Serious Robots.txt Misuse & High Impact Solutions seomoz.org

  12 Oct 2010 #seo #seomoz #web #robots.txt #via:daneastwell
• The country's new robots.txt file kottke.org

  [this is good]

  21 Jan 2009 #web #seo #robots.txt #transparency #funny #usa#bush#obama
• Improved SEO documentation galore! mattcutts.com

  4 Jun 2008 #documentation #google #robots.txt #search #seo #web
• nextthing.org » robots.txt Adventure nextthing.org

  Great article following up on a robots.txt survey. The linked HTTP headers survey is amusing, too.

  22 Sept 2007 #web #http #robots.txt #seo #statistics
• Official Google Blog: Robots Exclusion Protocol: now with even more flexibility googleblog.blogspot.com

  2 Aug 2007 #google #search #robots.txt #seo #web
• Official Google Webmaster Central Blog: What's new with Sitemaps.org? googlewebmastercentral.blogspot.com

  Every website can have a robots.txt file in a standard location, so we decided to let you tell us about your Sitemap in the robots.txt file. All you have to do is add a line.

  25 Apr 2007 #sitemaps #google #seo#sitemap #search #web #robots.txt