User-Agent: Googlebot Disallow: /js/pngfix/pngfix.htc # http://www.dotnetdotcom.org/ - DotBot does not recognize correctly URL-s in cyrillic and generates 404 errors. It may be because of CMS or Crawler bug?! User-agent: dotbot Disallow: / # Purebot doesn't read robots.txt? #User-agent: Purebot #Disallow: /