User-agent: * Disallow: / # Begin Exclusion From Directories from robots.txt Disallow: /cgi/ Disallow: /img/ disallow: /ftp/ disallow: /Public FTP/ disallow: /stats/ disallow: /vbulletin/ disallow: /blogs/ disallow: /twiki/ disallow: /wiki/ disallow: /mediawiki/