# http://www.searchengineworld.com/cgi-bin/robotcheck.cgi # Disallow: /directory/filename.ext # Disallow: /directory/ # Disallow: /*.ext$ User-agent:* Disallow: /downloads/ Disallow: /fwd/ Disallow: /img/ Disallow: /photo/ User-agent: MSIECrawler Disallow: / User-agent: MSNBot Disallow: /