#MSN Shopping robots.txt file #All pages in the root directory of this site may be crawled User-agent: * Crawl-delay: 2 Disallow: /css/ Disallow: /i/ Disallow: /passport/ Disallow: /scripts/ Disallow: /w3c/ Disallow: /xslt/ Disallow: /track/trackitemclick.aspx Disallow: /track/tracklinkclick.aspx