# This says to apply these settings to ALL search engine spiders/crawlers User-agent: * Sitemap: http://interstatedeals.com/sitemapindex.xml # These settings will keep spiders from indexing your unwanted pages Disallow: /adminpanel/ Disallow: /cache/ Disallow: /download/ Disallow: /email/ Disallow: /extras/ Disallow: /htmlarea/ Disallow: /temp/