User-agent: * # directed to all spiders Disallow: /db/ Disallow: /scripts/ Disallow: /Templates/ Disallow: /Images/ Allow: default.htm