# /robots.txt file User-agent: Googlebot Disallow: User-agent: MSNBot Disallow: User-agent: YahooBot Disallow: User-agent: Scooter Disallow: User-agent: KIT-Fireball Disallow: User-agent: Lycos_Spider_(T-Rex) Disallow: User-agent: Teoma Disallow: User-agent: htdig Disallow: User-agent: gsa-crawler Disallow: User-agent: gsa-crawler-zoa Disallow: User-agent: * Disallow: /