# disallow stress test User-agent: stress-agent Disallow: User-agent: wget Disallow: / # exclude help system from robots User-agent: * Disallow: /cgi-bin/ Disallow: /manual/ Disallow: /doc/ Disallow: /gif/ Disallow: /admin/