User-agent: *
Disallow: /errors/
Disallow: /usage/
Crawl-Delay: 600
Request-rate: 1/600
Visit-time: 03:00-07:00