User-agent

Allow Rules (one per line)

Disallow Rules (one per line)

Sitemap URL

Crawl-delay