🤖

Robots.txt Generator

New

Generate a robots.txt file for your website. Control crawler access.

User-agent

Allow

Disallow

Sitemap URL (optional)

robots.txt output

User-agent: *
Allow: /
Disallow: /admin
Disallow: /private

Common bot names

GooglebotGoogle Search

BingbotMicrosoft Bing

GPTBotOpenAI ChatGPT

Claude-WebAnthropic Claude

BaiduspiderBaidu