Robots.txt Generator

1. Choose a preset — pick WordPress, Shopify, Wix, or start blank
2. Add rules — allow or disallow paths for specific user-agents
3. Block AI crawlers — toggle individual AI bots on/off
4. Add sitemap URL — point crawlers to your sitemap
5. Preview & download — review the live preview and download your robots.txt file

Modern websites need to manage AI crawlers separately from search engines. This tool lets you block training crawlers like GPTBot, CCBot, and Google-Extended while keeping search bots like Googlebot and Bingbot allowed. You can also distinguish between AI training bots and AI search bots (like ChatGPT-User or PerplexityBot).

Choose from optimized presets for WordPress (blocks wp-admin, includes, plugins), Shopify (blocks cart, checkout, admin), Wix (blocks internal paths), or content sites. Each preset follows SEO best practices for that platform.

Switch to the Validate tab to check your robots.txt for syntax errors, duplicate rules, conflicting directives, and missing sitemap declarations. Get clear error messages and suggestions to fix issues before deploying.

Use the Test tab to check if a specific URL is allowed or blocked by your rules. Select a user-agent (Googlebot, Bingbot, GPTBot, etc.) and enter a URL path to see the matching rule and result instantly.

This tool runs entirely in your browser. No uploads, no accounts, no limits. Generate as many robots.txt files as you need. Your content never leaves your device — the most private robots.txt generator available.