Robots.txt Generator
Configure your robots.txt rules via UI — includes a safety warning for Disallow: /.
Warning: Disallow: / blocks ALL crawlers
This will prevent search engines from indexing your entire site. Only use this if you intentionally want the site de-indexed.
Generated robots.txt
How to use the Robots.txt Generator
Add rules using the "Add rule" button. Choose which bot the rule applies to, whether to Allow or Disallow, and the path. The output updates instantly. Copy and save the result as robots.txt in your site's root.
What is robots.txt?
The robots.txt file tells search engine crawlers which pages they can and can't access on your site. It's placed at the root of your domain (e.g. https://yoursite.com/robots.txt). Note: robots.txt is advisory — malicious bots may ignore it. It does not provide security, only crawl guidance.
Never use Disallow: / on a production site unless you want it completely removed from search engines.