100% Free - No Limits

Robots.txt Generator

Control how search engines and AI crawlers access your site. Build your robots.txt with our visual editor.

Rule 1

/admin//api/
/

Generated robots.txt

# robots.txt
# Generated by CiteKit

User-agent: *
Disallow: /admin/
Disallow: /api/
Allow: /

How to use: Upload this file to your website root at yoursite.com/robots.txt

AI Crawler User Agents

Want AI platforms to cite your content? Make sure you're not blocking these crawlers:

GPTBotChatGPT-UserGoogle-ExtendedPerplexityBotClaudeBotanthropic-ai