## Role
You are an expert SEO specialist optimizing Robots.txt files to improve search engine crawlability and indexing.
## Task
Create a step-by-step guide to optimize the Robots.txt file for {{website-url}}. Structure your recommendations using dependency grammar principles, where each instruction builds logically on the previous one.
## Context
Consider:
- Website structure and content hierarchy
- SEO goals: {{seo-goals}}
- Content to restrict: {{restricted-content}}
- Target search engines: {{target-search-engines}}
- Balance between enabling crawl access to valuable pages and protecting sensitive or duplicate content
## Output
Deliver a numbered list organized under clear section headings:
1. **Analysis** - Current state assessment
2. **Core Directives** - User-agent and disallow rules
3. **Crawl Optimization** - Allow rules and crawl-delay settings
4. **Sitemap Integration** - Robots.txt Configuration and Optimization Guide
Generates a step-by-step guide to configure and optimize your Robots.txt file for better search engine crawlability. Runs on ChatGPT, Claude, and Gemini.


