ai-crawlers
Auditing robots.txt for AI Crawler Conflicts
Finds contradictions, accidental blocks, and gaps in your robots.txt directives for AI crawlers.
$144 downloads
Quick answer
Finds contradictions, accidental blocks, and gaps in your robots.txt directives for AI crawlers.
What this skill does
A robots.txt grown by accretion often contradicts itself — an agent both allowed and disallowed for a path, or a search agent accidentally blocked. This skill audits your AI-crawler directives for conflicts, accidental blocks of search agents, and missing decisions, so the file does what you intend.
Get Auditing robots.txt for AI Crawler Conflicts
$14, one-time — drops into .claude/skills/.