Home/Skills/Designing an AI Crawler Access Policy
ai-crawlers

Designing an AI Crawler Access Policy

Builds a deliberate, per-agent allow/deny policy for AI crawlers in robots.txt — so you're represented where you want and protected where you don't.

$158 downloads
Quick answer

Builds a deliberate, per-agent allow/deny policy for AI crawlers in robots.txt — so you're represented where you want and protected where you don't.

What this skill does

Blocking an AI search agent silently removes you from its answers; allowing every training crawler gives your content away. This skill designs a per-agent allow/deny matrix for the major AI bots, reconciles it with your goals, and emits the robots.txt — with a resolver so you can prove what's allowed for any agent and path.

Get Designing an AI Crawler Access Policy

$15, one-time — drops into .claude/skills/.