ai-crawlers
Rate-Limiting Aggressive AI Crawlers
Throttles training crawlers that exceed a request budget while never throttling the search agents you want in answers.
$156 downloads
Quick answer
Throttles training crawlers that exceed a request budget while never throttling the search agents you want in answers.
What this skill does
Some AI training crawlers hammer sites; throttling them protects your origin — but throttle the wrong one and you vanish from an engine's answers. This skill applies a per-agent rate policy that limits aggressive training crawlers while always exempting the search-surfacing agents you want indexing you.
Get Rate-Limiting Aggressive AI Crawlers
$15, one-time — drops into .claude/skills/.