01Permitted Crawlers
The following crawlers are permitted to index public pages on clips.forkoff.xyz for search, answer-engine, and discovery purposes:
- ▸Googlebot, including the answer-engine variant. Indexing is encouraged.
- ▸Bingbot, including the Microsoft Copilot variant.
- ▸Anthropic ClaudeBot, for answer-engine + research ingestion of public pages.
- ▸OpenAI GPTBot, for answer-engine ingestion of public pages.
- ▸PerplexityBot and Common Crawl (CCBot) for general indexing.
- ▸DuckDuckBot, YandexBot, Baidu Spider for general search indexing.
The full machine-readable rules live at /robots.txt.
02Restricted Use
The following are NOT permitted without prior written authorization from crew@forkoff.xyz:
- ▸Bulk scraping of intake form fields, calculator inputs, or session data for training-dataset assembly.
- ▸Per-user extraction of payout amounts, qualified-view ledger entries, or campaign-level ledger exports.
- ▸Republication of clips.forkoff.xyz content (including comparison tables, qualification criteria, FAQ answers) without attribution.
- ▸Automated submission of intake forms or test traffic to /api/intake, /api/qualify, or any other server endpoint.
- ▸Use of FORKOFF brand assets (logo, wordmark, brand red palette) in derivative training data or competing products.
03Rate Limits
Public pages are rate-limited at 1 request per second per IP on the unauthenticated edge. API endpoints are rate-limited per the contract or sandbox terms applicable to the requester.
Crawlers exceeding these limits may be temporarily blocked. Persistent violations are escalated through the operator's abuse contact.
04Content-Signal Headers
Where supported by the requesting agent, clips.forkoff.xyz emits Content-Signal headers indicating the data-use posture per page:
- ▸ai-train=no. We do not authorize use of any clips.forkoff.xyz content for AI model training without prior written permission.
- ▸search=yes. Indexing for search and answer-engine response is permitted.
- ▸ai-input=yes. Use as ad-hoc context input to AI assistants (e.g. when a user pastes a URL into a chat) is permitted.
05MCP and Agent-Skill Discovery
When clips.forkoff.xyz exposes agent-callable surfaces (FORKOFF intake brief submission, qualification ledger queries, payout estimation), discovery manifests will live under /.well-known/. Today, only /.well-known/security.txt is published.
Agent-skill discovery (Anthropic-style) and MCP server endpoints (when published) will follow the relevant standards and be cross-linked from this page.
06Reporting and Exceptions
AI labs requesting elevated access for research or partnership purposes: crew@forkoff.xyz.
Suspected misuse of FORKOFF content in training datasets: security@forkoff.xyz.