Skip to content

RSL AI crawler licensing design spec#650

Open
jevansnyc wants to merge 1 commit intomainfrom
rsl-ai-crawler-licensing-spec
Open

RSL AI crawler licensing design spec#650
jevansnyc wants to merge 1 commit intomainfrom
rsl-ai-crawler-licensing-spec

Conversation

@jevansnyc
Copy link
Copy Markdown
Collaborator

@jevansnyc jevansnyc commented Apr 22, 2026

Summary

Adds design spec for Trusted Server's RSL-compliant AI crawler detection and licensing enforcement layer.

  • Edge-deployed AI crawler classification using six signals (UA, IP allowlist, JA4, ASN, H2, robots/license fetch correlation)
  • RSL 1.0 standards-compliant license publishing (/license.xml, robots.txt augmentation, Link header)
  • Public license.toml for RSL terms; private license.private.toml for enforcement secrets and commercial overrides
  • Standards-compliant 402/403 enforcement responses with inline RSL fragments
  • Permissive-by-default with per-publisher/per-route Strict override
  • Debug endpoints (/_ts/debug/rsl/summary, /_ts/debug/rsl/recent, /_ts/debug/rsl/license) and structured logging
  • Integrates with existing TS architecture; no changes needed to Edge Cookie, auction orchestrator, consent, or other integrations
  • Phase 2 preview for Open License Protocol (OLP) token-based access

Test plan

  • Review spec for accuracy against current TS infrastructure (integration hooks, JA4 signals, bot gate)
  • Verify RSL usage/payment vocabulary matches RSL 1.0 spec (https://rslstandard.org/rsl)
  • Validate onboarding flow assumptions against an existing TS publisher deployment
  • Confirm binary size estimates (~100 KB additional for IP allowlists + JA4 DB + new code)

🤖 Generated with Claude Code

@jevansnyc @claude
Add RSL AI crawler licensing design spec
8d08128 
Trusted Server RSL-compliant AI crawler detection and licensing
enforcement, MVP-ready. Six-signal classification (UA, IP, JA4, ASN,
H2, robots/license.xml correlation), permissive-by-default with strict
override, public license.toml + private license.private.toml split,
standards-compliant 402/403 responses, debug endpoints, structured
logging. Targets publishers already running TS. Phase 2 adds OLP
license server for programmatic token issuance.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jevansnyc jevansnyc linked an issue Apr 22, 2026 that may be closed by this pull request
Write Spec for RSL Support #649
Open
@jevansnyc jevansnyc mentioned this pull request Apr 22, 2026
Open
@aram356 aram356 assigned aram356 and jevansnyc and unassigned aram356 Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aram356 aram356 Awaiting requested review from aram356

@ChristianPavilonis ChristianPavilonis Awaiting requested review from ChristianPavilonis

At least 1 approving review is required to merge this pull request.

Assignees

@jevansnyc jevansnyc

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Write Spec for RSL Support

2 participants

@jevansnyc @aram356