# Agent Access Index data dictionary (The Walled Web)

Generated 2026-10-20T03:43:15.760Z from Checker run 2026-10-19. Code: Apache-2.0. Data: CC BY 4.0.

## sites
One row per tracked site, current state.
- **status**: blocked | partial | allowed | paid | unknown | opted_out (rules: /methodology/#status)
- **lane_training, lane_search, lane_user_directed**: none (every agent of that kind blocked) | full (all allowed) | mixed | restricted | unknown
- **door**: what the homepage did when WalledWebBot asked for it: open | refused | challenged | payment_required | skipped | error
- **robots_via**: live (read today) | archive (Internet Archive copy, at most 3 months old) | empty (unreadable)
- **checker_allowed**: 1 if robots.txt lets WalledWebBot fetch "/"
- **content_signal**: first Cloudflare Content-Signal value in robots.txt, if any
- **rsl_license**: 1 if robots.txt has an RSL License line

## agents
The tracked robots.txt tokens. **purpose**: training | search | user_directed. **major**: counts toward the Blocked rule. **obeys_robots**: what the operator's own documentation says (yes | implied | not_guaranteed | no | unverified | unstated | unknown).

## site_agent
Current access per site and agent. **access**: none (homepage and the rest of the site disallowed) | restricted (some of it) | full. **named**: 1 if the file names the agent; 0 if it falls under `User-agent: *`.

## monthly
The same as site_agent for every month since 2023-01, from the Internet Archive's first capture each month (carried forward up to three months) and the Checker's runs. Missing rows mean no copy that month.

## changes
AI-relevant changes. **source**: archive (dated by month; window_from and window_to bound when it happened) | checker (dated by day). **direction**: up (more blocking) | down (less) | mixed | other (signals only). **evidence_before / evidence_after**: URLs of the two files compared.

## tokens_in_wild
Every user-agent token named in a tracked robots.txt today, with how many sites name it and whether an operator documents it.

## runs
Checker runs: when, from where (vantage), how many requests.

## panel
The fixed set of sites used for trends: those with a robots.txt on record from the first quarter of the window through the current month. Comparisons over time should be restricted to these domains.