{"domain":"yelp.com","name":"Yelp","category":"local","subcategory":"reviews","country":"US","tags":["named-claim"],"notes":"Named in the @trikcode post (paid data licensing for agents)","status":"paid","status_label":"Charges AI","rationale":"Yelp says AI agents may use the site only through its paid data licensing program. Its robots.txt (the public rules file for bots) tells all 24 AI bots we track to stay out of the whole site. It names GPTBot, ClaudeBot, Google-Extended, CCBot, Meta-ExternalAgent and 6 more directly. A catch-all rule (<code>User-agent: *</code>) keeps the rest out too. The same rules ask our bot to stay out, so we read only robots.txt.","computed_at":"2026-10-20T03:43:14.824Z","last_checked":"2026-10-19T13:31:24.246Z","checker":{"allowed":false,"vantage":"residential connection, United States (manual run)","origin":"https://www.yelp.com"},"robots":{"via":{"type":"live","date":"2026-10-19","url":"https://www.yelp.com/robots.txt"},"kind":"file","http":200,"error":null,"url":"https://www.yelp.com/robots.txt","hops":[{"from":"https://yelp.com/robots.txt","to":"https://www.yelp.com/robots.txt","status":301}],"blob":"60c2dd139b30dea2ba79575abe4a04ef182ef90fe6ed5491bf07deb7f75893b3","text":"# By accessing Yelp's website (© 2026) you agree to Yelp's Terms of Service, available at\n# https://www.yelp.com/static?country=US&p=tos\n#\n# If you would like to inquire about crawling Yelp, please contact us at\n# https://www.yelp.com/contact\n#\n# As always, Asimov's Three Laws are in effect:\n# 1. A robot may not injure a human being or, through inaction, allow a human\n#    being to come to harm.\n# 2. A robot must obey orders given it by human beings except where such\n#    orders would conflict with the First Law.\n# 3. A robot must protect its own existence as long as such protection does\n#    not conflict with the First or Second Law.\n#\n# AI / LLM Crawlers and Agents:\n# Use of any robot, spider, service search/retrieval application, or other automated device, process or means to access, retrieve,\n# copy, scrape, or index any portion of the service or any content is prohibited, except as expressly permitted by Yelp.\n# If you would like to request permission to crawl or use Yelp content, contact us here: https://www.yelp.com/contact\n\nUser-Agent: Google-InspectionTool\nUser-Agent: Googlebot\nUser-Agent: Googlebot-Image\nUser-Agent: Googlebot-Mobile\nUser-Agent: Googlebot-Video\nAllow: /biz/taste-of-lurgan-lost-springs\nAllow: /biz_photos/*-albuquerque*\nAllow: /biz_photos/*-anaheim*\nAllow: /biz_photos/*-atlanta*\nAllow: /biz_photos/*-austin*\nAllow: /biz_photos/*-baltimore*\nAllow: /biz_photos/*-boston*\nAllow: /biz_photos/*-brooklyn*\nAllow: /biz_photos/*-charlotte*\nAllow: /biz_photos/*-chicago*\nAllow: /biz_photos/*-cincinnati*\nAllow: /biz_photos/*-columbus*\nAllow: /biz_photos/*-dallas*\nAllow: /biz_photos/*-denver*\nAllow: /biz_photos/*-fort-worth*\nAllow: /biz_photos/*-fresno*\nAllow: /biz_photos/*-honolulu*\nAllow: /biz_photos/*-houston*\nAllow: /biz_photos/*-indianapolis*\nAllow: /biz_photos/*-irvine*\nAllow: /biz_photos/*-jacksonville*\nAllow: /biz_photos/*-las-vegas*\nAllow: /biz_photos/*-long-beach*\nAllow: /biz_photos/*-los-angeles*\nAllow: /biz_photos/*-louisville*\nAllow: /biz_photos/*-manhattan*\nAllow: /biz_photos/*-miami*\nAllow: /biz_photos/*-minneapolis*\nAllow: /biz_photos/*-nashville*\nAllow: /biz_photos/*-new-orleans*\nAllow: /biz_photos/*-new-york*\nAllow: /biz_photos/*-oakland*\nAllow: /biz_photos/*-oklahoma-city*\nAllow: /biz_photos/*-orlando*\nAllow: /biz_photos/*-philadelphia*\nAllow: /biz_photos/*-phoenix*\nAllow: /biz_photos/*-pittsburgh*\nAllow: /biz_photos/*-portland*\nAllow: /biz_photos/*-raleigh*\nAllow: /biz_photos/*-richmond*\nAllow: /biz_photos/*-sacramento*\nAllow: /biz_photos/*-san-antonio*\nAllow: /biz_photos/*-san-diego*\nAllow: /biz_photos/*-san-francisco*\nAllow: /biz_photos/*-san-jose*\nAllow: /biz_photos/*-scottsdale*\nAllow: /biz_photos/*-seattle*\nAllow: /biz_photos/*-tampa*\nAllow: /biz_photos/*-tampa-bay*\nAllow: /biz_photos/*-tucson*\nAllow: /biz_photos/*-washington*\nAllow: /biz_photos/*?select_video=\nAllow: /biz_photos/video_url\n\nUser-Agent: LinkedInBot\nUser-Agent: Twitterbot\nUser-Agent: facebookexternalhit\nAllow: /article/\n\nUser-Agent: AdsBot-Google\nUser-Agent: BingPreview\nUser-Agent: Google-InspectionTool\nUser-Agent: Googlebot\nUser-Agent: Googlebot-Image\nUser-Agent: Googlebot-Mobile\nUser-Agent: Googlebot-Video\nUser-Agent: LinkedInBot\nUser-Agent: Mediapartners-Google\nUser-Agent: STC-bot\nUser-Agent: Twitterbot\nUser-Agent: Yahoo! Slurp\nUser-Agent: bingbot\nUser-Agent: facebookexternalhit\nDisallow: *US_CENSUS_NAME*\nDisallow: *US_FEMALE_NAME*\nDisallow: *US_MALE_NAME*\nDisallow: /1014943\nDisallow: /3584794\nDisallow: /5787254\nDisallow: /ad_acknowledgment\nDisallow: /ad_spice\nDisallow: /ad_syndication_user_tracking\nDisallow: /ad_visibility\nDisallow: /adredir?\nDisallow: /adtrack\nDisallow: /advertise?\nDisallow: /answers/*/delete\nDisallow: /answers/*/edit\nDisallow: /answers/*/vote\nDisallow: /biz/*?*&translate=1*\nDisallow: /biz/*?*&translate=True*\nDisallow: /biz/*?translate=1*\nDisallow: /biz/*?translate=True*\nDisallow: /biz/*destination=*\nDisallow: /biz/*entry_point=*\nDisallow: /biz/impression/*\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-0\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-1\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-2\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-3\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-4\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-5\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-6\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-7\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-8\nDisallow: /biz/outlook-autumn-market-fundamental-catwalk-flimsy-roost-legibility-individualism-grocer-predestination-9\nDisallow: /biz/taste-of-lurgan-lost-springs\nDisallow: /biz_attribute\nDisallow: /biz_link\nDisallow: /biz_photos\nDisallow: /biz_photos/*/log\nDisallow: /biz_photos/*/log_views\nDisallow: /biz_photos/feedback/\nDisallow: /biz_redir\nDisallow: /biz_share\nDisallow: /biz_update\nDisallow: /bn/\nDisallow: /bookmark?\nDisallow: /browse/reviews/recent\nDisallow: /client_errors\nDisallow: /collections/following\nDisallow: /collections/user\nDisallow: /cookie_sync\nDisallow: /csp_block\nDisallow: /csp_report_only\nDisallow: /elite?\nDisallow: /email_actions\nDisallow: /events/*/browse\nDisallow: /events/*/feed\nDisallow: /events/addphoto/\nDisallow: /events/cancel/\nDisallow: /events/cancel_rsvp/\nDisallow: /events/change_guest/\nDisallow: /events/create\nDisallow: /events/create/\nDisallow: /events/deletephoto/\nDisallow: /events/details/\nDisallow: /events/elite/\nDisallow: /events/email_notifications/\nDisallow: /events/event_talk_subscribe\nDisallow: /events/event_talk_unsubscribe\nDisallow: /events/export/\nDisallow: /events/get_media_slice/\nDisallow: /events/interested/\nDisallow: /events/load_more_messages/\nDisallow: /events/photos/\nDisallow: /events/record_rsvp/\nDisallow: /events/share/\nDisallow: /events/similar_snippet\nDisallow: /events/subscriber_snippet\nDisallow: /events/talk/\nDisallow: /events/uncancel/\nDisallow: /events/update\nDisallow: /events/update/\nDisallow: /events/update_primary_photo\nDisallow: /events/update_subscription\nDisallow: /events/updatecaption\nDisallow: /facebook_connect\nDisallow: /flag_content/submit\nDisallow: /flag_content?\nDisallow: /gamtarget\nDisallow: /location_suggest\nDisallow: /mail?\nDisallow: /menu/*/feedback_form\nDisallow: /message_the_business\nDisallow: /mss/review_feedback/\nDisallow: /mtb_composer\nDisallow: /not_recommended_reviews\nDisallow: /opportunity\nDisallow: /photo/\nDisallow: /possible_biz_owner\nDisallow: /proredir?\nDisallow: /px.gif\nDisallow: /questions/*/create_answer/\nDisallow: /questions/*/delete\nDisallow: /questions/*/edit\nDisallow: /questions/*/subscribe\nDisallow: /redir?\nDisallow: /reservations/*/notifyme\nDisallow: /review_feed_auto_fetch\nDisallow: /search_suggest\nDisallow: /send_to_friend\nDisallow: /sit_rep\nDisallow: /spice\nDisallow: /start_order\nDisallow: /syndicate\nDisallow: /syndication_cookie_sync\nDisallow: /talk/new_topic\nDisallow: /thanx?\nDisallow: /transaction_platform/start_order\nDisallow: /user_details\nDisallow: /user_details_answers_given\nDisallow: /user_details_bookmarks\nDisallow: /user_details_friends\nDisallow: /user_details_reviews_self\nDisallow: /user_details_thanx\nDisallow: /user_photos\nDisallow: /weekly/signup\nDisallow: /writeareview/\nDisallow: /yuv\n\nUser-Agent: Google-InspectionTool\nUser-Agent: Googlebot\nUser-Agent: Googlebot-Image\nUser-Agent: Googlebot-Mobile\nUser-Agent: Googlebot-Video\nDisallow: /biz_photos/*-albuquerque*/log\nDisallow: /biz_photos/*-albuquerque*/log_views\nDisallow: /biz_photos/*-anaheim*/log\nDisallow: /biz_photos/*-anaheim*/log_views\nDisallow: /biz_photos/*-atlanta*/log\nDisallow: /biz_photos/*-atlanta*/log_views\nDisallow: /biz_photos/*-austin*/log\nDisallow: /biz_photos/*-austin*/log_views\nDisallow: /biz_photos/*-baltimore*/log\nDisallow: /biz_photos/*-baltimore*/log_views\nDisallow: /biz_photos/*-boston*/log\nDisallow: /biz_photos/*-boston*/log_views\nDisallow: /biz_photos/*-brooklyn*/log\nDisallow: /biz_photos/*-brooklyn*/log_views\nDisallow: /biz_photos/*-charlotte*/log\nDisallow: /biz_photos/*-charlotte*/log_views\nDisallow: /biz_photos/*-chicago*/log\nDisallow: /biz_photos/*-chicago*/log_views\nDisallow: /biz_photos/*-cincinnati*/log\nDisallow: /biz_photos/*-cincinnati*/log_views\nDisallow: /biz_photos/*-columbus*/log\nDisallow: /biz_photos/*-columbus*/log_views\nDisallow: /biz_photos/*-dallas*/log\nDisallow: /biz_photos/*-dallas*/log_views\nDisallow: /biz_photos/*-denver*/log\nDisallow: /biz_photos/*-denver*/log_views\nDisallow: /biz_photos/*-fort-worth*/log\nDisallow: /biz_photos/*-fort-worth*/log_views\nDisallow: /biz_photos/*-fresno*/log\nDisallow: /biz_photos/*-fresno*/log_views\nDisallow: /biz_photos/*-honolulu*/log\nDisallow: /biz_photos/*-honolulu*/log_views\nDisallow: /biz_photos/*-houston*/log\nDisallow: /biz_photos/*-houston*/log_views\nDisallow: /biz_photos/*-indianapolis*/log\nDisallow: /biz_photos/*-indianapolis*/log_views\nDisallow: /biz_photos/*-irvine*/log\nDisallow: /biz_photos/*-irvine*/log_views\nDisallow: /biz_photos/*-jacksonville*/log\nDisallow: /biz_photos/*-jacksonville*/log_views\nDisallow: /biz_photos/*-las-vegas*/log\nDisallow: /biz_photos/*-las-vegas*/log_views\nDisallow: /biz_photos/*-long-beach*/log\nDisallow: /biz_photos/*-long-beach*/log_views\nDisallow: /biz_photos/*-los-angeles*/log\nDisallow: /biz_photos/*-los-angeles*/log_views\nDisallow: /biz_photos/*-louisville*/log\nDisallow: /biz_photos/*-louisville*/log_views\nDisallow: /biz_photos/*-manhattan*/log\nDisallow: /biz_photos/*-manhattan*/log_views\nDisallow: /biz_photos/*-miami*/log\nDisallow: /biz_photos/*-miami*/log_views\nDisallow: /biz_photos/*-minneapolis*/log\nDisallow: /biz_photos/*-minneapolis*/log_views\nDisallow: /biz_photos/*-nashville*/log\nDisallow: /biz_photos/*-nashville*/log_views\nDisallow: /biz_photos/*-new-orleans*/log\nDisallow: /biz_photos/*-new-orleans*/log_views\nDisallow: /biz_photos/*-new-york*/log\nDisallow: /biz_photos/*-new-york*/log_views\nDisallow: /biz_photos/*-oakland*/log\nDisallow: /biz_photos/*-oakland*/log_views\nDisallow: /biz_photos/*-oklahoma-city*/log\nDisallow: /biz_photos/*-oklahoma-city*/log_views\nDisallow: /biz_photos/*-orlando*/log\nDisallow: /biz_photos/*-orlando*/log_views\nDisallow: /biz_photos/*-philadelphia*/log\nDisallow: /biz_photos/*-philadelphia*/log_views\nDisallow: /biz_photos/*-phoenix*/log\nDisallow: /biz_photos/*-phoenix*/log_views\nDisallow: /biz_photos/*-pittsburgh*/log\nDisallow: /biz_photos/*-pittsburgh*/log_views\nDisallow: /biz_photos/*-portland*/log\nDisallow: /biz_photos/*-portland*/log_views\nDisallow: /biz_photos/*-raleigh*/log\nDisallow: /biz_photos/*-raleigh*/log_views\nDisallow: /biz_photos/*-richmond*/log\nDisallow: /biz_photos/*-richmond*/log_views\nDisallow: /biz_photos/*-sacramento*/log\nDisallow: /biz_photos/*-sacramento*/log_views\nDisallow: /biz_photos/*-san-antonio*/log\nDisallow: /biz_photos/*-san-antonio*/log_views\nDisallow: /biz_photos/*-san-diego*/log\nDisallow: /biz_photos/*-san-diego*/log_views\nDisallow: /biz_photos/*-san-francisco*/log\nDisallow: /biz_photos/*-san-francisco*/log_views\nDisallow: /biz_photos/*-san-jose*/log\nDisallow: /biz_photos/*-san-jose*/log_views\nDisallow: /biz_photos/*-scottsdale*/log\nDisallow: /biz_photos/*-scottsdale*/log_views\nDisallow: /biz_photos/*-seattle*/log\nDisallow: /biz_photos/*-seattle*/log_views\nDisallow: /biz_photos/*-tampa*/log\nDisallow: /biz_photos/*-tampa*/log_views\nDisallow: /biz_photos/*-tampa-bay*/log\nDisallow: /biz_photos/*-tampa-bay*/log_views\nDisallow: /biz_photos/*-tucson*/log\nDisallow: /biz_photos/*-tucson*/log_views\nDisallow: /biz_photos/*-washington*/log\nDisallow: /biz_photos/*-washington*/log_views\n\nUser-Agent: Yahoo! Slurp\nUser-Agent: bingbot\nDisallow: /search*start=\n\nUser-Agent: OAI-SearchBot\nAllow: /spk/\nAllow: /article$\nAllow: /article/\nDisallow: /\n\nUser-Agent: ChatGPT-User\nUser-Agent: Claude-User\nUser-Agent: PerplexityUser\nUser-Agent: Perplexity-User\nUser-Agent: Claude-SearchBot\nUser-Agent: PerplexityBot\nAllow: /article$\nAllow: /article/\nDisallow: /\n\nUser-Agent: GPTBot\nUser-Agent: ClaudeBot\nUser-Agent: CCBot\nUser-Agent: Google-Extended\nUser-Agent: anthropic-ai\nUser-Agent: Claude-Web\nUser-Agent: Meta-ExternalAgent\nUser-Agent: Meta-ExternalFetcher\nUser-Agent: RuxitSynthetic\nDisallow: /\n\nUser-Agent: *\nDisallow: /\n","refused":null,"warnings":[],"contentSignals":[],"contentUsage":[],"licenses":[],"aiComments":[{"line":15,"text":"AI / LLM Crawlers and Agents:"},{"line":17,"text":"copy, scrape, or index any portion of the service or any content is prohibited, except as expressly permitted by Yelp."}],"platform":null,"wildcard":{"exists":true,"access":"none"},"named":["Google-InspectionTool","Googlebot","Googlebot-Image","Googlebot-Mobile","Googlebot-Video","LinkedInBot","Twitterbot","facebookexternalhit","AdsBot-Google","BingPreview","Mediapartners-Google","STC-bot","Yahoo! Slurp","bingbot","OAI-SearchBot","ChatGPT-User","Claude-User","PerplexityUser","Perplexity-User","Claude-SearchBot","PerplexityBot","GPTBot","ClaudeBot","CCBot","Google-Extended","anthropic-ai","Claude-Web","Meta-ExternalAgent","Meta-ExternalFetcher","RuxitSynthetic","*"],"lines":365},"agents":[{"token":"GPTBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":361},"any":{"allowed":false,"line":361},"search":{"allowed":false,"line":361},"cart":{"allowed":false,"line":361},"checkout":{"allowed":false,"line":361},"account":{"allowed":false,"line":361}},"disallows":1,"lines":[352,361],"groupRanges":[[352,361]]},{"token":"ClaudeBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":361},"any":{"allowed":false,"line":361},"search":{"allowed":false,"line":361},"cart":{"allowed":false,"line":361},"checkout":{"allowed":false,"line":361},"account":{"allowed":false,"line":361}},"disallows":1,"lines":[352,361],"groupRanges":[[352,361]]},{"token":"Google-Extended","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":361},"any":{"allowed":false,"line":361},"search":{"allowed":false,"line":361},"cart":{"allowed":false,"line":361},"checkout":{"allowed":false,"line":361},"account":{"allowed":false,"line":361}},"disallows":1,"lines":[352,361],"groupRanges":[[352,361]]},{"token":"Applebot-Extended","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"CCBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":361},"any":{"allowed":false,"line":361},"search":{"allowed":false,"line":361},"cart":{"allowed":false,"line":361},"checkout":{"allowed":false,"line":361},"account":{"allowed":false,"line":361}},"disallows":1,"lines":[352,361],"groupRanges":[[352,361]]},{"token":"Meta-ExternalAgent","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":361},"any":{"allowed":false,"line":361},"search":{"allowed":false,"line":361},"cart":{"allowed":false,"line":361},"checkout":{"allowed":false,"line":361},"account":{"allowed":false,"line":361}},"disallows":1,"lines":[352,361],"groupRanges":[[352,361]]},{"token":"Bytespider","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"MistralAI-Training","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"AI2Bot","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"anthropic-ai","deprecated":true,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":361},"any":{"allowed":false,"line":361},"search":{"allowed":false,"line":361},"cart":{"allowed":false,"line":361},"checkout":{"allowed":false,"line":361},"account":{"allowed":false,"line":361}},"disallows":1,"lines":[352,361],"groupRanges":[[352,361]]},{"token":"OAI-SearchBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":340},"any":{"allowed":false,"line":340},"search":{"allowed":false,"line":340},"cart":{"allowed":false,"line":340},"checkout":{"allowed":false,"line":340},"account":{"allowed":false,"line":340}},"disallows":1,"lines":[336,340],"groupRanges":[[336,340]]},{"token":"Claude-SearchBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":350},"any":{"allowed":false,"line":350},"search":{"allowed":false,"line":350},"cart":{"allowed":false,"line":350},"checkout":{"allowed":false,"line":350},"account":{"allowed":false,"line":350}},"disallows":1,"lines":[342,350],"groupRanges":[[342,350]]},{"token":"PerplexityBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":350},"any":{"allowed":false,"line":350},"search":{"allowed":false,"line":350},"cart":{"allowed":false,"line":350},"checkout":{"allowed":false,"line":350},"account":{"allowed":false,"line":350}},"disallows":1,"lines":[342,350],"groupRanges":[[342,350]]},{"token":"Amazonbot","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"Amzn-SearchBot","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"DuckAssistBot","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"MistralAI-Index","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"Meta-WebIndexer","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"ChatGPT-User","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":350},"any":{"allowed":false,"line":350},"search":{"allowed":false,"line":350},"cart":{"allowed":false,"line":350},"checkout":{"allowed":false,"line":350},"account":{"allowed":false,"line":350}},"disallows":1,"lines":[342,350],"groupRanges":[[342,350]]},{"token":"Claude-User","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":350},"any":{"allowed":false,"line":350},"search":{"allowed":false,"line":350},"cart":{"allowed":false,"line":350},"checkout":{"allowed":false,"line":350},"account":{"allowed":false,"line":350}},"disallows":1,"lines":[342,350],"groupRanges":[[342,350]]},{"token":"Perplexity-User","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":350},"any":{"allowed":false,"line":350},"search":{"allowed":false,"line":350},"cart":{"allowed":false,"line":350},"checkout":{"allowed":false,"line":350},"account":{"allowed":false,"line":350}},"disallows":1,"lines":[342,350],"groupRanges":[[342,350]]},{"token":"Google-Agent","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"Meta-ExternalFetcher","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":361},"any":{"allowed":false,"line":361},"search":{"allowed":false,"line":361},"cart":{"allowed":false,"line":361},"checkout":{"allowed":false,"line":361},"account":{"allowed":false,"line":361}},"disallows":1,"lines":[352,361],"groupRanges":[[352,361]]},{"token":"Amzn-User","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]},{"token":"MistralAI-User","deprecated":false,"matched":"wildcard","access":"none","probes":{"root":{"allowed":false,"line":364},"any":{"allowed":false,"line":364},"search":{"allowed":false,"line":364},"cart":{"allowed":false,"line":364},"checkout":{"allowed":false,"line":364},"account":{"allowed":false,"line":364}},"disallows":1,"lines":[363,364],"groupRanges":[[363,364]]}],"refs":[{"token":"Googlebot","matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":227,"lines":[20,77,84,224,226,330],"groupRanges":[[20,77],[84,224],[226,330]]},{"token":"Bingbot","matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":128,"lines":[84,224,332,334],"groupRanges":[[84,224],[332,334]]}],"lanes":{"training":{"level":"none","none":9,"full":0,"restricted":0,"n":9},"search":{"level":"none","none":8,"full":0,"restricted":0,"n":8},"user_directed":{"level":"none","none":7,"full":0,"restricted":0,"n":7}},"door":{"door":"skipped","reason":"Disallowed for AgentAccessIndexBot by robots.txt"},"edge":{"cdn":["Fastly"],"botManagement":[]},"llms":{"state":"skipped","reason":"Disallowed for AgentAccessIndexBot by robots.txt"},"ai_txt":{"state":"skipped","reason":"Disallowed for AgentAccessIndexBot by robots.txt"},"tdmrep":{"state":"skipped","reason":"Disallowed for AgentAccessIndexBot by robots.txt"},"meta":null,"x_robots_tag":null,"tos":{"url":null,"source":null,"skipped":"No ToS URL curated or discovered"},"paid":{"domain":"yelp.com","summary":"Yelp says AI agents may use the site only through its paid data licensing program.","kind":"licensing_requirement","source":{"publisher":"TechCrunch","date":"2026-10-06","url":"https://techcrunch.com/2026/10/06/the-next-hurdle-for-ai-agents-getting-websites-to-let-them-in/"},"verified_on":"2026-10-19"},"program":null,"statements":[],"claim":{"domain":"yelp.com","company":"Yelp","claim":"Yelp requires paid data licensing for agents.","grade":"supported","grade_label":"Supported by the company","statement":"Yelp told TechCrunch that it doesn't permit non-human traffic on its platform unless the agent has paid to access Yelp's content and data through its data licensing program. Its robots.txt carries a section headed \"AI / LLM Crawlers and Agents.\"","sources":[{"publisher":"TechCrunch","date":"2026-10-06","url":"https://techcrunch.com/2026/10/06/the-next-hurdle-for-ai-agents-getting-websites-to-let-them-in/"},{"publisher":"Yelp robots.txt","date":"2026-10-19","url":"https://www.yelp.com/robots.txt"}]},"reports":[],"history":{"points":1,"first":"2026-10-19","archive_points":0,"series":[{"m":"2023-01","s":null},{"m":"2023-02","s":null},{"m":"2023-03","s":null},{"m":"2023-04","s":null},{"m":"2023-05","s":null},{"m":"2023-06","s":null},{"m":"2023-07","s":null},{"m":"2023-08","s":null},{"m":"2023-09","s":null},{"m":"2023-10","s":null},{"m":"2023-11","s":null},{"m":"2023-12","s":null},{"m":"2024-01","s":null},{"m":"2024-02","s":null},{"m":"2024-03","s":null},{"m":"2024-04","s":null},{"m":"2024-05","s":null},{"m":"2024-06","s":null},{"m":"2024-07","s":null},{"m":"2024-08","s":null},{"m":"2024-09","s":null},{"m":"2024-10","s":null},{"m":"2024-11","s":null},{"m":"2024-12","s":null},{"m":"2025-01","s":null},{"m":"2025-02","s":null},{"m":"2025-03","s":null},{"m":"2025-04","s":null},{"m":"2025-05","s":null},{"m":"2025-06","s":null},{"m":"2025-07","s":null},{"m":"2025-08","s":null},{"m":"2025-09","s":null},{"m":"2025-10","s":null},{"m":"2025-11","s":null},{"m":"2025-12","s":null},{"m":"2026-01","s":null},{"m":"2026-02","s":null},{"m":"2026-03","s":null},{"m":"2026-04","s":null},{"m":"2026-05","s":null},{"m":"2026-06","s":null},{"m":"2026-07","s":null},{"m":"2026-08","s":null},{"m":"2026-09","s":null},{"m":"2026-10","s":"blocked","l":{"training":"none","search":"none","user_directed":"none"},"n":24}]},"events":[]}