{"domain":"latimes.com","name":"Los Angeles Times","category":"news","subcategory":"regional","country":"US","tags":[],"notes":"","status":"partial","status_label":"Blocks some AI","rationale":"Its robots.txt (the public rules file for bots) treats AI bots differently from each other. It blocks 3 of the 24 AI bots we track from the whole site, including GPTBot, ClaudeBot and Bytespider. Google-Extended, Applebot-Extended, CCBot, Meta-ExternalAgent and 6 more can reach most of it.","computed_at":"2026-10-20T03:43:15.318Z","last_checked":"2026-10-19T13:36:15.733Z","checker":{"allowed":true,"vantage":"residential connection, United States (manual run)","origin":"https://www.latimes.com"},"robots":{"via":{"type":"live","date":"2026-10-19","url":"https://www.latimes.com/robots.txt"},"kind":"file","http":200,"error":null,"url":"https://www.latimes.com/robots.txt","hops":[{"from":"https://latimes.com/robots.txt","to":"https://www.latimes.com/robots.txt","status":301}],"blob":"6dc754f82f55902704f5e1cdd1f406e0f91246ce2238c92e9634f3335f57aab7","text":"User-agent: *\nDisallow: /search\nDisallow: /recipes/search\nDisallow: /changebrowser\nDisallow: /thirdpartyservice\nDisallow: /*/thirdpartyservice\nDisallow: /config/\nDisallow: /*/config\nDisallow: /dzcfg\nDisallow: /*/dzcfg\nDisallow: /*/test-template\nDisallow: /test/\nDisallow: /includes/\nDisallow: /_comments/\nDisallow: /deeplinkid/\nDisallow: /searchsuggest\nDisallow: /*/searchsuggest\nDisallow: /svgimageproc\nDisallow: /hive/\nDisallow: /get-galleryfragment*\nDisallow: /gs-*\n\nUser-agent: Googlebot-News\nDisallow: /*-ugc-*.html\nDisallow: /about/\nDisallow: /cbp-*.html\nDisallow: /*/cbp-*.html\nDisallow: /bp/*\nDisallow: /*-adstory.html\nDisallow: /*sns-ap\nDisallow: /*photogallery\nDisallow: /brandpublishing/\nDisallow: /*/brandpublishing/\nDisallow: /shopping/\nDisallow: /*/shopping/\nDisallow: /paid-posts/\nDisallow: /*/paid-posts/\nDisallow: /search\nDisallow: /*/search\nDisallow: /changebrowser\nDisallow: /thirdpartyservice\nDisallow: /*/thirdpartyservice\nDisallow: /config/\nDisallow: /*/config\nDisallow: /dzcfg\nDisallow: /*/dzcfg\nDisallow: /*/test-template\nDisallow: /test/\nDisallow: /_comments/\nDisallow: /deeplinkid/\nDisallow: /searchsuggest\nDisallow: /*/searchsuggest\nDisallow: /svgimageproc\nDisallow: /hive/\nDisallow: /get-galleryfragment*\nDisallow: /sponsored/\nDisallow: /*/sponsored/\n\nUser-agent: Twitterbot\nAllow: /\n\nUser-agent: Mediapartners-Google\nAllow: /\n\nUser-agent: AmazonAdBot\nAllow: /\n\nUser-agent: Bytespider\nDisallow: /\n\nUser-agent: ClaudeBot\nDisallow: /\n\nUser-agent: Diffbot\nDisallow: /\n\nUser-agent: discobot\nDisallow: /\n\nUser-agent: GPTBot\nDisallow: /\n\nUser-agent: ImagesiftBot\nDisallow: /\n\nUser-agent: omgili\nDisallow: /\n\nUser-agent: omgilibot\nDisallow: /\n\nSitemap: https://www.latimes.com/sitemaps/sitemap.xml\nSitemap: https://www.latimes.com/news-sitemap.xml","refused":null,"warnings":[],"contentSignals":[],"contentUsage":[],"licenses":[],"aiComments":[],"platform":null,"wildcard":{"exists":true,"access":"full"},"named":["*","Googlebot-News","Twitterbot","Mediapartners-Google","AmazonAdBot","Bytespider","ClaudeBot","Diffbot","discobot","GPTBot","ImagesiftBot","omgili","omgilibot"],"lines":93},"agents":[{"token":"GPTBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":81},"any":{"allowed":false,"line":81},"search":{"allowed":false,"line":81},"cart":{"allowed":false,"line":81},"checkout":{"allowed":false,"line":81},"account":{"allowed":false,"line":81}},"disallows":1,"lines":[80,81],"groupRanges":[[80,81]]},{"token":"ClaudeBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":72},"any":{"allowed":false,"line":72},"search":{"allowed":false,"line":72},"cart":{"allowed":false,"line":72},"checkout":{"allowed":false,"line":72},"account":{"allowed":false,"line":72}},"disallows":1,"lines":[71,72],"groupRanges":[[71,72]]},{"token":"Google-Extended","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Applebot-Extended","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"CCBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Meta-ExternalAgent","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Bytespider","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":69},"any":{"allowed":false,"line":69},"search":{"allowed":false,"line":69},"cart":{"allowed":false,"line":69},"checkout":{"allowed":false,"line":69},"account":{"allowed":false,"line":69}},"disallows":1,"lines":[68,69],"groupRanges":[[68,69]]},{"token":"MistralAI-Training","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"AI2Bot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"anthropic-ai","deprecated":true,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"OAI-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Claude-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"PerplexityBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Amazonbot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Amzn-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"DuckAssistBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"MistralAI-Index","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Meta-WebIndexer","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"ChatGPT-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Claude-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Perplexity-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Google-Agent","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Meta-ExternalFetcher","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Amzn-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"MistralAI-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]}],"refs":[{"token":"Googlebot","matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]},{"token":"Bingbot","matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":false,"line":2},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":20,"lines":[1,21],"groupRanges":[[1,21]]}],"lanes":{"training":{"level":"mixed","none":3,"full":6,"restricted":0,"n":9},"search":{"level":"full","none":0,"full":8,"restricted":0,"n":8},"user_directed":{"level":"full","none":0,"full":7,"restricted":0,"n":7}},"door":{"door":"open","http":200},"edge":{"cdn":["Amazon CloudFront"],"botManagement":[]},"llms":{"state":"absent","http":404,"url":"https://www.latimes.com/llms.txt"},"ai_txt":{"state":"absent","http":404,"url":"https://www.latimes.com/ai.txt"},"tdmrep":{"state":"absent","http":404,"url":"https://www.latimes.com/.well-known/tdmrep.json"},"meta":{"title":"News from California, across the nation and world - Los Angeles Times","robotsMeta":[],"noai":[],"noindex":false,"jsonLdTypes":["WebPage","Organization","ImageObject"],"platform":null,"textRatio":0.015,"llmsLink":false},"x_robots_tag":null,"tos":{"url":"https://www.latimes.com/terms-of-service","source":"discovered","http":200,"hash":"bebcf985cc860f3be48c55de067e35409215d039cb643d73764fa80567474816","chars":47727,"thin":false,"clauses":[{"keyword":"scrape","prohibition":true,"excerpt":"You may not scrape or otherwise copy our Content without our permission."},{"keyword":"data mining","prohibition":true,"excerpt":"…through the Site nor to insert any code or product or manipulate the Content or the Site in any way, and not to use any data mining, data gathering or extraction method."},{"keyword":"automated means","prohibition":true,"excerpt":"Collects any user content or information, or otherwise accesses the Site using automated means (such as harvesting bots, robots, spiders, or scrapers) without our prior permission;"},{"keyword":"agent","prohibition":true,"excerpt":"… statement by you that you have a good faith belief that the disputed use is not authorized by the copyright owner, its agent, or the law;"},{"keyword":"agents","prohibition":true,"excerpt":"…partners, suppliers, licensors, officers, directors, shareholders, members, employees, representatives, contractors and agents, and sub-licensees from any and all claims (including but not limited to claims for defamation, trade disparagement, privacy and intellectual property infringement) and dama…"},{"keyword":"robot","prohibition":false,"excerpt":"Violates any robot exclusion headers of the Site, if any, or bypasses or circumvents other measures employed to prevent or limit access to the Site;"},{"keyword":"scrape","prohibition":false,"excerpt":"Attempts to scrape or collect any personal or private information from other users or from the Site;"}],"door":{"door":"open","http":200}},"paid":null,"program":null,"statements":[],"claim":null,"reports":[],"history":{"points":44,"first":"2023-01-01","archive_points":43,"series":[{"m":"2023-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-09","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2023-10","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2023-11","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2023-12","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-01","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-02","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-03","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-04","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-05","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-06","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-07","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-08","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-09","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-10","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-11","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-12","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2025-01","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2025-02","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2025-03","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2025-04","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2025-05","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2025-06","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2025-07","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2025-08","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2025-09","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2025-10","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2025-11","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2025-12","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-01","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-02","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-03","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-04","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-05","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-06","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-07","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-08","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-09","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3},{"m":"2026-10","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":3}]},"events":[{"domain":"latimes.com","name":"Los Angeles Times","category":"news","date":"2025-08","precision":"month","source":"archive","provisional":false,"between":["2025-07-01","2025-08-01"],"summary":"Now blocked: ClaudeBot and Bytespider.","dir":"up","up":["ClaudeBot","Bytespider"],"down":[],"major":true,"signals":false,"status_from":"partial","status_to":"partial","evidence":["https://web.archive.org/web/20250701020919/https://www.latimes.com/robots.txt","https://web.archive.org/web/20250801061405/https://www.latimes.com/robots.txt"],"diff":[{"t":"ctx","s":"Disallow: /test/"},{"t":"add","s":"Disallow: /includes/"},{"t":"add","s":"Disallow: /_comments/"},{"t":"ctx","s":"Disallow: /topic/*?"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /test/"},{"t":"add","s":"Disallow: /_comments/"},{"t":"ctx","s":"Disallow: /topic/*?"},{"t":"gap","s":"…"},{"t":"ctx","s":""},{"t":"add","s":"User-agent: Bytespider"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"add","s":"User-agent: ClaudeBot"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"add","s":"User-agent: Diffbot"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"ctx","s":"User-agent: discobot"},{"t":"gap","s":"…"},{"t":"ctx","s":""},{"t":"del","s":"Sitemap: https://www.latimes.com/sitemap.xml"},{"t":"add","s":"User-agent: ImagesiftBot"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"add","s":"User-agent: omgili"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"add","s":"User-agent: omgilibot"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"add","s":"Sitemap: https://www.latimes.com/sitemaps/sitemap.xml"},{"t":"ctx","s":"Sitemap: https://www.latimes.com/news-sitemap.xml"}]},{"domain":"latimes.com","name":"Los Angeles Times","category":"news","date":"2023-09","precision":"month","source":"archive","provisional":false,"between":["2023-08-01","2023-09-01"],"summary":"Now blocked: GPTBot.","dir":"up","up":["GPTBot"],"down":[],"major":true,"signals":false,"status_from":"allowed","status_to":"partial","evidence":["https://web.archive.org/web/20230801012552/https://www.latimes.com/robots.txt","https://web.archive.org/web/20230901004036/https://www.latimes.com/robots.txt"],"diff":[{"t":"ctx","s":"Disallow: /recipes/search"},{"t":"del","s":"Disallow: /*/search"},{"t":"ctx","s":"Disallow: /changebrowser"},{"t":"gap","s":"…"},{"t":"ctx","s":""},{"t":"add","s":"User-agent: GPTBot"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"ctx","s":"Sitemap: https://www.latimes.com/sitemap.xml"}]}]}