Which websites block AI?
AI tools like ChatGPT, Claude and Perplexity read websites to learn, to answer questions and, more and more, to shop for you. Some sites welcome them. Others are shutting them out. Every day we send our own bot to 170 of the biggest sites to see which is which.
Any of the 170 we track, or any other site to check it now. How we check
47of 170
sites tell at least one AI company not to train its AI on their pages
13of 170
sites tell every major AI bot to stay out completely
48of 170
homepages turned our bot away or made it prove it was human, whatever their rules say
We couldn't read the rules of 11 sites or visit 20 homepages. Last check 24 Oct 2026.
Who blocks AI training, by type of site
Sites of each type that tell at least one AI training bot to stay out today. Show the full chart
The wall, month by month
Each column is one website, grouped by type. Each row is one AI bot. A solid brick means that site's rules tell that bot to stay out of the whole site. Press play to watch the wall go up since Jan 2023.
Numbers over each group: sites that block at least one AI bot that month. Older months come from the Internet Archive. How this works
How we check
What we read, and how. Everything we find is public and linked.
Websites post rules for bots
Almost every website has a public file called robots.txt that tells bots where they may and may not go. Anyone can read it. Here is the part of The New York Times' file that tells OpenAI's bot to stay out:
nytimes.com/robots.txtOpen the real file232User-agent: GPTBot 233Disallow: /
"User-agent" names the bot. "Disallow: /" means the whole site is off limits. We read those rules every day
We read them for 170 big sites, like Amazon, Reddit and The New York Times. Old copies saved by the Internet Archive take us back to Jan 2023.
Then we knock on the front door
robots.txt is a polite request, not a lock. 5 of the 7 AI assistants we track come from companies that say they may ignore robots.txt when a person asks for a page (OpenAI, Perplexity, Google, Meta, Amazon). So we also visit each homepage with a bot that says who it is, and write down whether the site lets it in, turns it away, or asks it to prove it is human.
Three kinds of AI bots
Sites often treat these differently, so the table shows each one.
AI training
Collects pages to train AI models like ChatGPT and Claude. Blocking these keeps a site out of future training. It does not stop AI from reading the site to answer questions.
Today 47 of 170 sites block at least one of these, and 12 block all of them.
AI search
Fetches pages so AI tools can quote and link to them in answers. Blocking these usually means the site stops showing up as a source.
Today 41 of 170 sites block at least one of these, and 7 block all of them.
AI assistants
Visits a page the moment a person asks an AI to, for example to check a price or book a table. Several AI companies say these visits may ignore robots.txt.
Today 33 of 170 sites block at least one of these, and 10 block all of them.
All 170 sites
Overall is our one-word summary. Training, Search and Assistants show what the site's rules say to each kind of AI bot: all blocked, some bots blocked, allowed. Let our bot in? is what happened when we visited the homepage. Click any site for the full story.
170 sites
| TrainingSearchAssistants | |||||
|---|---|---|---|---|---|
| Amazonamazon.comfact-checked | Retail | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Let in | Jun 2026 |
| Walmartwalmart.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Targettarget.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Best Buybestbuy.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | May to Oct 2026 |
| eBayebay.com | Retail | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Allowed | Turned away | May to Oct 2026 |
| Etsyetsy.com | Retail | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| Costcocostco.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| The Home Depothomedepot.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Lowe'slowes.com | Retail | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Turned away | · |
| Wayfairwayfair.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Macy'smacys.com | Retail | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | May 2026 |
| Nordstromnordstrom.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | May to Oct 2026 |
| IKEAikea.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | Sep 2025 |
| Sheinshein.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Temutemu.com | Retail | Allows AIa few smaller bots blocked | Training: Some bots blockedSearch: AllowedAssistants: Allowed | Let in | Sep 2025 |
| AliExpressaliexpress.com | Retail | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Allowed | No answer | Oct 2026 |
| Chewychewy.com | Retail | Blocks some AI | Training: AllowedSearch: Some bots blockedAssistants: Allowed | Turned away | Sep 2025 |
| Nikenike.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Instacartinstacart.com | Retail | Blocks some AI | Training: All blockedSearch: Some bots blockedAssistants: Some bots blocked | Didn't tryrules ask our bot to stay out | May 2026 |
| Krogerkroger.com | Retail | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Didn't tryrules file did not load | · |
| Sephorasephora.com | Retail | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Turned away | · |
| Zapposzappos.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Allbirdsallbirds.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Gymsharkgymshark.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Brooklinenbrooklinen.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Rothy'srothys.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Death Wish Coffeedeathwishcoffee.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| ColourPopcolourpop.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Bombasbombas.com | Retail | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | · |
| Fashion Novafashionnova.com | Retail | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Delta Air Linesdelta.comfact-checked | Travel | Blocks some AIcompany statement | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| United Airlinesunited.com | Travel | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Didn't tryrules file did not load | · |
| American Airlinesaa.com | Travel | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | Aug 2025 |
| Southwest Airlinessouthwest.com | Travel | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Alaska Airlinesalaskaair.com | Travel | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| JetBluejetblue.com | Travel | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| British Airwaysbritishairways.com | Travel | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Didn't tryrules file did not load | · |
| Lufthansalufthansa.com | Travel | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | · |
| Air Canadaaircanada.com | Travel | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Marriottmarriott.com | Travel | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | Feb 2025 |
| Hiltonhilton.com | Travel | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | · |
| Hyatthyatt.com | Travel | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Turned away | · |
| IHG Hotelsihg.com | Travel | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | Nov 2025 |
| Airbnbairbnb.com | Travel | Blocks some AI | Training: Some bots blockedSearch: AllowedAssistants: Allowed | Let in | Oct 2026 |
| Booking.combooking.com | Travel | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Expediaexpedia.com | Travel | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | Sep 2025 |
| KAYAKkayak.com | Travel | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | Apr 2023 |
| Tripadvisortripadvisor.com | Travel | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Allowed | Human check | Feb 2026 |
| Vrbovrbo.com | Travel | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | Sep 2025 |
| Hotels.comhotels.com | Travel | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Didn't tryrules file did not load | Mar 2026 |
| Amtrakamtrak.com | Travel | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| JPMorgan Chasechase.comfact-checked | Finance | Blocks some AIcompany statement | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Charles Schwabschwab.comfact-checked | Finance | Blocks AIcompany statement | Training: UnknownSearch: UnknownAssistants: Unknown | Turned away | · |
| Bank of Americabankofamerica.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Wells Fargowellsfargo.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Citiciti.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Capital Onecapitalone.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | May to Oct 2026 |
| American Expressamericanexpress.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| U.S. Bankusbank.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Discoverdiscover.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Fidelityfidelity.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Vanguardvanguard.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| E*TRADEetrade.com | Finance | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Didn't tryrules file did not load | · |
| Robinhoodrobinhood.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| SoFisofi.com | Finance | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| PayPalpaypal.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Venmovenmo.com | Finance | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | Jul 2024 |
| Coinbasecoinbase.com | Finance | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| Yelpyelp.comfact-checked | Local and reviews | Charges AI | Training: All blockedSearch: All blockedAssistants: All blocked | Didn't tryrules ask our bot to stay out | · |
| OpenTableopentable.com | Local and reviews | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Didn't tryrules file did not load | Aug 2025 |
| DoorDashdoordash.com | Local and reviews | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| Uber Eatsubereats.com | Local and reviews | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| Grubhubgrubhub.com | Local and reviews | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Angiangi.com | Local and reviews | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | Aug 2024 |
| Thumbtackthumbtack.com | Local and reviews | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | Apr 2024 |
| Nextdoornextdoor.com | Local and reviews | Blocks AI | Training: All blockedSearch: All blockedAssistants: All blocked | Didn't tryrules ask our bot to stay out | Sep 2023 |
| Zillowzillow.com | Local and reviews | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Allowed | Let in | Apr 2026 |
| Realtor.comrealtor.com | Local and reviews | Blocks some AI | Training: Some bots blockedSearch: AllowedAssistants: Some bots blocked | Turned away | Nov 2024 |
| Redfinredfin.com | Local and reviews | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Craigslistcraigslist.org | Local and reviews | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | Dec 2025 to Jul 2026 |
| Ticketmasterticketmaster.com | Local and reviews | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | Jul 2025 |
| Eventbriteeventbrite.com | Local and reviews | Blocks some AI | Training: Some bots blockedSearch: AllowedAssistants: Allowed | Let in | · |
| Indeedindeed.com | Local and reviews | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | Apr to Sep 2026 |
| Glassdoorglassdoor.com | Local and reviews | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Allowed | Human check | May to Oct 2026 |
| The New York Timesnytimes.com | News | Blocks AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Human check | Apr 2026 |
| The Washington Postwashingtonpost.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | No answer | Aug 2025 |
| The Wall Street Journalwsj.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Didn't tryrules ask our bot to stay out | Jun 2026 |
| The Guardiantheguardian.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Let in | · |
| BBCbbc.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Let in | Jan 2026 |
| CNNcnn.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Let in | Oct 2026 |
| Reutersreuters.com | News | Blocks some AI | Training: All blockedSearch: Some bots blockedAssistants: Some bots blocked | Didn't tryrules ask our bot to stay out | Sep 2026 |
| Associated Pressapnews.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Human check | Aug 2025 |
| Bloombergbloomberg.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Human check | Oct 2026 |
| Financial Timesft.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Human check | May 2025 |
| The Atlantictheatlantic.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Turned away | Apr 2026 |
| The New Yorkernewyorker.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Let in | Oct 2025 |
| NPRnpr.org | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Didn't tryrules file did not load | Oct 2026 |
| USA Todayusatoday.com | News | Charges AI | Training: Some bots blockedSearch: Some bots blockedAssistants: All blocked | Asked to pay | Apr to Jun 2026 |
| Fox Newsfoxnews.com | News | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Axiosaxios.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Allowed | Human check | May to Jul 2026 |
| Politicopolitico.com | News | Blocks AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Human check | Sep 2025 |
| The Vergetheverge.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Let in | Jun 2026 |
| WIREDwired.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Let in | Oct 2025 |
| The Economisteconomist.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Human check | Aug 2026 |
| Los Angeles Timeslatimes.com | News | Blocks some AI | Training: Some bots blockedSearch: AllowedAssistants: Allowed | Let in | Aug 2025 |
| Mediummedium.com | News | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Allowed | Human check | Sep 2025 |
| Substacksubstack.com | News | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Redditreddit.com | Social | Blocks AI | Training: All blockedSearch: All blockedAssistants: All blocked | Didn't tryrules ask our bot to stay out | Feb 2026 |
| Xx.com | Social | Blocks AI | Training: All blockedSearch: All blockedAssistants: All blocked | Didn't tryrules ask our bot to stay out | Sep 2026 |
| Facebookfacebook.com | Social | Blocks AI | Training: All blockedSearch: All blockedAssistants: All blocked | Didn't tryrules ask our bot to stay out | Oct 2026 |
| Instagraminstagram.com | Social | Blocks AI | Training: All blockedSearch: All blockedAssistants: All blocked | Didn't tryrules ask our bot to stay out | Jan 2025 to Apr 2026 |
| LinkedInlinkedin.com | Social | Blocks some AI | Training: All blockedSearch: Some bots blockedAssistants: All blocked | Didn't tryrules ask our bot to stay out | Mar to Jul 2026 |
| TikToktiktok.com | Social | Blocks AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Let in | Oct 2025 |
| YouTubeyoutube.com | Social | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Pinterestpinterest.com | Social | Blocks AI | Training: All blockedSearch: Some bots blockedAssistants: All blocked | Didn't tryrules ask our bot to stay out | Oct 2026 |
| Quoraquora.com | Social | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Some bots blocked | Human check | · |
| Tumblrtumblr.com | Social | Blocks some AI | Training: Some bots blockedSearch: Some bots blockedAssistants: Allowed | Let in | Sep 2024 |
| Discorddiscord.com | Social | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Threadsthreads.com | Social | Blocks AI | Training: All blockedSearch: All blockedAssistants: All blocked | Didn't tryrules ask our bot to stay out | Sep 2024 to May 2025 |
| Blueskybsky.app | Social | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Stack Overflowstackoverflow.com | Social | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | Feb 2026 |
| Twitchtwitch.tv | Social | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | May to Jul 2026 |
| GitHubgithub.com | Developer | Allows AIa few smaller bots blocked | Training: Some bots blockedSearch: AllowedAssistants: Allowed | Let in | Oct 2026 |
| GitLabgitlab.com | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| MDN Web Docsdeveloper.mozilla.org | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| npmnpmjs.com | Developer | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Human check | · |
| PyPIpypi.org | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Vercelvercel.com | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | 21 Oct 2026 |
| Stripe Docsdocs.stripe.com | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | May 2026 |
| Cloudflare Docsdevelopers.cloudflare.com | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | Oct 2025 |
| Supabasesupabase.com | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | 21 Oct 2026 |
| Hugging Facehuggingface.co | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Next.jsnextjs.org | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Sveltesvelte.dev | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Mintlifymintlify.com | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Dockerdocker.com | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Anthropicanthropic.com | Developer | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| OpenAIopenai.com | Developer | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| Wikipediawikipedia.org | Reference | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Britannicabritannica.com | Reference | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| Internet Archivearchive.org | Reference | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| IMDbimdb.com | Reference | Blocks AI | Training: All blockedSearch: Some bots blockedAssistants: Some bots blocked | Didn't tryrules ask our bot to stay out | May to Sep 2026 |
| Merriam-Webstermerriam-webster.com | Reference | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| Fandomfandom.com | Reference | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Human check | · |
| USA.govusa.gov | Government | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| IRSirs.gov | Government | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Social Security Administrationssa.gov | Government | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | · |
| CDCcdc.gov | Government | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| NIHnih.gov | Government | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| Medicare.govmedicare.gov | Government | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| HealthCare.govhealthcare.gov | Government | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| GOV.UKgov.uk | Government | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | May 2026 |
| Canada.cacanada.ca | Government | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| FTCftc.gov | Government | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Mayo Clinicmayoclinic.org | Healthcare | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | · |
| Cleveland Clinicclevelandclinic.org | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Johns Hopkins Medicinehopkinsmedicine.org | Healthcare | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | · |
| Kaiser Permanentekp.org | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Mount Sinaimountsinai.org | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | Oct 2026 |
| UnitedHealthcareuhc.com | Healthcare | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Turned away | · |
| Aetnaaetna.com | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Cignacigna.com | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Anthemanthem.com | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| Humanahumana.com | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | Oct 2026 |
| CVS Healthcvs.com | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | May 2024 |
| Walgreenswalgreens.com | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
| WebMDwebmd.com | Healthcare | Blocks some AI | Training: Some bots blockedSearch: AllowedAssistants: Allowed | Let in | Sep 2026 |
| GoodRxgoodrx.com | Healthcare | Blocks some AIat the front door | Training: AllowedSearch: AllowedAssistants: Allowed | Human check | Dec 2024 |
| Zocdoczocdoc.com | Healthcare | Unknown | Training: UnknownSearch: UnknownAssistants: Unknown | Human check | Feb 2026 |
| MyChart (Epic)mychart.org | Healthcare | Allows AI | Training: AllowedSearch: AllowedAssistants: Allowed | Let in | · |
Recent changes
All changes →- Oct 2026Airbnb Now blocked: GPTBot, ClaudeBot, Applebot-Extended, MistralAI-Training and AI2Bot.
- Oct 2026AliExpress Now blocked: GPTBot, ClaudeBot, Google-Extended, CCBot, Bytespider and Amazonbot.
- Oct 2026Bloomberg Now partly blocked: OAI-SearchBot, Claude-SearchBot, ChatGPT-User and Claude-User.
- Oct 2026CNN Now allowed: Google-Extended.
- Oct 2026Facebook Now blocked: GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, PerplexityBot and Amazonbot.
- May to Oct 2026Glassdoor Now blocked: ClaudeBot.
10 sites let one or two AI companies in and keep the rest out
These sites name one or two AI companies' bots and let them in, while blocking bots from at least three other companies.
| Site | Lets in | Keeps out | Since |
|---|---|---|---|
| Axiosaxios.com | Google Google-Extended | 3 bots from 3 other companies | Mar 2026 |
| The New York Timesnytimes.com | Amazon Amazonbot | 16 bots from 9 other companies | Oct 2025 |
| The Vergetheverge.com | Amazon Amazonbot | 15 bots from 11 other companies | May 2025 |
| The Atlantictheatlantic.com | OpenAI GPTBot, OAI-SearchBot, ChatGPT-User DuckDuckGo DuckAssistBot |
13 bots from 9 other companies | Jul 2024 |
| The Wall Street Journalwsj.com | OpenAI GPTBot, OAI-SearchBot, ChatGPT-User | 21 bots from 11 other companies | Jun 2024 |
| Airbnbairbnb.com | Meta Meta-ExternalAgent Perplexity PerplexityBot |
5 bots from 5 other companies | before 2023, or no record |
| Tripadvisortripadvisor.com | Perplexity PerplexityBot | 8 bots from 8 other companies | before 2023, or no record |
| Pinterestpinterest.com | DuckDuckGo DuckAssistBot | 23 bots from 11 other companies | before 2023, or no record |
| Quoraquora.com | Google Google-Extended Amazon Amazonbot |
9 bots from 5 other companies | before 2023, or no record |
| IMDbimdb.com | Amazon Amazonbot, Amzn-SearchBot, Amzn-User DuckDuckGo DuckAssistBot |
20 bots from 10 other companies | before 2023, or no record |
Take part
This is an independent project. It takes no money from the sites it tracks or from AI companies.