The Walled Web Newsletter

All sites / Local, listings and reviews

Zillow

zillow.com
Blocks some

Its robots.txt (the public rules file for bots) treats AI bots differently from each other. It blocks 4 of the 24 AI bots we track from the whole site, including Applebot-Extended, Meta-ExternalAgent, AI2Bot and Amazonbot. GPTBot, ClaudeBot, Google-Extended, CCBot and 6 more can reach most of it.

Last checked 19 Oct 2026. Rules file read today. Everything we checked · History

Bots that train AI Some blocked

3 of 9 blocked

Blocking these keeps the site out of future AI training. It does not stop AI from reading the site to answer questions.

Bots for AI search Some blocked

1 of 8 blocked

Blocking these usually means the site stops showing up as a source in AI answers.

AI assistants Allowed

0 of 7 blocked

5 of these 7 companies say their assistants may ignore robots.txt.

Bot by bot

AI botAllowed?Where the rule isCompany says it follows robots.txt?
Bots that train AI · Some blocked
GPTBotOpenAI Allowed Not named; the catch-all rule applies Says yes
ClaudeBotAnthropic Allowed Not named; the catch-all rule applies Says yes
Google-ExtendedGoogle · opt-out name, not a bot Allowed Not named; the catch-all rule applies Says yes
Applebot-ExtendedApple · opt-out name, not a bot Blocked Named, line 268 Says yes
CCBotCommon Crawl Allowed Not named; the catch-all rule applies Says yes
Meta-ExternalAgentMeta Blocked Named, line 268 Suggests yes
BytespiderByteDance Allowed Not named; the catch-all rule applies No statement found
MistralAI-TrainingMistral AI Allowed Not named; the catch-all rule applies Says yes
AI2BotAllen Institute for AI Blocked Named, line 268 Does not say
anthropic-aiAnthropic (legacy) · old name Allowed Not named; the catch-all rule applies Unknown
Bots for AI search · Some blocked
OAI-SearchBotOpenAI Allowed Not named; the catch-all rule applies Says yes
Claude-SearchBotAnthropic Allowed Not named; the catch-all rule applies Says yes
PerplexityBotPerplexity Allowed Not named; the catch-all rule applies Suggests yes
AmazonbotAmazon Blocked Named, line 268 Says yes
Amzn-SearchBotAmazon Allowed Not named; the catch-all rule applies Says yes
DuckAssistBotDuckDuckGo Allowed Not named; the catch-all rule applies Says yes
MistralAI-IndexMistral AI Allowed Not named; the catch-all rule applies Suggests yes
Meta-WebIndexerMeta Allowed Not named; the catch-all rule applies Suggests yes
AI assistants · Allowed
ChatGPT-UserOpenAI Allowed Not named; the catch-all rule applies Says not always
Claude-UserAnthropic Allowed Not named; the catch-all rule applies Says yes
Perplexity-UserPerplexity Allowed Not named; the catch-all rule applies Says it mostly ignores it
Google-AgentGoogle Allowed Not named; the catch-all rule applies Says it mostly ignores it
Meta-ExternalFetcherMeta Allowed Not named; the catch-all rule applies Says not always
Amzn-UserAmazon Allowed Not named; the catch-all rule applies Says not always
MistralAI-UserMistral AI Allowed Not named; the catch-all rule applies Suggests yes
For comparison
GooglebotRegular search engine, for comparisonAllowedNot named; the catch-all rule applies
BingbotRegular search engine, for comparisonAllowedNot named; the catch-all rule applies

Other AI names in this file

Names in this file that are not among the 24 bots we track. A rule for a name no real bot uses blocks nothing, but it shows which names the site has added.

YouBotBlockedline 268Not in our verified list
AI2Bot-DeepResearchEvalBlockedline 275Not in our verified list
Ai2Bot-DolmaBlockedline 276Not in our verified list

We test two pages: the homepage, and a made-up page that stands in for the rest of the site. "Blocked" means both are off limits. "Some pages blocked" means only one is. Rules for search results, carts, checkout and accounts don't change these labels, because almost every site keeps all bots out of those pages.

The rules file (robots.txt)

robots.txt · 329 lines · 35 lines about AI bots Source
# Access to and use of Zillow.com is governed by our Terms of Use. See https://www.zillow.com/corp/Terms.htm User-agent: * Allow: /homes/for_sale/$Allow: /homes/for_sale/fore_lt/$Allow: /homes/fsbo/$Allow: /homes/for_sale/1_open/$Allow: /homes/new_homes/$Allow: /homes/coming_soon/$Allow: /homes/for_rent/$Allow: /homes/for_rent/condo,apartment_duplex_type/$Allow: /homes/for_rent/house,townhouse_type/$Allow: /homes/recently_sold/$Allow: /homes/apartment-communities/$ Allow: /homes/for_sale/*_p/$Allow: /homes/for_sale/fore_lt/*_p/$Allow: /homes/fsbo/*_p/$Allow: /homes/for_sale/1_open/*_p/$Allow: /homes/new_homes/*_p/$Allow: /homes/coming_soon/*_p/$Allow: /homes/for_rent/*_p/$Allow: /homes/for_rent/condo,apartment_duplex_type/*_p/$Allow: /homes/for_rent/house,townhouse_type/*_p/$Allow: /homes/recently_sold/*_p/$Allow: /homes/apartment-communities/*_p/$ Allow: /homes/apartments-pet-friendly/$Allow: /homes/apartments-1-bedrooms/$Allow: /homes/apartments-2-bedrooms/$Allow: /homes/apartments-under-1500/$Allow: /homes/apartments-under-2000/$Allow: /homes/apartments-3-bedrooms/$Allow: /homes/pet-friendly/$Allow: /homes/rooms-for-rent/$Allow: /homes/apartments-with-washer-dryer/$ Allow: /homes/apartments-pet-friendly/*_p/$Allow: /homes/apartments-1-bedrooms/*_p/$Allow: /homes/apartments-2-bedrooms/*_p/$Allow: /homes/apartments-under-1500/*_p/$Allow: /homes/apartments-under-2000/*_p/$Allow: /homes/apartments-3-bedrooms/*_p/$Allow: /homes/pet-friendly/*_p/$Allow: /homes/rooms-for-rent/*_p/$Allow: /homes/apartments-with-washer-dryer/*_p/$ Disallow: /api/Disallow: /homes/Disallow: /?url=Disallow: /*_rectDisallow: /*_rect/Disallow: /*_ridDisallow: /*_sort/Disallow: /*.sdirectDisallow: /*1_fs/Disallow: /*3ab9b0d/Disallow: /*3bec89bDisallow: /*/_type/Disallow: /*/0-0_beds/Disallow: /*/admin-ajax.phpDisallow: /*/apartments-are_att/Disallow: /*/apartments-madden_att/Disallow: /*/apartments-offers_att/Disallow: /*/apartments-rent_att/Disallow: /*cbpartner=Disallow: /*/CountyAssessorPage.htm?*Disallow: /*/duplex-apartments_att/Disallow: /*/floor-map-annotator/*Disallow: /*/foreclosure--Disallow: /*/foreclosed/*Disallow: /*/GetResults.htm*Disallow: /*/house_type/Disallow: /*/house,apartment_duplex_type/Disallow: /*/house,mobile_type/Disallow: /*/house,mobile,land_type/Disallow: /*/house,mobile,townhouse_type/Disallow: /*/loft-apartments_att/Disallow: /*/mostrecentchange_sort/Disallow: /*/node_modules/.bin/Disallow: /*/plus/Disallow: /*/rent-house_att/Disallow: /*/rino-_att/Disallow: /*/qc/Disallow: /*-qc/Disallow: /*-qc-*Disallow: /*/townhome_att/Disallow: /*/u1_sch/Disallow: /*/z/internet-tv-deals-saver/?z*Disallow: /*/z/internet-tv-deals-speed/?z*Disallow: /*RentalConfirmedClaimDisallow: /*?searchQueryState=*Disallow: /ads/Disallow: /advice-thread/Disallow: /advicepages/Disallow: /aerial/Disallow: /advice/Disallow: /agent-websites/Disallow: /ajax/homedetail/CompareWithPublicFactsData.htmDisallow: /ajax/homedetail/HomeValueChartData.htmDisallow: /ajax/nav/UserNavAsync.htm?pageframe=trueDisallow: /ajax/rentals/ClaimedRentalProperties.htm?actionType=claimsDisallow: /ajax/ownerview/PerformSellerLeadContact.htmDisallow: /ajax/ownerview/SellerLeadData.htmDisallow: /app?Disallow: /b/*/rental-manager/resources/Disallow: /b/*bed*/Disallow: /b/*month*/Disallow: /b/*!*/Disallow: /b/*(*/Disallow: /b/*&*/Disallow: /b/*/*_ll/Disallow: /bal/impression/Disallow: /bal/impression/RecordImpression.htm?Disallow: /browse/homes/all/Disallow: /captcha/Disallow: /captchaPerimeterX/Disallow: /careers/apply/Disallow: /careers/jobs/Disallow: /chat/financeDisallow: /claiming/Disallow: /claiming/ClaimProperty.htm?Disallow: /cma-tool/create/*Disallow: /cms-constellation/Disallow: /community/error-subdivision-name-unavailable/Disallow: /contact/Disallow: /corp/Privacy.htmDisallow: /corp/Terms.htmDisallow: /CountyAssessorPage.htmDisallow: /del/Disallow: /discovery/Disallow: /editing/Disallow: /email/EnrollmentUnsubscribe.htmDisallow: /email/SubscriptionTypeUnsubscribe.htmDisallow: /email/Unsubscribe.htmDisallow: /for-sale-by-owner/?zpid=Disallow: /graphql/Disallow: /home-improvement-dueling-digs/Disallow: /homedetail/AuthRequired.htmDisallow: /homedetail/HomeDetail,*Border.*debug*DebugBar.*InspectorButton.link.sdirectDisallow: /homedetails/*view=ownerDisallow: /homedetails/*/rental-manager/resources/Disallow: /homedetails/*/*/?altIdDisallow: /homeimprovement/EmbeddedProfile.htmDisallow: /homepage/HomepageVideoMetrics.htmDisallow: /howto/GoodNeighborPolicy.htmDisallow: /HYx10rg3/*Disallow: /IMAGEDisallow: /imf-fetch-message/api/v1/request/fetch-message/MOBILE_APP_UPSELLDisallow: /instant-offers/version/Disallow: /jmp/questionnaire-eligibility-checkDisallow: /jmp/lead-eligibilityDisallow: /l/Disallow: /learnmore/Disallow: /license/Disallow: /m/Disallow: /make-me-move?zpid=Disallow: /make-me-move/?zpid=Disallow: /mobile/GetAdConfiguration.htmDisallow: /mobile/MobilePropertyDetails.htmDisallow: /mobile/WinmoPropertyDetails.htmDisallow: /mobile/WPPropertyDetails.htmDisallow: /mortgage/SmallMortgageLoanCalculatorWidget.htmDisallow: /mortgage/MortgageCalculatorWidgetLarge.htmDisallow: /mortgage/MortgageRateTable.htmDisallow: /mortgage/widgets/AffordabilityCalculatorWidget.htmDisallow: /mortgage/MortgageRateTableWidget.htmDisallow: /mortgage/MortgageLoanCalculatorWidget.htmDisallow: /mortgage/RateExplorerGraphWidget.htmDisallow: /myzillow/Disallow: /offers/request/Disallow: /ofr/api/v3/questionnaireDisallow: /pbflex/Disallow: /personalization/Disallow: /post-real-estate-listings?zpid=Disallow: /post-real-estate-listings/?zpid=Disallow: /post-rental-listings?zpid=Disallow: /postings/Disallow: /postings/PostForSale.htm?Disallow: /postings/PostMakeMeMoveStreamlined.htm?Disallow: /*/pre-foreclosure/*Disallow: /pricing/Disallow: /profile/*/Advice/Disallow: /profile/*/Reviews-Written/Disallow: /profile/*/photos/Disallow: /profiles/ProfileBorderTemplate,*myzillow*MyListingsTabulated.*postings*owners*OwnersProfileUpsell.*DirectLink.sdirectDisallow: /real-estate-listingsDisallow: /rental-manager/?source=topnav&itc=postbutton_topnavDisallow: /rental-manager/?zpid=Disallow: /rental-manager/properties?postingPath=trueDisallow: /rental-manager/properties/?source=*Disallow: /rental-manager/r/Disallow: /rentals/Disallow: /rentalsrevenue/Disallow: /rentalsrevenue/resume/UpdatePasswordInline*Disallow: /research/monthly-reports/Disallow: /reviews/write?s=Disallow: /reviews/write/?s=Disallow: /routing/email/Disallow: /s/Disallow: /savedhomes/Disallow: /schools/TestHistoryAsync.htmDisallow: /search/browse/HDPTree,*Border.*debug*DebugBar.*InspectorButton.link.sdirectDisallow: /search/HDPMapResults.htmDisallow: /search/SavedSearchLightboxContent.htmDisallow: /sell/?*Disallow: /sell/options/Disallow: /showcase/Click.htm?Disallow: /sitemap/catalog/Disallow: /sitemap/rss/Disallow: /sitemap/crawl-monitor/Disallow: /trk/Disallow: /ui/Disallow: /url-shortner/Disallow: /user/account/SaveSearchOptions.htmDisallow: /user/account/services/Disallow: /user/account/services/WhoAmI.htmDisallow: /user/acct/login/Disallow: /user/InlineLogin.htm?Disallow: /user/LoginDisallow: /user/RegisterDisallow: /util/ClientLogger.htmDisallow: /videodetails/Disallow: /walkscore/Disallow: /webservice/Disallow: /webtools/Disallow: /widgets/Disallow: /wikipages/Disallow: /zg-graph/Disallow: /ajax/profile-broker-websiteDisallow: /agent-resources/zgwp/app/themes/paxl/assets/fonts/stargate.ttfDisallow: /my-agent/eligibility*Disallow: /jsonp/Bdp.htm?*Disallow: /ajax/homedetail/MarkPropertyViewed*Disallow: /wiki/Help*Disallow: /*/allunits/*Disallow: /reviews/batch-request/Disallow: /myzillow/ChangePassword.htmDisallow: /editing/EditHomeFacts.htmDisallow: /user/account/Disallow: /reviews/request/Disallow: /error/PageNotFound.htmDisallow: /imf-fetch-message/Disallow: /app$Disallow: /*/node_modules/.bin/jestDisallow: /view-3d-homeDisallow: /b/*/*_p/Disallow: /rental-manager/login/?url/*Disallow: /homemedia/Disallow: /view-imx/Disallow: /view-skytour/Disallow: /view-floor-plan/Disallow: /homeloans/*zga_z_guid=Disallow: /tech/*s=Disallow: /async-create-search-page-stateDisallow: /deferredDisallow: /shopper-platform/  User-agent: facebookexternalhitUser-agent: LinkedInBotUser-agent: TwitterbotAllow: /view-imx/Allow: /view-3d-home/ … 266 lines not about AI bots# Bots with limited access to CMS resourcesUser-agent: YouBotUser-agent: YandexAdditionalUser-agent: YandexAdditionalBotUser-agent: Applebot-ExtendedUser-agent: AmazonbotUser-agent: amazon-kendraUser-agent: AI2BotUser-agent: AI2Bot-DeepResearchEvalUser-agent: Ai2Bot-DolmaUser-agent: meta-externalagentUser-agent: Meta-ExternalAgentAllow: /agent-resources/Allow: /blog/Allow: /data-science/Allow: /digs/guides/Allow: /engineering/Allow: /foreclosures/Allow: /home-buying-guide/Allow: /info/Allow: /lender-resources/Allow: /marketing/Allow: /mortgage-learning/Allow: /multifamily-knowledge-center/Allow: /promo/Allow: /public-engagement/Allow: /rent/guide/Allow: /rental-manager/resources/Allow: /resources/Allow: /research/Allow: /sellers-guide/Allow: /tech/Allow: /video/Allow: /z/Disallow: /Crawl-delay: 1 # Fully Disallowed BotsUser-agent: ActiveComplyBotUser-agent: CriteoBotUser-agent: GumGumBotUser-agent: TaraGroup Intelligent BotDisallow: / Sitemap: https://mortgageapi.zillow.com/sitemap.xml Sitemap: https://www.zillow.com/xml/indexes/us/static.xml.gz Sitemap: https://www.zillow.com/xml/indexes/us/hdp/for-sale-by-agent.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/hdp/for-sale-by-owner.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/hdp/new-construction.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/hdp/auction.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/hdp/pending.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/hdp/recently-sold.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/hdp/for-rent.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/hdp/off-market.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/hdp/other.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/bdp/buildings.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/bdp/apartments.xml.gzSitemap: https://www.zillow.com/xml/indexes/us/cdp/index.xml Sitemap: https://www.zillow.com/xml/indexes/us/srp/for-sale.xml.gz … 26 lines not about AI bots

Everything else we checked

Front doorLet in The homepage loaded normally for our bot, which says who it is (HTTP 200).GET https://www.zillow.com/ as WalledWebBot, from a residential connection in the United States
Page tags<meta name="robots" content="ALL">
Anti-bot softwareServed through Amazon CloudFront.From response headers and cookie names only. A cookie shows the product is there, not how it is set up.
Readable by AINo schema.org structured data on the homepage. Most of the page is script; an agent that does not run JavaScript sees little text. Built on Next.js.Homepage only. Useful for site owners; not part of the result.
Terms of Servicewww.zillow.com/corporate/terms-of-use/ (found on the homepage)

Sentences flagged by keyword. These are machine-selected excerpts, not a legal reading; open the source for context.

  • CIC does not use Artificial Intelligence (AI) in the collection or hygiene of criminal or housing court records.matched "AI" · reads like a restriction
  • …that you have a good faith belief that the identified use of the material is not authorized by the copyright owner, its agent, or the law and, and under penalty of perjury, that you are the copyright owner or are authorized to act on the copyright owner’s behalf in this situation.matched "agent" · reads like a restriction
  • conduct automated queries (including screen and database scraping, spiders, robots, crawlers, bypassing “captcha” or similar precautions, or any other automated activity with the purpose of obtaining information from the Services) on the Services;matched "automated queries"
  • upload invalid data, viruses, worms, or other software agents to the Services;matched "agents"
  • … defend, and hold harmless the Zillow Companies, our affiliates, and our respective directors, officers, employees, and agents from any and all claims and demands made by any third party due to or arising out of:matched "agents"
Not foundNo llms.txt, ai.txt, tdmrep.json, Content-Signal or Content-Usage lines or licensing signals (RSL License, HTTP 402).

History

40 versions on record since 1 Jan 2023

OverallJan 2023: Overall allowedFeb 2023: Overall allowedMar 2023: Overall allowedApr 2023: Overall allowedMay 2023: Overall allowedJun 2023: Overall allowedJul 2023: Overall allowedAug 2023: Overall allowedSep 2023: Overall allowedOct 2023: Overall allowedNov 2023: Overall allowedDec 2023: Overall partialJan 2024: Overall allowedFeb 2024: Overall partialMar 2024: Overall partialApr 2024: Overall partialMay 2024: Overall partialJun 2024: Overall partialJul 2024: Overall partialAug 2024: Overall partialSep 2024: Overall partialOct 2024: Overall partialNov 2024: Overall partialDec 2024: Overall partialJan 2025: Overall partialFeb 2025: Overall partialMar 2025: Overall partialApr 2025: Overall partialMay 2025: Overall partialJun 2025: Overall partialJul 2025: Overall allowedAug 2025: Overall allowedSep 2025: Overall allowedOct 2025: Overall allowedNov 2025: Overall allowedDec 2025: Overall allowedJan 2026: Overall allowedFeb 2026: Overall allowedMar 2026: Overall allowedApr 2026: Overall partialMay 2026: Overall partialJun 2026: Overall partialJul 2026: Overall partialAug 2026: Overall partialSep 2026: Overall partialOct 2026: Overall partialTrainingJan 2023: Training AllowedFeb 2023: Training AllowedMar 2023: Training AllowedApr 2023: Training AllowedMay 2023: Training AllowedJun 2023: Training AllowedJul 2023: Training AllowedAug 2023: Training AllowedSep 2023: Training AllowedOct 2023: Training AllowedNov 2023: Training AllowedDec 2023: Training Some blockedJan 2024: Training AllowedFeb 2024: Training Some blockedMar 2024: Training Some blockedApr 2024: Training Some blockedMay 2024: Training Some blockedJun 2024: Training Some blockedJul 2024: Training Some blockedAug 2024: Training Some blockedSep 2024: Training Some blockedOct 2024: Training Some blockedNov 2024: Training Some blockedDec 2024: Training Some blockedJan 2025: Training Some blockedFeb 2025: Training Some blockedMar 2025: Training Some blockedApr 2025: Training Some blockedMay 2025: Training Some blockedJun 2025: Training Some blockedJul 2025: Training AllowedAug 2025: Training AllowedSep 2025: Training AllowedOct 2025: Training AllowedNov 2025: Training AllowedDec 2025: Training AllowedJan 2026: Training AllowedFeb 2026: Training AllowedMar 2026: Training AllowedApr 2026: Training Some blockedMay 2026: Training Some blockedJun 2026: Training Some blockedJul 2026: Training Some blockedAug 2026: Training Some blockedSep 2026: Training Some blockedOct 2026: Training Some blockedSearchJan 2023: Search AllowedFeb 2023: Search AllowedMar 2023: Search AllowedApr 2023: Search AllowedMay 2023: Search AllowedJun 2023: Search AllowedJul 2023: Search AllowedAug 2023: Search AllowedSep 2023: Search AllowedOct 2023: Search AllowedNov 2023: Search AllowedDec 2023: Search AllowedJan 2024: Search AllowedFeb 2024: Search AllowedMar 2024: Search AllowedApr 2024: Search AllowedMay 2024: Search AllowedJun 2024: Search AllowedJul 2024: Search AllowedAug 2024: Search AllowedSep 2024: Search AllowedOct 2024: Search AllowedNov 2024: Search AllowedDec 2024: Search AllowedJan 2025: Search AllowedFeb 2025: Search AllowedMar 2025: Search AllowedApr 2025: Search AllowedMay 2025: Search AllowedJun 2025: Search AllowedJul 2025: Search AllowedAug 2025: Search AllowedSep 2025: Search AllowedOct 2025: Search AllowedNov 2025: Search AllowedDec 2025: Search AllowedJan 2026: Search AllowedFeb 2026: Search AllowedMar 2026: Search AllowedApr 2026: Search Some blockedMay 2026: Search Some blockedJun 2026: Search Some blockedJul 2026: Search Some blockedAug 2026: Search Some blockedSep 2026: Search Some blockedOct 2026: Search Some blockedAssistantsJan 2023: Assistants AllowedFeb 2023: Assistants AllowedMar 2023: Assistants AllowedApr 2023: Assistants AllowedMay 2023: Assistants AllowedJun 2023: Assistants AllowedJul 2023: Assistants AllowedAug 2023: Assistants AllowedSep 2023: Assistants AllowedOct 2023: Assistants AllowedNov 2023: Assistants AllowedDec 2023: Assistants AllowedJan 2024: Assistants AllowedFeb 2024: Assistants AllowedMar 2024: Assistants AllowedApr 2024: Assistants AllowedMay 2024: Assistants AllowedJun 2024: Assistants AllowedJul 2024: Assistants AllowedAug 2024: Assistants AllowedSep 2024: Assistants AllowedOct 2024: Assistants AllowedNov 2024: Assistants AllowedDec 2024: Assistants AllowedJan 2025: Assistants AllowedFeb 2025: Assistants AllowedMar 2025: Assistants AllowedApr 2025: Assistants AllowedMay 2025: Assistants AllowedJun 2025: Assistants AllowedJul 2025: Assistants AllowedAug 2025: Assistants AllowedSep 2025: Assistants AllowedOct 2025: Assistants AllowedNov 2025: Assistants AllowedDec 2025: Assistants AllowedJan 2026: Assistants AllowedFeb 2026: Assistants AllowedMar 2026: Assistants AllowedApr 2026: Assistants AllowedMay 2026: Assistants AllowedJun 2026: Assistants AllowedJul 2026: Assistants AllowedAug 2026: Assistants AllowedSep 2026: Assistants AllowedOct 2026: Assistants AllowedBotsblockedDec 2023: 1 of 24 AI bots blockedFeb 2024: 1 of 24 AI bots blockedMar 2024: 1 of 24 AI bots blockedApr 2024: 1 of 24 AI bots blockedMay 2024: 1 of 24 AI bots blockedJun 2024: 1 of 24 AI bots blockedJul 2024: 1 of 24 AI bots blockedAug 2024: 1 of 24 AI bots blockedSep 2024: 1 of 24 AI bots blockedOct 2024: 1 of 24 AI bots blockedNov 2024: 1 of 24 AI bots blockedDec 2024: 1 of 24 AI bots blockedJan 2025: 1 of 24 AI bots blockedFeb 2025: 1 of 24 AI bots blockedMar 2025: 1 of 24 AI bots blockedApr 2025: 1 of 24 AI bots blockedMay 2025: 1 of 24 AI bots blockedJun 2025: 1 of 24 AI bots blockedApr 2026: 4 of 24 AI bots blockedMay 2026: 4 of 24 AI bots blockedJun 2026: 4 of 24 AI bots blockedJul 2026: 4 of 24 AI bots blockedAug 2026: 4 of 24 AI bots blockedSep 2026: 4 of 24 AI bots blockedOct 2026: 4 of 24 AI bots blocked42023202420252026

One square per month, from Internet Archive copies and our own daily checks. Dark: blocked. Gray: some blocked. Light: allowed. Striped: no copy that month.

  • Apr 2026
    More blocking Now blocked: Applebot-Extended, Meta-ExternalAgent, AI2Bot and Amazonbot.
    Found in the Internet Archive's copies: changed between 1 Mar 2026 and 1 Apr 2026. Evidence: before · after
    Show exactly what changed
    - # Access to and use of Zillow.com is governed by our Terms of Use. See http://www.zillow.com/corp/Terms.htm
    + # Access to and use of Zillow.com is governed by our Terms of Use. See https://www.zillow.com/corp/Terms.htm
    …
    + Allow: /homes/apartments-pet-friendly/$
    + Allow: /homes/apartments-1-bedrooms/$
    + Allow: /homes/apartments-2-bedrooms/$
    + Allow: /homes/apartments-under-1500/$
    + Allow: /homes/apartments-under-2000/$
    + Allow: /homes/apartments-3-bedrooms/$
    + Allow: /homes/pet-friendly/$
    + Allow: /homes/rooms-for-rent/$
    + Allow: /homes/apartments-with-washer-dryer/$
    +
    + Allow: /homes/apartments-pet-friendly/*_p/$
    + Allow: /homes/apartments-1-bedrooms/*_p/$
    + Allow: /homes/apartments-2-bedrooms/*_p/$
    + Allow: /homes/apartments-under-1500/*_p/$
    + Allow: /homes/apartments-under-2000/*_p/$
    + Allow: /homes/apartments-3-bedrooms/*_p/$
    + Allow: /homes/pet-friendly/*_p/$
    + Allow: /homes/rooms-for-rent/*_p/$
    + Allow: /homes/apartments-with-washer-dryer/*_p/$
    +
    Disallow: /api/
    …
    Disallow: /*/foreclosure--
    - Disallow: */foreclosed/*
    + Disallow: /*/foreclosed/*
    Disallow: /*/GetResults.htm*
    …
    Disallow: /postings/PostMakeMeMoveStreamlined.htm?
    - Disallow: */pre-foreclosure/*
    + Disallow: /*/pre-foreclosure/*
    Disallow: /pricing/
    …
    Disallow: /sell/?*
    - Disallow: /sell/options/*
    + Disallow: /sell/options/
    Disallow: /showcase/Click.htm?
    - Disallow: /sitemap/catalog/*
    - Disallow: /sitemap/rss/*
    - Disallow: /sitemap/crawl-monitor/*
    + Disallow: /sitemap/catalog/
    + Disallow: /sitemap/rss/
    + Disallow: /sitemap/crawl-monitor/
    Disallow: /trk/
    …
    Disallow: /wiki/Help*
    - Disallow: */allunits/*
    + Disallow: /*/allunits/*
    Disallow: /reviews/batch-request/
    …
    Disallow: /editing/EditHomeFacts.htm
    - Disallow: /community/error-subdivision-name-unavailable/
    Disallow: /user/account/
    - Disallow: /user/InlineLogin.htm
    Disallow: /reviews/request/
    - Disallow: /CountyAssessorPage.htm
    … 80 more lines
  • Jul 2025
    Less blocking Now allowed: GPTBot.
    Found in the Internet Archive's copies: changed between 2 Jun 2025 and 4 Jul 2025. Evidence: before · after
    Show exactly what changed
    User-agent: *
    +
    + Allow: /homes/for_sale/$
    + Allow: /homes/for_sale/fore_lt/$
    + Allow: /homes/fsbo/$
    + Allow: /homes/for_sale/1_open/$
    + Allow: /homes/new_homes/$
    + Allow: /homes/coming_soon/$
    + Allow: /homes/for_rent/$
    + Allow: /homes/for_rent/condo,apartment_duplex_type/$
    + Allow: /homes/for_rent/house,townhouse_type/$
    + Allow: /homes/recently_sold/$
    + Allow: /homes/apartment-communities/$
    +
    + Allow: /homes/for_sale/*_p/$
    + Allow: /homes/for_sale/fore_lt/*_p/$
    + Allow: /homes/fsbo/*_p/$
    + Allow: /homes/for_sale/1_open/*_p/$
    + Allow: /homes/new_homes/*_p/$
    + Allow: /homes/coming_soon/*_p/$
    + Allow: /homes/for_rent/*_p/$
    + Allow: /homes/for_rent/condo,apartment_duplex_type/*_p/$
    + Allow: /homes/for_rent/house,townhouse_type/*_p/$
    + Allow: /homes/recently_sold/*_p/$
    + Allow: /homes/apartment-communities/*_p/$
    +
    + Disallow: /api/
    + Disallow: /homes/
    Disallow: /?url=
    …
    Disallow: /captcha/
    + Disallow: /captchaPerimeterX/
    Disallow: /careers/apply/
    Disallow: /careers/jobs/
    + Disallow: /chat/finance
    Disallow: /claiming/
    …
    Disallow: /homepage/HomepageVideoMetrics.htm
    - Disallow: /homes/*+
    - Disallow: /homes/*_rb
    - Disallow: /homes/*_zpid/
    - Disallow: /homes/*mapBounds*
    - Disallow: /homes/browse/
    - Disallow: /homes/com.zillow.service.search.SearchURLImpl
    - Disallow: /homes/comps_for_rent/
    - Disallow: /homes/comps_for_sale/
    - Disallow: /homes/comps/
    - Disallow: /homes/for_sale/*_agent_list/
    - Disallow: /homes/for_sale/1_favorite_list/1_rs/1_fr/
    Disallow: /howto/GoodNeighborPolicy.htm
    …
    Disallow: /sell/?*
    + Disallow: /sell/options/*
    Disallow: /showcase/Click.htm?
    …
    Disallow: /wikipages/
    + Disallow: /zg-graph/
    Disallow: /ajax/profile-broker-website
    …
    Disallow: /b/*/*_p/
    … 42 more lines
  • Feb 2024
    More blocking Now blocked: GPTBot.
    Found in the Internet Archive's copies: changed between 1 Jan 2024 and 1 Feb 2024. Evidence: before · after
    Show exactly what changed
    - # Access to and use of Zillow.com is governed by our Terms of Use. See https://www.zillow.com/corp/Terms.htm
    -
    + # Access to and use of Zillow.com is governed by our Terms of Use. See http://www.zillow.com/corp/Terms.htm
    User-agent: *
    - Disallow: /r/
    + Disallow: /?url=
    + Disallow: /*_rect
    + Disallow: /*_rect/
    + Disallow: /*_rid
    + Disallow: /*_sort/
    + Disallow: /*.sdirect
    + Disallow: /*1_fs/
    + Disallow: /*3ab9b0d/
    + Disallow: /*3bec89b
    + Disallow: /*/_type/
    + Disallow: /*/0-0_beds/
    + Disallow: /*/admin-ajax.php
    + Disallow: /*/apartments-are_att/
    + Disallow: /*/apartments-madden_att/
    + Disallow: /*/apartments-offers_att/
    + Disallow: /*/apartments-rent_att/
    + Disallow: /*cbpartner=
    + Disallow: /*/CountyAssessorPage.htm?*
    + Disallow: /*/duplex-apartments_att/
    + Disallow: /*/floor-map-annotator/*
    + Disallow: /*/foreclosure--
    + Disallow: */foreclosed/*
    + Disallow: /*/GetResults.htm*
    + Disallow: /*/house_type/
    + Disallow: /*/house,apartment_duplex_type/
    + Disallow: /*/house,mobile_type/
    + Disallow: /*/house,mobile,land_type/
    + Disallow: /*/house,mobile,townhouse_type/
    + Disallow: /*/loft-apartments_att/
    + Disallow: /*/mostrecentchange_sort/
    + Disallow: /*/node_modules/.bin/
    + Disallow: /*/plus/
    + Disallow: /*/rent-house_att/
    + Disallow: /*/rino-_att/
    + Disallow: /*/qc/
    + Disallow: /*-qc/
    + Disallow: /*-qc-*
    + Disallow: /*/townhome_att/
    + Disallow: /*/u1_sch/
    + Disallow: /*/z/internet-tv-deals-saver/?z*
    + Disallow: /*/z/internet-tv-deals-speed/?z*
    + Disallow: /*RentalConfirmedClaim
    + Disallow: /*?searchQueryState=*
    + Disallow: /ads/
    + Disallow: /advice-thread/
    + Disallow: /advicepages/
    + Disallow: /aerial/
    + Disallow: /advice/
    + Disallow: /agent-websites/
    + Disallow: /ajax/homedetail/CompareWithPublicFactsData.htm
    + Disallow: /ajax/homedetail/HomeValueChartData.htm
    + Disallow: /ajax/nav/UserNavAsync.htm?pageframe=true
    + Disallow: /ajax/rentals/ClaimedRentalProperties.htm?actionType=claims
    + Disallow: /ajax/ownerview/PerformSellerLeadContact.htm
    … 191 more lines

Reports from readers

No reports yet. Tell us what happened when your AI tool tried this site, even if it worked fine.