Quora
quora.comBlocks some
Its robots.txt (the public rules file for bots) treats AI bots differently from each other. It blocks 9 of the 24 AI bots we track from the whole site, including GPTBot, ClaudeBot, Applebot-Extended, OAI-SearchBot and Claude-SearchBot. Its catch-all rule keeps 13 other bots out of parts of it. Google-Extended can reach most of it. The homepage also showed our bot, which says who it is, a Cloudflare challenge page instead of the site.
Last checked 19 Oct 2026. Rules file read today. Everything we checked · History
Bots that train AI Some blocked
4 of 9 blocked
Blocking these keeps the site out of future AI training. It does not stop AI from reading the site to answer questions.
Bots for AI search Some blocked
3 of 8 blocked
Blocking these usually means the site stops showing up as a source in AI answers.
AI assistants Some blocked
2 of 7 blocked
5 of these 7 companies say their assistants may ignore robots.txt.
Bot by bot
| AI bot | Allowed? | Where the rule is | Company says it follows robots.txt? |
|---|---|---|---|
| Bots that train AI · Some blocked | |||
| GPTBotOpenAI | Blocked | Named, line 676 | Says yes |
| ClaudeBotAnthropic | Blocked | Named, line 750 | Says yes |
| Google-ExtendedGoogle · opt-out name, not a bot | Allowed | Named, line 209 | Says yes |
| Applebot-ExtendedApple · opt-out name, not a bot | Blocked | Named, line 659 | Says yes |
| CCBotCommon Crawl | Some pages blocked | Not named; the catch-all rule applies | Says yes |
| Meta-ExternalAgentMeta | Some pages blocked | Not named; the catch-all rule applies | Suggests yes |
| BytespiderByteDance | Blocked | Named, line 668 | No statement found |
| MistralAI-TrainingMistral AI | Some pages blocked | Not named; the catch-all rule applies | Says yes |
| AI2BotAllen Institute for AI | Some pages blocked | Not named; the catch-all rule applies | Does not say |
| anthropic-aiAnthropic (legacy) · old name | Some pages blocked | Not named; the catch-all rule applies | Unknown |
| Bots for AI search · Some blocked | |||
| OAI-SearchBotOpenAI | Blocked | Named, line 717 | Says yes |
| Claude-SearchBotAnthropic | Blocked | Named, line 758 | Says yes |
| PerplexityBotPerplexity | Blocked | Named, line 726 | Suggests yes |
| AmazonbotAmazon | Allowed | Named, line 76 | Says yes |
| Amzn-SearchBotAmazon | Some pages blocked | Not named; the catch-all rule applies | Says yes |
| DuckAssistBotDuckDuckGo | Some pages blocked | Not named; the catch-all rule applies | Says yes |
| MistralAI-IndexMistral AI | Some pages blocked | Not named; the catch-all rule applies | Suggests yes |
| Meta-WebIndexerMeta | Some pages blocked | Not named; the catch-all rule applies | Suggests yes |
| AI assistants · Some blocked | |||
| ChatGPT-UserOpenAI | Blocked | Named, line 738 | Says not always |
| Claude-UserAnthropic | Blocked | Named, line 754 | Says yes |
| Perplexity-UserPerplexity | Some pages blocked | Not named; the catch-all rule applies | Says it mostly ignores it |
| Google-AgentGoogle | Some pages blocked | Not named; the catch-all rule applies | Says it mostly ignores it |
| Meta-ExternalFetcherMeta | Some pages blocked | Not named; the catch-all rule applies | Says not always |
| Amzn-UserAmazon | Some pages blocked | Not named; the catch-all rule applies | Says not always |
| MistralAI-UserMistral AI | Some pages blocked | Not named; the catch-all rule applies | Suggests yes |
| For comparison | |||
| GooglebotRegular search engine, for comparison | Allowed | Named | |
| BingbotRegular search engine, for comparison | Some pages blocked | Not named; the catch-all rule applies | |
Other AI names in this file
Names in this file that are not among the 24 bots we track. A rule for a name no real bot uses blocks nothing, but it shows which names the site has added.
| Applebot | Has rules | line 337 | Documented by its operator |
| Diffbot | Blocked | line 672 | Documented by its operator |
| ImagesiftBot | Blocked | line 708 | Not in our verified list |
| PetalBot | Blocked | line 730 | Not in our verified list |
We test two pages: the homepage, and a made-up page that stands in for the rest of the site. "Blocked" means both are off limits. "Some pages blocked" means only one is. Rules for search results, carts, checkout and accounts don't change these labels, because almost every site keeps all bots out of those pages.
The rules file (robots.txt)
# NOTICE: All crawlers and bots, regardless of whether or not they are# specified below, are strictly prohibited from using Quora platform content # for the purposes of training AI models or similar machine learning systems, # except where explicit prior permission has been granted by Quora through a# contractual licensing agreement. Any use of Quora platform content for such # purposes without our explicit contractual permission is a violation of our # terms of service and intellectual property rights. # If you operate a search engine and would like to crawl Quora,# you can email robotstxt@quora.com. Thanks. User-agent: AASA-BotDisallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss… 74 lines not about AI bots User-agent: AmazonbotDisallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss User-agent: GooglebotAllow: */answer/ix_/Allow: /sitemap/collections/answer/Disallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/RssDisallow: */unanswered/*Disallow: */answer/Disallow: */answers/… 68 lines not about AI bots User-agent: Google-ExtendedDisallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss User-agent: FacebotDisallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss… 63 lines not about AI bots User-agent: ApplebotDisallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss User-agent: TwitterbotDisallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss User-agent: LinkedInBotDisallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss User-agent: FeedlyDisallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss User-agent: PingdomDisallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss # exclusive for AI training… 257 lines not about AI bots User-agent: Applebot-ExtendedDisallow: / # another archiving service User-agent: Arquivo-web-crawlerDisallow: / … 5 lines not about AI bots User-agent: BytespiderDisallow: / User-agent: DiffbotDisallow: / User-agent: GPTBotDisallow: / # People share a lot of sensitive material on Quora - controversial political# views, workplace gossip and compensation, and negative opinions held of# companies. Over many years, as they change jobs or change their views, it is# important that they can delete or anonymize their previously-written answers.# # We opt out of the wayback machine because inclusion would allow people to# discover the identity of authors who had written sensitive answers publicly and# later had made them anonymous, and because it would prevent authors from being# able to remove their content from the internet if they change their mind about# publishing it. As far as we can tell, there is no way for sites to selectively# programmatically remove content from the archive and so this is the only way# for us to protect writers. If they open up an API where we can remove content# from the archive when authors remove it from Quora, but leave the rest of the# content archived, we would be happy to opt back in. See the page here:# # https://archive.org/about/exclude.php# # Meanwhile, if you are looking for an older version of any content on Quora, we# have full edit history tracked and accessible in product (with the exception of# content that has been removed by the author). You can generally access this by# clicking on timestamps, or by appending "/log" to the URL of any content page.# # For any questions or feedback about this please visit our contact page# https://help.quora.com/hc/en-us/requests/new User-agent: ia_archiverDisallow: / … 28 lines not about AI bots User-agent: ImagesiftBotDisallow: / # Blocked since a lot of bad requests were made by this crawler. User-agent: JamesBOTDisallow: / … 5 lines not about AI bots User-agent: OAI-SearchBotDisallow: /# drinking waterAllow: /*drinking-waterAllow: /*Drinking-waterAllow: /*Drinking-WaterAllow: /*drinking-Water User-agent: PerplexityBotDisallow: / User-agent: PetalBotDisallow: / User-agent: ScrapyDisallow: / … 4 lines not about AI bots User-agent: ChatGPT-UserDisallow: / User-agent: BravebotDisallow: / User-agent: CriteoBot/0.1Disallow: / … 8 lines not about AI bots User-agent: ClaudeBotDisallow: / User-agent: Claude-UserDisallow: / User-agent: Claude-SearchBotDisallow: / User-agent: *Allow: /$Allow: /about$Allow: /about/Allow: /challenges$Allow: /press$Allow: /login/Allow: /signup$Disallow: /Disallow: /ajax/Disallow: /*_POST$Disallow: /*_POST/Disallow: /@asyncDisallow: /*/@asyncDisallow: /log/Disallow: /*/logDisallow: /*/aboutDisallow: /*/actionDisallow: /*/activityDisallow: /*/all_questionsDisallow: /*/all_posts$Disallow: /*/all_posts/Disallow: /*/blogs$Disallow: /*/blogs/Disallow: /*/followersDisallow: /*/followingDisallow: /*/link/Disallow: /*/manageDisallow: /*/mentionsDisallow: /*/mergedDisallow: /*/open_questionsDisallow: /*/posts$Disallow: /*/posts/Disallow: /*/questionsDisallow: /*/relatedDisallow: /*/reviews$Disallow: /*/reviews/Disallow: /*/shareDisallow: /*/top_questionsDisallow: /*/topic-bioDisallow: /*/topic_bioDisallow: /*/topicsDisallow: /*/commentDisallow: /comment/Disallow: /*/commentsDisallow: /*/all_commentsDisallow: /*/answer_commentsDisallow: /*/mobile_collapsedDisallow: /*/mobile_expanded_voter_listDisallow: /home/global_feedDisallow: /search?q=Disallow: /search/?q=Disallow: /digest/Disallow: /email_optout/Disallow: /qemail/Disallow: /invite/Disallow: /widgets/content_iframe/Disallow: /widgets/content_js/Disallow: /_/Disallow: /webnode2/Disallow: /anonymous/Disallow: /q/*/admin_logDisallow: /q/*/statsDisallow: /q/*/settingsDisallow: /q/*/queueDisallow: /q/*/suggestionsDisallow: /q/*/submissionsDisallow: /q/*/qualityDisallow: /q/*/earningsDisallow: /profile/*/RssDisallow: /topic/*/Rss … 72 lines not about AI bots
Notes about AI written in the file: "for the purposes of training AI models or similar machine learning systems," · "exclusive for AI training"
Everything else we checked
| Front door | Human check The homepage showed our bot, which says who it is, a Cloudflare challenge page instead of the site (HTTP 403).GET https://www.quora.com/ as WalledWebBot, from a residential connection in the United States |
|---|---|
| llms.txt | Disallowed for AgentAccessIndexBot by robots.txt |
| ai.txt | Disallowed for AgentAccessIndexBot by robots.txt |
| TDMRep | Disallowed for AgentAccessIndexBot by robots.txt/.well-known/tdmrep.json (W3C Community Group report) |
| Page tags | Homepage not read. |
| Anti-bot software | Served through Cloudflare.From response headers and cookie names only. A cookie shows the product is there, not how it is set up. |
| Readable by AI | Homepage not read.Homepage only. Useful for site owners; not part of the result. |
| Terms of Service | No Terms of Service page found. |
| Not found | No Content-Signal or Content-Usage lines or licensing signals (RSL License, HTTP 402). |
History
History starts with our first check
One square per month, from Internet Archive copies and our own daily checks. Dark: blocked. Gray: some blocked. Light: allowed. Striped: no copy that month.
One version on record so far (19 Oct 2026).
Reports from readers
No reports yet. Tell us what happened when your AI tool tried this site, even if it worked fine.