{"domain":"bestbuy.com","name":"Best Buy","category":"retail","subcategory":"big-box","country":"US","tags":[],"notes":"","status":"allowed","status_label":"Allows AI","rationale":"Its robots.txt (the public rules file for bots) mentions GPTBot, ClaudeBot, Google-Extended, CCBot and 8 more, but still lets all 12 major AI bots into the site.","computed_at":"2026-10-20T03:43:13.399Z","last_checked":"2026-10-19T13:24:42.244Z","checker":{"allowed":true,"vantage":"residential connection, United States (manual run)","origin":"https://www.bestbuy.com"},"robots":{"via":{"type":"live","date":"2026-10-19","url":"https://www.bestbuy.com/robots.txt"},"kind":"file","http":200,"error":null,"url":"https://www.bestbuy.com/robots.txt","hops":[{"from":"https://bestbuy.com/robots.txt","to":"https://www.bestbuy.com/robots.txt","status":301}],"blob":"cb65f883b9aeec87339db0e12aa3228f8b8b49613f0beeb58111bf3a153e8de9","text":"# --------------------------------------------------\r\n# BEST BUY OFFICIAL ENTERPRISE ROBOTS.TXT\r\n# Optimized for search engines, AI discovery, ads, and crawl control\r\n# --------------------------------------------------\r\n\r\n# ==================================================\r\n# BUCKET 1: AI SEARCH & DISCOVERY BOT ACCESS\r\n# Allows conversational answer engines to securely access clean landing pages.\r\n# Mirrors explicit production parameters to protect long-tail revenue.\r\n# ==================================================\r\nUser-agent: OAI-SearchBot\r\nUser-agent: ChatGPT-User\r\nUser-agent: Perplexity-User\r\nUser-agent: GPTBot\r\nUser-agent: Claude-User\r\nUser-agent: Claudebot\r\nUser-agent: PerplexityBot\r\nUser-agent: Applebot\r\nUser-agent: Google-Extended\r\nUser-agent: Gemeni-AI\r\nUser-agent: Gemini-Deep-Research\r\nUser-agent: AmazonBot\r\n\r\nAllow: /site/\r\nAllow: /discover-learn/\r\n\r\n# System Application Blockouts\r\nDisallow: /nex/\r\nDisallow: /m/e/*\r\nDisallow: /browse-api/*\r\nDisallow: /api/*.json\r\nDisallow: /availability/*\r\nDisallow: /cart\r\nDisallow: /cart/*\r\nDisallow: /cartitems/*\r\nDisallow: /fulfillment/*\r\nDisallow: /location/*\r\nDisallow: /pricing/*\r\nDisallow: /productfulfillment/*\r\nDisallow: /profile/*\r\nDisallow: /services/plans/*\r\nDisallow: /services/shoppingscheduler*\r\nDisallow: /site/builder/*\r\nDisallow: /site/modals/*\r\nDisallow: /site/dni/*\r\nDisallow: /site/offerbuilder/*\r\n\r\n# User Dynamic Input Paths\r\nDisallow: /site/questions/submission*\r\nDisallow: /site/answers/submission*\r\nDisallow: /site/reviews/submission*\r\n\r\n# Production-Safe Explicit Tab Fragments\r\nDisallow: /*template=_gameDetailsTab\r\nDisallow: /*template=_movieDetailsTab\r\nDisallow: /*template=_musicDetailsTab\r\nDisallow: /*template=_softwareDetailsTab\r\nDisallow: /*template=_accessoriesTab\r\nDisallow: /*template=_castAndCrewTab\r\nDisallow: /*template=_editorialTab\r\nDisallow: /*template=_episodesTab\r\nDisallow: /*template=_protectionAndServicesTab\r\nDisallow: /*template=_specificationsTab\r\nDisallow: /*template=_featuresSpoke\r\nDisallow: /*template=_buyingOptionsNewTab\r\n\r\n# Production-Safe Explicit Core Facet Controls (Brand/Category left open)\r\nDisallow: *armodelformat_facet*\r\nDisallow: *awardtype_facet*\r\nDisallow: *awardwinner_facet*\r\nDisallow: *awardyear_facet*\r\nDisallow: *blenderfunctions_facet*\r\nDisallow: *boxdesign_facet*\r\nDisallow: *cast_facet*\r\nDisallow: *cleaningpathwidthrange_facet*\r\nDisallow: *consumerlifestage_facet*\r\nDisallow: *decade_facet*\r\nDisallow: *defrosttype_facet*\r\nDisallow: *departmentid_facet*\r\nDisallow: *depthtype_facet*\r\nDisallow: *displayport_facet*\r\nDisallow: *doorstylesv_facet*\r\nDisallow: *drawertype_facet*\r\nDisallow: *dynamiccategory_facet*\r\nDisallow: *effectivemegapixels_facet*\r\nDisallow: *flashmodesmv_facet*\r\nDisallow: *framematerialmv_facet*\r\nDisallow: *giftcardthemesv_facet*\r\nDisallow: *handletype_facet*\r\nDisallow: *headsetstylesv_facet*\r\nDisallow: *heatingsource_facet*\r\nDisallow: *imagesensorsize_facet*\r\nDisallow: *imagesensortype11_facet*\r\nDisallow: *includedcolors_facet*\r\nDisallow: *inkmodelcompatibilitymv_facet*\r\nDisallow: *internalorexternal_facet*\r\nDisallow: *laptopuse_facet*\r\nDisallow: *lengthofcable_facet*\r\nDisallow: *mattressmaterial_facet*\r\nDisallow: *maximumsensitivity_facet*\r\nDisallow: *minfocallengthrange_facet*\r\nDisallow: *mobileinternalmemory_facet*\r\nDisallow: *monitorresponsetimerange_facet*\r\nDisallow: *monitorsnumberofusb30typecports_facet*\r\nDisallow: *monitorsnumberofusb31typecports_facet*\r\nDisallow: *mountmaterial_facet*\r\nDisallow: *networkingcabletype_facet*\r\nDisallow: *numberofbedroomsef_facet*\r\nDisallow: *numberofburnerssv_facet*\r\nDisallow: *numberofbuttons_facet*\r\nDisallow: *numberofdryingcycles_facet*\r\nDisallow: *numberoffanspeedssv_facet*\r\nDisallow: *numberofhdmiinputssv_facet*\r\nDisallow: *numberoflanports_facet*\r\nDisallow: *numberofspeakers_facet*\r\nDisallow: *numberofspeedssv_facet*\r\nDisallow: *numberofthunderbolt_ports_facet*\r\nDisallow: *numberofwashrinsecycle_facet*\r\nDisallow: *papersize_facet*\r\nDisallow: *playbackformatsmv_facet*\r\nDisallow: *productcategories_facet*\r\nDisallow: *screensizediagonalrange_facet*\r\nDisallow: *securityconnectivity_facet*\r\nDisallow: *securityresolution_facet*\r\nDisallow: *securitystoragetype_facet*\r\nDisallow: *softwareformat_facet*\r\nDisallow: *sqbprocessormodel_facet*\r\nDisallow: *sqbtotalstoragecapacity_facet*\r\nDisallow: *stringtype_facet*\r\nDisallow: *subclassid_facet*\r\nDisallow: *subgenre_facet*\r\nDisallow: *thruthedoordispenser_facet*\r\nDisallow: *toastingslotssv_facet*\r\nDisallow: *totaltechdeals_facet*\r\nDisallow: *touchscreentype_facet*\r\nDisallow: *transferrate_facet*\r\nDisallow: *transistorsize_facet*\r\nDisallow: *transmitterdistance_facet*\r\nDisallow: *treeheight_facet*\r\nDisallow: *treelightbulbcolor_facet*\r\nDisallow: *tubfinishmv_facet*\r\nDisallow: *varyinganglescreen_facet*\r\nDisallow: *vendorcollection_facet*\r\nDisallow: *videomemory_facet*\r\nDisallow: *viewfindertypemv_facet*\r\nDisallow: *virtualrealitycompatibility_facet*\r\nDisallow: *wanportssv_facet*\r\nDisallow: *wearingstyle_facet*\r\nDisallow: *currentprice2_facet*\r\nDisallow: *customerreviews_facet*Customer*Rating*5*\r\nDisallow: *customerreviews_facet*Customer*Rating*4*\r\nDisallow: *customerreviews_facet*Customer*Rating*3*\r\nDisallow: *customerreviews_facet*Customer*Rating*2*\r\nDisallow: *customerreviews_facet*Customer*Rating*1*\r\nDisallow: *soldout_facet*\r\nDisallow: */site/promo/*?*_facet*\r\nDisallow: */product/list/*\r\nDisallow: */awacs-ingestor/*\r\n\r\n# Exact Pagination Safeguards\r\nDisallow: *cp=1*\r\nDisallow: *cp=20*\r\nDisallow: *cp=6*\r\nDisallow: *cp=7*\r\nDisallow: *cp=8*\r\nDisallow: *cp=9*\r\nDisallow: *?*page=*&page=*$\r\n\r\n# Malformed URL Traps (Universal Protection)\r\nDisallow: *==$\r\nDisallow: */-?\r\nDisallow: */undefined$\r\nDisallow: */null$\r\nDisallow: *javascript:winopengld*\r\n\r\n# ==================================================\r\n# BUCKET 2: AI TRAINING / DATA INGESTION FIREWALL\r\n# Completely blocks models from scraping copy assets for foundational training.\r\n# ==================================================\r\nUser-agent: GPTBot-Web\r\nUser-agent: CCBot\r\nUser-agent: Anthropic-AI\r\nUser-agent: Claude-Web\r\nUser-agent: Bytespider\r\nDisallow: /[1.1]\r\n\r\n# ==================================================\r\n# BUCKET 3: UNIFIED WEB ENGINES & UNIVERSAL DEFAULTS\r\n# Consolidated grouping ensures zero functional divergence between indexing bots.\r\n# ==================================================\r\nUser-agent: Googlebot\r\nUser-agent: Bingbot\r\nUser-agent: DuckDuckBot\r\nUser-agent: Slurp\r\nUser-agent: *\r\n\r\n# Standard Asset Pathways\r\nAllow: /~assets/\r\nAllow: /discover-learn/\r\n\r\n# Core System Blockouts\r\nDisallow: /nex/\r\nDisallow: /m/e/*\r\nDisallow: /site/builder/*\r\nDisallow: /browse-api/*\r\nDisallow: /api/*.json\r\nDisallow: /availability/*\r\nDisallow: /cart\r\nDisallow: /cart/*\r\nDisallow: /cartitems/*\r\nDisallow: /fulfillment/*\r\nDisallow: /location/*\r\nDisallow: /pricing/*\r\nDisallow: /productfulfillment/*\r\nDisallow: /profile/*\r\nDisallow: /site/modals/*\r\nDisallow: /services/plans/*\r\nDisallow: /site/dni/*\r\nDisallow: /site/offerbuilder/*\r\nDisallow: /services/shoppingscheduler*\r\n\r\n# Staging, Internal Test, & Promotional Exclusion Matrices\r\nDisallow: /site/clp/test-page/*\r\nDisallow: /site/test-page/*\r\nDisallow: /site/misc/test-pages/*\r\nDisallow: /site/offer/*\r\nDisallow: /site/compare?skus=*\r\nDisallow: /site/clp/conditional-offers/*\r\nDisallow: /site/total-tech-support/*\r\nDisallow: /site/clp/annette-test-listing/*\r\nDisallow: /site/shop-view-test-page/*\r\nDisallow: /site/clp/test/*\r\nDisallow: /site/test-cloud-ready/*\r\nDisallow: /site/clp/mystery-rewards-certificate/pcmcat1524134875052.c?id=pcmcat1524134875052\r\nDisallow: /purchasehistory/gift-receipt*\r\nDisallow: /contact-us?*\r\nDisallow: /ease/email\r\nDisallow: /ihc/invoice*\r\nDisallow: /identity/newAccount?*\r\nDisallow: /identity/signin?*\r\nDisallow: /identity/signin/forgotPassword?*\r\nDisallow: /dossier/service/appointment/*/*-*\r\nDisallow: /dossier/service/appointment/search?*\r\nDisallow: */product-selector/*\r\nDisallow: */email-subscription/*\r\n\r\n# Legacy Template Controller Protections (Parameter Strictness Maintained)\r\nDisallow: /site/olspage.jsp?id=pcat17098*\r\nDisallow: /site/olstemplatemapper.jsp?id=pcat17098*\r\nDisallow: /site/olspage.jsp?id=pcat17089*\r\nDisallow: /site/olstemplatemapper.jsp?id=pcat17089*\r\nDisallow: /site/olspage.jsp?id=cat13503*\r\nDisallow: /site/olstemplatemapper.jsp?id=cat13503*\r\nDisallow: /site/olspage.jsp?id=pcat17007*\r\nDisallow: /site/olstemplatemapper.jsp?id=pcat17007*\r\nDisallow: /site/olspage.jsp?id=pcat17408*\r\nDisallow: /site/olstemplatemapper.jsp?id=pcat17408*\r\nDisallow: /site/olspage.jsp?id=cat12091*\r\nDisallow: /site/olstemplatemapper.jsp?id=cat12091*\r\nDisallow: /site/olspage.jsp?id=pcat17005*\r\nDisallow: /site/olstemplatemapper.jsp?id=pcat17005*\r\nDisallow: /site/olspage.jsp?id=cat12076*\r\nDisallow: /site/olstemplatemapper.jsp?id=cat12076*\r\nDisallow: /site/olspage.jsp?id=cat12080*\r\nDisallow: /site/olstemplatemapper.jsp?id=cat12080*\r\nDisallow: /site/olspage.jsp?id=cat12101*\r\nDisallow: /site/olstemplatemapper.jsp?id=cat12101*\r\nDisallow: /site/olstemplatemapper.jsp?id=pcat17006*\r\nDisallow: /site/olspage.jsp?id=pcat17006*\r\nDisallow: */site/olspage.jsp*\r\n\r\n# Asynchronous Layout Fragment Controls\r\nDisallow: /*template=_gameDetailsTab\r\nDisallow: /*template=_movieDetailsTab\r\nDisallow: /*template=_musicDetailsTab\r\nDisallow: /*template=_softwareDetailsTab\r\nDisallow: /*template=_accessoriesTab\r\nDisallow: /*template=_castAndCrewTab\r\nDisallow: /*template=_editorialTab\r\nDisallow: /*template=_episodesTab\r\nDisallow: /*template=_protectionAndServicesTab\r\nDisallow: /*template=_specificationsTab\r\nDisallow: /*template=_featuresSpoke\r\nDisallow: /*template=_buyingOptionsNewTab\r\n\r\n# User Input Paths\r\nDisallow: /site/questions/submission*\r\nDisallow: /site/answers/submission*\r\nDisallow: /site/reviews/submission*\r\n\r\n# Granular Facet Exclusions\r\nDisallow: *armodelformat_facet*\r\nDisallow: *awardtype_facet*\r\nDisallow: *awardwinner_facet*\r\nDisallow: *awardyear_facet*\r\nDisallow: *blenderfunctions_facet*\r\nDisallow: *boxdesign_facet*\r\nDisallow: *cast_facet*\r\nDisallow: *cleaningpathwidthrange_facet*\r\nDisallow: *consumerlifestage_facet*\r\nDisallow: *decade_facet*\r\nDisallow: *defrosttype_facet*\r\nDisallow: *departmentid_facet*\r\nDisallow: *depthtype_facet*\r\nDisallow: *displayport_facet*\r\nDisallow: *doorstylesv_facet*\r\nDisallow: *drawertype_facet*\r\nDisallow: *dynamiccategory_facet*\r\nDisallow: *effectivemegapixels_facet*\r\nDisallow: *flashmodesmv_facet*\r\nDisallow: *framematerialmv_facet*\r\nDisallow: *giftcardthemesv_facet*\r\nDisallow: *handletype_facet*\r\nDisallow: *headsetstylesv_facet*\r\nDisallow: *heatingsource_facet*\r\nDisallow: *imagesensorsize_facet*\r\nDisallow: *imagesensortype11_facet*\r\nDisallow: *includedcolors_facet*\r\nDisallow: *inkmodelcompatibilitymv_facet*\r\nDisallow: *internalorexternal_facet*\r\nDisallow: *laptopuse_facet*\r\nDisallow: *lengthofcable_facet*\r\nDisallow: *mattressmaterial_facet*\r\nDisallow: *maximumsensitivity_facet*\r\nDisallow: *minfocallengthrange_facet*\r\nDisallow: *mobileinternalmemory_facet*\r\nDisallow: *monitorresponsetimerange_facet*\r\nDisallow: *monitorsnumberofusb30typecports_facet*\r\nDisallow: *monitorsnumberofusb31typecports_facet*\r\nDisallow: *mountmaterial_facet*\r\nDisallow: *networkingcabletype_facet*\r\nDisallow: *numberofbedroomsef_facet*\r\nDisallow: *numberofburnerssv_facet*\r\nDisallow: *numberofbuttons_facet*\r\nDisallow: *numberofdryingcycles_facet*\r\nDisallow: *numberoffanspeedssv_facet*\r\nDisallow: *numberofhdmiinputssv_facet*\r\nDisallow: *numberoflanports_facet*\r\nDisallow: *numberofspeakers_facet*\r\nDisallow: *numberofspeedssv_facet*\r\nDisallow: *numberofthunderbolt_ports_facet*\r\nDisallow: *numberofwashrinsecycle_facet*\r\nDisallow: *papersize_facet*\r\nDisallow: *playbackformatsmv_facet*\r\nDisallow: *productcategories_facet*\r\nDisallow: *screensizediagonalrange_facet*\r\nDisallow: *securityconnectivity_facet*\r\nDisallow: *securityresolution_facet*\r\nDisallow: *securitystoragetype_facet*\r\nDisallow: *softwareformat_facet*\r\nDisallow: *sqbprocessormodel_facet*\r\nDisallow: *sqbtotalstoragecapacity_facet*\r\nDisallow: *stringtype_facet*\r\nDisallow: *subclassid_facet*\r\nDisallow: *subgenre_facet*\r\nDisallow: *thruthedoordispenser_facet*\r\nDisallow: *toastingslotssv_facet*\r\nDisallow: *totaltechdeals_facet*\r\nDisallow: *touchscreentype_facet*\r\nDisallow: *transferrate_facet*\r\nDisallow: *transistorsize_facet*\r\nDisallow: *transmitterdistance_facet*\r\nDisallow: *treeheight_facet*\r\nDisallow: *treelightbulbcolor_facet*\r\nDisallow: *tubfinishmv_facet*\r\nDisallow: *varyinganglescreen_facet*\r\nDisallow: *vendorcollection_facet*\r\nDisallow: *videomemory_facet*\r\nDisallow: *viewfindertypemv_facet*\r\nDisallow: *virtualrealitycompatibility_facet*\r\nDisallow: *wanportssv_facet*\r\nDisallow: *wearingstyle_facet*\r\nDisallow: *currentprice2_facet*\r\nDisallow: *customerreviews_facet*Customer*Rating*5*\r\nDisallow: *customerreviews_facet*Customer*Rating*4*\r\nDisallow: *customerreviews_facet*Customer*Rating*3*\r\nDisallow: *customerreviews_facet*Customer*Rating*2*\r\nDisallow: *customerreviews_facet*Customer*Rating*1*\r\nDisallow: *soldout_facet*\r\nDisallow: */site/promo/*?*_facet*\r\nDisallow: */product/list/*\r\n\r\n# Pagination Crawl Trap Controls\r\nDisallow: *cp=1*\r\nDisallow: *cp=20*\r\nDisallow: *cp=6*\r\nDisallow: *cp=7*\r\nDisallow: *cp=8*\r\nDisallow: *cp=9*\r\nDisallow: *?*page=*&page=*$\r\n\r\n# Malformed URL Traps\r\nDisallow: *==$\r\nDisallow: */-?\r\nDisallow: */undefined$\r\nDisallow: */null$\r\nDisallow: *javascript:winopengld*\r\n\r\n# ==================================================\r\n# BUCKET 4: PAID SEARCH / ADS / MEDIA BOTS\r\n# Leaves pathways unobstructed for digital advertising synchronization.\r\n# ==================================================\r\nUser-agent: AdsBot-Google\r\nUser-agent: Googlebot-Image\r\nUser-agent: Mediapartners-Google\r\nUser-agent: Google-Display-Ads-Bot\r\nDisallow:\r\n\r\n# ==================================================\r\n# BUCKET 5: BLOCKED AGGRESSIVE THIRD-PARTY AGGREGATORS / TOOLS\r\n# Absolute block to preserve server performance and block unauthorized scrapers.\r\n# ==================================================\r\nUser-agent: GetIntentCrawler \r\nUser-agent: DotBot \r\nUser-agent: MJ12bot \r\nUser-agent: thesis-research-bot \r\nUser-agent: TerraCotta\r\nUser-agent: BigSurAI\r\nUser-agent: MyCentralAIScraper\r\nDisallow: / [2.1]\r\n\r\n# ==================================================\r\n# DATA SITEMAPS\r\n# ==================================================\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_discover_learn.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_pdp.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_promos.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_qna.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_rnr.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_search_plps.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_standalone_qa.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_browse_plps.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_manually_curated_pages.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_combos.xml\r\nSitemap: https://sitemaps.bestbuy.com/sitemaps_bsin.xml","refused":null,"warnings":[],"contentSignals":[],"contentUsage":[],"licenses":[],"aiComments":[{"line":3,"text":"Optimized for search engines, AI discovery, ads, and crawl control"},{"line":7,"text":"BUCKET 1: AI SEARCH & DISCOVERY BOT ACCESS"},{"line":177,"text":"BUCKET 2: AI TRAINING / DATA INGESTION FIREWALL"},{"line":178,"text":"Completely blocks models from scraping copy assets for foundational training."},{"line":411,"text":"Absolute block to preserve server performance and block unauthorized scrapers."}],"platform":null,"wildcard":{"exists":true,"access":"full"},"named":["OAI-SearchBot","ChatGPT-User","Perplexity-User","GPTBot","Claude-User","Claudebot","PerplexityBot","Applebot","Google-Extended","Gemeni-AI","Gemini-Deep-Research","AmazonBot","GPTBot-Web","CCBot","Anthropic-AI","Claude-Web","Bytespider","Googlebot","Bingbot","DuckDuckBot","Slurp","*","AdsBot-Google","Googlebot-Image","Mediapartners-Google","Google-Display-Ads-Bot","GetIntentCrawler","DotBot","MJ12bot","thesis-research-bot","TerraCotta","BigSurAI","MyCentralAIScraper"],"lines":435},"agents":[{"token":"GPTBot","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":33},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":137,"lines":[11,174],"groupRanges":[[11,174]]},{"token":"ClaudeBot","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":33},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":137,"lines":[11,174],"groupRanges":[[11,174]]},{"token":"Google-Extended","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":33},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":137,"lines":[11,174],"groupRanges":[[11,174]]},{"token":"Applebot-Extended","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"CCBot","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":1,"lines":[180,185],"groupRanges":[[180,185]]},{"token":"Meta-ExternalAgent","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"Bytespider","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":1,"lines":[180,185],"groupRanges":[[180,185]]},{"token":"MistralAI-Training","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"AI2Bot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"anthropic-ai","deprecated":true,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":1,"lines":[180,185],"groupRanges":[[180,185]]},{"token":"OAI-SearchBot","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":33},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":137,"lines":[11,174],"groupRanges":[[11,174]]},{"token":"Claude-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"PerplexityBot","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":33},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":137,"lines":[11,174],"groupRanges":[[11,174]]},{"token":"Amazonbot","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":33},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":137,"lines":[11,174],"groupRanges":[[11,174]]},{"token":"Amzn-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"DuckAssistBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"MistralAI-Index","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"Meta-WebIndexer","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"ChatGPT-User","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":33},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":137,"lines":[11,174],"groupRanges":[[11,174]]},{"token":"Claude-User","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":33},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":137,"lines":[11,174],"groupRanges":[[11,174]]},{"token":"Perplexity-User","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":33},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":137,"lines":[11,174],"groupRanges":[[11,174]]},{"token":"Google-Agent","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"Meta-ExternalFetcher","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"Amzn-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"MistralAI-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]}],"refs":[{"token":"Googlebot","matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]},{"token":"Bingbot","matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":false,"line":208},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":182,"lines":[191,397],"groupRanges":[[191,397]]}],"lanes":{"training":{"level":"full","none":0,"full":9,"restricted":0,"n":9},"search":{"level":"full","none":0,"full":8,"restricted":0,"n":8},"user_directed":{"level":"full","none":0,"full":7,"restricted":0,"n":7}},"door":{"door":"open","http":200},"edge":{"cdn":["Akamai"],"botManagement":[{"vendor":"Akamai","cookies":["bm_sz"]}]},"llms":{"state":"absent","http":404,"url":"https://www.bestbuy.com/llms.txt"},"ai_txt":{"state":"absent","http":404,"url":"https://www.bestbuy.com/ai.txt"},"tdmrep":{"state":"absent","http":404,"url":"https://www.bestbuy.com/.well-known/tdmrep.json"},"meta":{"title":"Best Buy | Official Online Store | Shop Now & Save","robotsMeta":[],"noai":[],"noindex":false,"jsonLdTypes":[],"platform":"Next.js","textRatio":0.004,"llmsLink":false},"x_robots_tag":null,"tos":{"url":"https://www.bestbuy.com/site/help-topics/terms-and-conditions/pcmcat204400050067.c?id=pcmcat204400050067","source":"discovered","http":200,"hash":"23f8743a9a99d26b0ffa7363a9c017e664cd08ffd1928088b5a31afc4eea6e8e","chars":45550,"thin":false,"clauses":[{"keyword":"AI","prohibition":true,"excerpt":"AI used on or in connection with the Best Buy Properties to provide you with ideas, inspiration, summary information, or other contextual information, including those in response to topics or questions you input, is provided solely for your convenience and may not always be accurate."},{"keyword":"AI","prohibition":true,"excerpt":"We do not warrant that Product descriptions or other content, whether generated by AI or not, is accurate, complete, or error free."},{"keyword":"scrape","prohibition":true,"excerpt":"You may not copy, scrape, frame, modify, remove, delete, augment, add to, publish, transmit, participate in the transfer or sale, lease or rental of, create derivative works from or in any way exploit any of the Content, in whole or in part."},{"keyword":"AI","prohibition":true,"excerpt":"… PROPERTY, ANY PRODUCT OR SERVICE PURCHASED THROUGH A BEST BUY PROPERTY, OR FOR ANY INACCURATE INFORMATION GENERATED BY AI AND DISPLAYED ON THE BEST BUY PROPERTIES OR SHARED WITH YOU AS PART OF YOUR INTERACTIONS WITH THE BEST BUY PROPERTIES."},{"keyword":"agent","prohibition":true,"excerpt":"… a good faith belief that use of the material in the manner complained of is not authorized by the copyright owner, its agent, or the law"},{"keyword":"agent","prohibition":true,"excerpt":"You may not use or attempt to use any engine, software, tool, agent, data or other device or mechanism (including without limitation browsers, spiders, robots, avatars or intelligent agents) to navigate or search any Best Buy Property other than the search engine and search agents we provide and gen…"},{"keyword":"bots","prohibition":true,"excerpt":"…y CAPTCHAs or other mechanisms used or intended to distinguish computers from humans to prevent the use of automations, bots, or other technology from interacting with such content."},{"keyword":"agents","prohibition":true,"excerpt":"… or controversy arising out of or relating in any way whatsoever to your relationship or interaction with Best Buy, its agents, and its present and future subsidiaries, affiliates, and designees—including, but not limited to, GreatCall, Lively, Geek Squad, Magnolia, and Pacific Sales—whether based i…"}],"door":{"door":"open","http":200}},"paid":null,"program":null,"statements":[],"claim":null,"reports":[],"history":{"points":42,"first":"2023-01-01","archive_points":41,"series":[{"m":"2023-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-10","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-11","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-12","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-10","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-11","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-12","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-10","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-11","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-12","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-10","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0}]},"events":[{"domain":"bestbuy.com","name":"Best Buy","category":"retail","date":"2026-10","precision":"month","source":"archive+checker","provisional":true,"between":["2026-05-01","2026-10-19"],"summary":"Named GPTBot, ClaudeBot, Google-Extended, CCBot, Bytespider, anthropic-ai and 6 more explicitly, without changing what they may reach.","dir":"other","up":[],"down":[],"major":false,"signals":false,"status_from":"allowed","status_to":"allowed","evidence":["https://web.archive.org/web/20260501111319/https://www.bestbuy.com/robots.txt","https://www.bestbuy.com/robots.txt"],"diff":[{"t":"add","s":"# --------------------------------------------------"},{"t":"add","s":"# BEST BUY OFFICIAL ENTERPRISE ROBOTS.TXT"},{"t":"add","s":"# Optimized for search engines, AI discovery, ads, and crawl control"},{"t":"add","s":"# --------------------------------------------------"},{"t":"add","s":""},{"t":"add","s":"# =================================================="},{"t":"add","s":"# BUCKET 1: AI SEARCH & DISCOVERY BOT ACCESS"},{"t":"add","s":"# Allows conversational answer engines to securely access clean landing pages."},{"t":"add","s":"# Mirrors explicit production parameters to protect long-tail revenue."},{"t":"add","s":"# =================================================="},{"t":"add","s":"User-agent: OAI-SearchBot"},{"t":"add","s":"User-agent: ChatGPT-User"},{"t":"add","s":"User-agent: Perplexity-User"},{"t":"add","s":"User-agent: GPTBot"},{"t":"add","s":"User-agent: Claude-User"},{"t":"add","s":"User-agent: Claudebot"},{"t":"add","s":"User-agent: PerplexityBot"},{"t":"add","s":"User-agent: Applebot"},{"t":"add","s":"User-agent: Google-Extended"},{"t":"add","s":"User-agent: Gemeni-AI"},{"t":"add","s":"User-agent: Gemini-Deep-Research"},{"t":"add","s":"User-agent: AmazonBot"},{"t":"add","s":""},{"t":"add","s":"Allow: /site/"},{"t":"add","s":"Allow: /discover-learn/"},{"t":"add","s":""},{"t":"add","s":"# System Application Blockouts"},{"t":"add","s":"Disallow: /nex/"},{"t":"add","s":"Disallow: /m/e/*"},{"t":"add","s":"Disallow: /browse-api/*"},{"t":"add","s":"Disallow: /api/*.json"},{"t":"add","s":"Disallow: /availability/*"},{"t":"add","s":"Disallow: /cart"},{"t":"add","s":"Disallow: /cart/*"},{"t":"add","s":"Disallow: /cartitems/*"},{"t":"add","s":"Disallow: /fulfillment/*"},{"t":"add","s":"Disallow: /location/*"},{"t":"add","s":"Disallow: /pricing/*"},{"t":"add","s":"Disallow: /productfulfillment/*"},{"t":"add","s":"Disallow: /profile/*"},{"t":"add","s":"Disallow: /services/plans/*"},{"t":"add","s":"Disallow: /services/shoppingscheduler*"},{"t":"add","s":"Disallow: /site/builder/*"},{"t":"add","s":"Disallow: /site/modals/*"},{"t":"add","s":"Disallow: /site/dni/*"},{"t":"add","s":"Disallow: /site/offerbuilder/*"},{"t":"add","s":""},{"t":"add","s":"# User Dynamic Input Paths"},{"t":"add","s":"Disallow: /site/questions/submission*"},{"t":"add","s":"Disallow: /site/answers/submission*"},{"t":"add","s":"Disallow: /site/reviews/submission*"},{"t":"add","s":""},{"t":"add","s":"# Production-Safe Explicit Tab Fragments"},{"t":"add","s":"Disallow: /*template=_gameDetailsTab"},{"t":"add","s":"Disallow: /*template=_movieDetailsTab"},{"t":"add","s":"Disallow: /*template=_musicDetailsTab"},{"t":"add","s":"Disallow: /*template=_softwareDetailsTab"},{"t":"add","s":"Disallow: /*template=_accessoriesTab"},{"t":"add","s":"Disallow: /*template=_castAndCrewTab"},{"t":"add","s":"Disallow: /*template=_editorialTab"},{"t":"gap","s":"… 379 more lines"}]}]}