{"domain":"airbnb.com","name":"Airbnb","category":"travel","subcategory":"lodging-marketplace","country":"US","tags":[],"notes":"","status":"partial","status_label":"Blocks some AI","rationale":"Its robots.txt (the public rules file for bots) treats AI bots differently from each other. It blocks 5 of the 24 AI bots we track from the whole site, including GPTBot, ClaudeBot, Applebot-Extended, MistralAI-Training and AI2Bot. Google-Extended, CCBot, Meta-ExternalAgent, OAI-SearchBot and 5 more can reach most of it.","computed_at":"2026-10-20T03:43:14.325Z","last_checked":"2026-10-19T13:29:07.148Z","checker":{"allowed":true,"vantage":"residential connection, United States (manual run)","origin":"https://www.airbnb.com"},"robots":{"via":{"type":"live","date":"2026-10-19","url":"https://www.airbnb.com/robots.txt"},"kind":"file","http":200,"error":null,"url":"https://www.airbnb.com/robots.txt","hops":[{"from":"https://airbnb.com/robots.txt","to":"https://www.airbnb.com/robots.txt","status":301}],"blob":"e274a840ea29686b39867202b958149e3097a57bd608136ef09565e84c049fb2","text":"#\n#              ///////\n#             //     //\n#            //       //\n#           //         //                           ///             ///                      ///\n#          //           //                                          ///                      ///\n#         //     ///     //               //// ///  ///  /// ////   /// ////     /// ////    /// ////\n#        //   ///   ///   //            //////////  ///  ////////// ///////////  //////////  ///////////\n#       //   //       //   //          ///     ///  ///  ///        ///      /// ///     /// ///      ///\n#      //    //       //    //        ///      ///  ///  ///        ///      /// ///     /// ///      ///\n#     //      //     //      //        ///     ///  ///  ///        ///     ///  ///     /// ///     ///\n#    //        //   //        //        //////////  ///  ///        //////////   ///     /// //////////\n#    //         /////         //\n#    //         /////         //\n#     //      ///   ///      //\n#       //////         //////\n#\n#\n#    Hello bot, engineer, or very lost layperson! Welcome to your stay on the Airbnb site, we're happy to have you as a guest.\n#    If you're a human who likes solving interesting challenges with other humans, check out our careers page: https://careers.airbnb.com/positions/?_departments=engineering\n#    If you're a bot who likes crawling webpages, please mind the house rules and avoid accessing any disallowed subfolders to earn a 5-star review from us.\n#    Either way, thanks for stopping by! There's no need to collect your garbage at the end of your stay - we use Javascript for that.\n#    See you on the next crawl!\n\nUser-agent: Googlebot\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\n\nUser-agent: Bingbot\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\n\nUser-agent: YandexBot\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\n\nUser-agent: Baiduspider\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\n\nUser-agent: OAI-Searchbot\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\n\nUser-agent: ChatGPT-User\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\n\nUser-agent: meta-externalagent\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\n\nUser-agent: PerplexityBot\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\nDisallow: /rooms/\nDisallow: /experiences/\n\nUser-agent: anthropic-ai\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\nDisallow: /rooms/\nDisallow: /experiences/\n\nUser-agent: cohere-ai\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\nDisallow: /rooms/\nDisallow: /experiences/\nDisallow: /help/\nDisallow: /*/things-to-do\nDisallow: /*/stays/\n\nUser-agent: ImagesiftBot\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /\n\n\nUser-agent: GPTBot\nDisallow: /\n\n\nUser-agent: ClaudeBot\nDisallow: /\n\n\nUser-agent: Applebot-Extended\nDisallow: /\n\n\nUser-agent: Webzio-Extended\nDisallow: /\n\n\nUser-agent: AI2Bot\nDisallow: /\n\n\nUser-agent: MistralAI-Training\nDisallow: /\n\nUser-agent: *\nAllow: /calendar/ical/\nAllow: /.well-known/amphtml/apikey.pub\nDisallow: /.well-known/assetlinks.json\nDisallow: /*/skeleton\nDisallow: /*/sw_skeleton\nDisallow: /500\nDisallow: /account\nDisallow: /alumni\nDisallow: /api/v1/trebuchet\nDisallow: /associates/click\nDisallow: /book/\nDisallow: /calendar/\nDisallow: /contact_host\nDisallow: /disaster/lookup\nDisallow: /email/unsubscribe\nDisallow: /embeddable\nDisallow: /experiences/*?*scheduled_id\nDisallow: /experiences/*?*modal\nDisallow: /experiences/*/book\nDisallow: /external_link?\nDisallow: /fix-it\nDisallow: /fixit\nDisallow: /forgot_password\nDisallow: /google_place_photo\nDisallow: /api/v2/google_place_photos\nDisallow: /groups\nDisallow: /guidebooks\nDisallow: /help/feedback\nDisallow: /help/search\nDisallow: /help/search\nDisallow: /home/dashboard\nDisallow: /inbox\nDisallow: /login_with_redirect\nDisallow: /logout\nDisallow: /manage-listing\nDisallow: /messaging/ajax_already_messaged/\nDisallow: /my_listings\nDisallow: /oauth_connect\nDisallow: /payments/book\nDisallow: /reservation\nDisallow: /rooms/*/amenities\nDisallow: /rooms/*/enhanced-cleaning\nDisallow: /rooms/*/house-rules\nDisallow: /rooms/*/location\nDisallow: /rooms/*/photos\nDisallow: /rooms/*/reviews\nDisallow: /rooms/*/safety\nDisallow: /rooms/*?viralityEntryPoint\nDisallow: /rooms/*/cancellation-policy\nDisallow: /rooms/*/description\nDisallow: /s/guidebooks\nDisallow: /signed_out_modal.json\nDisallow: /signup_modal\nDisallow: /stories\nDisallow: /trips/upcoming\nDisallow: /trips/v1/\nDisallow: /update-your-browser\nDisallow: /users/*/listings\nDisallow: /users/show\nDisallow: /users/profile\nDisallow: /api/v3/LandingPageStaysQuery\nDisallow: /s/*/*\nDisallow: /things-to-do/places\n\nSitemap: https://www.airbnb.com/sitemap-master-index.xml.gz\n\n","refused":null,"warnings":[],"contentSignals":[],"contentUsage":[],"licenses":[],"aiComments":[{"line":19,"text":"Hello bot, engineer, or very lost layperson! Welcome to your stay on the Airbnb site, we're happy to have you as a guest."},{"line":20,"text":"If you're a human who likes solving interesting challenges with other humans, check out our careers page: https://careers.airbnb.com/positions/?_departments=engineering"}],"platform":null,"wildcard":{"exists":true,"access":"full"},"named":["Googlebot","Bingbot","YandexBot","Baiduspider","OAI-Searchbot","ChatGPT-User","meta-externalagent","PerplexityBot","anthropic-ai","cohere-ai","ImagesiftBot","GPTBot","ClaudeBot","Applebot-Extended","Webzio-Extended","AI2Bot","MistralAI-Training","*"],"lines":780},"agents":[{"token":"GPTBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":691},"any":{"allowed":false,"line":691},"search":{"allowed":false,"line":691},"cart":{"allowed":false,"line":691},"checkout":{"allowed":false,"line":691},"account":{"allowed":false,"line":691}},"disallows":1,"lines":[690,691],"groupRanges":[[690,691]]},{"token":"ClaudeBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":695},"any":{"allowed":false,"line":695},"search":{"allowed":false,"line":695},"cart":{"allowed":false,"line":695},"checkout":{"allowed":false,"line":695},"account":{"allowed":false,"line":695}},"disallows":1,"lines":[694,695],"groupRanges":[[694,695]]},{"token":"Google-Extended","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"Applebot-Extended","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":699},"any":{"allowed":false,"line":699},"search":{"allowed":false,"line":699},"cart":{"allowed":false,"line":699},"checkout":{"allowed":false,"line":699},"account":{"allowed":false,"line":699}},"disallows":1,"lines":[698,699],"groupRanges":[[698,699]]},{"token":"CCBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"Meta-ExternalAgent","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":422}},"disallows":61,"lines":[415,478],"groupRanges":[[415,478]]},{"token":"Bytespider","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"MistralAI-Training","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":711},"any":{"allowed":false,"line":711},"search":{"allowed":false,"line":711},"cart":{"allowed":false,"line":711},"checkout":{"allowed":false,"line":711},"account":{"allowed":false,"line":711}},"disallows":1,"lines":[710,711],"groupRanges":[[710,711]]},{"token":"AI2Bot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":707},"any":{"allowed":false,"line":707},"search":{"allowed":false,"line":707},"cart":{"allowed":false,"line":707},"checkout":{"allowed":false,"line":707},"account":{"allowed":false,"line":707}},"disallows":1,"lines":[706,707],"groupRanges":[[706,707]]},{"token":"anthropic-ai","deprecated":true,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":554}},"disallows":63,"lines":[547,612],"groupRanges":[[547,612]]},{"token":"OAI-SearchBot","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":292}},"disallows":61,"lines":[285,348],"groupRanges":[[285,348]]},{"token":"Claude-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"PerplexityBot","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":487}},"disallows":63,"lines":[480,545],"groupRanges":[[480,545]]},{"token":"Amazonbot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"Amzn-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"DuckAssistBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"MistralAI-Index","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"Meta-WebIndexer","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"ChatGPT-User","deprecated":false,"matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":357}},"disallows":61,"lines":[350,413],"groupRanges":[[350,413]]},{"token":"Claude-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"Perplexity-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"Google-Agent","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"Meta-ExternalFetcher","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"Amzn-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]},{"token":"MistralAI-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":720}},"disallows":61,"lines":[713,776],"groupRanges":[[713,776]]}],"refs":[{"token":"Googlebot","matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":32}},"disallows":61,"lines":[25,88],"groupRanges":[[25,88]]},{"token":"Bingbot","matched":"named","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":false,"line":97}},"disallows":61,"lines":[90,153],"groupRanges":[[90,153]]}],"lanes":{"training":{"level":"mixed","none":5,"full":4,"restricted":0,"n":9},"search":{"level":"full","none":0,"full":8,"restricted":0,"n":8},"user_directed":{"level":"full","none":0,"full":7,"restricted":0,"n":7}},"door":{"door":"open","http":200},"edge":{"cdn":[],"botManagement":[]},"llms":{"state":"absent","http":404,"url":"https://www.airbnb.com/404"},"ai_txt":{"state":"absent","http":404,"url":"https://www.airbnb.com/404"},"tdmrep":{"state":"absent","http":404,"url":"https://www.airbnb.com/404"},"meta":{"title":"Airbnb: Vacation Rentals, Cabins, Beach Houses, Unique Homes & Experiences","robotsMeta":[],"noai":[],"noindex":false,"jsonLdTypes":["WebSite","SearchAction"],"platform":null,"textRatio":0.006,"llmsLink":false},"x_robots_tag":null,"tos":{"url":"https://www.airbnb.com/terms","source":"discovered","http":200,"hash":"6f3bbd0be1cc766b298212aaae70c4a254b7b8d46539540c1200ecd68ca23416","chars":258026,"thin":false,"clauses":[{"keyword":"scrape","prohibition":true,"excerpt":"Do not scrape, hack, reverse engineer, compromise or impair the Airbnb Platform Do not use bots, crawlers, scrapers, or other automated means to access or collect data or other content from or otherwise interact with the Airbnb Platform."},{"keyword":"scrape","prohibition":true,"excerpt":"Do not scrape, hack, reverse engineer, compromise or impair the Airbnb Platform Do not use bots, crawlers, scrapers or other automated means to access or collect data or other content from or otherwise interact with the Airbnb Platform."},{"keyword":"automated means","prohibition":true,"excerpt":"If automated means have been used to review Content, and Airbnb has subsequently taken action with respect to that Content, Airbnb will notify the Member who posted the Content of the use of automated means to the extent that legitimate interests of Airbnb and its Members do not preclude such notice…"},{"keyword":"scrape","prohibition":true,"excerpt":"Do not scrape, hack, reverse engineer, compromise or impair the Airbnb Platform."},{"keyword":"agent","prohibition":true,"excerpt":"Airbnb is not acting as an agent in any capacity for any Member, except as specified in the Payments Terms of Service (“ Payment Terms ”)."},{"keyword":"agent","prohibition":true,"excerpt":"Your relationship with Airbnb is that of an independent individual or entity and not an employee, agent, joint venturer, or partner of Airbnb, except that Airbnb Payments acts as a payment collection agent as described in the Payments Terms."},{"keyword":"agent","prohibition":true,"excerpt":"Airbnb is not acting as an agent for any Member except for where Airbnb Payments acts as a collection agent as provided in the Payments Terms."},{"keyword":"agent","prohibition":true,"excerpt":"Your relationship with Airbnb is that of an independent individual or entity and not an employee, agent, joint venturer or partner of Airbnb, except that Airbnb Payments acts as a payment collection agent as described in the Payments Terms."}],"door":{"door":"open","http":200}},"paid":null,"program":null,"statements":[],"claim":null,"reports":[],"history":{"points":44,"first":"2023-01-01","archive_points":43,"series":[{"m":"2023-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-09","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2023-10","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2023-11","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2023-12","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-01","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-02","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":1},{"m":"2024-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-10","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-11","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-12","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-10","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-11","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-12","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-10","s":"partial","l":{"training":"mixed","search":"full","user_directed":"full"},"n":5}]},"events":[{"domain":"airbnb.com","name":"Airbnb","category":"travel","date":"2026-10","precision":"month","source":"archive","provisional":false,"between":["2026-08-25","2026-10-01"],"summary":"Now blocked: GPTBot, ClaudeBot, Applebot-Extended, MistralAI-Training and AI2Bot.","dir":"up","up":["GPTBot","ClaudeBot","Applebot-Extended","MistralAI-Training","AI2Bot"],"down":[],"major":true,"signals":false,"status_from":"allowed","status_to":"partial","evidence":["https://web.archive.org/web/20260825094005/https://www.airbnb.com/robots.txt","https://web.archive.org/web/20261001122330/https://www.airbnb.com/robots.txt"],"diff":[{"t":"ctx","s":"Disallow: /google_place_photo"},{"t":"add","s":"Disallow: /api/v2/google_place_photos"},{"t":"ctx","s":"Disallow: /groups"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /users/show"},{"t":"del","s":"Disallow: /s/*?"},{"t":"del","s":"Disallow: /s/*/homes"},{"t":"add","s":"Disallow: /users/profile"},{"t":"add","s":"Disallow: /api/v3/LandingPageStaysQuery"},{"t":"add","s":"Disallow: /s/*/*"},{"t":"ctx","s":"Disallow: /things-to-do/places"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /google_place_photo"},{"t":"add","s":"Disallow: /api/v2/google_place_photos"},{"t":"ctx","s":"Disallow: /groups"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /users/show"},{"t":"del","s":"Disallow: /s/*?"},{"t":"del","s":"Disallow: /s/*/homes"},{"t":"add","s":"Disallow: /users/profile"},{"t":"add","s":"Disallow: /api/v3/LandingPageStaysQuery"},{"t":"add","s":"Disallow: /s/*/*"},{"t":"ctx","s":"Disallow: /things-to-do/places"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /google_place_photo"},{"t":"add","s":"Disallow: /api/v2/google_place_photos"},{"t":"ctx","s":"Disallow: /groups"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /users/show"},{"t":"del","s":"Disallow: /s/*?"},{"t":"del","s":"Disallow: /s/*/homes"},{"t":"add","s":"Disallow: /users/profile"},{"t":"add","s":"Disallow: /api/v3/LandingPageStaysQuery"},{"t":"add","s":"Disallow: /s/*/*"},{"t":"ctx","s":"Disallow: /things-to-do/places"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /google_place_photo"},{"t":"add","s":"Disallow: /api/v2/google_place_photos"},{"t":"ctx","s":"Disallow: /groups"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /users/show"},{"t":"del","s":"Disallow: /s/*?"},{"t":"del","s":"Disallow: /s/*/homes"},{"t":"add","s":"Disallow: /users/profile"},{"t":"add","s":"Disallow: /api/v3/LandingPageStaysQuery"},{"t":"add","s":"Disallow: /s/*/*"},{"t":"ctx","s":"Disallow: /things-to-do/places"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /google_place_photo"},{"t":"add","s":"Disallow: /api/v2/google_place_photos"},{"t":"ctx","s":"Disallow: /groups"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /users/show"},{"t":"del","s":"Disallow: /s/*?"},{"t":"del","s":"Disallow: /s/*/homes"},{"t":"add","s":"Disallow: /users/profile"},{"t":"add","s":"Disallow: /api/v3/LandingPageStaysQuery"},{"t":"add","s":"Disallow: /s/*/*"},{"t":"ctx","s":"Disallow: /things-to-do/places"},{"t":"gap","s":"…"},{"t":"gap","s":"… 242 more lines"}]},{"domain":"airbnb.com","name":"Airbnb","category":"travel","date":"2024-11","precision":"month","source":"archive","provisional":false,"between":["2024-10-01","2024-11-01"],"summary":"Named GPTBot, ClaudeBot, Meta-ExternalAgent, anthropic-ai, OAI-SearchBot, PerplexityBot and 1 more explicitly, without changing what they may reach.","dir":"other","up":[],"down":[],"major":false,"signals":false,"status_from":"allowed","status_to":"allowed","evidence":["https://web.archive.org/web/20241001021608/https://www.airbnb.com/robots.txt","https://web.archive.org/web/20241101011254/https://www.airbnb.com/robots.txt"],"diff":[{"t":"add","s":"#"},{"t":"ctx","s":"#              ///////"},{"t":"gap","s":"…"},{"t":"ctx","s":"#    Hello bot, engineer, or very lost layperson! Welcome to your stay on the Airbnb site, we're happy to have you as a guest."},{"t":"del","s":"#    If you're a human who likes solving interesting challenges with other humans, check out our careers page: https://www.airbnb.com/careers/departments/engineering"},{"t":"add","s":"#    If you're a human who likes solving interesting challenges with other humans, check out our careers page: https://careers.airbnb.com/positions/?_departments=engineering"},{"t":"ctx","s":"#    If you're a bot who likes crawling webpages, please mind the house rules and avoid accessing any disallowed subfolders to earn a 5-star review from us."},{"t":"gap","s":"…"},{"t":"ctx","s":"Allow: /.well-known/amphtml/apikey.pub"},{"t":"add","s":"Disallow: /.well-known/assetlinks.json"},{"t":"add","s":"Disallow: /*/skeleton"},{"t":"add","s":"Disallow: /*/sw_skeleton"},{"t":"add","s":"Disallow: /500"},{"t":"ctx","s":"Disallow: /account"},{"t":"ctx","s":"Disallow: /alumni"},{"t":"del","s":"Disallow: /associates/click"},{"t":"ctx","s":"Disallow: /api/v1/trebuchet"},{"t":"add","s":"Disallow: /associates/click"},{"t":"add","s":"Disallow: /book/"},{"t":"ctx","s":"Disallow: /calendar/"},{"t":"add","s":"Disallow: /contact_host"},{"t":"ctx","s":"Disallow: /disaster/lookup"},{"t":"ctx","s":"Disallow: /email/unsubscribe"},{"t":"add","s":"Disallow: /embeddable"},{"t":"add","s":"Disallow: /experiences/*?*scheduled_id"},{"t":"add","s":"Disallow: /experiences/*?*modal"},{"t":"add","s":"Disallow: /experiences/*/book"},{"t":"add","s":"Disallow: /external_link?"},{"t":"ctx","s":"Disallow: /fix-it"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /forgot_password"},{"t":"add","s":"Disallow: /google_place_photo"},{"t":"ctx","s":"Disallow: /groups"},{"t":"del","s":"Disallow: /help/search"},{"t":"add","s":"Disallow: /guidebooks"},{"t":"ctx","s":"Disallow: /help/feedback"},{"t":"add","s":"Disallow: /help/search"},{"t":"add","s":"Disallow: /help/search"},{"t":"ctx","s":"Disallow: /home/dashboard"},{"t":"ctx","s":"Disallow: /inbox"},{"t":"add","s":"Disallow: /login_with_redirect"},{"t":"ctx","s":"Disallow: /logout"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /my_listings"},{"t":"del","s":"Disallow: /skeleton$"},{"t":"del","s":"Disallow: /skeleton/"},{"t":"add","s":"Disallow: /oauth_connect"},{"t":"ctx","s":"Disallow: /payments/book"},{"t":"del","s":"Disallow: /signup_modal"},{"t":"add","s":"Disallow: /reservation"},{"t":"add","s":"Disallow: /rooms/*/amenities"},{"t":"add","s":"Disallow: /rooms/*/enhanced-cleaning"},{"t":"add","s":"Disallow: /rooms/*/house-rules"},{"t":"add","s":"Disallow: /rooms/*/location"},{"t":"add","s":"Disallow: /rooms/*/photos"},{"t":"add","s":"Disallow: /rooms/*/reviews"},{"t":"add","s":"Disallow: /rooms/*/safety"},{"t":"add","s":"Disallow: /rooms/*?viralityEntryPoint"},{"t":"add","s":"Disallow: /rooms/*/cancellation-policy"},{"t":"add","s":"Disallow: /rooms/*/description"},{"t":"gap","s":"… 866 more lines"}]},{"domain":"airbnb.com","name":"Airbnb","category":"travel","date":"2024-03","precision":"month","source":"archive","provisional":false,"between":["2024-02-01","2024-03-01"],"summary":"Now allowed: GPTBot.","dir":"down","up":[],"down":["GPTBot"],"major":true,"signals":false,"status_from":"partial","status_to":"allowed","evidence":["https://web.archive.org/web/20240201011022/https://www.airbnb.com/robots.txt","https://web.archive.org/web/20240301015734/https://www.airbnb.com/robots.txt"],"diff":[{"t":"ctx","s":""},{"t":"del","s":"User-agent: GPTBot"},{"t":"del","s":"Disallow: /"},{"t":"del","s":""},{"t":"ctx","s":"User-Agent: *"}]},{"domain":"airbnb.com","name":"Airbnb","category":"travel","date":"2023-09","precision":"month","source":"archive","provisional":false,"between":["2023-08-01","2023-09-01"],"summary":"Now blocked: GPTBot.","dir":"up","up":["GPTBot"],"down":[],"major":true,"signals":false,"status_from":"allowed","status_to":"partial","evidence":["https://web.archive.org/web/20230801015834/https://www.airbnb.com/robots.txt","https://web.archive.org/web/20230901025526/https://www.airbnb.com/robots.txt"],"diff":[{"t":"del","s":"#                      ///////"},{"t":"del","s":"#                     //     //"},{"t":"del","s":"#                    //       //"},{"t":"del","s":"#                   //         //                           ///             ///                      ///"},{"t":"del","s":"#                  //           //                                          ///                      ///"},{"t":"del","s":"#                 //     ///     //               //// ///  ///  /// ////   /// ////     /// ////    /// ////"},{"t":"del","s":"#                //   ///   ///   //            //////////  ///  ////////// ///////////  //////////  ///////////"},{"t":"del","s":"#               //   //       //   //          ///     ///  ///  ///        ///      /// ///     /// ///      ///"},{"t":"del","s":"#              //    //       //    //        ///      ///  ///  ///        ///      /// ///     /// ///      ///"},{"t":"del","s":"#             //      //     //      //        ///     ///  ///  ///        ///     ///  ///     /// ///     ///"},{"t":"del","s":"#            //        //   //        //        //////////  ///  ///        //////////   ///     /// //////////"},{"t":"del","s":"#            //         /////         //"},{"t":"del","s":"#            //         /////         //"},{"t":"del","s":"#             //      ///   ///      //"},{"t":"del","s":"#               //////         //////"},{"t":"add","s":"#              ///////"},{"t":"add","s":"#             //     //"},{"t":"add","s":"#            //       //"},{"t":"add","s":"#           //         //                           ///             ///                      ///"},{"t":"add","s":"#          //           //                                          ///                      ///"},{"t":"add","s":"#         //     ///     //               //// ///  ///  /// ////   /// ////     /// ////    /// ////"},{"t":"add","s":"#        //   ///   ///   //            //////////  ///  ////////// ///////////  //////////  ///////////"},{"t":"add","s":"#       //   //       //   //          ///     ///  ///  ///        ///      /// ///     /// ///      ///"},{"t":"add","s":"#      //    //       //    //        ///      ///  ///  ///        ///      /// ///     /// ///      ///"},{"t":"add","s":"#     //      //     //      //        ///     ///  ///  ///        ///     ///  ///     /// ///     ///"},{"t":"add","s":"#    //        //   //        //        //////////  ///  ///        //////////   ///     /// //////////"},{"t":"add","s":"#    //         /////         //"},{"t":"add","s":"#    //         /////         //"},{"t":"add","s":"#     //      ///   ///      //"},{"t":"add","s":"#       //////         //////"},{"t":"ctx","s":"#"},{"t":"ctx","s":"#"},{"t":"del","s":"#    We thought you'd never make it!"},{"t":"del","s":"#    We hope you feel right at home in this file...unless you're a disallowed subfolder."},{"t":"del","s":"#    And since you're here, read up on our culture and team: https://www.airbnb.com/careers/departments/engineering"},{"t":"del","s":"#    There's even a bring your robot to work day."},{"t":"del","s":""},{"t":"add","s":"#    Hello bot, engineer, or very lost layperson! Welcome to your stay on the Airbnb site, we're happy to have you as a guest."},{"t":"add","s":"#    If you're a human who likes solving interesting challenges with other humans, check out our careers page: https://www.airbnb.com/careers/departments/engineering"},{"t":"add","s":"#    If you're a bot who likes crawling webpages, please mind the house rules and avoid accessing any disallowed subfolders to earn a 5-star review from us."},{"t":"add","s":"#    Either way, thanks for stopping by! There's no need to collect your garbage at the end of your stay - we use Javascript for that."},{"t":"add","s":"#    See you on the next crawl!"},{"t":"ctx","s":""},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /signed_out_modal.json"},{"t":"add","s":"Disallow: /.well-known/assetlinks.json"},{"t":"ctx","s":"Disallow: /device_id_bev_map"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /reservation"},{"t":"del","s":""},{"t":"del","s":"        Disallow: /rooms/*/safety"},{"t":"del","s":"        Disallow: /rooms/*/reviews"},{"t":"del","s":"        Disallow: /rooms/*/photos"},{"t":"del","s":"        Disallow: /rooms/*/location"},{"t":"del","s":"        Disallow: /rooms/*/house-rules"},{"t":"del","s":"        Disallow: /rooms/*/enhanced-cleaning"},{"t":"del","s":"        Disallow: /rooms/*/amenities"},{"t":"del","s":"        Disallow: /users/show"},{"t":"del","s":"        Disallow: /users/*/listings"},{"t":"del","s":"        Disallow: /contact_host"},{"t":"gap","s":"… 182 more lines"}]}]}