{"domain":"aliexpress.com","name":"AliExpress","category":"retail","subcategory":"marketplace","country":"CN","tags":[],"notes":"","status":"partial","status_label":"Blocks some AI","rationale":"Its robots.txt (the public rules file for bots) treats AI bots differently from each other. It blocks 6 of the 24 AI bots we track from the whole site, including GPTBot, ClaudeBot, Google-Extended, CCBot and Bytespider. Applebot-Extended, Meta-ExternalAgent, OAI-SearchBot, Claude-SearchBot and 4 more can reach most of it.","computed_at":"2026-10-20T03:43:14.136Z","last_checked":"2026-10-19T13:26:29.563Z","checker":{"allowed":true,"vantage":"residential connection, United States (manual run)","origin":"https://www.aliexpress.com"},"robots":{"via":{"type":"live","date":"2026-10-19","url":"https://www.aliexpress.com/robots.txt"},"kind":"file","http":200,"error":null,"url":"https://www.aliexpress.com/robots.txt","hops":[{"from":"https://aliexpress.com/robots.txt","to":"https://www.aliexpress.com/robots.txt","status":301}],"blob":"8b70ec63378682631ceaa01e5892d657aafeae1e0f14fa653093d38460d53097","text":"User-agent: *\r\nDisallow: /items/*\r\nDisallow: /bin/*\r\nDisallow: /search/*\r\nAllow: /wholesale.html$\r\nAllow: /wholesale-page-*.html\r\nDisallow: /productdetail/*\r\nAllow: /api/data_homepage.do\r\nDisallow: /api/*\r\nDisallow: /api*.do\r\nDisallow: /apps/*\r\nDisallow: /downloads/*\r\nDisallow: /wishlist/*\r\nDisallow: /shopcart/*\r\nDisallow: /brands/*\r\nDisallow: /cp/*\r\nDisallow: /item-img/*\r\nDisallow: /product/*\r\nDisallow: /p4p*list.html\r\nDisallow: /orderList.html\r\nDisallow: /ws*api-server.html$\r\nDisallow: /*mmend.htm$\r\nDisallow: /*Ajax.htm$\r\nDisallow: /*ajax.htm$\r\nDisallow: /store/*ajax.htm$\r\nDisallow: /detail/*Ajax.do$\r\nDisallow: /ajax*.do$\r\nDisallow: /__pc/*\r\nDisallow: /__mobile/*\r\nDisallow: /seo/*\r\nDisallow: /cross-domain/*\r\nDisallow: /*store/group/*\r\nDisallow: /*store/sale-items/*\r\nDisallow: /store/all-wholesale-products/* \r\nDisallow: /*store/top-rated-products/*\r\nAllow: /i/api/reviews\r\nDisallow: /i/api/*\r\nDisallow: /gcp/300000414/regulatedinformation\r\nDisallow: /img/wsproduct/sku/*\r\nDisallow: */aeglodetailweb/api/msite/item?productId*\r\n\r\nUser-agent: YandexBot\r\nCrawl-delay: 0.5\r\nDisallow: /bin/*\r\nDisallow: /search/*\r\nDisallow: /wholesale*\r\nDisallow: /productdetail/*\r\nDisallow: /api/*\r\nDisallow: /api*.do\r\nDisallow: /apps/*\r\nDisallow: /downloads/*\r\nDisallow: /wishlist/*\r\nDisallow: /shopcart/*\r\nDisallow: /brands/*\r\nDisallow: /cp/*\r\nDisallow: /item-img/*\r\nDisallow: /product/*\r\nDisallow: /p4p*list.html\r\nDisallow: /orderList.html\r\nDisallow: /ws*api-server.html$\r\nDisallow: /*mmend.htm$\r\nDisallow: /*Ajax.htm$\r\nDisallow: /*ajax.htm$\r\nDisallow: /store/*ajax.htm$\r\nDisallow: /detail/*Ajax.do$\r\nDisallow: /ajax*.do$\r\nDisallow: /__pc/*\r\nDisallow: /__mobile/*\r\nDisallow: /seo/*\r\nDisallow: /cross-domain/*\r\nDisallow: /*store/group/*\r\nDisallow: /*store/sale-items/*\r\nDisallow: /store/all-wholesale-products/* \r\nDisallow: /*store/top-rated-products/*\r\nDisallow: /i/api/*\r\nDisallow: /gcp/300000414/regulatedinformation\r\nDisallow: /img/wsproduct/sku/*\r\nDisallow: */aeglodetailweb/api/msite/item?productId*\r\n\r\nUser-agent: GPTBot\r\nDisallow: /\r\n\r\nUser-agent: CCBot\r\nDisallow: /\r\n\r\nUser-agent: Google-Extended\r\nDisallow: /\r\n\r\nUser-agent: ClaudeBot\r\nDisallow: /\r\n\r\nUser-agent: Amazonbot\r\nDisallow: /\r\n\r\nUser-agent: Bytespider\r\nDisallow: /\r\n\r\nUser-agent: SemrushBot\r\nDisallow: /\n\nSitemap: https://www.aliexpress.com/sitemap/en-https-item-item-2026-P0-sitemap-index.xml\nSitemap: https://www.aliexpress.com/sitemap/en-https-item-item-2026-P1-sitemap-index.xml\nSitemap: https://www.aliexpress.com/sitemap/en-https-wholesale-2025-w-commercial-local-sitemap-index.xml\nSitemap: https://www.aliexpress.com/sitemap/en-https-wholesale-2025-w-informational-localsitemap-index.xml\nSitemap: https://www.aliexpress.com/sitemap/en-https-wholesale-2025-w-brand-local-sitemap-index.xml\nSitemap: https://www.aliexpress.com/sitemap/en-https-wholesale-2025-p-transactional-local-sitemap-index.xml\nSitemap: https://www.aliexpress.com/sitemap/en-https-mlh-2026-sitemap-index.xml\nSitemap: https://www.aliexpress.com/sitemap/en-https-spot-2026-sitemap-index.xml","refused":null,"warnings":[],"contentSignals":[],"contentUsage":[],"licenses":[],"aiComments":[],"platform":null,"wildcard":{"exists":true,"access":"full"},"named":["*","YandexBot","GPTBot","CCBot","Google-Extended","ClaudeBot","Amazonbot","Bytespider","SemrushBot"],"lines":108},"agents":[{"token":"GPTBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":81},"any":{"allowed":false,"line":81},"search":{"allowed":false,"line":81},"cart":{"allowed":false,"line":81},"checkout":{"allowed":false,"line":81},"account":{"allowed":false,"line":81}},"disallows":1,"lines":[80,81],"groupRanges":[[80,81]]},{"token":"ClaudeBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":90},"any":{"allowed":false,"line":90},"search":{"allowed":false,"line":90},"cart":{"allowed":false,"line":90},"checkout":{"allowed":false,"line":90},"account":{"allowed":false,"line":90}},"disallows":1,"lines":[89,90],"groupRanges":[[89,90]]},{"token":"Google-Extended","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":87},"any":{"allowed":false,"line":87},"search":{"allowed":false,"line":87},"cart":{"allowed":false,"line":87},"checkout":{"allowed":false,"line":87},"account":{"allowed":false,"line":87}},"disallows":1,"lines":[86,87],"groupRanges":[[86,87]]},{"token":"Applebot-Extended","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"CCBot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":84},"any":{"allowed":false,"line":84},"search":{"allowed":false,"line":84},"cart":{"allowed":false,"line":84},"checkout":{"allowed":false,"line":84},"account":{"allowed":false,"line":84}},"disallows":1,"lines":[83,84],"groupRanges":[[83,84]]},{"token":"Meta-ExternalAgent","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Bytespider","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":96},"any":{"allowed":false,"line":96},"search":{"allowed":false,"line":96},"cart":{"allowed":false,"line":96},"checkout":{"allowed":false,"line":96},"account":{"allowed":false,"line":96}},"disallows":1,"lines":[95,96],"groupRanges":[[95,96]]},{"token":"MistralAI-Training","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"AI2Bot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"anthropic-ai","deprecated":true,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"OAI-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Claude-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"PerplexityBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Amazonbot","deprecated":false,"matched":"named","access":"none","probes":{"root":{"allowed":false,"line":93},"any":{"allowed":false,"line":93},"search":{"allowed":false,"line":93},"cart":{"allowed":false,"line":93},"checkout":{"allowed":false,"line":93},"account":{"allowed":false,"line":93}},"disallows":1,"lines":[92,93],"groupRanges":[[92,93]]},{"token":"Amzn-SearchBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"DuckAssistBot","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"MistralAI-Index","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Meta-WebIndexer","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"ChatGPT-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Claude-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Perplexity-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Google-Agent","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Meta-ExternalFetcher","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Amzn-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"MistralAI-User","deprecated":false,"matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]}],"refs":[{"token":"Googlebot","matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]},{"token":"Bingbot","matched":"wildcard","access":"full","probes":{"root":{"allowed":true,"line":null},"any":{"allowed":true,"line":null},"search":{"allowed":true,"line":null},"cart":{"allowed":true,"line":null},"checkout":{"allowed":true,"line":null},"account":{"allowed":true,"line":null}},"disallows":35,"lines":[1,40],"groupRanges":[[1,40]]}],"lanes":{"training":{"level":"mixed","none":5,"full":4,"restricted":0,"n":9},"search":{"level":"mixed","none":1,"full":7,"restricted":0,"n":8},"user_directed":{"level":"full","none":0,"full":7,"restricted":0,"n":7}},"door":{"door":"error","detail":"too many redirects"},"edge":{"cdn":[],"botManagement":[]},"llms":{"state":"absent","http":200,"note":"Served an HTML page, not a text file (soft 404 or redirect to homepage).","url":"https://www.aliexpress.com/s/error/404"},"ai_txt":{"state":"absent","http":200,"note":"Served an HTML page, not a text file (soft 404 or redirect to homepage).","url":"https://www.aliexpress.com/s/error/404"},"tdmrep":{"state":"absent","http":200,"note":"Served an HTML page, not a text file (soft 404 or redirect to homepage).","url":"http://www.aliexpress.com/maintain.html?app=23804-500"},"meta":null,"x_robots_tag":null,"tos":{"url":null,"source":null,"skipped":"No ToS URL curated or discovered"},"paid":null,"program":null,"statements":[],"claim":null,"reports":[],"history":{"points":43,"first":"2023-01-01","archive_points":42,"series":[{"m":"2023-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-10","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-11","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2023-12","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-10","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-11","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2024-12","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-10","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-11","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2025-12","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-01","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-02","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-03","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-04","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-05","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-06","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-07","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-08","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-09","s":"allowed","l":{"training":"full","search":"full","user_directed":"full"},"n":0},{"m":"2026-10","s":"partial","l":{"training":"mixed","search":"mixed","user_directed":"full"},"n":6}]},"events":[{"domain":"aliexpress.com","name":"AliExpress","category":"retail","date":"2026-10","precision":"month","source":"archive+checker","provisional":true,"between":["2026-09-10","2026-10-19"],"summary":"Now blocked: GPTBot, ClaudeBot, Google-Extended, CCBot, Bytespider and Amazonbot.","dir":"up","up":["GPTBot","ClaudeBot","Google-Extended","CCBot","Bytespider","Amazonbot"],"down":[],"major":true,"signals":false,"status_from":"allowed","status_to":"partial","evidence":["https://web.archive.org/web/20260910002748/https://www.aliexpress.com/robots.txt","https://www.aliexpress.com/robots.txt"],"diff":[{"t":"ctx","s":"Allow: /wholesale.html$"},{"t":"del","s":"Disallow: /wholesale*"},{"t":"add","s":"Allow: /wholesale-page-*.html"},{"t":"ctx","s":"Disallow: /productdetail/*"},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /i/api/*"},{"t":"add","s":"Disallow: /gcp/300000414/regulatedinformation"},{"t":"add","s":"Disallow: /img/wsproduct/sku/*"},{"t":"add","s":"Disallow: */aeglodetailweb/api/msite/item?productId*"},{"t":"ctx","s":""},{"t":"gap","s":"…"},{"t":"ctx","s":"Disallow: /i/api/*"},{"t":"del","s":"Host: https://aliexpress.ru"},{"t":"del","s":"Clean-param: initiative_id&site&spm&g&isrefine&needquery&blanktest&sorttype&cpt&aff_trace_key&ispremium&aff_platform&misspelltext&isonlyaffiliate&afref&aff_short_key&isdl&albcp&albch&albag&crea&netw&trgt&src&plac&aff_id&p4p&tracelog&useimatch&_csrf&utm_source&utm_campaign&utm_term&smsign&smtoken&utm_medium&utm_content&isafchannel&viewtype&aff_click_id&t&transabtest&callback&_csrf_token_&_&origin&rowan_id1&rowan_msg_id&ck&json&iscates"},{"t":"add","s":"Disallow: /gcp/300000414/regulatedinformation"},{"t":"add","s":"Disallow: /img/wsproduct/sku/*"},{"t":"add","s":"Disallow: */aeglodetailweb/api/msite/item?productId*"},{"t":"ctx","s":""},{"t":"del","s":"Sitemap: https://www.aliexpress.com/sitemap/https-en-ugc-desktop-sitemap-index.xml"},{"t":"del","s":"Sitemap: https://www.aliexpress.com/siteindex/https-en-ugc-desktop-xml-sitemap-index.xml"},{"t":"add","s":"User-agent: GPTBot"},{"t":"add","s":"Disallow: /"},{"t":"ctx","s":""},{"t":"add","s":"User-agent: CCBot"},{"t":"add","s":"Disallow: /"},{"t":"ctx","s":""},{"t":"del","s":"Sitemap: https://www.aliexpress.com/sitemap/https-en-ugc-desktop-sitemap-index.xml"},{"t":"del","s":"Sitemap: https://www.aliexpress.com/siteindex/https-en-ugc-desktop-xml-sitemap-index.xml"},{"t":"add","s":"User-agent: Google-Extended"},{"t":"add","s":"Disallow: /"},{"t":"ctx","s":""},{"t":"add","s":"User-agent: ClaudeBot"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"add","s":"User-agent: Amazonbot"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"add","s":"User-agent: Bytespider"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"add","s":"User-agent: SemrushBot"},{"t":"add","s":"Disallow: /"},{"t":"add","s":""},{"t":"add","s":"Sitemap: https://www.aliexpress.com/sitemap/en-https-item-item-2026-P0-sitemap-index.xml"},{"t":"add","s":"Sitemap: https://www.aliexpress.com/sitemap/en-https-item-item-2026-P1-sitemap-index.xml"},{"t":"add","s":"Sitemap: https://www.aliexpress.com/sitemap/en-https-wholesale-2025-w-commercial-local-sitemap-index.xml"},{"t":"add","s":"Sitemap: https://www.aliexpress.com/sitemap/en-https-wholesale-2025-w-informational-localsitemap-index.xml"},{"t":"add","s":"Sitemap: https://www.aliexpress.com/sitemap/en-https-wholesale-2025-w-brand-local-sitemap-index.xml"},{"t":"add","s":"Sitemap: https://www.aliexpress.com/sitemap/en-https-wholesale-2025-p-transactional-local-sitemap-index.xml"},{"t":"add","s":"Sitemap: https://www.aliexpress.com/sitemap/en-https-mlh-2026-sitemap-index.xml"},{"t":"add","s":"Sitemap: https://www.aliexpress.com/sitemap/en-https-spot-2026-sitemap-index.xml"}]}]}