# Robots.txt for Tosheroon.com User-agent: * Allow: / # Disallow search pages (all regions) Disallow: /*/search # Disallow LiveView transport (longpoll/websocket return JSON, not HTML) Disallow: /live/ # Disallow admin and system routes Disallow: /admin/ Disallow: /dev/ Disallow: /api/ Disallow: /webhooks/ Disallow: /up # Disallow affiliate redirects Disallow: /go/ # Disallow legacy product URLs Disallow: /products/ # Disallow alert management Disallow: /alerts/ # Sitemap Sitemap: https://tosheroon.com/sitemap.xml Crawl-delay: 1