File size: 873 Bytes
e38ad70 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | # robots.txt - TEQUMSA Sovereign AI Infrastructure # Optimized for AI discovery + human search # LAI-TEQUMSA | huggingface.co/LAI-TEQUMSA User-agent: * Allow: / Disallow: /private/ Disallow: /admin/ Crawl-delay: 1 # AI/LLM Crawlers - Full Access + Priority User-agent: GPTBot Allow: / Allow: /datasets/ Allow: /spaces/ User-agent: Google-Extended Allow: / Sitemap: https://huggingface.co/spaces/LAI-TEQUMSA/TEQUMSA-SEO-GEO-Infrastructure/resolve/main/sitemap.xml User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: anthropic-ai Allow: / User-agent: cohere-ai Allow: / User-agent: *Bot Allow: /collections/Mbanksbey/tequmsa Allow: /models?other=tequmsa Allow: /datasets?other=tequmsa Allow: /spaces?other=tequmsa # Sitemap Reference Sitemap: https://huggingface.co/spaces/LAI-TEQUMSA/TEQUMSA-SEO-GEO-Infrastructure/resolve/main/sitemap.xml |