# FinNotes robots policy. Allowlist model: only major search engines # and named LLM partners are permitted. Everything else falls through # to the catch-all Disallow. User-agent: Googlebot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Googlebot-News Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Googlebot-Image Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Bingbot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Baiduspider Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Slurp Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Applebot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: DuckDuckBot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Google-Extended Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: GPTBot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: OAI-SearchBot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: ChatGPT-User Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: ClaudeBot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Claude-Web Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: anthropic-ai Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: xAI-Grok Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Grok Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: PerplexityBot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Perplexity-User Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Meta-ExternalAgent Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Meta-ExternalFetcher Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: facebookexternalhit Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: Twitterbot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: LinkedInBot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: WhatsApp Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ User-agent: TelegramBot Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /account Disallow: /notes Disallow: /subscribe/success Disallow: /subscribe/cancel Disallow: /webhooks/ Disallow: /login Disallow: /fragments/ # Anything not matched above is denied. User-agent: * Disallow: / Sitemap: https://finnotes.com/sitemap.xml Sitemap: https://finnotes.com/sitemap-news.xml # AI crawler index (llmstxt.org convention). # https://finnotes.com/llms.txt # https://finnotes.com/llms-full.txt