User-agent: * Allow: / Sitemap: https://www.centraltrianglepressurewashingnc.com/sitemap.xml # --- Answer engines and AI crawlers, allowed explicitly --------------------- # The wildcard group above already permits these, so this section changes no # access. It is here because robots.txt group matching selects one group only: # a crawler that finds its own token stops reading "User-agent: *". Several # operators also check for their own token before treating access as consent. # # Each group repeats the wildcard group's Disallow lines. Without that repeat, # naming an agent would hand it every path the wildcard denies. User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: anthropic-ai Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Google-Extended Allow: / User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / User-agent: DuckAssistBot Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: cohere-ai Allow: / User-agent: CCBot Allow: /