#AI bots
Articles with this tag: 5.
What is llms.txt, does it work, and how to set it up for an online store
What llms.txt is, who actually reads it and whether Google uses it. Plus an example for a store and a way to check whether anyone fetches it.
How to optimize your website for Perplexity
PerplexityBot and Perplexity-User, robots.txt, the firewall and content that Perplexity shows as a numbered source. What to set up and how to check it.
How to optimize your website for ChatGPT
OAI-SearchBot, GPTBot, ChatGPT-User and OAI-AdsBot. Which bot decides whether you appear in ChatGPT search, how to set up robots.txt and how to measure results.
How to optimize your website for Grok
xAI publishes no documentation for its crawler. What official sources say about how Grok searches the web and X, and what you can do about it.
How to optimize your website for Claude
Anthropic's three bots, robots.txt, llms.txt and content Claude is happy to quote. Plus a simple way to check whether Claude already recommends your site.