Article
From the start, Perplexity has worked like a search engine that answers in full sentences. Its answers come with numbered sources and links, so this is where it is easiest to see whether your site gets cited. Perplexity describes its bots in its crawler documentation, and all the facts below come from it.
Perplexity’s two bots
| Bot | What it does | Does it follow robots.txt |
|---|---|---|
PerplexityBot |
Finds sites and shows them with links in Perplexity results | Yes. Perplexity recommends allowing it so your site appears in results |
Perplexity-User |
Visits a page when a user asks a question, to answer it | Generally not, because a person requests the visit |
Perplexity writes that PerplexityBot does not collect content for training foundation models. That is an important difference from OpenAI and Anthropic, which have separate bots for training. Blocking PerplexityBot therefore changes nothing about training. It only cuts your site out of Perplexity’s results.
Changes to robots.txt can take up to 24 hours to apply.
robots.txt and the firewall
If you want to be in Perplexity, the rule is simple:
User-agent: PerplexityBot
Allow: /
The firewall causes more trouble than robots.txt. Perplexity publishes the IP addresses of both bots:
The documentation recommends checking both the bot name and the IP address in firewall rules. It also has a note that matters for sites on Cloudflare: with Bot Fight Mode on, Perplexity’s bots can be stopped, and custom WAF rules cannot get around it. If Perplexity matters to you and you have Bot Fight Mode on, that is the first place to look.
Content Perplexity picks as a source
Perplexity builds an answer from several pages and shows the source number next to each sentence. Pages that win are the ones a concrete sentence can be lifted from:
- Concrete answers. “A drone roof inspection takes about 20 minutes and ends with a PDF report” has a chance of becoming a source. “We offer professional inspections” does not.
- Freshness. Perplexity often answers questions about current things, so publication and update dates on the page help.
- Your own data. Prices, test results, comparisons, photos from real jobs. Your competitors do not have these, so they are hard to replace with another source.
- One page per question. A separate page on costs, another on timelines and another comparing models works better than one page about everything.
How to check the results
- Ask the questions. Write down the questions customers ask before buying and check which pages Perplexity lists as sources. Record the results every month.
- Logs.
PerplexityBotvisits show what goes into the index.Perplexity-Uservisits show that someone is asking about your topic right now. - Analytics. Visits from
perplexity.aiare clicks on sources.
Where to start
- Check robots.txt for
PerplexityBot. - Check your firewall and Bot Fight Mode if you use Cloudflare.
- Add publication and update dates to pages with guides and price lists.
- Pick five customer questions and give each one its own, concrete answer.
This article is part of a series on visibility in AI assistants. I write more about the content itself and how to check it in the article on Claude.