InletsBot web crawler

InletsBot is operated by InletsAI. It reads publicly available pages to help prepare reports for our clients about how their brands appear in AI answers.

What it reads

InletsBot reads public pages from brands we work with, their competitors, and pages cited in AI answers. It analyses the text and structured data on those pages, including with AI models, to prepare client reports. It does not use pages to train AI models or build a search index.

How it crawls

• It follows robots.txt: the rules for InletsBot, or the rules for * when no rule for InletsBot is present.
• It honours Crawl-delay. Otherwise, it makes about one request per second to each site.
• It slows down after 429 or 503 responses and respects Retry-After.
• It only reads pages. It never submits forms or logs in.

Verify a request

User agent:

Mozilla/5.0 (compatible; InletsBot/1.0; +https://crawler.inlets.ai/bot)

Every request is signed with Web Bot Auth (HTTP Message Signatures). Our key directory is https://crawler.inlets.ai/.well-known/http-message-signatures-directory, and signed requests carry Signature-Agent: "https://crawler.inlets.ai".

InletsBot runs on shared cloud IP addresses, so we do not publish an IP list. A request using this name without a valid signature is not from InletsBot.

Block or slow InletsBot

Block it

To stop InletsBot from reading your site, add this to your robots.txt:

User-agent: InletsBot
Disallow: /

Slow it down

To set a lower crawl rate, add a Crawl-delay rule:

User-agent: InletsBot
Crawl-delay: 5

Contact

Questions or concerns about InletsBot? Please email engineering@inlets.ai.