Our crawler
ProxaAuditBot
ProxaAuditBot runs Proxa’s free agent-readiness audit. It visits a store only when someone asks us to check it, reads a handful of public pages, and reports how well AI shopping agents could use the store.
How to recognise it
Every request sends this user agent:
ProxaAuditBot/1.0 (+https://proxa.uk/bot)Requests are also signed with Web Bot Auth (HTTP Message Signatures with an Ed25519 key). Our public keys are at https://proxa.uk/.well-known/http-message-signatures-directory. A request that claims to be ProxaAuditBot but has no valid signature isn’t from us.
Requests come from Cloudflare’s network, because the audit runs on Cloudflare Workers.
When it visits
- Only when a person runs an audit of your store on our site. It doesn’t crawl the web on its own.
- Results are cached for 24 hours, so the same store isn’t checked again within that time.
- Audits are rate-limited per visitor, so nobody can use it to send repeated traffic to your site.
What it fetches
At most 14 requests per audit, including redirects, all within about 20 seconds:
/robots.txt- Your homepage
- One sitemap, plus one product sitemap if it’s a sitemap index
/.well-known/ucpand/.well-known/mcp.json- Up to 5 product pages
- On WooCommerce stores, one request to the public Store API for a single product
It only makes GET requests. It never logs in, submits forms, adds to cart or touches checkout, and it doesn’t run JavaScript or load images.
robots.txt
ProxaAuditBot follows robots.txt. It obeys rules for ProxaAuditBot, or for * when there’s no group for it. If your homepage is disallowed, the audit stops. It also honours Crawl-delay: it waits that long between requests and checks fewer pages if it runs out of time. To block it completely:
User-agent: ProxaAuditBot
Disallow: /What we do with the results
The audit records facts about your store’s public pages, such as whether product pages have structured data or whether robots.txt allows AI crawlers. It doesn’t collect personal data from your site. Our privacy notice has the details.
Contact
If ProxaAuditBot is causing a problem, or you want to verify a request, get in touch. Please include the time of the request and the URL it fetched.