Methodology
How do we compare AI bot traffic trackers?
Every report on Bot Trackers follows the same process, so that prices, features and crawler details can be checked by anyone.
Which sources do we use?
- Vendor and operator pages first. Pricing, plan limits and features come from each vendor's pricing, product and documentation pages. User agents, purposes and IP ranges come from the crawler operators' own documentation.
- Independent sources second. Studies, standards documents and reputable reporting are used for context and cited with a numbered source.
- Dated checks. Every table states the date on which it was checked. When sources disagree, we follow the vendor or operator page and say so.
Which criteria matter for a bot tracker?
| Criterion | What we look at |
|---|---|
| Data source | CDN integrations, log upload, log API or a JavaScript tag; supported hosts and log formats |
| Bot coverage | Which AI user agents are recognised, and whether new ones are added |
| Purpose classification | Training, search index and user-triggered fetches reported separately |
| Verification | IP-range or reverse-DNS checks to separate real crawlers from spoofed user agents |
| Page and status detail | Fetched URLs, status codes, blocked requests, crawl frequency |
| Connection to outcomes | Whether crawler data is linked to AI citations and referral visits |
| Price to start | Which plan includes bot analytics, and what it costs |
What do we not do?
- We do not publish star ratings, invented scores or anonymous reviews.
- We do not present illustrative examples as real traffic data; samples are labelled as such.
- We correct errors when a vendor or reader points them out and note the new check date.
How often are reports updated?
Crawler names, vendor features and prices change often. We re-check every report at least monthly and whenever an operator documents a new crawler or a vendor changes its pricing.