Crawlers Trfyx checks
Every AI assistant, search index, and training crawler classified on the server — because those bots request HTML without running your analytics script.
Other AI bots
AI crawler / 11 crawlers
OpenAI
OAI-AdsBot
Fetches landing pages for OpenAI advertising surfaces.
xAI
GrokBot
xAI crawler associated with Grok product fetches.
xAI
xAI-Bot
Generic xAI crawler token seen on product fetches.
xAI
xAI-Grok
Alternate Grok crawler token from xAI.
xAI
xAI-Web-Crawler
xAI’s general web crawler token.
xAI
Grok
Short Grok token xAI still sends on some requests.
Meta
FacebookBot
Meta crawler traffic associated with Facebook bot fetches.
ByteDance
Doubaobot
ByteDance crawler associated with Doubao product fetches.
Baidu
YiyanBot
Baidu crawler associated with Ernie Bot / Yiyan product fetches.
Alibaba
TongyiBot
Alibaba crawler associated with Tongyi product fetches.
Alibaba
AliyunBot
Alibaba Cloud crawler seen on product and research fetches.
See which of these
hit your site
Trfyx splits these crawlers from people, with the pages each one requested. One server-side package.
EU-hosted cloud