AI Agent Analytics: Tracking Agent Visits in Logs and GA4

Most guides to AI agent analytics stop at one step: build a GA4 segment for chatgpt.com and perplexity.ai, then call the result your AI traffic. That number is real, but it only counts the humans who clicked a link in an AI answer. The agents themselves, meaning the crawlers, the fetchers that read your page while a user waits, and the cloud browsers that fill in your forms, mostly never fire a GA4 tag. Some fire it and look exactly like a person on Chrome. To measure what AI is actually doing on your site, you need three data sources, and you need to know which question each one can answer.

What AI Agent Analytics Actually Measures

“AI traffic” covers four different things. Each one shows up in a different place.

• Training crawlers such as GPTBot and ClaudeBot collect pages that may be used to train models. They appear in server logs only.
• Search crawlers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot build the indexes that AI answers are drawn from. These also appear in logs only.
• User-triggered fetchers and agents such as ChatGPT-User, Claude-User, Perplexity-User and Google-Agent visit a page because a person asked a question or gave a task. They mostly appear in logs. Full cloud browsers may also execute your analytics tag.
• Referral clicks. A human reads an AI answer and clicks through. This is the only layer that GA4 reports reliably.

The layers form a funnel. Crawls make you eligible. Fetches mean your page is being read during a live conversation. Referrals mean someone cared enough to visit. Treat them as one number and you lose the most useful signal: where pages drop out between being read and being clicked.

Why GA4 Can’t See Most AI Agent Traffic

GA4 records a visit when its JavaScript tag runs in a browser. Crawlers and simple fetchers request the HTML and leave without running scripts, so GA4 never learns they came. That is how GA4 is built, and no setting changes it.

The agents that do run JavaScript run into a second filter. Google’s GA4 help documentation says known bot traffic is excluded automatically, using the IAB’s International Spiders and Bots List plus Google’s own research. It also says you cannot disable that exclusion or see how much traffic it removed. So when a headless agent executes your tag, you can’t tell whether GA4 dropped the hit as a bot or counted it as a user.

Agentic browsers make it worse. In an independent test published in October 2025, the KP Playbook found that ChatGPT Atlas reported itself exactly like Chrome on a Mac. It appeared as “Chrome” in GA4, and its search clicks were attributed to chatgpt.com / referral, so it couldn’t be separated from ordinary ChatGPT traffic. A human browsing in Atlas and Atlas’s agent working on that person’s behalf can look identical in GA4. Our guide to agentic browsers covers what that means for SEO.

The AI Bot User Agents Worth Tracking

The vendors publish their own lists, and the lists change. Check these pages rather than copying a list from a blog post, this one included. As of writing:

• OpenAI (developers.openai.com bots page): GPTBot (training), OAI-SearchBot (ChatGPT search), ChatGPT-User (user-initiated visits and GPT Actions) and OAI-AdsBot (checks ad landing pages). Each one has a published IP list, for example openai.com/chatgpt-user.json. OpenAI says robots.txt rules “may not apply” to ChatGPT-User because a user starts those actions.
• Anthropic (Claude help centre): ClaudeBot (training), Claude-SearchBot (search quality) and Claude-User (fetches for a user’s question). IP ranges are at claude.com/crawling/bots.json. Anthropic separately says that browsers it hosts for Claude to complete tasks identify as Claude-Agent.
• Perplexity (docs.perplexity.ai): PerplexityBot (search results) and Perplexity-User (user requests), each with its own IP JSON. Perplexity says Perplexity-User generally ignores robots.txt.
• Google (Google Search Central, user-triggered fetchers): Google-Agent, described as letting agents on Google infrastructure browse and act on a user’s request. Its IPs are in user-triggered-agents.json, and like the other user-triggered fetchers it generally ignores robots.txt.

Pay attention to the user-triggered tokens. They are the closest thing you have to demand data for AI answers, because every hit means a real person’s question led an AI to your page at that moment.

Reading AI Bot Traffic in Server Logs

Server logs record every request whether or not JavaScript ran. That makes them the foundation of AI agent analytics. You can get them from your origin server (Nginx or Apache access logs), your host’s log export, or your CDN. On many managed WordPress hosts you’ll need to ask support for raw access logs, because the dashboard only shows a summary.

A first pass doesn’t need special tooling. This counts requests per AI agent in a standard access log:

grep -oE 'GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-SearchBot|Claude-User|PerplexityBot|Perplexity-User|Google-Agent' access.log \
  | sort | uniq -c | sort -rn

From there, pull out three views:

• Top URLs per user-triggered agent. These are the pages AI answers are reading right now. Compare them against your money pages.
• Status codes by agent. A 403 from your WAF, a 404 or a 5xx served to ChatGPT-User means an answer was written without your page.
• Crawl recency by section. Find which directories the search crawlers haven’t touched in weeks.

If you already do log file analysis for SEO, this is the same workflow with a new list of user agents.

Verify Before You Count: Spoofing and Signed Agents

Anyone can put “ChatGPT-User” in a user-agent header. Scrapers do it all the time because some sites wave AI bots through. If you count user-agent strings without checking them, your AI bot traffic report will include impostors. There are two ways to verify.

IP verification. Match each request’s IP against the vendor’s published ranges (the JSON files above). This works today for every major vendor. Automate it, because the ranges change.

Cryptographic signatures (Web Bot Auth). Here the agent signs its requests using HTTP Message Signatures (RFC 9421) and publishes its public keys at a well-known URL. Per Cloudflare’s documentation, a signed request carries Signature, Signature-Input and Signature-Agent headers. OpenAI publishes a key directory at chatgpt.com/.well-known/http-message-signatures-directory naming https://chatgpt.com as the signature agent, and its ChatGPT agent allowlisting help article says Cloudflare verifies those signatures automatically. Google’s user-triggered fetchers page says it is experimenting with Web Bot Auth under the identity agent.bot.goog. The standard is still a draft: the IETF webbotauth working group’s protocol document (draft-ietf-webbotauth-httpsig-protocol) is a working-group draft, and no RFC has been published. Our Web Bot Auth explainer covers the mechanics.

The rule we use: count a hit as verified when it passes the IP check or the signature check. Count anything else that claims an AI token as “unverified”, and report it in a separate column.

Setting Up GA4 for AI Referral Traffic

GA4 still matters for the click layer, and the default setup is better than it used to be. Google’s default channel group now includes an AI Assistant channel. Its rule is a medium that exactly matches “ai-assistant”, which GA4 sets when the referrer is on its list of AI assistants (Google names ChatGPT, Gemini, DeepSeek, Copilot and Grok as examples). Google’s AI Overviews and AI Mode are not included. They count as Organic Search, so look for them in Search Console, not here.

The built-in list won’t cover every assistant your audience uses. To catch the rest, build a custom channel group, copy the defaults, and add an “AI Referral” channel where session source matches a regex such as:

chatgpt\.com|openai\.com|perplexity|claude\.ai|gemini\.google|copilot\.microsoft|deepseek|grok\.com

Order matters. Google’s documentation says traffic “is included in the first channel whose definition it matches,” so move your AI channel above Referral. Standard properties get only two custom channel groups. Review the regex every quarter, because assistants launch and change domains.

Remember what this channel measures: humans who clicked. Some AI clicks still arrive with no referrer and land in Direct, so read the channel as a floor rather than a total.

Agentic Browsers: The Blind Spot You Should Admit

Agentic browsers and cloud agents load your page in a real browser engine. They run your tags, scroll, click and submit forms. When the agent is a hosted browser that signs its requests, as OpenAI’s cloud browser does, your CDN or WAF can recognise it at the edge, even though GA4 can’t. When the agent runs inside the user’s own browser on their own machine, the vendors document no public signal that separates its actions from the user’s.

So don’t fingerprint sessions by guesswork (scroll speed, form-fill timing) and report the result as fact. Do three honest things. Label edge-verified agent sessions in your logs. Watch conversion paths for odd patterns, such as forms completed with no prior page views. And state in reports that some share of “human” GA4 sessions may be agent-driven, by an amount nobody can currently measure.

Should You Send Agent Hits Into GA4?

You can. Middleware can detect a verified agent and send an event through the GA4 Measurement Protocol, which needs no browser JavaScript. We’d advise against sending those events into your main property. They inflate users and sessions, distort engagement rate, and break year-over-year comparisons the moment you switch them on.

If stakeholders want everything in one dashboard, send agent events to a separate GA4 property or straight to a warehouse table, then join the two in Looker Studio. Keep the human property human. Your conversion rates and your agentic search numbers stay readable that way.

The Weekly AI Agent Analytics Report

A useful report fits on one page and answers four questions:

• Coverage: verified crawler hits by vendor and site section, and whether key pages were crawled recently.
• Live demand: verified user-triggered fetches by URL. This is the closest proxy you have for “AI answers are using this page.”
• Failures: non-200 responses served to AI agents. These are the fastest wins in the report.
• Clicks: AI Assistant and custom AI channel sessions and conversions from GA4, next to fetches for the same URLs.

The comparison between the last two is where the insight is. A page that is fetched often but rarely clicked is being answered in full inside the chat. Either add something worth clicking through for (a tool, a calculator, original data), or accept that the page now builds brand awareness rather than traffic. If you’re on Cloudflare, AI Crawl Control (formerly AI Audit) does much of the coverage and failure work for you. Cloudflare lists it as available on all plans, with per-crawler metrics and status-code charts, and its changelog notes that referrer analytics need a paid plan.

Where SEO Rocket Fits

SEO Rocket doesn’t parse your raw server logs. Keep that work in your log tool or CDN. It covers the steps on either side. Connect GA4 and Search Console in the app and you can ask in plain English how your AI Assistant sessions moved this month, and which landing pages they hit. AI-visibility tracking shows which prompts cite you, which is the part of the funnel no log can see. When your logs show agents getting 404s or slow responses, the real-crawler site audit (Screaming Frog-backed, with JavaScript rendering) finds the broken templates behind them. It’s the same playbook proven across 1,000,000+ ranking pages, applied to a new kind of visitor.

FAQ

Can GA4 track AI bot traffic?

Not reliably. Most AI bots don’t run JavaScript, and GA4 automatically excludes known bots in a way you can’t turn off or audit. GA4 is good for AI referral clicks. Use server or CDN logs for AI bot traffic.

Which AI agent traffic should I never block?

That’s a business decision, but most sites that want AI visibility allow the search crawlers and the user-triggered fetchers. Blocking ChatGPT-User or Perplexity-User at your WAF removes your page from live answers (robots.txt may not stop them, since both vendors say user-triggered fetches can ignore it). Blocking a training crawler only affects future model training.

Are AI agents in my logs real or spoofed?

Check each one against the vendor’s published IP ranges, or against a Web Bot Auth signature where the vendor supports it. Anything that fails both checks belongs in an “unverified” bucket, not in your AI traffic totals.

How often should I review AI agent analytics?

Weekly for failures and live-demand pages, and quarterly for the user-agent list and the GA4 regex. Bot names, IP files and domains change often.