Traffic and Server Logs

Two kinds of evidence

Visitors: in your analytics, look at referrer domains such as chatgpt.com, perplexity.ai, gemini.google.com, claude.ai and copilot.microsoft.com, and add UTM parameters to links you control. Note that some AI traffic arrives with no referrer and appears as "direct", so the numbers are a lower bound. Crawlers: your server access logs show which AI user agents fetch which pages, how often, and whether they get errors. Rising crawler hits on pricing and docs pages, and a healthy 200 response rate, are good signs that your content is reachable.

Counting AI crawler hits, run

I ran this on five made-up log lines: GPTBot 2, ClaudeBot 1, PerplexityBot 1 and one normal browser. Real logs are far larger, and you should also check IP ranges the providers publish, because user-agent strings can be faked.

import re
from collections import Counter
logs = [
 '1.2.3.4 - - [01/Oct/2026] "GET /pricing HTTP/1.1" 200 512 "-" "Mozilla/5.0 (compatible; GPTBot/1.1; +https://openai.com/gptbot)"',
 '1.2.3.5 - - [01/Oct/2026] "GET /docs HTTP/1.1" 200 900 "-" "Mozilla/5.0 (compatible; ClaudeBot/1.0)"',
 '1.2.3.6 - - [01/Oct/2026] "GET /pricing HTTP/1.1" 200 512 "-" "Mozilla/5.0 (compatible; PerplexityBot/1.0)"',
 '9.9.9.9 - - [01/Oct/2026] "GET / HTTP/1.1" 200 400 "-" "Mozilla/5.0 (Windows NT 10.0) Chrome/120"',
 '1.2.3.4 - - [01/Oct/2026] "GET /blog HTTP/1.1" 200 700 "-" "Mozilla/5.0 (compatible; GPTBot/1.1)"',
]
BOTS = re.compile(r"(GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-User|PerplexityBot|Google-Extended|CCBot)")
hits = Counter()
for line in logs:
    ua = line.rsplit('"', 2)[-2]
    m = BOTS.search(ua)
    hits[m.group(1) if m else "other"] += 1
print(dict(hits))

Output:

{'GPTBot': 2, 'ClaudeBot': 1, 'PerplexityBot': 1, 'other': 1}

Classifying referrers, run

Group referrers into AI assistants, search and other so you can chart AI-sourced visits over time.

from urllib.parse import urlparse
AI = {"chatgpt.com": "ChatGPT", "perplexity.ai": "Perplexity",
      "gemini.google.com": "Gemini", "claude.ai": "Claude",
      "copilot.microsoft.com": "Copilot"}

def source(ref):
    host = (urlparse(ref).hostname or "").removeprefix("www.")
    return AI.get(host, "search" if host in ("google.com", "bing.com") else "other")

refs = ("https://chatgpt.com/", "https://www.perplexity.ai/search?q=x",
        "https://www.google.com/", "https://news.example/")
print([source(r) for r in refs])

Output:

['ChatGPT', 'Perplexity', 'search', 'other']

Quick check: Why might AI-referred visits be undercounted in analytics?

• Some arrive without a referrer and show as direct
• AI assistants block analytics
• Referrers are never recorded
• They are always overcounted