How to Verify OAI-SearchBot in WordPress Server Logs

To verify OAI-SearchBot in WordPress server logs, check both the claimed user-agent and the request’s trustworthy client IP against OpenAI’s published search-crawler ranges. A bot name alone is easy to imitate. This tutorial produces a reviewable CSV showing matches, unresolved identities and delivery problems.

Quick Answer

Export request records, establish which field contains the original client IP, download the current searchbot IP list, and compare addresses using CIDR membership. Then inspect paths and response codes. A matched request documents crawler activity; it does not establish a ChatGPT citation or human visit.

Key Takeaways

• Use trustworthy client addresses, especially behind a CDN.
• Keep the IP-list snapshot alongside the dated report.
• Investigate failed retrieval separately from crawler identity.

Prerequisites: obtain the right WordPress logs

You need read access to hosting or CDN request logs, an export covering a defined period, Python 3, and a terminal with curl. WordPress administrator access alone may not provide these logs. Ask your host which download contains public HTTP requests for your exact hostname.

Prepare a UTF-8 comma-separated file named requests.csv with these exact headers:

timestamp,client_ip,path,status,user_agent

Export actual values into those columns using your host’s export or a spreadsheet. Include timezone offsets in timestamps, keep status codes, and retain full user-agent strings. Quote CSV fields containing commas. This script expects normalized CSV; it does not parse raw Apache or Nginx lines. Apache’s logging documentation explains why available fields depend on the configured format.

If a CDN serves cached pages, origin logs can miss those requests. Prefer edge records when available and state which layer you inspected. Remove unnecessary query parameters and keep logs private because request records can contain personal information.

Documentation and IP-list format checked on 12 October 2026.

Step 1: establish a trustworthy client IP

Behind a proxy, the connection IP may belong to that proxy. Confirm how your host restores the original visitor address. For Cloudflare deployments, its HTTP-header documentation describes CF-Connecting-IP and address-handling differences.

Do not copy an arbitrary browser-supplied forwarding header into client_ip. Have the administrator confirm the trusted proxy chain and logging configuration. Supply one plain IPv4 or IPv6 address per row, without a port or comma-separated chain. If provenance is unclear, label the identity unresolved rather than making an allowlist decision.

Step 2: download the current crawler ranges

OpenAI’s crawler reference identifies OAI-SearchBot as its search crawler and links to its published IP list. GPTBot and ChatGPT-User have different purposes and lists. This workflow checks only search-crawler claims; our WordPress crawler configuration guide covers those policy differences.

Download the list to your working folder:

curl --fail --silent --show-error --location https://openai.com/searchbot.json --output searchbot.json

Open the file and confirm it contains JSON with a prefixes array. Preserve the original file, download date and any supplied creation timestamp. Do not hard-code the address ranges from this article: a current list cannot conclusively classify an old request if ranges changed.

Step 3: verify OAI-SearchBot with a local script

Save this as verify_searchbot.py. It reads local files, uses Python’s ipaddress library for IPv4/IPv6 network membership, and writes only rows claiming the search crawler. No paid service or extra Python package is required.

import csv, ipaddress, json, re, sys

with open(sys.argv[2], encoding="utf-8") as f:
    data = json.load(f)
nets = []
for item in data["prefixes"]:
    for key in ("ipv4Prefix", "ipv6Prefix"):
        if key in item:
            nets.append(ipaddress.ip_network(item[key]))
if not nets:
    raise ValueError("No IP prefixes found")

with open(sys.argv[1], newline="", encoding="utf-8-sig") as f:
    reader = csv.DictReader(f)
    fields = ["timestamp", "client_ip", "path", "status", "user_agent"]
    if not set(fields).issubset(reader.fieldnames or []):
        raise ValueError("Missing required CSV headers")
    writer = csv.DictWriter(sys.stdout, fieldnames=fields + ["range_check"])
    writer.writeheader()
    for row in reader:
        if not re.search(r"\bOAI-SearchBot/\d", (row["user_agent"] or ""), re.I):
            continue
        try:
            ip = ipaddress.ip_address((row["client_ip"] or "").strip())
            match = any(ip.version == n.version and ip in n for n in nets)
            result = "range_match" if match else "no_range_match"
        except ValueError:
            result = "invalid_ip"
        out = {key: row[key] for key in fields}
        out["range_check"] = result
        writer.writerow(out)

Run it from the folder containing both inputs:

python3 verify_searchbot.py requests.csv searchbot.json > checked.csv

Open checked.csv, filter by range_check, and inspect paths and status codes. The script preserves original records and does not change WordPress, robots.txt or firewall rules. A header-only output means no matching user-agent claims were found in that export.

Step 4: interpret an illustrative report

The following cases are illustrative, not observations from a tested website. Assume the first address belongs to the downloaded snapshot and the original client-IP provenance is trustworthy.

Example	Output	Next check
104.210.140.129; article; 200	range_match	Inspect whether useful HTML was delivered.
203.0.113.10; article; 200	no_range_match	Investigate provenance or an imitated bot claim.
Matched address; article; 403	range_match	Review the corresponding security event.
Address contains a port or proxy chain	invalid_ip	Correct the export with administrator-confirmed fields.

A 200 can contain a challenge or error page, so status alone does not prove successful content retrieval. Follow redirects to their final response. Distinguish article requests from robots.txt or asset requests; fetching the policy file does not demonstrate an article crawl. Use our 404 troubleshooting guide for genuinely missing pages.

Troubleshooting and evidence checklist

• File errors: If JSON parsing or header validation fails, correct the input before interpreting output. Keep the files in the same folder and use their exact names.
• No rows: Check hostname, time window, rotated files, missing user-agents and CDN caching before concluding there were no requests.
• Unexpected non-matches: Recheck the trusted IP field and current snapshot. A non-match is a review flag, not proof of malicious activity.
• 403, 429 or 5xx: Correlate the timestamp and path with firewall, rate-limit or server events. Preserve evidence before proposing a narrow repair.
• Handoff: Record log layer, date range, timezone, snapshot, affected paths, response evidence and proposed next action. Never disable security broadly.

Review delivered policy separately using our robots.txt guide. Repeat the same evidence checks after a repair instead of assuming an editor save restored access.

Keep this report separate from the free AI search visibility workflow. Google’s current channel definitions assign recognized assistant referrers the ai-assistant medium and exclude Google AI Overviews and AI Mode from AI Assistant. Google’s AI features guidance instead describes ordinary Search eligibility. These systems cannot turn a crawler request into a measured citation or visit.

Frequently Asked Questions

Can I verify the crawler by changing curl’s user-agent?

No. That tests responses to your own connection using a chosen header. It does not place your request inside OpenAI’s published networks. Label it a diagnostic simulation. For a real log record, compare the trustworthy original client address with the appropriate dated IP-list snapshot.

Should every range match receive a firewall exception?

No. Confirm the client-address provenance, requested resource and specific security event first. A log report is diagnostic evidence, not an automatic authorization policy. Have the administrator review a narrowly scoped exception only when a genuine request to intended public content was incorrectly blocked.

Does a matched crawl prove ChatGPT cited my article?

No. Retrieval, answer citations and visitor sessions are different observations. Record citations only from actual responses containing the page link. A crawl does not show whether the content was used. Our free AI visibility workflow explains how to keep those measurement records separate.

Why might GA4 show AI Assistant instead of Referral?

Google’s current default-channel rules use the ai-assistant medium and assign it for recognized assistant referrers. This concerns attributed visitor sessions, not these crawler records. Check source and medium alongside channel; do not reclassify unidentified Direct visits or Google organic sessions as measured AI exposure.

Will successful crawler access improve Google AI visibility?

This report checks OpenAI requests only. Google’s AI features require ordinary Search eligibility, including indexed pages eligible for snippets; access to another provider proves none of that. Keep Google indexing checks separate. Neither an IP match, special schema nor a training-bot permission guarantees AI inclusion.

Conclusion

Verify OAI-SearchBot by joining trustworthy request records with the correct IP snapshot, then reviewing delivery separately. Keep the resulting report small and actionable: identity evidence, affected paths, observed responses and unresolved limitations. Measure citations and human visits through their own evidence rather than treating crawls as exposure.