Is AI actually reading your website? I had AI build a counter that tracks SEO and GEO exposure separately
On the first day my website went live, I wanted to know two things:
- Have search engines like Google, Baidu and Bing come to look at my website?
- Have AI tools like ChatGPT, DeepSeek, Doubao and Kimi come to read my website?
The first question is about SEO: whether people can find you when they search. The second is about GEO: whether AI mentions you when people ask it questions. In foreign trade, more and more buyers ask AI first and then go looking for suppliers, so the second question matters more every month.
Common analytics tools such as Baidu Tongji or Google Analytics can't answer either question. They only count real people opening pages in a browser. The programs sent by search engines and AI companies (called 'crawlers') aren't counted at all.
So I had AI build a counter for my website. It counts SEO and GEO separately and lists, for each platform, how many times it came, why it came, and which pages it read. These numbers are only visible in the website's admin area; visitors can't see them. I didn't write a single line of code and didn't spend a cent. Below is exactly how I did it, so you can follow along.
1. First, tell three numbers apart
'Exposure' actually comes in three kinds, from different sources:
| What is counted | Where the data comes from | What it tells you |
|---|---|---|
| How many times search engine and AI crawlers fetched your pages | The access log on your own server | They know you and are reading you. This comes before being indexed or cited |
| How many people clicked through from search results or AI answers | Also the server's access log | Real people have already found you through them |
| How many times your pages appeared in other people's search results | Official data from Google Search Console and Bing Webmaster Tools | True 'impressions', plus the words people searched for |
The server already records the first two numbers; nobody counts them. Google and Bing give the third number away for free; you just need to connect it. I put all three into my admin area.
2. SEO and GEO in plain words
- SEO exposure = search engines send crawlers to read your pages + people click through from search results.
- GEO exposure = AI companies send crawlers to read your pages + people click through from AI answers.
AI crawler visits come in three kinds, and they're not worth the same:
| Kind | Examples | Value |
|---|---|---|
| Reading live while a user asks a question | ChatGPT-User, Perplexity-User | Highest. Someone is asking AI a question, and AI came to read your page on the spot |
| Building an index for AI search | OAI-SearchBot, PerplexityBot | High. When someone asks a related question later, AI may cite you |
| Training and data collection | GPTBot, ClaudeBot | Moderate. Your content has been taken in, but that doesn't mean anyone saw it |
3. My real numbers on day one
| Channel | Who came | Visits |
|---|---|---|
| SEO | 34 | |
| SEO | Bing | 6 |
| SEO | Baidu | 6 |
| GEO | Google's research crawler (GoogleOther) | 29 |
| GEO | Anthropic's ClaudeBot | 26 |
| GEO | ChatGPT, DeepSeek, Doubao, Kimi, Yuanbao, Perplexity | 0 |
You can see at a glance what to do next: Google, Bing and Baidu have all come, so the SEO road is open. On the AI side only Claude has come; ChatGPT and the Chinese AI tools haven't visited yet. That's what I need to work on.
Search-result impressions from Google and Bing were both 0 at the time. That's normal: a new website usually takes 1 to 2 weeks to start appearing in search results.
4. What you need
| What | Notes |
|---|---|
| Your website on your own server | Any cloud server such as Alibaba Cloud or Tencent Cloud works, with Nginx (the most common web server software). The server records every visit automatically; that's the raw material for counting |
| An AI assistant that can connect to the server and do the work | I used Claude Code. Any AI that can carry out actions on your computer works, such as Codex, Cursor, Trae or WorkBuddy. One that can only chat won't work |
| An admin area for your website (optional) | I already had one for reading contact messages, so the counter went in there. If you don't, have AI build a password-protected page |
| Google Search Console and Bing Webmaster Tools accounts | Only needed for search-result impressions; both are free. It's best if your website is already verified on both |
If your website is hosted by a site builder (such as Shopify, or a WordPress site someone else hosts for you), you can't get the server's access log, so the first two numbers are out of reach. You can still connect the third, and you can always look at Google's and Bing's own dashboards.
5. The two free lists I used
How does AI know which visit is Google and which is ChatGPT? Every crawler announces its name when it visits. People on GitHub maintain lists of these names for everyone to use for free:
| List | What it covers | Address |
|---|---|---|
| ai.robots.txt | About 200 AI crawler names, with which company runs each one and what it's used for | github.com/ai-robots-txt/ai.robots.txt |
| crawler-user-agents | Over 1,500 crawler names, tagged as search engine, AI and so on | github.com/monperrus/crawler-user-agents |
Both are under the MIT open-source license, so you can use them freely; just credit the source. The lists get updated, so have AI download them again every few months.
6. What you do and what AI does
| You do it yourself | AI does it for you |
|---|---|
| 1. Say clearly what you want: count SEO and GEO separately, visible only in the admin area | Read the server's access log and count who came |
| 2. Approve changes to the live server (AI will stop and ask you) | Download the open-source lists and write the sorting and counting program |
| 3. Accept the Google Cloud terms of service | Build the admin page, test it, then put it live |
| 4. Type the Bing API key into the server | Create the Google Cloud project, set permissions and generate the key in the browser for you |
| 5. Open the admin page and check the numbers make sense | Make a shortcut that opens the admin page with a double-click |
Why you do items 3 and 4 yourself: accepting terms is something you agree to in person; a key is like a house key, so you should put it on the server yourself rather than pass it through AI.
7. Seven steps
Each step includes what to say to AI. You can copy it straight over.
Step 1: See who has come
Look at the Nginx access log on my server and count how many times search engine crawlers like Google, Bing and Baidu, and AI crawlers like ChatGPT, Claude, DeepSeek and Doubao have visited. Just look for now; don't change anything.
This step is only to get a picture. On my server, Google was already crawling page by page, and Claude's crawler had come too.
Step 2: Turn it into a counter that keeps counting
Using the open-source lists ai-robots-txt/ai.robots.txt and monperrus/crawler-user-agents on GitHub, build an SEO and GEO exposure counter:
- SEO = search engine crawler visits + visits clicked through from search results
- GEO = AI crawler visits + visits clicked through from AI answers (chatgpt.com, DeepSeek, Doubao, Kimi and so on)
- Read the server's access log automatically every minute; count only pages, not images or failed requests
- Show the results only in the admin area, behind a password, never on the public pages
- Test it locally on the server's real log first, show me screenshots, then put it live
AI tries it on your computer first, runs it on the server's real log and shows you, and only goes live when it works. Before going live it stops and asks for your approval, and only moves when you say yes.
Step 3: Catch impostors
Some visitors pretend to be Google's crawler. Use reverse DNS lookups to check whether crawlers claiming to be Google, Bing, Baidu and other search engines are real. Don't count impostors; list them separately. If a lookup returns nothing, don't treat that as an impostor.
This is the method Google and Bing themselves recommend: check whether the visiting address really belongs to Google. The last sentence matters a lot; the pitfalls below explain why.
Step 4: List by platform, including the ones that haven't come
In the admin area, list SEO and GEO exposure by platform: how many times each came, why it came (crawling, training, AI search, live visits while a user asks, clicks from answers), which pages it read most, and when it last came. For key platforms (Google, Bing, Baidu, Sogou, 360, ChatGPT, DeepSeek, Doubao, Kimi, Yuanbao, Tongyi, ERNIE Bot, Perplexity), list them even if they haven't come, showing 0.
That last sentence is one I added later, and it turned out to be the most useful. Seeing who came isn't enough; you need to see who hasn't come yet. That's where to put your effort.
Step 5: Connect Google's and Bing's official impression data
Connect the Google Search Console and Bing Webmaster Tools APIs, and bring impressions and clicks for every search term and every page into the admin area. Click whatever you can on the web pages yourself; stop and call me wherever terms need accepting or a key needs entering.
Both are free. AI does these things in the browser for you:
- Google: create a project in Google Cloud, enable the Search Console API, create a 'service account', add it to Search Console as a 'Restricted' user (read-only), and generate a key file that stays on your computer. No billing and no card needed. The first time you use Google Cloud you have to accept its terms of service; click that yourself.
- Bing: in Bing Webmaster Tools, click the gear in the top right, go to 'API access' and then 'API key', and click 'Generate'. Then type the key into the command AI gives you.
Step 6: Make a shortcut you can double-click
The admin area is only open inside the server. Make me a shortcut that opens it with a double-click, and pull the latest Google data every time before opening.
My admin area isn't open to the internet; you have to connect to the server first. AI made a file that, when double-clicked, connects, pulls the data and opens the browser. I enter the password once and I'm in.
Step 7: Check it yourself
Open the admin page and go through the 5 checks in section 9 below.
8. Pitfalls I hit, so you can avoid them
1. The admin page showed SEO as 0, when Google had actually come over 30 times
In the first test, SEO showed 0. My computer was running proxy software, so the reverse DNS lookups returned nothing, and the program treated the real Google as an impostor and threw everything out. I changed it so that a visitor only counts as an impostor when the lookup clearly points to someone else's address; when the lookup returns nothing, the visit still counts and is marked 'unverified'.
If a number looks wrong, always ask AI why. I only caught the problem because another conversation had found that Google came over 30 times, and the two didn't match.
2. Baidu's crawler nearly got counted as AI
One name on the AI crawler list is simply 'Spider'. That's far too broad: it would count Baidu's and Sogou's crawlers as AI. The fix is to check whether a visitor is a search engine first and only then whether it's AI, and to skip names that are too broad.
3. Some visits aren't exposure
Google visits when it verifies site ownership, Google's own testing tools visit, and various services fetch the site icon. None of this is exposure, so don't count it. Google also has a research crawler called GoogleOther that doesn't feed search results; I counted it under GEO.
4. Servers in mainland China can't reach Google
My server is on Alibaba Cloud in Hangzhou, and every connection to Google's API timed out in my tests. So the Google data is pulled on my own computer (which can reach Google) and then sent to the server. Bing is reachable from China, so the server pulls it itself every 6 hours.
5. Don't let AI type in the key
AI can click its way to 'Generate key' on the web page for you, but it won't put the key on the server for you, and that's right. The AI I used works in its own built-in browser, and clicking 'Copy' there didn't copy anything to my computer. In the end I typed the key into the terminal by reading it off the screen: about thirty characters, one minute.
6. AI stops and asks before changing the live server
Changing a live website affects every visitor, so a good AI assistant stops and waits until you clearly say you approve. Mine wouldn't even accept 'let's do it all together'; I had to say plainly that I approved deploying to the production server. Don't see it as a hassle; it's protecting you. Have AI back up the old files before changing anything, so you can always roll back.
7. Zero impressions? Don't panic
A new website usually takes 1 to 2 weeks to show up in search results, and Google's data itself runs 2 to 3 days behind. Zero for the first few days is normal.
9. When you're done, check these 5 things yourself
- The admin page shows the two big numbers for SEO and GEO, and neither is 0 (a day or two after launch, Google has usually been by)
- Under 'Where exposure comes from', platforms like Google and Bing have counts, and platforms that haven't come are greyed out as 'not yet'
- The admin page won't open without the password, and none of these numbers appear on any public page
- The Google and Bing cards both show 'last synced' with a time, not 'sync failed'
- The impressions and clicks on the two cards roughly match what Google Search Console and Bing Webmaster Tools show
10. What to do with these numbers
- The greyed-out platforms are your to-do list. ChatGPT, DeepSeek and Doubao are all at 0 for me right now, so I need to work out how to get them to read my site.
- Many Chinese AI tools borrow search engines to find information. When DeepSeek, Doubao, Kimi and Yuanbao search the web, many of them use results from Baidu, Bing, Sogou or Toutiao Search. So if they haven't sent a crawler, it doesn't mean they haven't used your content. To get cited by them, get indexed well by Baidu, Bing, Sogou and Toutiao first.
- Look at the most-read pages. Pages that AI and search engines come back to again and again are the ones they find valuable; write more like them.
- The most valuable signals are live visits while a user asks and clicks from AI answers. Once these start showing numbers, real customers are finding you through AI.
If you get stuck while following along, feel free to contact me through the website.
// FAQ
- I don't know how to code. Can I do this?
- Yes. I didn't write any code. What you do is say clearly what you need, approve a few times, type in one key, and check the result at the end.
- Does it cost money?
- No. Both lists are free, Google's and Bing's APIs are free, and reading your server's access log costs nothing. I didn't turn on any paid service.
- My website is on Shopify or hosted by someone else. Can I do this?
- Without the server's access log, the first two numbers are out of reach. You can still connect the impression data from Google Search Console and Bing Webmaster Tools, or just look at those two dashboards directly.
- Is this number how many people saw me?
- Not exactly. Crawler visits show that search engines and AI are reading you, which comes before being seen. How many times you actually appeared in search results is on the Google and Bing cards. Watch both.
- Can visitors see these numbers?
- No. They're only in the admin area, behind a password. My admin area can't even be opened from the internet; I can only connect to it from my own computer.