Most AI Visibility Checkers Are Lying to You

If your AI visibility tool does not use live AI, it is not checking visibility. It is guessing.

Let that sink in for a moment.

The AI visibility industry has a problem. A big one. Most of the tools claiming to measure your "AI presence" or "AI trust score" are, at best, making educated guesses based on stale data. At worst, they are inventing numbers to make you feel like you are getting value for money.

I have been testing these tools for months. What I have found is not encouraging. It is time someone said the quiet part out loud.

The Dirty Secret Nobody Wants to Admit

Here is how most AI visibility checkers actually work:

They do not ask ChatGPT anything. They do not query Claude. They do not check what Gemini or Copilot or Perplexity would actually say about your business.

Instead, they rely on:

• Static model outputs: Responses generated months ago, cached and reused
• Training data analysis: Guessing based on what was in the training set 6-18 months ago
• Scraped SERPs: Looking at Google results and assuming AI sees the same thing
• Proxy metrics: Domain authority, backlinks, content volume, none of which directly correlate to AI recommendations
• Entirely fabricated scores: "AI Trust Score: 78/100" based on... what, exactly?

They cannot see live AI answers. They cannot verify citations. They cannot know if a brand is truly being recommended when someone asks a real question.

They are smoke and mirrors. And the industry is selling it as insight.

The Three Lies You Keep Hearing

If you have looked at AI visibility tools, you have probably seen claims like these. Let me translate what they actually mean.

"We Simulate ChatGPT Responses"

Translation: We run prompts through a static model instance with no web access, no real-time data, and no connection to what ChatGPT actually says to users today.

The model they are "simulating" might have a knowledge cutoff from eighteen months ago. It definitely does not have web search enabled. And it absolutely is not grounded in current reality.

ChatGPT with web search enabled behaves completely differently from the base model. It pulls live data. It cites sources. It gives different answers depending on what it finds today. A "simulation" cannot replicate this.

"We Measure AI Trust"

Translation: We made up a metric that sounds impressive.

There is no such thing as a universal "AI trust score". Different AI systems use different training data, different retrieval methods, different grounding approaches, and different ranking signals. What works for ChatGPT might not work for Claude. What Perplexity finds relevant, Gemini might ignore entirely.

Any tool claiming to give you a single "trust score" across all AI platforms is either lying or so oversimplifying the reality as to be useless.

"We Score Your AI Presence"

Translation: We check if your website exists and has some content. Maybe we look at your schema markup.

Presence is not the same as recommendation. Your website can be perfectly indexed, beautifully structured, and completely ignored when someone asks "who should I hire for web design in Kettering?"

The question that matters is not "does AI know you exist?" It is "does AI recommend you when it matters?"

Why This Is Technically Impossible

Let me explain why measuring AI visibility the way most tools claim to is deeply flawed.

There is no global AI index. Unlike Google, which has a single (if constantly changing) index you can optimise for, AI systems do not share a unified view of the web. Each model has different training data, different knowledge cutoffs, and different retrieval mechanisms.

Different models, different grounding. ChatGPT with web search pulls from Bing. Perplexity has its own indexing system. Claude and Gemini have their own approaches. What one system sees and recommends, another might completely miss.

There is no API for "what would AI say?" You cannot query OpenAI and ask "when someone asks about plumbers in London, do you mention Smith & Sons?" The only way to know is to actually ask, in real time, with web search enabled, the way a real user would.

Any tool that claims otherwise is either ignorant of how these systems work or hoping you are.

"It's basically the same thing, right? It's like this is very visible on top and it's like, 'I hate it.' And then a year later, everyone's like, oh, how do I get in and how do I optimize for it?"

- John Mueller, Google Search Advocate, on AI Overviews and Featured Snippets

When I first read this, I smiled. Mueller was describing the pattern I have watched play out over thirty years: initial resistance, then frantic optimisation. Featured snippets followed this curve. AI Overviews are following it now. And AI visibility is next.

But here is the thing Mueller did not quite say: you cannot optimise for what you cannot measure. And most of the industry is measuring the wrong things entirely.

How the 365i AI Visibility Checker Actually Works

We built the 365i AI Visibility Checker because we were frustrated with the alternatives. We wanted something that shows the truth, even when it is uncomfortable.

Here is what happens when you run a check:

Step 1: We Parse Your AI Identity Files

First, we look for the files that tell AI systems who you are. These are the AI Discovery File Specifications: llms.txt, ai.txt, identity.json, brand.txt, faq-ai.txt, and others. We check if they exist, if they are valid, and what they tell AI about your business.

This is your AI identity, the story you are explicitly telling language models about who you are.

Step 2: We Run Live ChatGPT Queries

Then we do something most tools cannot or will not do. We ask ChatGPT directly, with web search enabled, in real time, the way an actual user would.

Eight carefully crafted queries covering:

• Brand identity recognition
• Website discovery
• Contact information accuracy
• Local "best of" recommendations
• Service recommendations
• Action-oriented queries ("Who should I contact for...")
• Natural language mentions
• Consistency across different phrasings

Not cached. Not simulated. Not guessed. Real queries, real answers, real citations.

Step 3: We Show You Everything

The exact prompts we asked. The raw responses ChatGPT gave. The citations it included. Where your brand appeared (or did not). And crucially, in what position.

When we tested our own services, we discovered something revealing. In some queries, we ranked first. In others, ChatGPT confused "365i" with completely unrelated businesses: paddleboard companies, refrigerator brands, anything with similar-sounding names.

That is the kind of insight you cannot get from a tool that just looks at your domain authority and invents a score.

Recognition vs Discovery: The Two Questions That Matter

Most AI visibility tools conflate two very different things. We separate them deliberately.

Recognition answers the question: "Does AI know who you are?"

This covers brand identity, website accuracy, contact details, and consistency. If someone asks "Tell me about [your company]", does AI respond accurately?

Discovery answers the question: "Does AI recommend you?"

This is harder. When someone asks "Who is the best web designer in Kettering?" or "I need help with WordPress hosting, who should I contact?", does your business appear? And if so, in what position?

You can have excellent recognition and terrible discovery. AI might know exactly who you are and still never recommend you. Our research shows most UK small businesses are completely invisible to ChatGPT, making this gap particularly common. Conversely, you might appear in recommendations despite AI having incomplete information about your business.

Understanding the difference is crucial. Fixing the wrong problem wastes time and money.

The Proof: What Real Results Look Like

Let me show you what honest AI visibility checking reveals.

Here is what happens when we run an action-oriented query, "I need help with WordPress Agency Hosting in England. Who should I contact?"

You see the exact query asked. You see the position (1 of 6 in this case). You see every business mentioned. You see the actual ChatGPT response text. You see the assessment.

No mystery scores. No black boxes. No "trust us, you're doing great".

And when things do not go well? You see that too.

This is why transparency matters. A tool that just shows you a green checkmark and a high score tells you nothing about where you are actually weak.

"Zero-click organic search is a branding opportunity. If your boss/team/client is pushing you to get traffic, you should push back with data about just how much you can influence an audience on Google results."

- Rand Fishkin, SparkToro, on The Shifting SEO Landscape

Fishkin has been ahead of this curve for years. What strikes me about this quote is how directly it applies to AI visibility. Being present in the conversation matters even when you do not capture the click. When ChatGPT mentions you positively, that builds brand recognition and trust, even if the user never visits your site directly.

But you cannot optimise for presence you cannot see. And you cannot trust tools that cannot show you reality.

The Market Reckoning Coming

AI SEO is becoming about credibility, not just ranking.

Traditional SEO rewarded volume. More pages, more links, more keywords. You could game your way to page one with enough effort and budget.

AI recommendations reward trust. Language models are trying to give the best answer, not show the most results. They cross-reference. They check consistency. They evaluate whether your content actually helps people.

The tools that lie, that invent scores and simulate responses and sell you confidence based on nothing, will eventually be exposed. When enough people realise their "AI Trust Score" of 85 means nothing because they never appear in actual AI recommendations, the credibility collapse will be swift.

We are betting on honesty. On showing you exactly what AI says, even when it is not what you want to hear. On measuring what matters, not what is easy to fake.

We wrote about this in more detail in our detailed comparison of AI visibility tools. The differences are stark.

What You Should Actually Do

Stop trusting scores you cannot verify. Start asking tools to show their work.

If a tool claims to measure your AI visibility, ask these questions:

1. Does it use live AI queries with web search enabled?
2. Can you see the actual prompts it asked?
3. Can you see the actual responses it received?
4. Does it show citations and sources?
5. Does it tell you your position relative to competitors?
6. Does it separate recognition from discovery?

If the answer to any of these is "no" or "I don't know", the tool is hiding something.

And then actually test it. Run a check with the 365i AI Visibility Checker. Open ChatGPT yourself and ask the same questions about your business. Compare the results.

You might discover you are doing better than you thought. You might discover AI has completely wrong information about you. Either way, you will know the truth, and that is worth more than any fabricated score.

Frequently Asked Questions