AI visibility tracking should help you decide whether something needs attention. That gets difficult when the evidence arrives as a screenshot in Slack: a buyer question, a competitor in the answer, and your company missing.
Imagine that screenshot lands before your marketing meeting. It deserves a closer look, but I wouldn't reorganize the content calendar around it. You don't yet know whether you've found a recurring gap, a different search mode, or one answer that changes when you ask again.
The useful job is building a tracking process that tells you what changed, whether the comparison holds, and who should investigate. Here's how I'd set that up before buying more software or assigning more content.
TL;DR & Takeaways
- Track a fixed panel of buyer questions repeatedly, with separate results for each engine and search mode.
- Count brand mentions, recommendations, and owned-site citations separately. They answer different business questions.
- Check capture quality and repeatability before treating a score change as a content problem.
- Use a short decision record to connect an observed change to an owner, a bounded action, and a follow-up check.
- Keep visibility alongside demand and revenue evidence. A sampled answer rate isn't market reach or attributed revenue.
Define what your AI visibility tracking measures
AI visibility tracking records how a brand appears in answers to a defined set of prompts over time. Its usefulness depends on the questions, collection method, and scoring rules. A score describes that sample; it doesn't tell you how often every potential buyer encountered your business.
Before collecting anything, agree on the events you care about.
| Signal | What counts | What it helps you decide |
|---|---|---|
| Brand mention | The answer names your brand, whether positively, negatively, or neutrally | Whether you're present in the tracked conversation |
| Recommendation | The answer presents your offering as an option for the stated need | Whether you're being considered for that use case |
| Owned-site citation | The answer links to a source on a domain you own | Which of your pages receive source credit |
| Accuracy issue | A factual claim conflicts with a verified current source | Whether information needs correction |
A third-party review can recommend you without linking to your website. Your article can be cited without the answer recommending your product. Count those events separately, and record third-party sources about your brand in their own field.
I'd count each event once per answer, even if your brand appears repeatedly. Write down accepted brand aliases and owned domains, and keep the sentence supporting each recommendation or accuracy judgment. Ambiguous cases need review rather than an automatic positive score.
This distinction is already established in resources such as Adobe's AI search KPI guide. The harder part is maintaining it when leadership wants one number.
Repeat the questions before explaining the movement
Prefer's September 2026 study asked 80 questions about AI search, GEO, and AEO three times across four engines. In its citation comparison, average pairwise cited-site overlap was 37% for ChatGPT, 35% for Gemini, and 96% for Perplexity.
Those percentages describe Jaccard overlap: sites shared by two answers divided by the distinct sites across both answers. They do not mean ChatGPT retained 37% of a brand's visibility, or that Perplexity will behave that way for your buyers.
Prefer collected the answers on September 13 through API calls, with repeats returning within 23 seconds of one another. The comparison excluded pairs where either answer cited nothing. It measures short-window citation variation in one topic area, not long-term brand performance or the consumer app experience.
Otterly's stability study addresses a different window: daily tracking across three categories. It found brand mentions generally more stable than citations and recommends using citations diagnostically beneath brand coverage.
Both companies sell tracking software. Their studies give you reasons to check your measurement method, not universal thresholds for deciding whether a decline is serious.
I'd begin with three scheduled runs per prompt in a bounded pilot, then repeat that schedule in the next reporting window. Three is a manageable starting rule, not a statistically validated sample size. Important or inconsistent findings need more runs across additional dates.
Keep the counts visible. A recommendation in three of three answers differs from one in two of three, even if both qualify as a majority under your reporting rule. Don't keep rerunning a prompt until the preferred answer appears.
Freeze a buyer-prompt panel without freezing your learning
Build the panel from questions your buyers ask during evaluation, supported by sales conversations, support records, and customer research. Keyword data can suggest topics, but it doesn't prove people use your exact wording in an assistant.
For a hypothetical CRM business, a non-branded prompt might be: “Which CRM suits a small sales team that needs reliable follow-up reminders?” A branded prompt might ask whether that company's CRM supports a specific workflow.
Keep those groups separate. Being named when the question supplies your name tells you something different from appearing in an unaided shortlist.
Give each prompt an ID, an exact wording, a buyer task, and a source for why it matters. Record a panel version. Use a separate exploratory list for new questions so you can learn without quietly changing the denominator in the trend report.
Semrush's reporting guide already recommends fixing the prompt set during a reporting period. I'd add an explicit change log: when prompts change, show a new baseline or compare only an unchanged subset, with its reduced coverage disclosed.
Also record the engine, model when exposed, search mode, language, location, collection time, API or interface, and session setup. Use fresh sessions for the controlled panel. If the model version isn't exposed, record that limitation rather than filling it in from memory.
An API run and a consumer-interface run belong in separate series unless you've established comparability. Personalized conversations can be useful exploration, but they shouldn't quietly replace your controlled checks.
Read the counts before the score
Consider a hypothetical panel of 20 non-branded buyer prompts, each run three times on one engine in each reporting window. Both windows produce 60 valid answers under the same method.
| Measure | Window A | Window B | Reading |
|---|---|---|---|
| Answers mentioning the brand | 24/60, or 40% | 24/60, or 40% | Mention rate is unchanged |
| Answers citing an owned domain | 18/60, or 30% | 12/60, or 20% | Citation rate fell 10 percentage points |
You can say owned-site citations declined in this sample. You can't say the brand disappeared from AI search. You also can't infer recommendation performance, because this table hasn't measured it.
Now inspect the prompt-level rows. Did several buyer topics lose citations, or did a few prompts account for the change? Did another page on your domain replace the old one, or did an outside source take its place?
Collection failures complicate this further. Suppose six of the 60 scheduled captures fail and the remaining 54 contain 24 mentions: the valid-answer rate is 44.4%, with capture coverage of 90%.
Reporting only 44.4% would hide the missing observations. If the failed captures cover topics where you rarely appear, the apparent improvement may reflect the changed sample. Recover those captures or show a matched subset before presenting a trend.
A valid answer with no mention belongs in the denominator. A failed capture is missing evidence. A valid answer with no citation is also different from an extraction failure; record whether search occurred when the interface exposes that information.
Report each engine separately before any combined score. If you blend engines or weight buyer topics, disclose the weights and retain the unweighted counts. Don't let a change in collection volume quietly become a change in business importance.
Use a decision record before commissioning work
The practical asset I'd put beside the dashboard is a short decision record. It forces the person reviewing a change to explain why it deserves work, rather than forwarding a red arrow to the content team.
Copy these fields into your existing reporting sheet:
| Field | What to record |
|---|---|
| Observation | Signal, prompt IDs, engine, windows, and underlying counts |
| Comparison check | Panel and method unchanged? Captures complete? Scoring reviewed? |
| Repeat evidence | Counts by run and date, with links to full answers |
| Interpretation | What the evidence supports, plus a plausible alternative explanation |
| Decision | Rerun, investigate, correct, or hold; name an owner |
| Follow-up | Bounded action, review date, and evidence needed to close the issue |
For the hypothetical citation decline, the record could say: “Mentions held at 24/60; owned-site citations fell from 18/60 to 12/60. Review the affected prompt rows and replacement sources before assigning a page rewrite.”
That is a proposed investigation, not a finding that the content became worse. The next observation may support a content change, a technical check, or no intervention.
I'd use these starting rules:
- One missing citation: rerun on the agreed schedule and preserve the answer. Don't commission content from that observation alone.
- Recurring absence on a relevant buyer task: inspect cited sources and your existing coverage. Check whether the answer favors a different use case before assuming you need another article.
- A verified false claim: correct inaccurate owned information promptly and pursue an appropriate correction elsewhere. You don't need weeks of trend data to fix a factual error.
- A broad shift across many prompts: check the collection method, scoring changes, and platform behavior before treating it as a site-wide content failure.
These are operating rules, not research-derived significance thresholds. Adjust the urgency to the consequence: a wrong eligibility claim can matter more than a larger change in educational citations.
Keep the business question beside the visibility report
The executive question is whether this work helps the business get considered accurately by relevant buyers, and whether that consideration contributes to demand.
Keep AI referrals, qualified inquiries, self-reported discovery, and pipeline evidence beside the sampled visibility data. Record the attribution method and time window. Branded search movement can add context, but other campaigns and market events can move it too.
I wouldn't convert a mention-rate increase into a revenue forecast. I also wouldn't dismiss the work because a buyer discovered the company in an assistant and returned later through another channel. Both conclusions outrun the evidence.
Start with a manual pilot small enough to inspect. Before automating, confirm that someone can reproduce the classifications, identify failed captures, and explain what action the report changes.
Then automate the repeatable collection work, preserve the exceptions for review, and document ownership. If the report never changes a decision, reduce its scope before increasing its frequency.
For your next review, choose the buyer-question panel, name the measurement owner, and complete one decision record against the baseline. The screenshot can start the investigation. The tracking process should determine what happens next.
More from the manifest.
Nov 18, 2026 · 10 min
Build an AI Content Workflow Without Losing Your Voice
Build an AI content workflow with author-led briefs, verified evidence, separate editing passes, exact-version approval, and measured review effort.
Read essayNov 16, 2026 · 10 min
How to Stop Good Leads from Going Cold
Build a lead follow-up workflow with routing, ownership, useful responses, stop conditions, and tests for the failures that lose inquiries.
Read essayNov 13, 2026 · 9 min
When More Leads Won't Fix Your Revenue Problem
Investigate lead quality, readiness, handoffs, and conversion before buying more demand, using a worked funnel and investigation checklist.
Read essayNov 9, 2026 · 10 min
Your Marketing Plan Has Too Many Priorities
Build a marketing plan that fits real capacity, compares evidence, and keeps deferred ideas on a useful not-now list.
Read essayNov 6, 2026 · 9 min
Why Your CRM Can't Answer Basic Growth Questions
Repair CRM reporting through clear definitions, account-to-billing joins, and a revenue-impact-led cleanup plan with explicit owners.
Read essayNov 4, 2026 · 9 min
Which Marketing Channel Should You Invest In Next?
Choose your next marketing channel using buyer behavior, readiness gates, customer economics, and a bounded test scorecard.
Read essayNov 2, 2026 · 11 min
Marketing Dashboard: KPIs, Examples and Build Guide
Build a marketing dashboard around decisions: illustrated examples, CAC, LTV:CAC, payback, data integrations, AI build steps and reporting tools.
Read essayOct 30, 2026 · 10 min
Fractional CMO, Agency, or Marketing Hire?
Choose a fractional CMO, agency, or marketing hire by the leadership, capacity, and expertise you need. Compare costs, ownership, and fit.
Read essayOct 28, 2026 · 10 min
Growth Bottleneck: Find It Before Spending More
Find your growth bottleneck before spending more. Diagnose demand, qualification, conversion, delivery, and retention, then test the right intervention.
Read essayOct 26, 2026 · 13 min
Marketing Automation: What to Keep Human
Decide what to automate in marketing with a practical scorecard for data quality, risk, judgment, human review, and measurable time savings.
Read essayOct 14, 2026 · 11 min
Fractional CMO: What to Expect in the First 90 Days
What a fractional CMO should deliver in 90 days, and how MakeReality’s AI Growth Operator adds hands-on systems, testing, and handover.
Read essayOct 8, 2026 · 10 min
How to Set a Marketing Budget
Work backward from revenue goals to a marketing budget, then test acquisition costs, cash, and capacity with a worked example and scenario comparison.
Read essayOct 7, 2026 · 11 min
The 3 Most Important Metrics for SaaS Growth & Marketing
Understand CAC, LTV:CAC, and CAC payback, calculate your SaaS acquisition economics, and improve growth efficiency without straining cash flow.
Read essayApr 24, 2026 · 6 min
Why "The Manifest" exists
Content has gotten sloppy. I don’t just mean “AI slop” either… even though that is a major contributing factor. The ability to generate content at scale and cover topics in days that in aggregate…
Read essay