A lot of agencies added “AI SEO” to their service list without changing anything about how they actually work. Here is how to tell the difference between an agency that has genuinely adapted and one that just updated a slide deck and what to ask instead.
1. They still only report Google rankings
If your monthly report is exclusively keyword positions and organic traffic, your agency is not measuring the channel that is increasingly deciding who gets recommended. Ask directly whether they track AI-answer citations, and how.
2. “AI SEO” means keyword-stuffed blog posts
Generic, AI-written content with no specific, quotable answer is exactly the kind of content AI models are learning to skip over in favor of sources that actually answer a question directly. If the strategy is volume over specificity, it is not built for how these models actually select sources.
3. They can’t explain how AI models choose what to cite
This should be a confident, specific answer — directory profiles, review text, quotable on-page content, and source consistency. A vague or evasive answer here usually means the agency has not actually studied the mechanism, just the marketing term.
4. Their own client work doesn’t show up in AI answers
Ask to see it. Run a real buyer question through ChatGPT or Perplexity for one of their existing clients, live, in front of them. An agency doing this work well will have already run this exact test and will not be surprised by the result.
5. Everything is a one-time audit, nothing is a tracked trend
AI visibility moves faster than classic rankings, in both directions. A one-time PDF audit from six months ago describes a web that no longer exists. If there is no monthly re-measurement, there is no real program — just a report that was sold once.
Frequently Asked Questions
What is the fastest way to test my agency on this?
Ask them directly: how do you measure AI-answer citations, separately from Google rankings? A confident, specific answer with a methodology is a good sign. A vague answer or a pivot to a generic “GEO score” is not.
Is this actually a fireable offense for an agency to not track?
Not by itself — but a flat refusal to even discuss it, or a claim that it “doesn’t matter yet,” is worth pressing on given how fast buyer behavior is shifting toward AI answers.
Can a small or generalist agency still do this well?
Yes, if they are willing to measure and adapt. This is a mindset and process gap, not a size gap — some of the fastest-moving work in this space comes from small, focused teams.
Should I switch agencies over this alone?
Not on its own — but combined with other stalled reporting or outdated tactics, it is a legitimate signal worth raising directly with your current agency before deciding.
See What a Real Baseline Looks Like
Leedly’s free AI visibility scan shows exactly which sources are getting you cited — or not — across ChatGPT, Perplexity, and Google AI Overviews, with a methodology you can see for yourself. Run your free scan.
