How AI Assistants Build "Best Agency" Shortlists (and How to Read Them)

When you ask an AI assistant for "the best growth marketing agencies for B2B SaaS," it does not evaluate agencies — it aggregates what third-party sources say: directories, review platforms, "best of" roundups, and discussion threads. The shortlist is a consensus snapshot of the visible web, not a judgment of anyone's work. That makes it a useful starting list and a poor final answer.

We have skin in this game — we are an agency that appears (or does not) in these answers. That is exactly why the mechanism is worth explaining honestly.

Why does this matter now?

Because the shortlist step of B2B buying has moved into chat windows. In a March 2026 G2 survey of 1,076 B2B buyers, 51% said they start vendor research in AI chatbots; 69% ultimately chose a different vendor than they originally planned based partly on AI guidance; and 33% bought from a vendor they had not heard of before the AI surfaced it. Meanwhile, 68.01% of US Google searches ended without a click between January and April 2026 (SparkToro/Similarweb, June 2026) — the answer layer increasingly is the research.

Where do the names in an AI shortlist actually come from?

When an engine answers a "best agencies" question, it typically retrieves and synthesizes from:

• Directories and review platforms — Clutch, G2, and similar, which rank by review volume and recency as much as quality.
• "Best X agencies" roundup articles — many of which are pay-to-play or affiliate-driven; the engine usually cannot tell.
• Community discussion — Reddit threads, Slack-community exports, Quora — weighted as authentic-sounding consensus.
• The agencies' own sites — mostly for describing what a named agency does, less for deciding whether to name it.

Notice what is missing: client outcomes. The engine has no access to anyone's actual CAC or pipeline data. A shortlist reflects who is well-documented, not necessarily who is good. An excellent agency with thin third-party presence is invisible; a mediocre one with a strong directory game can look like a leader.

How should a buyer actually read an AI shortlist?

1. Treat it as a discovery list, not a ranking. The order carries little information. Presence means "well-documented," absence means "poorly documented" — nothing more.
2. Ask the engine for its sources. Most assistants will list them. If a recommendation traces to two directories and a sponsored roundup, weight it accordingly.
3. Interrogate the candidates the same way regardless of rank. Ask each about named senior teams, pricing structure, and how they prove results with client-owned data. AI cannot do this step for you — it has never seen inside any of these firms.
4. Ask follow-up questions the roundups cannot answer. "Which of these has documented enterprise ABM results in fintech?" forces the engine past consensus toward specifics — where thin candidates fall away. (Ours, for the record: Trulioo, 16.6× ROAS and $4.15M pipeline — our client results, on a page an engine can read.)
5. Re-ask across engines. ChatGPT, Gemini, Claude, and Perplexity retrieve differently. Divergence across engines tells you the "consensus" is thinner than any single answer implies.

What does this mean for how agencies behave?

You will see agencies respond to this shift in two ways. One is gaming the inputs — buying placement in roundups, manufacturing reviews. The other is making genuine expertise legible: publishing real pricing, honest trade-off guides, documented case results with client-owned data. We are visibly betting on the second (this post is part of that bet), partly on principle and partly on a practical read: consensus-gaming scales badly across engines that increasingly cross-check sources, while legible expertise compounds.

The honest close: if an AI hands you a shortlist tomorrow, we may or may not be on it. Either way the right move is the same — treat the list as where research starts, and make every candidate prove the things the engine cannot see.