STAT Director Addresses AI Search Measurement Challenges

Search Engine Journal reported that Tom Capper, Director of Search Product Strategy at STAT, addressed the measurement problem of AI search in a recent webinar. Capper explained that AI answers can recommend products without linking to the company's website, making it difficult to track the impact of these recommendations.

Capper suggested measuring visibility alongside business outcomes rather than relying on clicks to describe the entire customer journey. He emphasized the importance of separating brand mentions from source citations, as a report focused only on domain citations may miss the recommendation itself.

The distinction between citations and mentions is crucial, according to Capper. He illustrated this with an example of an AI-generated car recommendation, where the brand being recommended was Audi, but the cited links pointed to other websites. Capper advised defining the brand entities to track, including relevant sub-brands, to get a more accurate picture.

Capper also discussed the importance of building a representative prompt sample to measure AI visibility. He cautioned against simply putting a keyword list into a prompt tracker and suggested using People Also Ask questions as a starting point. The goal is to sample relevant situations broadly enough to recognize patterns.

Capper recommended using citations to investigate what changed, rather than relying on them as a headline KPI. He described starting with pages cited when competitors were recommended and investigating third-party sources worth exploring. This approach creates a more useful question than whether the citation count increased.

Capper suggested adding visibility metrics alongside business outcomes, including presence, recommendation and context, and SERP prominence. He also urged caution with sentiment scores, as they may amplify small differences in otherwise neutral answers. Instead, he recommended focusing on qualitative themes behind a mention.

During the Q&A session, Loren Baker asked how teams could connect visibility to a customer journey when multi-touch attribution could not observe the earlier exposure. Capper suggested combining evidence, such as post-purchase surveys, to reveal customers who said an AI recommendation helped them discover or choose a brand.

Search Engine Journal reported that Capper's approach provides a fuller account of influence, rather than trying to reconstruct every person's path or classify all direct traffic as AI-driven. To put this discussion to work, businesses can start by auditing what their report misses, defining a consistent comparison, and connecting source research to campaign work.

Capper's measurement approach emphasizes the importance of understanding the impact of AI on a business's online presence. By tracking brand mentions and recommendations, businesses can gain a more accurate picture of their visibility and make informed decisions about their marketing strategies.

According to Search Engine Journal, the full session connected these measurement choices with examples and audience questions, providing a comprehensive guide for businesses to improve their AI search measurement.

STAT and Search Engine Journal plan to continue exploring the topic of AI search measurement in future webinars, including an upcoming session on AI shopping and how AI agents decide which products shoppers consider.