Faced with the proliferation of tools measuring visibility in generative search engines, the author of an analysis published on Search Engine Journal urges companies not to entrust their strategy to software. According to him, an immature market exposes teams to unstable tools, poorly comparable to each other, and sometimes difficult to leave.
To support his point, the author states he has identified 194 tools launched over the last two to three years, from Product Hunt, G2, and LinkedIn. According to him, just over 60% of these are artisanal experiments, side projects, or solo founder creations. He estimates the mortality rate of this category to be between 55% and 60%, not counting tools he could not identify. Each month, he says, services cut their APIs, let their domain names expire, or pivot to unrelated applications.
Cost is, according to the author, the main factor of fragility. Querying a traditional search engine and analyzing its HTML code costs a fraction of the price of a series of complex queries, in multiple exchanges, on AI models with active web navigation. He also cites Google’s efforts to combat this type of data collection, which force the publishers of these tools to constantly reinvest. For a self-service tool billed at $100 per month, margins would be, he says, non-existent. A change in inference pricing, query limits, or a model provider’s interface is enough to derail these projects.
Measurements That Are Not Alike
Changing tools is not, according to the author, like replacing one instrument with an equivalent one. Multi-year contracts complicate departures, and data export is not always sufficient, as there is no common standard. He cites in this regard an IAB document which notes the absence of a universally accepted standard for measuring conversational AI responses.
The differences lie in several areas. One platform may query the model directly, while another uses web navigation comparable to that of a logged-in user. Some tools simply check if the brand is mentioned, others assess its placement and tone. A brief mention may count as a success on one and penalize the score on another. Finally, one platform may present a fixed ranking, while another measures the frequency of variations across multiple tests. According to the author, one might announce a top placement in 80% of responses, while another never mentions the brand, as AI responses vary randomly.
Building Your Own Data
For the author, the company must internally own its list of queries, prompt sets, and collected responses. He recommends storing these elements outside of any external subscription and favoring platforms that allow the export of response texts, timestamps, source URLs, and queries, via webhook, BigQuery, or standardized CSV. This way, if a provider changes its calculation method, the raw history remains usable.
The author finally reminds us that visibility is, in his view, an outcome and not a lever. Language models, which generate responses, would cite brands with strong entity definitions, unambiguous architectures and schemas, and an active digital presence, rather than brands that have purchased software. He predicts that consolidation in the sector will accelerate and that most tools will disappear.
