GEA and the Open Agentic Web | Oasy<br>35 years ago this week, Tim Berners-Lee published the first website, ushering in the age of the World Wide Web. The following decades produced millions of websites, ideas, conversations, and of course memes.
One of the web's finest contributions to civilisation.<br>In a curious development, the World Wide Web became a very open place. Anyone, anywhere, could pretty much post whatever they wanted, there really weren't any gatekeepers. In what is usually called the Grand Bargain, Google indexed the near infinite content on the web, enabled discovery, and reaped the benefits by earning billions in ads revenue. Publishers let Google scrape their sites, as they got traffic in return. Advertisers paid for traffic they otherwise never would have gotten. Everybody won. We had a system where humans were incentivised to post interesting new content on the web.
On the back of this abundance of content, we trained the first LLMs. On November 30, 2022, OpenAI released ChatGPT, firing the starting gun on the largest capital expenditure boom in human history. In the following years, AI became a presence in everything from refrigerators (really) to the Google results page. By introducing their AI Overview feature (the AI generated section at the very top of the page), Google changed their site from a gateway to a destination.
Google used to simply be a way for publishers to have their content discovered. Now, in over two-thirds of the cases, it is the end destination for users. They have a query, Google it, and find the answer without ever clicking a blue link. The information Google uses to answer them is of course mined from those very pages they used to refer to.
The result: Google-referred visits have fallen 50% year-on-year, a deathblow for publishers who depend on referral-links and ads to monetise their website. Say you have a blog called ParisForParents, where you post fun ideas for parents who want to spend a weekend in the city. Pre-AI, parents who googled “paris with toddlers” would find your site, visit your page, and see the ads, maybe even clicking one of your affiliate links. Nowadays, they will get the exact same content in the AI Overviews section, consuming your content but never monetising your website.
These developments have led to discussion of The Death of the Ad-Supported Web and the Zero Click Web. Faced with these dynamics, roughly three-quarters of top publishers have chosen to simply block AI crawlers from scraping their websites. This is where we stand today. The “human” web was monetised by ads, and largely open. The Agentic Web looks to consist of a few walled gardens, a closed web, with most visits never leaving the AI platform, and most sites shutting themselves off from them. In turn, this will lead to most AI platforms operating on increasingly stale data, like 10 year old blogs, because people are no longer incentivised to post novel content.
There has to be a better way.
In June this year it was reported that bot web traffic has overtaken human web traffic. Eventually we will arrive at the Agentic Web: where Agents use the web thousands of times more than humanity. These Agents will research, converse and make transactions. As Harvard researcher Shuwei Fang wrote:
“Every major information revolution, from printing to mobile, expanded the market for knowledge, often dramatically. AI is likely to do the same, but is also categorically different: the world is entering the age of machine audiences.”
June 2026: the internet is no longer mostly human.<br>Cloudflare, sitting in front of a fifth of all web traffic, has tried to monetize these audiences with tolls. Their Pay Per Crawl marketplace let publishers charge AI companies per fetch. It didn't stick: this July, Cloudflare announced it was evolving the model into Pay Per Use, a watered down version. This model seems doomed for the same reason its predecessor was: the platforms have all the leverage, and no interest in setting a precedent of paying for the open web. Remember: the human web was never monetised by charging readers per page either. Every attempt at micropayments died, and ads won, because ads brought in a third party who was genuinely eager to pay. There is no reason to believe the Agentic Web will be different.
The advertisers certainly seem as eager as ever. Many companies have decided to jump at this new Agentic channel, investing in their AI Visibility. Inspired by the Web 2.0 practice of SEO (Search Engine Optimization), platforms like Promptwatch and Peec are building the industry of GEO (Generative Engine Optimization), raising tens of millions of venture funding and reaching million dollar revenues in record time.
So the web is closing, right at the moment there is a huge increase in demand for both information and visibility. This is not an inevitable reality, but a case of incentive misalignment. If these Agents are the new audience, and making transactions, why not monetise...