For years, the digital publishing model was largely based on search engines and ads. Google’s content of the traffic in exchange for scanning, it became the main source of revenue of publishers. But with the rise of artificial intelligence technologies, this balance began to be shaken. Cloudflare attempted to redefine the equation by offering a new solution to this transformation.
Cloudflare, which manages about one -fifth of the Internet, has established a new system to control the access of AI bots to content. With this platform called “Pay Per Crawl ,, site owners will be able to charge the boats to scan their content. This system, which is currently tested within the scope of special beta, carries the long -term AI and publisher relationship in a different direction. However, the extent to which this system will become widespread is not yet known.
Cloudflare has established an infrastructure where publishers can sell content to AI bots
Cloudflare CEO Matthew Prince said that the foundation of this platform was laid last year. The single -click systems that block AI boots, the panels showing the scanning history of the boots were parts of this infrastructure. Now the publishers can make their content on AI boats paid, free or completely without access. This offers content owners a long time they need.
However, the system is not limited to content access. Cloudflare states that AI bots can also show the purpose of scanning the content. So a bot content uses for model training, to produce search responses, or for another analysis; This information will be open to the publisher. Such transparency was not seen in digital publishing before.
The technical functioning of the new system is very simple: Publisher and AI company opens an account through Cloudflare. The two sides then determine the amount to be paid per screening. When the scan occurs, the fee is deducted from the AI company and transferred to the publisher by Cloudflare. In spite of everything, how much this system will gain will depend on the demand and participation rate.
According to Cloudflare’s data, the current imbalance between AI bots and publishers is quite remarkable. For example, while Google scans 14 for each referral to content providers; OpenAI increases this ratio up to 17,000 and Anthropic up to 73,000. In other words, many publishers have to share their contents for free for free, but they cannot get almost no traffic in return. In addition, this clearly shows that a sustainable model is no longer working.
Cloudflare also announced that the system will block AI bots by default for every newly opened website. Site owners will only allow access to certain boots. Big broadcasters such as TIME, Associated Press, The Atlantic announced that they support this model. This shows that many institutions in the sector are looking at the permission -based system.
This transformation, faced by traditional content producers, has actually been giving signals for a long time. The fact that users now reach information directly through AI chat boots has reduced dependence on sites. Platforms such as chatgpt provide rapid access to information, while these systems do not provide traffic to the source of the content. This new platform intends to provide a solution to some of this problem.
On the other hand, in order for the system to be implemented, AI companies need to participate in this model. Currently, many companies are a question mark why they will agree to pay fees when scanning content free of charge. Nevertheless, Cloudflare offers a suitable structure in terms of testing this model with its technical power and sector position. At least there is a starting point for publishers to provide more control over their content.