The news: Publishers are tackling AI scraping with a new strategy—pay per crawl.

Rather than one-time licensing deals, usage-based compensation models would have AI companies pay publishers and content providers based on how often their work is used in AI-generated responses.

This kind of structure could already be in motion at Perplexity, whose publisher payment model is partially based on usage, per The Information.

The problem: Website owners and publishers often struggle to shield their content from AI bots and web crawlers. In response, some—like The Atlantic, Axios, and the Associated Press (AP)—are signing content licensing agreements.

But as revenues skyrocket for firms like OpenAI, those deals may no longer seem adequate.

• If premium users are paying per query, it seems only fair that the companies supplying those answers get paid on a similar basis.
• “These are not little startups that can’t fund fair access to content,” said Paul Bannister, CSO of Raptive, per The Information.

And the infrastructure to support usage-based deals already exists through companies like Cloudflare. Condé Nast, Time, AP, Adweek, and Fortune are some of its early adopters, per TechCrunch.

Why it matters: A shift to usage-based payments could reshape how content is discovered, monetized, and surfaced by AI-driven platforms.

• Less “free” exposure. Brands that depend on media mentions or SEO could see visibility shrink if AI firms limit usage to cut costs.
• New ad models. As AI search platforms like Perplexity start running ads and sharing revenues with content creators, brands may need to navigate multiple, smaller AI-native ad platforms.
• Content strategy shift. AI engines may focus on including brands with the lowest usage rates, creating a price war.

Yes, but: Unlike royalty models in music and media, pay per crawl isn’t a legal requirement—yet. AI firms can still negotiate flat-fee deals or avoid payments entirely unless regulation or industry norms change.

Websites may choose to participate in programs like Cloudflare’s, but AI companies can still bypass or ignore those agreements.

Our take: These usage-based models could be a more equitable deal for publishers whose content powers AI engines that are earning tens or hundreds of millions of dollars per year.

To avoid getting locked out of monetization, brands should act now to review existing content agreements, explore licensing opportunities, and push for fairer models that recognize the value of original content.

This content is part of EMARKETER’s subscription Briefings, where we pair daily updates with data and analysis from forecasts and research reports. Our Briefings prepare you to start your day informed, to provide critical insights in an important meeting, and to understand the context of what’s happening in your industry. Non-clients can click here to get a demo of our full platform and coverage