The news: Major web publishers, including Reddit, Yahoo, Medium, and Quora, are joining forces to push for a new content licensing system for AI publishers. By working together, the publishers hope to strengthen their bargaining power with AI firms, per The Verge.

The group is backing Really Simple Licensing (RSL), an open standard that lets publishers dictate how AI bots scrape their content and includes payment and royalty requirements.

What is RSL? The standard builds on robots.txt, a line of code that websites use to tell bots which content they can and can’t scrape. But because robots.txt isn’t legally binding, bots can ignore it.

RSL adds licensing and royalty terms to robots.txt code, letting publishers charge AI companies through models like subscriptions, pay per crawl, or pay per inference, in which AI companies pay whenever they surface content in a response.

Why it matters: Both publishers and AI firms face struggles in the battle for online data.

• Publishers are seeing drops in web traffic and ad revenues due to the rise of generative AI (genAI) search tools. They’re also concerned about original content losing value when it’s scraped and repurposed by chatbots.
• AI companies are dealing with a dwindling pool of data for model training, which could push bots to be more aggressive and even ignore robots.txt restrictions.

Looking ahead: If licensing standards like RSL become the norm, AI companies may face higher operating costs that could reshape how they price tools or structure partnerships.

Gatekeeping content in this way could also fragment the flow of online data, making AI search tools less comprehensive and potentially nudging users back to traditional search—restoring some of the referral traffic that publishers have lost.

Our take: If publishers’ collective action can successfully enforce licensing terms for content scraping, regulators may follow with broader mandates. Visibility inside generative engines could change, pushing marketers to further prioritize generative engine optimization (GEO) strategies and comprehension of how AI responses source, cite, and surface branded content.

This content is part of EMARKETER’s subscription Briefings, where we pair daily updates with data and analysis from forecasts and research reports. Our Briefings prepare you to start your day informed, to provide critical insights in an important meeting, and to understand the context of what’s happening in your industry. Non-clients can click here to get a demo of our full platform and coverage.