AI Today: One switch to say no, if the AI companies listen

A news site that wants to keep its articles out of AI training has had to name each company's crawler, one at a time, in a small file called robots.txt: GPTBot for OpenAI, ClaudeBot for Anthropic, Google-Extended for Google, and so on down the list. Miss one, and that company's bot is not covered.

AI Today is a series from Fíor, a natural language classifier that predicts whether a piece of text was more likely written by a person or by AI. Each piece explains one development in AI and shows where the claims come from.

This week, The Verge and AFP reported that OpenAI, Google, Microsoft and Anthropic are backing a shared alternative. A publisher would add one new line to robots.txt, stating its preference on AI training once, and every participating company would read the same signal. The proposal has been submitted to the Internet Engineering Task Force, the body that sets many of the internet's technical standards, as part of its work on AI preferences. It is still a draft.

Support will arrive at different times. According to the reports, Anthropic has started work on it, OpenAI plans to support it in the coming months, Microsoft from February 2027, and Google at some point in 2027. Meta called a common signal an important step but has not clearly committed to honouring it.

Simpler, not settled

The change is real for publishers. One line is easier to get right than a growing list of crawler names, and many news sites already block AI training bots one by one.

What it does not change is who decides. Like robots.txt itself, the signal works only if the companies choose to respect it, and only those that sign up. Each company also decides how it reads the preference. That gap has mattered before. In a 2025 court case, Google acknowledged that its own training opt-out covered only some of its AI work.

There are practical catches too. Existing rules that block all bots do not automatically count as an AI training opt-out, so publishers have to add the new line explicitly. And the signal covers training, not everything. Some AI services also use web pages to answer questions and point readers back to sources, so publishers who opt out may still want to stay visible there.

Asking, or being told no

The bigger argument is about who has to act. At an Australian parliamentary inquiry this month, Kate Gilchrist of the ABC, Australia's public broadcaster, argued that opt-out systems put the burden on rights holders, who would have to keep watching where their work turns up. Jeffrey Bleich of Anthropic argued that licensing every piece of content on the internet used for training would be technically impossible.

Both points can be true at once. A single switch makes saying no easier. It does not turn saying no into asking permission.

For journalism, the stakes are plain. The articles publishers are deciding whether to protect are the same kind of writing AI is learning to produce. One switch helps them decide. It does not decide for them.

Try the browser extension at Fíor.