Google says it will honor one shared robots.txt signal for AI training opt-outs, but not until 2027

Google said on October 30, 2026 that it plans to honor a shared robots.txt signal that lets a website owner opt out of AI training in one place. The announcement, posted on the Google Search Central blog by Search Advocate John Mueller, does not turn anything on today. Google says it aims to support the control, alongside other AI providers, in 2027.

What is the new signal?

Today, a site that wants to limit AI training has to name each company's crawler token one by one. Google's token for this is Google-Extended. Other firms publish their own. The new idea is a single token that any participating AI provider could read.

The proposal comes from an Internet-Draft titled “Machine-Readable Training Preference Signal”, version 00, dated October 30, 2026. It lets a site write an ordinary robots.txt group with the token __usage_ai-training in the user-agent line. A group that disallows paths states a preference against AI training on those paths, and an allow rule states a preference in favor. The draft adds no new directive names and leaves the existing robots.txt rules from RFC 9309 unchanged. For example, it shows Disallow: / with Allow: /press-kit/.

Two cautions. The IETF page says the document is an individual submission that is not endorsed by the IETF, so it is not a standard yet. And the page lists Kevin Kelley of Anthropic as author. Anthropic makes the Claude AI models; we note the connection so readers can weigh it.

How would it work with Google-Extended?

Google says the shared signal builds on Google-Extended, which it has offered since 2023. The two will run side by side. If a site uses the new token to opt out of training and does not explicitly mention Google-Extended, Google says it will treat the site as if Google-Extended were disallowed. A site that allows Google-Extended stays eligible to appear in, and get clicks from, Gemini Apps grounding responses, according to the post.

Google's crawler documentation describes Google-Extended as a standalone token that controls whether content Google crawls can be used to train future Gemini models. It also covers grounding, where Search index content is given to the model at prompt time. The page states that the token “does not impact a site's inclusion in Google Search” and is not a ranking signal. The same page says Google intends to support the training preference signal in early 2027.

What should a small business do now?

For most local businesses, nothing urgent. The new token does not exist in Google's systems yet, and the draft could change before then. Still, there are practical steps:

1. Open yourdomain.com/robots.txt and read it. Many site builders and plugins edit it without telling you.
2. Decide your position on AI training for your own pages. Some owners want maximum exposure; publishers with paid or original reporting may not.
3. If you opt out today, use the company-specific tokens each provider documents. Blocking Googlebot itself is a different matter and can keep pages out of Search.
4. Keep a note of the date and the change, so you can compare traffic and referrals later.
5. Check back when Google publishes support documentation in 2027.

What this does not do

A robots.txt rule is a request, not a lock. Responsible crawlers follow it. Others may not, and the file cannot remove content that has already been collected. The signal also says nothing about licensing, payment or credit. Google's post says the proposal reflects feedback from publishers, civil society and regulators, and that it is consistent with the EU Code of Practice for General Purpose AI Model Providers, which recognizes robots.txt as a way to express machine-readable preferences.

The open questions are how many other AI providers sign on, and whether the IETF's working group adopts the draft or changes it. Google gave a 2027 target, not an exact date. We will update this report when either answer changes.