Google-Extended
Google-Extended is a robots.txt token, not a separate crawler. It controls whether content crawled by Google may be used to train future Gemini models and for grounding in Gemini Apps and Vertex AI. Google says it does not affect a site's inclusion in Google Search.
What it does
Google-Extended has no user agent of its own. Crawling is done with Google's existing user agents, and the Google-Extended token is used in a control capacity: it tells Google whether content it has crawled may be used for training future generations of Gemini models and for grounding in Gemini Apps and Vertex AI.
Google states that Google-Extended does not impact a site's inclusion in Google Search. Google's AI features in Search, such as AI Overviews, are governed by the Googlebot rules and snippet controls, not by this token.
In our 2026-11 study, 18.1% of popular sites with a robots.txt blocked it. →
robots.txt token
Google-Extended
How to allow it
User-agent: Google-Extended
Allow: /How to block it
User-agent: Google-Extended
Disallow: /How to verify real visits
There is nothing to verify in your logs: Google-Extended never appears as a user agent. The requests behind it come from Google's regular crawlers, which you can verify with a reverse DNS lookup (googlebot.com, google.com or googleusercontent.com) followed by a forward lookup, or against Google's published IP ranges.