Google-CloudVertexBot
Google-CloudVertexBot crawls sites at the request of their owners to build Vertex AI Agents on Google Cloud. Google says its robots.txt rules have no effect on Google Search or other products. Blocking it only matters if you or a Google Cloud customer use your site as an agent's data source.
What it does
Google says crawling preferences addressed to Google-CloudVertexBot affect crawls requested by site owners for building Vertex AI Agents. It has no effect on Google Search or other Google products. Its user agent contains the substring Google-CloudVertexBot.
In robots.txt it follows rules for the Google-CloudVertexBot token and, if there are none, rules for Googlebot. Google lists it among its common crawlers, which always obey robots.txt when crawling automatically.
robots.txt token
Google-CloudVertexBot
How to allow it
User-agent: Google-CloudVertexBot
Allow: /How to block it
User-agent: Google-CloudVertexBot
Disallow: /How to verify real visits
Google says user agent strings can be spoofed and recommends a two-step DNS check. Run a reverse DNS lookup on the requesting IP: the hostname should end in googlebot.com, google.com or googleusercontent.com. Then run a forward lookup on that hostname and confirm it returns the same IP. Google also publishes the IP ranges of its common crawlers as JSON, so you can match addresses automatically.