Free tool · Reference

Googlebot user agent strings

Every user agent Googlebot and Google's other crawlers send, as Google publishes them, the robots.txt token each one obeys, and how to check that a visit claiming to be Googlebot really comes from Google.

Googlebot's user agents

Crawler	User-Agent	Copy
Googlebot Smartphone	Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot Desktop	Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36
Rarely used	Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Rarely used	Googlebot/2.1 (+http://www.google.com/bot.html)

Copied from Google's list of common crawlers. W.X.Y.Z is Google's own placeholder for the Chrome version the crawler uses; Googlebot renders with a current Chromium, so in your logs it is a real, recent Chrome version that changes as Chrome does. The part to match is Googlebot/2.1 (or the token Googlebot), not the version.

Googlebot Smartphone does most of the crawling for Google Search, since Google indexes the mobile version of pages. Both crawlers obey the same robots.txt token, Googlebot, so you cannot address them separately there.

Every Google crawler

Common crawlers do Google's product crawling and obey robots.txt. Special-case crawlers work for a product with an agreement between you and Google, such as AdSense or Ads, and may treat robots.txt differently (Google's list).

Crawler	robots.txt token	User-Agent
Googlebot ImageCommon crawler	Googlebot-Image	Googlebot-Image/1.0
Googlebot VideoCommon crawler	Googlebot-Video	Googlebot-Video/1.0
Googlebot NewsCommon crawler	Googlebot-News	No string of its own: it crawls with the Googlebot strings above.
Google StoreBotCommon crawler	Storebot-Google	Desktop: Mozilla/5.0 (X11; Linux x86_64; Storebot-Google/1.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Safari/537.36Mobile: Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012; Storebot-Google/1.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36
Google-InspectionToolCommon crawler	Google-InspectionTool	Mobile: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Google-InspectionTool/1.0)Desktop: Mozilla/5.0 (compatible; Google-InspectionTool/1.0)Search Console's URL Inspection and the Rich Results Test.
GoogleOtherCommon crawler	GoogleOther	Mobile: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; GoogleOther)Desktop: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GoogleOther) Chrome/W.X.Y.Z Safari/537.36
GoogleOther-ImageCommon crawler	GoogleOther-Image	GoogleOther-Image/1.0
GoogleOther-VideoCommon crawler	GoogleOther-Video	GoogleOther-Video/1.0
Google-CloudVertexBotCommon crawler	Google-CloudVertexBot	Google lists only the substring Google-CloudVertexBot.
Google-ExtendedCommon crawler	Google-Extended	A robots.txt token only: crawling happens with the other Google strings.
APIs-GoogleSpecial-case crawler	APIs-Google	APIs-Google (+https://developers.google.com/webmasters/APIs-Google.html)
AdsBot Mobile WebSpecial-case crawler	AdsBot-Google-Mobile	Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; AdsBot-Google-Mobile; +http://www.google.com/mobile/adsbot.html)
AdsBotSpecial-case crawler	AdsBot-Google	AdsBot-Google (+http://www.google.com/adsbot.html)
Mediapartners-Google (AdSense)Special-case crawler	Mediapartners-Google	Desktop: Mediapartners-GoogleOn mobile it sends a phone's string followed by (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html).
Google-SafetySpecial-case crawler	Google-Safety	Google-SafetyIgnores robots.txt.

Google also runs user-triggered fetchers (Google Site Verifier, Feedfetcher and others) that act on a person's request and generally ignore robots.txt; they are listed on Google's crawler overview.

How to verify Googlebot

A user agent is self-reported: any scraper can send Googlebot's, and many do, because some sites let Googlebot through where they block everyone else. Google publishes two ways to check a visit (Verifying Googlebot):

DNS: look up the IP's reverse DNS name; it must end in googlebot.com, google.com or googleusercontent.com. Then look that name up and confirm it resolves to the same IP. The second step matters: anyone who controls an IP's reverse zone can make it point at a Google-looking name, but not make Google's DNS point back.
IP ranges: compare the IP with Google's published lists. The same answer without two DNS lookups per visit; refresh the list daily.

# 1. Reverse DNS: the name must end in googlebot.com, google.com or googleusercontent.com
host 66.249.66.1
# 1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.

# 2. Forward DNS: the name must resolve back to the same IP
host crawl-66-249-66-1.googlebot.com
# crawl-66-249-66-1.googlebot.com has address 66.249.66.1

The names and addresses in the example are what these lookups returned on 19 October 2026, and both Python functions gave the results shown for the same two IPs.

Google's IP list	Reverse DNS names
common-crawlers.json	Common crawlers (Googlebot and the others above): crawl-*---.googlebot.com or geo-crawl----*.geo.googlebot.com
special-crawlers.json	Special-case crawlers (AdsBot and others): rate-limited-proxy-*---*.google.com
user-triggered-fetchers-google.json	User-triggered fetchers run by Google: google-proxy-*---*.google.com
user-triggered-fetchers.json	User-triggered fetchers on Google Cloud: *---*.gae.googleusercontent.com

All four files live under developers.google.com/static/crawling/ipranges/. A visit that claims Googlebot and fails both checks is not Google; one that passes is, whatever else it sends.

Googlebot in robots.txt

Rules address crawlers by token, not by the full user agent. For example:

# Googlebot (smartphone and desktop)
User-agent: Googlebot
Disallow: /search/

# Google Images
User-agent: Googlebot-Image
Disallow: /private-images/

# Opt out of use for Google's AI models; does not affect Search
User-agent: Google-Extended
Disallow: /

Googlebot-Image, Googlebot-Video and Googlebot-News fall back to the Googlebot group when no group names them. Check what your own browser sends with what is my user agent.

FAQ

What is the Googlebot user agent?

Googlebot Smartphone, which does most crawling for Google Search, sends Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html). Googlebot Desktop sends Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36. W.X.Y.Z stands for the Chrome version Googlebot renders with.

What Chrome version does Googlebot use?

A current one. Google's documentation prints Chrome/W.X.Y.Z as a placeholder for the version of Chrome the crawler uses, and Googlebot has rendered pages with an up-to-date Chromium since 2019, so the number moves with Chrome releases. We do not print a number here because we have not observed Googlebot's requests ourselves; match on the Googlebot token, never on the Chrome version.

How do I verify that a request is really from Googlebot?

Run a reverse DNS lookup on the IP address; the name must end in googlebot.com, google.com or googleusercontent.com. Then look that name up and check that it resolves to the same IP. Or compare the IP with Google's published list, common-crawlers.json. The user agent alone proves nothing, because anyone can send it.

Where is googlebot.json?

Google renamed it. The old address, developers.google.com/static/search/apis/ipranges/googlebot.json, now redirects (HTTP 301, checked on 19 October 2026) to common-crawlers.json, which covers Googlebot and the other common crawlers. AdsBot and the other special-case crawlers are in special-crawlers.json.

How do I block or allow Googlebot in robots.txt?

Use its token: User-agent: Googlebot. The more specific tokens (Googlebot-Image, Googlebot-News, Storebot-Google and others) let you treat those crawlers differently; a crawler follows the most specific group that names it, and falls back to the Googlebot group. Google-Extended only controls whether content is used for Google's AI models; it does not affect Search.

Does Googlebot ignore robots.txt?

Googlebot obeys it. A few of Google's special-case crawlers work differently: AdsBot ignores the global User-agent: * group (it must be named), and Google-Safety ignores robots.txt entirely. User-triggered fetchers, such as Google Site Verifier, generally ignore robots.txt because a person asked for the fetch.

Can I test my site as Googlebot by changing my user agent?

You can see what your server sends to that string, which helps find user-agent-based cloaking or blocks. It is not how Google sees the page: Google's IP addresses, rendering and timing differ. Search Console's URL Inspection (the Google-InspectionTool crawler) shows the real result. How to change your user agent covers the methods.

Clearcote puts this into practice

An open-source Chromium with fingerprint control compiled into the engine. A drop-in for Playwright & Puppeteer.

Get it free Read the docs

Free for one browser with GitHub. No card.