{"id":147696,"date":"2015-07-08T15:22:45","date_gmt":"2015-07-08T15:22:45","guid":{"rendered":"https:\/\/www.similarweb.com\/blog\/?p=147696"},"modified":"2025-01-21T15:40:48","modified_gmt":"2025-01-21T15:40:48","slug":"googlebot","status":"publish","type":"post","link":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/","title":{"rendered":"Googlebot\u2019s Mission: Scan, Index, Cache and Rank the Web"},"content":{"rendered":"<p>Googlebot, the Google robot, also known as a spider crawls, indexes, and ranks the entire World Wide Web. It combs the internet for new and existing material and using an algorithm of ranking factors determines the position of your website in Google search. Googlebot&#8217;s mission is to read the code files of a website, save information in Google&#8217;s cache index and make available the Google algorithm as part of the indexing of your site in Google\u2019s massive worldwide server storage units.<\/p>\n<p>When someone searches and queries Google for a search keyword, it scans Google\u2019s complete website index in a split second using the most advanced technology (which started as an invention by Larry Page and Sergey Brin of Stanford in 1997, called PageRank). Then based on the algorithm\u2019s best choices, it returns the SERPs.<\/p>\n<p>Google takes into account many things when ranking a website, you can work on ranking in one area and it might help you, the algorithm is looking at a broad aspect of a website.<\/p>\n<p>Googlebot crawls the web and indexes all the sites on the web, meaning that Google takes a picture of a site and keeps it on file in a cache. To see this cache, go to any search page, find the down-pointing arrow menu next to the site description, and click on Cached.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/rank-ranger\/blog_1\/algo2.png\" alt=\"Google SERP looking at cache\" border=\"0\" \/><\/p>\n<p>You will see the full version Google cached with links to other coded aspects of the site. Google caches three site elements; Full version (actual page), Text-only version (page without images or CSS formatting) and View Source (code). By looking at those three elements, you get a good idea of the scope of what Google sees.<\/p>\n<p>For instance, by looking at just the Text-only version, you can see the site without images or color; you get an idea of what text elements Google sees. This is important, John Mueller says again and again, because Google doesn\u2019t find context for your website in images or video, it counts on the text for keyword terms, category, and context. So be sure to <a href=\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/image-optimization\/\">add Alt text to your images<\/a> and video to make sure Googlebot can contextually define your site\u2019s full value including images.<\/p>\n<h2>Google Cached Full Version<\/h2>\n<p><img decoding=\"async\" class=\"alignnone wp-image-180202 size-full\" src=\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/53435454.png\" alt=\"Google Cached Full Version\" width=\"732\" height=\"427\" srcset=\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/53435454.png 732w, https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/53435454-300x175.png 300w, https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/53435454-512x299.png 512w\" sizes=\"(max-width: 732px) 100vw, 732px\" \/><\/p>\n<h2>Google Cached Text-Only Version<\/h2>\n<p><img decoding=\"async\" src=\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/rank-ranger\/blog_1\/Ford-cached-text-version.png\" alt=\"text-only version of ford.com google cache\" \/><\/p>\n<h2>Google Cached View Source Code Version<\/h2>\n<p><img decoding=\"async\" src=\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/rank-ranger\/blog_1\/Ford-cached-source-version.png\" alt=\"View source version of ford.com Google cache\" \/><\/p>\n<p>Use the Search Console for a more detailed approach to the subject, and see exactly what Googlebot sees, and thus get an idea if there are missing pages to your site that Googlebot isn\u2019t seeing or parts of a page that Googlebot&#8217;s missing. Using the Search Console \/ Search Console &gt; <a title=\"\" href=\"https:\/\/support.google.com\/webmasters\/answer\/6066468?hl=en\" target=\"_blank\" rel=\"noopener\">Fetch as Google<\/a>\u00a0tool, you get a Googlebot view of your website &#8211; seeing the site exactly as Googlebot sees it.<\/p>\n<p>By understanding the basic Googlebot functionality, you can get insights into how your website looks in Google\u2019s massive index of every website it can find on the web, and every fact it can glean about the Universe.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Googlebot, the Google robot, also known as a spider crawls, indexes, and ranks the entire World Wide Web. It combs the internet for new and existing material and using an algorithm of ranking factors determines the position of your website in Google search. Googlebot&#8217;s mission is to read the code files of a website, save [&hellip;]<\/p>\n","protected":false},"author":499,"featured_media":150632,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[2803,6345],"tags":[],"class_list":["post-147696","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-marketing","category-seo"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Google&#039;s Robot Mission | Similarweb<\/title>\n<meta name=\"description\" content=\"Did you know you can view your site&#039;s Google cached versions? In addition to crawling, indexing &amp; ranking, Google saves snapshots of a page code. Learn more\" \/>\n<meta name=\"robots\" content=\"max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Darrell Mordecai\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/\"},\"author\":{\"name\":\"Darrell Mordecai\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/#\/schema\/person\/645f1730e736ea84615ff69fc556fbbc\"},\"headline\":\"Googlebot\u2019s Mission: Scan, Index, Cache and Rank the Web\",\"datePublished\":\"2015-07-08T15:22:45+00:00\",\"dateModified\":\"2025-01-21T15:40:48+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/\"},\"wordCount\":502,\"publisher\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/googlebots-mission.jpg\",\"articleSection\":[\"Marketing\",\"SEO\"],\"inLanguage\":\"\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/\",\"url\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/\",\"name\":\"Google's Robot Mission | Similarweb\",\"isPartOf\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/googlebots-mission.jpg\",\"datePublished\":\"2015-07-08T15:22:45+00:00\",\"dateModified\":\"2025-01-21T15:40:48+00:00\",\"description\":\"Did you know you can view your site's Google cached versions? In addition to crawling, indexing & ranking, Google saves snapshots of a page code. Learn more\",\"breadcrumb\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#breadcrumb\"},\"inLanguage\":\"\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#primaryimage\",\"url\":\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/googlebots-mission.jpg\",\"contentUrl\":\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/googlebots-mission.jpg\",\"width\":708,\"height\":420},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.similarweb.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Googlebot\u2019s Mission: Scan, Index, Cache and Rank the Web\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/#website\",\"url\":\"https:\/\/www.similarweb.com\/blog\/\",\"name\":\"Similarweb\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.similarweb.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/#organization\",\"name\":\"Similarweb\",\"url\":\"https:\/\/www.similarweb.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2021\/03\/1587374135933.png\",\"contentUrl\":\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2021\/03\/1587374135933.png\",\"width\":200,\"height\":200,\"caption\":\"Similarweb\"},\"image\":{\"@id\":\"https:\/\/www.similarweb.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/Similarweb\",\"https:\/\/x.com\/Similarweb\",\"https:\/\/www.youtube.com\/channel\/UCVCI01HR6iB4AA4ChW08cvQ\",\"https:\/\/www.instagram.com\/similarwebinsights\/\",\"https:\/\/www.linkedin.com\/company\/similarweb\",\"https:\/\/en.wikipedia.org\/wiki\/Similarweb\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/#\/schema\/person\/645f1730e736ea84615ff69fc556fbbc\",\"name\":\"Darrell Mordecai\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"\",\"@id\":\"https:\/\/www.similarweb.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2023\/06\/darrelm.jpg\",\"contentUrl\":\"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2023\/06\/darrelm.jpg\",\"caption\":\"Darrell Mordecai\"},\"description\":\"Darrell Mordecai is a content marketing manager at Similarweb. After working as an SEO manager and geeking out on Google patents, he acquired a deep understanding of SEO, which he regularly pulls from to create content and copy for the SEO industry.\",\"sameAs\":[\"https:\/\/www.similarweb.com\/\",\"https:\/\/www.linkedin.com\/in\/darrell-mordecai-6a401316\/\",\"https:\/\/x.com\/https:\/\/x.com\/MordecaiDarrell\"],\"url\":\"https:\/\/www.similarweb.com\/blog\/author\/darrell-mordecai\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Google's Robot Mission | Similarweb","description":"Did you know you can view your site's Google cached versions? In addition to crawling, indexing & ranking, Google saves snapshots of a page code. Learn more","robots":{"max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/","twitter_misc":{"Written by":"Darrell Mordecai","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#article","isPartOf":{"@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/"},"author":{"name":"Darrell Mordecai","@id":"https:\/\/www.similarweb.com\/blog\/#\/schema\/person\/645f1730e736ea84615ff69fc556fbbc"},"headline":"Googlebot\u2019s Mission: Scan, Index, Cache and Rank the Web","datePublished":"2015-07-08T15:22:45+00:00","dateModified":"2025-01-21T15:40:48+00:00","mainEntityOfPage":{"@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/"},"wordCount":502,"publisher":{"@id":"https:\/\/www.similarweb.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#primaryimage"},"thumbnailUrl":"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/googlebots-mission.jpg","articleSection":["Marketing","SEO"],"inLanguage":""},{"@type":"WebPage","@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/","url":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/","name":"Google's Robot Mission | Similarweb","isPartOf":{"@id":"https:\/\/www.similarweb.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#primaryimage"},"image":{"@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#primaryimage"},"thumbnailUrl":"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/googlebots-mission.jpg","datePublished":"2015-07-08T15:22:45+00:00","dateModified":"2025-01-21T15:40:48+00:00","description":"Did you know you can view your site's Google cached versions? In addition to crawling, indexing & ranking, Google saves snapshots of a page code. Learn more","breadcrumb":{"@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#breadcrumb"},"inLanguage":"","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/"]}]},{"@type":"ImageObject","inLanguage":"","@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#primaryimage","url":"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/googlebots-mission.jpg","contentUrl":"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2015\/07\/googlebots-mission.jpg","width":708,"height":420},{"@type":"BreadcrumbList","@id":"https:\/\/www.similarweb.com\/blog\/marketing\/seo\/googlebot\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.similarweb.com\/"},{"@type":"ListItem","position":2,"name":"Googlebot\u2019s Mission: Scan, Index, Cache and Rank the Web"}]},{"@type":"WebSite","@id":"https:\/\/www.similarweb.com\/blog\/#website","url":"https:\/\/www.similarweb.com\/blog\/","name":"Similarweb","description":"","publisher":{"@id":"https:\/\/www.similarweb.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.similarweb.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":""},{"@type":"Organization","@id":"https:\/\/www.similarweb.com\/blog\/#organization","name":"Similarweb","url":"https:\/\/www.similarweb.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"","@id":"https:\/\/www.similarweb.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2021\/03\/1587374135933.png","contentUrl":"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2021\/03\/1587374135933.png","width":200,"height":200,"caption":"Similarweb"},"image":{"@id":"https:\/\/www.similarweb.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Similarweb","https:\/\/x.com\/Similarweb","https:\/\/www.youtube.com\/channel\/UCVCI01HR6iB4AA4ChW08cvQ","https:\/\/www.instagram.com\/similarwebinsights\/","https:\/\/www.linkedin.com\/company\/similarweb","https:\/\/en.wikipedia.org\/wiki\/Similarweb"]},{"@type":"Person","@id":"https:\/\/www.similarweb.com\/blog\/#\/schema\/person\/645f1730e736ea84615ff69fc556fbbc","name":"Darrell Mordecai","image":{"@type":"ImageObject","inLanguage":"","@id":"https:\/\/www.similarweb.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2023\/06\/darrelm.jpg","contentUrl":"https:\/\/www.similarweb.com\/blog\/wp-content\/uploads\/2023\/06\/darrelm.jpg","caption":"Darrell Mordecai"},"description":"Darrell Mordecai is a content marketing manager at Similarweb. After working as an SEO manager and geeking out on Google patents, he acquired a deep understanding of SEO, which he regularly pulls from to create content and copy for the SEO industry.","sameAs":["https:\/\/www.similarweb.com\/","https:\/\/www.linkedin.com\/in\/darrell-mordecai-6a401316\/","https:\/\/x.com\/https:\/\/x.com\/MordecaiDarrell"],"url":"https:\/\/www.similarweb.com\/blog\/author\/darrell-mordecai\/"}]}},"lang":"en","translations":{"en":147696},"pll_sync_post":[],"_links":{"self":[{"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/posts\/147696","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/users\/499"}],"replies":[{"embeddable":true,"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/comments?post=147696"}],"version-history":[{"count":3,"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/posts\/147696\/revisions"}],"predecessor-version":[{"id":199284,"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/posts\/147696\/revisions\/199284"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/media\/150632"}],"wp:attachment":[{"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/media?parent=147696"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/categories?post=147696"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.similarweb.com\/blog\/wp-json\/wp\/v2\/tags?post=147696"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}