New Google help document says frequent crawling is a good sign

Google posted a new help document on “Things to know about Google’s web crawling.” While many of those “things to know” are already known, Google felt it would be a good idea to make this document in order to provide “basic educational information about crawling to better highlight various resources about crawling that are available to site owners.”

The document has 9 items posted in it right now including:

Frequent crawling is a good sign! Google wrote,

• “If we’re crawling your site a lot, it’s an indication your pages have fresh or highly relevant content that people want to find, and that our systems are recognizing that demand. Online shopping is a great example: we crawl ecommerce sites often so that our results will display retailers’ most up-to-date prices, promotions, and inventory status.”

Other items. Here is the full list, but make sure to check out the help document to read it all. None of it is new but it is a helpful refresher:

• What is crawling? In short, crawling is how Google “sees” the web
• We have many crawlers; they each have important jobs
• We perform repeat crawls to find the latest updates and to provide the freshest search results
• Frequent crawling is a good sign!
• Google’s crawling has grown over time as pages have become more complex
• We optimize crawling automatically
• Google crawlers never go into paywall or subscription content without permission
• Site owners have control over what gets crawled, and how
• Our standard crawlers always respect websites’ choices about how their content is accessed and used

Why we care. Crawling is a fundamental requirement for SEO and being found in Google Search and other Google surfaces. This help document might help you quickly understand how Google crawling works and what you can aim to do to improve your site’s crawlability.

View the full article