Jump to content




Featured Replies

google-robot-on-phone-1920-800x457.jpg

Google has added a new user agent to its help documentation named Google-CWS. This is the Chrome Web Store user agent that is a user-triggered fetchers.

More details. Google posted about the new user agent over here, it reads; “The Chrome Web Store fetcher requests URLs that developers provide in the metadata of their Chrome extensions and themes.”

google-cws-agent-scaled.png

What are user-triggered fetchers. A user-triggered fetchers are initiated by users to perform a fetching function within a Google product.

The example provided by Google was “Google Site Verifier acts on a user’s request, or a site hosted on Google Cloud (GCP) has a feature that allows the site’s users to retrieve an external RSS feed. Because the fetch was requested by a user, these fetchers generally ignore robots.txt rules. The general technical properties of Google’s crawlers also apply to the user-triggered fetchers.”

Why we care. If you see this user agent in your crawl logs, you now know where it is from. The Chrome Web Store fetcher requests URLs that developers provide in the metadata of their Chrome extensions and themes.

View the full article





Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.