Thursday, 9 May 2024

Google had a crawling issues that impacted some sites

Between November 11th and November 17th, Googlebot slowed crawling on some websites.

Google confirmed. “We saw that crawling for the AMP cache had slowed down for some sites,” a Google spokesperson told Search Engine Land, “This bug has since been resolved, and an internal analysis showed it wasn’t widespread, and that there were no negative effects overall.”.

The issue. It seems the issue impacted only certain crawling of caches for some sites. “We saw that crawling for the caches had slowed down for some sites. This bug has since been resolved, and an internal analysis showed it wasn’t widespread, and that there were no negative effects overall,” Mueller said on Twitter.

First to notice. Olivier Papon from Seolyzer first spotted the issue and posted about it on Twitter, saying that he noticed Googlebot, Google’s web crawler, has essentially stopped crawling much of the web. He said it began on November 11 at 6PM (GMT) and then crawling went back to normal on November 17 at 8PM (GMT), he said in a follow-up tweet.

Here is the chart he shared showing the drop in crawl activity from Googlebot:

personally did notice a number of sites show a decline in crawling in the Google Search Console crawl stats report but it seemed not to impact most of the sites I have verified access to in Search Console.

Why we care. If you noticed any issues with new pages being indexed or old but updated pages not reflecting those changes in Google Search, this may be why. Google seems to have resolved the bug and going forward, all should be fine. Google said this wasn’t widespread and “there were no negative effects overall.”

Hopefully, your business and website were not impacted by this bug but if it was, you probably have zero recourse.