blog Release 2024-02-27

We will now crawl your website (with some limitations) to discover your URLs to ensure your site is indexed as comprehensively as possible. We’ll only crawl your site if you allow it. This is helpful if your sitemap does not include every single page of your site.

• Site crawling (not working for SPAs yet) to discover your URLs in the event of not having a sitemap in Webmaster Tools or you do not yet have a sitemap enabled
• We’ll now check the indexing status of crawled URLs to better help auto-index your content (your sitemap may be missing or incomplete, and non-indexed URLs are not going to show in our organic impression data for you)