Why your pages are stuck in crawled-currently not indexed. And what to do about it.

I have had a number of site owners reach out to ask for help with indexing issues lately. In most cases, I am finding that Google has classified their pages as “crawled-not currently indexed” in the page indexing report in Google Search Console.

In almost every case I’ve examined these pages have quality issues. They are usually “commodity content” - essentially rehashing what many others have already written on a topic without offering anything new or more helpful than what currently exists online.

In this article, I’ll share how I look at the crawled-currently not indexed report in GSC. I’ll give you a tool to help you find the pages in this report that you should analyze further. And I’ll give you some tips for improving so you can possibly recover. I must give fair warning though. For most sites, if you have a large number of pages you want indexed, but they are stuck in crawled-currently not indexed, recovery will be difficult.

What Google said about crawled-currently not indexed at the Google Search Central event in Toronto.

I attended the Google Search Central event in April of 2026. The organizers asked us not to attribute quotes directly to any Googler, but they did give us permission to share what was said.

One presenter shared about how Search works. He said that when Google crawls a page it essentially means they download it. Then, “If we think it’s useful we might put it in a database,” or in other words, in Google’s index.

Then he talked about what kinds of things Google wants to put in the index. He said that AI has made the threshold for creating things lower. If anyone can create content on anything, then the type of content Google wants to add to their index is content that offers two things: personal experience, and knowledge no one else has.

He said that if Google has crawled your page and has decided not to index it, there could be two reasons:

1. There could be a technical issue.

I have found this to be rare. However, just last week I reviewed a site that had undergone a migration and all of their pages were stuck in crawled-currently not indexed. Note: This is not the same as “Discovered-not currently indexed” which means that Google is aware of the pages, but has not yet crawled them.

My first step was to analyze whether Google could see the content on pages. I used the page inspection tool in GSC by clicking the magnifying glass next to the url in the crawled-currently not indexed list and clicked, “Test Live URL” Surprisingly, when I viewed the live tested page, all it showed was a heading, a few boilerplate words and no content whatsoever.

In this case, the site owner did indeed have a technical issue. Their robots.txt had this line, Disallow: /*?*. The idea was to block crawling of urls with parameters like ?replytocom or ?utm_source. But, their new theme relied on these parameters for their CSS files and javascript so they were essentially blocking Google and all other search engines from seeing most of their content.

We have since removed this block and very slowly, pages are starting to appear back in the index again.

If your live test shows Google can indeed see the content on your pages, it’s very unlikely to be a technical issue that is causing crawled-currently not indexed problems.

I should also mention that some pages should be in your crawled-currently not indexed list if they are not the canonical version. If you see /feed/ pages or pagination or pages with url parameters, this is normal.

2. Quality

The Googler in Toronto went on to explain another cause for Google to crawl a page and not index it. He said it could be because “we looked at it and found it not to be good.” He said that if thousands have covered the exact same topic they might decide that your page is unlikely to be useful in Search. It might be that there are other options that are more popular or better quality.

He also said that sometimes Google experiments by allowing your page to be indexed for a while to see if users like it, “We are experimenting with seeing which one produces happier users.” That is a pretty wild statement!

My bet is that if you have pages that you want indexed, but Google has them in the crawled-not currently indexed bucket, then your main issue is related to commodity content.

Commodity content is the most likely cause

Google talked a lot about commodity content at this event.

Commodity content is content that almost anyone could write about a subject. It's generally repeating what already exists online on other sites. Non-commodity content brings a unique viewpoint or has content that others lack or can’t easily replicate. It usually demonstrates first-hand knowledge or experience.

Take this article you are reading right now. Anyone could use AI to write a helpful article defining crawled-currently not indexed pages. My article, however, talks about my experience as a professional who is paid to give my opinion on this subject. I’ve shared the real world technical example above, I’ve shared first hand information I learned from attending a Google event, and I’m about to share my observations on pages that have been deemed not worthy of indexing.

My observations of pages stuck in crawled-currently not indexed

These pages are usually not junk. They’re good, decent articles - as good as the pages that Google is ranking. And that’s just the point. The pages aren’t special or any more valuable than what currently exists.

Here is the process I use to analyze these pages.

To find the list, click on “Pages” under Indexing in GSC. Then click on Crawled - currently not indexed:

Below this you’ll see a list of urls to investigate. (Below I’ll share more about a tool I’ve created to help you filter this list to see the urls that really matter.)

I’ll find a url in this list that really is one that we want indexed.

First, I’ll search for some queries that you would expect the page to rank for. On the SERP, there is usually an AI answer that is very helpful. Often, a user will find the answer to their question there. If this is the case, then why would they want to click through to your website to read the exact same thing?

I’ll expand the AI overview and then open up Gemini in the Chrome sidebar. Then I hold down CTRL/Cmd and click on the top websites linked to from within the AIO. If you do this when you have Gemini in the Chrome sidebar opened, you’ll find these tabs get added to your Gemini conversation.

Then I type “/” which opens up the skills I have saved at chrome://skills/ and choose my Non-commodity check. (If you’re a member of my paid community, you can find this full skill here.)

This skill is a very long prompt that looks at some of the things Google tells us their algorithms aim to reward in their documentation on creating helpful content, including, but not limited to:

• Does the content provide original information, reporting, research, or analysis?
• Does the content provide insightful analysis or interesting information that is beyond the obvious?
• If the content draws on other sources, does it avoid simply copying or rewriting those sources, and instead provide substantial additional value and originality?
• Does the content provide substantial value when compared to other pages in search results?

And Gemini gives me some of the reasons why the pages linked to offer value to the reader. Note: Sometimes pages are ranking not because of their non-commodity value but because they are an authoritative source. When you're a known authority you can get away with a bit more "commodity-ness."

Now, we need to acknowledge that Gemini does not have inside insight into Google’s ranking systems. It does not know why certain pages are ranking. What we are trying to learn here is what types of things could be helping a page be worthy of presenting to searchers.

Then, I open up my client’s page and prompt this, “Now analyze this page according to the same criteria.. This page is not ranking well. It is our client. Please share where you think it is lacking. No need to suggest improvements at this point.”

Here is the result for one crawled-currently not indexed page I used this prompt on.

John Mueller and Martin Splitt discussed crawled-currently not indexed in a recent podcast

As I was about to publish this, Google published a Search Off the Record Podcast on “How to read the Indexing Report”. There’s a lot in here so I bolded the parts that I thought were important.

This discussion starts at 20:32 in the video

Chapter 9: Discovered vs. Crawled Not Indexed: Is it a technical or site quality issue?

“And also if you add or change your site or if your site is very new, then you can actually also use this report to see a little bit how your site goes through the different stages, because at some point, you're going to see pages in Discovered currently not indexed. Which tells you we know they exist, but we haven't actually visited them. And if we haven't visited them, we can't put them in the index versus. Crawled-currently not indexed, which means we visited them and we didn't put them in the index. And that can have all sorts of different reasons. Would you say that is often or only sometimes a sign of a quality issue?

So it's definitely the case if our systems are seriously worried about the quality of a website, that they will reduce the number of pages that they index. Because if we have strong concerns about the overall quality, then it doesn't make much sense for our systems to spend a lot of time on the website.

So we'll probably crawl a lot less, we'll index a lot less, and then you'll see things like crawled, not indexed or discovered, not indexed, which from our point of view is basically our system saying, we know about this, we looked at it, and once we're happy, we will take another look and see if we can index it. It's not so much that I would say you should take these situations and try to fix them. From a technical point of view, it's not that you need to fix this technical issue that Google is not indexing this page at the moment, but rather you almost need to when you recognize a bigger pattern like this, that Google is not indexing a lot of your pages, and there's no technical reason, you almost need to take a step back and think about the quality overall.

And thinking about quality is really challenging because a lot of times, it's your website and it's your baby. And of course, it's the best baby ever. But taking a step back and trying to look at it with the eyes of someone who is not directly involved with your website. Sometimes that opens up some ideas for areas where you can improve, where maybe if most of your website is AI generated and it worked for a while, it might be that people look at this AI generated site and they're like, well, I can tell this is AI generated. There's nothing unique or valuable that is available here for me. That's not to say that all AI generated content is bad, but sometimes you just run across websites where you're like, anyone could have written this. This tells me nothing. Yeah, that's true. And I think what makes this difficult is not only the fact that obviously the way you wrote it is the way you thought is best, and that's why you think it's high quality, of course. So that's really, really hard to step out of your own perspective. But sometimes it's also there's so much other stuff that is just as good. So why would we add it to the index.

And then that can tell you, like maybe this content isn't as valuable as I thought it is because other people are covering the same thing. And then what's the value of this version of it being in the index. Yeah that's true. I feel we could have a whole podcast about quality. I think maybe one other thing that is worth mentioning with regards to quality is it's not just the text. So a lot of times people will say, well, my text is unique or my articles are good and they're packaged in a page that is terrible to access where anyone who, when they try to load it like their computer fan spins up and they're like, oh my gosh, I have to run away to make sure my computer doesn't explode. So maybe that's an extreme case, but you've all seen these pages where basically the text is there, but it's almost hidden away, hidden behind ads, hidden behind interstitials, hidden behind other things that are moving and coming and going, maybe hidden below a bunch of filler content, which we sometimes see, for example, with recipes where there's this really long story on top that maybe most people don't really care about. And then the recipe comes. These are all the kind of things where the overall quality is much more than just that piece of text that you say, this is my main content. This is what Google should be counting for my site. And from our point of view, we almost have to take into account the full experience on a page, because that's what users see. It's not that users go to a web page and turn on some magic mode that just pulls out the text, but rather they have the full experience of this website with all of the 3D, 4D animations and everything. I agree very much. Agree oh my God.”

How can you fix this issue?

Oh boy, this is the tough part of this article, because in a lot of cases I feel that it is extremely difficult to get pages out of crawled-currently not indexed. I mean, if excessive ads and filler are to blame, there are obvious things to improve on there. If there’s a technical issue, fix it and request reindexing via GSC - or just be patient and wait till Google tries to crawl your pages again.

If it’s a quality issue though, you’re likely going to have to put significant effort into improving these pages.

For many sites that I analyze, their superpower in the past was the ability to cover a topic thoroughly. Lately there is a trend to not only cover a topic, but to anticipate all of the fan out queries and cover those as well. This was mentioned in the Google Search Central event a few times. If you are creating loads of content based on this method, you run the risk of facing a scaled content penalty. I cannot prove this yet, but I suspect that the June 2026 spam update impacted a number of sites that were creating commodity content at scale. If this is true, you won’t see a manual action in GSC. You’ll just see a drop in organic traffic with no explanation.

I fear for a lot of SEO agencies because for many, your number one tool in your toolbox is content creation. AI has made it much easier to cover content on any subject. I am not against using AI to help with content creation. But, if your SEO company can use AI to create content on your topics, then it’s likely not original, insightful and substantially more helpful than what currently exists. There are exceptions. I know of some agencies that use clever AI pipelines to interview a business, extract their relevant experience and turn that into good, original content.

Although I don’t recommend using AI to write your content for you without any human input, I do think you can brainstorm with AI to help improve it. The problem though is that the solutions will require effort. The word “effort” is used 120 times in Google’s Quality Rater Guidelines. You will want to find ways to draw from your experience to create content that adds to the body of knowledge that currently exists on your topics.

Try this simple prompt. Give your content to an LLM or open up Gemini in the sidebar and ask this, “Is this content likely to be considered commodity content?”

I just opened up Gemini in Google docs and asked about this very article you are reading now:

Next, try this for some ideas.

“Give me 20 ideas that help me draw from my first hand experience to make this article even more helpful, and substantially better than anything else that exists on this topic on the web.”

Damn there are some good ideas in here.

Some tools to help you assess your crawled-not currently indexed pages

I created a couple of tools using Google’s Antigravity. You can find them at tools.mariehaynes.com.

There are two new tools:

1. Filter your crawled-not currently indexed urls. Export your crawled-not currently indexed urls from GSC. If you export as CSV, open the zip file and find the table.csv file. You can upload it to this tool and it will strip out /feed/ pages and others so that you can see and click on the urls that you want to investigate.

2. GSC Index Checker. You will need to login in to your Google account to use this tool, but know that I don’t see any of your data. It will check a list of urls to see what their indexing status is. You can choose from the most recent pages in your sitemap, paste a list of urls in manually, or have the tool grab your top trafficked pages from GSC.

What you’re looking for here is whether these pages that matter to you are indeed indexed, or whether they’re stuck in crawled-currently not indexed.

I hope this article helps! Google does seem to be getting more strict on what it is indexing these days.

Marie

Or, join us in the Search Bar for up to the minute news on Search and AI.