There are many changes in the latest update to Google’s search quality evaluator guidelines. Google has given us much more information to understand whether the topics our sites cover are considered YMYL. In this article we will discuss what we can learn from these clarifications. They may help us understand what Google’s algorithms want to see in terms of E-A-T.
We will also discuss some other wording changes including:
- how YMYL is classified
- how low quality pages can exist on any website, even “official” or highly authoritative sites
- how intention to harm is on Google’s radar
- more focus on whether content is designed to incite violence or hatred
- more clarity on interstitials
- a move away from the raters focusing on searches on mobile phones
- some interesting information about language - possibly a foreshadowing of MUM?
- possibly some hints as to whether the information gleaned from quality rater tests is used for machine learning to determine ranking factor weightings
Before we dig into the changes made in this update, let’s look at a resource Google linked to at the end of the newly changed QRG document.
Clear documentation on the importance of the Search Quality Rater Guidelines
At the end of the most recent QRG document Google includes a link to this document that is an overview of the guidelines. It gives a very clear picture of what they are trying to accomplish with the QRG. (Note, I’ll refer to the Search Quality Rater Guidelines as the QRG or “Quality Raters’ Guidelines throughout this article.)
I’d recommend reading the entire document. If you’re short on time, here is a summary of what I found important or interesting.
Who are the quality raters?
There are over 14,000 quality raters. Their ratings don’t directly impact search, but instead, they manually evaluate pages based on the instructions in the QRG. If Google is considering a change to their algorithms, they will generate a few hundred sample searches and then show the raters the current results compared to results with the new change implemented. They then ask them which results they prefer and why.
They evaluate pages based on the page quality (i.e. how well a page achieves its purpose) and whether a searcher’s needs have been met.
The information that is fed back to Google helps them improve search.
Is Google using machine learning to weight their ranking factors?
I personally believe that when Google says, “ratings are used to improve our systems by giving them positive and negative examples of search results” that this implies that the feedback from the raters gives Google information they can use in their machine learning systems.
In an October 2021 Google video called Trillions of Questions, Nick Fox, VP of search said, “The people at Google aren’t deciding what’s a good result from a bad result….Rather, the raters are basically teaching our computers what’s good and what’s bad...Is this a high quality result or a low quality result?”
The QRG has one small change that hints at this. The old version explained that Google's purpose in using quality raters was to determine how well their algorithms were working. In my opinion, the new version implies that the raters are helping Google's systems learn what a good result is and what a bad result is. If so, it's not a stretch to think that that information could be fed into machine learning algorithms that Google can use in their ranking system.
October 2021 version
July 2022 version
Google has not shared much about whether or not they use machine learning to determine ranking factors, but we think that the information from the quality raters helps Google’s machine learning algorithms determine what types of results are good and helpful and trustworthy.
It is interesting to note that Bing has been open about their use of machine learning to determine the weighting of ranking factors.
It's important to know that nobody is coding how Bing ranks the results... It's machine learning-based. You tell the ML model what you want & it works out the best results. It's a satisfaction & optimization problem after that: https://t.co/xIZHTWSufG pic.twitter.com/K7ZUWrEJ1X
— Glenn Gabe (@glenngabe) April 29, 2020
While the QRG are not an exact blueprint of Google’s search algorithms, they tell us what is important to Google. As Google’s Ben Gomes told CNBC, “You can view the rater guidelines as where we want the search algorithm to go…They don’t tell you how the algorithm is ranking results, but they fundamentally show what the algorithm should do."
If we can understand how quality raters assess page quality it can help us create better pages that Google’s algorithms will want to reward.
How do raters assess page quality?
The raters are given many instructions on how to determine what the purpose of a page is and whether it is potentially harmful to searchers. They then give it a rating on a scale from lowest to highest quality.
The main criterion they use is E-A-T which, according to the QRG includes:
- whether the content has taken a significant amount of time, effort, expertise or talent/skill to produce
- whether the website or the content creator has a reputation for being experts in their topics
The other important thing quality raters do is assess whether content is likely to meet the needs of searchers. This is incredibly important. If we can create content that meets user intent, then these are the types of pages Google wants to rank well!
How do quality raters determine if a searcher’s needs have been met?
The QRG has many pages of instructions and examples on how to determine whether a searcher’s needs have been met. In future podcast episodes I hope to go over many of these examples in great detail. I’d recommend you open up the QRG and read through them! Every time I do I learn something new.
Google says raters are told consider the extent to which the result:
- “fits” the query
- is comprehensive
- is up-to-date
- comes from an authoritative source
- satisfied the user, or whether the user would want to see additional results
Now that we know how the raters work and why their work is important, let’s look at what we can learn from the new changes to the QRG.
Changes in the July 2022 QRG
Less emphasis on YMYL "pages" - more on YMYL "topics"
YMYL, or “Your Money or Your Life” can be hard to understand. In many places of the QRG this edition has removed “YMYL page” and replaced it with “YMYL topic”.
Here are a few examples:
October 2021 version
July 2022 version
October 2021 version
July 2022 version
October 2021 version
July 2022 version
Google also removed (YMYL) from the webpage column in their examples of low and high quality content.
October 2021 version
Instead of calling the pages YMYL, instead they add detailed explanations as to why the topic should be considered YMYL. The explanations tend to center around why information on this topic has the potential to harm people.
July 2022 version
Why is this important? Google wants to ensure that content that could significantly impact a searcher’s life is of the highest quality.
They give even more information in this version of the QRG as to how to determine whether the topic you are covering is YMYL. As SEOs we need to be able to identify which parts of our website are covering topics considered as YMYL because these pages absolutely need to be demonstrating strong E-A-T.
More clarification on what YMYL is
Section 2.3 of the guidelines which describes YMYL has changed quite a bit. The previous version listed pages that could impact a person’s future happiness, health, financial stability or safety.
October 2021 version
The new version discusses topics and whether or not these topics have a risk of harming people.
They describe that content could be deemed harmful if the topic itself is dangerous or if the topic could cause harm where the content is not accurate or trustworthy.
July 2022 version
YMYL is measured on a spectrum
Something very interesting to me is that they now describe YMYL as being on a spectrum. They instruct raters to determine whether a topic is
- clearly YMYL
- definitely not YMYL
- something in between
Here is the chart they give with very specific examples to help us decide whether our topics are clearly YMYL, may be YMYL or not likely to be YMYL.
Guidelines on how to determine if a topic is YMYL
They also give clear instructions on how to determine if a topic is YMYL. This is extremely helpful!
From the guidelines:
- Would a careful person seek out experts or highly trusted sources to prevent harm? Could even minor inaccuracies cause harm? If yes, then the topic is likely YMYL.
- Is the specific topic one that most people would be content with only casually consulting their friends about? If yes, the topic is likely not YMYL.
I really like that second question!
Why does it matter if your topics are YMYL?
Why does this matter? Again, if you are writing on YMYL topics, Google’s standards are higher.
Google tells us in their guide to how they fight disinformation that when their algorithms determine that a query relates to a YMYL topic they will put more weight in their ranking systems to their understanding of E-A-T of the pages they present in response.
I’ve seen some studies and twitter threads lately where people said, “We added author bios and saw no benefit. Therefore E-A-T doesn’t work!” It’s important to know that:
1) E-A-T is so much more than just author bios. E-A-T represents many aspects of quality and authenticity.
2) If your topics are not YMYL, E-A-T is much less important
The level of E-A-T required to rank depends on the topic and purpose of the page
The following (highlighted) was added to section 6.0 on Low Quality Pages.
The required level of E-A-T varies depending on the topic and purpose of a page. Google may want to see more information about who created the content for a clearly YMYL topic. They mention throughout the guidelines the importance of being recognized by others online as an expert in your topic.
In other cases, all that may be needed to satisfy Google’s algorithms is:
- personal experience
- everyday expertise
- time and effort creating content
It is also interesting to note that in a few places the guidelines have been changed to say that low quality content can be found on academic websites, authoritative websites, and government websites as well.
A few other interesting changes
I did not list every change in this version as there are many. I did think the following changes were worth noting.
Pages intended to manipulate
In a few places Google mentions that pages intended to manipulate people into actions that benefit the website while causing harm to the user are to be seen as low quality.
Content that incites violence or hatred
Another addition in two places is to call out content that is intended to incite violence or hatred towards others.
Interstitials
While interstitials (ads that appear over top of the page and require a click to close) were mentioned previously in the guidelines, this part is new.
Changes specific to mobile phones
There were quite a few places where Google has changed the raters instructions to be less specific to doing their work on a mobile device. I’m not sure the purpose of this. Perhaps with mobile first indexing it is less important to distinguish between mobile searches and desktop now?
Language changes
These examples were added which I find very interesting. They are saying that when a searcher is in a Hindi locale for example, while an English result might be more relevant, a Hindi result would be more likely to be useful to the searcher.
I am wondering whether these changes are there so that Google can do better work in bringing us the full power of MUM which can glean and generate information from multiple languages.
Conclusions
I’m always excited to read the QRG when Google has made changes. They often tell us what direction Google is heading when it comes to improving search.
I took the following away from these changes:
- Page Quality and whether a page meets a searcher’s needs is important.
- Google is likely using machine learning to help determine which factors can be algorithmically determined to measure whether a searcher’s needs have been met.
- A searcher’s needs may be to have comprehensive information, up to date information, or information from an authoritative source.
- It is very important to determine whether the topics we are covering are considered YMYL. If so, good E-A-T is so important!
- YMYL topics are ones where the content, if inaccurate, has the potential to cause people harm.
- YMYL is measured on a spectrum. Some topics are more YMYL than others.
I hope you have found this interesting. If so, you might be interested in my book on how my team and I use the information in the QRG to assess quality in websites. This book was written in 2020, but is still relevant today. I will be producing a third edition of the book in the next year or so, but it will be quite different from the current version and will not completely replace it.
I’d love to hear your comments on what you think of the QRG changes!
Comments are closed.