Google 'Army' to Flag 'Anti-Islam, Anti-Semitic And Racist' Search Results

Google is undertaking a new effort to better identify content that is potentially upsetting or offensive to searchers. It hopes this will prevent such content from crowding out factual, accurate and trustworthy information in the top search results.

"We're explicitly avoiding the term 'fake news,' because we think it is too vague," said Paul Haahr, one of Google's senior engineers who is involved with search quality. "Demonstrably inaccurate information, however, we want to target."

The effort revolves around Google's quality raters, over 10,000 contractors that Google uses worldwide to evaluate search results. These raters are given actual searches to conduct, drawn from real searches that Google sees. They then rate pages that appear in the top results as to how good those seem as answers.

...Quality raters use a set of guidelines that are nearly 200 pages long, instructing them on how to assess website quality and whether the results they review meet the needs of those who might search for particular queries.

Those guidelines have been updated with an entirely new section about "Upsetting-Offensive" content that covers a new flag that's been added for raters to use. Until now, pages could not be flagged by raters with this designation.

The guidelines say that upsetting or offensive content typically includes the following things (the bullet points below are quoted directly from the guide):

• Content that promotes hate or violence against a group of people based on criteria including (but not limited to) race or ethnicity, religion, gender, nationality or citizenship, disability, age, sexual orientation, or veteran status.
• Content with racial slurs or extremely offensive terminology.
• Graphic violence, including animal cruelty or child abuse.
• Explicit how­ to information about harmful activities (e.g., how tos on human trafficking or violent assault).
• Other types of content which users in your locale would find extremely upsetting or offensive.

The guidelines also include examples. For instance, here's one for a search on "holocaust history," giving two different results that might have appeared and how to rate them:

The first result is from a white supremacist site. Raters are told it should be flagged as Upsetting-Offensive because many people would find Holocaust denial to be offensive.

The second result is from The History Channel. Raters are not told to flag this result as Upsetting-Offensive because it's a "factually accurate source of historical information."

In two other examples given, raters are instructed to flag a result said to falsely represent a scientific study in an offensive manner and a page that seems to exist solely to promote intolerance:

What happens if content is flagged this way? Nothing immediate. The results that quality raters flag is used as "training data" for Google's human coders who write search algorithms, as well as for its machine learning systems. Basically, content of this nature is used to help Google figure out how to automatically identify upsetting or offensive content in general.