Search Quality, SEO, The Google Farmer Update and The Aftermath

The declining search quality on Google

In the last couple of months, the online world seemed to be buzzing about Google’s declining search quality.

Google’s Response:  The Google Farmer Update

In late February, Google announced a major update to improve search result quality, and tighten the screws on content farms on the Web.  This algorithmic update, dubbed the “Farmer Update” (presumably because it tried to address the issue related to content farms) has created a scenario with winners and losers, and has also left a trail of devastation.

Analysis of the effects of the Google Farmer Update

Given that nearly 12% of search results were affected by this update, many industry experts have chimed in with analysis of winners and losers in the aftermath of this Google Farmer Update:

  • Google Farmer Update: Quest for Quality – SEO company SISTRIX published a list of big losers (based on their SISTRIX VisibilityIndex, calculated from traffic on keywords, ranking and click-through rate on specific positions). Web 2.0 company, Mahalo.com is one of the companies in the losers list, which according to SISTRIX, lost nearly 70% of their top-ranking keywords on Google.
  • Number Crunchers: Who Lost in Google’s “Farmer” Algorithm Change? — Danny Sullivan wrote a comprehensive post analyzing winners and losers from this update, citing data from multiple sources.
  • Google Farmer Update: Who’s really affected? – SearchMetrics SEO blog shares analysis of specific sites that have been hurt badly in this update, including Suite101.com, Helium.com and others.
  • Google’s Farmer Update: Analysis of Winners and Losers – Rand Fishkin of SEOmoz shares his company’s analysis of the effect of this algorithmic update on the rankings of sites.  Of particular note is the analysis of possible causes factors that could have caused lost rankings.  Initial speculation is that factors like “user/usage data”, “quality raters’ inputs”, and “content analysis” are likely to be involved, and that “link analysis” of sites was not a likely factor.
  • Correlation between Google Farmer Update and Social Media Buzz — Liam Veitch at Zen Web Solutions has done some analysis on whether Google must have considered social buzz as a factor in determining site to whack or reward.  His initial analysis seems to support the hypothesis, but requires more study.
  • How Demand Media Used PR Spin to Have Google Kill Their Competitors — Aaron Wall at SEOBook.com presents a provocative analysis about how eHow (a service of Demand Media which had an IPO recently, and often cited in the context of content farms) not only escaped the Google axe with this update, but might actually be thriving in an environment where many competitors have been killed.

Collateral Damage?

There are a lot of sites that seem to have been caught up in this “cleanup” act of Google – collateral damage as it were, in Google’s act of slashing “content farms”.  Here are some sample comments from site owners on various blogs that are telling:

My Personal blog was almost completely removed from Google’s SERPs.

Searching for my name IN QUOTES will not pull up my Blog (url is the same exact as my name).

I didn’t do anything to my site, nor did I do any SEO (white or black hat) but my search traffic is now 1 hit a day from 15-20 a day.

Google probably axed a lot of innocents in this update.

BLueSS on SEOmoz.org

We made the Sistrix list and I am currently freaking out right now. We literally lost about 70% of our US-based traffic overnight. What’s worse, we are a discussion forum with editorial who employs absolutely no black hat techniques, no duplicate content, we’re really tough on spam, and I don’t know what on earth I can do to get back into Google’s good graces. I’m convinced we somehow got caught up in the mix because I was under the impression Google was targeting “content farms” and “Made-for-AdSense” sites, and not forums. In fact, like most forum owners, I was eagerly awaiting this update with anticipation because I thought it would help us sites that deliver 100% unique, quality content.

Dani of Daniweb.com on SearchEngineLand

Well, there may be anecdotal reports of recovery, but not for my site (freegeographytools.com; 4 years old, 100% original content all by me, no farming or scraping). Google referrals are still down 20%, and AdSense earnings are down 40%+, and the trend is downwards. Thanks, Google!

leszekmp on SearchEngineLand

The real impact on hundreds of thousands of small sites may not be known for a long time.  But this has turned out to be a situation where for every loser there seems to have emerged a winner – whether the winner was deserving to win or not, and whether the loser was deserving to lose or not will remain debatable for some time for many sites.

Is search quality all good now?

Turning to a question which I’m personally interested in … Ok, now that Google has deployed an update (that by some accounts has been a year in development), is search quality all cleaned up?  Noted sites has Technorati, Songkick, PRNewswire have all been hit in this recent update, and it is kind of difficult to consider them as being similar to content farms.  So, personally I’m not sure.  I’ve not run tests myself yet with the new update, so I can’t speak to this from personal experience yet.  Some commentators like AJ Kohn point out that this update was more about demoting content deemed of low quality, not promoting better content.  According to AJ Kohn, the results are different, not necessarily betterJoe Devon comments on a ReadWriteWeb post about this:

The new results are different, but not better. I think it has exposed that Google has an immense problem.

They’ve taken care of many of the open content farms…yes. But it just pushed up a bunch of scrapers that are being a little more low key than the content farms going public or selling for millions. Results are awful…

In the mean time, Google is claiming that they are working to help good sites caught by this cleanup operation.

The Google – SEO Industry dance

We’ve seen this dance before:

  • The SEO industry at large doggedly pursues the task of finding how Google’s ranking algorithms might be working, figures out loopholes in the process, and soon, large numbers of sites out there are exploiting those loopholes.
  • Search Quality declines, and the whining from users begins, and sometimes reaches a crescendo.
  • Google pays attention, comes back at it with some algorithmic updates, fixes some issues, opens up other issues, leaves some collateral damage along the way.
  • The SEO industry (again, I mean this broadly to include all manner of SEO specialists, white-hat, gray-hat, black-hat) goes to work again to learn about the ranking updates … and the cycle continues!

This is a classic cat-and-mouse game.

To think that a chunk of the business transacted online is dependent on this, or to consider that the livelihood of many small companies (maybe even larger companies too) or solopreneurs might depend on the outcome of this game at any given time – to me this is frightening!

What do you think?

Disclosure: Readers of this blog know that my startup Zakta, will soon officially launch SearchTeam, a real-time collaborative search engine that enables personal as well as collaborative content curation.  It represents a very different approach to solve the information search problem and the attendant search quality and search relevance issues.  I’ll be writing more about SearchTeam here in the coming weeks.

The Buzz about Google’s Search Quality

We may have reached a tipping point in our tolerance of the declining quality of web search results on Google. At least that is how it appears with the growing commentary on the subject from influential bloggers, writers in the news media and searchers as well.

A meme in the making?

Anil Dash writes about the decline of Google search quality citing the negative experiences and observations of Paul Kedrosky, Alan Patrick, and Jeff Atwood:

What is worth noting now is that, half a decade after so many people began unquestioningly modifying their sites to serve Google’s needs better, there may start to be enough critical mass for the pendulum to swing back to earlier days, when Google modified its workings to suit the web’s existing behaviors.

Many more bloggers are chiming in about the same matter, as this BlogPulse Conversation Tracker listing shows.

Meanwhile, at LifeHacker, over 77% of readers say that Google’s search results are less useful lately:

We asked readers last week whether what influential bloggers said was true—that Google was losing the war against search result spam. Your response? More than three quarters found Google prone to spam, with one-third tagging the decline as significant.

Michael Rosenwald wrote in the Washington post about the losing battle against spam in search results:

Google’s success rate, as measured by the percentage of users visiting a Web site after executing a search, fell 13 percent last year, according to Experian Hitwise, which monitors Web traffic. Microsoft’s Bing search engine increased its search efficiency by 9 percent over the same period.

Although there could be several reasons for the disparity, one is most certainly spam in Google’s results, analysts said.

“It’s clear that Google is losing some kind of war with the spammers,” said tech guru Tim O’Reilly, who often cheers Google’s technology. “I think Google has in some ways taken their eye off the ball, and I’d be worried about it if I were them.”

For years, Google’s organic search results have been experiencing a slow decline in quality. Paul Kedrosky writes about this in a recent blog post:

What has happened is that Google’s ranking algorithm, like any trading algorithm, has lost its alpha. It no longer has lists to draw and, on its own, it no longer generates the same outperformance — in part because it is, for practical purposes, reverse-engineered, well-understood and operating in an adaptive content landscape. Search results in many categories are now honey pots embedded in ruined landscapes — traps for the unwary. It has turned search back into something like it was in the dying days of first-generation algorithmic search, like Excite and Altavista: results so polluted by spam that you often started looking at results only on the second or third page — the first page was a smoking hulk of algo-optimized awfulness.

Would Google care?

One thing I personally do wonder about is just how important is this issue to Google in 2011 and beyond, as compared to when they started in 1999!  In my earlier post on this blog, I wrote about the ongoing relevance of search relevance to Google:

I have no doubt that Google’s relevance with organic search results will improve yet again, given the rise in negative commentary about it in influential pockets of the Internet.  However, the pertinent question to ask is this: Why would an advertising giant care more about relevance of organic search results any more than absolutely necessary to keep their revenue proposition in tact?  Or, asked another way, is search relevance ever likely to be as relevant as ad relevance to Google?

Should Google care?

What is the practical threat to Google? It seems that all this isn’t really affecting Google’s stable search market share or its growing ad revenues in any meaningful way! But there are others who see 2011 as a turning point for Google’s invincibility.

Niall Harbison wrote recently about the perfect storm coming together to unhinge Google, that too in 2011.  He cites real time search, friend recommendations, the Like button from Facebook, the rise of spam, the possibility of categorized human knowledge and Bing as some of the key factors that could unseat Google as the search king.:

For years it seemed as if Google could do no wrong and the competition be it search start ups, Yahoo or Microsoft was generally batted away with disdain. The landscape has changed over the last 18 months though and Google faces a very real danger that it’s core product could come under threat and I think 2011 will be the year where we see the first cracks start to appear in Google’s once invincible search armor.

I am not ready to predict anything about Google’s future.  I myself have been a Google addict, and I respect their talent and innovations to date greatly.  I am interested in all this in a very deep and personal way.  I been in the search engine space, dabbling with search engine technologies since 1996. But more importantly, my startup, Zakta is set to introduce an innovative search tool that we hope is very relevant to the problems we face with search today, and very useful as well.  SearchTeam.com is in private beta now, and offers unique ways to search the Web, curate what you need, personally or in collaboration with others you trust.

What is your take on this?

Follow

Get every new post delivered to your Inbox.