Does the Web need Collaborative Search Tools?

Search engine interfaces have historically been designed to let just an individual search the Web for their needs.  In over 15 years since the first Web search engine hit the market, search engine use has become ubiquitous, with many searches actually being collaborative in nature. But search engines have remained in the domain for individual use only.  Why are search engines designed only to be used alone?

Before answering this, I think it is useful to see if search engines really are being used collaboratively today? Let us look at one example in a bit of detail

Planning a vacation with friends / family: Whether it is spring break with friends, or a summer vacation with family, vacation planning involved web searching and communication, coordination and collaboration with friends or family members. When my family went on a summer vacation to Toronto, Canada recently, I had to engage my family members in the process, seeking input about places to go, places to stay, and myriad other details. Here’s how I ended up doing this job:

  • Suffering from Google addiction as many out there are, I googled many times to find interesting information about places in and around Toronto, day trips of interest, interesting places to stay etc.
  • I copied links of interest over into my email and pruned that list and would periodically pass it around for comments from the family
  • I visited many different specialty sites like Expedia, Travelocity, Hotels.com, Priceline, Kayak etc. to find possible flight itineraries, and places to stay
  • And in turn, I copied interesting links of places to stay, as well as possible travel itineraries, in email and sent that around for comments from the family
  • My wife or son would pass along interesting links via email along the way from some searches they did, or tidbits they heard from other family members / friends who had been to Toronto before.  Some more conversations would ensue.
  • Many iterations of this, and many email conversations and many in-person conversations (where that was possible) later, many days from when we started this process, we arrived at the decisions we needed.  We had firmed up an itinerary, places to stay, details of places to see, day trips to try out, and lists of links of interest towards our visit (all scattered across multiple emails).

Does this sound familiar?  This is collaborative searching at work.  Albeit with search engines that weren’t built to support it.

Let’s look at another example in a little detail.

Researching a disease or medical condition: It is not uncommon these days to have a good friend or a family member get diagnosed with some new disease or medical condition. That kicks off the process of trying to learn more about the disease or condition, finding treatment options, and finding ways to cope with the condition.  Recently, in my family, a relative of mine was diagnosed recently with high cholesterol and diabetes at the same time. They reached out to me for input on food or lifestyle changes that might help with the management of the diseases along with their regular mainstream treatment. They were keen to know about herbs or supplements that might help, or how methods like Yoga or energy healing might contribute towards a return to wellness faster.  The process that ensued was like this:

  • I googled many different queries related to these conditions, and additional queries related to diet, nutrition, supplements / herbs, lifestyle changes, looking for good authoritative information that I could pass along
  • I started collecting links into an email and sent them along in small batches to my relative
  • In turn, I’d get emails back with links and questions about the legitimacy / believability of various claims made about certain supplements or herbs.  And I’d check them out to see the sources and citations and so forth and write back about each
  • Occasionally, they would find me online on Skype and reach out to me to chat about some additional things they had read.  In the process, we’d discover some more interesting resources to keep for future use, which I’d go copy into an open email or new email
  • Dozens of queries, hundreds of pages sifted, and many email threads later, we had collected dozens of links of use for my relative. They finally had the information they needed to make their own decision in concert with their doctor

Sound familiar again?  This too is an example of collaborative search in action today.

The problem with this is that, this process is inefficient, time consuming, prone to redundant work (people doing the same queries, seeing the same sites that were not useful etc.), and at the end of all this, the useful information is spread across multiple emails and possibly some instant messaging / chat sessions, and not easily discoverable or usable when you need to consult it later on.

Here are more examples at home or in other personal contexts, where I’ve run into this need:  Shopping for an appliance or a big ticket item;  Looking for a new home; Finding suppliers for a craft project;  Finding learning resources for gifted kids etc.

Plenty of such examples also exist in the academic context or business context as well.

What is common across all of these examples is that there’s more than one person involved in the finding, collecting, organizing, sharing or using of that information.  i.e. These are prime examples of collaborative searching, which cry out for a new breed of collaborative search tools.

So, yes, I think that the Web needs collaborative search tools now.  What do you think?

My startup Zakta, is about to launch SearchTeam (sometimes mistakenly referred to as Search Team), a real time collaborative search and curation engine.  It combines traditional search engine features, with semantics, curation tools, real-time and asynchronous collaboration tools to deliver the world’s first commercial tool for real-time collaborative searching with trusted people.  SearchTeam is designed from ground up to enable users to search the Web together with others they trust, curating, sharing and collaborating on what they need on any given topic.  I’ll be sharing more information about this in the coming days and weeks.

Search Quality, SEO, The Google Farmer Update and The Aftermath

The declining search quality on Google

In the last couple of months, the online world seemed to be buzzing about Google’s declining search quality.

Google’s Response:  The Google Farmer Update

In late February, Google announced a major update to improve search result quality, and tighten the screws on content farms on the Web.  This algorithmic update, dubbed the “Farmer Update” (presumably because it tried to address the issue related to content farms) has created a scenario with winners and losers, and has also left a trail of devastation.

Analysis of the effects of the Google Farmer Update

Given that nearly 12% of search results were affected by this update, many industry experts have chimed in with analysis of winners and losers in the aftermath of this Google Farmer Update:

  • Google Farmer Update: Quest for Quality – SEO company SISTRIX published a list of big losers (based on their SISTRIX VisibilityIndex, calculated from traffic on keywords, ranking and click-through rate on specific positions). Web 2.0 company, Mahalo.com is one of the companies in the losers list, which according to SISTRIX, lost nearly 70% of their top-ranking keywords on Google.
  • Number Crunchers: Who Lost in Google’s “Farmer” Algorithm Change? — Danny Sullivan wrote a comprehensive post analyzing winners and losers from this update, citing data from multiple sources.
  • Google Farmer Update: Who’s really affected? – SearchMetrics SEO blog shares analysis of specific sites that have been hurt badly in this update, including Suite101.com, Helium.com and others.
  • Google’s Farmer Update: Analysis of Winners and Losers – Rand Fishkin of SEOmoz shares his company’s analysis of the effect of this algorithmic update on the rankings of sites.  Of particular note is the analysis of possible causes factors that could have caused lost rankings.  Initial speculation is that factors like “user/usage data”, “quality raters’ inputs”, and “content analysis” are likely to be involved, and that “link analysis” of sites was not a likely factor.
  • Correlation between Google Farmer Update and Social Media Buzz — Liam Veitch at Zen Web Solutions has done some analysis on whether Google must have considered social buzz as a factor in determining site to whack or reward.  His initial analysis seems to support the hypothesis, but requires more study.
  • How Demand Media Used PR Spin to Have Google Kill Their Competitors — Aaron Wall at SEOBook.com presents a provocative analysis about how eHow (a service of Demand Media which had an IPO recently, and often cited in the context of content farms) not only escaped the Google axe with this update, but might actually be thriving in an environment where many competitors have been killed.

Collateral Damage?

There are a lot of sites that seem to have been caught up in this “cleanup” act of Google – collateral damage as it were, in Google’s act of slashing “content farms”.  Here are some sample comments from site owners on various blogs that are telling:

My Personal blog was almost completely removed from Google’s SERPs.

Searching for my name IN QUOTES will not pull up my Blog (url is the same exact as my name).

I didn’t do anything to my site, nor did I do any SEO (white or black hat) but my search traffic is now 1 hit a day from 15-20 a day.

Google probably axed a lot of innocents in this update.

BLueSS on SEOmoz.org

We made the Sistrix list and I am currently freaking out right now. We literally lost about 70% of our US-based traffic overnight. What’s worse, we are a discussion forum with editorial who employs absolutely no black hat techniques, no duplicate content, we’re really tough on spam, and I don’t know what on earth I can do to get back into Google’s good graces. I’m convinced we somehow got caught up in the mix because I was under the impression Google was targeting “content farms” and “Made-for-AdSense” sites, and not forums. In fact, like most forum owners, I was eagerly awaiting this update with anticipation because I thought it would help us sites that deliver 100% unique, quality content.

Dani of Daniweb.com on SearchEngineLand

Well, there may be anecdotal reports of recovery, but not for my site (freegeographytools.com; 4 years old, 100% original content all by me, no farming or scraping). Google referrals are still down 20%, and AdSense earnings are down 40%+, and the trend is downwards. Thanks, Google!

leszekmp on SearchEngineLand

The real impact on hundreds of thousands of small sites may not be known for a long time.  But this has turned out to be a situation where for every loser there seems to have emerged a winner – whether the winner was deserving to win or not, and whether the loser was deserving to lose or not will remain debatable for some time for many sites.

Is search quality all good now?

Turning to a question which I’m personally interested in … Ok, now that Google has deployed an update (that by some accounts has been a year in development), is search quality all cleaned up?  Noted sites has Technorati, Songkick, PRNewswire have all been hit in this recent update, and it is kind of difficult to consider them as being similar to content farms.  So, personally I’m not sure.  I’ve not run tests myself yet with the new update, so I can’t speak to this from personal experience yet.  Some commentators like AJ Kohn point out that this update was more about demoting content deemed of low quality, not promoting better content.  According to AJ Kohn, the results are different, not necessarily betterJoe Devon comments on a ReadWriteWeb post about this:

The new results are different, but not better. I think it has exposed that Google has an immense problem.

They’ve taken care of many of the open content farms…yes. But it just pushed up a bunch of scrapers that are being a little more low key than the content farms going public or selling for millions. Results are awful…

In the mean time, Google is claiming that they are working to help good sites caught by this cleanup operation.

The Google – SEO Industry dance

We’ve seen this dance before:

  • The SEO industry at large doggedly pursues the task of finding how Google’s ranking algorithms might be working, figures out loopholes in the process, and soon, large numbers of sites out there are exploiting those loopholes.
  • Search Quality declines, and the whining from users begins, and sometimes reaches a crescendo.
  • Google pays attention, comes back at it with some algorithmic updates, fixes some issues, opens up other issues, leaves some collateral damage along the way.
  • The SEO industry (again, I mean this broadly to include all manner of SEO specialists, white-hat, gray-hat, black-hat) goes to work again to learn about the ranking updates … and the cycle continues!

This is a classic cat-and-mouse game.

To think that a chunk of the business transacted online is dependent on this, or to consider that the livelihood of many small companies (maybe even larger companies too) or solopreneurs might depend on the outcome of this game at any given time – to me this is frightening!

What do you think?

Disclosure: Readers of this blog know that my startup Zakta, will soon officially launch SearchTeam, a real-time collaborative search engine that enables personal as well as collaborative content curation.  It represents a very different approach to solve the information search problem and the attendant search quality and search relevance issues.  I’ll be writing more about SearchTeam here in the coming weeks.

Better late than never

Hello world!  Welcome to The Sharer blog.  My name is Sundar Kadayam, a technology entrepreneur, with over 24 years of experience in the software industry.

Writing this first personal blog post at Kadayam.com, I feel so much like a latecomer to a revolution that has been underway for almost 8-10 years.  100s millions of blogs and personal journals, 600M-700M users in social networks, billions of tweets and more later, does the world need another blog?  Really what is the use of another pile of opinions, another addition to the ranks of self-important megaphone wielders?

I did not take these questions lightly. It is true that I’m a Johnny-come-lately to the online world of social media and social networks.  But that is only partly true.

My professional journey had me leading teams of people measuring and monitoring the social media revolution from a time before it was called “social media”, a journey that left me with no time or energy to devote to being on the front lines wielding my own megaphone.

At the same time, I’ve been on an incredible personal journey that has brought greater clarity, healing and incredible moments of awakening.

The problem with blogging for me is that I’ve never been fond of the idea of wielding a megaphone – there’s enough noise out there already, and the world doesn’t need me to add to that noise in any way.  Will I have something useful to say?  Why should anyone care?

But then, despite my reticence to start blogging, I’ve found that people have enjoyed hearing me share my stories, experiences and tidbits from life.

Finally, the resolution to this dilemma came from a dear friend and mentor. He assuaged my doubts about the value I can deliver and pointed out just how sharing came naturally to me.

And so, and here I am, deciding to honor the sharer in me, through this blog, The Sharer, where I plan to share tidbits of learning from the experiences of my life.

Let me get started with a few things right away …

Since 1996, I’ve worked with search engine technologies and it has been an area of passion for me.  In fact, as I write this, at my current startup, Zakta, we are running a private beta of the world’s first real-time collaborative search engine.  It enables friends, classmates, colleagues or other trusted people to search together in real-time.

Another area of passion for me has been entrepreneurship.  Given my background and upbringing, I am an unlikely entrepreneur! My entrepreneurial journey has been fascinating and full of personal learning so far, and I hope to share some of that too here!

I’m also fascinated with topics related to health, wellness, healing and the deeper aspects of life and living itself.  I do healing work, and have been doing so for almost 10 years now. That has opened up a view of the fabric of life itself that has changed me deeply and permanently in the way I relate to people and the world around me.  I hope to share some of these life changing experiences here as well.

There … I’ve gotten started!  I think I can do this – be The Sharer!  And as they say, “better late than never”!

Follow

Get every new post delivered to your Inbox.