Social Search and Bing

Bing recently made major updates to their social search capability which leverages their relationship with Facebook.  With this latest update, Bing now uses “Likes” from a user’s Facebook Friends, as well as the collective wisdom gained from opinions of users at large, to better rank and present search results.

As reported widely, Bing’s updates also includes the availability of the Bing Bar, which makes it easy for users to Like any page on the Web. This is another source of social signals for Bing. On the heels of this major update to Bing.com, Microsoft has also added social search features to its mobile search as well.

Equally noteworthy has been the aggressive ad campaign around the themes “Bing and decide with your friends“, and “Friends don’t let friends decide alone“.

In the days following these major updates to Bing, there’s been a lot of discussion on the impact of these changes to the search landscape.  Here are a few topics related to this, that interested me greatly:

  • Users seem to like these new social search features in general. But is this enough to convert regular Google search users to Bing?  This is covered in a good post at Brafton.com.
  • All these social search additions from Bing (and earlier from Google) are changing the nature of search itself. The social impact on search was a hot topic at SEMPO, and that is covered well at this post from SearchAdvisory.net.
  • So, who is creating the most social search engine now?  Google or Bing?  This is a topic covered in this post, also from SearchAdvisory.net.
  • Twitter is impacting Web search results.  Facebook is altering Web search results.  Other social signals are increasingly changing the search results we see from Google and Bing. SearchEngineWatch has a collection of posts on this page related to this topic of how social signals are impacting mainstream Web search engines.
  • With Bing’s aggressive integration of social search, comes the natural question around the impact of the Facebook-Microsoft alliance on Google.  This AdAge article calls out why Microsoft’s Facebook alliance is a real threat to Google.

In my blog post on The Evolution of Social Search, I predicted that “Social Search, as we now know it, becomes a mainstream search engine feature”. Bing’s recent social search moves seem to cement that claim.

The current wave of “social search” has been around the concept of using social signals of recommendation from friends and the Web at large to alter the rank ordering and presentation of Web search results.  Good strides have been made in this regard, and I expect even more activity and integration from Google and Bing in the coming months.

My startup, Zakta, has taken the next steps in deepening social search.  Where the current generation of social search involves leveraging signals of recommendation from friends / social connections in presenting Web search results, SearchTeam.com from Zakta enables users to search the Web together with their friends and other trusted people.  SearchTeam provides the capability for friends to search together, classmates to research together, for colleagues to work together, in real-time or asynchronously, curating the best search results together from the Web.

Just as the current generation of social search features promises to improve the quality of search results for transactional searches and some simple informational searches by leveraging social signals, SearchTeam delivers the social search solution for improving the quality and experience and value of deeper informational searches through its collaborative search and curation paradigm.

What is your take on social search and its long term impact on the search landscape?

Does the Web need Collaborative Search Tools?

Search engine interfaces have historically been designed to let just an individual search the Web for their needs.  In over 15 years since the first Web search engine hit the market, search engine use has become ubiquitous, with many searches actually being collaborative in nature. But search engines have remained in the domain for individual use only.  Why are search engines designed only to be used alone?

Before answering this, I think it is useful to see if search engines really are being used collaboratively today? Let us look at one example in a bit of detail

Planning a vacation with friends / family: Whether it is spring break with friends, or a summer vacation with family, vacation planning involved web searching and communication, coordination and collaboration with friends or family members. When my family went on a summer vacation to Toronto, Canada recently, I had to engage my family members in the process, seeking input about places to go, places to stay, and myriad other details. Here’s how I ended up doing this job:

  • Suffering from Google addiction as many out there are, I googled many times to find interesting information about places in and around Toronto, day trips of interest, interesting places to stay etc.
  • I copied links of interest over into my email and pruned that list and would periodically pass it around for comments from the family
  • I visited many different specialty sites like Expedia, Travelocity, Hotels.com, Priceline, Kayak etc. to find possible flight itineraries, and places to stay
  • And in turn, I copied interesting links of places to stay, as well as possible travel itineraries, in email and sent that around for comments from the family
  • My wife or son would pass along interesting links via email along the way from some searches they did, or tidbits they heard from other family members / friends who had been to Toronto before.  Some more conversations would ensue.
  • Many iterations of this, and many email conversations and many in-person conversations (where that was possible) later, many days from when we started this process, we arrived at the decisions we needed.  We had firmed up an itinerary, places to stay, details of places to see, day trips to try out, and lists of links of interest towards our visit (all scattered across multiple emails).

Does this sound familiar?  This is collaborative searching at work.  Albeit with search engines that weren’t built to support it.

Let’s look at another example in a little detail.

Researching a disease or medical condition: It is not uncommon these days to have a good friend or a family member get diagnosed with some new disease or medical condition. That kicks off the process of trying to learn more about the disease or condition, finding treatment options, and finding ways to cope with the condition.  Recently, in my family, a relative of mine was diagnosed recently with high cholesterol and diabetes at the same time. They reached out to me for input on food or lifestyle changes that might help with the management of the diseases along with their regular mainstream treatment. They were keen to know about herbs or supplements that might help, or how methods like Yoga or energy healing might contribute towards a return to wellness faster.  The process that ensued was like this:

  • I googled many different queries related to these conditions, and additional queries related to diet, nutrition, supplements / herbs, lifestyle changes, looking for good authoritative information that I could pass along
  • I started collecting links into an email and sent them along in small batches to my relative
  • In turn, I’d get emails back with links and questions about the legitimacy / believability of various claims made about certain supplements or herbs.  And I’d check them out to see the sources and citations and so forth and write back about each
  • Occasionally, they would find me online on Skype and reach out to me to chat about some additional things they had read.  In the process, we’d discover some more interesting resources to keep for future use, which I’d go copy into an open email or new email
  • Dozens of queries, hundreds of pages sifted, and many email threads later, we had collected dozens of links of use for my relative. They finally had the information they needed to make their own decision in concert with their doctor

Sound familiar again?  This too is an example of collaborative search in action today.

The problem with this is that, this process is inefficient, time consuming, prone to redundant work (people doing the same queries, seeing the same sites that were not useful etc.), and at the end of all this, the useful information is spread across multiple emails and possibly some instant messaging / chat sessions, and not easily discoverable or usable when you need to consult it later on.

Here are more examples at home or in other personal contexts, where I’ve run into this need:  Shopping for an appliance or a big ticket item;  Looking for a new home; Finding suppliers for a craft project;  Finding learning resources for gifted kids etc.

Plenty of such examples also exist in the academic context or business context as well.

What is common across all of these examples is that there’s more than one person involved in the finding, collecting, organizing, sharing or using of that information.  i.e. These are prime examples of collaborative searching, which cry out for a new breed of collaborative search tools.

So, yes, I think that the Web needs collaborative search tools now.  What do you think?

My startup Zakta, is about to launch SearchTeam (sometimes mistakenly referred to as Search Team), a real time collaborative search and curation engine.  It combines traditional search engine features, with semantics, curation tools, real-time and asynchronous collaboration tools to deliver the world’s first commercial tool for real-time collaborative searching with trusted people.  SearchTeam is designed from ground up to enable users to search the Web together with others they trust, curating, sharing and collaborating on what they need on any given topic.  I’ll be sharing more information about this in the coming days and weeks.

Beyond spam: Big Problems with Search

The current discussion around declining search quality on Google goes to the main bread and butter issue in organic search: How good are the search results in the first page?  And in this context, the discussion is dominated by the topics of search spam and content farms and gaming of the Google algorithm. That makes sense!

In my opinion, there are a lot of unaddressed “big problems” in search beyond fixing the spam issue.  I’m citing just a few of these here.

The content explosion: There is a growing diversity of content types, explosive growth of online content, and multi lingual content, all of which contribute to the complexity of what the current and next generation search engine needs to handle. No single search engine really is able to cover the complete set of information on the Web today, and this will remain a big challenge for search engines into the future.

Hidden content sources: Part of the content explosion continues to be the proliferation of specialized content sources and databases, content from within which we can’t readily discover from mainstream search engines. This phenomenon is called the Invisible Web or the Deep Web, first written about in the late ’90s (my previous startup Intelliseek, delivered the first search engine for the Invisible Web in 1999), and continues to remain a big open issue. Attention on it has lessened only because of the sheer noise around other memes like social search, real-time search and so forth in the past few years.

Understanding user intent: Then there are age-old issues that haven’t been addressed around understanding user intent.  Much of the quality of search results has to do with not knowing what the heck the searcher really needs.  We are still feeding keywords into a single search box and expecting the magic to happen on the part of the search engine to give us what we need.  Not finding our answers, more of us are doing longer queries, hoping that will give us the answers we need. i.e. We are compensating as users for something that search engines fundamentally do not understand today: our search intent.

Understanding the content: 16+ years since the first Web search engine, we are still processing textual information with little understanding of the semantics involved. Search engines do not understand the meaning of the content that they index. This is another contributing factor that limits the quality of results delivered by a search engine to users. For long, there’s been a buzz about the semantic Web, which is supposed to usher in richer search and information experiences starting from more meaningful data and sophisticated software that can make inferences from the data in ways that is not possible today. Hailed as “Web 3.0″, it is seen as the next phase in the evolution of the Web, and that is a realm of new problems and opportunities for search engines.

Handling User input: For the most part, search interfaces have continued to use the age old search box for typing keywords as input. While promising work has been done with accepting natural language questions as input, nothing commercially viable has really turned up that works in Web scale. Without solving this problem first, there is no hope of being able to speak to a search engine to have it bring back what you are looking for.

Presenting search results: The 10 results per page read-only SERP interface that first came about in the mid 90’s is what we are essentially stuck with even today (granted that there have been recent touches like page previews / summaries added to it, and showing videos / images etc. along with links to pages / sites).  A retrospective look at this 2007 interview with usability expert Jakob Nielsen which looks into possible changes in search result interfaces by 2010 is very revealing about the relatively slow pace of change with the SERP interface.  Others have attempted purely visual searches, and still others have tried to categorize / cluster search results. Still, what the mainstream search engines offer in terms of a interface for search results consumption is not noticeably innovative.

Personalizing: For the most part, search results are a one-size-fits-all thing.  Everyone gets the same results regardless of your interests and your connections.  Some attempts have been made to personalize search results both based on some model of individual interests and on the likes / recommendations of their social group, but that is a really challenging problem to solve well.  At Zakta, our Zakta.com service made the SERP read-write, and personalizable. Other services have tried to bypass the search engine itself with Q&A services that flow through a user’s social network.

Leveraging social connections and recommendations: First generation attempts have been made to have search results be influenced by the recommendations of others in a person’s social circle. Some speculate that Facebook might be sitting on so much recommendation data that they might have a potent alternative to Google in the search arena.  Regardless, this remains an unsolved search problem today.

Facilitating collaboration in search: Web searching has been a lonely activity since its inception. Combined with the limiting read-only SERP interface, searchers have never really been able to leverage the work, findings or knowledge of others (including those that they deeply trust) in the search process.  In the post Web 2.0 world we are in, this remains an noticeable gap in search. One area of opportunity is for search engines to let people search together to find what they need.

Specialized searches in verticals and niches: For a while in the early and mid 2000’s, the buzz was all about vertical search engines and somehow that meme just faded away. The core reasons for the attractiveness of vertical / specialized search engines remain. Shopping, Travel, and plenty of other verticals represent areas which could benefit from continued development of specialized search solutions that go beyond the mainstream search engine experience.

These are but a few examples of the many open “big problems” with Search.  Seeing this, we cannot but acknowledge that we are still in our infancy in meeting the search needs of an increasingly online, connected and mobile populace.

At Zakta, my startup, we are working on solutions for some aspects of these big search problems.  We are combining semantics, curation, and collaboration technologies with traditional Web searching to deliver a new search engine called SearchTeam.  Perfect for collaborative search, or as a research tool for personal or collaborative curation of web content, we hope that SearchTeam will become a very useful part of people’s search toolset.  At this time, SearchTeam is in private beta.

What do you think are open problems, big or small, with search engines?

The Google seduction – Are instant search results blinding you to the real cost of your searches?

I’ll admit it.  I’m hooked on Google!

To start with, back in 1999, I was impressed with the speedy and mostly relevant search results delivered by Google.  I got mesmerized by their growing coverage, the advanced tools, and mostly by their simple, uncluttered, spartan interface that never changed in years.  I got lazier and hooked even more when Google started suggesting queries I could use – now I don’t even type my query fully often, and pick from the query suggestions list instead.  My addiction to Google has only gotten worse with Google Instant, where I get results as I type (granted, that this thing works well some times, and is downright irritating at other times when it slows down my typing of the query).  There are these delightful situations where I type in the name of a company, and I get the company’s web site, address, phone number and a map, ready for use – exactly what I needed.  Examples like this abound.  In all, these are ingredients of the Google seduction!  Masterfully crafted tools and features that keep me hooked and coming back for more.

You know what I mean!  It’s highly likely, from seeing market share stats, that you, like me, are hooked on Google too!

If you’ve seen the previous posts here, you might have noticed that I’m an entrepreneur, and my team has created an alternative search engine that will be released to the market soon.  Why am I admitting to being seduced by and hooked on Google?  Because it is true, and it is a reality I expect to encounter again and again in releasing our brand new search engine to market soon.

As productive and as satisfying my searches like these are on Google, there are different types of searches I do often which are downright frustrating and painful.

Example 1:

Shopping for a laptop computerTake for instance the time when I wanted to find a good laptop suitable for use by son who was heading to college (this past summer).  I was buying a laptop after a 2 year gap, so I had to find out what laptop technologies were out there, research specific features of interest to us, find reviews of models, find deals etc. – the whole effort lasted many hours. During this time, I used Google out of habit, and had to wade through tons of irrelevant and commercially hijacked results.  It wasn’t easy keeping things together that I found interesting along the way, and it certainly wasn’t easy to share findings readily with my son.  And in parallel, my son did some searches and we couldn’t easily get in synch with our efforts.  Irrelevant results, lot of time spent in sifting through results, no easy way to save what was useful, no easy way to share, no easy way to find things together, and no easy way to pause the searching process and continue from where I left off.

Example 2:

Finding a hypoallergenic natural sunblock or sunscreenHere’s another example … recently I had to research hypoallergenic sun screen / sun block solutions because my family had developed an allergy to something in traditional sun screen products.  I recall spending over 8 hours, Googling over and over across dozens of different queries, pouring over pages and pages of results, sifting through the gunk to isolate useful nuggets.  Since I had to do this across multiple sessions, often I had to repeat my searches and sift through the same results, often irrelevant, over and over again.  Post-it notes, clippings in a Word document, patchy email notes sent to other family members about this — this was my toolset for collecting, sharing and collaborating.  I did find 2-3 products finally, but it wasn’t easy.  And I’m an above-average searcher myself!

Example 3:

Planning a reunion in central FloridaIn late 2009, my extended family decided to have a family reunion in Florida.  In that context, we had to search for travel options, accommodations for 16 people, attractions, food/eating choices and ideas and a whole lot more.  Of course, I Googled over and over and over across many days doggedly, used emails to collect information and share with other family members across the country, and eventually we did have a great reunion in the Orlando area.  But, Googling offered little support in accomplishing this whole task!

If you think these searches are outliers, think to your own experiences of researching to purchase a gadget, an appliance or any big ticket item.  Think about the time you started researching places for a vacation with family or friends.  Or when you had to find more about a disease or medical condition and treatment options for a dear one.  Or the time when you had to find a supplier for a product / service at work.  The list is quite large, of searches like these, where the search itself is a process, and not something that yields an answer with a single query and the desired result on page-1.  Whether these searches emerge from our need as a consumer, or as a student, or as a business professional, the problem is the same!  The instant gratification that seduced me and you into using Google, and has us addicted to it, doesn’t do a darned thing to help here.

I don’t know how to say it, except quite bluntly – Google isn’t designed for searches like these, at least not today! Neither are other mainstream search engines.

The irrelevant results, coming from the search result pages that have been hijacked for queries with a commercial intent, make it worse.  Check out Paul Kedrosky’s post: Dishwashers, and How Google Eats Its Own Tail, Alan Patrick’s post: On the increasing uselessness of Google and Jeff Atwood’s post: Trouble in the House of Google for more examples where Googling isn’t helping as it might have some time ago. The subject of declining relevance in search results is a separate topic that I’ll write more about later.

These searches are expensive in terms of time – your and my precious time lost in googling over and over.  And yet, each one of us goes back to Google every so often for a need like this, and go through the same time wasting process over and over again!

So, when you factor in these sorts of searches that you do, what is the real cost of your searches in all?

Seduced by the tools of instant gratification, I personally believe we have been enslaved into using Google (and same could be said for regular Bing or Yahoo or Ask or AOL users too) even when it is not the right tool for the job.  “If the only tool you have is a hammer, everything looks like a nail”, the saying goes, and it seems that the vast majority of searchers are using the Google hammer all the time, even when it is clearly not the right tool for the job. The cost of doing so?  I don’t have concrete numbers to share yet, but I think it is safe to say that there’s a HUGE collective productivity loss from using the wrong tool for a searching job like this!

What is your opinion on this matter?

My startup, Zakta, is set to launch SearchTeam, the world’s first real-time collaborative search engine soon.  By combining tools to search, collaborate and curate into a single integrated solution, we hope to provide a useful search tool for finding information like this individually, or together with friends, family members, colleagues or other trusted people. I’ll share more information about this in the coming days and weeks.

Better late than never

Hello world!  Welcome to The Sharer blog.  My name is Sundar Kadayam, a technology entrepreneur, with over 24 years of experience in the software industry.

Writing this first personal blog post at Kadayam.com, I feel so much like a latecomer to a revolution that has been underway for almost 8-10 years.  100s millions of blogs and personal journals, 600M-700M users in social networks, billions of tweets and more later, does the world need another blog?  Really what is the use of another pile of opinions, another addition to the ranks of self-important megaphone wielders?

I did not take these questions lightly. It is true that I’m a Johnny-come-lately to the online world of social media and social networks.  But that is only partly true.

My professional journey had me leading teams of people measuring and monitoring the social media revolution from a time before it was called “social media”, a journey that left me with no time or energy to devote to being on the front lines wielding my own megaphone.

At the same time, I’ve been on an incredible personal journey that has brought greater clarity, healing and incredible moments of awakening.

The problem with blogging for me is that I’ve never been fond of the idea of wielding a megaphone – there’s enough noise out there already, and the world doesn’t need me to add to that noise in any way.  Will I have something useful to say?  Why should anyone care?

But then, despite my reticence to start blogging, I’ve found that people have enjoyed hearing me share my stories, experiences and tidbits from life.

Finally, the resolution to this dilemma came from a dear friend and mentor. He assuaged my doubts about the value I can deliver and pointed out just how sharing came naturally to me.

And so, and here I am, deciding to honor the sharer in me, through this blog, The Sharer, where I plan to share tidbits of learning from the experiences of my life.

Let me get started with a few things right away …

Since 1996, I’ve worked with search engine technologies and it has been an area of passion for me.  In fact, as I write this, at my current startup, Zakta, we are running a private beta of the world’s first real-time collaborative search engine.  It enables friends, classmates, colleagues or other trusted people to search together in real-time.

Another area of passion for me has been entrepreneurship.  Given my background and upbringing, I am an unlikely entrepreneur! My entrepreneurial journey has been fascinating and full of personal learning so far, and I hope to share some of that too here!

I’m also fascinated with topics related to health, wellness, healing and the deeper aspects of life and living itself.  I do healing work, and have been doing so for almost 10 years now. That has opened up a view of the fabric of life itself that has changed me deeply and permanently in the way I relate to people and the world around me.  I hope to share some of these life changing experiences here as well.

There … I’ve gotten started!  I think I can do this – be The Sharer!  And as they say, “better late than never”!

Follow

Get every new post delivered to your Inbox.