Search fatigue

Improving search has less to do with goosing numbers than honing the relevancy of results

A year ago March, Neil Holloway, president of Microsoft’s Europe, Middle East, and Africa operations, boasted about how good Microsoft’s search engine would become in six months.

“What we’re saying is that in six months’ time we’ll be more relevant in the U.S. market place than Google. The quality of our search and the relevance of our search from a solution perspective to the consumer will be more relevant,” Holloway said.

This got him into a lot of hot water, at least in the blogosphere where he was roundly lambasted for making the claim. He went on John Battelle’s Searchblog to explain what he really meant to say.

“[W]e are committed to investing in R&D aimed at providing a search service, initially in the US in six months, that performs better than the current industry wide standard of one in two urls being connected to the subject of the original query. I also said that our aim is to perform as good, or better, in that respect than Google. This is a long term goal. I did not put a date to it as this is work in progress,” he posted.

Yes, you did put a date to it, Mr. Holloway. See your quote directly above.

Nevertheless, a year later I decided to perform a search or two on Google, MSN Search, and Yahoo to see what kind of results I would get.

First, I searched for the “Gospel of Judas,” a recent archeological discovery of a lost Gospel written about A.D. 185.

Google gave me 1,390,000 returns; Yahoo, 1,800,000; and MSN Search, less than 500,000.

That’s quantity, but what about the quality — or relevancy, as it is called — of the search results?

This time I searched on the phrase “search fatigue” (this time, quotes included), which is only now becoming recognized as a problem, mostly by reference librarians who see it every day, in online searching. More about that later.

Here, Google returned 1,890 results; Yahoo, 431; MSN, 329.

Only Google returned multiple results on the phenomenon of search fatigue as it refers to online searches on the first page. MSN and Yahoo had only a single relevant result on the second page, nothing on the first.

As you can see, one year later Microsoft loses out to Google both on quantity and quality of results. Why the largest software company in the world can’t do better is beyond my ken.

However, the truth is that all of the current consumer search engines are inadequate. Only the most dedicated researcher would wade through 431 results, let alone a million or more. The rest of us will definitely suffer from search fatigue.

Is there a cure?

I suggest that you get a copy of the March 2007 print edition of American Libraries magazine and read Jeffrey Beall’s article, “Search Fatigue: Finding a Cure for the Database Blues.” Beall is a catalog librarian and assistant professor at the Auraria Library, University of Colorado at Denver and Health Sciences Center. Beall offers some search-fatigue antibiotics.

Search fatigue, according to Beall, is a feeling of dissatisfaction when search results do not return the desired information.

“The root cause of search fatigue,” Beall told me, “is a lack of rich metadata and a system that can exploit the metadata.”

For example, metadata-enabled searching, as you find in a library when it searches through its databases for resources, “allows for precise author, title, and subject searches,” Beall says. In other words, it looks only in the fields you request, rather than searching through the entire document. If you name the author, it looks only in the author field of each document, thus returning only relevant hits.

Beall says libraries also use “controlled” vocabularies, meaning that if someone searches for information on cannibalism, they will also receive resources that use the synonym “anthropophagy.”

After my talk with Beall, my advice to Microsoft and Holloway is to go to the folks sitting behind the library reference desk. They can probably tell you how to create a better search engine than Google more than anyone else on the planet.