Where was desktop search when we needed it?

Desktop search engines are sprouting like weeds. Google’s showed up in October, then Microsoft’s in December, and now Yahoo’s this month. It’s a story full of bizarre twists and turns. Did you know, for example, that Microsoft gave us a capable desktop search engine years ago?

I’d used the Windows indexing service back in the late ’90s, when it powered search on my NT-based Web server. But I’d forgotten — or maybe never knew — that the same indexing and search capabilities have long been available on the desktop, as Simon Burns  and Wesner Moise  pointed out in a pair of recent articles.

For performance reasons, the indexing service is disabled by default in Windows. Burns, however, made a remarkable discovery. Even when the index is active, Windows almost never uses it. Searches are still dog-slow unless you prefix your query with an exclamation point; then the index kicks in, and searches become lightning fast.

This seemed so improbable that I had to verify it for myself. But it’s true. According to Greg Sullivan, lead product manager of the Windows client at Microsoft, the search function starts bypassing the index as soon as even one unindexed file lands on your disk. Better to be slow than to be incomplete, in other words — a strategy that Sullivan admits looks lame in hindsight.

Well, it’s water under the bridge now. The new MSN Desktop Search improves both the engine and its user interface. Unlike the current indexer, it also reaches into the Outlook and Outlook Express mail stores.

Of course, there are lots of other ways to skin that cat. All the newcomers search mail, as do established players such as Copernic, Isys, and X1. I’ve revisited all of these recently. On balance, I think X1 — which Yahoo has licensed — is the pick of the litter. I like its interactive feel and dense information display.

For lots of people I know, any one of these choices will produce a life-changing productivity boost. For me, though, that’s no longer true. The Gmail  experiment has become a lifestyle choice. I still maintain a local Outlook mail store, and it’s indexed several ways, but I rarely need to search it. Similarly, most of the documents I create — InfoWorld stories, Weblog postings — are pushed to the cloud and are searchable there.

Few of you can or should live in the network cloud to the extent that I do. But if you refocus on the corporate intranet, cloud-based storage has compelling advantages. The first and most obvious one is the ability to access your stuff anywhere, anytime, from any client. A subtler point is that documents in the cloud are documents that other people can help you tag and find.

The gating factor for desktop search is metadata. Finding documents based on words they contain is a huge benefit, something we should all have been able to take for granted years ago. But the future isn’t faster or prettier full-text search; it’s more context and better relevance. And your personal hard drive isn’t the garden where these flowers will grow.

Google’s PageRank showed us that relevance is a collective judgment. Services such as del.icio.us, Flickr, and Furl are likewise showing us that metadata tagging wants to be a group effort. One of the ironies of desktop search may prove to be that, by the time it went mainstream, the personal hard drive was about to become an endangered species.