Does Microsoft have a new search strategy?

Can David help Goliath slay Google? Probably not. Microsoft is a couple of light-years behind Google in market share. But buying Powerset, a San Francisco startup focused on semantic search, is a smart move that will help the stumbling software giant get back in the game.

And in much the same way that competition from Mozilla and other upstarts has forced Microsoft to (finally) improve its Internet Explorer, Google (which has a nearly comparable lock on much of the search market) will be forced to improve its capabilities.

Before delving deeper into the story, here’s a heads-up: Microsoft has hung out a “help wanted” sign for engineers skilled in search technologies. In the blog post announcing the Powerset acquisition, Satya Nadella, who heads Microsoft’s search business, said, “We’re buying Powerset first and foremost because we’re impressed with the people there … We’re looking to add even more talented engineers to the San Francisco team to accelerate our shared progress. If you’re interested in joining the team, drop us a line.”

And if you’re skilled in Ruby on Rails, you have a leg up, since Powerset uses the hot language for programming.

A smarter search

Comparing the smarts of Powerset’s search to Google’s isn’t fair. Google has indexed much, if not all, of the Web. Powerset has “only” indexed Wikipedia — a massive, but limited, task. Still, the premise of semantic search, developing algorithms that parse the meaning of a query, as opposed to the brute-force keyword matching of conventional search is promising.

Semantic search, by the way, is not exactly a synonym for natural language search. It’s about how the engine analyzes the information on a Web page. The fact that a query can be framed more naturally is helpful, but not really the point. The real breakthrough is being able to understand text by using a series of powerful grammar engines, says Powerset CTO Barney Pell.

His company’s core technology — a natural language parser called the Linguistic Environment platform — was licensed from the Palo Alto Research Center (the former Xerox PARC), one of the most fertile beds of technology innovation in the industry.

Here’s how PARC explains the semantics problem: “‘The company is ready to sell’ is not easy for a computer to understand because the sentence is syntactically ambiguous — is the company opening for business, or does it want to be acquired?”

Resolving this ambiguity requires understanding the context: Is the sentence in the middle of an article on mergers and acquisitions? Or is the sentence followed by “Its shelves are stocked with all the hot products”? This succeeding sentence is helpful only if the computer understands that the possessive pronoun “its” refers to the company, and that “stocked” and “products” are more relevant to selling goods than to being acquired.

Google doesn’t work that way. And because it doesn’t, its searches can be frustratingly inaccurate if the context is ambiguous. Adding natural language capabilities to Google (or any other search engine) is certainly possible, but would require a huge investment in time and money to re-index billions of Web pages.

A smarter strategy for Microsoft, too

Microsoft, of course, faces the same challenge. But having saved $46 billion by not buying Yahoo, Steve Ballmer and company can easily afford the effort. Indeed, it has little choice. The search market is really about ad dollars, and building a better search engine is the way Microsoft figures to snag them.

Despite a fair amount of effort and cash pushed into MSN and the search business, Microsoft is still an also-ran in that arena, with just 6 percent of all U.S. searches, compared to 20 percent for Yahoo and 68 percent for Google, according to Hitwise, a research firm. Aside from raw share numbers, Google has turned its name into a verb (as in “to Google”), the ultimate marketing coup. Putting a dent in that kind of mind share will be extraordinarily difficult.

Powerset will give Microsoft incremental advantages, and that’s a good thing. But to really challenge Google in search, Microsoft will have to do something extraordinary.

One possibility: an engine that does a good job searching multimedia content, and not just tags or captions. There’s already a developing market in facial recognition technologies used for law enforcement. A search engine that allows consumers to find an image or a song would be the game changer Microsoft desperately needs.

(Disclosure: I own a small number of shares Microsoft shares.)