Browser usage reports: Lies, damn lies, and statistics

With news so breathless it rated a press release, StatCounter announced yesterday that over the weekend David trounced Goliath, with Chrome taking 32.71 percent of global market share, trumping IE’s lowly 32.5 percent. The crowds went wild, with bloggers dissecting the way Chrome usage goes up on the weekends, per StatCounter’s numbers, while IE’s goes down, only to reverse itself when the work week begins.

The people publicly crunching the numbers seem to forget that there’s another firm with Web usage statistics, and Net Applications and its numbers show Chrome very gradually trending upward at around 19 percent, compared to IE’s 53 percent.

StatCounter says IE has 33 percent; Net Applications says 53 percent. That’s quite a difference. Who’s right?

Not surprisingly, Microsoft sides with Net Applications — to the point of posting a detailed explanation on the Exploring IE blog. There are three points where the companies differ: prerendering, geoweighting, and unique visitors.

Back in June 2011, Chrome started “prerendering” pages based on search strings typed into Google. As Google describes it, “Sometimes a site may be able to predict with reasonable accuracy which link the user is most likely to click on next — for example, the ‘next page’ link in a multi-page news article. In those cases, it would be faster and better for the user if the browser could get a head start loading the next page so that when the user clicks the page is already well on its way to being loaded.” Google.com uses prerendering for searches; it isn’t at all clear what other sites use the Chrome-only prerendering technology.

Google has a “page visibility API” that tells Web stat packages when a page is being requested for prerendering.

In February, Net Applications stopped counting prerendered pages, claiming that “prerendering in February 2012 accounted for 4.3 percent of Chrome’s daily unique visitors.” Not surprisingly, the adjusted figures showed Chrome taking a big hit in February, with IE as the prime beneficiary.

StatCounter, on the other hand, states: “Looking at the trajectory of Chrome, however, it’s apparent that this change did not have any significant impact on our stats at all. There is no bump or sudden jump in Chrome in our stats.” And they give the data to prove their point: Chrome’s growth has been steadily upward, with no resounding bump, prerendering or not. StatCounter counts page hits, regardless of the browser’s willingness to show the pages to the user.

The second contentious issue is geoweighting. Neither Net Applications nor StatCounter have reporting sites that accurately reflect the number of Internet users around the world.

StatCounter says that in January it counted 18.1 billion hits — but only 0.17 billion of those reported hits came from websites based in China. StatCounter just doesn’t have that many tracking sites in Asia, even though there are somewhere around 50 percent more Internet users in China than in the United States.

Net Applications doesn’t have enough data collection sites in China either, but it adjusts its numbers to account for how many Internet users are in each country. “Weighting by country nearly eliminates the effects of the geolocation of our client websites, and provides a much more accurate representation of internet usage market share.” Net Applications began adjusting the numbers in August 2009.

According to Microsoft, Net Applications normalizes its numbers based on the CIA World Factbook Internet population numbers — which is to say, the estimated number of (frequent or infrequent) Internet users in each country. To use Net Applications’ own example, “We compare our traffic to the CIA Internet Traffic by Country table, and weight our data accordingly. For example, if our global data shows that Brazil represents 2 percent of our traffic, and the CIA table shows Brazil to represent 4 percent of global Internet traffic, we will count each unique visitor from Brazil twice. This is done to balance out our global data.”

Ignoring for a moment the question of how the CIA estimates the number of Internet users in China and whether the average Internet use per CIA-counted person in Brazil is close to the average use in, say, India, Net Applications hasn’t made it clear if it’s normalizing visitors based on their country of origin or based on the physical location of the Web page that’s getting hit.

In the final point of disagreement, StatCounter counts hits — page views — while Net Applications counts daily unique visitors. That’s a big difference, particularly when you consider that “unique” doesn’t mean unique individual, it means unique IP address. Now you see why the numbers vary so widely.

Which one’s right? Neither. Or both.

Clearly, StatCounter takes a head-in-the-sand, call-’em-as-we-measure-’em approach. If you’re primarily concerned about browser use in StatCounter’s major service areas — the United States, Turkey, India, Brazil, the United Kingdom, Canada, Germany, and Thailand, in that order — and don’t worry much about prerendered pages in Chrome, its numbers have merit.

Just as clearly, Net Applications is faced with an impossible task of extrapolating worldwide data from tiny samples. Net Applications admits “although we have significant data from China, it is relatively small compared to the number of internet users in China.” With geoweighting part of the Net Applications shtick, the small sample in China turns into big numbers in overall browser usage — 50 percent more than the United States.

I’ve come to two conclusions. First, the facile StatCounter explanations for Chrome’s increase on the weekends, counterbalanced by IE’s increase during the week, don’t mean much. Sure, it’s possible that people like to play with Chrome on the weekend, then they’re forced to drudge with IE during the week. But it’s equally possible that the number of IE users in India decreases over the weekend, while the total number of browser users in the United States and Turkey (StatCounter’s two largest contributors) stays the same.

Second, Microsoft’s continued harping and harping and harping about Internet Explorer 9 market share on Windows 7 treads a fine line between self-serving and disingenuous. Microsoft quotes Net Applications numbers exclusively, and without knowing a lot more about Net Applications servers in China — certainly the most heavily weighted number in the mix — it’s hard to give them much credence.

At least there’s a silver lining: With Windows 8 and Internet Explorer 10 in the not-too-distant future, we’re going to be treated to some fancy talk about worldwide browser market share on Windows 7 and Windows 8, encompassing both IE9 and IE10. Somehow, Microsoft will spin out numbers showing IE is taking over the world. I can already hear the obfuscating machines whirring up to speed. It should be entertaining.

This story, “Browser usage reports: Lies, damn lies, and statistics,” was originally published at InfoWorld.com. Get the first word on what the important tech news really means with the InfoWorld Tech Watch blog. For the latest developments in business technology news, follow InfoWorld.com on Twitter.