Has IE8 run its course as a default test browser?
WebPagetest enthusiasts – a group that includes 100% of Strangeloop employees - have grown accustomed to using the popular site to get a reliable measurement of how fast pages load across different browsers. Pat Meenan and his team have done a great job of maintaining relevance in the face of constantly shifting browser trends.
Recently, however, some here at Strangeloop began wondering why WebPagetest still uses Internet Explorer 8 as its default browser, when both Chrome and Firefox exceed it in popularity.
So we started digging…
Why is this a relevant question?
While modern browsers increasingly embrace common standards, they’re far from equal. As web pages become increasingly complex, front-end optimization (FEO) techniques that can make pages faster in some browsers can slow them down, or even break them, in others.
The problem with conducting all your tests on a single browser - such as IE8 - is you might just be getting a pinhole view of your site’s performance. This problem worsens if the majority of your traffic uses Chrome or Firefox, as it renders the findings largely irrelevant.
How many people use WebPagetest’s default settings?
To get an idea of just how many people rely on default test settings, here’s a quick breakdown of roughly 24,000 public tests on WebPagetest from July 6th.
As this graph shows, most users are indeed using the default settings:
Global browser usage
Compare these findings with the global browser breakdown for July 1-5, courtesy of Statcounter:
Comparing WebPagetest and global browser stats
A quick side-by-side comparison of these two data sets shows some striking discrepancies:
• Just 13% of global internet users use IE8, whereas roughly 65% of WebPagetests are done on IE8.
• Chrome 20 holds the largest global usage share at 23%. IE9 is second at 17%, and Firefox 13 is third at just under 15%.
• While WebPagetests are split almost equally between IE9 and IE7, Statcounter indicates IE9 to be vastly more popular.
• Further to the preceding point, the number of Safari iPad users is almost double the number of IE7 users, at 2.52% and 1.38% respectively.
Based solely on this data, an ideal set of testing options for WebPagetest might look something like this:
Or would we?
We then compared these numbers to data gathered by Akamai IO between July 1-5:
While Akamai’s results aren’t identical to ours, they’re quite similar, further validating our belief that the sample data generated represents a typical user segment in the ecommerce market.
WebPagetest versus Strangeloop browser stats
We then overlaid Strangeloop’s browser usage data on WebPagetest’s data:
The data shows strong correlation on IE8 usage, however, there’s a significant gap when it comes to IE9. Most of the browsers we believe to be the most widely used - Chrome 20, Firefox 13, and Safari — are not yet on WebPagetest’s map.