How to use a waterfall chart to interpret web performance data

Waterfall diagrams are visual representations of data that are generated cumulatively and sequentially across a process. In web performance, waterfalls allow us to see the series of actions that take place between user and server when a user accesses a website.

Here at Strangeloop, we work with waterfalls every day in an effort to not only educate customers on their own website performance, but also help guide our internal operations. Our team encounters plenty of executives, site owners and other customers who aren’t sure how to interpret waterfall charts. This post is intended to help beginners understand a typical performance waterfall, both pre- and post-acceleration. 

Here’s what a waterfall looks like for a typical website that hasn’t been optimized:

Looks like a mess, but a few basic facts make it seem less complicated: 

1. Each row represents a different object within the page.

2. An object could be text, image, Javascript files, etc. 

3. Each object makes its own roundtrip between server and browser.

Now let's take a closer look at the same image:

Each of the colored bars represents a different activity that happens as the object is delivered to the user’s browser:

Dark green = DNS lookup

This is when the browser looks up the domain of the object being requested, similar to looking something up in a massive directory. DNS lookups don’t cause much of a performance problem on most sites. 

Orange = TCP connection

Also called the “three-way handshake,” this is the process by which user and server make friends by sending and receiving acknowledgment that a connection exists, and that data transferring can begin.

It’s not easy to speed up a TCP connection, but limiting the number of connections will improve performance.

Bright green = Time to first byte

This is the window of time between when content is requested from a server, and when the first bit arrives back.

The user’s internet connection is a factor here, but there are other variables at play: the amount of time it takes your servers to “think” of what content to send, and the distance between servers and the user.

Blue = Content download

This is the time it takes for every piece of content to be sent from the server to the browser. 

You can speed up content download by making the total amount of stuff you need to send smaller. More on that later.

Back to our original chart:

Has IE8 run its course as a default test browser?

WebPagetest enthusiasts – a group that includes 100% of Strangeloop employees - have grown accustomed to using the popular site to get a reliable measurement of how fast pages load across different browsers. Pat Meenan and his team have done a great job of maintaining relevance in the face of constantly shifting browser trends.

Recently, however, some here at Strangeloop began wondering why WebPagetest still uses Internet Explorer 8 as its default browser, when both Chrome and Firefox exceed it in popularity.

So we started digging…

Why is this a relevant question?

While modern browsers increasingly embrace common standards, they’re far from equal. As web pages become increasingly complex, front-end optimization (FEO) techniques that can make pages faster in some browsers can slow them down, or even break them, in others.

The problem with conducting all your tests on a single browser - such as IE8 - is you might just be getting a pinhole view of your site’s performance. This problem worsens if the majority of your traffic uses Chrome or Firefox, as it renders the findings largely irrelevant. 

How many people use WebPagetest’s default settings?

To get an idea of just how many people rely on default test settings, here’s a quick breakdown of roughly 24,000 public tests on WebPagetest from July 6th.

As this graph shows, most users are indeed using the default settings:

Global browser usage

Compare these findings with the global browser breakdown for July 1-5, courtesy of Statcounter:

Comparing WebPagetest and global browser stats

A quick side-by-side comparison of these two data sets shows some striking discrepancies: 

• Just 13% of global internet users use IE8, whereas roughly 65% of WebPagetests are done on IE8.
• Chrome 20 holds the largest global usage share at 23%. IE9 is second at 17%, and Firefox 13 is third at just under 15%.
• While WebPagetests are split almost equally between IE9 and IE7, Statcounter indicates IE9 to be vastly more popular.
• Further to the preceding point, the number of Safari iPad users is almost double the number of IE7 users, at 2.52% and 1.38% respectively.

Here’s a graphical illustration of the differences:

Based solely on this data, an ideal set of testing options for WebPagetest might look something like this:

• Add the capability to test on Chrome 20, and make this the default browser.

• Add Firefox 13, Safari 5.1, and Safari iPad.

• Implement the ever-popular Browser Tax on those who test on Firefox 6.

Or would we?

These global usage numbers do not reflect the real-world data seen here daily at Strangeloop, so we gathered traffic data for the same time period from a handful of Strangeloop customers - approximately 350,000 unique visits in total - and broke down the data by browser version:

 

We then compared these numbers to data gathered by Akamai IO between July 1-5:

While Akamai’s results aren’t identical to ours, they’re quite similar, further validating our belief that the sample data generated represents a typical user segment in the ecommerce market.

WebPagetest versus Strangeloop browser stats

We then overlaid Strangeloop’s browser usage data on WebPagetest’s data:

 

The data shows strong correlation on IE8 usage, however, there’s a significant gap when it comes to IE9. Most of the browsers we believe to be the most widely used - Chrome 20, Firefox 13, and Safari — are not yet on WebPagetest’s map.

Conclusions 

1. IE8 remains relevant as a default test browser, but IE9 is the future.

2. WebPagetest could generate more accurate results if they offered different defaults based on country and/or user/application type.

The bottom line

Browsers matter, and before running tests, it's wise to examine your analytics to to get a better picture of who your users are.

Questions about browsers and web performance?

Talk to a Strangeloop Performance Expert.

13 questions (and answers) about Google, site speed, and SEO

When Google announced in 2010 that page speed factors into search rankings, a flood of questions followed: How do they do it? How much does it factor into the search algorithm? Should site owners care?

As Google’s ultimate concern is delivering a better end-user experience, it’s no surprise that faster pages can be rewarded with higher rankings. Strangeloop president Joshua Bixby has done some digging to find out the how and why of page speed and SEO. Read on for his findings:

1. Does the Google search bot track page load time?

A lot of people assume that Googlebot measures page load, but in fact, no, the bot has nothing to do with measuring speed.

2. Does Google use synthetic tests or real end user monitoring (RUM) to gather its data about page speed?

We’ve heard speculation that Google uses tools like Page Speed to score sites' performance, but this isn't the case. Google uses real end-user monitoring (RUM) to check site speed. This is the right thing to do. They’re measuring from users’ actual web browsers and from real bandwidths — no simulations.

3. So how does Google gather real-world performance data?

Google crowdsources page measurement, and the process is actually kind of nifty. If you use the Google toolbar with "PageRank checking" activated, then the toolbar measures the load time of every page you visit and sends the results back to the mothership. The results are then aggregated and used to determine real-world speed for each page.

4. What browsers does the Google toolbar use?

The toolbar functionality is, as you would expect, already embedded in Chrome. The Google toolbar is also available as an add-on for Internet Explorer 6+. It used to be available for Firefox 2-4, but Google recently discontinued Firefox support, which has led to speculation that they have another plan for capturing this data from Firefox browsers; however, no other details have emerged.

5. What exactly does the Google toolbar measure?

It measures "onload time": the time it takes for all page resources to render in the browser -- from resources you can see, such as text and images, to those you can't see, such as third-party analytics. (Geek definition: "onload time" is also known as "document complete time" or "load time".)

There's a big caveat here: While onload time is an important measuring stick for performance, it needs to be taken with a grain of salt, because it isn't an indicator of when a page begins to be interactive. A page with a load time of 10 seconds can be almost fully interactive within 3 to 5 seconds. That's because onload time can be inflated by third-party content, such as the aforementioned third-party analytics, which users can't even see. Lesson learned: Even if your pages feel fast to you and your users, if they take a long time to get to onload, then according to Google they're slow.

6. What pages does Google measure?

Google measure every page visited by users on your site.

7. What? Even pages I’ve marked as non-crawlable?

That's right. Because the Google toolbar grabs all user-generated data via each participating user's browser, it allows Google to measure pages your users use, not what you have told Google is crawlable.

8. What if my page is personalized and has very different content for authenticated users, but the same URL?

The Google toolbar makes no distinction between personalized content if the URL remains the same. All results are averaged together to determine the final score.

9. Does Google use its Google Analytics Site Speed feature to calculate load times into its search algorithm?

Last year, Google added a new feature to Google Analytics that measures and reports real-world page speed to Analytics users who turn that feature on. Site Speed lets site owners know which pages are fastest and slowest, how page load time varies geographically, and how fast pages load for different browsers. You would think that all this data would be useful for factoring page speed into search ranking, but based on their silence on the subject, Google doesn't use any of the data collected in Google Analytics for this purpose. In our opinion they should, as it would allow them to sample more modern browsers.

10. Will preloading content on a page hurt my ranking?

"Preloading" is a fantastic front-end optimization technique that we use in our FEO solutions at Strangeloop. It lets us constantly track and analyze how visitors use a site. Then, using this information, it predicts what pages people are most likely to visit next and then pushes page resources to the visitor's browser so they're waiting on standby before the visitor even clicks.

Preloading is great because it gives the illusion of nigh-instantaneous page load, but site owners often ask me if it's a trick that Google will reject and possibly even penalize them for. The short answer is no. As mentioned above, Google's score is based on the onload time measurement. Preloading doesn't cheat the system. It simply improves your onload time.

11. Will deferring page resources help my rankings?

"Deferral" is another excellent optimization technique. It allows you to defer non-essential page resources -- such as third-party scripts -- so that they load after the onload event. Deferral is an honest technique in Google's books. Anything that helps a page improve its onload time will improve that page's score.

12. Will having pages start render faster help my rankings?

"Start render time" is different from "onload time". As its name suggests, "start render" indicates when content begins to display in the user's browser. While it can be measured, start render doesn't indicate whether the first content to populate the browser is useful or simply ads and widgets.

Having said that, start render time is still a useful measurement because, over time, it gives good insight into the performance health of a page. This is neither here nor there, because, as I've already mentioned, Google doesn't factor it into its results, instead focusing exclusively on onload time. In my opinion, it would be great to augment Google's search algorithm with some way to benefit pages that start loading faster.

13. How much should I care about page speed?

The previous questions address the "How do they do it?" question. It's tougher to answer the "How much should I care?" question. I've read arguments that making pages faster did little to nothing to improve search ranking, and I've read case studies from companies that say they've made their pages faster and grown organic traffic anywhere between 20 and 40 percent.

The long and the short of it is, better organic search rankings are yet another benefit of adding advanced front-end acceleration for site owners. 

Looking for more information on how faster pages can boost organic search rankings, conversion rates, and the overall profitability of web applications? Contact a Strangeloop Performance Expert.

Web Performance Digest: June 2012 - User experience tips, case studies, and the best of Velocity

At the beginning of each month, we update our Web Performance Hub with a collection of the most compelling articles, posts, videos and presentations from across the performance community. 

This month's Web Performance Digest features 20 new links. Find out what went down at Velocity 2012, including product unveilings, memorable presentations, and an exciting announcement from us here at Strangeloop. Elsewhere, get user experience tips, read case studies from the performance trenches, and learn the 10 golden rules by which all third-party providers should abide.

New links include:

10 Golden Rules for 3rd Party Providers [blog post]
Catchpoint – June 27, 2012
Summary: Murphy's Law reigned supreme throughout June, with a flood of large-scale outages taking down some of the world’s most popular websites. Given the inevitability of online failures, third-party providers must be prepared to deal with the worst. Find out the 10 Golden Rules by which all third-party providers should live by.

Browser Speed Tests: Chrome 19, Firefox 13, Internet Explorer 9, and Opera 12 [article]
Lifehacker – June 12, 2012
Summary: It’s a battle of startup speed, tab loading times and other KPIs between the four titans of Windows browsing. Lifehacker’s speed tests are always entertaining for what they’re not afraid to say, and this article is no exception.

How complex systems fail [research paper]
CTALab.org – June 26, 2012
Summary: As a complex and interdependent system, the web is prone to catastrophe at the highest levels. In this fascinating paper on resilience engineering, Dr. Richard Cook outlines the reasons why all complex systems are intrinsically hazardous, why disaster is always just around the corner, and how failure-free operations require experience with - you guessed it - failure.

Ghosts of Velocities Past: 9 presentations that are still relevant today [blog post]
Web Performance Today – June 20, 2012
Summary: Velocity’s short (yet very important) history is filled with memorable moments, and these 9 presentations from past conferences remain relevant today. Perhaps not trendsetting anymore, but certainly trend-affirming, which may just be better.

How to Make Progress Bars Feel Faster to Users [video]
UXMovement – June 4, 2012
Summary: The human perception of time is anything but linear, and with just minor visual tricks, it gets even more skewed. After reading this post, you’ll never trust a progress bar again.

Check out the rest of the Hub and see hundreds more links to the best selection of performance-related resources on the web. (Have we missed something? Send us your link recommendations!)

Three free new performance measurement tools for managing mobile and third-party content

The annual Velocity Web Performance and Operations Conference is always a great time to discover new technology, and this year was no different. Here are three interesting (and free) tools revealed at last week's conference that we at Strangeloop are already big fans of:

3PO

A handy bookmarklet from Stoyan Stefanov, 3PO checks pages for integration with third-party content such as analytics and social sharing buttons. The tool installs by simply dragging it to your bookmarks, and it launches with just one click.

After a quick analysis, 3PO generates a report for each page – complete with letter grade – that gives tips on how to fix common performance issues. This tool is still in its infancy. Stoyan plans to add additional checks to this tool, but it’s already very useful and has the potential to become much more so. 

SPOF-O-Matic

SPOF-O-Matic is a fast and easy way to test your site’s vulnerability to third-party outages. This third-party measurement tool from Patrick Meenan is a Chrome extension that detects likely third-party single points of failure (SPOFs) while browsing and allows site owners to simulate how a page will perform if third-party resources are unavailable. 

One of the interesting features of this tool is its ability to show visible alerts any time a user is on a page with potential third-party issues. With this feature, users can quickly discover that the vast majority of web pages are prone to third-party interference.

WebPagetest for mobile

Another tool from Pat Meenan, WebPagetest now lets users test on two mobile devices, iPhone 4 (iOS 5.1) and Nexus S (Android 2.3), from the Dulles, VA, location, thanks to Akamai's recently open-sourced Mobitest agents. The test agents are also available for people to use for private instances. Given the difficulty of creating tests that generate reliable mobile results, this new feature will surely get plenty of use. 

We at Strangeloop salute the many hardworking and creative developers who so generously make these tools available for free. Their fearless experimentation with new features and platforms is leading the change in our industry.