Four important performance measurement terms explained (so normal people can understand them)

Response time. Load time. Start render time. Time to first byte. We encounter these terms every day, but sometimes it's hard to get a bead on what they really mean. That's because, depending on which measurement tools you're using, these terms can have completely different meanings. 

The result of this confusion: Site owners can make the mistake of believing their sites are much faster than they actually are.

To cut through the noise, here's a plain-language guide to understanding these terms (based on this popular post by our president, Joshua Bixby), why they're important, and some important caveats for using them. At the end of this post, we'll explain how to get a reliable, ground-level look at how your site actually performs for real visitors.

Response time

What it means: Response time is incredibly tricky, and it causes a lot of confusion. It can refer to any number of things, depending on whom you ask: server-side response time, end-user response time, HTML response time, time to last byte with no bandwidth/latency, and on and on. Long story short: There’s no single definition.

Caveats: If someone starts talking to you about response time, ask them to clarify which response time they mean. Be wary of anyone who tries to sell you on the idea that there’s only one definition. If user experience matters to you, ask how whatever type of response time you’re looking at relates to what the end user actually sees.

When it’s useful: Different types of response time measurements tell you different things, from the health of your back end to when content starts to populate the browser. You need to know what you’re measuring and why.

Time to first byte

What it means: Time to first byte is measured from the time the request is made to the host server to the time the first byte of the response is received by the browser.

Caveats: Time to first byte doesn’t really mean anything when it comes to understanding the user experience, because the user still isn’t seeing anything in the browser.

When it’s useful: For detecting back-end problems. If your website’s time to first byte is more than 100 milliseconds or so, it means you have back-end issues that need to be examined. (Web performance consultant Andrew King has written a good post about this, as has Google performance expert Pat Meenan.)

Start render

What it means: As its name suggests, “start render” indicates when content begins to display in the user’s browser. This term seems to have evolved as an alternative to “end-user response time”, but it’s not yet widely used outside of hardcore performance circles.

Caveats: Doesn’t indicate whether the first content to populate the browser is useful or important, or simply ads and widgets.

When it’s useful: When measuring large batches of pages, or performance of the same page over time, it’s good to keep an eye on this number. Ideally, visitors should start seeing usable content within 2 seconds. If your start render times are higher than this, you need to take a closer look.

Load time

What it means: The time it takes for all page resources to render in the browser — from those you can see, such as text and images, to those you can’t, such as third-party analytics scripts. (Geek version: “Load time” is also known as “document complete time” or “onLoad time”. It’s measured when the browser fires something called an “onLoad event” after all the page resources have fully loaded. No matter what you call it, it’s used as a primary measuring stick for site performance.)

Caveats: Needs to be taken with a grain of salt, because it isn’t an indicator of when a site begins to be interactive. A site with a load time of 10 seconds can be almost fully interactive in the first 5 seconds. That’s because load time can be inflated by third-party scripts, such as analytics, which users can’t even see.

When it’s useful: Load time is handy when measuring and analyzing large batches of websites, because it can give you a sense of larger performance trends.

How to get a real-world look at how your site loads

Numbers in a spreadsheet are a good way to spot larger patterns and trends, but if you want to get a ground-zero look at your site’s performance, capturing videos and filmstrip views of your pages' load times are one of the best ways to go.

WebPagetest is an excellent (and free) online tool that lets you do this. Simply enter the URL of the page you wish to test, click the "Video" tab, and select "Capture Video". Your test results will include an excellent video showing how your site loads in realtime, as well as a filmstrip view of the same.

Questions about web performance?

We have the answers. Take a moment to book your phone meeting with one of our Performance Experts. 

Valentine's Day Online: How, when and where people are spending [INFOGRAPHICS]

1 out of 3 North Americans love to spend online, but slow sites equal lost sales. The average Valentine's Day e-commerce site takes a whopping 21.5 seconds to load -- driving away the 57% of shoppers who say they'll wait just 3 seconds before they bounce.

Click through to see our complete set of Valentine's Day e-commerce infographics and get a big-picture look at how much people are spending, what they're buying, and whom they're buying for on Valentine's Day.

Find out:

• Are we spending more on our partners... or on our pets?
• If condom sales peak in February, sales of what other product peak in March?
• What percentage of women would kick their boyfriends to the curb if they didn't get a Valentine's Day gift?
  
  

Enjoy -- and please feel free to share on your own site! (Get a high-res version here.) 

Talk to a Performance Expert

Slow web pages are a year-round concern -- one that doesn't just affect florists and candy stores. Do you have questions about your website's performance, or performance in general? We have answers. Talk to us today.

Hooman Beheshti, Strangeloop VP Technology, to moderate Cloud Performance Summit

On February 13, Hooman Beheshti will be co-moderating the Cloud Performance Summit, a special one-day event at Cloud Connect 2012. Hooman will be joined by fellow moderator Alistair Croll in leading an all-day panel that features speakers from Google, F5, Keynote, Akamai, and other performance industry leaders. 

Summit topics will focus on important current issues, challenges, and opportunities around performance in on-demand environments. 

In addition to moderating the Summit, Hooman will be presenting a session entitled "Application Acceleration Through Front-End Optimization".

From the session description:

Accelerating applications can mean different things to different people. In web applications, performance is impacted by everything from infrastructure to code to back-end processing to browser capabilities. This can get even more complicated in cloud environments. In this discussion, we'll focus on the issues surrounding the "front-end" performance of the application which includes all interactions between the browser and the app after the dynamic content (the base HTML) has been generated and delivered to the browser. We will discuss the major front-end performance pain points and some strategies for mitigating them (including hidden complications and gotchas), ultimately leading to a better perceived user experience.

UPDATE: See the slides from Hooman's session: Front End Optimization: what it is and how to fix it

Website Performance and the Modern Browser: It's complicated

Browser performance was one of the most talked-about aspects of our recent State of the Union report on page speed and web performance. In our survey of the Alexa-ranked top 2,000 e-commerce sites, we found that pages loaded 29% faster in Internet Explorer 9 than Internet Explorer 7. IE9 and Firefox 7 were each about 5% faster than Chrome. 

Browser speed is a competitive -- and contentious! -- issue, and we were not surprised to see that this finding sparked a lot of excellent debate and discussion online.

From our report, this was our take on these findings:

In the past year, speed has emerged as a highly competitive issue in browser development. Every major browser now markets speed as a key feature, from Chrome's self-described "lightning speed" to Internet Explorer 9's slogan "Fast is beautiful." We cannot claim that this study definitively answers the question of which browser offers the best performance, but we do feel that this sample size is significant enough to merit including these findings in the ongoing debate. It is encouraging to see that browser developers appear to take speed seriously. All indicators point to the fact that speed will remain a top priority.

In our tests, we simulated (using WebPagetest, an excellent third-party tool) how fast each site loads for a real user who is ostensibly viewing just that one primarily HTML-based site. What we were not able to test for:

1. Browser performance under stress from having multiple tabs open simultaneously.
2. Browser performance degradation over time (i.e. the longer the browser remains open, its likelihood of crashing).
3. Browser performance when visiting sites that use HTML5 or Flash, or when watching videos.
4. Usability. (This gets tricky, as it can often boil down to personal preference. Some people like the sleekness of Chrome, whereas others prefer Firefox's many add-ons.)
   
   

As with performance in general, browser performance is nuanced and cannot be summed up simply. As we stated in our report, our numbers are just one part of the bigger picture.

Download the free report: 2012 Annual State of the Union: E-Commerce Page Speed and Website Performance