Four important performance measurement terms explained (so normal people can understand them)

Response time. Load time. Start render time. Time to first byte. We encounter these terms every day, but sometimes it's hard to get a bead on what they really mean. That's because, depending on which measurement tools you're using, these terms can have completely different meanings. 

The result of this confusion: Site owners can make the mistake of believing their sites are much faster than they actually are.

To cut through the noise, here's a plain-language guide to understanding these terms (based on this popular post by our president, Joshua Bixby), why they're important, and some important caveats for using them. At the end of this post, we'll explain how to get a reliable, ground-level look at how your site actually performs for real visitors.

Response time

What it means: Response time is incredibly tricky, and it causes a lot of confusion. It can refer to any number of things, depending on whom you ask: server-side response time, end-user response time, HTML response time, time to last byte with no bandwidth/latency, and on and on. Long story short: There’s no single definition.

Caveats: If someone starts talking to you about response time, ask them to clarify which response time they mean. Be wary of anyone who tries to sell you on the idea that there’s only one definition. If user experience matters to you, ask how whatever type of response time you’re looking at relates to what the end user actually sees.

When it’s useful: Different types of response time measurements tell you different things, from the health of your back end to when content starts to populate the browser. You need to know what you’re measuring and why.

Time to first byte

What it means: Time to first byte is measured from the time the request is made to the host server to the time the first byte of the response is received by the browser.

Caveats: Time to first byte doesn’t really mean anything when it comes to understanding the user experience, because the user still isn’t seeing anything in the browser.

When it’s useful: For detecting back-end problems. If your website’s time to first byte is more than 100 milliseconds or so, it means you have back-end issues that need to be examined. (Web performance consultant Andrew King has written a good post about this, as has Google performance expert Pat Meenan.)

Start render

What it means: As its name suggests, “start render” indicates when content begins to display in the user’s browser. This term seems to have evolved as an alternative to “end-user response time”, but it’s not yet widely used outside of hardcore performance circles.

Caveats: Doesn’t indicate whether the first content to populate the browser is useful or important, or simply ads and widgets.

When it’s useful: When measuring large batches of pages, or performance of the same page over time, it’s good to keep an eye on this number. Ideally, visitors should start seeing usable content within 2 seconds. If your start render times are higher than this, you need to take a closer look.

Load time

What it means: The time it takes for all page resources to render in the browser — from those you can see, such as text and images, to those you can’t, such as third-party analytics scripts. (Geek version: “Load time” is also known as “document complete time” or “onLoad time”. It’s measured when the browser fires something called an “onLoad event” after all the page resources have fully loaded. No matter what you call it, it’s used as a primary measuring stick for site performance.)

Caveats: Needs to be taken with a grain of salt, because it isn’t an indicator of when a site begins to be interactive. A site with a load time of 10 seconds can be almost fully interactive in the first 5 seconds. That’s because load time can be inflated by third-party scripts, such as analytics, which users can’t even see.

When it’s useful: Load time is handy when measuring and analyzing large batches of websites, because it can give you a sense of larger performance trends.

How to get a real-world look at how your site loads

Numbers in a spreadsheet are a good way to spot larger patterns and trends, but if you want to get a ground-zero look at your site’s performance, capturing videos and filmstrip views of your pages' load times are one of the best ways to go.

WebPagetest is an excellent (and free) online tool that lets you do this. Simply enter the URL of the page you wish to test, click the "Video" tab, and select "Capture Video". Your test results will include an excellent video showing how your site loads in realtime, as well as a filmstrip view of the same.

Questions about web performance?

We have the answers. Take a moment to book your phone meeting with one of our Performance Experts. 

Valentine's Day Online: How, when and where people are spending [INFOGRAPHICS]

1 out of 3 North Americans love to spend online, but slow sites equal lost sales. The average Valentine's Day e-commerce site takes a whopping 21.5 seconds to load -- driving away the 57% of shoppers who say they'll wait just 3 seconds before they bounce.

Click through to see our complete set of Valentine's Day e-commerce infographics and get a big-picture look at how much people are spending, what they're buying, and whom they're buying for on Valentine's Day.

Find out:

• Are we spending more on our partners... or on our pets?
• If condom sales peak in February, sales of what other product peak in March?
• What percentage of women would kick their boyfriends to the curb if they didn't get a Valentine's Day gift?
  
  

Enjoy -- and please feel free to share on your own site! (Get a high-res version here.) 

Talk to a Performance Expert

Slow web pages are a year-round concern -- one that doesn't just affect florists and candy stores. Do you have questions about your website's performance, or performance in general? We have answers. Talk to us today.

Hooman Beheshti, Strangeloop VP Technology, to moderate Cloud Performance Summit

On February 13, Hooman Beheshti will be co-moderating the Cloud Performance Summit, a special one-day event at Cloud Connect 2012. Hooman will be joined by fellow moderator Alistair Croll in leading an all-day panel that features speakers from Google, F5, Keynote, Akamai, and other performance industry leaders. 

Summit topics will focus on important current issues, challenges, and opportunities around performance in on-demand environments. 

In addition to moderating the Summit, Hooman will be presenting a session entitled "Application Acceleration Through Front-End Optimization".

From the session description:

Accelerating applications can mean different things to different people. In web applications, performance is impacted by everything from infrastructure to code to back-end processing to browser capabilities. This can get even more complicated in cloud environments. In this discussion, we'll focus on the issues surrounding the "front-end" performance of the application which includes all interactions between the browser and the app after the dynamic content (the base HTML) has been generated and delivered to the browser. We will discuss the major front-end performance pain points and some strategies for mitigating them (including hidden complications and gotchas), ultimately leading to a better perceived user experience.

UPDATE: See the slides from Hooman's session: Front End Optimization: what it is and how to fix it

Website Performance and the Modern Browser: It's complicated

Browser performance was one of the most talked-about aspects of our recent State of the Union report on page speed and web performance. In our survey of the Alexa-ranked top 2,000 e-commerce sites, we found that pages loaded 29% faster in Internet Explorer 9 than Internet Explorer 7. IE9 and Firefox 7 were each about 5% faster than Chrome. 

Browser speed is a competitive -- and contentious! -- issue, and we were not surprised to see that this finding sparked a lot of excellent debate and discussion online.

From our report, this was our take on these findings:

In the past year, speed has emerged as a highly competitive issue in browser development. Every major browser now markets speed as a key feature, from Chrome's self-described "lightning speed" to Internet Explorer 9's slogan "Fast is beautiful." We cannot claim that this study definitively answers the question of which browser offers the best performance, but we do feel that this sample size is significant enough to merit including these findings in the ongoing debate. It is encouraging to see that browser developers appear to take speed seriously. All indicators point to the fact that speed will remain a top priority.

In our tests, we simulated (using WebPagetest, an excellent third-party tool) how fast each site loads for a real user who is ostensibly viewing just that one primarily HTML-based site. What we were not able to test for:

1. Browser performance under stress from having multiple tabs open simultaneously.
2. Browser performance degradation over time (i.e. the longer the browser remains open, its likelihood of crashing).
3. Browser performance when visiting sites that use HTML5 or Flash, or when watching videos.
4. Usability. (This gets tricky, as it can often boil down to personal preference. Some people like the sleekness of Chrome, whereas others prefer Firefox's many add-ons.)
   
   

As with performance in general, browser performance is nuanced and cannot be summed up simply. As we stated in our report, our numbers are just one part of the bigger picture.

Download the free report: 2012 Annual State of the Union: E-Commerce Page Speed and Website Performance

FREE REPORT: 2012 State of the Union for E-Commerce Page Speed and Website Performance

The average e-commerce website takes 10 seconds to load, web pages are getting bigger, and Internet Explorer 9 outperforms other browsers. These were just a few of the findings of our second annual State of the Union: E-Commerce Page Speed and Website Performance.

Trying to identify performance trends over long periods of time is like aiming at a constantly moving target. This is why we've undertaken an ongoing initiative to benchmark and analyze the performance of the home pages of the same top 2,000 Alexa-ranked retail websites.

For the past two years, we've measured performance criteria such as load time, page composition, and adoption of core optimization best practices. In this report, we look at the newest round of data on its own, and as compared with data from the previous year.

A few key findings:

• The average site is 10% faster now than it was one year ago. While this is encouraging, it is far from the ideal load time of 2 seconds, as identified by decades of research into human-computer interaction.
• Top-ranked sites are slower, not faster, than the rest of the pack. 
• Web pages are getting bigger, year after year. The average home page contains 98 page objects, a 13% increase from last year. (For a bit of context: In 1995, the average page contained just 2.3 objects.)
• Internet Explorer 9 outperformed other browsers.

What does it all mean? Our CEO Jonathan Bixby puts it like this:

"The key takeaway here is that the pursuit of faster websites is a neverending race. As pages continue to grow in size and complexity, many site owners are barely managing to stay ahead. Newer browsers help somewhat – as does using a content delivery network to cache your content closer to your visitors – but only somewhat. Site owners who want to do more than keep their heads above water need to expand their acceleration toolkit."

Download the free report: 2012 Annual State of the Union: E-Commerce Page Speed and Website Performance

Wine.com optimizes site by 45%, reduces page load to 2 seconds

In a recent interview, Strangeloop customer Wine.com told Retail Touch Points why they chose us to accelerate their premiere e-commerce site:

"A four-second download once was acceptable but today is far too slow, given advancements in computer technology and growing consumer expectations," stated Geoffrey Smalling, CTO of Wine.com. "Two seconds was an ambitious goal but we had to reach it or lose potential customers, especially first-time visitors."

Mr. Smalling talked about the challenges of optimizing content-rich pages for every browser type, the complexity of optimizing location-sensitive pages for customers around the world, why the pursuit of high-performing web pages is a neverending task, and how the Strangeloop Site Optimizer addresses all of these challenges. 

"It was an engineering challenge for Strangeloop to develop optimizations based on our unique, regulated business," said Mr. Smalling. "Strangeloop best handled enterprise scale via appliance or cloud and has an aggressive plan for further product development to keep us at the front edge of acceleration."

"The promise of a front-end optimization tool like the one we selected from Strangeloop Networks is that we don’t have to think about it ― the optimization is done for us. That promise has included a 45% improvement in speed on our New York homepage, 39% on Virginia’s and 35% on the Los Angeles page."

Read the full article on Retail Touch Points.

Test your site

Find out why companies like Wine.com, eBay/PayPal, Visa, and Petco consider us the only choice in front-end website acceleration. Send us your URL. We'll analyze your site, then show you how much faster we can make it.

Advanced Mobile Optimization: How does it work, and how do you measure success?

If you're in the San Francisco area, join Strangeloop president Joshua Bixby on Thursday, January 12th, as he speaks at a meeting of the SF/Silicon Valley Web Performance Meetup Group.

Joshua will walk through several advanced mobile optimization techniques and explain how they work around the unique constraints (and opportunities!) of the mobile platform. Joshua will also present data from detailed case studies that show how real-world companies have optimized their mobile sites, and as a result have experienced dramatic improvements in key performance indicators such as page views, conversions, cart size, and revenue.

Joshua's research is based on the development and beta deployment of the Strangeloop Mobile Site Optimizer, the most advanced front-end tool currently available for delivering optimized pages to mobile devices.

The event is free to Meetup group members, but spaces are limited and filling up fast. Go to the event page to learn more.

2012 predictions: The average web page will hit 1 MB, Google and Siri will face off, and Chrome, Windows 7, and RUM will rise

Wondering who's going to pull ahead in the browser races? Who's the new dark horse in the mobile industry? How will the CDN and RUM industries evolve? Strangeloop president Joshua Bixby puts on his wizard cap and makes a handful of predictions in his latest blog post, starting with prediction #1:

The average web page will surpass 1 MB in size.

Between December 2010 and now, the average web page grew from 716 KB to 965 KB, according to the HTTP Archive. That’s 30% growth in slightly less than one year. This kind of growth is the norm, as pages have grown at a rapid rate since 1995, when the average page size was just 14.1 KB. It’s pretty safe to assume that this growth will continue. We’re going to see sites grow by at least another 30%, taking them well over the 1 MB mark — a number that would have blown our minds 10 years ago. The main culprits: images (which account for more than half of the average page size) and third-party scripts like analytics, ads, and social sharing widgets.

This prediction has already been picked up by GigaOM, BBC News, and TechVibes.

Check out the rest of Joshua's predictions on his blog, Web Performance Today.

You can't get away with a bad mobile experience anymore

Mac Slocum of O'Reilly Radar recently interviewed Strangeloop president Joshua Bixby, and they talked about common mobile optimization mistakes, mobile apps vs mobile sites, whether or not the Kindle Fire will revolutionize front-end optimization, and why the 2011 holiday season caught online retailers with their mobile pants down.

MAC: Should companies be thinking "mobile first" now?

JOSHUA: We talk about "the web" and "the mobile web" as if the two are different, but they aren't. I'm the first to admit that I'm as guilty of doing this as the next person. Using these terms is helpful for discussing differences in how people browse via different devices, but at the end of the day, it's all one web. Users want the same breadth and depth of content, no matter what device they're using. They want a consistent, reliable user experience. They don't want to interact with your site one way at their desks, then learn a whole new way when they're tablet-surfing on the couch, and then learn a third way when they're roaming around with their phones. Site owners who can deliver an experience that feels the same, regardless of the platform, are the ones who are going to own the web of the not-too-distant future.

MAC: Will all phones be smartphones at some point? Or will non-smartphones remain an important segment for years to come?

JOSHUA: It really depends on which market you look at. Last summer, comScore published a survey that showed that, despite the rapid rate of smartphone adoption, 155 million American mobile phone users still don't have smartphones. That's obviously a pretty significant number. But if you take a global view, things flip around and we see that, especially in developing countries, new mobile users are jumping right on the smartphone wagon. This allows users to bypass both dumb phones and the desktop web. It's a whole different way of interacting with the Internet.

I think an even better question might be: How great a disruptor will tablets be to the mobile market? Site owners have been caught with their mobile pants down, so to speak, in their inability to recognize the importance of this market. Forrester did a survey of retailers and found that, on average, about 20% of holiday mobile traffic came via tablets, with some retailers reporting that more than half their mobile traffic came via tablets. But most "mobile-optimized" sites look terrible on tablets. In 2012, site owners are going to be scrambling to catch up with this paradigm shift.

Read the rest of the interview on O'Reilly Radar.

CDN Solutions and Strangeloop team up to bring advanced FEO to Japan

We are very excited to announce our newest partnership, this time with Japan's leading performance solutions provider, CDN Solutions. 

CDN Solutions' expertise is in providing content delivery and traffic management services for web content, streaming media, and web-based applications. This is a perfect marriage with our ability to optimize web content itself. Effective immediately, CDN Solutions will be bundling our Site Optimizer service, which optimizes individual page code so that it can render up to three times faster in a visitor’s browser, with its existing content delivery network (CDN) service.

"Partnering with Strangeloop is a great opportunity to introduce a new web performance solution to our market, and allows us to add a powerful new complementary solution to our current service offerings," said Kazunori Higashiguchi, Director of Sales, CDN Solutions. "Companies come to us to make their sites faster, and today, front-end optimization presents the greatest area where we can deliver significant new acceleration gains. Strangeloop’s Site Optimizer is the most proven FEO solution on the market, allowing us to provide fast ROI for our customers on improved website and application performance."

CDN Solutions joins our growing list of top-tier partners, which includes Level 3, Akamai, Neustar, Exceda, and Seven Group.

• Read the full press release.
• Learn more about our partner program.