Performance Impact

Performance Impact, Part Two: More findings from the front lines of web acceleration

In this presentation at the Velocity 2010 conference, Strangeloop president Joshua Bixby discusses research findings on the relationship between website performance and key business metrics.

 

Video Transcript

Joshua Bixby: It’s a pleasure to be here. Thank you for attending. I know there is a lot of competition. My name is Joshua Bixby, I’m the president of Strangeloop. We are a company that accelerates websites, that’s our business. I’m not here to talk about what we do specifically but I’m here to talk about the outcomes of acceleration, particularly to focus on the relationship between performance and key business metrics for normal companies. There is been so much research that’s talked about Yahoo and Google and Amazon and eBay and I want to share some results of companies that aren’t like that, that might actually be more like you.

We’re really here to talk today about money. That’s what drives all of this, Tim talked about in his speech about how performance and the key KPIs drive business, you know, I liked his discussion about heroes and innovators. We’re here to talk about money but we’re actually here to talk about something slightly different. We’re here to talk about money and the fact that we have to make decisions about it. And really what I want to, what I want to, everyone to take away from this, if they can, is how does performance change your priorities? What is your performance curve? Is performance the number one priority that you have in your business? And most importantly, when you make other decisions, is performance a factor?

I want to tell you a quick story. My son, I have a four-year old son and I’ve got, almost four, August 1st and I’ve got a – my other son is seven weeks old. So, I want to do a shout out to my wife for being a local hero given that I’ve been here all week. I was telling my son, he’s like “Dad, I don’t want you to go down to San Francisco” or “Franfisco,” I think is how he said that, I said “I’m sorry, I have to go” and he said, “why?” And I said “because I’m going to go talk to people about performance, it matters.” And he said, “look at me!” and so we ran from where we were, we were just walking out of the house, straight down the stairs and right to the car and he looked around and he said, Dad, I’m fast, it’s better.

So, it’s like, that’s really cute. You get it and I’m going to go talk to a whole bunch of people who get it and whose businesses might or might not get it. I want to start talking about the history of this really briefly. I want to spend a lot of time on the actual results. So, one of the questions when we think of our performance curve, because what we want to talk about is how do we get good enough performance? Is there such a thing as good enough performance? In, you know, Alistair Croll and Hooman Beheshti did this, the first part of this, as a webinar with Watching Websites Group. I can’t come close to doing as good a job as them, so I apologize in advance. Zoner Research came out in 1999 with a bold statement about what good enough was and they came and told us that it was 8 seconds.

Some of you might remember the 8 seconds rule. Now, it wasn’t just 8 seconds, it was 8 seconds times a whole bunch of factors that if you wanted to go try to sell to your business executives and/or you are a business executive, you say, you know, it’s 8 seconds for us but or you just have a lot of tenacity, you know, we are very tenacious users. It’s we have – we have users that have a very high natural patience. Well, this just doesn’t going to work. This research has obviously evolved. We now get into things that I like to call the sort of Barbie standard. Forrester came out and, you know, we have a good relationship with Forrester, so certainly not to mock him but Forrester came out with a report in conjunction with Akamai, the new standard is now 2 seconds. If your site isn’t, you know, is doesn’t render in 2 seconds, you’re failing your users, you’re losing money. All of these standards are good ways for us to simplify what’s actually an incredibly complex problem.

What we want to do today is share some of the results from increased performance and again what we know from performance and we did a, we did a session yesterday that some of you would have attended because I see some familiar faces where we talked about the fact that some of these performance enhancements like turning on compression, doing keep alives is simple. This is, should be a check box. Some of these, some of the things that get talked about in the context of performance are much, much more complicated and take much more effort. So, again, it’s this question of cost benefit and we go from easy to hard and organizations have a very difficult time, we’ve found, because we’re constantly talking to them about where to invest, how much to invest and where benefit is going to come from.

There are two people, we have the honor of talking to a lot of organizations. There are two people that we talked to. On the right hand side, we talk to the IT guy. I love these stereotypes. I had to bring them back as Alistair’s at home awaiting his newest child which should be coming in a few weeks and he couldn’t be here but in honor of him, we’ve got the IT person on the right who, you know, thinks of different things. We’ve got the businessperson or the marketer who’s thinking about different things. Rarely do they meet, rarely do they find something where the energy is going in the right direction, they both find themselves really thinking and acting on the same metric. I want to talk about how KPIs change this in a second, but first I want to talk about what we know: total, absolute, gratuitous flog of my blog, Web Performance Today, where we are trying to accumulate all of the resources, basically a cheat sheet on what’s going on.

I saw Tim do it. I rushed back up to the hotel room, added pictures of my blog because I figure if he can do it, I can do it. I want to talk about the research that’s been done to date and I want to show you this, again I’m not the best person to interpret some of the analysis that’s come out from Google and Bing from last year, but I want to set a level playing field, some of you might know some of this, some of you might not, but I want you to see it. I want you to understand what has been, what has been qualified or quantified in the industry.

First thing I want to do is I want to present it in the form of videos. Patrick, who I see is here, the – continues to get shout outs. I understand he doesn’t drink so in one session it was get him beers. So, don’t get Patrick beers, buy him food or luxury items. If you want to thank him for all of his great work over Webpage test, what we’ve done is we’ve tried to simulate some of the tests that happened. The first one we’re going to do is Google and I want to show you what 400 milliseconds feels like. Let’s take a look. So, if you went to your executive team and you said look at the amazing work I did, 400 milliseconds, we’re going to make more money, you’re probably going to get that’s not faster. Let’s look at the results. If you’re Google and I’m sure some of you are in here and you can make a 0.5 or 0.6 or a 0.7 percent difference, impact on the average daily searches per user, this is a big deal.

So, one of the things that we find when we talk about this space is that, is that actual numbers, real numbers are really powerful. Let’s take another example. Bing came out with – at the same time actually a joint session last year where they talked about, in conjunction with Google, some of the same numbers. Let’s take a look at what 2 seconds feels like. Okay, the grays, don’t think about the grays, just think about when you first started seeing the page. Okay, I get it. That’s 2 seconds. The impact is huge. What’s interesting as we go through this is the – and in all of these slides I find this fascinating, the stats that come out reflect the key KPIs that are important. What’s interesting when we look at the Bing stats is we’re not just looking in this case about user searches per day, 2 seconds, 200 milliseconds creates a huge change in the number of key business metrics, from satisfaction to revenue per visit, really, really interesting data, 2 seconds.

You can also see the impact of one second or 500 milliseconds or 200 milliseconds only one analysis that was conducted in this particular survey where AB testing showed no impact and that was 50 milliseconds. Let’s keep going, Shopzilla, anyone here from Shopzilla? I’d like to congratulate all of the Shopzilla people because I just think they put out some of the best data out there in this space and as you’ll see when we talk about our customers and the customers we encourage to put out data, that’s a hard thing for companies to justify doing. Shopzilla, they got 5 seconds. Let’s take a look at that. Okay, that’s, that’s pretty dramatic, that’s incredibly impressive.

If you look at what went into it, we’re talking about a 16-month engineering project and I love this graph. Any graph that shows like this is just a dream. Let’s talk about what was important. What was important was a 5 to 12 percent increase in revenue. That’s what was important. That’s the money conversation. The cost conversation comes from what was 16 months of engineering, how many people, how much effort, was it worth it? We got to think of that too but what would you do for a 5 to 12 percent increase in your revenue? Most companies would do a lot. Let’s keep going. AOL released some data as well. I don’t have a video for this because of the way the data was presented to us. All we know is percentiles and page views but what’s fascinating about this stat is what they are measuring.

What’s important to a lot of Google properties unlike users, you know, searches per day, we’re talking here about page views. We don’t know what the bottom axis is, we don’t know if that’s going from zero seconds to 10000 and what the percentiles are. What we do know is how important performance is. The difference between eight seconds, eight pages per visit and three pages per visit is huge. It’s the difference between those that are successful in the industry and those that aren’t. Let’s keep going. Firefox did this really cool study that came out after Velocity last year, certainly it wasn’t presented here, where they tweaked their homepage, that page where you download Firefox and they saw what happened. Take a look at those stats. And anything that looks like a slide that you’d look through a microscope just makes me very excited, it looks so technical.

Average page loads from five seconds here down to 2.8, decent sample size, conversion, better conversion, more downloads of Firefox, more Firefox. Let’s keep going. I want to make sure you all have the same background as we do when we go into the real numbers here. About, anyone see this today, this came out today or yesterday, this is right off the press, very excited about the people who brought this to my attention. About.com undertook in conjunction with the Google team an acceleration project. They did a number of things, a number of the things that get talked about a lot in this industry from resource in, you know, combination to cashing content, things you’d find on Page Speed, YSlow. I want to highlight one down here. Lot of talk about Third-party, Third-party is this, Third-party is that, what I really like about this initiative was the initiative that this study took to go back to the ad industry to take performance seriously and we’re really encouraging that within our customer base and with the prospects we talk about.

There are some things you can’t control directly but as influencers when you are a large customer or a group of medium size customers, go back to the vendor and they hear that they might lose your business, you might find somebody else, they act. What happened? They did AB testing obviously over, this is, this is new and it has a limited, we know very little about it, just what was released in the press release. Performance went up on average by 19 percent. And we’re going to come back and talk in the section where we actually revealed some of the data we’ve been talking about. These numbers around performance are, we like to simplify so 19 percent and then what does that mean? In this case it meant 0.9 percent increase in revenue per 1000 pages.

For a network of, like, about that has 700 sites that’s constantly driving every, you know, squeezing every inch out, this is very important, very, very important. Again, all of these organizations are sharing their data. I’ll get back to that. All of them are seeing positive correlation. We’ve heard it before and I know we all like to believe it and we want to believe it but we’re seeing study after study which correlates this difference. Now, one thing that was mentioned today in the keynotes that also is very, very important. It came out since Velocity was the fact that our rankings are now influenced by performance. There is very little data about this, Google has made available a great, great tools to understand how to make your site faster but the alchemy of the algorithm remains a closely guarded state secret.

What we do know or what’s been made public is that it’s one of a number of different variables, I’ve heard anywhere from, you know, 180 to 250 but I’m certainly no expert, and we’re dying for results. So, if you’re seeing results since this announcement was made, we’d love you to share them with us. We have one example of, of the companies we’re going to talk about that are “model companies,” companies that don’t have, that can’t raise armies of developers or where that’s a very expensive endeavor, where we did see some results about search but it wasn’t directly related to using site speed as a factor. Notice the trend, the trend in this is that all of the data, all of the public data comes from the giants of the industry and I get the opportunity, the wonderful opportunity, and I’m very grateful for it, to have a chance to speak either in person or on the phone with dozens of customers or prospective customers a day where we’re pitching our story of we’re going to make you faster automatically.

And, you know, I ream out these stats just every time, any time you’re on a conversation with me, I’ll say do you know what eBay did? Have you heard about the Amazon stats? And I had a CTO stopped me and he said I don’t really care what they do, it’s not me, doesn’t impact me. I’m different. I don’t have the visitors, the same, I don’t have the same number of visitors. I don’t know if my visitors act the same way. I just don’t know, so what we’re here really to speak about is, what are the stats when we look at mortal companies, when we look at companies that aren’t in the top 20? What do they look like? We put out an initial set of slides about six months ago where we did show graphs and we did them anonymously because it’s pretty difficult and continues to be difficult to get our customers to share.

One of the things I promised Steven Jesse when I submitted this talk and was graciously accepted was we’re going to tell you who they are, we’re going to tell you what we did as much as we can and we’re going to show you the impact. One of the things that we see people agree on, and you’ve noticed this is in honor of Alistair, they’ve switched, this gentleman is now looking at her – is KPIs unite our teams. Parag here, who is the CTO of AutoAnything is going to come up and talk but he was instrumental in us seeing this change. We were in his office in San Diego about a year, year and a half ago and AutoAnything is – Parag is going to come tell us about what they do and I’ll butcher it, I’m sure, but the way I like to think about it is they are everything for your car, replacement parts, accessories.

We were in talking to Parag and we were talking about, you know, 2 X Parag, look we’re 2 X. Payload is this, round trip production is that and he said, you know, I sit at a management and executive level every week and anytime I’m looking for budget, I have to make a business case. I’m not allowed – our CEO said I’m not allowed to come talk about performance stats. I’m not allowed to come talk about antivirus threat stats. That’s just not interesting. When you tell me that you’re going to speed up my site, you’re like a windshield wiper. It’s nice to have a good windshield wiper, it’s nice to have, but the stock one I got for my car is probably good enough. For me to make a case at the executive level, I need you to be my transmission. I need you to be an engine. I have to talk business and since that talk and he’s here and we’re very proud that he’s here. We’re very proud he’s a customer.

Since that talk our business has changed and we really focus on the KPI side and there is a meeting of the minds there. I have yet to find a technical person, although I know that there is the reputation and the all we care about is numbers, I’ve yet to find one person in this entire industry who when I say you can go talk to your CEO or executive director about making more money, I’ve yet to see someone say that’s not interesting or that’s not compelling or I’m not going to pitch it that way. I haven’t seen it yet. There is a myth that all we care about is speeds and feeds and making things faster and that’s just not true. It’s just not true. There is a meeting of the minds here around KPIs. I want to talk about the process that we’re going to go through and the experimentation, but first I want to talk about just general flow, this is, you know, people come to your site.

Once they’ve come to your site, they are going to bounce or you’re going to keep ‘em and they are going to become new visitors, you’re going to grow or good repeat visitors. They are going to look at lots of pages. They are going to spend lots of time on your site and then it’s going to translate into a KPI, and one of the key areas that’s really important is we go through these KPIs, as you saw from the KPIs that have been presented before, you don’t fit necessarily into every single one of these categories. Some of our customers really care about search. It’s all about search. We have one customer today who makes that is their metric. Natural search is what they care about.

We have some customers where it’s all about ad clicks. That’s their metric, that’s where they make money. Other people come to us and its about user satisfaction or user productivity, maybe its an internal app and obviously, I’m going to spend a lot of time today talking about the customer that cares about conversion and cart size as you put those two together and you bump up revenue. Let’s talk about the experiment, here is what we did. Visitors came in from a whole number of, visitors came in from the wild. They hit Strangeloop, let me just digress very quickly and tell you, Strangeloop has a virtual appliance, a physical appliance and we recently announced a service. What’s important about what we do is we sit in the traffic flow, request comes in, we see it, response goes out, we change it. That’s our business attempting to improve performance, most importantly improve KPIs. So, we see requests coming in. We’re not on the server. We see the server generate the HTML and the content that it normally would. That normal content comes out. We insert a segmentation marker. We insert a cookie. You’re accelerated, you’re unaccelerated.

We then make a decision based on that segmentation marker. We accelerate that page. We make it faster. We rewrite that HTML to be just as optimized as we possibly can or we send it unaccelerated and no matter which option is chosen, again, randomly, once you’re in a group, you’re going to stay there. I believe for most of these experiments it’s three months. The page is received, the script is processed and analytics, not our analytics, although we have a component here, but the customers’ analytics, Google analytics, or Omniture or Web Trends or whatever tool you have is going to be shipped this information, so you can segment every key KPI that you manage based on which group you’re in.

In all the studies today we did 50/50 splits which again when you look at most of the studies that are done by the large organizations or you know you’re taking 0.01 percent of traffic, we have some brave souls. When we talk about analytics, this is the kind of graph you’d see in Google analytics. Now, obviously as a revenue graph, you know, our customers are generous but they are not going to share this. What you’d see if you pulled up your revenue graph is you’d see a line which would be Strangeloop accelerated and you would see a line which would be not accelerated. And what I hope to do for any business is to just send this picture without the blurs to a CEO and start collecting my money. And if you do your own acceleration in house, you should be able to provide and present something like this and start collecting more money for your budget, because this shows me if you’re thinking any of these intervals in any of these days that one line’s above the other and when you look at a graph like this that’s all good as long as it’s the right line.

And frankly, if it’s the wrong line, you probably learn something as well. Now, not only do we send data through to the customers’ analytics but we also collect some ourselves. We radio back to our data warehouse for customers that opt into this service and I’m going to talk a lot more about that later, performance data, because it’s very difficult to send a performance data, the raw performance data too, a lot of the tools that are out there today. We hope we can get out of this business because that’s actually not – it’s not interesting to us but today to send performance data and really analyze it in something like Google analytics or Web Trends is really, really challenging.

Let’s talk about acceleration. When I talked to Steve about this, the question was, I need to know what you did. So, I said great, I’ll show you what we did. Some of you might have seen this graph before. This was presented in a session about how to win the war in Afghanistan and was picked up by the New York Times as why PowerPoint is a failure and why the military doesn’t know what they are doing. I want to focus on the fact that I started to put together a presentation that showed all the interactions around acceleration and decided I did not want my slide to be showed up as how PowerPoint is dysfunctional. The reality is let’s take a look at one the datasets and I want to explain to you why this is complicated.

One of the datasets has over 500 million pages. One of the datasets has a 120, a 130 000 individual pages, 52 000 of those are landing pages where somebody started their flow through the site. We’re talking about 10 major browsers, 110 versions and nine page views per customer. So, when I come and tell you we sped up this site by 2 X what does that mean, on what – for who? Across the flow on the home page, what does acceleration mean? This is a really challenging question. To be frank as vendors in this phase, we try to dumb it down to one number because we think it’s easier for you to digest. We are trying to mature and explain to you that this process actually is really complicated. You want to treat a landing page differently than you want to treat a page that you think people don’t land on. You want to treat things different and you want to, you know, the browsers are different. This is a very complicated process.

I want to just talk to you about what acceleration means. What acceleration means to us is we have a number of inputs. One of the inputs obviously is the URL, we need to know what page it is, www.example.com. We need to know what browser you’re on because if you’re going to do acceleration and you’re going to do it effectively. Browsers are all different. What you’re going to do for an iPhone if you’re going to really optimize, it’s going to be different than what you’re going to do for IE6 et cetera. We then need to know what the user flow is. Have you been to the site before? Is this a landing page for you? Is it not a landing page? Were you back? Who are you? Not only that, but we need to know what other people have done in order to really do optimal acceleration. Who, you know, you are just as likely to follow that nine page flow if someone else, if there are consistencies there, we can take advantage of those trends and we can accelerate even more.

So, what does acceleration mean in the context of what we’re talking about today? Oh, we optimize caching. We’re going to make sure ideally that you never come back for the same thing twice. We’re going to make sure that when you do – when a resource does change, you don’t have stale content. So, when we talk about this bucket of acceleration and we talk about the business benefits across these sites in various degrees depending on where you are in this incredibly complicated flow we’re going to optimize caching. We’re going to also minimize round trips as much as we can. And, again, how you would minimize round trips on a landing page is different than how you would minimize round trips on a page that’s not a landing page. For example, we’re going to minimize the payload size as much as we can across the pages that we’re going to talk about in the sites we’re going to accelerate.

We want to optimize browser rendering. This involves, you know, moving scripts around, putting things at the right place, in the right browser, et cetera. We’re going to optimize images. The image I get in my iPhone doesn’t necessarily need to be the image I get on my desktop. We’re going to automatically rewrite the CDNs. Most customers have CDNs. They want to take advantage of that. We need the ability to be able to say hey we want to turn it on, in a couple of the examples and we’ll talk through it. They all own CDNs and one of the examples they own the Acamai GSA product which is actually full site acceleration through a CDN. We’re going to flush the buffer early as often as we can automatically on a lot of these sites, talked about, is Eric here? I probably bored him on my last chat so he decided this wasn’t interesting. I overheard one of his conversations and that was just not detailed, then our last chat. You know, one of the challenges before I talk about that in this, in this environment is the levels.

We have people, we did a session yesterday, Hooman and I about acceleration and we had people who came up to us and said that was way too fast and we had people who came up and said yeah, it wasn’t interesting until the 78th minute, kind of like the football game, score, Germany won on the subject of last minute. We also were going to do a lot of preloading and take advantage of the fact that not only is the link between the browser and the server waiting while the server is thinking of churning out data, we can take advantage of that but we can also take advantage of when a user is waiting. So, when I talk about acceleration, we’re talking about some or all of these things happening on the pages that the customers are going through to look at the KPIs. And what does it look like? It looks like an output, which is different for each specific browser group.

So, that’s what I mean when I talk about acceleration. I’m not going to look at one waterfall and dissect it here. We’ve done that before. We have an example of what a waterfall might look like before and after on a first and repeat view page. Obviously, I’ve picked a beauty. They don’t know look this beautiful. We did a talk yesterday where we went through and actually talked about each of the individual steps up to a point. We optimized the Velocity homepage. You can see that and the slides to it there. If you’re interested in finding out more about what acceleration looks like, you know, there are great resources out there. Let’s talk about the site data. Let’s get into it and my timer over here is broken, so I’m going to just do a quick check of time to make sure that we are still on board here and I’m going to invite Parag to come up and give you an introduction.

One of the things we hear when we talk to customers is I’m not going to share my data with you. Some people are local heroes and I want to talk through them. I’m just going to go through quickly here, sorry, Parag and talk about this data. So, I wanted Parag to introduce AutoAnything to us and then I’m going to take it from there.

Parag Patel: Hello, hello. Thanks Joshua. So, AutoAnything is a small retailer as Josh said, not an Amazon, obviously of automotive accessories primarily. So, we are a family owned business and I’ve been in business for about 30 years and made the transition to the web about 11 years ago. So, our, our team because of the size of our company and the complexity of the product that we sell is for, you know, we have a small team in the technology department and we focus really on providing a good user experience for customers on how they buy the product not necessarily focused on, you know, how fast they can navigate the site because we can only put our resources into so many, into so many areas given the limits. And given it’s a family own business, you know, we’re very ROI focused, so almost everything we do, you know, has to have a positive – overwhelmingly positive return. So, you know, we partnered with Josh, like I said, a year and a half two years ago to help us on that piece of our infrastructure that is actually very difficult to influence ourselves given all the complexities.

Joshua Bixby: Thanks sir. Parag was generous enough so that I wouldn’t butcher that. I wanted to start here just couple of minutes of time, we’ve got about 10 minutes left. So, I want to flow through this, then I want to come back through some of the other examples but I wanted to make sure that Parag got a chance to introduce AutoAnything. I want to show you what a homepage looks like from my house. Optimize, not optimized – more importantly I want to tell you about the business impact. We saw average order size, in the study that we did go up by six percent.

Again, remember revenue is average order size times conversion rate 9 percent jump in conversion after we accelerated the site. We saw that 29 percent of people were more likely to buy on their own first visit, which is a fascinating step. We also found that 38 percent were more likely to return for second visit. Obviously our data fidelity degrades overtime in this particular stat because people can clear cookies, we might lose some data fidelities so we tried to stick just to second visit. But the stats were overwhelming and for business that I mean, Parag says mom and pop but let’s be honest, this is quite a large organization, it put through many zeros of revenue I can’t actually share that revenue figure with you but overwhelmingly positive data and influential data. Again, one of the mortal companies that we’re bringing as part of our study. Thank you sir.

Let’s move on. I’m just going to go back a few slides because again cognizant of time I wanted to make sure we got that in, just going to go back really quickly to two other examples. One of the examples I want to talk to you today about is JC Whitney. Again, we put up Alexa Rank here because obviously sharing, you know, key stats like pages and other things our customers weren’t necessarily comfortable with our sharing but we certainly did share some of the public data. Probably a rank that might feel like you 6 to 800 person company, what’s fascinating about this business is that the CEO is on the conversion team. I love it. These guys get it. They not only get it but they spend a lot of money on conversion and optimization already, they own the Acamai DSA product. They are running in a lot of stuff quickly.

Let’s take a look at what performance difference we saw on the homepage from my house. Again in line with some of the things we saw before, let’s look at the business impact. Revenues been up to 2 percent, conversion up 2 percent, customers are 7 percent more likely to buy on the first visit. Obviously different statistics than what Parag has said and what’s so important and what I really want to emphasize is what is your curve. It’s not the same for everybody, different revenue here, different statistics here, different performance, different investment already in acceleration. I want to take you through one more example then I want to talk to you about some aggregate stats because we’re just almost out of time.

Everyday Health, larger organization and the ones we saw before, these guys collect, these are all of their web properties. They are the go-to medical health diet. My wife actually was funny because she didn’t know that this is one of our customers and we just had a baby, my 7-week-old, and she kept sending me updates from whatoexpect.com about what my baby looked like. It can look like a little morphed thing and then it grew and she was fascinated, I thought it was quite interesting I guess. Everyday Health, what’s the difference here, this is the whattoexpect site. Now we’ve talked about conversion, revenue I want to talk about a different stat. This one is really interesting to me, which is for this organization, search is just critical. What they found when they started to put acceleration techniques into place was and what they looked at and what they focused on was a cross stat.

They found that their time spent downloading the page for the crawler was almost half and more importantly instead of 40, 000 pages per day we crawled, they got were getting up to 80. You can pretty much see where we went in, right here. And when they started to make the investment in acceleration. So, again coming back to my first point, it’s not just about – it’s not just about who, you know, conversion or ad clicks. It’s about what’s important to you. Look at that when you make optimization. Just skip through this and talk about one last stat because I’ve got a minute and a half here. Aggregated data. So, we are as I said radioing back performance data. When we combine that with our other data we’re able to start looking at some fascinating trends.

The trend cycle that we’re going to look at here is all related to landing page. What happens, what is the relationship between the performance of your landing page and key metrics? This is five companies all in a very similar space with very similar conversion rights that have aggregated their data all together we’re trying to find some curves. What happens here bounce rate is performance changes. Well, this is fascinating, with a benchmark it is about one second, that’s base line. When your cycle is from one second to three seconds in this aggregated group, your bounce rate goes up by 50 percent. This curve although, you know, we have a lot of data we expect it to continue to smooth out, the first data points roll over the place as we started to add more and more data in, it got more smoother and smoother.

Let’s take a look at pages per visit, correlates pretty well with the stats that we saw from AOL – five customers all aggregating data into one source, let’s look at the big one: conversion rate. Anyone who is going to tell me there is a three second rule or an eight second rule or even a two second rule when I look at data like this I just don’t believe them. One of the things we’re doing is business stats that we collect for ourselves, we’ve done a blog post where we really encourage you to go in – we want to open this up to the world. I don’t know if it is a good idea but we would like to open this up so we create this data warehouse where other people can be subscribing to it, sending us data.

It involves a change in Java Script on your site. I want to hear if this is a good idea. We want more data in that repository. So, if you do want to comment or have input I’d love to hear it. That concludes our session, we have a booth, Parag will be at the booth at 3:30 today and a few times tomorrow if you want to ask him questions, we will also be at the booth to answer any and all of your questions about the data. I really appreciate your time, thank you very much.