Buddy Brewer (LogNormal)

Podcast Transcription

Joshua Bixby: Hello. It’s Joshua again. President of Strangeloop Networks.  Welcome back to the next installment in the webperformancetoday podcast series. When I was at Velocity in London in October, I had a chance to sit down with Cliff Crocker.  Cliff, who is a friend, has a really neat perspective on performance because he has worked on both sides of the wall.  He has spent time as a consultant and worked at Keynote, then crossed the wall and started to work at Walmart where he was the senior engineering manager, which is where I met him selling front-end optimization into Walmart.  We became friends. We were both driving towards the same goal, which was trying to show Walmart that performance mattered. It was friendship that struck up there and has continued through his new role where he is VP Product at SOASTA, which is a cool company and I’m gonna spend a bit of time talking on the podcast about them in one of the podcast coming up. I got a chance to talk to Cliff about his experience in all three companies, the emergence of big data, among other things. I hope you enjoy. Here at Velocity Europe with my friend, Cliff Crocker, formerly of Walmart fame, currently of more famous SOASTA fame. Hey, buddy, how are you doing?
Cliff Crocker: Doing well, doing well. How are you?
Joshua Bixby: I’m great. Thanks for joining me. Tell me about Velocity Europe. How is it going for you?
Cliff Crocker: It’s my first time to Europe period and to come here for a web performance conference I think just really makes it kind of the best trip that I’ve been on for a while. It is going great.  I think that Velocity Europe seems to be raising the bar from what I have seen in terms of the talks and the presentation and slides that came out of Berlin last year and then the quality of the talks and the people here speaking today, just, it’s amazing. It’s going really well.
 Joshua Bixby: Nice. See, they’re loving us in the background too. We’re actually, like, right in the middle here, you can hear the applause from the crowd.
Cliff Crocker: Yeah.
Joshua Bixby: That wasn’t for us, but…
Cliff Crocker: I think it was.
Joshua Bixby: You think we can take credit for that? People are filing past us as we’re chatting here. We’re sort of tucked into corner. I want to rewind to Keynote days.
 Cliff Crocker: Yeah, absolutely. 
Joshua Bixby: How long were you in Keynote? 
Cliff Crocker: I was at Keynote for six years, I believe.
Joshua Bixby: And you started as?
Cliff Crocker: I started as actually doing load testing, as a load-testing consultant. I was coming in to do web performance consulting, but interestingly enough that team wasn’t growing and wasn’t big enough at the time, although a really good friend of mine there, Ned Rushlow [Phonetic] [0:02:22] was doing a great job, kind of, early days doing evangelism, so I jumped onto the load testing team and worked there and did load testing for several e-commerce sites for a few years then switching over and switching into web performance consulting.
Joshua Bixby: So, tell me about the early days of web performance consulting. This would have been, what, 2008?
Cliff Crocker: Yeah, I guess it would’ve been 2008-2007 potentially.
Joshua Bixby: How was the world different back then than it is now?
Cliff Crocker: Well, it is interesting. I think, this industry moved so fast; however, I think, honestly, we still have so many of the same things that we’re still talking about today that we were talking back then. So, it was interesting. I think people kind of were not as attuned to the fact that performance had an impact on the bottom line, so it was a bigger challenge back then than it is today, but I still find myself having the same conversations that I had, you know, six year ago.
Joshua Bixby: So, fundamentally, the similar dialogue?
Cliff Crocker: Absolutely.
Joshua Bixby: Hey, speed’s important, why, maybe you didn’t have the artillery before to tell people, now there seems to be a lot of artillery out there, but…
Cliff Crocker: Yeah, and I think that the performance of browsers back then made it a lot easier to be a consultant, because there were so many best practices that you could take into account and things you could do and now that the browsers are getting better and faster, there is obviously still a lot of optimizations that hold true, but at the time it was a lot easier to kind of go through your checklist prior to or right around the YSlow days and things as they were coming out and right around, you know, the book, when the book was published.
Joshua Bixby: The book. I like that. Steve would like that. What were your lessons from those web consulting days, like, what do you take into SOASTA now that you guys are doing, you know, also in the performance business, what from that time, when that new kid comes into SOASTA and works in your division and you’re saying, listen man, over a beer, let me tell you about the good old days, there were some key lessons I learned, what would you share, grandfather like?
Cliff Crocker: I think the biggest thing, because interestingly, and I will rewind for one second, before SOASTA, after leaving Keynote, I went to Walmart, so really I was able to, and I knew this was part of my journey and part of my career is that I wanted to hop on to the other side of the fence and kind of, you know, see what it was like and what I have been telling people all these years, what were the real challenges, why was it so hard to, you know, combine the JavaScript or, you know, do any number of optimizations that should be simple and easy to do, so I wanted to get a taste for why that was hard and I think what I learnt at the enterprise level and specially when we’re dealing with a site that is so large and sort of forced to moving so slow, there is a lot of things you learn about patience and there is a lot of things you learn about choosing your battles and it is the same old truce, so now that I’m a vendor again and we’re back on that other side, I think that, you know, you can’t make the assumption that you know exactly what’s going on, on the other side of the fence and just because something is easy to do in practice doesn’t mean that it is easy to do in process.
Joshua Bixby: Yeah, and it doesn’t mean they’re idiots.
Cliff Crocker: Yes, exactly, right cos they’re not, I mean, they’re really smart guys, I mean…
Joshua Bixby: So, you evolved out of keynote. How did you get the Walmart gig?
Cliff Crocker: Actually it as a guy that I had been doing consulting with on the Walmart side that finally said, man, why don’t you just come over, just come over here and do this for us, so a lot of that was because they wanted to focus a lot of load testing as well and sort of build up a center of excellence, but, Subir, my boss at the time, Subir Sengupta, brought me in and was just great and kind of gave me a bunch leash and just let me run and build a team and really work on sort of brining performance in as a culture more than anything at Walmart.
Joshua Bixby: And you guys did some amazing stuff. You were there for two years, three years?
Cliff Crocker: Yep, two years.
Joshua Bixby: I mean, you guys, as far as I can tell were one of the leading, if not the leading organization looking at this stuff.
Cliff Crocker: Well, thanks. I think definitely we tried to raise the level of awareness and we tried to, again, change the culture and I think that we got some sponsorship, executive sponsorship really, interestingly enough, not in the engineering side, but more on the business side that said, you know what, this is important, you guys run with it. It’s okay if you go and talk about it. It’s okay if you go and talk about the fact that Walmart isn’t the fastest site on the Internet and the things that we’re trying to do to fix that, so I think it was about community, it was about awareness. A big huge thing for me was actually the web perf meetups that are happening and specifically the San Francisco web perf meetup that Aaron Kulick founded, so I got to meet Aaron very early on when I was actually looking for performance engineers and then started really getting tied into that community that’s where I met Buddy who you’ll talk about from LogNormal, you know, the company that SOASTA has just acquired, as well as Philip Tellis and then, you know, Aaron Kulick who is, you know, a very dear friend and still fighting the fight with Walmart.
Joshua Bixby: Brilliant, brilliant man.
Cliff Crocker: Yes, he is. Yes, he is.
Joshua Bixby: Where did vendors go wrong at Walmart, I mean, as you cross the bridge and were on the Walmart side, you were attacked by vendors, of course, you were a target. What did they do wrong? What were the flaws that you saw continuously from enterprise sales guys trying to sell to Walmart?
Cliff Crocker: You know, I guess, there was a lot assumption, there was a lot of fear tactics that were tried, right? I mean, the vendors coming in and saying, you know, are you just gonna let your site crash or you just gonna this, the other guys are, you know, are better than you, they’re faster than you. I think that that there was just not as much empathy and there wasn’t as much trying to really understand the positioning and understand that, you know, Walmart is a large organization, they’re not gonna move extremely fast on the cell side, but that shouldn’t discourage them from actually coming in and trying to work with them and I think patience was probably the biggest thing.
Joshua Bixby: Just didn’t last…
Cliff Crocker: It’s a long sales cycle and I hate it on this side, I hated it on the Keynote side, but I understand where it comes from.
Joshua Bixby: I know what it is like to sell into Walmart. You know that.
Cliff Crocker: Absolutely. Absolutely, you do. You do. But, you know, you guys kept smiling and there definitely was a lot of patience.
Joshua Bixby: I think it’s the Canadian side of us.
Cliff Crocker: I think it is.
Joshua Bixby: Because there is a side of that. Coming back to Walmart side cos I am always fascinated by the business model that Walmart has, forget the technical side, which is using some of this data, working with vendors very closely to optimize the shopping experience and the price for the shopping experience, I mean, everything that I hear about how they treat vendors, whether you like it or not is they’re close relationships, there’s strong direction, was part of the culture being brought over on the technical side? I mean, was part of the idea that we figured out data for warehousing and shipping to stores and we should figure out how data can be used on the website, was there any crossover there, was there anything you pulled from that culture?
Cliff Crocker: I would say, well, we definitely pulled a lot of things from the vendor management perspective and some of the people there actually really care about on the vendor management side today made it very clear that, you know, our success is depending on the shoulders of all these vendors that we work with, so that culture was definitely there. However, I think that some of the separation between Bud and Bill and, you know, e-commerce side at Walmart and the website itself, there was a pretty big disconnect.  It wasn’t until the formation of Walmart labs that I started to see some that come on and actually teams that were dedicated to data and dedicated to big data, so, that’s where we gotta start playing with people and playing with the cool toys and, you know, the very large city hoop clusters and obviously we got Boomerang up and running at Walmart, started collecting all this rich data and now that is actually driving a lot of what’s going on on that side. It is just about better understanding the customer, being closer to the customer, who is this customer that is shopping at a Walmart versus an Asda in the UK versus, you know, Sam’s Club and that type of a thing, so and then the mobile store gets extremely interesting and, you know, you saw, I don’t know if you saw Dion and Ben talking this morning, but they’re always great to watch and great to talk to.
Joshua Bixby: They are. They have a great interplay between the two of them, like, it’s a good, sort of, on stage gig they have.
Cliff Crocker: We were chatting last night in the lobby and they were finishing each other’s sentences every other word. Obviously those guys have been doing some cool stuff for years, but the things they’re doing and the opportunities they have there in mobile, if you think about the number of people that are in a Walmart store or in some type of a property all across the US at any time with a smart phone in their pocket, you start to connect the dots and see what a huge opportunity that is and innovation that goes on there between those gentlemen as well as the great team that they have there, the mobile team at Walmart labs, you know, it’s fun stuff. It’s pretty cool.
Joshua Bixby: I don’t know if you read that article in New York Times about Target and how they use data…
Cliff Crocker: Yeah, the whole thing about the dad finding out that his daughter is pregnant, yeah.
Joshua Bixby: How did that type of thing resonate in how you guys were using data and thinking about data, you know, and I ask this also from a perspective of SOASTA, I mean, there are things we can find out about people, what they’re looking at, how much they’re spending, when you think of that whole challenge of privacy around this, how do you think about that?
Cliff Crocker: I think, you know, there is a creepiness factor, right? There is always a creepiness factor and a big brother factor thinking about, oh my gosh, you know, just me as a consumer thinking about I don’t know that I want everyone knowing all this stuff about me and, you know, predicting, you know, what I’m doing. I just joined a CrossFit last week and I need to, you know, be buying these products or, you know, longer socks or whatever it might be, but to be honest I think that it’s actually, it’s not an intention of trying to, you know, invade privacy or, you know, predict, you know, all types of things to really maximize revenue, it’s more of a competitive advantage and something that, you know, all companies are having to do if they want to serve the customer better, right? So, I love it. I think that it’s great. I think big data is amazing. I think that what we’re doing at SOASTA or beginning to kind of embark on with Buddy and Philip from LogNormal and me coming in and having experience from Walmart and really delivering that product line, I think it’s not always gonna be about that performance, it’s gonna really be about human behavior and how can we sort of predict what that user is going to do next or what their behavior might be on a site, so we can think more about what should we actually be testing, on which we actually should be spending our time, what should we be optimizing and, you know, what kind of things and behaviors are driving people away. Buddy introduced something yesterday that he has talked about before called the LD50. The lethal dose basically where you guys have the performance poverty line at Strangeloop, out stick may be the LD50 where you’re looking at what point the user starts bouncing or exiting a site when performance gets to that level and sort of understanding that in a broad spectrum, but in a multifaceted way where you’re looking at multiple dimensions, whether it is browser, device type, geography, whatever it might be, it’s just really about creating better quality and also providing more input into that whole development life cycle where functional requirements come in from the business and marketing and all the way through to where we’re supporting production.
Joshua Bixby: Yeah. I saw an interesting startup today on the New York Times, New York based.  Its goal is to allow people to sell their information, so, you know, I could track all of my browsing history and then sell it back to either a conglomerate of vendors or vendors who might want to buy that information.
Cliff Crocker: Wow.
Joshua Bixby: Yeah. I don’t know where it’s gonna go, but I thought, you know, this whole world of privacy and tracking and what we do and how we interact with it, how we time it, it’s interesting and I thought that, you know, it definitely caught my interest. I was like wow, that’s interesting.
Cliff Crocker:  Yeah. Crowdsourcing is very powerful and I think what we wanna do is, well, start to feed all that data back into the industry whether it’s in a way that’s free, that we can all kind of consume it and understand how different industries perform, different verticals perform so we can kind of provide more contextual intelligence and get more contextual intelligence to our own data. It’s extremely powerful and really why I’m doing it cos the questions always change and more things you can do with the data.
Joshua Bixby: So, you have this great gig at Walmart, you’re collecting information, you’re analyzing it, you’re in big data heaven and SOASTA obviously had something pretty attractive. What attracted you, I mean, I don’t think SOASTA is a household name for most people yet, so I guess, give us, what was attractive about it and why, like, you’re a high-value asset, why did you go there?
Cliff Crocker: You know, I think that I had given myself a timeline and said, hey, I wanna do this for a while, but I don’t wanna stay here, I wanna move, I wanna actually, you know, do something that’s gonna have more impact at a global level as opposed to use within one organization, so it was parting on great terms first of all with Walmart, but really the reason that I thought SOASTA was so attractive was back to your point about vendors and partnerships. Before I even get into the technology and the things they’re good at, they were a company with a lot of integrity, you know, when I was dealing with a sales rep it wasn’t dealing with a sales rep, it was dealing with an account manager and a guy that I could call and talk to and, you know…
Joshua Bixby: Who knew something about the product.
Cliff Crocker: Exactly, yeah.
Joshua Bixby: Not just what the discounts were this quarter.
Cliff Crocker: Exactly. Exactly. You know, not how much he was gonna let me beat him up on price or whatever, it was more about getting the work done and just the innovative people there and how much the people really loved working there and really loved the direction of the company. But all that aside and that’s all nice and warm and fuzzy, these are guys are really smart, these guys have done this, you know, the two founders, Ken Gardner, who I report to, the executive chairman, and Tom Lounibos, have been doing this for a while and have had several other successful startups and the thing that Ken, really attracting about Ken and the reason that we work well together was really his history with real-time analytics and data and that was something that they started doing years ago and had perfected with one of their previous companies and we started putting the dots together and realizing, hey, we can start to deal with this data in a real-time way and also do it with some real-time visualizations and the visualization part of it kind of had me sold in terms of, you know, playing with the data, making it beautiful, breaking the mold, you know, not just developing yet another monitoring product, but doing something what could be different and really kind of change the way people look at real user measurement.
Joshua Bixby: And then you guys obviously bought a real user measurement company two days, yesterday. Tell me about that.
Cliff Crocker: Yeah. I might’ve had a little something to do with that. So, Buddy was actually the guy who I brought on to help me to get up and running with Boomerang at Walmart and I, again, met him through the web perf meetup through Aaron’s meetup there and I always loved working with Buddy, but he wasn’t a guy that I was ever gonna be able to hire and he and Philip started this great thing with LogNormal. I loved the way that they were actually, the approach that they were taking with the data, very statistical and analytical approach to measuring data and a sensible way of doing it, so, you know, rather than kind of strike out and, you know, try and find and build a team of people that could adapt and learn about performance and really sort of build that up again, which will take some time and is hard to do, I thought what a great idea to actually go out and talk to these guys and see if your interests are in line enough for them to actually come join us and do this together.  So, it was a process of several months. Taking about partnership, looking, you know, do we do this ourselves, do we, you know, how do we actually win in this space and it just ended up making sense and I, you know, couldn’t be happier about the decision. I’m very excited to work with those guys, very humbled and, you know, when I sit down and talk with Philip or Buddy about their ideas and their thoughts, it’s just exciting and it’s fun again, so.
Joshua Bixby: And, I mean, I’ve met Buddy a number of occasions and I echo your sentiments, smart dude, we’re actually gonna have him on the podcast here, so, people will get to hear for themselves. Tell me about where measurements goes, I mean, as you said, and I’ve always thought this, the market hasn’t changed that much over the last 10 years, I mean, the browsers have changed, but not much else has. We have more simulation of real end users, we’re getting closer to, you know, people are understanding that you have to use a real browser and it’s important to have a location that mimics a real end user and a bandwidth that mirrors a real end user, what’s changing?
Cliff Crocker: Well, I think what’s changing is the fact that simulations, you know, while they have served us well and they have done a good job to this point, there are absolutely no replacement for human behavior and I think that’s really what’s driving this, is that instead of a world where we base everything on synthetic and live and die by our Gomez Keynote web page test numbers, I’d rather live in a world that we’re actually basing it on what we’re measuring from the end user. So, I think leading with that as opposed to leading with synthetic is really what’s changing and how it is going to be different and also, you know, setting some new metrics or coming up with some new metrics that just make more sense.  I think what we’re big on is the fact that, you know, whether you’re at Walmart or, you know, Target or whatever sites you might be as our test go, what’s important to that company is their number and what their goal is and really looking at their own data instead of trying to base their, you know, their studies on something that Google did or Amazon did or Shopzilla did or Walmart did, when I take that same model and really see, you know, what makes sense for our customers and our users, where is the LD50 or performance poverty line for our businesses as opposed, you know, the rest of the industry.
Joshua Bixby: And do you think it’s varied? I mean, do you think Walmart is different than Target?
Cliff Crocker: Well, I think that everyone and every company that I’ve ever talked to or consulted with thinks they’re different and I think that’s…
Joshua Bixby: Are they?
Cliff Crocker: I think that there are different types of customers. I think that, you know, whether you’re…if you’re selling something at Walmart that someone could easily pop over to Amazon and buy because the page is slower and it’s not loading, it’s different than if you’re a specialty store and selling something that someone absolutely has to have from that store, like a cheese shop or something, right? So, I think that they are different. I think that what’s not different is really the fact that we do see some level of drop off, we do see some level of engagement where people are just getting more and more and more impatient or more expecting, you know, to be delighted as Philip would put it, you know, by providers like, you know, Walmart and everyone else, so I think that expectations are growing at the same rate, but I think that behaviors might vary enough and not even just between industries, but, you know, one of the study that we did that you guys have posted on your blog as well and talked about was basically where performance changes is depending on the type of product you’re looking at, that patience or that tolerance for someone who is buying something like an iPad is going to be, you know, much greater than someone who is buying PowerBait or, you know, something else off the site, right?
Joshua Bixby:  Yeah and that’s very interesting, almost within segments of the product line one has different tolerance, right?
Cliff Crocker:  Exactly.
Joshua Bixby:  I know that for myself.  I’m buying a car right now and I’ll spend a good 20 minutes on a page, you know, and I’ll wait for it because I want to see what capabilities and…
Cliff Crocker: A very Flash-heavy page.
Joshua Bixby: Yeah, I mean, I don’t like it, but I definitely wait because, you know, it’s a big purchase and I’m gonna take some time, so I get that, I mean, as somebody who preaches that every second counts, I must admit, in my own behavior sometimes I definitely will spend more time on one thing than I will on the other. That makes a lot of sense to me.
Cliff Crocker: And I think, you know, back to sort of your original point, that it hasn’t really changed that much.  I think that it has and it hasn’t, but also this whole mobile thing that, you know, isn’t just a fad anymore, has really changed the game as well because it has made it harder again for developers to get give a fast user experience that people are expecting, you know, same speeds over carrier lines that they are over, you know, DSL or whatever.
Joshua Bixby: Yeah. No, I know, that’s something I’ve been spending my time thinking about the presenting on, which is how do cell phones work and why are they slow and, you know, that’s definitely a real area of interest.
Cliff Crocker: Right.
Joshua Bixby: What have you learnt at Velocity that’s new, net new, like, you’re gonna take back to the shop and say, holy smokes! Everyone needs to download this slide, anything?
Cliff Crocker: Well, actually, I, embarrassingly, wasn’t able to attend your talk.
Joshua Bixby:  So, there you go.  That was like six of the slides I was thinking, that was like a soft ball for you.  Other than the genius that I presented, any other one or two slides that stick out?
Cliff Crocker: And it was only because Steve organized the track in a way that LogNormal presenting at the same time as you were, otherwise I would’ve been there.
Joshua Bixby: That’s true. That’s true. You’re one of the only ones that has an excuse, although someone else had to do an intro so I figured they had an excuse too, I can’t remember who it was.
Cliff Crocker: I think aside from that, which I’m sure to be inspired by and motivated by…
Joshua Bixby: I love it. This is good. Keep going. I can handle this forever.
Cliff Crocker:  I think, actually, one session I just came out of with Pat Meenan who I’m a huge fan of, he’s my hero.  He was doing his whole presentation that had nothing to do with Webpagetest, but had everything to do with single point of failure and SPOF-O-Matic and I think that’s been introduced, you know, for a while and then it sort of died off and now it’s getting really hot again at Velocity in US and here and something that I can talk to customers about from a performance perspective, in terms of beingready for holiday or something.  It is not just about load testing, but look for these things and so I think SPOF-O-Matic is great.  I’m excited about that.
Joshua Bixby: That’s cool. I was beta testing that early on and loved it, like, I was so excited using it, so I’m a big fan of that stuff too.
Cliff Crocker: Yeah. Absolutely. So, if I had to say there’s obviously something I learn with every talk even if it’s someone who has recycled their slides 12 times and I’m not gonna call out any names or anything, but I think that every single one of those talks…
Joshua Bixby:  We all recycle a bit, but there should be some, you know, especially a conference of this magnitude, you should put some original thought into I think.
Cliff Crocker: Yeah. I was actually, you know, sort of silently referring to Steve’s slides that aren’t so great, but… 
Joshua Bixby: Of all the guys, he is kind of known for the same shtick.
Cliff Crocker:  Absolutely.
Joshua Bixby: It’s like when you go McDonalds, you expect a burger to be the same every time, you know.
Cliff Crocker: But you get something new every single time. I take something new from it every time I hear it or I hear something in a different way or I think of a new question to ask and what I love of Velocity is the fact that it’s the hallway track…
Joshua Bixby: Yeah, we can hear it in the background here, you can hear the hallway track, we’re in the hallway track.
Cliff Crocker:  Absolutely. It’s the networking, it’s the talking, it’s the ideas and the community and the companies that get started and then companies that have successful exits and all circle around Velocity that I think is just amazing. So, it’s an amazing community. It’s been amazing to me, I mean, I have a lot to be thankful for this whole movement or whatever you wanna call it because it certainly has created endless opportunities for me.
Joshua Bixby: No, it’s a good tribe. Cliff, thank you.
Cliff Crocker: Yeah.
Joshua Bixby: Take care. I want you to get out, I’m gonna stop this so you can get out.  I see some clouds over there…
Cliff Crocker: That’s right.
Joshua Bixby:  …kind of in the background.
Cliff Crocker: That’s right, yeah.
Joshua Bixby: So, get out there and enjoy London.
Cliff Crocker: And what was the name of the place again, one more time?
Joshua Bixby: Borough Market.
Cliff Crocker: Borough Market, got it, got it. 
Joshua Bixby: Right near London Bridge. 
Cliff Crocker: Thanks.
Joshua Bixby: That’s my current…I don’t know when it’s open till, but it’s definitely a lunch place.  It’s fantastic.
Cliff Crocker: Okay. Excellent. Thanks for having me.
Joshua Bixby: Okay. Take care. Well, that was great. Thanks for listening and thanks again to Cliff for taking the time out of what was a very, very busy schedule to chat at Velocity. All the links and slides that Cliff and I spoke about are available on the blog, webperformancetoday.com/podcast.  If you have a suggestion for a future podcast, topic or guest, please drop me a line at joshua@webperformancetoday.com.  Any and all suggestions are welcome.  I’ve had some crazy ones and I’m trying to book some of those crazy chats and I look forward to more.  Have a wonderful day.  Thanks for listening.

Joshua Bixby: Hello everyone its Joshua here. Welcome back to the next installment in the Web Performance Today podcast series. I am excited to bring you another chat from my time in London at the Velocity EU Conference. This time I got a chance to sit down with Buddy Brewer. Buddy is the Co-Founder and CEO of LogNormal. I got a chance to sit down with Buddy the day after they were acquired by SOASTA. My good friend over there Cliff Crocker who will be running that division decided that RUM and Synthetic should live together which is a world I have always espoused.

Buddy and I got a chance to talk about Real User Monitoring which is really a hot topic in the industry, I have written a lot about it recently on webperformancetoday.com. We got to talk about how to teach site owners to learn to love RUM and really interesting to talk to someone who took a company from bootstrapping to acquisition in a year. So we got a chance to talk about that as well, I hope you enjoy.

Here at Velocity Europe with Buddy Brewer who has something to tell us, what’s the big news, now it’s going to be late because these pods, this is like the world of podcasting when you have exciting news and then the podcast goes up like a week later, but let’s pretend that the news that aired what a day ago is still modern.

Buddy Brewer: Yeah right at the beginning of Velocity.

Joshua Bixby: Give it to us man.

Buddy Brewer: 8 O’clock A.M. London time and right at midnight back on the West Coast, LogNormal was acquired by SOASTA, so we are really excited about it.

Joshua Bixby: Good work congratulations.

Buddy Brewer: Yeah thanks a lot Josh.

Joshua Bixby: First exit.

Buddy Brewer: It is our first – it’s both my co-founder Phil Tellis and mine it was our first startup that we actually started ourselves, so I have been part of startup before but the first one that we actually did, you know, from inception to exit, so we are super excited about it.

Joshua Bixby: And are you infected with the bug now?

Buddy Brewer: We are.

Joshua Bixby: Yeah.

Buddy Brewer: I got to say we are.

Joshua Bixby: So we got to tell Cliff and the SOASTA guys that what it’s like a two year horizon until you are going to start a new one.

Buddy Brewer: But probably a little bit longer than that --.

Joshua Bixby: See you are still talking the party line.

Buddy Brewer: The beauty of it though is that, you know, SOASTA is not a big company right, so we are going from a tiny company to small to medium sized company that’s had a lot of early success and I think has a really great trajectory ahead of them and I mean we are super stoked that we got to be part of that so.

Joshua Bixby: Yeah I think they are doing some cool stuff. Tell me – give me the birth story of LogNormal?

Buddy Brewer: I think it’s really apropos that we did the announcement here at Velocity because we are a company that really never would have happened if it hadn’t been for the web performance community conferences like Velocity as well as local meet ups. Philip and I actually met in the San Francisco Bay Area, I had just recently moved out there and Philip was living out there and had been for a few years and --.

Joshua Bixby: When is this? What circa?

Buddy Brewer: This was a little bit over a year ago.

Joshua Bixby: Okay.

Buddy Brewer: So LogNormal was incorporated on, if I have this right, it was July 29th of last year.

Joshua Bixby: I love it.

Buddy Brewer: So we are just a little bit over a year old. A few months prior to that, we were going to some of the same events the SF WebPerf Meetup like I said over that they both had in the City of San Francisco as well as occasionally down in the peninsula. And I was looking for something new to do; I just left a company that I have been at for about seven years and I was trying to figure out, you know, did I want to do, I knew I wanted to stay in web performance, but I didn’t know if I wanted to do consulting or if I wanted to go work for another company and I had this opportunity to start something. So after spending all these years in synthetic monitoring, I wanted to spend a little bit of time looking into Real User Measurement, I got really excited about that and that’s how I learned about Boomerang, which was a free library that Phillip had written for people to use to do their own Real User Measurement.

Joshua Bixby: Yeah.

Buddy Brewer: There is just one problem there is no backend for it.  So I sat down I said well you know see this is a great opportunity to sit down and write a backend and you know maybe build a nice little sort of business around providing that. So I hooked up with Phillip went and saw him out and sat down and you know figured maybe he’d tweet something about it or something right and you know he says you know what I am thinking about doing the same thing. So once we discovered that we were both on the same track and we were in same sort of geographical area, we sat down and started plan this out and that’s really sort of the beginnings of how LogNormal came about. And we built the product and we got our early thing out there and started to get some interest and thanks to people that we both knew in the web performance industry, we were able to get ourselves into some conversations to people to get some early users that turned into some early customers. 

But we didn’t really start to get a lot of exposure until Velocity Santa Clara, that was the opportunity that we had to actually get into a room with hundreds of people and sort of get our perspective on Real User Measurement out there and thanks to the exposure that we got at that conference we really started to kind of ramp our growth to bring on more customers and started to attract the interest of another business that was looking to kind of getting to the Real User Measurement business. And so not only did the web performance community and conferences like Velocity gave us an opportunity to sort of meet and start this company but it also enabled us to grow it and then ultimately was the basis for the introductions that we made that allowed us to exit.

Joshua Bixby: Where is the name from?

Buddy Brewer: Which is the beginning of just something of -.

Joshua Bixby: Which is the beginning yes, the entry.

Buddy Brewer: Yeah right, right.

Joshua Bixby: What’s the story around the name?

Buddy Brewer: Yeah so LogNormal, it’s interesting because Phillip and I knew of each other but we didn’t really know each other super well, so at the same time that we were starting the company, we were kind of getting to know each other’s personalities and how each other ticks. So we went back and forth on the name of our company over and over and over again while we were writing the initial code and you know trying to figure out which law firm to use and all that kind of stuff that you do when you start a business. But the interesting thing about the exercise of naming the business was, it was the first non-trivial kind of not that it was a difficult conversation.

Joshua Bixby: It’s like getting a tattoo.

Buddy Brewer: Yeah I mean I was --.

Joshua Bixby: It’s like getting a tattoo.

Buddy Brewer: Exactly right, so that was a great sort of team building exercise that we went through and we knew that we wanted to start out with Real User Measurement, but you know being that it was a small company and we were just getting started on this we weren’t exactly sure where we were going to go other than that we definitely wanted to stay in web performance. So we wanted to pick a name of the business you know that said something about web performance but wasn’t necessarily or really specific, so we set that on LogNormal because the – when you plot all of the load times in a histogram for any dataset it tends to follow a LogNormal distribution.

Joshua Bixby: Yeah.

Buddy Brewer: So hence that’s the name of our company.

Joshua Bixby: Yeah that’s cool it’s good, it seems very appropriate, it’s like one of those things that feels like it should be named that you know like you hear the name you know that that’s well placed, it’s – I always I liked the name of the business.

Buddy Brewer: And the other thing for us it was kind of like a little bit of kind of like a captured a little bit of kind of this sort of radical upstart notion that we wanted to capture in the way that we were approaching business because we knew that we were small. We were going to be a bootstrap startup; we weren’t starting out seeking capital. So in order to differentiate it was going to be really hard for us to sort of go head to head with a lot of these big incumbent players in the space. So we said you know what we will do is we won’t try to pretend to be something that we are not, we will just try to out geek everybody.

Joshua Bixby: Yeah.

Buddy Brewer: So we will pick the nerdiest name that we can come up with the name of our business and then we will sort of wear that as a badge of pride, so that’s what we did.

Joshua Bixby: Tell me about RUM and you know you guys have been leading this for a year now. Tell me about where it’s been and where it’s going, give me a bit of the history, the Boomerang history and then follow that through to where we are here today and to the future, for people who don’t know about where RUM comes from, give us that story and then pitch us through.

Buddy Brewer: Right, so Real User Measurement first of all, let me preface all of this by saying that I think that there is a place in everyone’s monitoring toolkit for both Synthetic and Real User Measurement. So it’s not an either or proposition, they are both actually quite complementary, but if you look at the history of web performance testing and I have been involved in it in one way or another pretty much since 2001 and when I got started in it really synthetic monitoring was the way to go and one of the reasons why synthetic monitoring worked really well was because there was really only one web browser that mattered and that was IE6. 

So you know and that worked really well for a really long time, but then you know some very important things happened right like you know sort of the resurgence of Netscape and then Chrome and Safari and these other browsers on the desktop that entered the field and then the rise of the Mobile Ecosystem. They create all these problems that it was really difficult for Synthetic to deal with, one being browser diversity. It’s just so difficult and also the pace of the revisions of browsers, right. I mean IE6 was IE6 for so long before IE7 showed up. But then you know once that happened and now you have got Chrome like it seems like they released a new version of browser like every week or something.

Joshua Bixby: They are like forty now I don’t know.

Buddy Brewer: Yeah who knows, it’s the things like you know -.

Joshua Bixby: Maybe four.

Buddy Brewer: Maybe we will talk about navigation timing later but it’s easy to remember you know when navigation timing was introduced in Firefox and IE but I couldn’t even tell you, I can’t remember when it was introduced in Chrome because they ran so fast. But you know for those types of things really started to strain Synthetic and especially with mobile where you have got a situation where you know for one thing it’s so dependent on that single strength and the state that the device is in and all of these things that have to do with human activities where they roam around you on the planet and what state their telephones is in and all that kind of stuff that’s hard to capture in the datacenter and I remember in some of their early days when the company I was at was trying to setup mobile synthetic monitoring, they actually had problems getting signals from where they had their phones racked up like they put them in datacenters and things.

Joshua Bixby: Sure, yeah of course.

Buddy Brewer: So -.

Joshua Bixby: That’s a lot of radio frequency disruption when you are sitting in the basement of a datacenter somewhere, right.

Buddy Brewer: Yeah totally, exactly right and one of the things that I was noticing is that a lot of the incumbence were, they had done such a great job building this ecosystem around synthetic monitoring and then they started to add Real User Measurement, but I think where one of the places where Phillip and I wanted to differentiate is there was still this perspective where you lead with synthetic and you did most of your monitoring around synthetic and then where there are gaps like for example these browsers that should you know have covered you over locations which is another huge one, then you would fill in those gaps with real user and where we came in as we said you know we think that that’s backwards. 

So we would rather sort of encouraged people to lead with Real User Measurement because there are all these reasons why that’s great, that we can certainly go into and then where there are gaps like for example competitive benchmarking or object level data currently is something that you can’t really get out of Real User Measurement, then go ahead and fill those gaps in with synthetic but our approach was to sort of turn it around the other way and lead with real user. That’s something that people are starting to get now. At Velocity in Santa Clara, we heard so much about RUM; it was great we loved it right.

Joshua Bixby: Yeah and that was good, great timing.

Buddy Brewer: Yeah and this year it seems like RUM is a big thing you know here in London as well as Mobile, which is something that we are also you know very passionate and excited about going through in the future. But the message that we have been trying to get out there around Real User Measurement though especially here at this conference I think is that people are just starting to understand how to use Real User Measurement effectively. So the analogy that I always come back to is, it’s kind of like the early days of television in the 1940’s right, people treated TV initially like it was radio with pictures. So you got this guy that’s you know just reading off of sheets of paper for the nightly news and just reading into the microphone and there is a camera pointing at him and he is not doing anything interesting, he is like smoking a cigarette and reading these sheets of paper right.

Joshua Bixby: Yeah.

Buddy Brewer: And it took a while before people realized that television was actually fundamentally different than radio in some pretty key ways and you could do a lot of interesting stuff you know with those fundamental differences and to bring that back to RUM versus Synthetic I think that what a lot of people are doing now is sort of this first order Real User Measurement where they do the same things that they did with Synthetic like create these lists of my fastest and slowest geographies or my fastest and slowest browsers or you know types of pages, home versus search, you know versus product detail or whatever and that’s important. Those are things that should be done, but that’s to me not the most interesting part about Real User Measurement that the second order things that you can with it or where you sort of take that performance stuff and you correlate it against the behaviors of the real human beings that you collected those measurements from and that’s something that’s fundamentally not possible to do in the world of Synthetic Measurement that’s really powerful when you do it in Real User Measurement because you can discover things from that that allowed you to set goals and make discoveries about how performance relates to your business goals, that was just not possible you know prior to some of the advancements that have happened recently by RUM.

Joshua Bixby: Tell me about it because I really believe that same story, that the power of RUM is that I can connect it to every other analytics, real time or analytics, web analytics tool everything see in Google analytics, everything I see in Omniture can be connected back to time and I can get such a rich dataset.  How does this look long term, does Google Analytics and Omniture somehow become the RUM vendor of the future, given that they already have this incredibly rich in user business metric dataset, does it get absorbed there, is it a standalone piece five years from now, where do you, how do you see that evolve for the typical business?

Buddy Brewer: I think it’s going to be really exciting to see what direction that goes in and it’s, I think there is a lot of different things that could happen as far as you know what those businesses choose to do and pull in.  But it’s got to start with what I think that we are in the middle of at LogNormal and now are at SOASTA and that’s creating conversations where they didn’t use to happen. Right, so what historically you have had is you have had you know your ops and development people that look at certain datasets from their synthetic monitoring vendors or you know if they are doing Real User Measurement it’s very specific to timing pages and collecting all this technical data. And then you know overall on the business side and the marketing group they are neck deep in their Omniture data and stuff and they like to talk about the metrics that they get out of Omniture and Google Analytics and  really have no interest over in the synthetic data and vice versa and they are just speaking two different languages. So by correlating the performance metrics and the business metrics and all of the sudden you are creating this scenario where you know if you relate things like page load times to revenue or to conversions, which is something that you can readily do using Real User Measurement. Then all of the sudden you know the marketing folks get really interested in what those page load times are, the engineering and operations groups I think also got excited about the fact that they can now for the first time understand exactly how you know their inputs to the organization effect the you know the revenues in the top line. So once you create those conversations and then the marketing analytics companies and everybody start to find out that you know that they start to get asked about these types of features.

Joshua Bixby: Yeah.

Buddy Brewer: Then you know I think that for Real User Measurement you will start to see some consolidation and people building things out and you will start to see it happen in both directions. You will start to see more of the performance tools companies getting more involved in that business data and I think you will start to see some of those marketing companies start to pull in more of that stuff that you know it used to not be in the RUM.

Joshua Bixby: Yeah I have been thinking a lot about where that, you know where those worlds collide because they, I think as you said and such a good point the synthetic world has nothing to do with the business metrics right, they are just it’s totally disconnected, there may be some long term correlation between the speed of a synthetic test and performance but there is no direct correlation. What’s so cool about RUM is it’s directly correlated to everything else I am calculating and I think that’s correlation is fascinating. You guys built the business for a year and then sold it why, why not build it, why not and get capital and build it out, what was the choice to join, to merge, get acquired by SOASTA?

Buddy Brewer: Yeah we had essentially I think three different options right you know stay on the track that we were on continue to bootstrap and you know when we needed money to pay our bills and stuff like that while our ramping customers go do consulting.

Joshua Bixby: Yeah.

Buddy Brewer: You know that’s maybe option A, option B is you know go get capital somewhere and go on that track and then option C was to you know to join forces with another company and we were going down that first path and we were starting to see, first of all that people were starting to get that message and as the space heats up of course other people start to come in and the environment gets more competitive and in order for us to continue, to sort of enjoy all of the kind of relevance and mindshare and everything that we are getting, we are going to have to start growing faster and to start building things faster so we are going to need to have some kind of leverage in that right.

Joshua Bixby: Get some capital.

Buddy Brewer: So as we started to move away from that bootstrap model we were looking at capital and we were looking at you know opportunities to be acquired. Really us and SOASTA just kind of like met each other in the right place at the right time and it was one of those things that you know we weren’t necessarily looking at the exact time that we met specifically to go be acquired by someone or have an exit but it was really about the people and we originally started talking about you know sort of this arms linked things about well this could be in anything we could join up with them or maybe we could do a partnership or wherever and the more that we talked, the more that we discovered that we really had a very shared vision around the real user measurement and analytics, so we were really trying to achieve the same thing and that this could be a really symbiotic relationship if we were on the same team because they already had you know this great team of these world class engineers and the sales organization that’s out you know selling their existing products and doing a great job of that and really winning in the market in the things that SOASTA was currently doing and they were looking to get into this other space where we had a lot of expertise, but we just didn’t have that kind of distribution you know channels and everything so by joining forces that it really did create this environment where I think we can continue to do exactly what we always wanted to achieve in the first place which is to get our message about Real User Measurement and everything out there, but do it at a much bigger scale and a lot faster and at the same time you know about this personnel relationships and everything with a group of people that we just genuinely liked and thought that we would have a lot of fun working together over the next you know several years getting it done.

Joshua Bixby: Was there a moment where you thought this is going to be an acquisition, like was there a – did you go out for a phone call or a conference call and you are like this is real.

Buddy Brewer: Yeah, yeah so me and Cliff Croker who you also interviewed go back a little ways and at one point we were talking and he shared a little bit of his roadmap with us and I remember getting off the call and thinking this is the same roadmap, we either need to be competing with each other or we need to be working directly together right you know this is no longer a partnership conversation, this is we are doing the same thing so do we want to do it independently or do we want to do it jointly and again you know like I said it really comes back to the people because we met the rest of the executive team at SOASTA and met the engineering staff and the more people that we talk to and the more we realize that not only did we share the same goals, but this was a great just sort of team to be a part of and a great place to spend the next few years working on technology and really making Real User Measurement happen.

Joshua Bixby: That’s very cool.  Any memorable slides from Velocity so far, anything that sticks out you are going to take home?

Buddy Brewer: I would say probably the biggest thing especially here in the UK, probably more so than even Santa Clara is the focus around mobile because you know like I said I spent so much time in the synthetic world pushing desktop stuff and then with Real User Measurement you know just getting our business started and doing all the things that that come out of that has consumed so much of our time and everything that you know I have to admit that frankly I have been probably sleeping on this whole mobile thing more than I should right, even to the point that prior to about a week ago I really knew very little about this whole kind of middle ground where it’s not just about native, it’s not just about HTML5 or pure web applications but there are this whole hybrid -.

Joshua Bixby: Yeah this while hybrid world.

Buddy Brewer: And you know I think that’s been really fascinating, I think it poses a lot of really interesting problems with respect to how you measure performance and how you make those things, how you make those things fast right and that’s something that we are also looking into from a RUM perspective is how we get people insight not only into things that happen in the web browser or a web view but also what happens in the native code and how do we discover a set of metrics that we can use and maybe create some kind of common language where we can you know relate about performance in the native world as usually as we do in a regular browsing world.

Joshua Bixby: Yeah that’s interesting. Lastly, I want to ask about a problem I face which is I go to customers and push RUM a lot as you know I am a big RUM supporter. I face the challenge where customers are so used to synthetic world where they can get an answer in five minutes or they have perceived that they have a viable answer in five minutes and in the RUM world I need to say be patient, you don’t have enough data yet, there is no statistical viability in the data that you have, you need to wait, teaching people about outliers, teaching people about basically stats that we probably all learned in great ‘11 but most people have long forgotten. When you go into organization and they start experimenting with RUM isn’t it a dangerous weapon if used improperly? Do you find the fact that people have to become stats animals, you know animals that think statistically, that that’s a problem for senior executives or other people actually looking at the tools.

Buddy Brewer: Well and there is lies, damn lies in statistics right I mean you can make the numbers say just about anything that you want to make him say. At the end of the day one of the things that we built into LogNormal right from the beginning was putting the margin of error around as many metrics as we possibly could and really trying to give people a sense of the uncertainty around the numbers and be upfront about it.  We try to do it in an easy way and we try to, one of the things that we think about a lot and are going to think about a lot more going forward is how we take some of those complex concepts and put them into visualizations and things where people can sort of just inherently understand things about the data without having to, have a degree in statics right I think when you start to confront people with big lists of numbers and all these Greek letters and symbols and stuff that they haven’t seen before all that kind of stuff and yeah -.

Joshua Bixby: Yeah here is the standard deviation and it’s like you have lost all bunch of executives at that point right.

Buddy Brewer: Yeah it really starts to scare, yeah, yeah exactly so I think that the biggest challenge with anything that’s data oriented is finding ways to conceptualize the data and to present it in a visual way that communicates some kind of useful message without over simplifying and giving people the wrong impression.

Joshua Bixby: And there is that balance right?

Buddy Brewer: And that’s one of the things that you know I think some of the early tools and one of the traps that people fall into a lot is you know allowing outliers to secure their numbers and they put these you know numbers up in great big huge giants fonts and they are just not very realistic for all sorts of problems right. So we try to use, it’s one of the reasons why we tend to focus more on percentiles and putting things in histograms. 

Joshua Bixby: Yeah.

Buddy Brewer: You know rather than showing arithmetic means and stuff like that and also why we really try to be upfront with people and in fact there are some metrics that we don’t even show unless we have you know a minimum sample size.

Joshua Bixby: See that’s what I want, that’s what I have always pushed forward, don’t even give, don’t even give them the gun you know until, until it’s like you know until you are in a position where you know it’s safe, just don’t even give them the tool because they are going to make bad judgments. The tools can be an interface to help educate us on when judgments can be made and when they can’t and when we all see a number we make judgments right?

Buddy Brewer: Yeah and that was one of the things too with kind of going back to the why SOASTA thing you know that we got really excited when we were working with those guys because we had seen some of the visualizations and the UI work that they had done in their existing products, so it wasn’t just about getting any old group of the bunch people and just having them you know help us scale our business. It was about you know this very specific set of expertise around, I think especially around the visualization side where we could learn things and apply that expertise to Real User Measurement and accomplish some pretty interesting things.

Joshua Bixby: I wish you all the best of luck, we will be in touch. I appreciate you taking the time.  Are you going to go out and see a bit of London now?

Buddy Brewer: Yeah I think so I hope Cliff has been waiting on me -.

Joshua Bixby: I know, I know I got to get you out of here because this is Cliff’s first time to Europe so you guys have to get out, the clouds are coming in, enjoy, have some good beer and let’s stay in touch.

Buddy Brewer: Thanks a lot Josh.

Joshua Bixby: Okay take care. 

Joshua Bixby: Thank you for taking the time to listen to today’s podcast.  Many thanks to Buddy who took time in a very busy Velocity Conference, for him a very busy schedule, he took time to sit down and visit with me. I wish him all the best of luck and I am very excited about what they are doing over there at SOASTA. Check out the rest of the podcast in the series at webperformancetoday.com/podcast and if you have any suggestions for future topics or even better, if you would like to volunteer yourself to put yourself forward, to speak out on performance, on web performance, acceleration any topic related please reach out to me at joshua@webperformancetoday.com, I have been really enjoying these conversation and look forward to talking to more really interesting people. Have a great day.