Geoffrey Smalling (Wine.com)

Podcast Transcription

Joshua Bixby: Hello. It’s Joshua again. President of Strangeloop Networks.  Welcome back to the next installment in the webperformancetoday podcast series. When I was at Velocity in London in October, I had a chance to sit down with Cliff Crocker.  Cliff, who is a friend, has a really neat perspective on performance because he has worked on both sides of the wall.  He has spent time as a consultant and worked at Keynote, then crossed the wall and started to work at Walmart where he was the senior engineering manager, which is where I met him selling front-end optimization into Walmart.  We became friends. We were both driving towards the same goal, which was trying to show Walmart that performance mattered. It was friendship that struck up there and has continued through his new role where he is VP Product at SOASTA, which is a cool company and I’m gonna spend a bit of time talking on the podcast about them in one of the podcast coming up. I got a chance to talk to Cliff about his experience in all three companies, the emergence of big data, among other things. I hope you enjoy. Here at Velocity Europe with my friend, Cliff Crocker, formerly of Walmart fame, currently of more famous SOASTA fame. Hey, buddy, how are you doing?
Cliff Crocker: Doing well, doing well. How are you?
Joshua Bixby: I’m great. Thanks for joining me. Tell me about Velocity Europe. How is it going for you?
Cliff Crocker: It’s my first time to Europe period and to come here for a web performance conference I think just really makes it kind of the best trip that I’ve been on for a while. It is going great.  I think that Velocity Europe seems to be raising the bar from what I have seen in terms of the talks and the presentation and slides that came out of Berlin last year and then the quality of the talks and the people here speaking today, just, it’s amazing. It’s going really well.
 Joshua Bixby: Nice. See, they’re loving us in the background too. We’re actually, like, right in the middle here, you can hear the applause from the crowd.
Cliff Crocker: Yeah.
Joshua Bixby: That wasn’t for us, but…
Cliff Crocker: I think it was.
Joshua Bixby: You think we can take credit for that? People are filing past us as we’re chatting here. We’re sort of tucked into corner. I want to rewind to Keynote days.
 Cliff Crocker: Yeah, absolutely. 
Joshua Bixby: How long were you in Keynote? 
Cliff Crocker: I was at Keynote for six years, I believe.
Joshua Bixby: And you started as?
Cliff Crocker: I started as actually doing load testing, as a load-testing consultant. I was coming in to do web performance consulting, but interestingly enough that team wasn’t growing and wasn’t big enough at the time, although a really good friend of mine there, Ned Rushlow [Phonetic] [0:02:22] was doing a great job, kind of, early days doing evangelism, so I jumped onto the load testing team and worked there and did load testing for several e-commerce sites for a few years then switching over and switching into web performance consulting.
Joshua Bixby: So, tell me about the early days of web performance consulting. This would have been, what, 2008?
Cliff Crocker: Yeah, I guess it would’ve been 2008-2007 potentially.
Joshua Bixby: How was the world different back then than it is now?
Cliff Crocker: Well, it is interesting. I think, this industry moved so fast; however, I think, honestly, we still have so many of the same things that we’re still talking about today that we were talking back then. So, it was interesting. I think people kind of were not as attuned to the fact that performance had an impact on the bottom line, so it was a bigger challenge back then than it is today, but I still find myself having the same conversations that I had, you know, six year ago.
Joshua Bixby: So, fundamentally, the similar dialogue?
Cliff Crocker: Absolutely.
Joshua Bixby: Hey, speed’s important, why, maybe you didn’t have the artillery before to tell people, now there seems to be a lot of artillery out there, but…
Cliff Crocker: Yeah, and I think that the performance of browsers back then made it a lot easier to be a consultant, because there were so many best practices that you could take into account and things you could do and now that the browsers are getting better and faster, there is obviously still a lot of optimizations that hold true, but at the time it was a lot easier to kind of go through your checklist prior to or right around the YSlow days and things as they were coming out and right around, you know, the book, when the book was published.
Joshua Bixby: The book. I like that. Steve would like that. What were your lessons from those web consulting days, like, what do you take into SOASTA now that you guys are doing, you know, also in the performance business, what from that time, when that new kid comes into SOASTA and works in your division and you’re saying, listen man, over a beer, let me tell you about the good old days, there were some key lessons I learned, what would you share, grandfather like?
Cliff Crocker: I think the biggest thing, because interestingly, and I will rewind for one second, before SOASTA, after leaving Keynote, I went to Walmart, so really I was able to, and I knew this was part of my journey and part of my career is that I wanted to hop on to the other side of the fence and kind of, you know, see what it was like and what I have been telling people all these years, what were the real challenges, why was it so hard to, you know, combine the JavaScript or, you know, do any number of optimizations that should be simple and easy to do, so I wanted to get a taste for why that was hard and I think what I learnt at the enterprise level and specially when we’re dealing with a site that is so large and sort of forced to moving so slow, there is a lot of things you learn about patience and there is a lot of things you learn about choosing your battles and it is the same old truce, so now that I’m a vendor again and we’re back on that other side, I think that, you know, you can’t make the assumption that you know exactly what’s going on, on the other side of the fence and just because something is easy to do in practice doesn’t mean that it is easy to do in process.
Joshua Bixby: Yeah, and it doesn’t mean they’re idiots.
Cliff Crocker: Yes, exactly, right cos they’re not, I mean, they’re really smart guys, I mean…
Joshua Bixby: So, you evolved out of keynote. How did you get the Walmart gig?
Cliff Crocker: Actually it as a guy that I had been doing consulting with on the Walmart side that finally said, man, why don’t you just come over, just come over here and do this for us, so a lot of that was because they wanted to focus a lot of load testing as well and sort of build up a center of excellence, but, Subir, my boss at the time, Subir Sengupta, brought me in and was just great and kind of gave me a bunch leash and just let me run and build a team and really work on sort of brining performance in as a culture more than anything at Walmart.
Joshua Bixby: And you guys did some amazing stuff. You were there for two years, three years?
Cliff Crocker: Yep, two years.
Joshua Bixby: I mean, you guys, as far as I can tell were one of the leading, if not the leading organization looking at this stuff.
Cliff Crocker: Well, thanks. I think definitely we tried to raise the level of awareness and we tried to, again, change the culture and I think that we got some sponsorship, executive sponsorship really, interestingly enough, not in the engineering side, but more on the business side that said, you know what, this is important, you guys run with it. It’s okay if you go and talk about it. It’s okay if you go and talk about the fact that Walmart isn’t the fastest site on the Internet and the things that we’re trying to do to fix that, so I think it was about community, it was about awareness. A big huge thing for me was actually the web perf meetups that are happening and specifically the San Francisco web perf meetup that Aaron Kulick founded, so I got to meet Aaron very early on when I was actually looking for performance engineers and then started really getting tied into that community that’s where I met Buddy who you’ll talk about from LogNormal, you know, the company that SOASTA has just acquired, as well as Philip Tellis and then, you know, Aaron Kulick who is, you know, a very dear friend and still fighting the fight with Walmart.
Joshua Bixby: Brilliant, brilliant man.
Cliff Crocker: Yes, he is. Yes, he is.
Joshua Bixby: Where did vendors go wrong at Walmart, I mean, as you cross the bridge and were on the Walmart side, you were attacked by vendors, of course, you were a target. What did they do wrong? What were the flaws that you saw continuously from enterprise sales guys trying to sell to Walmart?
Cliff Crocker: You know, I guess, there was a lot assumption, there was a lot of fear tactics that were tried, right? I mean, the vendors coming in and saying, you know, are you just gonna let your site crash or you just gonna this, the other guys are, you know, are better than you, they’re faster than you. I think that that there was just not as much empathy and there wasn’t as much trying to really understand the positioning and understand that, you know, Walmart is a large organization, they’re not gonna move extremely fast on the cell side, but that shouldn’t discourage them from actually coming in and trying to work with them and I think patience was probably the biggest thing.
Joshua Bixby: Just didn’t last…
Cliff Crocker: It’s a long sales cycle and I hate it on this side, I hated it on the Keynote side, but I understand where it comes from.
Joshua Bixby: I know what it is like to sell into Walmart. You know that.
Cliff Crocker: Absolutely. Absolutely, you do. You do. But, you know, you guys kept smiling and there definitely was a lot of patience.
Joshua Bixby: I think it’s the Canadian side of us.
Cliff Crocker: I think it is.
Joshua Bixby: Because there is a side of that. Coming back to Walmart side cos I am always fascinated by the business model that Walmart has, forget the technical side, which is using some of this data, working with vendors very closely to optimize the shopping experience and the price for the shopping experience, I mean, everything that I hear about how they treat vendors, whether you like it or not is they’re close relationships, there’s strong direction, was part of the culture being brought over on the technical side? I mean, was part of the idea that we figured out data for warehousing and shipping to stores and we should figure out how data can be used on the website, was there any crossover there, was there anything you pulled from that culture?
Cliff Crocker: I would say, well, we definitely pulled a lot of things from the vendor management perspective and some of the people there actually really care about on the vendor management side today made it very clear that, you know, our success is depending on the shoulders of all these vendors that we work with, so that culture was definitely there. However, I think that some of the separation between Bud and Bill and, you know, e-commerce side at Walmart and the website itself, there was a pretty big disconnect.  It wasn’t until the formation of Walmart labs that I started to see some that come on and actually teams that were dedicated to data and dedicated to big data, so, that’s where we gotta start playing with people and playing with the cool toys and, you know, the very large city hoop clusters and obviously we got Boomerang up and running at Walmart, started collecting all this rich data and now that is actually driving a lot of what’s going on on that side. It is just about better understanding the customer, being closer to the customer, who is this customer that is shopping at a Walmart versus an Asda in the UK versus, you know, Sam’s Club and that type of a thing, so and then the mobile store gets extremely interesting and, you know, you saw, I don’t know if you saw Dion and Ben talking this morning, but they’re always great to watch and great to talk to.
Joshua Bixby: They are. They have a great interplay between the two of them, like, it’s a good, sort of, on stage gig they have.
Cliff Crocker: We were chatting last night in the lobby and they were finishing each other’s sentences every other word. Obviously those guys have been doing some cool stuff for years, but the things they’re doing and the opportunities they have there in mobile, if you think about the number of people that are in a Walmart store or in some type of a property all across the US at any time with a smart phone in their pocket, you start to connect the dots and see what a huge opportunity that is and innovation that goes on there between those gentlemen as well as the great team that they have there, the mobile team at Walmart labs, you know, it’s fun stuff. It’s pretty cool.
Joshua Bixby: I don’t know if you read that article in New York Times about Target and how they use data…
Cliff Crocker: Yeah, the whole thing about the dad finding out that his daughter is pregnant, yeah.
Joshua Bixby: How did that type of thing resonate in how you guys were using data and thinking about data, you know, and I ask this also from a perspective of SOASTA, I mean, there are things we can find out about people, what they’re looking at, how much they’re spending, when you think of that whole challenge of privacy around this, how do you think about that?
Cliff Crocker: I think, you know, there is a creepiness factor, right? There is always a creepiness factor and a big brother factor thinking about, oh my gosh, you know, just me as a consumer thinking about I don’t know that I want everyone knowing all this stuff about me and, you know, predicting, you know, what I’m doing. I just joined a CrossFit last week and I need to, you know, be buying these products or, you know, longer socks or whatever it might be, but to be honest I think that it’s actually, it’s not an intention of trying to, you know, invade privacy or, you know, predict, you know, all types of things to really maximize revenue, it’s more of a competitive advantage and something that, you know, all companies are having to do if they want to serve the customer better, right? So, I love it. I think that it’s great. I think big data is amazing. I think that what we’re doing at SOASTA or beginning to kind of embark on with Buddy and Philip from LogNormal and me coming in and having experience from Walmart and really delivering that product line, I think it’s not always gonna be about that performance, it’s gonna really be about human behavior and how can we sort of predict what that user is going to do next or what their behavior might be on a site, so we can think more about what should we actually be testing, on which we actually should be spending our time, what should we be optimizing and, you know, what kind of things and behaviors are driving people away. Buddy introduced something yesterday that he has talked about before called the LD50. The lethal dose basically where you guys have the performance poverty line at Strangeloop, out stick may be the LD50 where you’re looking at what point the user starts bouncing or exiting a site when performance gets to that level and sort of understanding that in a broad spectrum, but in a multifaceted way where you’re looking at multiple dimensions, whether it is browser, device type, geography, whatever it might be, it’s just really about creating better quality and also providing more input into that whole development life cycle where functional requirements come in from the business and marketing and all the way through to where we’re supporting production.
Joshua Bixby: Yeah. I saw an interesting startup today on the New York Times, New York based.  Its goal is to allow people to sell their information, so, you know, I could track all of my browsing history and then sell it back to either a conglomerate of vendors or vendors who might want to buy that information.
Cliff Crocker: Wow.
Joshua Bixby: Yeah. I don’t know where it’s gonna go, but I thought, you know, this whole world of privacy and tracking and what we do and how we interact with it, how we time it, it’s interesting and I thought that, you know, it definitely caught my interest. I was like wow, that’s interesting.
Cliff Crocker:  Yeah. Crowdsourcing is very powerful and I think what we wanna do is, well, start to feed all that data back into the industry whether it’s in a way that’s free, that we can all kind of consume it and understand how different industries perform, different verticals perform so we can kind of provide more contextual intelligence and get more contextual intelligence to our own data. It’s extremely powerful and really why I’m doing it cos the questions always change and more things you can do with the data.
Joshua Bixby: So, you have this great gig at Walmart, you’re collecting information, you’re analyzing it, you’re in big data heaven and SOASTA obviously had something pretty attractive. What attracted you, I mean, I don’t think SOASTA is a household name for most people yet, so I guess, give us, what was attractive about it and why, like, you’re a high-value asset, why did you go there?
Cliff Crocker: You know, I think that I had given myself a timeline and said, hey, I wanna do this for a while, but I don’t wanna stay here, I wanna move, I wanna actually, you know, do something that’s gonna have more impact at a global level as opposed to use within one organization, so it was parting on great terms first of all with Walmart, but really the reason that I thought SOASTA was so attractive was back to your point about vendors and partnerships. Before I even get into the technology and the things they’re good at, they were a company with a lot of integrity, you know, when I was dealing with a sales rep it wasn’t dealing with a sales rep, it was dealing with an account manager and a guy that I could call and talk to and, you know…
Joshua Bixby: Who knew something about the product.
Cliff Crocker: Exactly, yeah.
Joshua Bixby: Not just what the discounts were this quarter.
Cliff Crocker: Exactly. Exactly. You know, not how much he was gonna let me beat him up on price or whatever, it was more about getting the work done and just the innovative people there and how much the people really loved working there and really loved the direction of the company. But all that aside and that’s all nice and warm and fuzzy, these are guys are really smart, these guys have done this, you know, the two founders, Ken Gardner, who I report to, the executive chairman, and Tom Lounibos, have been doing this for a while and have had several other successful startups and the thing that Ken, really attracting about Ken and the reason that we work well together was really his history with real-time analytics and data and that was something that they started doing years ago and had perfected with one of their previous companies and we started putting the dots together and realizing, hey, we can start to deal with this data in a real-time way and also do it with some real-time visualizations and the visualization part of it kind of had me sold in terms of, you know, playing with the data, making it beautiful, breaking the mold, you know, not just developing yet another monitoring product, but doing something what could be different and really kind of change the way people look at real user measurement.
Joshua Bixby: And then you guys obviously bought a real user measurement company two days, yesterday. Tell me about that.
Cliff Crocker: Yeah. I might’ve had a little something to do with that. So, Buddy was actually the guy who I brought on to help me to get up and running with Boomerang at Walmart and I, again, met him through the web perf meetup through Aaron’s meetup there and I always loved working with Buddy, but he wasn’t a guy that I was ever gonna be able to hire and he and Philip started this great thing with LogNormal. I loved the way that they were actually, the approach that they were taking with the data, very statistical and analytical approach to measuring data and a sensible way of doing it, so, you know, rather than kind of strike out and, you know, try and find and build a team of people that could adapt and learn about performance and really sort of build that up again, which will take some time and is hard to do, I thought what a great idea to actually go out and talk to these guys and see if your interests are in line enough for them to actually come join us and do this together.  So, it was a process of several months. Taking about partnership, looking, you know, do we do this ourselves, do we, you know, how do we actually win in this space and it just ended up making sense and I, you know, couldn’t be happier about the decision. I’m very excited to work with those guys, very humbled and, you know, when I sit down and talk with Philip or Buddy about their ideas and their thoughts, it’s just exciting and it’s fun again, so.
Joshua Bixby: And, I mean, I’ve met Buddy a number of occasions and I echo your sentiments, smart dude, we’re actually gonna have him on the podcast here, so, people will get to hear for themselves. Tell me about where measurements goes, I mean, as you said, and I’ve always thought this, the market hasn’t changed that much over the last 10 years, I mean, the browsers have changed, but not much else has. We have more simulation of real end users, we’re getting closer to, you know, people are understanding that you have to use a real browser and it’s important to have a location that mimics a real end user and a bandwidth that mirrors a real end user, what’s changing?
Cliff Crocker: Well, I think what’s changing is the fact that simulations, you know, while they have served us well and they have done a good job to this point, there are absolutely no replacement for human behavior and I think that’s really what’s driving this, is that instead of a world where we base everything on synthetic and live and die by our Gomez Keynote web page test numbers, I’d rather live in a world that we’re actually basing it on what we’re measuring from the end user. So, I think leading with that as opposed to leading with synthetic is really what’s changing and how it is going to be different and also, you know, setting some new metrics or coming up with some new metrics that just make more sense.  I think what we’re big on is the fact that, you know, whether you’re at Walmart or, you know, Target or whatever sites you might be as our test go, what’s important to that company is their number and what their goal is and really looking at their own data instead of trying to base their, you know, their studies on something that Google did or Amazon did or Shopzilla did or Walmart did, when I take that same model and really see, you know, what makes sense for our customers and our users, where is the LD50 or performance poverty line for our businesses as opposed, you know, the rest of the industry.
Joshua Bixby: And do you think it’s varied? I mean, do you think Walmart is different than Target?
Cliff Crocker: Well, I think that everyone and every company that I’ve ever talked to or consulted with thinks they’re different and I think that’s…
Joshua Bixby: Are they?
Cliff Crocker: I think that there are different types of customers. I think that, you know, whether you’re…if you’re selling something at Walmart that someone could easily pop over to Amazon and buy because the page is slower and it’s not loading, it’s different than if you’re a specialty store and selling something that someone absolutely has to have from that store, like a cheese shop or something, right? So, I think that they are different. I think that what’s not different is really the fact that we do see some level of drop off, we do see some level of engagement where people are just getting more and more and more impatient or more expecting, you know, to be delighted as Philip would put it, you know, by providers like, you know, Walmart and everyone else, so I think that expectations are growing at the same rate, but I think that behaviors might vary enough and not even just between industries, but, you know, one of the study that we did that you guys have posted on your blog as well and talked about was basically where performance changes is depending on the type of product you’re looking at, that patience or that tolerance for someone who is buying something like an iPad is going to be, you know, much greater than someone who is buying PowerBait or, you know, something else off the site, right?
Joshua Bixby:  Yeah and that’s very interesting, almost within segments of the product line one has different tolerance, right?
Cliff Crocker:  Exactly.
Joshua Bixby:  I know that for myself.  I’m buying a car right now and I’ll spend a good 20 minutes on a page, you know, and I’ll wait for it because I want to see what capabilities and…
Cliff Crocker: A very Flash-heavy page.
Joshua Bixby: Yeah, I mean, I don’t like it, but I definitely wait because, you know, it’s a big purchase and I’m gonna take some time, so I get that, I mean, as somebody who preaches that every second counts, I must admit, in my own behavior sometimes I definitely will spend more time on one thing than I will on the other. That makes a lot of sense to me.
Cliff Crocker: And I think, you know, back to sort of your original point, that it hasn’t really changed that much.  I think that it has and it hasn’t, but also this whole mobile thing that, you know, isn’t just a fad anymore, has really changed the game as well because it has made it harder again for developers to get give a fast user experience that people are expecting, you know, same speeds over carrier lines that they are over, you know, DSL or whatever.
Joshua Bixby: Yeah. No, I know, that’s something I’ve been spending my time thinking about the presenting on, which is how do cell phones work and why are they slow and, you know, that’s definitely a real area of interest.
Cliff Crocker: Right.
Joshua Bixby: What have you learnt at Velocity that’s new, net new, like, you’re gonna take back to the shop and say, holy smokes! Everyone needs to download this slide, anything?
Cliff Crocker: Well, actually, I, embarrassingly, wasn’t able to attend your talk.
Joshua Bixby:  So, there you go.  That was like six of the slides I was thinking, that was like a soft ball for you.  Other than the genius that I presented, any other one or two slides that stick out?
Cliff Crocker: And it was only because Steve organized the track in a way that LogNormal presenting at the same time as you were, otherwise I would’ve been there.
Joshua Bixby: That’s true. That’s true. You’re one of the only ones that has an excuse, although someone else had to do an intro so I figured they had an excuse too, I can’t remember who it was.
Cliff Crocker: I think aside from that, which I’m sure to be inspired by and motivated by…
Joshua Bixby: I love it. This is good. Keep going. I can handle this forever.
Cliff Crocker:  I think, actually, one session I just came out of with Pat Meenan who I’m a huge fan of, he’s my hero.  He was doing his whole presentation that had nothing to do with Webpagetest, but had everything to do with single point of failure and SPOF-O-Matic and I think that’s been introduced, you know, for a while and then it sort of died off and now it’s getting really hot again at Velocity in US and here and something that I can talk to customers about from a performance perspective, in terms of beingready for holiday or something.  It is not just about load testing, but look for these things and so I think SPOF-O-Matic is great.  I’m excited about that.
Joshua Bixby: That’s cool. I was beta testing that early on and loved it, like, I was so excited using it, so I’m a big fan of that stuff too.
Cliff Crocker: Yeah. Absolutely. So, if I had to say there’s obviously something I learn with every talk even if it’s someone who has recycled their slides 12 times and I’m not gonna call out any names or anything, but I think that every single one of those talks…
Joshua Bixby:  We all recycle a bit, but there should be some, you know, especially a conference of this magnitude, you should put some original thought into I think.
Cliff Crocker: Yeah. I was actually, you know, sort of silently referring to Steve’s slides that aren’t so great, but… 
Joshua Bixby: Of all the guys, he is kind of known for the same shtick.
Cliff Crocker:  Absolutely.
Joshua Bixby: It’s like when you go McDonalds, you expect a burger to be the same every time, you know.
Cliff Crocker: But you get something new every single time. I take something new from it every time I hear it or I hear something in a different way or I think of a new question to ask and what I love of Velocity is the fact that it’s the hallway track…
Joshua Bixby: Yeah, we can hear it in the background here, you can hear the hallway track, we’re in the hallway track.
Cliff Crocker:  Absolutely. It’s the networking, it’s the talking, it’s the ideas and the community and the companies that get started and then companies that have successful exits and all circle around Velocity that I think is just amazing. So, it’s an amazing community. It’s been amazing to me, I mean, I have a lot to be thankful for this whole movement or whatever you wanna call it because it certainly has created endless opportunities for me.
Joshua Bixby: No, it’s a good tribe. Cliff, thank you.
Cliff Crocker: Yeah.
Joshua Bixby: Take care. I want you to get out, I’m gonna stop this so you can get out.  I see some clouds over there…
Cliff Crocker: That’s right.
Joshua Bixby:  …kind of in the background.
Cliff Crocker: That’s right, yeah.
Joshua Bixby: So, get out there and enjoy London.
Cliff Crocker: And what was the name of the place again, one more time?
Joshua Bixby: Borough Market.
Cliff Crocker: Borough Market, got it, got it. 
Joshua Bixby: Right near London Bridge. 
Cliff Crocker: Thanks.
Joshua Bixby: That’s my current…I don’t know when it’s open till, but it’s definitely a lunch place.  It’s fantastic.
Cliff Crocker: Okay. Excellent. Thanks for having me.
Joshua Bixby: Okay. Take care. Well, that was great. Thanks for listening and thanks again to Cliff for taking the time out of what was a very, very busy schedule to chat at Velocity. All the links and slides that Cliff and I spoke about are available on the blog, webperformancetoday.com/podcast.  If you have a suggestion for a future podcast, topic or guest, please drop me a line at joshua@webperformancetoday.com.  Any and all suggestions are welcome.  I’ve had some crazy ones and I’m trying to book some of those crazy chats and I look forward to more.  Have a wonderful day.  Thanks for listening.

Joshua Bixby: I am Joshua Bixby, President of Strangeloop Networks. Welcome to another edition of the Web Performance Today Podcast. If you have been following along, you will know that I have been talking to a lot of players in the performance solution space. Today I am going to flip things around a bit and talk to a site owner, someone who actually buys the solution. Geoffrey Smalling is a Chief Technology Officer at Wine.com, the world’s number one online wine store. Geoff and I have had a chance to sit down and talk about Wine.com’s performance epiphany, his experience in ramping up the company’s performance efforts for their extremely complex and dynamic site and probably most importantly, the impact all of this has had on Wine.com’s business, enjoy.

Joshua Bixby: So you are the first e-commerce customer on the podcast. We’ve actually had some really interesting people already from other companies but you are the first e-commerce guy and as you know I talk a lot about e-commerce. So tell me about Wine.com, I am interested and just give me the, you know, you have been there since, what is it 2002, so you have been there for a decade. Tell me about Wine.com?

Geoffrey Smalling: I am old, yeah well Wine.com is a great intersection of technology and wine, you know many people go into the wine store and then see isles and isles of wine and it's like you know organized by varietal or a region but it's really hard to slice and dice and move them around, so we’ve sort of built this huge database of wine and we try to bring the consumer information about wine and let them access it, let them research, let them track what they like, get recommendations, it's just all about empowering the consumer with wine knowledge and making it easy and fun to slice and dice and find the right wine out of the thousands and thousands we carry.

Joshua Bixby: And you are the CTO, you are the head, you are the head honcho?

Geoffrey Smalling: Yeah I am the guy that keeps it running, mostly these days and builds the new features.

Joshua Bixby: Tell me about how lives change, I mean it's 10 years in the e-commerce business is like the guy who started the model T and has come up to you know modern production standards so tell me about that, tell me about that evolution, how has that been?

Geoffrey Smalling: Oh it’s been – it's been quite a rise. I really started before even Wine.com in the e-commerce as a consultant so you know it's been a pretty fun, I mean the technology always think that it can’t any better and boom it gets better every year, people are – our vendors are coming like you, are coming up with great ideas all the time and there is just this vast array of knowledge to learn and new techniques and new marketing methods I mean that’s been the most fun from being a guy whose electrical engineering degree is learning all the marketing and that keeps changing, so it's a space that I think it's going to keep evolving for quite a long time.

Joshua Bixby: What's hot on the plate right now, I mean we just came, we just came through the Thanksgiving Day weekend and obviously you know Christmas and all the rest of it, what's high on your agenda, has this lockdown time for you, is this or you’ve done your experimenting and now put it into the black box and pray or where are you at?

Geoffrey Smalling: Theoretically, yes but we are still small dot com, I mean we have been around 10 years but it's, the issue with wine is there is a lot of state regulation, so it's been you know we haven’t boomed and grown like Amazon or others. We are at this cusp of always wanting to lockdown but always struggling to grow, so we are pretty risky. We are, it makes it fun, we have done all of our big projects, we actually have a release tonight. It's Wednesday, so it's 11:28 but we’ve put some best practices in so we ensure that we are not going break stuff and roll back the plans and then we are going to actually release on Monday too.

Joshua Bixby: Oh cool.

Geoffrey Smalling: So we keep releasing through the holidays as long as it's low risking, a new one that I am pretty excited about is we are going to do something that we have always wanted to do and it's something you can do in this store, but you can’t do very well online and it's probably going to affect our performance while we are watching it, all of our bottles were, not all but a majority of bottles will turn into a spinable bottles where you can actually sort of reach out and grab it, where you grab it, spin it, zoom in the label and actually be able to read the back label see if it has an interesting story, look at the alcohol content so trying to make it more engaging.

Joshua Bixby: And so do the suppliers provide you with the pictures or is that something you guys do in house, I mean how do you get that?

Geoffrey Smalling: We decided to do it in house.

Joshua Bixby: Well that’s a great repository, that’s a great dataset though I mean I am sure that that’s going to be valuable long term to have all those pictures and the history.

Geoffrey Smalling: Yeah I mean integrating with you know people like Amazon have launched a wine marketplace and other kinds of marketplace out there, that bigger picture they keep asking for bigger and bigger pictures so it's just we decided to make an investment so we have got you know domes, light domes that spin the bottle -.

Joshua Bixby: Cool, when am I going to see this, is that live now or is that part of the new release?

Geoffrey Smalling: It's going to be a new release probably on Sunday or Monday.

Joshua Bixby: Nice okay.

Geoffrey Smalling: We are still getting through a couple of kinks with it, it's on our QA site, I can send you a link.

Joshua Bixby: Nice yeah, yeah sure I am not going to post that on the blog but yeah send me a link I would love to take a look. What's -.

Geoffrey Smalling: We are a little worried about performance and so hopefully we have done our best to ensure its delay loaded and all that stuff but -.

Joshua Bixby: Yeah we got you covered on that side. Tell me about what's keeping at night you know are there things that you know either micro, macro trends that you can share with us that keeps a CTO up at night?

Geoffrey Smalling: Right now it's just a lot of stuff going on with Cyber Monday and Cyber Tuesday and you know that the peak is coming and unlike most e-commerce companies I think and I would say most because I am sure there is others like us but we have a lot of warehouses because of the state laws so this time of the year the backend keeps me up at night more than the frontend just making sure all the warehouses are operational, running, making sure it's fast and then you know watching performance of the website you know there is always something we uncover that’s a little glitch, you know maybe a feature we added in June and now we are at a hundred percent capacity of June and boom the glitch uncovers itself.

Joshua Bixby: Yeah, yeah.

Geoffrey Smalling: So we are always watching out for that it's like I think every year we have some stresses in that uncovers a little weakness.

Joshua Bixby: And you also have children?

Geoffrey Smalling: Oh yeah.

Joshua Bixby: Which I mean keep me up at night I don’t know about yours but you know -.

Geoffrey Smalling: Yeah the little guys yeah.

Joshua Bixby: There is – that element is, it also keeps one up.

Geoffrey Smalling: It's pretty embarrassing when your child goes hey daddy let's go to the office and brings his toy computer. That means you have been at the computer away too much.

Joshua Bixby: Yeah mine is that when they grab the iPhone and or iPad and start playing with it as if they have spent their entire life on it and then you are looking to other parents like no, no that’s occasional right I don’t use that to babysit my children so yes I am, you know I am interested in your perspective something that’s come up a lot across my desk has been performance measurement, real end user monitoring, synthetic monitoring you know the idea that you know as you said how fast is my site, is it fast tell me about how you see that problem, how you guys look at performance measurement?

Geoffrey Smalling: Oh my god it's in art form these days, I don’t know if there is a quiet of a science to it yet. We are pretty addicted to a new tool, called New Relic.

Joshua Bixby: Yeah I love those guys.

Geoffrey Smalling: We watch it like a hawk because it has the eyes and ears of everything so we can watch transactions from the web user, all the way down to the database and it's been an incredible tool, we are always watching New Relic and just seeing the trends and seeing the averages that’s sort of real user monitoring and then we use Gomez just to make sure the network level is all working all properly. I also use Gomez to sort of get an SLA on vendors and you know I will reach out to the CEO or VP of a beacon company and say hey what's going on here you are firing, you are slowing down and so there is this sort of monitoring tools that we use, we try to monitor I think my team is sick of me about performance that is like, that’s a sort of my pet area because it's the one thing marketing and the business side can’t really get and it's like you know they can’t really voice their opinion on it, they just like that’s it's faster.

Joshua Bixby: Yeah, yeah.

Geoffrey Smalling: And so this is an area that you know we are always trying to make the site faster with every release and every new feature and sometimes we do that, sometimes we don’t, so we have got you know a New Relic we can actually undeploy to a little bookmark in and so we can watch ourselves over time and that’s what tonight release is, it's just actually a performance tuning release, we are trying to speed up a couple of key pages with doing some optimizations and -.

Joshua Bixby: Do you find, how long have you guys been using New Relic, how long has that been sort of operational?

Geoffrey Smalling: About a year.

Joshua Bixby: And so you have had a long enough history to have sort of used it through a lifecycle of the product and it's, you know well I am a big fan of what those guys are doing and a lot of our customers are using them as well so it doesn’t surprise me you are seeing a good experience, I have also, our experience is also in the customer service side quite a high standard there too so that’s always nice?

Geoffrey Smalling: Yeah that’s been really great.

Joshua Bixby: Tell me about obviously you are a CTO of an important e-commerce site in North America, it puts you squarely in sort of scope of every sales guy in the world, right every sales guy wants a piece of you, they want your budget, they want your time, they want you to try it, try it, try it; tell me about the sales guys and this is a topic that really interest me obviously you know we all have wears to pitch. Tell me about when they make big mistakes, what are the big, what are the you know if you are to train my sales team and say here are the top three things that I just can’t stand that technology vendors do, can you tell me what those are, are top fifty?

Geoffrey Smalling: I guess the biggest kind of any kind vendor of me is when they, haven't even looked at our site and don’t even know what we do that you just can’t immediately tell that they got our, my name from a database.

Joshua Bixby: Yeah.

Geoffrey Smalling: And have my title or even an old title and if you don’t know what we do it's like you know that’s just pretty much almost hang up at that time it's, that’s my biggest pet peeve – but yeah, you know we get a lot of calls I would like to listen to almost everybody at least but sometimes you know I can’t afford you in some, I pretty much go there at first and you know I like a lot of technology you know I think I have picked out a lot of the technology that we really like and you know it fits sort of like Strangeloop did or New Relic did with sort of our core cost or core initiative, but we are going to look at it and work on it. I guess the other one for sales teams is I have actually had a great experience with a CDN vendor recently and the guy was really persistent but not overly persistent, you know you just ping every quarter and he knew I was in a longer contract and couldn’t get out of it but he pinged in just to say hey how are you doing and nothing pushy and then the time just happened where he pinged me and I needed him and I remembered oh god this guy was great, he wasn't pushy and you know we did a little business deal so it worked out to be persistent but not pushy.

Joshua Bixby: Yeah that’s a balance I spend time on with our sales guys trying to strike that balance and man do people ever get it wrong in the technology side you know it's just an, I get inundated with these calls and I find it so I sometimes just pick up the phone and think you know I want to educate you as a sales guy because you are never going to meet your quota this way you know it doesn’t, you got to have a long term horizon and you have to add some value and yeah so I have the same level of frustration.

Geoffrey Smalling: Really my biggest issue as a vendor is not really the sales, it's the sales to implementation and that transition and I think almost every company has gotten it wrong for us a lot of times and then you know it's a big issue and I don’t know how people can do that better. Except for the sales guys bring the transitional guy so there is nothing lost in the whole sale.

Joshua Bixby: Yeah that’s, that’s I mean that’s such a struggle, either overpromising on delivering or just or which is the worst but even just things that get lost in translation right?

Geoffrey Smalling: Exactly that’s the biggest issue and we have a, we have a simple site in theory selling wine but you know our state selection process has thrown every company for a loop and you know the sales guy won't, we will think through it and maybe talk to the sales engineer that come up with solution, that solution theoretically does not get translated down the line to the implementers so then you will back at round one, level one.

Joshua Bixby: Yeah I have unfortunately experienced that from both sides so I am well aware of, I am well aware of dropping both balls. Tell me about the cloud, you are a buyer of technology, you actually spend money on it where does the cloud fit into your vision and is it a game changer for you in anyway?

Geoffrey Smalling: So you know I love the cloud personally and if I were to do my own startup I would probably make the technology cloud friendly. Unfortunately we have been in business for twelve years so we have got a lot of technology that we have built and we have got investment in the datacenter.

Joshua Bixby: Yeah.

Geoffrey Smalling: So for us it's quite an investment that we don’t have yet to move to the cloud but we use it for services that we you know we, services like you know Strangeloop or you know one cloud vendor we are loving right now is called Mailgun so we part here with a lot of the cloud vendors for technology in using their API’s and you know we love people that have taken core functions that we don’t have to manage anymore like e-mail management and you know so that you know our vision is using the services that other people have and you know we use Amazon S3 for sort of community image and content stores, we use things but we have a pretty big database and a pretty big legacy one and you know that we caused it out, it's almost cheaper to stay in the datacenter right now because we have that investment.

Joshua Bixby: Yeah.

Geoffrey Smalling: Maybe down the road it won't be the case for us and what we are looking to do is use the cloud for better disaster recovery. So this is the strategy we have sort of got is to have some virtual machines that are off in the cloud and or parked if you will and then you know we pop them up every week to deploy to it and then bring back down just to save cost and you know just have the DR in the cloud for now and maybe we will switch to it down the road. But the barrier right is we have, we don’t have the distributed database model that many of the new startups would have. We have sort of got the legacy, huge database model, big iron and you know getting the eye ops we need in the cloud is very expensive.

Joshua Bixby: Yeah and that’s a very common theme that I am hearing it's exactly that which is if it was new which is not the case for almost of any of the established e-commerce businesses right.

Geoffrey Smalling: Correct.

Joshua Bixby: Then I would do this or if it was a new project or I did it on my own so I am hearing that same theme continuously and actually envy, like this envy of people that you know, they go to these conferences or read these papers on guys who spin up a thousand instances for twenty minutes, pull them all down I mean people are envious of that type of utility costing but can’t take advantage of it yet for themselves?

Geoffrey Smalling: Right, we don’t really need it, I mean we are e-commerce I mean it's not like we are doing huge calculations.

Joshua Bixby: Yeah fair.

Geoffrey Smalling: It's a – you know we have got people knock it a lot down here in the Bay Area that we are probably one of the Bay Area companies still on the Microsoft dot net platform and you know those web servers can really scale out, I mean they are – we don’t have that many web servers like somebody running Ruby might have and we have got you know really optimize the sharp code in it, it doesn’t take much to run a pretty high transaction volume side on it. So we don’t have the need of a lot of VM’s, but what we have done has virtualized everything in our datacenter.

Joshua Bixby: Nice.

Geoffrey Smalling: So we have got the private cloud I guess you would call it in marketing terms and but we have got the private cloud where we have a lot of VM’s, every web service is a VM right now, the databases are physical, app servers and web servers are VM’s and it's really coming handy in a lot of ways you know being able to snapshot a server and bring it up to test or QA or figure out a production issue or -.

Joshua Bixby: Yeah, that’s actually how we run our infrastructure as well for a lot of our dev and also non-sort of appliance based tasks as everything is virtual and it's made a big difference, we are able to handle away more servers per  sys admin and just be much more efficient so I hear here we have a similar infrastructure. Tell me about mobile, because I know this is something that that you know is on the minds of all CTOs, everyone is trying to grapple with, do I outsource it, if I did outsource it do I bring it back in, how important is it. So first on the metric side, has – have you seen a dramatic growth in people shopping for wine either – on a mobile device be it a tablet or phone?

Geoffrey Smalling: So we are mobile crazy here and it's a game changer in a lot of different ways. We invested early in iPad and iPhone apps, iPhone app hasn’t really panned out for us. iPad has done amazing things. So on the iPad, our app converts as well as the website and people use it a lot who like it.

Joshua Bixby: Okay.

Geoffrey Smalling: So we’ve done a lot of investment in the iPad app area. The bigger exciter is the iPad web traffic.

Joshua Bixby: Interesting yeah.

Geoffrey Smalling: So the iPad web, I will just give you the stat I looked up yesterday, 143% growth in November.

Joshua Bixby: Wow.

Geoffrey Smalling: Over last November.

Joshua Bixby: Wow, that’s impressive.

Geoffrey Smalling: And conversion is  good or better than our website and just get into your company performance it's a big challenge in performance.

Joshua Bixby: Yeah it is.

Geoffrey Smalling: Because we have, it's our slowest web browser when you go look at New Relic it's so, it's converting well but it's you know not a great experience so we are really looking to make that a better experience next year, probably not this Christmas but next year because of the growth in the iPad.

Joshua Bixby: Do you now have iPad’s across the test team, how embedded is the mobile experience within your own team?

Geoffrey Smalling: Yeah we have had an iPad across the dev team, but now we have an iPad 2 and an iPad Mini and an iPad Retina just I bought it Monday.

Joshua Bixby: Nice, nice.

Geoffrey Smalling: Yeah we, our iPad app had a little issue with iOS 6, so we have been hard at work fixing it and we just decided oh my god with a hundred and forty three percent revenue growth we would better just be iPad first for a while.

Joshua Bixby: Yeah let's be serious about it.

Joshua Bixby: Not mobile first, but iPad first.

Joshua Bixby: Yeah, well we have seen I mean we published a study on two customers last year, which tracked the amazing iPad growth and it was similar to the growth that you are talking about. I mean just you know a hundred percent plus and more importantly real conversions and real money. I mean this customer was making about fifty cents for every one hundred dollars on mobile in 2010 and in 2011 or 2012 actually they were making seven dollars on mobile of which a big chunk of that was on the iPad so I have certainly seen that trend in terms of just raw conversion and dollar figures, which is as you say changes the game, I mean performance is different, the browsers are different, the capacity of the phones or the tablets are different, it makes you know as we have seen these browsers sort of coalesce around similar standards we just have had that blown up with all these different phones and different tablets that’s for sure.

Geoffrey Smalling: Yes.

Joshua Bixby: Tell me about, tell me about the you know if I give you the crystal ball and I ask you to look out a year what is a higher priority, what's new, what's changing, I mean you talked a bit about that in terms of how you focus on performance, on mobile but what else is in that list for you?

Geoffrey Smalling: Next year, you know this year we did a lot of experimenting on social and we haven’t cracked the code. I think next year we are going to keep trying to figure out how to get you know wine and social, it should be a social commerce kind of play and we haven’t figured it out so we are going to keep probably at it. But for you know things that we can count on it's definitely going to be the iPad website and trying to figure out how we can now that it's growing at a hundred forty three percent, I just was on an airplane flying to Texas for ThanksGiving and I looked around and almost everybody has the iPad, I didn’t see many androids and it's just crazy like kids playing on it.

Joshua Bixby: Yeah.

Geoffrey Smalling: Mom’s playing on it, everybody so it's just like this ubiquitous device almost and we got a, we got to optimize on it and you know we built our, we focused on our M.Wine.com you know a really big effort on the M.Wine.com and it's a beautiful site but it just doesn’t convert very well so it's fast you know we looked at making it the fastest M. that we could we you know have all the content and people just don’t convert so we are going to maybe put that same effort on the iPad this year. We are also impacted heavily from mobile on e-mail, so e-mail is you know still a big revenue driver for e-commerce and now you know I commute so I am waiting for the ferry or the bus and I am standing there with my phone so I am reading all these marketing e-mails on my phone and then I go to the site and I don’t convert so we are trying to think through that problem, not just how to deliver e-mail better to the phone but how to you know convert them better from the phone.

Joshua Bixby: Interesting.

Geoffrey Smalling: And if you think about it as iPhone traffic is thirty percent of your traffic and they are not converting, your conversion just tanked so -.

Joshua Bixby: Yeah so absolutely and I don’t think it's going down, I mean I don’t see that being a blip in the radar that’s for sure.

Geoffrey Smalling: No it's so – we have got a lot of ideas there and we are going to start testing them. We have tested a few this holiday season and so that that’s the team. We’re also still trying to make the experience better and more and more personalized on the website and this year, we’ve done a lot in that area, you know I have been 10 years and I have always wanted to do this and we haven’t had the team and now we have got a great team that we have built and we are doing amazing things so the site is getting more and more personalized, more rich which makes the performance problem, so that that’s definitely that this year we will continue on with making the site more personnel, making it work on any device and really focusing on not mobilized there but you know iPad or iPhone we unfortunately haven’t seen a lot of Android conversion or android revenue or android I don’t know if we are the only ones but we definitely have -.

Joshua Bixby: I don’t, I don’t think so I mean I think there is a real demographic skew to, a demographic that would be a wine aficionado/connaisseur to an Apple lifestyle and I think they have done an amazing job at capturing that audience so I think, I have seen the same think in other e-commerce sites and it doesn’t surprise me as always it's a pleasure to chat, I appreciate you taking the time out of a very busy schedule to chat about performance and your world at Wine.com. I am very excited for when you start shipping to my house in Vancouver.

Geoffrey Smalling: Oh just one day.

Joshua Bixby: I know, I know that’s not there yet but I am going to be a top customer when that occurs so keep -.

Geoffrey Smalling: We will get it to move down to Seattle you know.

Joshua Bixby: I could, actually I get a lot of stuff shipped to Point Roberts which is just across the border but forty five minutes from the house and I drive down, I drive back actually I was just doing this the other day, drove back and they said do you have any declare, I am like yeah I was in, how long were you in the United States, it's seventeen minutes. Do you have any declare? Yes I have four hundred dollars worth of Christmas gifts and they just always let me pass through so I might get a case of wine delivered there and see if they let me get through, I will talk to you about that discount code, you know the one that gives me like sixty percent off, I will – you got to hook me up.

Geoffrey Smalling: All right.

Joshua Bixby: Thanks man, take care have a great, have a great rest of the year and we will connect next year for sure.

Geoffrey Smalling: Yeah keep monitoring our site for us.

Joshua Bixby: Thanks man, we are on it don’t worry.

Thanks for listening and thanks so much to Geoff for taking the time to visit with me. I encourage you to check out webperformancetoday.com/podcast to hear from some of the other really smart people that I have had a chance to sit down with. If you have a suggestion for a future podcast or you want to drop me a line feel free, joshua@webperformancetoday.com have a great day.