Today on Build, Maggie sits down with Justin Dilley, Head of Product at FullStory, a customer experience data platform. Before he joined FullStory, Justin jumped into the world of product management after business school first at Amazon where he worked on their payments and Kindle teams. Then it was back to the east coast where he joined Home Depot’s mobile team before making the move to FullStory.
Together Maggie and Justin discuss the “why” behind what we build as product people and how to organize your product team for success. Tune in to the full episode.
You can get Build on Apple Podcasts, SoundCloud, Spotify, Stitcher or wherever you get your podcasts. Or listen to the full audio version below ?
Like this episode? Be sure to leave a ⭐️⭐️⭐️⭐️⭐️⭐️ review and share the pod with your friends! You can connect with Maggie and Justin on Twitter @maggiecrowley @JustDilley.
Subscribe & Tune In
Full Transcript
Maggie Crowley: So we’re back. Welcome to Build, this is Maggie and today I’m really excited because I have Justin Dilley, the head of Product at Fully Story on the show. If you don’t know, Full Story is a magical app that I actually used in the past that captures all of your customer experience data in one place and reveals the truth about their digital experience. So Justin Welcome to build.
Justin Dilley: Thank you, Maggie appreciate it. It’s great to join and be able to chat with you about everything related to product management.
Maggie: Awesome. So, as we talked about, we’re going to dig into two topics today specifically understanding the why behind what we’re building as product people, as well as how to organize product teams for success. But first I’m curious, Justin, how you got into product and how you made your way from Amazon, all the way over in Seattle, to the complete other side of the country to Full Story in Atlanta?
Justin: Yeah, no, it’s a really good question. I would say, like a lot of product managers, I kind of fell backwards into product management. So, starting out my career, I started my career in IT and I always knew that I wanted to be close to technology, close to customers, close to the actual product. And so I started in IT to be close to technology, realized that I was not going to be a good fit to be a long term career IT professional, so I went back to business school and got my MBA as kind of a transition point to get another job.
And so Amazon came calling, and I went out to Amazon really not knowing much about the company. I mean, I knew about the company as a customer, but it certainly wasn’t the big behemoth that it is now. And I started on the Amazon Payments team, they had a product that is now Amazon Pay, which was Checkout by Amazon. And I started in kind of a business development, general management type role, just basically helping them with a lot of the go to market motions. And then I ultimately transitioned into product management from there, and so I was more or less asked to join the Kindle product management team as the Kindle had just launched shortly after I joined and they were looking-
Maggie: Oh cool.
Justin: Yeah, smart product managers to help out, and so I just fit the bill in a logical way. So that was kind of my transition and starting point into project management.
Maggie: Cool, so then how did you go from the West Coast all the way back to the East Coast?
Justin: Yeah, yeah, good question as well. I took a number of product management roles in Amazon. Amazon’s one of these companies where you can do a lot of very different things, and still work at the same company; and so I bounced around to a couple different businesses. I mentioned, I helped lead and build the product management team for the Kindle Fire, and then I ultimately transitioned over to the Amazon Dash team and helped them with the Amazon Dash service.
And then, honestly, it came a point where my wife and I, we’ve got two little kids, and so we wanted to move a little bit closer to family, and so we started just to do some research on “What are some interesting locations where there are interesting companies doing the kind of work that we both do?” And so I ended up reaching out to a couple companies, one being Home Depot, and I met my former boss there and we just really hit it off.
And I was really blown away by Home Depot’s passion for growing their team and technology, and the chance and the opportunity to step into kind of a general business role while also still having my hands on the product. And so what I did is to transition from Amazon to Home Depot to lead their mobile app team, and it was just a really great opportunity for me to work at a really interesting, innovative big company that is growing their technical and digital chops. And so I felt like I could make a big impact and work on a pretty cool, interesting product.
Maggie: Right, you had an interesting story about how you found Full Story as well, right?
Justin: Yeah, yeah and again in classic fashion, is the same way I fell backwards into-
Maggie: Right.
Justin: Project management. I serendipitously found this role at Full Story, so one of the co-founders of Full Story, his daughter and my son happened to go to the same school here in Atlanta. And I was at a kid’s birthday party, and our kids were playing on the playground and we just started chatting and we just really hit it off, as well; and he mentioned that they were looking for someone to kind of come in and help them build up their product management function. And I was a little skeptical because I had never really worked for a startup, I had always worked at mostly big companies like Amazon and Home Depot, and so I was skeptical and then I came in and I met the rest of the kind of leadership team; so the other two co founders, and then the VP of engineering.
And I was just completely blown away by the team and the technology, and I have a real passion for this space; I care a ton about customer experience. And the idea of being able to work at a company that is delivering software to help companies create better experiences for their users, all of those three things combined was just kind of a no brainer opportunity for me.
Maggie: Was the jump from big company to startup as dramatic as you thought it would be?
Justin: Yeah, it actually wasn’t. I did all my research, and I have a bunch of friends and colleagues that had worked at startups before and you-
Maggie: Right.
Justin: Always hear about kind of just the crazy dynamics of being at a startup, and how fast pace everything is, and how uncertain everything is. And, certainly, there is an element of that at any real startup, but kind of all of the negative things that you hear about working at a startup, I have not experienced at all at Full Story. I’ve gotten kind of mostly the excitement of trying to define a market and push our product forward as, certainly, outweighing any sort of concerns that I have related to “Working at a startup.”
Maggie: Yeah, I think the same thing about working at a startup myself. I find a lot of those things that someone might call a negative to be actually part of the positives, because if there is chaos, that means you get to do something to figure it out; you have the opportunity to change things, you have a lot of impact. So, people who might have been comfortable in a big company who come to a startup might not see those things as positives, but I think that’s part of the magic of working at a startup.
Justin: Yeah, and the idea that you can both be really connected to the business, and also extremely connected to the product, and being able to sit in the center and say, “If we deliver on this set of features, this can actually materially push the business forward.” Is something that, at a bigger company, that connection is just a lot further apart, and so you as an individual product manager, it’s harder to see how what you are doing impacts the business in a pretty material way. Versus at a smaller company, and certainly at Full Story, being able to directly connect to the things that the team is working on with the business and the growth that we’re seeing is really, really great.
Maggie: Right. So then I guess that brings us to our first question which is, when we were talking, you mentioned that product managers and product leaders might not be spending enough time on why we’re building; and so, of course, part of that’s that connection to sort of business results. But what made you sort of come up with that hypothesis, and why is that something that’s been on your mind recently?
Justin: Yeah, it’s something that I think a lot about, and I think it really stemmed from joining a company like Full Story that has so many great ideas, and has an incredible team thinking about a number of different things that we could do. How do we prioritize all of those things in the most pragmatic way possible that does move the needle on the business, while also satisfying and delighting our customers?
And I just think back to my own time as a product manager and realizing that I didn’t really go into a lot of features, or projects, or products with enough ammunition around kind of why we’re doing what we’re doing, and then, how should we even think about prioritizing that very specific thing? And so just doing a lot of research out there in terms of really trying to solve problems effectively, it is so clear to me that we, as product managers, spend a lot more of our time in the kind of execution phase of a particular initiative.
And so the design, and the development, and the launch milestones, and then once you get the product shipped out you iterate on it and we don’t spend enough time thinking about taking a minute to really pause and say, “Why are we doing this, and who is this actually for, and how will this actually help them?” And then how does that help us, as an input, to being able to ruthlessly prioritize all of the things that we’re working on?
And so no matter if you’re a big company, or a small company, or a small team, you’re always going to have to prioritize those things, and if you don’t have a strong sense of why, then it’s really hard to effectively prioritize the multiple things that you’ve got on your plate.
Maggie: Why do you think that people aren’t doing that? Because I would imagine that most PMs, they want to do that or they think they are, maybe they think they already know that or, what do you think is getting in the way of us being able to spend that time?
Justin: I think it’s really easy to get on the feature factory treadmill, and the idea that you, as a product manager, or you as a product team, you’re delivering value by the number of features that you’re churning out, and then how are those features impacting the business; I think it’s easy for us, as product managers, to get stuck on that treadmill a bit. One, because I think it’s really hard to define success, and there’s not a lot of great tools and support out there about “What is the scorecard for a product manager, and then how do we hold ourselves accountable for that and how does the organization hold us accountable for that?”
I think what you end up falling back on is just “How many features can we get out the door, how quickly can we get those features out?” It is really well intentioned in that there’s a problem, you want to solve that customer’s problem, and you think you have a feature that is a fit to be able to solve that problem. And so you end up, naturally, falling into this rut and you don’t take a moment to step back and say, “I do see this problem, but why is this a problem?” And really trying to dissect that particular problem from a number of different angles.
Maggie: Yeah, I think a lot about this topic we’ve been talking here just about, recently, which is management by proxy, and you can apply that type of thinking to any area of the business. But what I’m hearing you say that I’m thinking about, it’s hard to know if we’re actually solving the right problems and, and I think, generating the outcomes we’re looking for for our customers. And so instead of trying to figure that out and measure it, were just measuring the features we build, assuming that we’re going to get those outcomes.
Justin: Yeah, yeah, that’s exactly right.
Maggie: Yeah. As a leader and head of product at Full Story, how are you, evaluating your PMs? Have you come up with that scorecard, do you have an idea of how you’re going to solve the problem?
Justin: Yeah, so this is a little bit of don’t necessarily preach what I practice in all cases, because I certainly have my own faults as a product manager, and I tend to try and fall back into that track; and so I actively try and pull myself out of that rut fairly frequently. We do have a pretty good scorecard process here at Full Story in terms of “How do we hold the entire product team accountable for what we’re building?”
And so the entire product team is really made up of product management, design, and engineering, and so you’re not as specifically focused on “What did this product manager do?” Per se, but just as a team, do we have the right metrics and measurements in place to say that we’re making the right decisions for our customers? And so the obvious things like engagement or conversion rate are helpful things, but you have to start to look at some other additional things as it relates to how happy your customers are.
I mean, one of the great things about working at Full Story is that we use Full Story to understand and empathize with our own customers. And so it’s hard to wrap something like that up into a dashboard, but we’ve got a process built in to understand how our customers are experiencing the features and functionality that we’re launching. And so that’s just built into kind of our processes we release in a feature. And so, there’s a couple more strategic and tactical ways that we do that here at Full Story.
Maggie: One of the things I think about is, if we can measure the outcome for the customers and whether we’re actually creating that can be sort of hard sometimes, I think, to talk about internally. The sentence is that you have to say to describe the thing that you’re hoping to get for the business and the customer, can take so much time that it’s oftentimes so much easier just to say “Oh we need to build feature X.”
Justin: Yeah.
Maggie: Just a shorthand, and I always think about how it’s easier to say the shorthand even though what I really want to talk about is the outcome, but that’s just an awkward way to talk as a human being. And so, sometimes I think that that’s one of the barriers in the way of me, as a product leader, describing the outcome I want to create, because instead of telling a paragraph every day, I just want to say “Oh, can we just build this thing?”
Justin: Yeah, that shorthand concept is a really interesting one in terms of how you rally the team and rally the organization; that becomes the easy rallying cry to get people excited, internally, about what we’re doing. But it’s not, necessarily, as aligned to customers as I think, as product managers, we really want to be. And so even just taking a bit of time and defining “What does success look like for this feature? What are the key metrics and key things that we care about as it relates to this specific product or feature?”
And taking the time to write those things down at the beginning of a feature arc, I think is just as important as being able to then measure that impact. And so, you may not always be right on “Hey we want to increase engagement by 12 percent by launching this very specific thing.” But just the act of writing it down and then holding yourself and the team accountable to something, and then adjusting, as necessary, in terms of “Why did we or didn’t we reach that metric?” I think is a really helpful, grounding exercise and gets you more into the why and more into the outcome focused thinking about things from the customer standpoint, and then working backwards.
Maggie: Right. What advice would you give to maybe a PM who’s more junior, who might feel like they are being handed things to work on? Right, there’s definitely places, and I’ve been in this situation in the past, where by the time an idea gets to you, it’s sort of the Y has already been established and you just have to execute on something. So if someone’s in that position and they feel like maybe they don’t understand the why, how would you coach them to bring that to their team, even if they’re not the one who’s sort of in charge?
Justin: Yeah, I think, even if you don’t necessarily have all of the details around the why, just coming up with a customer problem statement that you feel is specific to the long analysis that maybe was or was not done, and in your head and in your team, getting everybody aligned and rallied around “What is the two to three sentence problem statement that feature X is going to address?” And if you’ve already been handed the feature, then it’ll be easier for you, as maybe a more junior product manager, to at least take the time to write down the problem statement. And then write down some of the key measurements of how you’ll, ultimately, define success.
And then maybe, most importantly, is to understand how to think about the priority of this new feature that you were just handed down, versus all of the things on your roadmap. So having and taking the time to come up with a framework that works for you, that works for your business to evaluate “How does this fit in with all of the other things that I have going on?” And if it is the most important thing, well that’s obvious; you should just do it. If it’s not, then how do you start to have the conversation with your team or your leadership about the trade offs that have to be made in order to get this specific feature done?
Maggie: Yeah, yeah. I think that’s something that people I work with struggled with this in the past which is, they have a situation and they think that they’re making the wrong trade off, but they feel they have to make it based on leadership. And then the ways and methods in which you can sort of bubble that back up and sort of highlight the trade offs and say “This is what I’m going to do, this is what you asked for, but these are the other factors at play.” And I’ve seen PMs struggle with that over and over again.
Justin: Yeah, and even the simple act of saying, “You want me to do this, we all think this is a great idea, but we’re not going to be able to do X, Y, and Z as well.” And you’re kind of pushing the prioritization discussion, in some ways, back to leadership or someone on your team that ultimately push this down, is an interesting kind of Jedi mind trick to get them to say, “Is this really the most important thing that we need to get done right now? I told you those other two things were actually more important, so let’s actually stay the course and stay focused on those two things and don’t let this new kind of out of left field feature request be the most important thing that’s going on.”
Maggie: Right, which I think brings us to the second topic which is how to organize a team to be effective. So when we talked on the phone about the trend that we have that we see in product where everyone wants to be set up into small, decentralized teams that execute on a portion of the product, think that sounds really great; I know it’s how we’re organized at Drift, and I think it’s how you’re also organized a Full Story. But, where do you think that kind of came from and how have you at Full Story figured out the right operating model?
Justin: Yeah, that’s a good question. What I hear most frequently about where it came from was this excellent, but maybe a bit dated, video from the folks at Spotify.
Maggie: Right.
Justin: And I think their concept is much broader than decentralized teams and, I think, they pioneered a lot of the thinking on structuring a product organization. I’ll also say from my experience at Amazon, while not the same as Spotify, the idea of the two-pizza team concept that “We want to organize around small teams because you can make faster and higher quality decisions.” I think that is now starting to become the defacto standard when you’re creating a product team, or as your product team scales; thinking about creating these sub-teams or smaller teams; and we absolutely do that at Full Story.
So at Full Story we call each of the organization’s ‘families’, so there’s the product family and then within the product family, we have these sub-families that have clearly defined and separate missions; which, again, I think is a really helpful thing as you start to decentralize your teams and your decision making. Having everybody, both within that small team and then across the family and across the organization, if there’s a really good, defined mission then that can be used as a way to really move everything forward.
So in terms of “What do you work on, and why do you work on it, and how do you think about the next thing and the next initiative that you might want to take on as a small team?” And I think the biggest benefits are that you can make decisions at a smaller level because you’re closer to that area of the product, that segment of your customers. And by trying to get your teams closer to customers in these smaller units, they’ll have more depth as it relates to a particular area of your product, and they’re actually in the best position to make those decisions.
And so I think that’s the biggest reason that I’ve seen in the past, and that’s how we’ve tried to organize our teams at Full Story is for those quick, high velocity, high quality decisions about whatever we’re working on a Full Story.
Maggie: Right. But then one thing that I struggle with is, how do you as you scale and start to scale that model, how do you make sure that those teams are staying aligned? Because I think it’s really easy when you think about “Okay, there’s this two-pizza team, and they’re focused on this one area of the product, and they’re completely autonomous, and they’re just crushing it. But then when you step back and you look at the full user journey throughout your product, that’s made up of a bunch of different teams; how do you keep them aligned so that the overall product experience is still coherent?
Justin: Yeah, it’s a really good question. The way we’ve tackled that at Full Story is that we have these sub-teams who own a piece or portion of the Full Story app experience, and then in addition to, we have kind of what I would determine or what I would call kind of more of your breath product teams; and so these are the folks that are looking at, specifically, those workflows.
So where you might have one small team working on an individual feature, we’ve got another team that’s in parallel thinking more broadly, but not as deep in one area, and they’re the ones that are kind of thinking through that core application experience, and thinking through the end-to-end workflows, and they become kind of the connective glue between each of the individual families and sub-families at Full Story. They become that connective glue to make sure that things aren’t being done in complete isolation with no regard for what the end-to-end experience looks like; which I agree, is really easy to do as you start to create more of these teams.
And so the way we’ve addressed that is to just have a separate team that does nothing but really think about that end-to-end experience, and then work across those smaller teams to make sure that the things they’re working on get pulled in to become a cohesive product.
Maggie: Right. Yeah, that makes sense. Another thing that we’re doing is also bringing in more design systems that help us be, at least visually, more consistent across the board, so it does look like one product even though we have very autonomous teams sort of running in all different directions.
Justin: That’s good validation, we are just embarking on that mission ourself here at Full Story to come up with kind of standard components, and a standard design language, and design systems to help support that across our organization as well.
Maggie: Yeah, I’ve been part of these efforts in the past, and prior to the one that we’re doing right now, I think it always felt like “Oh, we’re going to do another design system, there’s going to be a style guide, and we’re never going to look at it and whatever.” But then this one, I think, what’s been really great is to see “Okay, when we are using the right components, the speed to execution is much higher.” And so now everyone says “Okay, well can we get that into the system, because I want to use it because it’s going to make this go faster?”
Justin: Yeah, it’s the carrot and the stick model. What you’re talking about is describing as I would describe the carrot of, “Hey, your life now just becomes a ton easier because we have these design systems in place.” And then people just, organically, will flock to that versus more of the stick around, “We’re creating this brand new design system, and you’ve got to use this style guide, and it becomes outdated the moment you publish it.” And so you have to have both, in some ways, but it’s easy to forget about the carrot approach of “Hey, this is going to make your life easier as a developer, or designer, or product manager.”
Maggie: Yeah, absolutely. So then I want to switch gears a little bit, because one thing that I try to get from everyone I have on the show is just selfishly advice on how to be better at our jobs. So, what’s your advice for people who are looking to grow in product, from product to product leaders and what do you look for when you’re evaluating people to join your team?
Justin: That’s a really good question. I think the biggest transition from being an individual product manager to maybe managing a small team of product managers, is that you kind of have to take your hands off the wheel in terms of the day-to-day product management. And you go from this really active role in the product being able to transition to be more of a servant leader to your team, and helping them think through some of their own particular problems. I think, that’s really the key transition as you move from an individual product manager into a manager of product managers, is to have the ability and skills to take your hand off the wheel of the individual execution, and be the one that can take the step back and be that servant leader for your team, and really dig in and try and help them on their journey as product managers; that is the hardest part about the transition.
And what I would say is try and work with your team, work with a mentor, a leadership coach, to really change your mindset into being more of that manager of product managers; and so that’s my biggest advice as it relates to that. And then just in terms of product managers, I will again reinforce the take the time upfront to really understand the direction you’re going and why you’re going on that direction. And so I’ll use the factory line analogy, if you can fix something on the factory line, upstream, before it gets into a mode, things become a lot less expensive.
And I think the same thing is true in product management, the more time you spend deeply defining the problem, deeply thinking about how to prioritize that particular problem, the faster the kind of development and iteration goes, and I hope that it leads to more success. And so challenging you as a product manager, and certainly myself, I feel like I’m challenging myself all the time to think about that, is just take a minute and pause and spend more time than you’re spending right now. And it doesn’t have to be a huge amount, and I don’t think this is a totally shift away from what you’re doing today, but take more time that you’re spending now and start there, see if that is helpful for you; I think and I hope that you will.
Maggie: Yeah, I love that, that’s something that I’ve found more and more important in my career is being able to tell the stories of the products. It’s not just “As a person, I want to X so that I can Y.”
Justin: Right.
Maggie: But really being able to paint the picture of what’s going on with your customer and why they need this thing, and what problem they’re having, and bringing a little color to it helps that sort of live in my mind, in the team’s mind and helps them execute.
Justin: Yeah, no, you’re totally right.
Maggie: Yeah. Okay, so last question, what are you reading or listening to these days that you think is helping you think differently or execute better, or that you’re suggesting to your team?
Justin: A couple things. I’m a big podcast fan.
Maggie: Okay.
Justin: Shout out to you, this podcast and all podcasts out there. And so a couple of my favorites that I’ll throw out there, one is The 20 Minute VC with Harry Stebbings, it’s just a really good, well done, a very diverse set of venture capital investors, and founders, and product people; and so I find that is a really good one.
And then the other one is Reed Hasting’s Masters of Scale, there’s a ton of really great content from a lot of incredibly smart, talented people that have been through the battlefield. And so I always find myself learning a couple little tidbits from each of those; and so that’s really good.
Justin: 29:03 And then in terms of books, this is very much a non-product management recommendation, but I’m maybe 25 percent of the way through the book Sapiens.
Maggie: Oh yeah.
Justin: And I cannot recommend that book enough. It is such an interesting take on who we are as human beings, and how we got to the place we got to, and why we have some of the tendencies that we have. And again, I’m not all the way through it, but it’s a really interesting, unique read.
Maggie: Awesome. Well thanks Justin, I really appreciate you taking the time to come on the show and sharing everything that you’ve learned with us. I know I’m going to be thinking a lot more about making sure that we’re really clarifying the why we’re spending time on it, and making sure we know exactly how we’re going to measure our success. So again, thank you.
Everyone, leave Justin a review, give him a shout out and the comments, five stars; I’ll take six, but really five stars will be great. And let me know who else I should have in the show, what else I should cover at Maggie@drift.com. Thanks and see you next time.
