Last month, I sat down with Jim Semick of ProductPlan to discuss his “Guide to Product Roadmaps” book, and to delve into questions like:
- Why do roadmaps turn into big ol’ list of features?
- How do you teach junior PMs to think strategically?
- Are roadmaps really the “root of all evil”?
- Can product managers also have passing tests?
Jim details his views on how to avoid some of these perils by building your strategy chops, and by using an accessible roadmap that stresses vision and adaptation rather than a prescriptive list. It’s worth reading twice, so get settled with a fresh cup of coffee and dig in!
JC: Why do roadmaps tend to slip into a list of features? We know in our hearts that that is not the direction we want to take it, but it just seems to evolve that way over time.
JS: I have some potential reasons why that happens. One is, I think that there is a misconception that the product manager is “The Keeper of the List.” The other misconception is that the roadmap is a list of features. It is not. The roadmap really should be focused on strategy and on the why. It should be boiled up to a higher level of themes and strategic goals that you want to accomplish.
Also, I think there are a lot of people in product management that do not really have formal product management training. Really, there are very few resources out there for becoming a qualified product manager. Sure you have Pragmatic Marketing, and other resources like that, and there are plenty of articles around it, but there is really nothing for people around becoming a product manager. You have a lot of people with a project management background who kind of fall into the role of being a product manager, and there is a huge difference.
A Project Manager is The Keeper of the List. They are also the keeper of how many person hours it will take to accomplish those features.
Product Management is a completely different skill-set. It is being a politician. It is being able to say “No” tactfully. It is being able to be a visionary, adhering to the vision of the product, and coming up with innovative ways of solving the problems.
That is one of the reasons why you have a lot of feature list driven roadmaps. The other thing is that as a company matures, I think there is a tendency to get into “maintenance mode.” The product may have been very innovative in the beginning, but then it kind of boils down to the list of features, or performance improvements, or tasks that kind of keep the wheels on the bus, rather than innovation.
JC: How do you teach strategic thinking to junior product managers?
JS: Yeah, that is a great question. I think you are bringing up a really great point about the junior product managers, or associate product managers being in that routine of just making sure that the scrum or team keeps their queue filled. You are needing to feed this monster that needs stories and features. I think there is a real tendency to want to feed it features, so I think that reinforces it.
I think that it is the responsibility of the VP or the Director to give even the associate product managers opportunities to engage with customers, and think strategically in an innovative way about what is going to solve problems for those customers.
I think that if a junior product manager’s job is 40 or 50 hours a week of being inside the building, and sitting at their computer, and managing the backlog … well that is only going to perpetuate the problem.
I think that a certain amount of time needs to be allocated by the VPs so that these folks can do Product Discovery, because that is the stuff that is really exciting. That is the stuff that is going to get their creative juices flowing.
JC: When we were at the same conference, and I have seen these quotes come out on Twitter often where Marty Cagan will say something like “Roadmaps are the root of all evil,” or things like that. I am guessing that if we were all in the same room, we would actually probably see it all the same way.
JS: We would agree.
JC: Where is that passion on his side coming from, do you think? What is he responding to?
JS: That is a great question. I think he is responding to bad product management, which is in many, many companies. The executive team or product team will sit around a table and deem what the right thing is to build in advance. “We are going to do this particular feature,” let’s say. Then that feature winds up on the roadmap, and then from that point forward the train is moving, “We are going to do this thing.”
Marty’s point is that if people are looking at that roadmap, there is a presumption that that roadmap is the “right” thing to do. That is where product roadmaps are the root of failure because that process that they use to get the right features in the queue is broken. Then it is on the roadmap, broken down into stories, scheduled, added into the “agile” backlog, which this is not really agile. That is the problem. You wind up at the end of the day with products that are ineffective, that do not really solve customer problems, and you put the company and product at risk.
I think that is where his frustration comes from, and the better way that I have seen it done is doing this in the customer discovery process–making sure that you are solving real problems.
The reality is that you need something in order to communicate the decisions that are being made about the direction of the product. That is going to happen in any company, and that document is the roadmap.
Whether you communicate this roadmap through a spreadsheet, or through a PowerPoint presentation, or through product roadmap software, it is going to get it done. The stakeholders need to know the direction.
JC: Or you can just stand near the water cooler. Something is going to happen!
JS: Yeah, so I think that is probably where his frustration comes about, and I am planning on writing about this, this topic of “are roadmaps really the roots of product failure?” I do not think that roadmaps are the root of product failure, but bad process and customer understanding, that is the root of failure.
JC: How do you impart best practices within your product? How prescriptive do you get?
JS: It is a tough one. I think the ultimate vision for our product, for ProductPlan, is that there are certain best practices that we can enable within the product that will help them, because so many people are looking for best practices.
There are so many new product managers out there that are looking for the better way of doing it. They want to find the “right” way of doing it, but there is no actual “right” way of doing it.
They are looking for it–because there is a lack of product management training out there–they are looking for this best way of doing it, so they looked at products like ours for that. We build in some of that into our product, but the truth is that our product is still … there is a lot more for our product to do before we fulfill that ultimate vision.
It is a double-edged sword because there is a certain amount of flexibility we have to have in our product that adapts to the way that people want to use them. Right now, our lack of prescriptive best practices is actually, for a lot of our customers, a positive.
JC: You have customers in HR and marketing as well, right?
JS: We find that all the time. Our product starts out on the product team and they use it for product roadmaps, and then the IT team sees it and they started adopting it, and then at some point the marketing team says “I love that, I want that too.” Our product is flexible enough that it works to all of those use cases, but I think it is our job as product management advocates to start building in some of these best practices.
JC: When are dates on a roadmap timeline appropriate, and when does it cause a toxic situation?
JS: That is a great question. In fact, we had just conducted a webinar this morning and that was one of the audience Q&A questions that came up. I do not think the dates in themselves are bad for certain situations. There are a lot of organizations that are mature organizations, they are Fortune 1000 companies, and their executives need to know what is happening in Q3. Then there are all these other departments that are dependent on them to deliver something in Q3. So, I think as organizations get more mature, product lines get more mature, that there is a tendency towards that. Especially when you have 100 different stakeholders that need this information. I think that is a natural thing, whether we think that is right or wrong.
Not all organizations are agile. There are still a surprising number of these big companies that are waterfall. They may say they are agile, but they are waterfall. That model fits for them. The danger happens when a company is sharing information with people that should not have it shared with them. For example, you are promising 12 months out to the sales team what features are going to be delivered, and then the sales team is using that to close deals. That is dangerous.
Or you are in a smaller organization where there is no possible way that you know what is happening three-quarters out, no way. In our organization, which is smaller, there is no possible way. This is not just because we are agile, and we continue to re-prioritize, but because the whole world will change in a few quarters. We are going to have new competitors on the market, or our teams are going to change in size, and so our velocity is going to increase. Or the thing that we thought was really important, that was the needle-mover, and suddenly we are getting different feedback from our customer base about what is really important, and we are recognizing it. Or new technology comes along that makes a new feature feasible.
JC: We would not be doing this if there was not that level of uncertainty because it would have probably been figured out already. (Laughing)
JS: Yeah, exactly, and what makes us successful is that we are able to adjust to the changing needs. In our case, dates are really useless. Maybe not dates from a sprint standpoint–certainly our engineers need to know what is happening in the next couple of sprints. In that context, dates become important. Dates two quarters out, I don’t know. It is important for them to understand the priority, and the vision and the direction, but not assign dates to that.
I think there is a movement towards not having dates, and often moving up and moving on to more of a combination style roadmap, which from what I have seen, is much more prevalent in smaller organizations. When you get to some of the larger organizations, and our customers are often Fortune 1000 companies, dates are just the nature of it. Sometimes we will see roadmaps that are 18-month or two-year roadmaps, and that is the way that they plan. We just have to accept that I think.
JC: Often engineers feel that the product managers are not really being held to the same standards. What are their “passing tests” ? How can you use roadmaps to instill that sense that we are all in it together, and we are all measuring ourselves against some common goal? Can the roadmap help with that?
JS: Yeah, great question. I think roadmaps can completely help with that.
I think that measuring the development team, and measuring things like velocity, measuring the number of bugs and things like that, those are all important things to do. When it comes to the product, those are not the things that create a good product.
If your velocity is fantastic and you have zero bugs, it does not necessarily mean that you have a great product that is loved by customers.
You need to start measuring the things that actually do matter, and that should be represented in the roadmap. The roadmap should be tied back to strategic goals, and those strategic goals are customer-oriented metrics, business-oriented metrics that matter.
Things like reducing churn, that is a great one. In our business, that is something that we want to focus on, and a lot of other software companies want to focus on that. Things like customer growth or customer satisfaction. Improving NPS, things like that. Those are the things that really matter, and should be included on the product roadmap so that everybody can understand “This is why I am doing this feature. I am not doing a feature just because I think it is cool, I am doing a feature because I think it is going to move the needle in some direction.”
Maybe my strategic goal is to move into a new geographic area, so how is a feature going to help with adoption in this new geographic area? That is what gets represented in the roadmap, and that is what should be measured, in addition to all these other metrics that engineers are measured against. Everybody needs to know that we are going to improve NPS from a 50 to a 60, or whatever the number happens to be. Then people can see where they fit into that.
JC: How do you make people feel the roadmap is accessible for them to be able to chime in with ideas?
JS: I think a lot of product managers come to the table with a solution for the problem. They already know in their mind what this feature is going to be. That’s a problem. I saw features work out very well at AppFolio where you bring to the table this problem that you want to solve, or the customer job to be done. Then the engineers will come up with a solution, because they are smart, or because they know of a different way to solve it technologically that the product manager may not know about.
I think that combination works out really well, where the product manager’s job is to bring the customer story, or the customer job to be done, or the customer problem that needs to be solved to the table, and then the engineers can help figure out “Okay, what is a good solution for that?”
I think a smart product manager is also technical, and they kind of understand what it is feasible, and what can be accomplished. I think there is a good mix there, and then I think a healthy product team invites collaboration with those folks. When I have been validating new products in the past, bringing somebody in from the engineering team to listen to those customer interviews and the discussions that happen after, coming up with possible solutions … that makes for something kind of magical. When it is not just one person in their head, thinking of what the solution is. I think that sort of collaboration works out really well.
JC: One thing that struck me from the book was that you mentioned you had a matrix of value versus complexity. You chose the word “complexity” versus the word “cost” which was interesting. When you are thinking about costs or complexity, how deep do you personally go in trying to think about what the ramifications and long-term costs of that feature might be?
JS: I think that if you think short-term, those costs are very apparent, in engineering time.
JC: Or opportunity cost maybe?
JS: Yeah, opportunity cost is a good one, I like to think about that. “There is only limited bandwidth, so what are we not doing because of this?” There certainly are ongoing costs, like service costs. Then there is another cost that I think that people overlook, which is this … I am not quite sure what to call it, but it is this “feature-bloat-debt” that you accumulate.
JC: I call it the psychic drag–like you just have more to think about. You have more to juggle, and every conversation gets a little bit more complicated. You probably have that with sales, right? If we add something new, it is one more thing that you have to talk around. It is kind of interesting.
JS: Yeah, and so there is this feature and unless it is immensely valuable, it is kind of a drag on the organization in small ways, and it is also a drag on the customer experience. That is part of this cost I think. It is not just things like “We need to write another help topic,” it is not just that. It is also we are building in all of this stuff that every single time you do a release, now it needs to be tested, and things like that.
JC: So if you’re on a call with someone for the first time, what little hints do you pick up on that suggest to you that, okay this is maybe a little bit more of a forward-thinking product team?
JS: Yeah. You know it’s funny because I preach strategy. Being strategic, tying goals to strategic goals to the roadmap and so on. But this is easy for me to say, especially if you’re dealing with a large organization. Some of our customers have 20 or 30 different product managers that they’re managing. These are product managers of all different skill levels. The first thing I notice is whether there is a formal product management role. Is there a structure in place for it? A lot of organizations don’t. They have business analysts or they have program managers that are actually project managers, and I think that there needs to be kind of a formal function.
JC: Do you guys use personas, and incorporate them into your roadmaps?
JS: We don’t do that formally. Perhaps we should, but right now the people who make the product decisions are a very small group. Those folks have been involved since day one, from the very first calls. So I understand, intimately, that persona and I can draw up distinctions between the different personas. I suspect as ProductPlan grows, then the product team becomes bigger, then that sort of formalized persona process documenting “jobs to be done” becomes more important.
I think that if things started to go awry for some reason, maybe we start building features that don’t really resonate or the company is growing quickly, or … those sorts of things. Maybe there’s a disconnect between marketing and product, or these things that happen as a company tends to grow. I think that if those things were to happen then we would adopt, more formally, some of these processes. But, everything for us right now, that’s going swimmingly. I feel like, okay, well then there’s not really a problem to be solved here.
JC: When dealing with stakeholders, a healthy level of tension or there’s a lot of guests, how do you make sure that they’re productive versus either falling apart or just kind of satisfying and settling on the kind of least objectionable idea?
JS: Yeah, that’s a tough one because I think that if you have a strong product manager, that strong product manager is bringing problems to the table where there can be a healthy discussion and debate. And then the product manager, then, is not bringing some things to the table and is just making unilateral decisions.
I think PMs need to be strong with what they’re doing. Not that they’re jerks about it. You have to say no in nice ways and be politically correct and all of this.
But I think the product manager should be bringing the things that really matter to the table.
So in ProductPlan we have a feature called the planning board and we’ve talked to several of our customers who used that scoring model that we built into it as a way of driving the conversations.
So, I actually think that there’s something going on there where the fact that there’s this, kind of a third party product out there where everybody can focus their attention on that, and then have a discussion of whether something warrants a two, or a five, or whatever the scoring mechanism happens to be.
That’s a healthy discussion and the product manager can foster that, kind of broker that and mediate it. It encompasses two things. One is that you kind of get opposing ideas, and people’s minds can change about things with information that they didn’t know about. The other thing is that it builds in transparency into the process and it also gets people on board. Once that decision’s been made and the conversation’s been had, hopefully people aren’t rehashing that down the road and they understand why you’re building a certain feature.
JC: Why are product managers nervous about sharing the roadmap?
JS: Obviously someone is saying that for a reason. The roadmap is seen as almost like a liability that can spin everything out of control.
JC: Do you think it’s worth it to be more transparent even if it causes interim pain for the product team? Or can you see scenarios where trying to keep the roadmap kind of tight-lipped inside the team, is that ever worth it?
JS: Yeah, I think that sort of attitude of, you know, I don’t want to send this in email. I think that this gets to Marty’s point. Which is, people are under the perception that the roadmap is like this commitment and that it’s this artifact that once it gets out into the wild, all these misconceptions can happen and people are going to feel like I’m locked into this roadmap. And this is valid because people get a document or a spreadsheet or whatever it happens to be and believe that that’s the truth.
JC: Or, we’re going to be selling that widget in this quarter . . .
JS: Yes. That’s what people want. People want certainty. So, I understand a PM’s hesitation to not want to send it out in email. But that’s also a sign of, I think, maybe a little dysfunction–of not trusting the organization–or not properly setting expectations about what this thing means. That’s one of the benefits of ProductPlan. That it’s a living document that’s always updated and changing. It’s not like this locked in Power Point presentation where somebody a year from now is going to look at it and say, “why didn’t you deliver that thing that you said?”
JC: When you first did the survey in 2015 that you reference in the book, what were you surprised by? What data came back that you weren’t expecting?
JS: I thought it was really interesting how communication was the number one reason why people wanted to have a product roadmap. I need to communicate this thing out. The number one challenge was “I have a problem communicating my roadmap,” and this challenge was probably for all sorts of reasons. But I think that’s interesting. It means that roadmaps aren’t doing their job.
JC: What tensions do you deal with in your roadmap?
JS: I think that in organizations like ours, there’s only so much bandwidth.
There’s always a tension between wanting to improve customer satisfaction or drive more sales with a certain target group, and you can’t do it all.
We’re faced with this question every day. How much of our resources do we allocate to usability improvements that will make people happier? How much of our effort do we allocate to improving performance? How much effort to we allocate to enterprise features that we know are going to move the needle for sales?
This is an ongoing problem, and it’s true that work in any one of those areas will bleed over and benefit the other areas. I know that’s the reality but I personally tend to like to bucket things. Not just do it just because we think it’s a good thing to do and we think it’s going to benefit all these areas. It’s kind of wishy-washy. I want to be very specific.
JC: How do you know when you need to make the bold move?
JS: I think that’s a great question. I think that’s what separates out the really great products from the products that just kind of plod along and just do okay. And I think that’s strong product management, being able to step out and say, “I know that none of our competitors are doing this yet, but I see in the future that the trend is going to be this and I’m going to make this bold move.”
But you also need to balance it with the other realities of releasing features, and improving performance, and fixing bugs. And all these other things that have to happen. But the really good product managers pick one thing, right? And that kind of becomes the thing that they focus on, the thing that’s going to move the needle for the company. And I’m sure they don’t always get it right. At AppFolio there were initiatives like that, that were kind of trail-blazing initiatives that we still didn’t know whether it would actually work. But you’re guided by the things that are important to your customers.
JC: Sometimes I feel we might fetishize the product itself versus just keeping in mind these customers and doing these things. What do you think about treating the product as an entity unto itself?
JS: I think it’s really cool to be really passionate about product, and wanting your product to be unique and different, and done well, and [wanting to have] a great customer experience. But I think you have to start with the customer. Even backing up even further, you have to start with the customer pain. What is the pain that you’re solving? When you focus on that, now you can think of, wow, some really amazing things that you can do that will move the needle. Because your competitors are stuck in the product weeds. They’re locked into this paradigm of, “I now have to add a feature to my accounting package, what is that feature going to be?”
Jim Semick is the Founder and Chief Strategist of ProductPlan, a product that helps you plan, visualize, and communicate your product strategy. Watch our webinar with Jim on building data-driven product roadmaps and don’t forget to download Jim’s latest book, A Guide to Product Roadmaps.