Category Archives: How to’s

Small tweaks, Big Differences: Lowering your flake rate on customer interviews

31 October 2023

Leave a comment

Are you having trouble getting customers to interview for your product? Are too many flaking out and missing your scheduled calls?

It’s a common challenge new and veteran PMs face every week: You want to be interviewing customers, but they’re not happening as often as you’d like.

While I’ve written a bunch about how to interview customers, and how to take great notes, I haven’t discussed the important step of actually getting customers to show up for a call.

I was just helping a client with this issue recently, so wanted to share what I’ve learned helping dozens of product managers over the years. Here’s my best advice for improving your success rate at both getting customers to interview and getting them to show up:

How to get more of your customers to show up for their scheduled customer interviews

You can’t interview customers who don’t set up a time to speak with you.

Or, put more simply: you have a 100% flake rate for the customers who never scheduled a call.

So if you haven’t set up any calls yet, you’ll need to do that first. Here’s where you can start if you’re new to customer interviews:

  • Ask your existing user base: If you have only 10 customers, go interview all 10. When you have much more than that, you can narrow your focus, but it’s always best to start with people already familiar with your product.
  • Focus on power users: Every product tends to have some people that are fired up about using it. They log in more, complete more actions, and may even evangelize it to others. These people are gold. One of the best ways to thank them is to interview them and take their feedback and ideas seriously. They often reveal insights that can help guide you to your next stage of growth.
  • Mix up your methods: Every customer type I’ve ever worked with has been a little different what works best. Try various communication mediums to see what works best (text, email, messages in your platform, Linkedin, etc).
  • Act on what you learn: This should go without saying, but I’ve seen too many PMs miss the forest for the trees on customer interviews. You should be learning new things in interviews that influence your roadmap, improve designs, and fix bugs. Taking action on what you hear is what signifies to those you interview that it was worth their time to talk to you.
  • Give credit and thanks to those that helped: This may seem like a small thing, but it is a big deal. Customers *love* hearing that you built the thing they complained about or suggested. Taking a few minutes to message them to let them know can go a long way, and is often how you then build relationships where customers will volunteer for interviews again and again.

And if you need more ideas for getting customers to speak with or just need to find your first set of customers, I have a post that will give you tons of ideas: 95 ways to find your first customers.

Rules of thumb to remember when interviewing customers:

Before we dive into the tactics to use to get your customers to show up for their interviews, keep in mind a few important rules of thumb for these interviews:

  1. It’s hard to start from scratch: If you’ve never interviewed customers before, it will take time to get your customers used to talking with you. The first ask will have the lowest response rate, and then it will get better from there.
  2. Your industry matters a lot: B2B customers are typically much easier to get calls with than B2C (consumers), but that varies, too. Think about your customer and how often they’re by a computer or phone and what their schedule is like. The more they are available and near a phone/computer, the easier to get calls, while the less they are, the harder interviews will be. So yes, it’s harder to interview college students than an IT department. But it’s also harder to interview a construction worker than a retiree.
  3. How you ask matters, too: Being friendly, explaining what’s in it for them, and making it as easy as possible makes a big difference, too. You should experiment on the best methods and wording to ask for interviews just like you’d test other messaging in your product and emails.
  4. Ask the right people: Make sure you know who you want to talk to and why. Power users are more likely to speak to you, as are those who are most motivated to use your product (like a sales rep mandated to log their calls) and have been using it. Asking people who never log into your product will both be harder to get on the phone, and have fewer useful insights (unless you’re specifically trying to learn how to activate users of their type).

All of this is to say that practice and persistence makes perfect; it takes time to master the entire process of interviewing customers, but the rewards are huge. You’ll make better product decisions, improve your product, and gain insights that can improve every team and department.

Fixing the flake rate: How to get more customers to show up to your interviews

No matter what you do, some customers won’t show up to your interviews. It’s an unfortunate fact of life. Emergencies come up, conflicts hit their schedules, and some people flat out forget.

Yet, there’s a tolerable amount of flaking, an expected amount of flaking, and then amounts that make it too hard to do your job, and leave you and colleagues feeling frustrated.

The rule of thumb I like to use is that in an easy to reach, engaged audience in B2B, 10% is a good flake rate (meaning 1 in 10 calls scheduled is a no-show), while in B2C and other difficult industries, a 40% flake rate is not unusual.

Regardless of your industry, we want to make that number as low as we possibly can. These tactics will help you accomplish that.

Oncehub and Calendly are your friends.

Before we get into the nitty gritty details, it’s important to call out the most important tool in the process of scheduling customer interviews: your scheduler.

If you work in tech, you probably use Calendly, and if you’re really old school, you may remember the old days of Tungle, who invented the tech back in ~2009. I personally use a tool called Oncehub, which I find more robust and customizable than Calendly, but both will work.

The key with either is that they automate parts of the process you don’t want to have to do yourself like:

  • Proposing times you can make based on your current schedule in any moment.
  • Creating a calendar invite and sending it to your interviewee and any colleagues joining.
  • Avoiding any double bookings no matter when your customers are ready to find a time.
  • Handling reminders, and easy rescheduling for the customer.

And if you’ve ever used them, I’m sure you can think of other benefits as well.

All of their functionality turns many frustrating back and forth emails into a few quick clicks and you’re done. And unless your customers are egotistical VCs on Twitter, they’ll be grateful to have something so simple and fast to use. (…and a dirty secret about VCs is that the same people who *hate* calendar scheduling links have assistants who *love* to use them.)

Tweaks to Your Customer Interview Scheduling Process to Reduce Flaking

Ok. Now that you have a calendar scheduling tool set up, let’s go check those settings so that you maximize the chances that the majority of your customers who schedule a time will actually speak with you.

1) Set a lot of reminders:

Your first instinct may be to minimize reminders. You sent a calendar invite, right? You make all the meetings and events on your calendar, so your assumption is everyone else is the same.

That’s a rookie mistake.

Instead, set multiple reminders: 24 hours before, 1 hour before, and 10 minutes before is a your best bet. This makes sure the day before they see, “oh right, I’m supposed to talk to [your company] tomorrow.” Then, the 1 hour and 10 minute reminders ensure you’re at the top of their inbox and they are less likely to get sidetracked and forget.

Even if you think they’d be annoying, realize these reminders really help you out, so take advantage of the fact scheduling tools will automatically send these for you….if you turn them on.

2) Ask for a phone number at sign up, then call them:

You may use Zoom or MS Teams or Google Hangouts for all your calls at work, but that doesn’t mean your customers always do. For any number of reasons they may be unable to or unfamiliar with the technology.

And even if they are familiar, there may be times they have issues with the system working due to spotty internet, not being near a computer, or IT restrictions. You never know until you lose some calls because of it.

However, unfortunately many customers who miss calls never tell you why they didn’t make it.

That’s why the best thing you can do is ask for their phone number when they sign up for a call, so you can call them instead of wait for them to dial in.

3) Offer a gift card or credits for their time:

One of the easiest ways to recruit more people to speak to you is to offer them a reward for their time. This is particularly helpful when you’re working on a consumer product.

This can also often help with flake rates, too, because people are more likely to show up when they have a reward they’ll get after.

Just realize there’s quickly diminishing returns on this; while $20 might work better than $5, you typically won’t find much difference between $20 and $50, or $50 and $100. The exception is if you’re trying to talk to high net worth individuals, or executives; in that case you may need to pay a lot more, or donate the money to a charity instead.

Regardless, offering compensation (or credit with your product) is a great way to not just get people to schedule a call, but if you remind them of the offer, it can help with getting fewer of your interviewees to no-show.

4) Remind them what the purpose of the meeting is in the title, description, and initial ask:

As you’ve likely learned in building products, the little details matter, and they add up. Getting the details right in your customer interview reminders and calendar invites is no different.

Assume that your customers are busy, distracted, and thinking about a lot of other things, which means repeating all the most important details will help.

Go through your calendar scheduling tools settings and:

  • Make your title for the calendar event something clear and descriptive (i.e.- “[Your company] product interview about [Feature]”)
  • Put in the description all the key information about the event to further jog their memory and remind them of the reward they’ll get (i.e.- “You scheduled this call so we can talk to you about your experience with [Feature]. As a thank you, you’ll receive a $X gift card for your time.”)
  • Put all this info in the reminders, too. Remember those 24 hour, 1 hour, and 10 minute reminders? Usually you can add a note or customize them, so add the same info there, too.

You never know what your customer will or will not look at, so by always having the most useful information, you avoid them saying, “Who the heck is Joe and why did I agree to call with them?”

5) Configure messages to come from your email as much as possible:

Your interviews are a chance to build a relationship with your customers. One of the best ways to do that is for them to feel like they can reach you.

Sending messages from “noreply@” and default systems feel impersonal and prevent your customers from reaching you directly. This can rob you of all kinds of valuable opportunities:

  • Questions or concerns they have before speaking with you (which if unanswered they just won’t show up)
  • Feedback about your product. Most feedback never makes it to the PMs who can do something about it, so the best thing you can do is give your customers a direct line to you to cut out all the middlemen.

Where possible, swap any default email addresses and noreply@ systems for your own email, or an alias you can forward to you.

6) Resurrect some missed calls by sending followups:

There are a million reasons why someone might forget about your call with them. Don’t take it personal.

Instead, assume they meant to make the call and something came up. If that happens, the best thing you can do is follow up. I like to do so twice for anyone who seems to be flaking:

  1. 5 minutes after the start time: This is a last ditch effort to get them to join you. Be brief, polite, and simply include the link or dial in info to your call.
  2. 30 minutes after the start time: At this point they’re obviously not making it, so now this follow up is a “Sorry we missed you email” which then you give them a link to reschedule to a better time.

Neither of these are perfect, but I’m always amazed at how effective they are.

I find that anywhere from a quarter to a half of my flakes will respond to one of these and ultimately lead to us having a call.

And a couple settings for your benefit:

While you’re in your calendar scheduling tool setting things up, a couple other settings that will really help you be more effective:

  • Allow for no more than 3 calls in a day. Trust me, you’ll be drained, and this way you can still get other things done in your work day.
  • Leave a 30 minute buffer between meetings. This helps with any bio breaks you need, making sure you’re prepared well, and avoids fatigue.

With a few tweaks and tactical additions to your process, you can significantly change how often your customers flake on your customer interviews.

How do you avoid customers flaking on your calls? Leave a comment with any other tactics you recommend.

Further Reading:

Want to learn more about having great customer interviews? These other posts can help you:

And if you want hands on help creating a repeatable process for interview customers and turning those insights into great products, I’m available as a coach and consultant for a limited number of engagements. You can sign up for a call here, or learn more about my work here.

A Simple, Fun Way to Develop Your Product Manager’s Eye for Design

24 October 2023

Leave a comment

How do you teach taste? Can you teach it?

It’s a good question, and an important one for any good product leader to think about.

While designers work painstakingly on developing their eye for design, product managers have many things they need to master across a wide variety of fields.

Yet, it can be incredibly helpful for PMs to have their own sense of taste; it benefits them and their teams in a variety of ways:

  1. You can more easily appreciate quality work from your designer, knowing what to praise and recognize.
  2. You avoid suggesting ill-advised things that would hurt the user experience or drive your designer crazy.
  3. You can help raise the bar for your engineering team, by making useful critiques during the testing phase of releasing a new feature that otherwise only your designer would.
  4. You can more easily find good opportunities for inspiration to bring into your product from other apps, sites, and tools you use and discover.
  5. You’ll earn the respect of the rest of your product team as they see you make quality suggestions, and catch valid issues.

That all sounds great, but… how do you do that?

Or, if you’re a senior product leader, how can you teach others to have more taste? It’s certainly better to teach it than wait for it to naturally develop.

Today, I want to teach you one of my favorite, light-weight ways to help develop taste as a PM. It’s something I *looked forward to* early in my career when I was learning, and has kept my skills sharp as I now teach others the same way.

Taste Sessions: How to Develop Design Sense in You or Your Product Managers.

You don’t have to be Tim Gunn to help teach people to have better design sense. All you need is a simple routine and a little prep.

What you’ll do is set aside some time for a special meeting I’ll teach you to have. The key components of the meeting are as follows:

  1. Meet once a month: Make it a recurring event on everyone’s calendar so that you know it’s coming and can prepare/anticipate it accordingly.
  2. Choose an app or product category: Each month, choose a category or product type that you and the team will all download or visit and try out. This works for websites and mobile apps.
  3. Ask everyone to note a few things they like and don’t: If you have limited time in the meeting, ask people to prepare in advance. If you have more time, then you can do all of this live in the meeting.
  4. Go over the apps everyone downloaded together: Knowing that at least everyone has downloaded all the apps or visited & signed up for their products, you can now go app by app reviewing them for good, bad, and ugly of each one.
  5. Get specific about the good, bad, and ugly: When you or anyone else bring up something they have strong feelings about, take time to dig into why. Make it a discussion. Why is that interface ugly/clunky/awkward? What’s smooth or delightful about that screen, animation, or feature?
  6. Praise your team members: Praise is like watering flowers. To develop their taste, be sure to call out and praise the things you really like they noted.
  7. Suggest where you can apply this to your product: All of this work isn’t just theoretical exercise. It’s a great way to find inspiration for your product. Challenge your team to think about how the best things they saw can be applied to current and future projects.

I know it’s a cliche on the internet, but it’s true here: in just 7 simple steps, you can develop you and your team’s taste.

Here’s a few more tips around these Taste Session meetings to really run them like a pro.

1) Zoom is GREAT for this!

Too often, remote work makes things harder. I’m still looking for a good way to whiteboard remotely and have the same energy and emotion as being in person.

Yet, in this case, Zoom is a huge asset, especially if you’re looking at mobile apps. When you share your screen on Zoom, one of the options you have is to pair your iPhone, which then means everyone dialed in can now see the presenter’s phone in giant form. This is perfect for calling out little details that you’d never see looking over someone’s shoulder.

2) Have someone take notes and share them across the team

As much as this is about learning about different designs, it’s also about making your product better. Having a record of past discussions with call outs (and if you’re a pro, screenshots) to how these things can apply to your product, ensures that these discussions shift from the academic exercise to the practical application.

Best of all, if you save these in an easy to reference place, when you’re building new features, you have a library of interesting, high quality interfaces you can pull out and reference in your product spec or product thesis. This will impress your designers who are used to having to do most of the work to find inspiration.

3) Embrace your inner critic

The beauty of this process is that it’s fun, it’s easy, and it’s safe. Rather than only talking about products when you’re critiquing each of your team’s own work, you set up a safe place to critique other people’s products.

This helps people learn to be more vocal editors; since the creator isn’t in the room, it’s easier to say something they don’t like or doesn’t work for them just as much as what they love. And a big part of taste is recognizing all the ways you can do things wrong, frustrate users, or confuse them.

Establish the quality bar.

Being a good PM means being willing to stand up when something doesn’t meet the necessary quality bar. You can use these discussions to talk about tradeoffs like this and where your company feels the standard must be (i.e.- When would you delay a ship because it’s not good enough yet?)

4) Share responsibility

There’s not a ton of work to do to run a monthly meeting like this, but you do have an important decision to make: choosing the category or type of app/product.

One of the best ways to handle this is to rotate within your group who picks the app. Chances are, your example will inspire others to choose good categories. All they have to do is let everyone know a few days beforehand and the rest is the same each time.

Remember: Make it fun! You can learn from just about any app category regardless of what your app or software does. I’ve gotten great ideas for enterprise SaaS from consumer games, product led growth ideas from social networks, and many more unexpected crossovers.

5) When it’s your turn, be strategic

When it’s your turn to organize again (as you delegated to rotate through everyone) you should be strategic in your choices; for example, if your team is focused on onboarding experiments, you can focus on specifically looking at onboarding and choose a category and products you think does it well.

Don’t shortchange your efforts on this; leadership by example is one of the most important parts of making this successful. If you take looking at apps seriously, and choose a good category, so will your team. Yet, if you are careless, sloppy, or forgetful, it will be a lot harder for this habit to stick and grow with your team.

6) Encourage your junior team members to level up on their own

If some of your team members are very new to this, give them some reading to help raise their self-awareness and perspective when they use apps. This will help them both have a better critical eye, and most importantly, start to understand *why* they may like or dislike something.

In my experience, these books can really help:

  • The Non Designer’s Design Book: This book covers all the basics of fonts, colors, layout, and other fundamentals.
  • The Design of Every Day Things: This book will change how you look at the products and objects you use every day. This then can translate to looking at software.
  • Don’t Make Me Think: This book lays out fundamental rules of web design. The title gives you the most important lesson of all: don’t make your users think, and teaches it well through colorful examples.

By meeting once a month, they’ll have time to chip away at these books; every time you meet, they will understand more and more, while not being dependent on this one meeting for their learning. They’ll also likely then impress their colleagues as their critical eye will seem to rapidly improve.

7) The Veteran Move: Invite Design in

I would highly recommend you start out with just you and other PMs at your company (or if you’re too small, PM friends at other companies). It gives you all a chance to get comfortable and sharpen your skills without fear of being judged by others.

However, as you do all get more comfortable and develop your critical eyes, consider inviting designers from your team to join. It’s a great way for your designers to teach your PMs a bit and for your PMs to earn some respect and build more rapport with people they work with regularly.

Keep in mind that the size of your group does change things. Once you have more than 6 people in your group it may become difficult for everyone to get a chance to speak. When that happens try the following:

  • Keep track of who participates: In large groups, it’s easy for people to fade into the background. Make sure that doesn’t happen by making note and calling on anyone too quiet or seeming to check out.
  • Call on junior team members first: It’s easy to agree with your boss whether due to intimidation, a demand to be a yes man, or simply wanting to sound smart. By calling on junior team members first none of those things can happen.
  • Split the group up: If your group is creeping into double digits, it’s time to split it up. It’s better to meet in small groups and everyone actively participates, than a large audience only watching. I personally prefer groups of 2-6, and would split any group larger than 8, but use your best judgment.

While a little preparation can go a long way here, the best thing you can do in these meetings is to be a good moderator. That’s how you recognize people are checking out, ensure junior team members participate, and see when the group is getting too big. You’ll also see who may work best so that when you split your group, you match people up well.

A little taste goes a long way…

Developing the skills of your PMs, or even honing your own skills, can be something that you put off week after week after week. It’s hard to get it to the top of your to do list.

That’s why rituals like a monthly Taste Session can be the best hack. With limited effort, you and your peers or PMs can improve your design sense.

Want help developing the PMs in your org? Feeling isolated as the only PM at your company? I coach product managers to help them be at their best. Whether you need help mastering interview customers, reducing churn, improving your stakeholder relationships, or teaching new PMs all the skills they need to succeed, I can help you.

You can learn more about the work I do here, or sign up for a free call to discuss your challenges to see if I can help you here.

How to Take Notes like a Pro during Your Customer Interviews to Maximize Learning

17 October 2023

1 Comment

“Wait, you do what? …how!?!?”

I was as surprised as they were when I learned that it’s not typical to be able to take notes while also asking the majority of the questions in a customer interview.

To me, it comes naturally. Yet, now that I have taught dozens of PMs how to interview customers and regularly coach product managers, I recognize this is not normal. In fact, I haven’t really met anyone who can do the same.

Knowing this, the questions become:

  • How do you make the most of your customer interviews?
  • How do you take notes, so that you make the most of every discussion?

As I’ve helped dozens of PMs over the years master their customer interviews, I’ve noticed a number of approaches that can help, so I’m sharing them here to help you today.

How to Take Notes like a Pro during Your Customer Interviews

Before we dive into how to take notes, there’s an important step that comes before: Having a good customer interview script. Fortunately for you, I have a detailed post walking you through everything you need to know to create a great customer interview script.

Start there, then come back to this post once you have a great script; it’s the foundation of a great interview that brings the best insights and feedback for your product.

Okay, now that you have a good interview script ready, how can you make sure you never miss a valuable insight you hear from a customer? Try this.

1) Have a template you use every time.

A customer interview script is more than just a list of a few questions you plan to ask your customer. It should be a complete template that organizes and prepares you for the interview.

When I make an interview template, it typically includes:

  • Title area: This includes the person’s name, background, any existing customer and behavior data from your product, and links to relevant information (like their linkedin, any admin profiles in your systems, analytics data, etc)
  • Introduction: If the customer has never spoken to you before, having a few notes about how you’ll introduce you, anyone else on the call, and the goals of the interview can help.
  • People: Understand how your product fits in their life. Get to know what their job is like if your product is B2B, or their life around the aspects your consumer product relates to. Best of all, this warms them up as they also happen to be good rapport building questions.
  • Problem: This is the meat of your customer interview script. It should include a collection of questions to learn about their problems, priorities, and feedback.
  • Solution: Once you have learned all about them and their problems, the last step is to share any solutions for feedback. This can be walking through a new feature, having them try a clickable prototype, or sharing mockups.
  • Key Takeaways: I use this after an interview to summarize what I learned. I place this near the top so that I can easily compare notes and find the right interview to reference later.

A little structure and organization goes a long way. The hour or so of prep I do before my first interview to create such a template and then brief prep before each call allows me to get 10X out of my interviews compared to most people.

2) Do your prep ahead of time, every time.

A good customer interview is only as good as the prep you do for it. Not only does that include the template we just went over, but it also includes filling in answers to any questions you can find the answers out on your own.

When you already know something about your customer, fill it out. You can find out many of these things ahead of time with just a little bit of digging:

  • What are they reading on your company blog and other content?
  • How and what parts of your product do they use? Which features?
  • How long have they been a customer?
  • What support tickets or feedback have they submitted?
  • Which plan are they on? Have they upgraded or downgraded?
  • Any recent experiments they were a part of and which version they saw?

This saves you having to ask about these things live on the call, and can lead to unique questions you’d only ask based on what you learn.

For example, if I see they use an obscure feature, or use our product in a unique way, I make sure to ask about that. I also look for signs to determine if they’re a power user or brand new. Both will have valuable insights, but different perspectives you’ll want to differentiate.

Yet, you’ll only be able to pull on unique threads like that if you do prep and research beforehand.

3) Bring a friend.

The best way to take notes is to have a team member join you. It does not have to be the same person for all interviews. In fact, it can be beneficial to rotate through others, so many of your colleagues get the experience.

For example, on a typical product team, you may rotate through:

  • An intern eager to learn
  • An engineer that wants to hear straight from the customer
  • Your designer, who also should be eager to talk to customers
  • The product marketer launching the feature you’re researching and looking for copy inspiration

Having a coworker from your team with you means you have someone who can not only then take notes, but also ask questions you missed or would not have thought of. As you transition to various steps in your template, pause and see if they have a question they’d like to ask. Often, they notice something you missed, or bring a perspective different than you would, deepening your learning.

4) Embrace a little silence.

Sometimes the best question you can ask is a good pause.

By taking a minute to review your notes to find the next question, write down something important they said, or otherwise take a breath, you give space to your interviewee to think more, too.

Often, the person you’re interviewing will fill that space with more detail beyond their initial answer. This often leads to more great insights you wouldn’t expect, or would have missed if you had already moved onto the next question.

This is particularly relevant when they’re recalling a story or example for you; typically, you’ll first receive the high points from them. With some time to pause, I find you’ll usually hear more rich details, some of which will be extremely interesting and relevant to you.

Important: You need to ask good questions to get good answers from customers. Avoid asking “Yes or no” type questions, which will provide limited insights. Instead, focus on asking “What” and “How” questions, which get customers talking because it requires details to answers them. (i.e. – compare asking, “Do you use [Feature X] in the morning?” to “What prompts you to use [Feature X]?”)

5) No matter what, record it!

Even with the best help taking notes, there is no better record to have than a full audio recording (and video if appropriate). Here’s why:

  • Easy to review something later. You’ll sometimes have a conversation you want to revisit. Hearing it live is better than relying on your memory, or even the best notes.
  • Extract exact wording. This is great for marketing copy, as well as copy inside your product. Use their words, which are best revealed in a recording.
  • Build credibility with your team. Too many PMs make stuff up. To differentiate yourself and build confidence especially with skeptical engineers, sharing the recording of exactly what you heard builds huge credibility.

Important: Always ask for permission before starting a recording. Some will say no (definitely respect that, especially given some state laws require it), but for those that say yes, having it for future reference is priceless.

6) Follow up after.

After the interview, your work is not done! There’s a couple things you should still do to maximize your learnings and customer insights:

  1. Review and clean up your notes: Sometimes your notes will get sloppy in the moment. You’ll jot down the best shorthand you can. Be sure to go back, review them, organize them, and add anything else you remember that you didn’t notice the first time. If you think you missed a lot, it can be helpful to listen to the recording to add more notes.
  2. Add Key Takeaways at the top of your notes: This is the veteran move. By putting your biggest 5-10 takeaways at the top, it will make it much easier to revisit the notes later once your memory has blurred or faded. This saves you re-reading all the notes instead of quickly scanning the takeaways to find the right interview.
  3. Followup with your customer: Always message your customer thanking them for their time. Follow up on any issues they raised with you, and of course deliver the gift card or other compensation. This is also the perfect time to ask 1-2 follow up questions if there’s something simple you missed asking.

The little details add up and are what really separate good and average PMs from great ones. By following up after the interview, you can fill in any gaps and easily make up for any mistakes or misses during the interview.

Remember: Interviewing customers is a habit.

Becoming great at interviewing customers is a skill to build like any others. You may not be great at it in the beginning, but by making it a habit and practicing these 6 steps, you’ll make the most of interviewing customers.

Fortunately, these get easier and become more natural the more you do them. Your customers also will become more open to interviews the more it seems like it’s a consistent thing your team or company does.

Further Reading:

Want to improve your customer interview habits and learning? I’ve written more on the subject as I’ve mastered them and taught others:

Or, if you want hands on help building an amazing and insightful customer interview process, with help crafting a script, running interviews, and leveraging the best insights to build great products, then my coaching is the best option for you. Sign up for a free call to discuss your needs here.

Practical Product Ep 11: JTBD (Jobs to Be Done) – What it is, why it matters, how to use it, and a real life example

6 January 2023

“I don’t want a drill, I want a quarter inch hole.”

If you’ve worked in product long, you’ve probably heard that phrase talking about Jobs to Be Done (JTBD). The goal of the framework is to help you think deeper about why your customers buy or use your product. Often, it goes much deeper than you’d expect, and are even more significant than wanting a “quarter inch hole.”

Over the years, I’ve found learning the Jobs to Be Done for your product to be incredibly helpful not just for product teams, but also to inform sales and marketing materials. When you know the true, full buyer’s journey for your customers, you can attract more of them faster, and know how to better meet their needs.

Yet, most PMs are terrible at Jobs to Be Done, even if they claim to know what it is.

They pay lip service to the phrase, kind of like how some people think they’re a “lean startup” because they keep a tight budget. 🤦‍♂️

That’s why ever since I learned how to properly do a JTBD interview from the creators Clayton Christensen and Bob Moesta at a seminar in 2012, I’ve taught many friends, colleagues, and clients how to do Jobs to Be Done interviews, too.

In my experience, doing a live example is the best way to learn it, which is why I’ve typically taught people by doing an interview of them with a recent purchase.

While that works, it doesn’t scale well.

That’s why this week’s episode of Practical Product is a live recording of doing one of these interviews.

How to do a Jobs to Be Done interview: Why it matters, how to use it, and a live example for you to follow along to

On this episode we sit down with my former client, Ryan Findley, to go through the buyer’s journey and Jobs to Be Done for Ryan buying a new mattress.

Ryan, who is the Chief Learning Officer at Learn to Win and has spent his career working at startups and scaleups. He’s a builder who helped launch his current company, Learn to Win, and served as the company’s founding Head of Product (which is when we worked together).

In this episode, we show you:

  • How to do a JTBD interview
  • How to map what you learn to the buyer’s journey
  • What to do with what you learn in an interview
  • Common pitfalls to avoid and key moments to recognize

Highlights of the episode include discussing:

  • (0:35) – Introducing JTBD: What is Jobs to be done?
  • (3:03) – Setting the stage with the product Ryan recently bought
  • (3:45)- When did you first start thinking it was time for a new mattress?
  • (4:34) – Who was involved in the purchasing decision?
  • (7:51) – How did budget play a role here?
  • (9:14) – Where did you go to get ratings and reviews?
  • (13:45) – Zooming out: Black Friday & Forcing Functions
  • (21:36) – The purchase moment
  • (33:37) – How did this purchase differ from other things you buy?
  • (47:17) – Did you visit any third party locations when you were in the process?
  • (48:39) – Digging into Ryan’s experience using what he purchased
  • (49:59) – Advice for marketers applying JTBD
  • (54:53) – Ryan’s thoughts on this experiment

Key Show Notes & Further Reading:

We covered a lot of ground in this episode, so we have a ton of links for you to check out.

Mapping the Buyer’s Journey

The image above is the timeline we discuss in the episode. As you do a JTBD interview, you will learn what the various steps were in your buyer’s journey from “First Thought” all the way to “Buying” and “Consuming.”

If you want to follow along more closely in the interview, open up this companion post on how to do the jobs to be done interview. I use this post and the questions listed there every time I do an interview.

Additional helpful links:

Learn more and connect with Ryan Findley

Want to be the first to hear about new episodes of the Practical Product podcast, and blog posts I write about Product Management? Sign up here:

Practical Product Ep 9: The Harsh Truth of Interviewing & Hiring Product Managers

3 December 2022

Unfortunately, the product management interview process at most companies is poor. Navigating the interview process, or creating a good one at your company is a tall task.

In this wide-ranging interview we cover both perspectives to help you think about both the perspective of the interviewer and the interviewee. You’ll learn how to prepare to run a great interview process, when a project is appropriate and how to make it effective, as well as tips for your resume, and how to handle imperfect interviews for your next job.

This episode is with Willis Jackson, a long time friend of mine who has been the first PM at recently IPO’d Grove Collaborative, as well as VP of Product at Apto. He’s now hard at work on his own startup, but he took some time to share a lot of hard earned knowledge on the interview process in this episode of Practical Product.

The Product Management Hiring Process: How to thrive as the interviewer or interviewee

On this episode, we cover terrible PM interview practices, the key fundamentals of hiring you need to follow, how to ask behavioral questions the right way, making good PM assignments, and how to build your resume like a pro.

Highlights of the episode include discussing:

  • (0:44) – Introducing Willis Jackson 
  • (2:18) – The different types of Product Management and how they affect interviews
  • (8:09) – Recommended resources to learn to be great at hiring.
  • (17:15) – Handling ridiculous hypothetical questions and what to do instead.
  • (26:51) – The importance of networking, reputation and interviewing stories
  • (38:33) – How to make good, fair PM assignments for your interview process
  • (52:43) – Whether you should include company problems in your interview process
  • (59:13) – Resume crafting do’s and don’ts for PMs
  • (1:22:45) – Finding the right type of PM roles and filtering opportunities to save all sides tim

Key Show Notes & Further Reading:

We covered a lot of ground for both the interviewer and those seeking their next job, so some key takeaways are grouped below for each.

For the interviewer:

  • If you know you’ll be hiring down the road, start planning now. Think about the skills you want, the values you want, and the process you’ll follow. 
  • Interviewing is a skill. Spend time reading and learning how to do it well. 
  • It’s much easier to create your interview plan in small, incremental steps leading up to when you need them than being buried, desperately needing help and spread too thin.
  • Avoid puzzles, brain teasers, and hypothetical situations that are nothing like the job they’d have. Research shows it has no bearing on evaluating candidates effectively.
  • If you’re going to make an assignment, make it:
    • A reasonable time request (a few hours, not days worth of effort)
    • Consistently applied to everyone (don’t give one person a day and someone else 2 weeks)
    • Involves what the job would really include. (Willis’s example is a plan after an experiment / launch fails) 
    • Extremely clear what you’ll evaluate them on and what you will not. (Like whether you care about design or format)
  • Be proactive in communicating with your recruiting team. Enlist their help and expertise to find & close great candidates.
  • Remember that hiring the wrong person is extremely expensive in time wasted by your team, cost on your budget, and setbacks on your projects. 

For the interviewee:

  • Make your resume succinct and include data & numbers as much as covering skills and actions
  • If you do not have numbers now, start working on it now. Get in the habit to look up numbers and see what work you did has moved the needle.
  • Your resume becomes talking points and great questions in the interview.
  • Prepare good questions to ask an interviewee to make sure the company does the kind of product management you like doing.
  • Reflect on your current job regularly. Willis recommends weekly journaling on subjects like:
    • What wins have you had recently? What happened?
    • What did you learn from a project that recently didn’t go well?
    • What do you enjoy about your work and want future jobs to also offer you? 
    • What’s changed over time in my notes?

Helpful links mentioned in this episode:

Learn more and connect with Willis Jackson

Want to be the first to hear about new episodes of the Practical Product podcast, and blog posts I write about Product Management? Sign up here:

Practical Product Ep 8: How to Write Product Specs Your Team & Executives Actually Want to Read

23 November 2022

Are your product specs high quality? Do they succinctly and clearly convey what you’re working on, why you chose them, and what your engineering and design partners need to do their jobs well?

Or are they kind of random, with each one different than the last?

I’ve helped dozens of PMs improve their product specs, and I’ve been lucky to learn from one of the best how to make a great product spec. Which is why I knew I needed to do an episode on the subject to help everyone improve their product specs.

Today, we cover:

  • The most common mistakes PMs make in their product specs
  • How I learned the right way to make a spec
  • The key, fundamental concepts underlying good product specs
  • and most importantly: Exactly what goes into a great product spec (aka- Product Thesis)

How to Write Product Specs Your Team Actually Wants to Read (AKA – The Product Thesis)

Everyone writes product specs regularly in their job as a PM, but few do a great job with them. These poorly constructured specs then cause all kinds of problems on product teams including:

  • Engineers and designers confused and uninspired about what they’re making
  • Delays in shipping due to misunderstandings and miscommunication about priorities
  • Disappointed execs who don’t get what they expect

And a lot more. Yet, it keeps happening because PMs don’t realize that the root cause in their specs that:

  • Do not cover the right topics
  • Are wayyyyy too long, and filled with fluff
  • Tend to be overly prescriptive on the solution instead of collaborating with your team on it
  • Lack data to back up your decision
  • Fail to share an inspiring WHY to motivate your and convince your team
  • Are inconsistent spec to spec making it harder to read and digest

That’s why we need to hit the reset button and reshape how you make product specs with something called The Product Thesis. Listen in to learn more about it:

Highlights of the episode include discussing:

  • (0:49) – Mistakes made on the average Product Spec
  • (3:17) – Introducing you to The Product Thesis 
  • (10:06) – What goes into a Product Thesis?
  • (12:05) – Section 1: Why are we working on this next?
  • (14:57) – Section 2: When and how do people use this feature? (Aka – what are the use cases?)
  • (18:24) – Section 3: What problems do we need to solve, and in what priority?
  • (24:19) – Section 4: How much time is budgeted for this project? When does this need to be completed by?
  • (25:56) – Section 5: What are the future considerations that must be accounted for?
  • (27:28) – Section 6: What is our KPI or metric for this thesis?
  • (29:59) – Optional: For larger companies: Who are the stakeholders and how/when do they need to be involved?
  • (31:14) – Optional: What kind of launch or marketing/sales efforts go with this feature?
  • (32:27) – Section 8: Further Reading

Key Show Notes & Further Reading:

Want to be the first to hear about new episodes of the Practical Product podcast, and blog posts I write about Product Management? Sign up here:

Practical Product Ep 7: How to Supercharge Growth with Free Tools & Side Products with Michael Novotny

11 November 2022

Have you ever thought about building a free tool for your company? Do you want to build more buzz, get a ton more inbound SEO links, or drive signups and leads for your core business?

Or maybe you are a free tool skeptic, worried it will distract your team, take to much time, or not pay off?

I’ve always been curious about free tools, but haven’t been directly involved in many myself. So when I met Michael Novotny, who has become an expert in creating free tools, I knew I had to have him on the Practical Product podcast.

Michael is a product manager turned founder, who has helped build and launch dozens of free tools / side products now and studied hundreds of others with his company, Product and Build Co.

In this episode we go deep on this topic covering everything including keys to success, pitfalls to avoid, tons of examples, and how to convince yourself or your boss to take a shot at making some free tools or side products.

How to Use Free Tools & Side Products to Grow Your Business

Today we talked about how building free tools (aka – side projects) for your company can help drive major growth for your company.

Building these tools helps you a few ways:

  1. People who use your free tool may directly sign up for your paid product when they see you made the free tool.
  2. People using your free tool may give you their email address, which you can market to later.
  3. Others will link to your free tool, boosting your SEO through improved backlinks.

Michael shares a lot wisdom and experience doing these, and the most important tips are:

  • Build a portfolio: You need to launch many tools (ideally 4-5 or more) so that some will hit, and others won’t. If you only launch one, the odds work against you on the moon and stars aligning for you. 
  • Build in public/test with your community: To increase your success rate, validate and test the ideas you have for tools to see if they resonate and what are the most important things it needs to do to provide value. 
  • Use low and no-code tools: You can build and launch a lot faster using these tools, and since it doesn’t touch your core product, it doesn’t need the perfect architecture. 

There’s a lot more to this episode, so I encourage you to give it a listen on your favorite platform or in the player below:

Highlights of the episode include discussing:

  • (2:04) – How Michael discovered the power of free tools to drive sign-ups for another product
  • (11:46) – What are a couple of your favorite examples of these tools?
  • (16:31) – Cases where free tools didn’t work out.
  • (21:55) – Are there businesses that shouldn’t be creating free tools?
  • (29:05) – What is Michael’s Side-Product Framework?
  • (34:56) – How should PM’s think about budgeting for Side-Products?
  • (42:08) – How do you come up with good ideas?
  • (47:55) – How can you start to validate some ideas for tools to see if you’re on the right track?
  • (54:09) – What should people do to make these free tools successful?
  • (58:05) – What are the best ways to tie a free tool to your product?
  • (1:03:32) – How much ongoing maintenance should you expect?
  • (1:09:18) – What are your favorite tools that help you piece this process together?
  • (1:13:58) – Keys to convincing your boss or peers to try free tools.

Key Show Notes & Further Reading:

Case studies and examples of Free tools:

No Code and low code tools to help you build your free tools:

– Connect with and learn more about Michael Novotny:

Want to be the first to hear about new episodes of the Practical Product podcast, and blog posts I write about Product Management? Sign up here:

How PMs can Master Stakeholder Management to Build Better Products & Win Together

16 September 2021

“We can’t trust product to deliver.”

“Sales is just a bunch of sleazy, coin-operated machines”

“These bugs never get fixed. Product doesn’t care about CS…”

At the companies I’ve worked at, and those where I’ve coached PMs, relationships between departments have not always been great. In fact, they have sometimes been down right toxic, like the quotes above.

As much as it may feel good to point fingers, it doesn’t really help in such situations, and it’s the responsibility of product managers like you to improve them.

Yet, to fix them requires real change, new habits, and a lot communication. It’s absolutely possible (and common) to be well-liked as an individual human at your company, while the reputation of your product team or the product org as a whole is poor.

Today, we dig into those common issues, and how to hit the reset button so you can have great relationships with other departments and especially your key stakeholders within them.

How to Reset Your Stakeholder Relationships to Improve How You Work with Other Departments

Before we dive into how to fix this, let’s look at where things go wrong. There are a few common mistakes that lead to stakeholders and other departments being at odds with product teams. If any of these resonate with you, then read on:

  1. Sales sells features that don’t exist yet without consulting product.
  2. Sales and/or marketing complain about ship dates slipping often.
  3. Customer Success keeps trying to come up with new ways to get the attention of product to fix things they see daily.
  4. An internal feature request board filed by customer success, support, and/or account management has become a bloated graveyard rarely reviewed or followed up on.
  5. Executives keep asking for new ways to be updated on product.
  6. Product managers feel misunderstood, or overwhelmed by demands and requests.
  7. New feature launches are extremely stressful, and rarely well coordinated with needs from other teams like marketing promotion, sales training, or help doc creations and updates by support.

And there’s likely others you’ve experienced. The underlying theme of all of them is that you have a stakeholder relationship issue one or both ways:

  • PM -> Outward: You’re having trouble getting what you need from other departments for launches, customer insights and needs, sales demands, etc.
  • PM <- Inward: Other departments are complaining that product isn’t informing them effectively and delivering on what they need to be at their best in their role.

As a product manager, your job is to be a coalition builder. A large portion of your job is about listening and learning. You’re also challenged with building buy in and support for the things that you believe need done. Fortunately, that makes you perfectly suited to reset your relationships with other departments and stakeholders, regardless of who is to blame for past issues.

Build Stakeholder Partnerships; don’t manage them.

One of the phrases you hear commonly that PMs need to do is “Stakeholder Management”, which I think is a misnomer.

Your job is not to manage them. Your job is also not to keep them happy.

Instead, you should treat them like a Partner. 

That means:

  • You share your goals with them and listen to theirs, so you can see where you’re aligned and work through areas where there may be conflicts.
  • You let them know what you need and what you’re working on so they can help you.
  • You treat them as a valued source of information, and share what information you have that could help or impact them.
  • You bring structure to your partnerships with them by thinking ahead, asking good questions, and communicating the information they must know based on their needs and role.
  • When they make requests, you genuinely listen and then explain your priorities so they understand how their requests fit in, or why you have to say “not yet.”

This is all instead of “Management”, which often means:

  • Fielding their requests
  • Trying to keep them happy, or avoiding conflict
  • Dealing with their frustrations or outbursts
  • Selling them on your product decisions

So the question becomes: How do you build stakeholder partnerships?

How to build Stakeholder Partnerships

There are 3 key ingredients that underpin all of the recommendations in the remainder of this post: 

  • Empathy: Understand your stakeholders, and help them understand you.
  • Transparency: When both sides understand clearly the other’s goals, it gets easier to resolve conflicts.
  • Structure: Product managers should lead these discussions, and in doing so, there will be better discussions, and you will be less likely to have significant conflict. 

If you’re looking to hit the reset button to stop managing stakeholders and start making them partners, here’s the key things to do:

1.  Understand the needs and motives of your stakeholders


The first thing to do is to consider who your stakeholders are. Based on that, you’ll be able to think about what you need to get to know about them. 

For example, if someone is in the sales org, we know:

  • Quota: They have a number they have to hit that will always be top of mind.
  • Closer’s mindset: When you tell them something, they’ll be thinking about it through the lens of how it will help them close more deals. 
  • Successful customers: No one wants to sell a lemon, so they’ll be interested in being sure they’re selling something that will work for the leads they work to close. 

Also keep in mind that depending on their level in the organization, they’ll be thinking differently:

  • VP: Is thinking about the high level number the entire organization has to hit, and how changes impact the entire organization. They’ll also likely be thinking more long term as they consider head count, and multiple future quarters numbers they have to build towards.
  • Director: Will be thinking about the sales team that reports to them. If that’s a specific vertical, territory, or other type, recognize it will change what they’re interested in and what they’ll be lobbying for.
  • Individual Contributor in Sales: They’ll be focused on their specific leads for their territory or vertical and their number to hit, and are less likely to be aware of things going on outside of their team. They’ll be the most short term thinking of all. 

As you prepare for what stakeholders you’re going to engage, keep in mind what level they are in the organization and how that may impact what they need from you, what they may lobby you for, and how they can help you most. 

If you’re unsure, the best thing you can do is ask a few questions to really learn what matters to them like:

  • What’s your biggest priority this quarter? (or next quarter if approaching EOQ)
  • How do you fit into your organization? 
  • Can you help me understand how your responsibilities are part of your department’s larger goals?
  • Based on what I’ve shared we’re working on, who else should I talk to? Why did you choose them?

While the first few questions give you the foundation for your relationship with the stakeholder you’re talking to, the last one is really underrated; they can help you navigate the other departments more effectively as they’ll know who else to talk to who could be helpful, has been talking about things you want to learn about, or that complement what they share. 

Most importantly: Don’t assume. It’s always better to ask and be sure, than make a guess and then jump to the wrong conclusions.

2. Consider them one of many inputs you have

You should be sourcing data, input, feedback, and information from all over. Just a few of an endless number of examples would include:

  • Data and analytics from Looker, SQL queries, your analytics tools, Full Story, heat maps, etc.
  • Support ticket patterns, bugs, and biggest pains from the Customer Success team.
  • Top reasons you’re losing deals from the Sales team and Salesforce reports.
  • Key requests from big and important customers from your Account Managers. 
  • Market research and analysis as well as surveys and other data from Marketing. 
  • Financial impacts to changes in pricing, payouts, and onboarding experiments from Finance and Ops.
  • Error and bug reports, site and specific page speeds, and overall health of your product from the Engineering team.
  • Usability issues and customer-product interaction problems from the Design team. 
  • Insights from your own direct customer interviews, surveys, and feedback mechanisms.

A key skill of being a great PM is turning requests from these many teams into actionable, and prioritizable insights. 

To do that, you need to develop the skill of asking them the right questions when they come to you with a request. Some simple questions that can tell you a lot include:

  • How often does this happen? 
  • What is the root problem or cause you think that’s leading to this? 
  • How does this impact customers? (What kinds of deals are we losing because of this? Are people churning or rage tweeting about it?) 
  • How can you help me quantify the potential impact of the change you’re suggesting?
  • My priorities right now are to focus on [X]. Do you see a way this would help with that goal? How? 

What you’re doing here is teaching them how to take their requests and put them into a format you can act on one way or the other (either address it, or have good reason not to). It also gives you a way to easily push back, because something is far outside your focus right now.

A Sales Example…

For example, if an Account Executive comes to you and says, “We really need to build X! I just lost a deal because of it!” 

If you then ask, “How often does this happen?” and they say, “Well, it’s the first time,” you can easily then tell them: 

“Okay. I understand it’s not fun to lose deals like this. If it happens 5 more times, I’ll sit down with you and we’ll go over all 5 deals to look for patterns and figure out what we can do to help.” 

What’s particularly important here is you’re teaching them how you work. You’re not only letting them know the information you need, but also the threshold (“5 more times”) for it to matter to you. 

I used such an approach with a past Customer Success team I worked with. That led them to only raise concerns for issues they had to address 10 or more times in a single month. This took me from getting dozens of requests every time we met to an organized list of 2-3 asks per month. 

This proved to be a huge win-win:

  • We fixed their most important problems, reducing their time spent working around the same issues over and over.
  • Because they knew I was listening, they put in more work to organize the information I needed so I could quickly and easily take back the bug information to the engineering team to fix. 

Your other department stakeholders can be a wealth of information, and even help you prioritize problems when you bring the right questions to the table, and teach them how you think like in the examples above. 

Remember: “Not yet”, instead of “No.”

One thing to keep in mind is that constantly hearing “No” from the product team can be really frustrating, especially if they don’t feel like other things are coming fast enough.

That’s why a key phrase for you to remember is to say, “Not Yet” often.

When you say that, you can then explain your current priorities and reasoning to them. That will often help them understand why they have to wait for what they asked for. It may even help them remember something else they need that *is* related to your current goals. 

If you’re interested in a deeper dive into this topic, there’s more to Not Yet you can read about in detail here.

3. Bring the structure to your relationship and every meeting

Every meeting you have with a stakeholder should have a purpose (or multiple). 

Prepare an agenda and bring it with you to the meeting. Let them know so they can add things as well, and plan ahead for any questions you ask (especially if it involves asking for numbers or other data).

Most importantly, keep in mind you should be consciously driving the relationship.

By bringing an agenda, questions to ask them, and a plan for what you want to get from the relationship you will effectively make the switch from stakeholder management to building stakeholder partnerships.

Just like the Product Thesis challenges you to make sure you’ve thought through everything you need to kickoff a major project, following the approach outlined here in this post will help you think through everything you need to do with your stakeholders. 

Take time to plan ahead, and the meetings with your stakeholders will go much better. 

4. Iterate on the relationship as you work more with them

Stakeholder management is not one-size-fits all. You will need to adapt to their work styles, personalities, how helpful they are, and how relevant their work is to yours. 

This is all likely to change over time, so be prepared to make changes accordingly. A few examples that you should keep in mind:

  1. Frequency: Meet with stakeholders that are key to your project more often, and as they become less critical, meet less often. These recurring meetings are sometimes called Peer 1 on 1s.
  2. Helpfulness: If someone is difficult to work with, and doesn’t add a lot of value to your efforts, consider if someone else is better to meet with in that department.
  3. Triangulation: If another team is critical to your work, a single point of contact may not be enough, so consider getting multiple perspectives varying by seniority, focus, or other vectors that give you the best view.
  4. Your Needs: When you’re doing exploratory research, someone who can introduce you to the perfect fit is most helpful. Meanwhile, when thinking strategically, someone more senior may be a better fit. Adjust and adapt accordingly 

Over time, you’ll develop instincts around this, which will help you know who to reach out to and how often will be ideal to talk to them. You’ll also start to recognize what questions are best to ask each person you speak with. 

If you’re new to this, or hitting the reset button in a big way, keep in mind this is a process. It will take time.

The goal of this post is to help you re-frame your relationships around this new approach, and you’ll work with them and your other stakeholders over time to perfect these partnerships. 

Recap: Remember these 3 things

There’s been a lot in this post, so here is a reminder of the most important things for you to do to turn stakeholder management into stakeholder partnerships:

1) Set the rules and manage their expectations

Teach them to fish by setting thresholds for escalating to you vs. sending everything. Communicate why you choose that and be open to tweaking it with their feedback.

Frame the relationship by bringing an agenda every time and asking good questions of them. Over time they’re likely to start doing the same.

2) Show your thinking and priorities:

When they understand what you’re working towards, they can share ways to help you with insights, data, and introductions to key customers.

They’ll also understand why you may need to say “Not yet” to some of their requests.

But that only happens if you share it with them, so work to provide a concise overview of your work and thinking.

3) Get to know them and iterate:

When you build rapport with them, and listen to their needs, too, they’re more likely to have empathy for you as well.

You and their needs will change. Iterate on the frequency of meeting, topics you cover, how you explain your thinking, and questions you ask them to evolve the relationship effectively.

Tell them this is an iterative process as well, so they can help shape and improve them, too. You never know where a good idea or insight may come from!

Further Reading

Have questions? Leave a comment, or if you want hands on help for yourself or your product organization, you can learn more about my coaching and consulting here: BeCustomerDriven.com

To continue your learning to improve your stakeholder relationships, read these:

And if you’re looking for an experienced product leader to help coach you or a team member, or need hands on consulting to solve your hardest B2B Product challenges, see what others are saying about working with me and sign up to talk here.

Why Feature Voting Creates Poor Products (and what to do instead)

23 April 2021

1 Comment

“Let’s see what the highest voted features were on our feature voting site.” – Said no great product team, ever.

Whether a company is using the new hotness of a Canny, or the old clunky of a UserVoice site, public feature voting systems are an example of terrible product management practices.

Feature Voting is not talking to customers

I’m a HUGE fan of founders and product managers talking to customers. In fact, I’ve written a variety of posts to specifically help more people confidently and effectively “get outside the building” to learn from their customers to build better products.

Unfortunately, feature voting apps are the kind of shortcut that make some people think they’re getting customer input, when really they’re making a mess.

Just looking at this page makes me 🤢

I’ve had a personal disdain for these tools for many years, as I’ve tweeted about issues a few times, and had many conversations with other founders and PMs about it. And today, I’m finally pulling together a comprehensive case for:

  1. Why great product teams would never dream of using a Feature Voting tool
  2. All the reasons feature voting leads to a worse products and bad decisions
  3. What to do instead to be truly customer driven and deliver real customer value

With that in mind, let’s dive in…

Why Great Product Teams Avoid Feature Voting Tools like the Plague

There’s more than one way to build a great product, but there are a few traits that great product teams have in common:

  • Data Informed: They measure the results of their work and use analytics and data to help them focus their efforts and see where they need to ask questions.
  • Customer Driven: They think with the end user in mind, whether they understand it innately because they are it, or they deeply dig in to get to know and speak with end users of all types.
  • Detail Oriented: The details are what separate okay and good from great when it comes to products. One of my favorite sites is dedicated to them: LittleBigDetails.com
  • Product Sense: This obviously takes time to develop, but the best product teams include people who have and enforce having taste; they don’t fall for every fad, and bring a stamp to their work that people can see a bit of their fingerprints on.
  • Collaborative: The Holy Grail of great product teams is being able to operate with the Cauldron approach that Steve Jobs used; everyone brings their best ideas, no one cares whose idea was what, and everyone focuses on creating the best possible solution.

These are all hard to build and can take time to develop. If you’re trying to level up a product org, you likely can only improve one at a time, but that’s a post for another day.

When it comes to feature voting, it works against all of these:

  • Data Informed: As we’ll dig into more below, the data from feature voting is trash. Bad data is worse than no data.
  • Customer Driven: It may seem like it brings some customer voice in, but feature voting is so distorted and warped, it represents a bastardized version of listening to and understanding your customer.
  • Detail Oriented: When people vote on a feature, what are they really saying? You don’t know. You just know they clicked an up arrow, not the nuance of their needs.
  • Product Sense: Feature voting tries to turn your customers into your product team. Don’t. do. that. You build a product team so your PM, designer, and engineers can create the best solution, which customers usually haven’t thought of.
  • Collaborative: There’s nothing collaborative about a wall of feature votes on a screen. And without the real context of the problems to be solved, there’s no way to talk tradeoffs and iterate to something great.

Yet, despite all these being true, these are minor issues compared to the biggest issues with feature voting: They are fatally flawed from the start.

Let’s talk about why.

How Feature Voting Fails

Let’s deconstruct all the failed parts of using a feature voting system. It starts with a faulty foundation, and falls completely apart from there.

1) You get *features* not problems.

If you look at the average results of feature voting sites, what you’ll see is a lot of people asking for a feature. Typically posts say things like:

  • Make a CSV export
  • Build an integration with X
  • Add tangential feature Y

Here’s the thing: Any of those requests should be the *start* of a conversation, not the answer by itself.

For instance, with a CSV export, all of these questions come to mind putting on my PM hat:

  • What specifically do you want to export?
  • Why do you need an export? How will you use the export?
  • How would you like the export formatted?
  • How often would you expect to need the export? Why that frequency?

All of these questions can lead you down a rabbit hole that realizes any of the following:

  • A new report in your product would be better, and more up to date, than an export.
  • The export is for a key weekly meeting for the customer, so automatically emailing the numbers would be even better.
  • The export is only needed once a year, and needs to cover multiple sections of your product to really meet their needs.

The only way to get these kinds of insights is to talk directly to customers, and a feature voting site does not allow you to have those 1 on 1 conversations effectively.

2) People get easily sidetracked…

You’re using a product. Suddenly you notice something annoying, confusing, or missing. You decide you’ll share the feedback that their stacked bar chart that only has two shades of blue is very difficult to read and you’d like more control and granularity.

You notice they have a feedback button, or a link to their feature site and eagerly head over.

When you get there, you’re greeted by a list of dozens, if not hundreds of other options.Without meaning to, you start reading the other options, maybe clicking on a few. “Oh that sounds interesting…” you think.

Without meaning to, you start reading the other options, maybe clicking on a few. “Oh that sounds interesting…” you think.

But then you forget why you came, and either never get around to sharing the feedback you meant to, or giving only a partial explanation of your original thought.

Either way, the product team loses, as the feedback that motivated you to come to the page is much more valuable than a random upvote or two.

3) …Too many votes for something discourages future posting

While distraction is one problem, the sibling of it is people flat out giving up. If you look and see the top upvoted item was posted in 2017 and has 400 upvotes, what makes you think your new idea will ever get any attention? And do you think that anyone at the company is even really listening?

When feature voting apps were first getting popular about 5-10 years ago, I used to dig into them to try to get an idea of how product teams used them.

It was depressingly rare how often I’d see someone from the company actually active.

Even worse, when they were active, it was usually trying to explain why they’re not going to build the most popular items.

Now, feature voting sites can try to help with this by building some kind of algorithm to show “trending” or “most recent” items, but that’s really lipstick on a pig; this is just one of many significant problems.

(Fun aside: At a past job, I looked at our competitor’s UserVoice and noticed a number of posts from people asking for features we had that they didn’t. I passed this to our VP of Sales who then figured out who these people were and got some of them to switch to us.

So your Feature Voting page not only hurts your product, but makes it easier for your more ambitious competitors to steal your frustrated customers.)

4) Nobody wants yet another log in

One of the most famous stories in e-commerce history is the discovery that you can make *millions* more, and have double-digit improvements in checkout conversion by allowing people to check out as guests:

No one wants yet another log in, yet in many feature voting tools, the first thing you have to do is create another account to upvote or leave a suggestion.

As anyone who has worked on growth teams or on e-commerce conversion rates knows, adding steps to a process will always lead to more drop-off.

So let’s think about it…which is easier:

Feature Voting:

  1. Leave product you’re in to go to feature voting site
  2. Land on the overall page filled with existing suggestions
  3. Click on an item or to post your own feedback
  4. Sign up for an account (or if you’re a masochist, sign in with your account from last time)
  5. Find the way to add your suggestion and post it

Or, the direct way:

  1. Click on Intercom, a Feedback button, or link
  2. Send your message

If you’re measuring the rate of customers having feedback to actually sending, the latter option will far outperform the former.

But we’re getting ahead of ourselves on the solution, so let’s continue with the problems of Feature Voting.

5) Not all votes are equal.

Let’s look at two different Product Managers and see what they have:

PM 1: Talks to customers directly, gets feedback passed to them from other teams, and with help, organizes it all.

PM 2: Relies on the feature voting page, when convenient, to show how many people are asking for something like what they dreamed up.

—

PM 1 has 25 logged conversations where they or a colleague they trained asked some follow up questions for context, and understands this is “very important” not just a “nice to have.”

PM 2 has 50 upvotes for a feature that kinda sounds like what they’ve spec’d out.

Question: Who has more real data to back up their decision making?

  • Answer: PM 1 in a landslide.

Upvotes != real customer feedback

The number of upvotes a Feature Voting submission has does not mean all of those people want the same thing.

Let me repeat that: The number of upvotes does not mean every upvote wants the same thing.

Here are some of the reasons someone may upvote it:

  1. “I want that feature exactly as described.”
  2. “Well, this has 25 votes already, and it’s close enough, I’ll upvote that instead of post mine.”
  3. “Oh that looks interesting, I wouldn’t mind that. Click.”
  4. “I think my coworker wanted that…”
  5. “I could have used that a few months ago” (and haven’t needed it since)
  6. “That sounds cool.”

All those upvotes and you don’t really have a quantitative count of customer input. Something could have 250 upvotes, but it’s the least important item. Or they could want different functionality or features as part of it.

You don’t know though, because all you have is an upvote, not a conversation, or even a few sentences from each of those people.

QUICK PM QUIZ:

You’re a SaaS PM that has a mix of SMB and mid-market customers. Which feature should you build:

  • A) A feature that all 5 of your biggest customers say is critical to their workflow
  • B) Dark Mode, because it’s highly upvoted on your Feature Voting tool

If you choose B, please think about a career change

6) Customers forget what they wanted if you wait to reach out

Do you remember what you were thinking a year ago? How about a month ago? A week?

For the vast majority of people, ideas, and feedback, are fleeting. When you’re in the moment doing something is the time you’re most in tune with the situation.

If I come back to you a year later asking, “Hey – I saw you upvoted Feature X last year. We’re finally working on it. What were you looking for?” Unless you have continued regularly experiencing the issue that prompted the vote, you’re unlikely to remember the request well.

You might have also changed jobs and be unreachable, gone on vacation when I reach out, or not even remember voting for something.

That means following up with all those upvoters when you finally get around to a feature, you’re unlikely to get nearly as many useful insights as you would in the moment.

Feedback is like milk…it goes bad quickly when raw.

That’s why it’s so important to talk to customers regularly, and ask them in the moment what the underlying problem is and why it matters to them. That’s when they’ll remember the context you need to truly understand their request.

Just because you can’t build something right away does not mean you can’t talk to customers about their needs and save it for later. You do that by talking to them (in chat, email, calls, etc), not by collecting votes and checking your Feature Voting app once a quarter for ideas.

—

From distractions to lost feedback, and murky data to blurred meaning, Feature Voting is fundamentally flawed from the start, and only gets worse the longer you use it.

Now, let’s talk about what to do instead.

What to do instead of Feature Voting:

By now you understand why it’s a terrible idea to add feature voting to your product, but that’s only half the battle.

You need to then focus on what to do instead.

1) Source feedback where it naturally occurs

Done right, your company can be a customer / feedback gathering machine. We did it when I was the 1st PM at KISSmetrics.

There are great sources of feedback available all around you:

  • Sales Teams: They know what deals are closing and what deals they’re losing. And the best sales people know the difference between good losses (not a great fit) and bad losses (could have won and the customer would be happy).
  • Customer Support: They deal with the angry, the frustrated, the annoyed, and the confused. Tap into their knowledge and fix their biggest problems…and you’ll fix your customers’ problems, too. They also will get feedback, so give them a way to pass it to you.
  • Account Management: They’re in charge of keeping customers and making them successful, so they see the gap between the promises your sales team made, and the reality of using your product. Another gold mine of feedback and insights, and even a potential source of future junior PMs.
  • UX Researchers: As they do usability testing and test new features, they likely hear customer gripes, questions, and feedback. Make sure that gets captured.
  • Data Teams: Ever wanted to ask a specific subsegment of your audience a question? Your data team is your best friend in helping create all kinds of great segments to analyze, survey, and reach out to for customer interviews.
  • You: If you’re a PM and not regularly talking to customers, what are you doing? Make the time, and build the habit both for features you’re actively thinking about and to generally get feedback.

Now, that sounds like a lot, and it is. Which is why you need to dig into one of the key, unheralded jobs of great product managers: Relationship Building.

Have Peer 1 on 1s with key people on other teams.

Choose some of the best people, or those most related to the part of your product you work on, and have peer 1 on 1s with those people in customer success, sales, and account management.

If you’re new to these, this post dives deep into how to have awesome peer 1 on 1s as a product manager.

As you meet with them every 4-8 weeks or so, remember to teach them how to fish! What this means is that you will:

  • Explain to them your goals of gathering feedback and understanding customer needs they’re hearing, so they get what you’re trying to accomplish.
  • Teach them how to ask a good followup question or two (“How important is this to you? What is most important to you in this request? Why?”) before passing information to you.
  • Involve them in prioritization by telling them the threshold for passing it to you (i.e.- “Once you hear something 10 times, let me know” or “If it’s a customer >$N per year, pass immediately to me.”), thus keeping your signal to noise ratio strong.

You can also then share back with them how you’re listening and acting on what you hear from them. 

Nothing puts a smile on the face of customer success like hearing “Yes, we’re finally fixing that bug you have been dealing with for months” or telling a sales person about a hot new feature you both know they can sell like crazy.

When you start sourcing information from all across your company, you’ll see that feature voting is so incredibly low quality compared to all the ways you can get detailed, context-filled, specific information from your peers.

2) Choose your focus

Feature Voting apps don’t know what your company’s priorities are this quarter, and neither do your customers. Yet, as far as a Feature Voting app knows, all requests are created equally.

That’s why it’s important for you to narrow your efforts as a PM by starting with a focus.

Some examples of proper focus would be:

  • Retention: You need to improve retention and resurrection of old accounts for your social app.
  • Churn: Your SaaS app is leaking too many customers, bringing a drag on growth.
  • Growth: You need to create more organic growth through an improved viral coefficient.
  • Conversion: Customers are adding to cart, but not buying. You need to find out why, because it is crushing your CAC.
  • Activation: Why are customers signing up, but not completing the setup process?
  • New Bookings: Your company is trying to go up market, and needs to identify the features that will allow for more medium sized business deals to close.

For each of these problems, a different segment of your customer base would be your target to talk to. Along with this, different problems and features would be most important to focus on.

By starting with your focus in mind, it narrows down your efforts significantly, and can help you ask the right questions of all your peers you’re now meeting with and talking to semi-regularly.

This context helps you make better decisions, and changes who you talk to and what you ask them.

3) Get Quantitative, too!

While ultimately the goal is to understand who your users are, how they use your product, the problems you solve (or need to solve in the future) and how you fit into their world, you must balance that qualitative information with quantitative data.

Being customer driven also means understanding the numerical side of your customer base. You likely have a few different personas and company types who use your product. You need to understand how that translates numerically to your business with answers to questions like:

  • What % of our customer base is each type of business? How much of our total revenue do they correspond to? 
  • What customer types have the highest LTV? 
  • How do our core metrics compare when we slice our customer base by various properties (like company size, business type, various demographics, plan type, device, location/region, etc):
    • Churn rate
    • Rate of expansion 
    • Referral rate 
    • Conversion rate

And I’m sure you can think of many more. This kind of quantitative work is a priceless exercise, especially if you haven’t done it before. 

When I was at KISSmetrics we dove deep into these and discovered that company size didn’t really matter, but when it came to business model, SaaS and Ecommerce businesses converted 2X as well, churned half as much, and thus had a much higher LTV.

Do you think that impacted our future product decisions?

Surveys are your friend, too.

Now, ideally you’d have infinite data you could easily query across your product to answer every question. However, that’s neither feasible, nor really desirable (it would be too costly, hard to maintain, etc)

For snapshots in time, and to jump-start efforts, surveys then become your friend.

This allows you to then take input from your customers (especially those interested enough to take a survey) and segment it based on the questions you ask.

That’s why for instance with my startup, Lighthouse, we ask people what department they work in. Certain departments convert better than others, while one department has proven to be a massive time waste. We automatically filter out the latter’s input, because we know it’s not useful.

However, surveys are not a guaranteed silver bullet. In fact, most people make a ton of mistakes using surveys by making them too long, asking too many open ended questions, or using confusing language.

That’s why I regularly revisit this awesome post from Profitwell on how to do product surveys.

The TLDR is simple:

  1. Keep it short
  2. Make as many questions multiple choice as possible
  3. Ask customers to mark the *most* and *least* important things instead of rating everything

Now, taking a step back, not only are you building an engine to gather all your feedback and to better understand your customers, but you should also be using your analytics and other quantitative data (sales & marketing numbers, conversions, custom queries from your data team, etc) to help prioritize what metrics you want to move.

And once again, this kind of data informed approach gives you much better information than the random, muddy data of feature votes. Here you’re understanding problems, not starting with features.

4) Make people feel heard!

I hinted at this above in the problems with Feature Voting section, and it bears repeating: your customers want to feel heard. 

Posting or upvoting on a Feature Voting site is like the Suggestion Box in a Dilbert Comic:

The real way to make customers feel heard is for them to get a response from someone on the product team to things they ask for, and to occasionally see things they ask for fixed or added.

It could be its own post on how to do this, but for starters, here are 3 of my favorite approaches:

1) Tell your customers about new features launched

Any progress is good progress in the eyes of customers. It gives them hope you’ll get to some of their requests, and can really make their day when you finally build something they really wanted. 

That’s why I’m a big advocate of sending semi-regular product update emails.

Doing so let’s them know you’re listening, gives you a place to thank those that gave feedback, and reduce churn as people recognize you’re improving the product regularly. 

2) Make all your product emails have a real reply address so you can talk to people

I’m always stunned when companies send announcements and product emails and make them noreply@company.com. That’s a big missed opportunity.

Instead, make it a google group that sends to some of the product team. You’ll get fewer emails than you may be worried, but those that do really care.  Your customers will appreciate being able to respond (often saying positive things and showing gratitude!) and with a simple, quick reply, you (or a coworker) can make them feel heard. 

3) When you launch a feature based on feedback from a customer, email them personally.

How do you feel when you get personal notes from friends, family, or people you work with? Pretty good, right?

You can do the same for your customers simply by sending them a quick thank you note when their input is acted on. 

If you’re a small startup, then this should be pretty easy. Sending 10 thank you’s to those that hopped on a call should take you 5 minutes.  

As you scale, this can scale too. If need be, pull the names and emails of those that submitted feedback, had a flagged support ticket, did a usability test, etc and do a mail merge to send all of them a similar form note thanking them. Anything is better than nothing. 

You can also enlist your coworkers, like for instance asking your Account Manager to reach out to their customers involved and let the AM share the good news. 

This is win-win; it’s less work for you AND the Account Manager looks good to the customer, as it shows they can effectively pass along feedback that gets acted on by the product team.

Doing this not only makes people feel heard, but it also helps you build relationships that create power users and customer advisory boards. The more people feel heard, the more they’ll reach out and make your life easier as a product manager seeking out feedback, problems, and insights.

—

Isn’t this a lot of work?

Yes, this is a lot of work, but you’ll notice quite a bit of this is collaborative. That means you’re sharing the workload. And best of all, your hit rate on features built will go way up, so there’s less drama, more excitement, and you overall become more efficient.

All because you roll up your sleeves and do the work.

Take the time to be a product person who truly cares about their craft and builds processes and paths to directly learn from and speak with their customers. You’ll find that feature voting is then the last thing you’d want when you have all this direct, quality customer insight coming in.

Thanks to Ray Wu , Adil Majid, and Willis Jackson for feedback on this post.

Need help for you or your product team? I can be your coach.

I’m doing a limited number of engagements now helping with getting your first customers and early adopters, overcoming churn issues, and validating new features. You can sign up for a free intro call here, or learn more about what I do here.

 

How Being Customer Driven United a Startup & Doubled Feature Engagement

3 March 2021

2 Comments

What’s your proudest moment as a product manager?

For some, it’s shipping a massive feature that really struck a chord in the market. For others, it’s navigating a really complex challenge and finding an elegant solution. Or it could be any number of things like:

  • Amazing ROI on an opportunity you identified.
  • The moment you know you’d guided the product to product market fit.
  • Recognition from the CEO or a mentor you really respect.

After over a decade working in product, I have quite a few of those, and the story today definitely ranks way up there.

One of my proudest moments at KISSmetrics

Way back in 2012, I was the second, 1st PM at KISSmetrics. When I joined, it had been almost 4 months since the original, 1st PM had departed, and at that point, things had gotten pretty messy as no one was really doing what a good product manager does.

For instance, there wasn’t a lot of process, or commitment to talking to customers regularly, nor a way to channel feedback from those that were talking to customers into something actionable for the product team.

Over the course of my first 6 months there, with Hiten’s support, I slowly worked to turn the ship in a variety of ways to get us to be more customer driven. We were already product-led, well before “product-led” was a thing, but as an analytics company, we found it far more comfortable to rely on quantitative data in our KISSmetrics reports than qualitative data from our customers.

That’s why it was a really proud moment when the story you’ll read below describes when every single employee (over 30 at the time) talked to at least one customer that week. It led to one of the biggest morale boosts we had while I was there, and had a huge impact for our customers.

A big win for customers and us

When the KISSmetrics blog sold and some posts were no longer up, I had to rely on the Wayback Machine of the Internet Archive to reference the story of this moment when we got everyone in the entire company to talk to at least 1 customer in the same week.

To ensure this story is preserved going forward, I’m reposting this story as told by my coworker, Chuck Liu, back in November 2012. Everything between the lines is his writing.

Getting Things Done: How Moving Fast Doubled Our Feature Engagement

As a SaaS business, we regularly make improvements in our software product because we care about our customers. We also want to give our customers a competitive advantage with our customer data so they can make better business decisions.

When we started working on our new version of Live two weeks ago, we had a lot of discussion about whether we should rewrite the whole thing or just improve the visual designs. I’ll dive more into that a little later, but one of the big influencers for a rewrite was that we wanted to make a dramatic improvement in reliability and uptime — one that wouldn’t be possible with just a simple design upgrade. What’s a new design worth if it doesn’t work?

Funny thing is, when we finished building Live, our customers said it was fantastic…but still there was something missing. We were not getting the engagement or adoption levels we had hoped for.

What went wrong?

Instead of going back to the drawing board, we kept it simple. We figured we’d waste time making decisions and changing a lot of things. It was going to take too long to plan everything out again. We learned that, a lot of times, all it takes is small changes here and there to get that bump in engagement.

In our case at KISSmetrics, we increased our engagement by making small alterations in design thanks to our customer-driven data. Test quicker, faster, and get more things done. Here’s our story:

Building the New Version of Live

KISSmetrics Live lets you monitor top activity trends, filter for specific activity you’re looking for with a new launch or campaign, and see information for individual customers — all in real time.

When we set out to improve our Live tab (which provides people with a real-time data stream of customer activity), we first looked to our secret sauce — customer feedback.

Thanks to our awesome customers, we were able to define a list of requirements and use cases that our Live tool needed to help customers get their jobs done better and faster.

Some key requirements included:

  • Reliability*** — Flash was causing all sorts of trouble
  • A way to drill down on specific people, events, or properties
  • A better way to view your own activity AND monitor the live stream
  • Getting into individual customer profiles more obviously

Flash was a big offender. It caused loading problems. The different versions caused different errors. It crashed. Customers were not able to see it at all because of their device. Customers lost their whole session.

The old version of Live, complete with annotations, after customer feedback was summarized by Jason Evanish, our product manager. Oof.

Before we got to any visual design improvements, we started with the back end. If our customers couldn’t use our feature, there would be no point in updating the visuals or functionality. Our engineers did the hard part by building a robust back end that didn’t depend on Flash anymore. They were able to deliver the behind-the-scenes magic that powers our new Live tab now.

With reliability improved, we could confidently move forward and implement the rest of our improvements.

Getting to Customer Needs

What do customers actually need? To help answer this, we started with sketches, mockups, and wireframes.

One of the earliest sketches in our design phase from Eric, our support engineer.

We wanted to work with something low fidelity to show customers’ rough user experiences so we could see if our ideas were actually helping them solve their problems. This allowed us to focus on the jobs customers were trying to get done without having colors and major layouts get in the way of the feedback. It allowed us to differentiate what they needed (ways to filter, search, etc.) from what they wanted (button colors, perfect alignment, etc.).

Another idea from one of our designers, Jason Caldwell, with UI notes.

As a company that helps other businesses get to know their people, it was obvious to us that we needed to keep in close contact with our customers. And we did just that. After several cycles of interviews and testing, we were able to get to a point where customers agreed that they would be able to do the jobs they wanted to accomplish with our new improved tool. So we started building.

Problem Solved! …or so we thought.

Final design that was made by our lead designer, Ian, and implemented. Huzzah!

A job well-done, everyone! Let’s move on to the next thing, we thought.

Not so fast.

When we launched the feature two weeks ago, our feedback box started filling up with messages.

Some feedback messages all about stream activity being too big. Yowza. Thanks to everyone who sent in feedback!

A lot of the initial negative feedback focused on how the stream items were so large that it was impossible to scan for customer activity. Some people even wanted to switch back because it was not valuable without the ability to scan easily!

Luckily, we received positive feedback with regard to reliability, search filters, and trend monitoring. All the jobs were accomplished (yeah!), but we had a design issue to solve.

Here’s How We Got The Crazy Boost In Engagement

Yes, this is the actual metric chart with the huge jump. Since our data is tracked on a per-person basis, we knew something great happened.

The small bump in the middle of the graph was when we originally launched the new version of Live. We saw about a 50% increase in that week alone after an email to our existing customers to check it out and use it over the course of the week where it leveled off.

After the feedback wave, we hustled. We didn’t waste any time trying to come up with a quick change that would alleviate the main problem: customers wanted to see more activity and data in the stream.

What we rolled out a week later as Version 2. We increased how much activity you could see, while maintaining some Version 1 design elements on hover.

In the old design, you could see two customer activities, maybe two and half activities, in the stream. In the new design we came up with, you can see ten activities in the same resolution.

Here’s how some of our customers responded:

Valuable feedback turned into fix. Thanks Pejman!
Happy customer with the new fixes and improvements. Thanks Evan!

Best part? We didn’t do any additional marketing or launch emails when we implemented the new version. Our customers organically started using the feature A LOT.

So What Did We Do?

1. We started with tracking customer data. We made sure we established tracking for our Live tab with KISSmetrics. Tracking on a per-person basis steers you away from dangerous vanity metrics and makes you start analyzing the behavior of real people. One event each from a million people is very different from a million events from just one person.

We also got customer feedback data and made sure their problems were solved by our solutions.

2. With customer data, we were able to look into the whole lifecycle of our engagement patterns, all the way back to the first occurrence, as well as drill down into specific customers, if necessary. We were able to benchmark our performance to measure against dips, or in this case, gains.

3. We made small design changes according to most common customer requests. And we did it fast. It’s hard for any business to get everything right the first time. Or the second time. And so on. If you iterate quicker, you’ll learn faster about what worked, what didn’t work, and how to prioritize what’s next.

What a great story!

Because the company had become really in tune with our customers across a variety of methods (as you can see above, it even included tweeting at Hiten), we were able to understand our customer’s frustrations and quickly engage the whole team in a fix.

As I’ve learned over and over again in my career, if you get engineers and designers hearing the words straight from customers, it motivates them deeply to do their best work that customers love.

And this situation was no different as we quickly fixed the feature to be exactly what customers needed.

With that win under our belt, it became commonplace for everyone to ask what was best for the customer, and to go talk to customers if we didn’t feel we knew the answers yet.

You can do this, too.

Wish your company was more product-led, or your product team was more customer focused? I can help you.

I’m doing a limited amount of product consulting helping product-minded founders and 1st product managers learn and apply all the best product skills I’ve learned from some of the greatest product people in Silicon Valley.

If you enjoy what I’ve written here on my blog, then you’ll love when I get into the specifics of your business to help you accelerate your learning and take the actions that have a big impact…without the years of painful trial-and-error.

Want to see if there’s a fit for us to work together to help you? Then sign up for a free call by clicking here.