When it comes to the development and commercialization of new products, no one holds more accountability for the product’s ultimate success than the Product Manager.
“Product Manager” isn’t exactly a title that fits neatly into a box.
In most cases, a product manager is someone who works cross-functionally with development, marketing, and customer support in order to map out use cases for new products. In many cases, they also act as the voice of the customer to make sure that new products are aligned with what the customer wants. Their key responsibilities are to make sure that must-have features are prioritized, that project milestones are being hit, and that the project is staying on budget.
In order to be successful, product managers need to use whatever tools and tactics they have at their disposal to understand and engage with their target customers to gather information about what features to build and when.
In this article, we’ll go over 3 ingredients needed to devise, develop, and roll out successful products and features:
-
Utilizing the right tools
-
Understanding key assumptions then designing and running effective behavioral experiments to validate or invalidate those assumptions
-
Enlisting the experience of an innovation coach to motivate, empower, and offer guidance during the process
But before we dive in, it is important to address the problems facing many of today’s product leaders.
Problems Product Managers Face
According to recent research conducted by Christian Bonilla from Mind the Product, the biggest challenge facing product managers is being able to conduct proper market research to validate whether the market truly needs what they’re building.
It is a common concern that many PMs lack the time to even do any market validation at all.
On top of that, product managers have reported that stakeholders who are higher up in the organization tend to cloud the product vision based on their own opinions and assumptions, rather than hard evidence.
Put that together with difficulties in tracking pertinent details of each project while being hit with a steady flow of new information from leadership and other key stakeholders, it becomes clear how a product roadmap might become muddied and lose focus.
Many teams find themselves in a situation where projects go over budget, the products they deliver are filled with features that no one wants, and the important work around customer segmentation doesn’t get done, which leads to wasted time and money.
Fortunately, there are steps that can be taken to avoid winding up in the aforementioned situation.
Having the right tools, running solid experiments, and being open to receiving outside help from a third-party firm or innovation coach can mean all of the difference between rolling out a product that performs successfully in the market versus wasting resources on a product that is destined to crash and burn.
Having the Right Tools
In today’s business landscape, there are tools and software to serve just about every need, and more being developed and launched every day. But building the right product management stack does not only encompass having your typical project management system, feature and bug tracking software, wire-framing applications, spreadsheets, and product planning roadmap visualization tools (as this article seems to imply).
There is one key ingredient missing from their list, and has to do with the single most important variable in the equation: the customer.
Since the PM is the eyes and ears of the customer, it only makes sense to include tools that address who the target customer is, and what their needs are before jumping in and developing a product roadmap.
On top of using computer-based tools for gathering customer feedback like Typeform or SurveyMonkey, we recommend having something in your arsenal that allows you to identify your early adopters and anticipate their needs based on how you believe they will behave in order to guide how you set up your customer feedback interviews and experiments.
You can use a printed tool like the Customer Zoom to “zoom in” on your target customer and identify what your early adopter looks like. Using a tool like this can help you conceive a product roadmap that includes all of the must-have features that an early adopter of a product might want.
After you’ve spent some time as a team laying out who you think this person is and coming up with some ideas around what they want, you can take a deep dive into their biggest challenges using another print out called the Problem Zoom, followed by the Solution Zoom, which helps the team define the best solution to the customer problems identified in the previous exercise.
These tools not only provide product teams with an immersive experience, they also help to uncover the key assumptions around customer behavior that can then be validated or invalidated by using experimentation.
Did you know MOVES THE NEEDLE OFFERS PRODUCT INNOVATION WORKSHOPS?
At these public workshops along with learning to use MTN’s experimentation tools, our expert coaches will teach you the strategic and tactical skills and leadership competencies to drive innovation and increase your influence and business impact.
Experimentation: How to Setup and Execute A Good Experiment
Experimentation is an important part of the product management process. Structured experiments are used to expose any weak points in the product strategy and identify knowledge gaps that need to be filled with evidence gathered from customer insights.
The most effective experiments measure real behavior, while also delivering real value to the customer.
What Makes an Experiment Good?
As mentioned, good experimentation is all about focusing in on very specific behaviors. If you set up your experiment with too many variables, your team won’t be able to see what they actually learned at the end. The goal is to drill down to very specific individual behaviors (one variable changes) and isolate those variables to see what has an effect.
Typically, an experiment will use a hypothesis (“if, then” statement) based on assumption about the customer, problem, or solution that the team is looking to validate or invalidate.
A good hypothesis will clearly state what is going to change and what the expected effect of the change is going to be by relating an independent variable to a dependent variable. The effect on the dependent variable occurs as a result of what happens when you change the independent variable, and this is what gets measured and reported on.
When you use these assumptions as a means to direct experiments, you can gather data around these specific behaviors (something we refer to as the “customer truth”) in order to create your MVP.
And when we talk about minimum viability, there are minimum levels of functionality that need to be included, but you don’t need to have every supporting function or feature outlined at this stage. You just need what is required in order to deliver the core value.
How Long Should an Experiment Last?
Something that usually comes up with regard to experimentation is a question about length. In our experience, shorter is usually better. What is most important to remember is that it’s not necessarily about how many people you can get to behave in the desired way, it’s about what percentage of your customers take the specific desired action in a predetermined time frame. To measure an experiment effectively, you need to define what your target goal looks like before you begin the experiment and set parameters to determine what success looks like upfront.
That way, you can be precise about your results and can determine whether to run the experiment longer if needed.
How Product Teams Should Approach Experimentation
Whether your team is looking to improve upon a product feature that has already been rolled out to customers or to test out an idea before launch, experimentation can be leveraged during any point in the product lifecycle.
The experimentation process cycles through the following steps:
-
Planning and prioritization
-
Setting up and implementing the experiment
-
Tracking and measuring results
-
Taking action based on insights gathered
Ultimately, the goal of experimentation is to make products better meet the needs of your customers, while also falling in line with actions they are willing to take. Once you go through the steps of the experimentation cycle, you should know whether you need to run another experiment or make optimizations to your product. If the results of the experiments you are running are not guiding your team towards actionable insights, there is something wrong with your process.
Sitting down and going through a five whys exercise regarding your experiment should reveal what went wrong. It could be you designed a bad experiment, targeted the wrong behavior, or environmental factors skewed the results.
Planning and Prioritization
The first step is to narrow down the assumptions (everything you’ve accepted as true about your customer, their problems, and your solution) that you want to test, prioritize based on the riskiest assumption (the one that if you are wrong about it, everything would need to change), and create a plan to test various hypotheses that will provide you with an answer to whether the assumption is valid or not.
We start with the “riskiest” assumption because if it is proven to be false, it would be a waste of time to test any of other assumptions since we would have proof that our entire thought process about a product or feature was wrong.
From there, prioritization should be based on how crucial validation is to the business model and whether there is already supporting evidence available in the marketplace.
It is important to note that one test of the hypothesis is not going to necessarily provide the answer to the question alone. To get clarity around the validity of the underlying assumption, you may need to run multiple experiments, at increasing levels of fidelity.
To get started, gather your team and ask the question “what is the experiment we are going to run to elicit that behavior”, then, turn the answer into your hypotheses and figure out what confirmation of that behavior looks like in each situation in order to set up your experiments.
You may also want to create an experimentation log to keep tabs on the specific details of your experiments to serve as a reporting tool when you want to share out results to the rest of the organization.
Set Up and Implementation
After the planning process, the next step is to set up and implement the experiment.
If you wrote a good hypothesis, it should be fairly straightforward to set up and run your experiment. Now is the time to set your goal, determine which target metrics and KPIs you want to track and measure, and set a timeline for your experiment.
When figuring out your KPIs, you need to determine:
-
What the desired customer behavior is,
-
What the business activity that has to be done in order to elicit that behavior is,
-
What should be measured to know that the customer behavior is happening as a result of the business activity.
As a rule, it is okay for the target metric and goal to be based on an educated guess if you have no previous information to pull from. But after running the experiment, you should be able to look at what was learned and dig into the “why” behind the actual results.
At this point, you may find that you were wrong about the time needed to run the experiment or the number of people needed to complete a desired action in order to get viable results, in which case, you can choose to iterate on the experiment using the learnings you derived to make the needed adjustments.
Once the experiment has completed, it is time for your team to measure the results, and create a plan of action to integrate the learnings back into the current project or launch a new experiment.
Examples of Experiments a Product Team Can Run
There are plenty of behavioral experiments out there that can be used to help shape your products. The following are some examples of experiments product teams might run.
Pricing Experiments
Running a pricing experiment is a great way to maximize profit for a particular product. It is well-known that asking a person how much they would pay for something is a poor way to determine proper pricing: people don’t always do what they say they will. It is also asking your customer to imagine what your solution might look like, how it might function, then speculate on what they might be willing to pay.
One way to set a baseline price for a new product is to run an experiment designed to determine how much time, energy or resources someone is willing to expend solving their problem the typical way and assign some dollar value to that time. This gives you a baseline to speculate whether your product has to be less expensive, more convenient, have a better experience, etc, in order to deliver value.
Feature Experiments
In this type of experiment, you are looking to gauge demand for a new feature before spending any time on development.
Let’s say you have a digital SaaS style product in your customer’s hand, but you think you see a need for a new feature. You could create a button with a simple tracking link and embed it in a relevant place in your product. Once customers click through, you inform them the feature is still being developed. You track how many people click through to determine demand. Now, you can compare that number to the metric you hypothesized and using the evidence you gained from running your experiment, you can determine if the need justifies the development of the feature.
Asking those customers if they desire to be added to an interest list in order to be the first to test the new feature helps you understand if the feature would be of value to your customers before you even build anything. This concept of experiment currency helps you weed out the “tire kickers” from those who are truly interested.
Problem Validation Experiments
Before you begin building a full-featured solution, can you manually help your customers solve the problem?
Concierge experiments (experiments in which you perform all of the required processes and actions that would eventually be performed by the technology you are looking to build by hand) help you determine the customer value of solving the problem, and gives you a deeper understanding of the budget (time, energy, resources) they are willing to expend to get a solution. This also helps you understand how painful the problem really is to your customer.
Flexing Your Experimentation Muscle
Learning to run good experiments is tough.
It takes a lot of time and willingness to fail in order to get better. With experience, you can come up with good experiments, but there is no shortcut: like a muscle, the more you use it, the better you will get at it.
Adding in the guidance and experience of an innovation coach helps teams run successful experiments sooner. It’s like sports, having a coach allows for better, faster feedback which results in optimized decision-making and quicker learning.
Benefits of Utilizing an Innovation Coach
Before we dive into this section, it is worth noting that using the expertise of an innovation coach is not a requirement for success. Sometimes, there are budget constraints that don’t allow for the utilization of an external resource such as a coach. However, what we’ve found is that having a coach can be very beneficial for product teams because:
-
An innovation coach can offer a third-party perspective. They are not stuck in the weeds of the product itself, but can offer insights on which assumptions to test, how to feed insights back into the product development process, and identify any areas for improvement in the go to market strategy.
-
A coach is someone who is proficient at designing good experiments. This “borrowed” experience allows them to aid teams in creating, implementing and reporting on good experiments. Their experience also ensures that the teams they coach are following process correctly, and can offer feedback or corrections earlier than if the team is going it alone.
-
Coaches offer motivation and accountability. They are able to act as a mediator in times of discord, and they can provide insight to problems that may arise between the team and outside stakeholders.
-
An innovation coach can teach teams how to communicate their evidence when presenting to leadership and help them understand how to tell the story in a way that makes sense to leadership so they can secure buy-in.
Experimenting Toward Success: Final Thoughts
While the role of a product manager is to guide the development and launch of new products and features into welcoming markets, there is a lot of complexity to the process. Having the right tools to engage with customers and the proper experiments and guidance is a surefire way to continuously roll out products and features that people want.
But, tools aren’t the entire answer.
Tools, experiments, and coaches can make the road easier, but Product Leaders have to be honest about the process and about what the data says in order to produce successful products that truly provide value.
Looking for guidance on how to experiment your way to success? We are always happy to help. Drop us a line and let us know about your biggest challenges with experimentation.