3 Frameworks to Define Success Metrics for your Feature or Product
We want our products and features to be successful. We strive to shape products in a way to make them valuable for both consumers and investors. To make this goal achievable, we need to understand when we can say, “Yes! This feature has been worth the effort,” or, “We are about to fail. Let’s do something about it”. To put it another way, we need to understand when we are succeeding or facing the risk.
Often decision making on what to do next with a product or feature becomes a real challenge since many of us are used to go it blind. Should we maintain a feature? Is it cost-effective? Does it generate a business benefit? Is feature X more important for customers and business than feature Y? To find answers to these questions, we should have evidence to avoid making decisions based on our assumptions or gut feeling.
We need to collect quantitative data.
The knowledge about customers’ and users’ behavior opens new horizons. We become data-informed decision-makers. We don’t rely exclusively on what we already know from the past. We have a chance to monitor a market response dynamically and react to it correspondingly.
We need to define success metrics.
In my previous post “What Gets Measured Gets Managed: Measure What Matters” , I already discussed the importance of metrics for product success. Today I would like to focus on approaches that help to define success metrics and wrap up the results of my research.
If you were asked, “ How would you define the success metric for product or feature X? “, how would you answer?
Up until now, it would be difficult for me to come up with the answer. However, I’ve been lucky to find a few frameworks.
The GAME Framework
The GAME framework is a 4-step process that can define metrics for any feature or product:
- Articulate your GOALS
- List the ACTIONS that matter
- Define your METRICS
- EVALUATE your metrics
I really like the article “4 Steps to Defining GREAT Metrics for ANY Product” by Vince Law. The author proposes to make use of a mental model for metrics.
Imagine your product or feature is a box. Customers, users, and business put their problems, wishes, needs, money into this box and take solutions out of it. They provide input and receive value in return. And a GOAL of a product or feature is to produce this value.
However, along the way, a user is required to take ACTIONS within the product to extract value and achieve goals, e.g. sign up, subscribe, start using a solution actively, and come back. Create a list of these actions.
You can use ARM, AARRR, or other metrics frameworks to map a user journey.
To make our products and features better, we need to get insights about how (and how many) users get value from a product or feature. For this purpose, we need to track actions over time. It’s realized by METRICS.
A product or feature benefits from metrics since they allow to EVALUATE and adjust goals, actions, and metrics themselves. Evaluations allow to improve each phase of a user journey and maximize the value of a product or feature.
The Framework by PMExersices
PMExersices provides the How to Answer Metrics Questions framework that is quite similar to the GAME framework.
However, it contains more steps that help to be more specific since the goal of this framework is to enable a candidate to be prepared for a product management job interview. I see this approach to be used not only during the job interview. Product managers, especially those who are new to metrics, can take the maximum advantage of this framework.
The framework consist of 7 steps.
- Describe the feature
- Choose a goal
- Walk through the customer journey
- Map and quantify (through metrics) user behaviours in the customer journey
- Evaluate your metrics
- Prioritize the metrics
- Summarize your answer
In addition, PMExersices provides to different metrics questions using the described above framework.
The Framework by Mixpanel
Mixpanel, a business analytics service company, says, “we’ve put together a framework to help you find the right metrics for your product”. I couldn’t overlook their approach.
To learn more about it, I found the informative and practical Guide to Product Metrics by Mixpanel.
What I like about this framework is that it introduces a hierarchy of metrics. I think that this approach creates alignment, improves focus, and allows metrics contributing to each other.
There are 3 levels of metrics:
Credit: mixpanel.com
The focus metric is the top priority. It shows what matters most to a business. Mixpanel recommends choosing a focus metric tied to active usage, such as Weekly (WAU) or Monthly (MAU) Active Users. This metric shows whether more people are using the product over time.
It’s crucial to define what makes a user active. For example, a video streaming service may consider users who watch at least one video a week as active. Typically, active users are those who receive value over time.
The focus metric sum up trends in other metrics, such as acquisition and retention.
Level 1 metrics complement the focus metric. There are 5 categories of the level 1 metrics: Reach, Activation, Engagement, Retention, and Business-specific.
They help to make sure that a product is growing in a healthy direction. For example, a video streaming service may notice a drop in a number of weekly active subscribers. This decrease may be caused by a fall in an average amount of time an active user spend on watching a video. Since engagement is reduced, some users find a solution less valuable and stop using it.
Level 2 metrics are more specific and drive the level 1 and focus metrics.
Wrap Up
The success of our products and features should be measurable. We should know when we can celebrate success and when we need to make an extra effort to improve the situation.
Definition of success metrics and collection of quantitative data help us to make educated decisions on the future of our products and features.
All frameworks that I considered in this post force us to be a top-down.
We should start with what matters the most to customers, users, and business including setting user and business goals.
We need to track actions of users by walking through the user journey.
We need to evaluate and prioritize metrics in a way to make our products and features more valuable for consumers and business.
Did you like my post? You can easily share it with your network by using the buttons below.
