How to build a value-based metric system
A short guide from Anna Buldakova, Product Lead at Facebook
A successful product should be both valuable and viable, meaning that it should create some value and deliver it under certain business constraints. But founders often focus on the second part only — measuring sales, acquisitions, and unit growth.
Using just these benchmarks would be difficult to predict performance in the long run. So we need to have a separate category of metrics that would signal if we are creating a valuable product.
Here are 5 steps you can take to build such a metric system, according to Anna Buldakova, ex-Product Lead at Facebook.
Step 1. Understand that the metrics you were using before were not about product value.
Yes, even retention. It's a lugging indicator that depends on product value but not measuring it.
Let's take a closer look at the metric frameworks you are using: AARRR (acquisition, activation, retention, referral, revenue) or HEART (happiness, engagement, adoption, retention, task success).
All of them kind of suggest that you already know the value of your product. They are built around this value at its core. For example, ‘happiness’ is about exceeding customer expectations about value. Or ‘engagement’ — depends on the expected frequency of access to the value.
Step 2. Define the core value of the product.
Any product should have a clear mission or problem statement. In this stage, you need to answer the main three questions:
Who is your customer, and what is the use case?
How do you transform customers’ experience? What is the problem you solve?
Why do you do that? What is unique about you solving that problem?
You can use any standard methods to answer those questions — JTBD Research, BCG Matrix, SWOT Analysis, Kano Model, etc.
Step 3. Connect your mission with the actual metric that would caption its essence.
For example, a mission for Google Search is to help discover a broad range of information from a wide variety of sources.
Three possible metrics to capture this mission could be:
A number of long clicks on SERP per month.
Time spent on SERP per month.
A number of result queries per month.
The experience you’ll be designing with each of those metrics will be very different. If you focus on long clicks — you’ll be driving people away from the search engine. If you focus on time spent — you actually can show some of this information on SERP itself so people can access it right away.
Thus, we should pick out just one core metric and validate it through the number of experiments and questions.
Is it reflective of value? Do a round of user research studies, product/market-fit validation.
Is it predictive of retention and revenue? We want to make sure that we align these metrics with each other.
Is it simple, measurable, and actionable? We should be able to use it for day-to-day decision-making.
Step 4. Identify levers to grow this metric.
What are they going to be? It depends on your product. Consider what is relevant and work on awareness, acquisition, activation, engagement, and all other metrics you are used to boosting.
Step 5. Build a system around product value.
You can't have one metric that rules it all. Instead, you need a system that will help prioritize tasks: should I build a feature A or feature B first? Let's see what's more critical regarding core value.
How to develop such a hierarchy? You should determine what metrics affect the value the most and arrange them in descending order. To brainstorm metrics with a team, Anna uses three buckets:
Reliability. Making sure that users can access the product.
Usability. Making sure that they can complete the task they came for.
Experience. Actors and actions they take, surfaces, functionality, and inventory. In this bucket, you should consider not only a happy past but also bad experiences.
In the case of eBay, this could look like that if their core metric would be the number of sold items.