Key Metrics That Matter
Key Metrics That Matter
How do top 50 consumer companies measure their success?
👉 This week’s post is done in collaboration with “Corporate waters”
This is a guest post by Mikhail Shcheglov, Group Product Manager at Bolt.
Thank you, Mikhail, for sharing the knowledge.
And thank you, reader, for visiting Tallinn Product Group.
Since the moment I started doing interviews about 7 years ago, I’ve noticed one stable trend.
Most product managers struggle to succinctly answer, “What metrics would you track?”. The question is deceptively simple, but has layers of hidden meaning.
The typical response is a long list of things, ranging from net revenue to satisfaction scores and NPS, often without any clear rationale.
I understand where this is coming from. The “measure everything” mentality has infected not only the product teams, but entire companies. This FOMO is reinforced by the classic adage “If you can’t measure, you can’t manage”.
The truth is, you don’t need to manage everything. You need to cut through the noise and discard the stuff that is not important.
But how do you navigate this amidst the abundance of misleading frameworks? Directionally, stuff like AARRR, RARRA, or HEART makes sense, but it over-simplifies the bigger picture behind the business.
There’s so much noise out there and so little clarity on the why.
The goal of this article is to provide you with that clarity.
Today’s article
Key Metrics That Matter. How 50 top consumer companies across 4 key business models (Advertising, Marketplaces, SaaS, DTC) measure success.
Guiding framework. How to select key success metrics for your business.
📝 Word count: 1735 words
⏱️ Reading time: ~12 minutes
Companies reviewed
We looked into top 50 consumer companies (mostly tech) and identified core business models, looked into their key metrics and success indicators.
In essence there are four viable business models - advertising, middle-man take rate, paid subscription and direct-to-consumer sales (or a combination of them).
📊 Key Metrics That Matter
📢 Advertising
Most “subscription-free” services at their core are advertising businesses. Take Facebook, TikTok, YouTube, Pinterest, X e.t.c.
Key success formula for such a business is:
Average ad revenue per user * Active users * Clickthrough rate (Clicks/Impressions) * Add fill rate > x3 CAC
Two key things matter: the number of users and their engagement, and how effective the ads are. The more users return and engage with the platform, the higher the ad revenues.
The more relevant our ads are and the higher the inventory of ads we can show, the higher the ad revenues.
These revenues are measured against the customer acquisition cost (CAC). The rule of thumb here is that your user’s lifetime value (LTV) needs to be three times the CAC. This helps to generate the upside needed to cover the remaining operational costs.
Personal experience case from OLX
Our advertising inventory consisted of Google ads 90% and 10% direct sales with long-term contracts. If we lacked inventory to show, our fill rate would go down as well as ad revenues.
Making adds more personalized was a double-edge sword as the ads started competing with the search listings on the platform.
More on that later on.
🏪 Middle-Man Take Rate
In essence, most middle-man take rate businesses are either two or three-sided marketplaces.
Marketplaces charge a premium for balancing the interests between demand and supply.
For a ride-hailing business, demand is represented by riders and supply by active drivers on the platform. For a food delivery business, the demand side consists of eaters, the supply side consists of restaurants, and the third side includes available couriers.
The higher-level success metric is either Gross Bookings or Gross Orders, but the net revenue comes from the take rate percentage.
Case of an Airbnb:
Net Gross Booking Value = Gross Booking Value - Gross Booking Value Cancelled (Cancelled bookings) - Gross Booking Value Altered (Altered Bookings)
Source
Gross bookings or gross orders will not come out of thin air. For this, liquidity needs to happen on the platform. To enable liquidity, you’ll have to incentivize both supply and demand at different stages of marketplace development.
Hence, the success metric would look like the following:
Net rate = (Take rate - Demand subsidies - Supply Subsidies) / Gross Booking Revenue
If you want to learn more about marketplaces, I suggest you to jump into this article.
☁️ SaaS
Netflix, Tinder, Strava, Calm, and even WhatsApp (which sells Business APIs) are all subscription-as-a-service businesses.
The king of all metrics for such businesses is Average Recurring Revenue (ARR). Seemingly simple, this metric comes in many shapes and forms.
For instance, Dropbox measures ARR as Subscription revenue * 12.
Zoom as Monthly recurring revenue * 12.
SproutSocial as FY Value of Subscription revenue
Source
The universal way to measure net ARR is:
Net New ARR = New ARR (From new customers) + Existing ARR (current customers) - Churned ARR (lost customers)
Bu the real success hides behind the following formula:
LTV (Lifetime Value) > x3 CAC (Customer acquisition cost)
CAC Payback < 12-18 months
Put simply, the average revenue each customer generates needs to be three times their acquisition cost. These costs have to be paid back with recurring revenue in 12-18 months for the business to have attractive ROIs.
Of course, ARR is an output metric. The interesting twists come from finding and measuring sensitive proxies that can influence the ARR.
A case of Netflix:
Back in the day company used to measure “Valued hours” as a predictor to the ARR.
Valued hours = (Subscriber viewing hours of the show / Total viewing hours of the subscriber)
The more hours you watch, the less valued they are. However, if you watch really little or a single show keeps you on the platform - your hours are extremely valued.Netflix also considers such a show to be valuable for the platform.
SourceA case of Duolingo:
One of the core metrics for Duolingo is paid subscriber conversion, with 72% of its revenue coming from subscriptions.Since the service operates on a freemium model, it sometimes takes months or even years for a user to convert to a paid subscriber. Therefore, a user's tenure on the platform is considered a significant factor in predicting conversion.
🛒 DTC (Direct to consumer)
An old-school direct-to-consumer sale of goods and services.
It requires ownership of assets (e.g., cars for rentals, planes) or supply chain/production of goods (e.g., glasses).
Success is measured directly by the Profit and Loss statement:
Net profit = Operating profit - interest - taxes - other expenses
Operating profit = gross profit - COGS - operating expenses - depreciation/amortization
The more you sell, the lower the cost, the more efficient you are as a business.
Typically, such companies have significant upfront CAPEX (capital expenditure) investments, which can lower the ROI (return on investment) initially and reduce margins due to ongoing depreciation expenses.
Case of an average airplane company
Typically airlines increase their ROI by maximizing the profit via higher prices and selling limited amount of bookings in the plane.
Ryannair introduced novel strategies like demand based pricing, services upsell and ruthless cost reduction.
🔀 Combinations of models
Many businesses combine multiple models to maximize revenues. Subscriptions can be paired with ads (e.g., Spotify, Behance, Discord). The same applies to the “middle man model”, which can also be bundled with advertising (e.g., Uber, OLX).
Bundling two models might come at a cost, though. For instance, ad inventory can compete with your supply (such as product listings) and negatively impact marketplace liquidity.
Personal experience case from OLX
We measured ad revenue metric, but looked at the buyer conversions as a guardrail. If the buyer conversions dropped, we assumed the test to be unsuccessful. This allowed to align both business models on the platform.
Blended models can positively impact CAC and LTV and have a multiplier effect for the ROIs of the business model as well.
Signal to Noise ratio
🚧 Data vomit
An abundance of data sources has created the phenomenon called “data vomit”.
On one side, you have hard financial metrics. On the other, a diverse set of Looker/Tableau dashboards. Add Mixpanel to the mix, and you’re right in the epicenter of the “metric galore”.
As companies mature, they start tracking more and more data. Some of it is regular reporting metrics, but some are ad-hoc and exploratory. All those efforts just amplify the noise.
The typical strategy for dealing with this ambiguity is sticking to the status quo. What other teams and stakeholders say is important, we assume is important by default. Just follow the crowd.
🚧 Metric dimensions
Another aspect that adds complexity is the metric dimension.
A metric can indicate a business goal (Gross Bookings, Net Revenue, ARR, EBITDA, or Net Profit). Alternatively, it can represent a product goal (User Acquisition, Monthly Retention, etc.) or a goal of any other function.
Teams speak different languages. An operational team pushes for net profitability while a product team struggles to identify and explain which input metrics can act as proxies.
Many times, those high-level aspirational goals amplify the status quo even further. Since net profitability encompasses everything, teams start doing low-impact busy work and label it as “meaningful for the business”.
🚧 Framework noise
There’s an exceeding amount of data-frameworks floating around the internet.
Pirate metrics (AARRR), RARRA (Retention-first), HEART, you name it.
The concepts make sense and directionally those frameworks are ok. But many times they are an extreme simplification of the bigger picture behind the business.
Usage of these frameworks in isolation can be very misleading. I’ve seen numerous cases when teams shaped their KPIs to the AARRR requirements purely for pro forma reasons.
🚧 Pattern seeking
When you stare in the abyss for too long, eventually you’ll start seeing patterns.
We’re hiredwired to look for patterns even when none exist.
The problem with data abundance is that a product manager can be easily misled into optimizing for the wrong things.
The larger the pool of available data, the more of a sense will product team will make out of it.
Putting it all into framework
✅ Think like a board member
What’s important for a company owner/ board member?
Those folks are interested in getting the highest returns on the money they and their VC funds have invested.
A rule of thumb answer to this is >30% IRR within ~4 years. That should be your starting point. Work backwards from there in estimating the revenues, costs and the payback period.
✅ Identify the business model
As we mentioned above, there are four viable business models for consumer services - advertising, middle-man take rate, paid subscription, direct-to-consumer sales (or a combination).
The business model informs the key metric selection.
✅ Identify the key business risk
When breaking down IRR/ROI, what’s the biggest leverage and risk for the business?
Risks vary by company stage (pre-seed, scale-up, corporation, etc.). Early on, proving product-market fit with stable topline growth (e.g., Gross Bookings/GMV) might be crucial. Later, optimizing for profitability and cost reduction becomes key.
✅ Split your metrics into actionable proxies
Your success indicator is typically a low sensitivity output metric (say Net bookings). It needs to be broken down into sensitive and actionable proxies (total number of listings, listing conversion e.t.c.).
✅ Keep it simple
Remember, you’re not building an academic model. You need to give direction to yourself, the team and get buy from the stakeholders. The simpler your selection of metrics and the success formula, the higher the chances it’s going to get adopted.
A long list of metrics, complexity and arcane definitions will play against you.
📋 TL;DR
A metric definition starts with the definition of success—namely, the ROI, the expected returns, and the payback period.
Metric selection is then dependent on the business models.
In a nutshell, there are only four key business models or combinations of them: advertising, marketplaces, SaaS, and direct-to-consumer.
Advertising models are highly dependent on advertising revenue and corresponding user engagement.
Marketplace models are all about liquidity and a net rate uplift.
SaaS models focus on average recurring revenues and the proxies leading to them.
Direct-to-consumer models are focused on sales and show a direct impact in the P&L.
Keep both your metrics and the success equation simple to understand and pitch if you want them to be adopted by the team and stakeholders.
 | A guest post by
|