5.3 Product Value Metrics & Evidence-Based Management (EBM)

5.3 Product Value Metrics & Evidence-Based Management (EBM)

Core Value Axiom: Delivering a high volume of features quickly provides zero value if those features do not solve customer problems or improve business outcomes. In Scrum, value is measured by market impact and customer outcomes, not by internal activity metrics, story point throughput, or output volume.

To help organizations make empirical decisions in complex, uncertain environments, Scrum.org developed Evidence-Based Management (EBM). EBM is an empirical guide designed to help Product Owners, Scrum Masters, and leadership steer product development toward customer outcomes and business value using objective, measurable evidence rather than subjective assumptions.

---

The Four Key Value Areas (KVAs)

EBM organizes organizational performance into four Key Value Areas (KVAs). These KVAs reflect an organization's current performance, market opportunity, operational speed, and capability to deliver future value.

                             +-----------------------------------+
                             |     EVIDENCE-BASED MANAGEMENT     |
                             |         (Key Value Areas)         |
                             +-----------------------------------+
                                               |
         +---------------------+---------------+---------------------+
         |                     |               |                     |
         v                     v               v                     v
[ Current Value (CV) ]  [ Unrealized Value ] [ Time to Market ]   [ Ability to Innovate ]
 (Delivered Value Now)    (Potential Value)     (Speed of Feedback)     (Capability Focus)

1. Current Value (CV)

Measures the value that the product delivers to customers and the business today. It answers: How happy are our users right now, and how sustainable is our current revenue stream?

• Key Metrics: Customer Satisfaction Index (CSAT), Net Promoter Score (NPS), Active User Retention, Monthly Recurring Revenue (MRR), Employee Satisfaction.
• Product Owner Focus: High CV indicates a mature, successful product. Low CV signals that current features are failing to satisfy user needs, requiring immediate attention to user experience and quality.

2. Unrealized Value (UV)

Measures the potential future value that could be captured if the product met all potential customer needs. It answers: Is there additional market opportunity worth investing in?

• Key Metrics: Market Share Gap, Customer Satisfaction Gap, Desired Customer Outcome Metrics, Competitive Opportunity.
• Product Owner Focus: High UV combined with low CV justifies ongoing R&D investment to capture untapped market share. When UV drops to near zero, the product has reached market saturation, and further R&D yields diminishing returns.

3. Time to Market (T2M)

Measures the organization's ability to quickly deliver new capabilities, services, or products to customers. It answers: How fast can we test a new hypothesis and learn from real user feedback?

• Key Metrics: Release Cadence, Build/Deployment Frequency, Lead Time (from idea generation to production deployment), Cycle Time (from work initiation to release).
• Product Owner Focus: Reducing T2M shortens empirical feedback loops, allowing the team to adapt strategy rapidly before competitors react.

4. Ability to Innovate (A2I)

Measures the organization's effectiveness in delivering new capabilities that meet customer needs. It answers: How much of our engineering capacity is spent creating new value versus maintaining existing legacy software?

• Key Metrics: Feature-to-Defect Ratio, Technical Debt Ratio, Installed Version Diversity, Time Spent on Unplanned Maintenance and Context-Switching.
• Product Owner Focus: High technical debt and bug rates severely impair A2I, forcing teams to consume capacity on bug fixes and refactoring rather than valuable new features.

---

Goal Hierarchy in Evidence-Based Management

EBM structures organizational direction into three nested levels of goals to align strategy with execution:

1. Strategic Goals: Broad, long-term organizational targets characterized by high uncertainty (e.g., "Become the leading mobile payment platform in South America within 3 years").
2. Intermediate Goals: Specific milestones that advance the organization toward Strategic Goals (e.g., "Achieve 500,000 active monthly users on iOS by Q3"). In Scrum, the Product Goal serves as an Intermediate Goal.
3. Tactical Goals: Short-term operational objectives evaluated in rapid feedback cycles. In Scrum, the Sprint Goal serves as a Tactical Goal.

---

The Flaw of Velocity in Value Evaluation

One of the most persistent anti-patterns in Agile management is using Velocity (the number of story points completed per Sprint) as a productivity target or value metric. The PSM I exam repeatedly tests candidates on this critical distinction.

Why Velocity is NOT a Value Metric

1. Velocity is Arbitrary and Relative: Story points are subjective estimates unique to a specific Scrum Team. A velocity of 50 points for Team A cannot be compared to a velocity of 30 points for Team B.
2. Point Inflation and Gamification: When management pressures teams to increase velocity, teams artificially inflate point estimates (e.g., re-classifying 3-point items as 8-point items) without delivering a single extra unit of customer value.
3. Ignores Quality and Business Utility: A team can achieve record velocity while delivering features that customers despise, software riddled with hidden technical debt, or capabilities that degrade system performance.
4. Internal Capacity Tool Only: Velocity is strictly an internal, relative capacity-planning tool for Developers during Sprint Planning, never a measure of team productivity, success, or product value.

---

EBM Strategic Experimentation Loop

EBM connects high-level strategic direction to daily execution through an empirical experimentation loop:

[ Formulate Hypothesis ] ---> [ Run Experiment (Sprint) ] ---> [ Measure KVAs ] ---> [ Adapt Strategy ]

By evaluating KVAs after each production release, the Product Owner makes empirical decisions backed by hard market evidence rather than subjective opinions, executive intuition, or output vanity metrics.

---

Integrating Evidence-Based Management into Scrum Events

EBM metrics are not analyzed in isolated management silos. They are integrated directly into standard Scrum events:

• Sprint Planning: The Product Owner shares current KVA trends (such as declining Current Value or low Time to Market) to contextualize the selection of Product Backlog items.
• Sprint Review: Stakeholders and the Scrum Team inspect both the functional Increment and the updated market telemetries (CV and UV metrics) to inform future backlog ordering.
• Sprint Retrospective: The team evaluates Ability to Innovate (A2I) metrics—such as technical debt, code maintenance load, and build frequency—to formulate actionable process improvements.