The Complete Guide to Value-Oriented Metrics in the AI Era: Shifting from Output to Outcome

Artificial Intelligence (AI) technologies are fundamentally reshaping how we work. With automation, data analysis, and personalization capabilities, teams can operate faster and more efficiently than ever before. However, this speed and efficiency bring a critical question to the forefront: Is it more important how fast we run, or if we're running in the right direction?

Traditional Agile metrics, particularly common among Scrum teams like 'velocity' or 'story points completed,' often focus on 'output' – how much work is being done. In a world where AI handles a significant portion of the work, focusing solely on output can be misleading. An AI algorithm can complete thousands of operations in seconds, but whether these operations create real business value is a separate matter.

In this guide, we'll explore an approach that enables teams in the AI era to not just do work, but to truly create value: the Evidence-Based Management (EBM) framework. We'll discover how to shift from output to outcome, with practical examples and actionable steps.

Many Agile teams rely on metrics like 'velocity,' 'number of tasks completed,' or 'lines of code' to measure progress. These metrics were once valuable for understanding a team's internal efficiency. However, with the rise of AI, this paradigm has shifted.

For instance, an AI-powered code completion tool can significantly increase a developer's coding speed. While the team's 'velocity' might appear to increase, it's unclear how much value this newly written code generates for users or contributes to business goals. In fact, the complexity or maintenance cost of AI-generated code could impose additional burdens on the team in the long run.

Similarly, a product team might develop automated reporting features using AI. This could translate into many new reports as 'output.' However, whether these reports truly help in making better decisions, simplify users' work, or contribute to business growth cannot be understood by merely looking at output. Traditional metrics risk rewarding 'effort' while overlooking 'impact'.

This is precisely where Evidence-Based Management (EBM) comes into play. Developed by Scrum.org, EBM is a framework that approaches product development and value creation with an empirical mindset. EBM helps teams focus on measuring not just how much work they do, but how much value that work genuinely creates.

At the core of EBM are four Key Value Areas (KVAs):

These four areas serve as a compass for teams on their journey to value creation. By measuring their current state in these areas and setting improvement goals, teams can effectively transition from output to outcome.

The product team at 'TechFlow Solutions,' a SaaS company, had long been tracking progress with metrics like 'number of features shipped weekly' and 'story point velocity.' The team was working on an AI-powered recommendation engine, completing many new algorithms and integrations each sprint. However, Product Lead Sarah noticed that this high output wasn't translating into the expected impact on sales or customer satisfaction. Churn rates remained high, and new customer acquisition was slow.

Sarah introduced her team to the EBM framework. First, they focused on the 'Current Value' area, seeking to understand the existing impact of their recommendation engine. Instead of traditional metrics, they started tracking metrics like 'purchase rate from personalized recommendations' and 'customer lifetime value.' For 'Unrealized Value,' they used customer feedback and market research to form hypotheses about 'what new personalization features customers desired.'

The team realized that their AI engine needed to deliver recommendations that truly engaged customers and influenced purchasing decisions, rather than just generating more suggestions. They retrained their AI models to optimize for these new value-oriented metrics. As a result, within a few months, churn rates decreased, customer lifetime value increased, and TechFlow Solutions concretely saw how AI could contribute to real business outcomes.

Struggling to define strategic metrics and structure your product backlog to align with these value areas? AgileKoc's Product Backlog Architect tool helps you easily align your product vision with value-driven goals. Explore the Product Backlog Architect tool and start building a value-oriented backlog.

Integrating EBM into your team is an iterative, empirical process. Here's a framework to guide you:

Finding the right metrics for each KVA will vary based on your team's context and the nature of your product. The key is that these metrics should be directly linkable to business outcomes. Need coaching support to help your team embrace EBM principles and cultivate a value-driven mindset? AgileKoc Learn offers in-depth EBM training and resources specifically designed for Scrum Masters and product leads. Explore EBM resources on AgileKoc Learn and transform your team.

The transition to value-oriented metrics may not be easy. You might encounter obstacles such as cultural resistance, difficulty in finding the right metrics, and pressure for short-term output. However, patience, continuous communication, and a team open to continuous learning will enable you to overcome these challenges.

EBM is not a one-time implementation; it's an ongoing journey of learning and adaptation. As AI technologies evolve and market dynamics change, the definition of value will also shift. Therefore, it's crucial for teams to regularly review and refine their metrics and approaches.

Remember, in the AI era, true competitive advantage lies not just in doing work faster or more, but in delivering products that genuinely create value and solve customer problems. EBM is a powerful framework to guide you on this journey.