Measurement is essential for success.
Many companies collect product development metrics such as R&D Spending as a % of sales or revenues from new products. But too many companies are measuring the wrong things and ending up with the wrong results. Teams are tasked with collecting product development metrics because they’re easy to gather rather than useful. Or they focus on validating whether they succeeded or failed instead of measuring the behaviors that drive success.
Predictive product development metrics are the solution to this problem. Predictive metrics are fundamentally different from results metrics in that, instead of measuring an outcome, the organization measures a process or behavior that drives a clearly defined result. These metrics provide a direct line of sight between the project goal and the three to five key measures that, when defined and frequently gathered, will best predict the likelihood of achieving that goal. Effective new product development metrics track the critical few factors that allow project leadership to seed improvements throughout the organization.
Your strategy should also include operational goals and metrics for product development, such as goals for customer satisfaction or the percentage of revenue from new products. For example, within your portfolio of products, you may direct 10% of the R&D budget toward “new, new” products (those products that involve new technologies and new markets). In this case, the metric you choose should depend on your company’s risk profile, which governs the balance between breakthrough and incremental product development projects. However, this strategic metric measures a result. While it points the arrow toward ‘True North’ it does not measure how effective your 10% investment really is. Ideally, organization should also have a predictive metric that demonstrates that the team is executing effectively, for example milestone hit rate.
Effective product development metrics systems have a clear line of sight to broader, strategic goals. The key is to measure behaviors that are tied to specific initiatives that support theses goals. The sweet spot is between two and four predictive metrics. Most organizations have too many metrics. Metrics should be retired after they have served their purpose. Our tools include a metrics quality checklist to ensure that teams are measuring the critical few factors that predict positive change.
Who are Product Development Metrics for?
TCGen works with product companies across industries. We specialize in Consumer Electronics, High Tech, Medical Devices, and Computer Hardware and Software. But any company that wants to change its culture and create more predictable improvements can take advantage of our expertise in product development metrics.
Larger Companies Managing Many Programs At Once
As companies grow and programs multiply, discipline can erode. Operational measures as well as predictive metrics to drive specific behaviors instill discipline and encourage continuous improvement as businesses mature. It takes small steps and steady discipline to keep the culture of a company intact – or to improve the culture as the need arises.
Companies seeking Agility
Not just for software any more, product companies are beginning to take advantage of the numerous tools in the Agile kit that can help them get an edge. Having the right product development metrics is essential for motivating change and pushing the right levers to shift a culture in the Agile direction.
Does this sound like you?
How do we structure Product Development Metrics engagements?
Typically, TCGen’s Product Development Metrics engagements have four steps: assessment, developing the key metrics, training; and then piloting metrics to manage change by monitoring behavior.
Assessment and Prioritization
First, we conduct an assessment of the current state and then use these findings to pinpoint a small number of improvement levers. Desired organizational changes are teased apart and divided into incremental improvements. We then choose the most effective levers for these small improvements, and then sequence and prioritize them.
Target Metric for Each Improvement
For each of these key improvement levers, we establish a target metric with the aim of seeding the changes within the organization so that they take root. Only when our metrics indicate that an improvement is taking hold do we move on to the next transformation. Metrics that we often use to predict outcomes are behavioral. We measure behavior to manage improvement. For example, when putting in place a new, and lean set of milestones, we measure the percent of teams that use the milestone’s new names in their status reports. Very simple, yet precise metrics can be instrumental in gauging a company’s progress.
Half Life Target Curves
Our Half-Life Diagram predicts how fast you can expect to improve or make an impact in your organization depending on organizational and technical complexity. This prediction provides a continuous estimate of progress, against which you can plot your own actual progress and course correct if you see an early deviation. The Half-Life Diagram consists of an equation and a graphical plot of expected improvement over time. It uses estimated degrees of technical and organizational complexity (high, middle, and low) as an input, and then estimates the rate of improvement based on a study of nearly 100 similar projects. This tool generates a target curve over time that a project team can use to gauge its progress.
We then test our assumptions around predictive metrics with several teams and key executives. We pilot metrics with particular teams. This is largely a learning exercise to understand the specifics of implementation. Having tested the veracity of the method and its implementation, we then provide training programs, in the form of workshops, for all relevant stakeholders.
We phase in the new product development metrics by first requiring only a subset of teams to use them. New teams adopt the process as they form, and we offer retraining at that time. We then attach compliance metrics with a clear goal, on the order of 95% compliance within three months.
In what kind of timeframe do organizations see change after implementing Product Development Metrics?
It depends on the type of metrics we are asked to implement. Typically, metrics are part of a consulting engagement – used primarily to detect progress and identify hot spots. Sometimes metrics engagements are stand alone in areas such as: Innovation, execution effectiveness, and Agility. If an organization has a craving for metrics as a management tool, we have training and implementation services that follow a phased assessment/implementation approach.
TCGen’s product development metric assessments take about one to two months. It takes approximately another month or two to perform further research and make recommendations on the most important levers of improvements and the predictive metrics that will establish them in the organization. We also build the required training.
Teams will tend to see improvement in the mid-term, over the following two to three quarters following our engagement. Managers will begin to see a learning organization that is able to improve rapidly and predictably, with the right measures in place to monitor behavior over time. Our half-life diagram can then track the speed of adoption of these new behaviors.
What kind of internal resource commitment should a company plan for when considering a Product Development Metrics improvement engagement?
A Point of Contact
Companies should plan to provide a contact who will serve as an internal point person for the Product Development Metrics improvement engagement. This point person, usually an individual contributor or team manager, serves as the resident expert and as our guide through the organization. He or she helps to organize meetings and functions as a project manager. Helping TCGen to improve the organization’s product development metrics should be this individual’s top priority.
Typically, in a product development metrics engagement we also work with an internal consultant or someone at the Director or Vice-President level who serves as our internal champion. This individual will act as the driver of the engagement for the client organization, providing access to data, and attracting the attention of upper management as needed. If we are implementing metrics as part of a management strategy, the champion is the most senior product development executive. SVP of Product, SVP of Engineering are typical titles.
Teams may be asked to provide access to project data such as schedules, budgets, project plans, roadmaps, staffing by function, project completion rate, or other artifacts for use in project histories and other research tied to our assessment phase.
What do people say about their experience after a Product Development Metrics Engagement?
Our product development metrics clients report greater team empowerment and more rapid and continuous improvement. With the tools we provide, teams can begin to stop measuring things that do not make a difference and begin to measure the critical few metrics that drive behavior and root improvements across the organization. Teams fully empowered are a key to consistent innovation.
The best testimonial we receive is repeat business. Companies that have taken advantage of our product development metrics expertise often ask TCGen to solve wider challenges or to duplicate our prior success in another division. Metrics involve system-level transformation. Companies that take advantage of our metrics expertise may want to apply our best practices to more granular challenges, at the program or project level.