Taking Credit When Credit Is Due

Many factors determine the success of a new strategy. But which one is responsible for real results? Here's how to find out.

One of the most challenging issues in reporting the value of a particular project or program is isolating the effects of that project. Often managers and professionals involved will make this comment, "There are just too many factors to understand the impact of this project; we cannot separate these other influences." This can actually be done in a credible way. Much of our efforts in the last two decades have focused on this issue and there are a variety of methods to accomplish this important part of showing project results.

Consider this scenario: you have just implemented Salesforce.com on-demand customer relationship management (CRM) application tool. The goal is to ensure that the tool helps develop consistency in practice and ultimately boost sales, increase market share, and improve customer satisfaction. However, when it is implemented there are many other influences driving success in these key measures. In fact, most dynamic market situations involve multiple influences at the same time -- some implemented during the same period of this CRM implementation. So the question is, "How do we show impact of the CRM application?" The good news is that it can be done credibly with minimum effort.

First, let's discuss why it is important. Some may argue that all of these resources are working and increased sales, market share, and customer satisfaction have resulted. So, why do we care what caused it? Today, executives need to know more. They need to know what is causing success, and therefore, we must understand the cause and effect relationship between the variety of projects and programs implemented to move key measures. Doing nothing about this is an unacceptable strategy.

So, what can we do?

First, perhaps the best method to consider comes from research -- the control group. In research practice, this requires an experimental group versus control group. Here we implement the project or program in one area (experimental group) and compare it to a similar group where the project or program is not implemented (control group). The post-implementation improvement in the measures suggests the impact is contributed to the project or program. The key is to find the groups and to make sure they are matched with criterion that is affecting the measures in question. Unfortunately, this is not a commonly used technique. According to a survey at the Global ROI Conference in November 2007 using iDNA Systems keypads, twenty-four percent of the studies conducted by the participants are using this method. When this is not feasible, other methods are available.

Analytical approaches involving forecasting the influence of the other factors is another approach. Outlining how these techniques work is beyond the scope of this particular column, but they are relatively straightforward and not too difficult if certain conditions are sustained. Unfortunately, they do not work in most situations. In our own work, about twenty percent of the time we can use them, which means eighty percent of the time we can not. There is a group of techniques involving expert input, and the key is finding the expert. Sometimes it can be the customer involved, but often it is the people who are involved in the project and program. After all, it is their performance that we are concerned about. They may know more about the affect of all the influences available than any individual. This approach is the estimation approach using participants, and it works like this. The individuals are provided a list of the factors that are influencing the outcome measures. In addition, they may add other factors. Then they discuss the various influences and their affect on the outcome measures. In a structured format, they then provide estimations as to an allocation of the improvement to each influence. In essence, they are developing a pie chart allocating the success achieved to the various influences. Because this is an estimate and the allocations are adjusted, they are asked an additional question, "What is your confidence in the allocation on a scale from 0 percent to 100 percent, where 0 represents no confidence and 100 represents certainty?" These two percentages are multiplied to make adjustments.

For example, in the situation involving the implementation of Salesforce.com -- if the sales force achieves several outcome measures, such as revenue growth, it is asked to indicate the connections. They are provided the other factors that could have influenced this, such as new promotion, new product development, market growth, and so forth.

In a focus group arrangement, the cause and effect relationship between each factor is discussed. Each person is given the same amount of time in a round-robin format, considering one factor at a time. After this discussion, allocations are made which represent the percentage of improvement that is directly related to each influence. Then they are asked to provide confidence in the allocation, as described above. If a sales representative indicated that forty percent of her sales increase is due to the use of Salesforce.com and that she is eighty percent confident in that allocation, then multiply forty percent by eighty percent -- a thirty-two percent increase would be allocated to the software. When this is repeated by the entire sales forecast and the totals are obtained, this is a credible way to allocate the success of a particular project. In essence, when the confidence adjustment is made, the value is understated. In other words, the confidence adjustment is the error adjustment. This process removes the error from the estimate.

Fortunately, we have seen the success of this in hundreds of settings and situations. When a more credible and analytical method can be used to isolate the effects of a program and estimates are used, as well, the estimates are usually more conservative. There is much research on the power of estimations and the power of input from the people who are doing the work.

In short, whenever there is a project and program implemented, the critical issue is to take only the credit that is linked to your project. There are ways to do this even if estimates have to be used. Estimates can be credible. They are not preferred as a method, but are used as a last resort.

For more information, go to any of our publications, for example Show Me The Money: How to Determine ROI in People, Projects, and Programs: A Step-by-Step Guide to Forecasting and Measuring Six Types of Value.