CONNECTING WITH QUALITY

Did That Process Change Work?
Four Steps to Better Processes

by Alan S. Koch, PMP

There is an old saying that goes, "You can't manage what you don't measure." This saying has survived the test of time because it is an essential truth. And it applies to more than just organizational management or even project management. It is just as true for process change.

Coming up with good ideas for process changes is no different from coming up with good ideas for product requirements; it is only the first step. The real work follows that step. And that real work is what must be carefully managed—and carefully measured!

Why are we changing our process?

The first step in any process-change is to determine precisely what we are trying to accomplish. What are our goals? Why are we making the change? Often our goals are to remove some pain from our projects.

• Perhaps we have experienced communication breakdowns, so we are trying to open better lines of communication.
• Maybe technical mistakes cost us a fortune, so we are trying to adopt more reliable engineering methods.
• Or it could be that we have a customer who is requiring that our processes comply with a given standard.

Identifying our primary goal is an important start. But as with a software product, there are may other kinds of requirements that are needed by the various stakeholders. A process must accept inputs from different sources, provide outputs to various stakeholders, and allow people to do their jobs effectively. In addition, a process must be efficient, user-friendly, easy to learn, and on and on.

If this is sounding a bit like Requirements Engineering for software then I am making my point. And just as with software requirements, process requirements must be specific, concrete and measurable. They are our key to determining if what we did had the effect that we intended.

How will we measure it?

After we have established the goals and requirements for a process change, our next step is to identify the metrics that will tell us if we have achieved what we set out to do. How will we be able to objectively judge how effective the change was?

For each goal and requirements, we must decide how to measure success. For example:

• If our goal was to improve communication, then we might measure the number of bungled hand-offs per month, or the frequency of disagreements about prior decisions.
• If our goal was to make fewer technical mistakes, then we might measure the number of person-hours expended in rework.
• If our goal was to comply with a given standard, then we could measure the number of compliance issues each week.

And we don't want to focus only on our primary goals. Each Requirement should be supported by a metric to help us to determine if we are achieving it.

Establish the Baseline

With our goals and metrics identified, we're almost ready to make some changes. Almost!

The last step in preparation is to collect the metrics on our current process, before we make any changes. This is an easy step to skip, especially if the need for a change is pressing. But without these baseline measures, how will we know if our changes had the desired effect?

You might argue that it will be "obvious" if things get better, but this is problematic for several reasons. For example:

• People generally find process changes to be disruptive to their work, so they may judge the experiment as a failure when in fact, it is producing the desired effect.
• If things appear to be "a little better," that will raise questions about whether it will be "worth it" to continue with the new process.
• In some circumstances, when the current process is very painful, people may judge any change to be "good", whether it actually achieved the intended results or not.

The good news is that you may be able to glean your baseline metrics from information that is already available in your records. That would give you the benefit of a clear baseline without postponing implementing a change.

But even if you find that you must collect the metrics from scratch, you still may not experience a significant delay. Deciding on how the process will be changed, then preparing the necessary guidance, forms, procedures, and systems can take significant time. If you are collecting your baseline data while you are doing this process development, then your new process and your baseline data may be ready at about the same time.

Keep on measuring

As you begin the process change, you will want to continue to measure the same things that you did to establish your baseline. Trend information will be a much better indicator of the current and future effect of the change than comparing your baseline with an isolated point in the future.

Consider the case of a change that is supposed to increase people's productivity. Because every process change takes time for people to assimilate, we would expect an immediate productivity decline followed by a recovery after people become proficient with the new process. Suppose our baseline productivity is 145 Primlads per Quarbrel (P/Q), and it is 145 P/Q one month after the change. Can I conclude that the change had no effect on productivity? I really can't know for sure.

If I had continued measuring productivity as the change was implemented, I might have seen this:

• Baseline = 145 P/Q
• Week 1 = 75 P/Q
• Week 2 = 80 P/Q
• Week 3 = 105 P/Q
• Week 4 = 145 P/Q

The trend in the data is one of acceleration! Productivity increased:

• 15 P/Q between weeks 1 & 2
• 25 P/Q between weeks 2 & 3
• 35 P/Q between weeks 3 & 4

With this trend, I see that I must continue measuring, because productivity levels have not yet stabilized. I can guess that the change will be successful in achieving better productivity, but I must measure to bear that out, and to quantify how much of a difference it made.

What if you are not concerned with productivity? Trend data will still be important to you. The ups and downs in the metric that you are interested in may not be as predictable as productivity. If you aren't watching trends, then you will simply be guessing about them!

When can you stop measuring? If your goal is something that is important to your organization's health, then you may want to continue to check that metric, at least periodically, just to be sure that things are still going well.

Your organization must adjust their processes from time to time. Sometimes these changes do not achieve the results that you expect. To be sure of your results, you must:

• Define your process goals
• Identify informative metrics
• Establish the process baseline
• Continue watching the numbers

These four simple steps can be the difference between making random process changes with uncertain returns, and taking well-defined steps in the right direction.

---