Investing in Quality: When is "Enough" Enough?

by Alan S. Koch, PMP

There is no doubt that we need to invest in producing a quality product. But it is not clear how much we should invest. Invest too much and we waste precious resources; invest too little and our product will be judged as poor-quality, and our project may be labeled a "failure." (Yikes!)

So, how do we find that sweet spot? How do we determine the "right" amount of quality investment?

You already have the information you need to answer that question. It is in the records from your prior projects. Both the projects that produced "good" quality and those that did not set the stage for you to make the right quality investment decisions this time around.

Harvesting Your Quality Data

We start by going back through our recent projects. Sort them into three piles: Sufficient Quality, Insufficient Quality, and Exceptional Quality. If you are lucky, you will have at least one project in each category. In this case, finding the sweet spot is a matter of comparing what we did on the projects that fell into the different categories, and using the differences as our guide.

But most of us are not lucky. We will find that the majority of our projects fall into only one of these three piles. And, unfortunately, it may be the "Insufficient" category that is overpopulated. Although this gives us much less information to go on, it still provides a good starting point. After all, if we know that the quality investments we made on those projects were insufficient, we know that we must invest more (or at least smarter)!

Look back at the projects that you put into each category. What kinds of quality investments did you make on each of them?

Counting the Cost of Quality

As we look back at each project, we must be careful to fully count the costs of quality on those projects. Some of those costs come to mind immediately, but others are less obvious. So it is important to take a few minutes to look at each.

Testing -- How much testing was done? And more importantly, what types of testing did we do? On software projects, several types of testing are likely.

  • Unit testing of the individual components programmers write establishes a good basis for building a high-quality product out of those components.
  • Software integration testing verifies that the components work well with each other and that interfaces work as designed.
  • System testing verifies that the whole software product actually satisfies its requirements.
  • System integration testing assures that the software product can coexist with the other products, devices, and databases in its environment.
  • Acceptance testing ensures that the end-users can actually use the product to do real work.

If your project is building a service, a physical thing, or anything else other than software, you can probably identify a similar set of tests that may be called for. Regardless of the nature of the product of your project, investing in the right kinds of testing is important.

Preparation for testing -- What kind of investment was made in testing before it started? For each of the types of testing that you did, how much time and effort went into planning it, defining a strategy, and mapping out the specific tests that would be run?

Random, flow-of-consciousness testing can rarely be expected to produce consistently good results. The testers may stumble across serious defects, but they could just as easily miss them. Well-planned testing that follows a defined strategy will have a much greater likelihood of finding the important problems sooner, rather than later. (This goes for unit testing as well as for any of the others!)

Review and Analysis -- Testing (using the product to observe its behavior) is not the only way to find defects. In fact, for many types of defects, visual or automated analysis of the product is a much more effective way to identify problems for removal.

It is obvious that using a tool to analyze work products will be very efficient. But many organizations have reported that having another person or a few of them (usually peers of the person who created the product) perform a technical review of the work is a very good use of the engineers' time, because it is so efficient at finding many kinds of problems.

Are you making use of these types of evaluations on your projects? And if so, what percentage of the product is being analyzed?

Defect Detection -- For the kinds of testing and reviews that you have employed, how effective have they been at finding defects? How many defects are detected per hour? We want to ensure that our time is well spent, so we may want to invest in the more efficient defect-detection activities. Of course, we may also find that we can improve the efficiency of those activities (for example, by planning testing instead of doing it randomly).

Defect Correction -- For each kind of testing and review we employ, how much effort does it take to correct the defects found? Organizations that measure the engineering time spent on defect correction have found that it is generally less expensive to fix defects found earlier in the project rather than later. If you find that this is true on your projects, you may want to shift more of your quality investment to activities that find defects when they are cheaper to fix.

Where Is This Data?

Many of us will find that we don't have good data about the hours that were invested in specific activities on our prior projects. We know that our testers reported a certain amount of time against the project, but we don't know how each hour was spent. By the same token, we know how many engineer hours were billed but we have no idea how many of those hours were spent fixing defects.

This is not an uncommon problem. If this information was not previously identified as important, then it probably wasn't measured and documented. But that doesn't mean your hands are tied. There are two things you can do about it.

First, you can estimate the missing information based on the data you do have. You may have to ask people to try to remember how much of their time went into which activities, and then use their best recollection to estimate the numbers. But that is a good starting point. It may not be terribly accurate, but it is better than no information at all.

Second, start collecting the pertinent information today. Even if you are in the middle of a project and have no detailed data about what has gone on to date, there is no reason not to start collecting useful data right now. That data will very quickly become a goldmine for you. You will discover that any data is valuable. And, of course, the sooner you start, the sooner you will have a trustworthy collection of data you can use to make important project decisions!

Using This Information

As you look back over past projects, you can learn from your experiences. Projects that produced acceptable quality can be your template for appropriate quality investments. Projects that produced unacceptable quality can serve as a baseline that you must exceed on future projects in order to have a chance of producing better quality. And your data can help you to pick out the quality investments that produce the best returns for the lowest investment. Imagine! Instead of investing more, we may be able to simply invest smarter! That is a recipe for wildly successful projects!

©Copyright 2000-2018 Emprend, Inc. All Rights Reserved.
About us   Site Map   View current sponsorship opportunities (PDF)
Contact us for more information or e-mail
Terms of Service and Privacy Policy