When the Capability Maturity Model^® first came out, it confused a lot of people. It had a Key Process Area (KPA) named "Software Quality Assurance," but that KPA didn't have the first thing to do with testing, or with peer reviews of technical artifacts. And those topics were addressed in KPAs that don't even mention Quality Assurance!

Wait a minute! What's going on here? Why is this getting so confusing?

It's confusing because we in the software industry have gotten into a bad habit of using the term "Quality Assurance" to refer to things like testing and technical reviews, which are actually Quality Control activities. We've been doing this for so long, and so consistently that most of us don't even know what true Quality Assurance is!

Assuring Quality vs. Controlling It

Most of us have heard the truism that quality cannot be tested into a product; it must be built in, or it will not be there. This is the heart of the difference between these two concepts.

Quality Control is evaluative. It consists of activities that are designed to determine if quality was built into the product, and if not, where the deficiencies reside (so they can be corrected). These activities are done after the product has been built. Clearly, testing and reviews fall into this category. You can't test a program until after it has been built, and you can't review a document until after it has been written. Both sets of activities serve to evaluate the product to see how well it was built. In software, as in manufacturing, both testing and reviews are Quality Control activities.

In contrast, Quality Assurance is proactive. It consists of activities designed to assure that quality will be built into the product. These activities either precede the building of the product, or happen concurrently with it. What proactive activities do we engage in to assure the quality of our software? Unfortunately, in too many projects, there are few (if any) Quality Assurance activities. This presents a wonderful opportunity for us to improve the quality of the software we build, and make our projects run more smoothly at the same time!

QA: Process Assurance

In the CMMI^® (the next-generation successor to the original CMM) the Process Area for "Process and Product Quality Assurance" focuses on the processes and standards being used by the project. Its intent is to ensure that the processes and standards support the needs of the project, and that they are used properly and consistently. Appropriate processes and standards (if consistently used) will provide an environment in which our programmers can do their best work.

Is anyone doing process assurance on your projects? This may be an opportunity for you! The CMMI recommends that an independent oversight group do these things. The Agile methods and a few others like the Team Software Process^SM include the role of a process coach who operates as a member of the development team.

QA: Continuous Learning

Beyond the base of solid processes and standards, Quality Assurance also includes learning from past projects. Holding a project retrospective session (or lessons learned or postmortem, if you prefer) after each project or phase gives the team an opportunity to step back from the busy-ness of the day-to-day work and look for ways to improve. The key questions in these sessions include:

• What worked particularly well? How can we make those successes the norm on our projects?
• What went wrong? How can we make those problems less likely, or eliminate them altogether?

Of course, these sessions are only worthwhile if we actually do something with the information. Based on what we learn, we should make changes to our processes, procedures, methods, tools, or standards to make the next project or phase better. Then at our next retrospective, we can ask if those changes had a positive impact; and if we should adopt them as the norm, or abandon them.

QA: Avoiding Errors

Finally, QA includes learning from our defects. All of those defects that were identified by our Quality Control activities should be analyzed periodically to look for patterns. Are there certain types of defects that we inject regularly, or that are inordinately expensive to diagnose and fix? If so, the key questions we want to ask are:

• Is there a way to avoid these defects altogether? Would it help to adopt a different method or standard, or to change the way certain things are done? For example, are we investing sufficiently in software design? Would a different design method be more appropriate?
• If we can't see how to avoid these defects, how can we detect and remove them more cheaply? Can we find them in earlier testing (e.g. Unit instead of System test)? Or could we tailor code or design reviews to catch them before testing even starts?

Affording QA

But how can we afford all of this extra QA work that we don't do today? This is the great part: Quality Assurance activities pay for themselves! Most QA activities are a modest investment (e.g. a 2-hour retrospective session), but the benefits they yield in improved productivity and better quality are likely to far surpass those costs.

Integrating Quality Assurance activities into our project will assure that we build better products. Then our Quality Control activities will be less a matter of finding so many problems, and more a matter of confirming that we did, in fact, build quality in!

Related Links

Previous Columns by Alan

Placing the Quality Bet - The time you need for reviews is already built into your schedule. Just look under "Testing."
Planning for Quality - Without proper attention to defining what we mean by "good" and then planning for it, achieving the level of quality we need is far from assured.

Quality Assurance

Development Process Quick Reference
Lessons Learned Survey
Lessons Learned Meeting Agenda
Lessons Learned Meeting Report
Conducting a Basic Pareto Analysis

Quality Control

Guidelines: Completion Criteria
Project Overview Test Plan
Master Test List
Beta Test Plan
User Acceptance Test Plan