Products
and Services > Inspections and Defect Prevention > Software
Appraisals
|
Articles:
|
Product
Descriptions:
|
White
Papers:
|
Software
Appraisals
Software
appraisals include a range of activities performed to find and remove defects in
software early in the development life cycle prior to the start of testing. They are various types of appraisals including personal
reviews (also called bench checks), walkthroughs, and inspections. Personal
reviews are an efficient technique for cleaning up relatively simple logical
errors in a product prior to submitting it for inspection.
Walkthroughs are appropriate during the early stages of the development
of higher level products like requirements or system architecture. Inspections
are best used for finding inconsistencies, misunderstandings, and errors in
products like requirements, designs, code, and test cases. Appraisals can be one
of the most effective tools for producing high quality software on time and on
budget.
Over
the years there has been a lot of data published indicating that appraisals are
one of the most effective methods to lower cost and cycle time while
simultaneously improving product quality. Appraisals
work because they find defects early, when the removal cost is lowest.
The
cost of appraisals has two components: the time to review the product and the
time to fix the defects found in review.
The cost of running tests has a lot more components: compiling and
linking the product that will be tested, executing the test case, isolating
defects, and fixing the defects.
When
performing an appraisal, the time required to review the product is independent
of the number of defects found. The
time required to fix the defects is at worst the same as the time need to fix
them when they are found in test. It
can be a lot less if a defect is found and corrected in a spec before the spec
is actually used to generate products. The
cost of testing increases linearly with the number of defects in the product
under test, because the compile-link-test-isolate-fix cycle is typically
repeated for each defect. Unless
the required time is very small, it is likely that appraisals will have a significant cost
advantage over testing.
Well-managed
appraisals remove defects at a very predictable rate and cost. Testing is
notoriously unpredictable. It can
take anything from five minutes to a week to find a defect that requires only a
minute to fix. Eliminating the time
spent isolating defects and re-running tests to verify the fixes is what gives
appraisals their cost advantage. As more defects are found via appraisals and
less in test, there is a major reduction in the variability of test time
resulting in more predictable schedules and costs.
Finally,
removing defects with appraisals is much less prone to injecting secondary
errors than removing them in test. When
a defect is fixed in test, it is frequently in a high-pressure situation where
there is an intense focus is on getting the test case to pass.
This can make it easy to implement a fix that is inconsistent with
another element of the design and inject a defect somewhere else.
The later this happens in the development life cycle, the more likely the
secondary defect is to escape into the finished product.
The focus in an appraisal is typically much broader and the probability
of injecting a secondary defect is usually an order of magnitude lower.
However,
many organizations don’t perform appraisals at all, and many more don’t perform
them effectively. Without adequate
training and good
team leadership, appraisal teams are prone to waste time, and
tend to find a preponderance of minor defects.
Without measurements and pro-active management, even the best appraisal
process will not function close to its optimum performance levels.
The net result is higher costs and lower yields than necessary.
Unfortunately this situation is closer to the norm than the exception.
Conversely, with good training, good team leadership, and with measurement
driven management of the inspection process, it is possible to remove nearly all
of a system’s defects prior to the start of integration & test at a much
lower cost than for a code & test life cycle.
If
your organization is not performing appraisals, introducing them will bring a
quick improvement in development cost, product quality, and schedule
performance. There will also be soft benefits such as cross-training,
knowledge transfer, and enhanced standards compliance.
If
your organization is already doing appraisals, but you find yourself curtailing
them to make schedule, or your development staff complains about them and says
all the major defects are really found in test, then your need to stop and fix a
broken process. It is almost
certain that your appraisal process is ineffective and could very well be a
waste of effort.
If
your organization does not measure the effectiveness of its appraisal process,
and use the measurements to manage the appraisal process to target performance
levels, then it is almost certain that your process is not performing anywhere
near its optimum level. There are
many factors that degrade appraisal process performance.
Do you know what they are? Are you managing them to optimum targets?
If not, you won’t get the performance you could.
| 
