Products
and Services > CMM
and CMMI >
Model-Based
Improvement Pitfalls
|
Articles:
|
Product
Descriptions:
|
White
Papers:
|
Model-Based
Improvement Pitfalls
Model
based improvement is a very strong paradigm and forms the basis of many very
successful software process improvement initiatives. However, in order to use a
model-based approach effectively, you need to understand how to avoid its
pitfalls:
Implementing
the practices verbatim
The key practices are meant to be representative.
Implementing the practices verbatim can be costly and may not reflect your organization's specific process needs very well.
This approach can result in an overly proscriptive process that actually
increases costs and slows release cycles. Expending funds to
make “improvements” that turn out to be counterproductive is much worse than doing nothing at all. It
can rapidly destroy the credibility
of your process improvement implementation with management and with the software
development staff.
Delaying
your metrics program
The
models lack an explicit measurable coupling between maturity levels and
business results such as cost and cycle time. The models don’t require
substantial process performance measurements until the higher maturity levels.
Consequently organizations that are just beginning a process improvement effort
often delay implementation of a
measurement program. Without measurements, an organization can achieve
maturity level 3 without understanding the true costs and benefits associated
with process improvement. Without a solid understanding of the
return on investment (ROI) and its relationship to business results, it can be easy to
lose sponsorship
during a re-organization. Worse yet, without measurement it is quite possible to implement
practices that meet the goals of the KPAs (PAs) but are not cost-effective from a
business standpoint. The basic problem is that the models
provide qualitative KPA (PA) goals and typical
practices to meet these goals. If your organization has an efficient
implementation of the practices that meets the goals, you can realize improved business results.
However, if your implementation is inefficient relative to your organizational
needs, you may meet the goals, but at a cost that will outweigh any
benefits. Process performance may be static or might actually degrade relative to
business goals.
Open
loop management
Another
consequence of deferring measurements is that even good process will under perform
without active management. Management means feedback and
control. With active management, actual process performance is continually
checked against process capability. When they begin to diverge, management
actions are used to drive the process back in efficient operation. Without
good measurements the process goes “open loop” and tends to degrade in
performance over time.
Strict
adherence to the staged representation
Often
times, the staged improvement model is taken too literally.
Inspections are a level 3 practice in the CMM. When implemented
effectively and efficiently, inspections can be one of the
best ways to quantitatively improve cost, product quality, and cycle time. Many
organizations with a level 2 goal ignore inspections totally because they are a
level 3 practice. There is certainly room to introduce a few higher
maturity level practices early on if there is a good business case for them.
Sometimes, the staged improvement model may not be the
best choice for an organization. If
applied literally across a large organization, it can result in excessively long
cycle times for improvements and repeated, costly false starts. Consider a
500 person software organization with 20 active development projects. In a strict
application of the staged representation, we would try and get all projects practicing at
maturity level 2
before moving any of the projects to level 3. This can take a very long time.
Without the benefit of experience, an organization could roll out an
inefficient implementation of a process across an entire organization, expending a considerable amount of time, money, and good will before
it can be fixed.
One
alternate approach is to select several pilot projects
and move them rapidly to a high maturity level. Experience gained from
these projects will make it less likely that there are organization wide false
starts with ineffective improvements. Having a few projects operating at higher
maturity level permits an accurate assessment of costs and benefits.
Moreover,
these projects will generate pull for other projects to upgrade their practices
as well, overcoming organizational resistance to change.
The
CMMI offers a continuous representation as an alternative to the traditional staged
representation, but many organizations have a "staged" mindset left over from
experience with the CMM and they are reluctant to experiment with it.
Emphasis
on documentation
Both
maturity models are fairly “document
centric”. There is a long list of required policies and procedures that
need to be in place for each KPA (PA). These policies and procedures
must be available during an assessment in order to provide evidence of the
institutionalization of the practices. In the quest for a
level, it is easy to generate a lot of policies and procedures that satisfy the
KPAs (PAs) but are never used. This is a waste of time and money, and it squanders the
credibility of the improvement process. Policies and procedures need to be
short and they need to be used by everyone concerned. These
procedures are often written to have minimal impact on existing practices in an
attempt to shelter a
busy
development staff from unnecessary upheaval. The problem is that this pre-supposes there is no need for improvement. If the practices of most of
the organization are not affected, how can there be a significant improvement?
In this case, what is the point of the investment in model based improvement? If
you really believe things are fine the way they are, you are better off doing nothing!
Poor
organizational alignment
The
final failure mode concerns organizational responsibilities. The CMM recommends creating a Software Engineering Process Group (SEPG)
chartered with the responsibility of defining and improvement the
organization’s software process. The CMMI talks about an
Engineering Process Group (EPG), reflecting its broader focus. If the responsibility for deploying improved practices
is given to the SEPG (EPG), and the project managers have the responsibility to
get product out the door, it is possible to set up a situation where the SEPG (EPG) is
busy putting practices in place and the project managers don’t use the new
practices on their projects. After all, project managers' goals are based
upon shipping product. They
have no incentive to take a chance trying new practices on their project. The result is
frustration for all parties and a waste of organizational time and resources. The solution
is that senior management must make continuous improvement everyone’s
responsibility.
Summary
It
should be emphasized that there is nothing wrong with the models. They are
just not
prescriptive enough to avoid these problems. In some sense the models are like a
“necessary but insufficient condition” for improvement relative to business
goals. Failing to meet the model goals indicates a high likelihood of
consistently missing business objectives. Meeting the model goals can correlate
with a higher likelihood of meeting business goals, provided the practices used
to meet the goals are efficient and effective. There will be no correlation
when the practices are not efficient and effective.
The
net result can be the all too typical situation where an organization expends
considerable effort moving up maturity ladder without making significant improvements
in cost, quality, or cycle time. In some government contracting circles
this can be considered an acceptable cost of doing business because contract awards
are often predicated on maturity level. In most other business, however, it collapses
when someone in management notices that the are significant expenditures without
measurable returns beyond a process maturity level.
| 
