This is an article about lifecycles. It covers the problems with lifecycles, in particular their propensity to be created, redesigned and reinvented with disturbing regularity across the IT industry. Lifecycles are very expensive to define and implement, so should we continue this cycle or attempt to break it? This article discusses the underlying features of all lifecycles together with the strengths and weaknesses of each. Based on these features, it presents techniques to review existing lifecycles and to achieve the elusive goal of providing workable, durable lifecycles.
We Need a Lifecycle
New, appropriate lifecycles. Each time, numerous resources are involved in workshops and working groups. Each time, it is planned as the new-and-improved magic bullet to replace old and faulty working practices. However, the inevitable complications cause it to be implemented with difficulty and often with a degree of watering down. As time passes, new markets are addressed and new technologies appear, the lifecycle becomes less able to support the changing activities. Eventually it falls into disuse or misuse. Soon after, a strategy is agreed to define a new lifecycle. And so it continues.
Hidden Costs of Lifecycles
The provision of a lifecycle framework is aimed to aid communications and support quality as everyone sings from a standardised, improved procedural hymn sheet. These principles are noble but the practice is flawed exactly because the lifecycle provides a framework. As the underlying structure on which everything else is built, it must be slipped under existing developments and product releases with the agreement (and understanding) of its managers and users. Also, to ensure its appropriateness each stage of the lifecycle needs to document appropriate activities, deliverables and controls in a high level of detail. This is often harder than anticipated as it must include the often ad-hoc needs of the business.
Reinvention – Problem or Necessity?
Developers of lifecycles might be rebuked for wasting time and money. Attempts to define the ultimately flexible lifecycle often end in failure, as either a monster is created which is too complex for anyone to use or as working groups discover that the plethora of conflicting requirements make the whole thing impossible. Specific lifecycle developments are also scorned – after all, what purpose is served by reinventing the wheel with time-wasting workshops, yielding a result so limited that its future replacement is inevitable?
What is right? Should we strive for the ultimate, one-size-fits-all and infinitely flexible lifecycle? Probably not. Alternatively, should we dispense with lifecycles altogether and adopt a reinvention policy, using the project plan to define its own sequence of activities? Unlikely. So we are faced with finding a middle ground, balancing the needs of structure and repeatability against those of flexibility and durability.
Macrocycles and Microcycles
The place to start in defining this middle ground is to recognise what we are trying to achieve. There are some fundamental truths for any activity or sequence of activities, namely that there will be a problem to solve, there will be a solution to define and there will be some work done to achieve the solution. Terminology is always contentious – one teams stage is anothers step and one persons project is anothers task. For the purposes of this article the following working definition is used:
A lifecycle is an ordered group of activities to achieve a goal
It is of fundamental importance to note that a lifecycle can exist at any level, and its purpose is to define the relationship between the activities.
Lifecycles, activities and goals range from strategic (or macro) to purely tactical (or micro), for example:
– defining a business strategy and subsequent running of the business
– developing and implementing a programme of change
– running a project or developing a product
– carrying out a project task, for example writing product documentation
– managing the logging and resolution of a fault
– running a meeting.
Activities at a given level will consist of lower level activities. For example, programmes are made up of projects which consist of tasks. As lifecycles group activities, so can they contain lifecycles. Just as each activity should be aware of the requirements of the levels above and of its impacts on the levels below, so lifecycles need to take into account both vertical and horizontal relationships.
There are essentially three types of relationship between activities, namely concatenation, iteration and verification.
A set of activities has to take place in the right order. More importantly, the right activities have to happen in the right order, either to give the right response to the input event or to conduct the right process on the input data.
Apart from the central do it!, activities may include:
checking the validity of input
defining and agreeing what is expected
agreeing a way forward
checking the suitability of the response
testing and/or reviewing any outputs
delivering a product or response in an appropriate manner.
Activities should take place in the correct order, for example requirements must be defined before they can be met. This does not necessarily mean that one activity has to complete before another can start. It may be sufficient that one activity reaches a suitable state.
A common weakness of concatenation is that micro level activities are carried out without sufficient regard to the macro level. For example, a system specification may be produced by a specification team without due regard to design aspects of the system, or marketing may sell to a customer a solution which would be too costly to develop. This weakness is exacerbated should communications difficulties exist between the teams (the so-called over-the-wall mentality).
Golden rules are:
focus on the required result of a given activity but be aware (hence guided) by the activities surrounding it
involve the business during all activities in the lifecycle
optimise communication between teams involved in the different activities
An example of a concatenation lifecycle is the Waterfall lifecycle for software development, which employs concatenation through the stages Feasibility, Analysis, Design, Implementation, Test, Integration and Validation. The Waterfall lifecycle expects each activity to be completed before another can start; also it has no look-ahead such that impacts of earlier decisions on later activities can be determined. Backtracking is permitted, but generally this is to solve problems discovered following the completion of a stage – for example a flaw is discovered, during the design phase, in the specification. For these reasons it is normally the first lifecycle on the evolutionary scale of lifecycles, as it is normally superceded by lifecycles supporting Verification and/or Iteration, described below.
Any activity at any level is made up three stages – defining the requirement, meeting the requirement and checking that the requirement has been met. The checking stage, called verification, is sometimes forgotten in the rush to complete, but that doesnt make it any less important.
There are two rules of thumb:
verification is an essential part of any activity because it is necessary to check whether a requirement is met before delivery takes place
verification should be considered throughout the activity – for example, when the requirement is being defined, the question how can we check this? can be asked.
Requirements can be specified to include the mechanisms for their verification; activities, once complete, can be checked against these requirements. The resulting lifecycle construct is a V-shaped sequence of activities. Vs usually exist at the macro level, where each stage of a lifecycle can have a corresponding verification stage. Vs are equally valid at the micro level, where the requirements of an activity should be documented in a way that the can be (and are) verified, for example that a meeting covers all its agenda items. A hierarchy of activities will be a hierarchy of Vs.
There are few, if any, shortfalls of incorporating verification as a lifecycle feature. More time is required up front, but this is generally paid back tenfold as a reduction in the rework when the inevitable problems are found.
An example of a verification lifecycle is the traditional V lifecycle for software development which has been widely adopted in the industry. In this lifecycle, verification works with concatenation to provide the required look-ahead. However, it shares the main weaknesses of Waterfall, namely the required completion of a stage before another can start plus the use of backtracking to solve problems rather than improve understanding.
Iteration (or looping) of activities is an essential lifecycle feature as it enables activities to expect and deal with the effects of change.
There are two main drivers for iteration:
situations change rapidly, and as a result so do requirements, hence it must be possible to return to activities that were previously considered complete
the result of carrying out activities is that the problem to be solved is better understood, hence it becomes possible to carry out previous activities in more detail
Iteration is far more than backtracking, which refers to solving problems caused by previous stages. Iteration is where an activity is carried out in full knowledge that it will be returned to. For this reason, activities are rarely completed in the traditional sense, rather each activity is only carried out to the level of detail necessary for the next activity to start. This implies that the activity takes into account the requirements coming from the next activity, for example which areas to concentrate on first. An activity can be returned to at any time to further develop its ideas and correct its mistakes, in parallel with other activities.
This blurring of the boundaries between activities is very different to traditional ways of working and requires a whole new management style. For example, teams need to be involved in all stages otherwise the ease of iteration becomes crippled by inter-team communications. Iteration requires a fast, flexible approach to be workable but there is a tendency for the process to descend into backtracking.
The weakness of an iterative approach is that, as it becomes harder to define what each activity should deliver at any given moment, it tends to cause spiralling backwards whilst losing sight of moving forwards. Good management, again, ensures that the project moves forwards.
The spiral of iteration may exist at a macro level but is just as necessary both between and within activities. It must always be possible to move forward with only part of the answer and with the door always open to come and improve the question. There is no such thing as a fixed requirement, nor is there any such thing as the complete answer. Fitness for purpose remains the main criterion to judge whether an activity is complete.
Lifecycles incorporating iteration include those proposed for Rapid Application Development (RAD). In the past these have yielded mixed results, with bad experiences stemming from using RAD as an excuse for throwing systems together and hacking code. Other iterative lifecycles include the Spiral lifecycle, DSDM and others which incorporate the concept of the prototype as the central element which holds the process together. The prototype concept is not suitable for all applications.
All activities, at the macro and micro levels, can benefit from the advantages of concatenation, verification and iteration and should be prepared to recognise and avoid their downsides. These qualities and issues should be taken into account when either reviewing existing or developing new lifecycles.
New lifecycles should never be defined from scratch and should rarely be bought off the shelf as they should be appropriate to the needs of their users and managers. Templates such as the traditional V, the Spiral and DSDM can be used as a foundation, a starting point. The users existing processes, activities and terminology will provide the requirements for the new lifecycle. Hence a new lifecycle should:
contain the proven advantages of industry best practice as a template
incorporate and be driven by the requirements of the business
be reviewed against the strengths and weaknesses of concatenation, iteration and verification
The implementation of a lifecycle is, itself, a series of activities which should walk their own talk – that is, should adopt good lifecycle principles – for the implementation to be successful. Any user training (and there should be some) should reinforce the underlying rationale and purpose, which are traditionally hidden and largely forgotten by the time the detail of standards, controls, document templates and development methods are rolled out.
We need lifecycles as they provide a structure within which we can work. There is no one-size-fits-all lifecycle, this is an impossible goal and drain on resources. However there is industry best practice to build on and existing working practices from which to source requirements. By looking at the underlying qualities of lifecycles, their advantages and shortfalls, we can define, design and implement lifecycles which are both understandable and durable.