Are green fields always swamps?

My family are currently building a pond in our back garden. Two days, say the manuals. Two months, says my back – if you’re lucky.

The trouble is, the pond is only one project we have in hand. Half the garden in being redeveloped – an 8-metre fir has been uprooted, the old (and, it turns out, asbestos-lined) summerhouse has been demolished, a new greenhouse (right by the pond) has gone up, the beds have all been massively redesigned, half the major shrubs have been moved (and probably killed), the shed has had to be completely rearranged to accommodate a ton of drying firewood…

Not so much green field then as builder’s yard – or now, after we have tramped all over it for weeks on end and dug up everything and dumped tree tons of topsoil and dug up half of it at lest twice just to get going - swamp. No wonder it’s take so long to complete the pond. The plan is in flux, everything around it is being changed, the very earth is unstable.

How like a so-called ‘green field’ site. And how very unlike a real green field. Had we just left everything else alone, we could have simply dug a pond into nice stable earth and lawn, and quite possibly done it in two days. Instead, just like our back garden, on a ‘green field’ site, everyone is trampling over it, struggling to execute their conflicting projects and making hundreds of quick fixes and claiming this space here and that system there.

So like most of the programmes I have worked on – the whole management processes, architecture, systems and data is often the last thing to be created, so everything is just like a swamp – so many interim and temporary solutions, almost all of which will gel into permanent – and permanently obstructive – blots on the landscape.

Ho hum…

Advanced test scripts

All IT and most business organisations will be familiar with the idea that they have to test systems. A basic tool in this is the test script.

A test script sets out a succession of a step-by-step instructions to carry out a test. It is crucial that testing be scripted to an extent commensurate with the risks of not testing properly, but when looking at either corporate standards or individual projects it’s striking how seldom even the minimum standards are met. Do your scripts state the expected result, so you tell unequivocally whether the test has been passed or failed? In my experience, most don’t. Nor do they tell the user what preconditions must be satisfied before the test can begin (navigate to…, using these privileges…), or tell you how to check the result, and so on.

So here is my recipe for a truly complete test script. You won’t want to use it because it’s long and complicated and you can’t see the point of it, but if you can tell me which fields are not needed by anyone with a legitimate interest in your testing, feel free to take them out.

The script falls in five sections:

1. Document control
2. Setup information
3. Execution
4. Outcome
5. Result

1. Document control data
   
   • Identifier.
   • Test name.
   • Reference no.
   • Parent identifiers.
   • Author.
   • Authoriser.
   • Preparation date.
   • Version.
2. Set up information
   
   • Planned execution date.
   • Tester
   • Function.
   • Summary of test’s objective or purpose.
   • Test condition(s) implemented.
   • Positive/negative test?
   • Start point (i.e., navigating to appropriate screen/process/field.)
   • Set up/Initial conditions.
   • User ID/privileges required.
   • Preceding actions/tests to prepare the application.
   • File selection, parameter settings, etc.
   • The preconditions for the test as a whole (Eg, account no., currency, etc.)
3. Execution (probably in table format)
   
   • Step no. (if sequence is significant.)
   • Location (Screen/form/field name at which testing should begin enter, for example, “Go to screen…”, “Select ‘Reports’ menu”, etc.)
   • Input data.
   • Data to be entered.
   • Option(s) to be selected.
   • Test actions (Step by step, checklist-style – e.g., “Enter data”, “Select option A”, “Click on Submit”, etc.)
   
   
4. Outcome
   
   • Actual test time/date.
   • Actual result.
   • Checked boxes (against each test step, to confirm completion.)
   • Notes, with a general prompt to record anomalies, unexpected results, unplanned steps, & unusual system behaviour.
   • Narrative/commentary (to support re-runs & regression testing.)
   • Sign off.
   • Tester’s name & signature.



5. Test result
   
   • Expected result.
   • Method for checking actual against expected (if not just “Check actual vs expected results” - including automated file comparisons, etc., as appropriate. E.g., checking back-end systems, end-of-day report, messages, etc.)
   • Pass/fail.
   • Cause of failure.(Eg, “Comm320 failure”, “Data feed”)
   • Defect reference field (to locate defect reports, anomalies, etc. my be needed at both step and script levels.)

Try it. Really, it works.

Invest in your corporate brain

The brain consumes about 20% of the body’s energy, and that is why we are by far the most dominant organism the world as ever seen.

From the point of view of development, I would say that formal processes represent about half of every company’s brain – a good deal of its memory, its practical skills, its controls of perception and behaviour, quite a lot of its capacity for reasoning, balance and coordination, and most of its basic language and social skills.

On the other hand, most companies' formal processes - their lifecycles, their operational mechanisms, their ways of working - are designed more like the brain of a crocodile. (The analogy is more exact than you might imagine.) They are spread out across they organism, they barely speak to one another, they are almost never tailored to the practical needs of their users, they are seldom created with a meaningful outcome or benefit in mind and never tracked or measured to see whether they are doing their job and seldom fixed even when they plainly aren't, they have no effective direction or ownership, they are formally changed in an arbitrary and impulsive manner, they are never changed to anticipate a real problem, and so on.

Without major evolutionary leaps processes like this will never evolve into something truly intelligent.

Why is this?

It is, I think, because we don’t bother to invest in them to the extent necessary. Perhaps we just can’t believe that they need maybe 20% of all resources. Perhaps, like the brain itself, the significance of these processes is lost on people who, almost by definition, cannot see what they contribute. After all, we only need to invest so much in a corporate nervous system because most the useful things an organisation does require a span of knowledge, control and attention no individual possesses. Hence the paradox of processes – they are installed because we cannot manage such vast and complex systems, but even when they are installed and doing their job perfectly well, we seldom construct, implement or support them in a manner that really improves our view of the whole. We are still cogs in a machine that, although it may now be working better, we still cannot grasp processes as a whole.

So what needs to be done to improve our grasp? There are quite a few things that can be done:

1. Engineer processes properly in the first place!
   
   1. Make it a collective activity, don’t under-estimate how much effort it will take, or the hidden cost of getting it wrong.
   2. Start with a clear end in mind – and constantly test processes to see whether they are doing their job.
2. Make sure everyone understands what the processes are for – i.e., don’t allow people to wander blindly through their work.
   
   1. Train everyone and train in detail. The ROI on training is higher than the ROI on almost anything else in management.
   2. Not training is not only completely counter-productive but also extremely demoralising and defeats the purpose of hiring intelligent, capable people.
      
3. Do not over-engineer processes:
   
   1. Define standards and processes in terms of functional goals, not detailed technical steps, so they can be implemented and adapted locally.
   2. Conversely, explicitly Maximise local discretion and flexibility, so decisions made remotely cannot inadvertently force absurd actions locally.
4. Make sure that you can fix any problems with the process.
   
   1. Instrument the processes to make sure they tell you how well they are doing without having to ask specially.
   2. Have an intelligent waiver/exemption process so local teams can escape the worst excesses of processes not defined for their purposes.
   3. Part of every well-defined process is the possibility of changing it. It will need changing, and you need that fact into it. So make sure that all processes are owned, with clear local accountability and authority to improve. Invest in the time and re-training needed to do that.

There’s a lot more, but if you are already doing all this, you probably have other things you should be focusing on instead.

Your SDLC is your company's brain

I have recently been asked to review a large insurance company's delivery methodology, and found a state of affairs I have not witnessed for about 20 years. It’s a sad thing that major companies routinely neglect their lifecycles, methods and processes, presumably because they do not appreciate just how valuable the potentially area. It’s almost like they can’t see what good their brains do, so they neglect them in favour of other, more obviously useful organs (mainly the stomach, I think), and as a result their brains shrink and they become still less able to evaluate those same brains' purpose, effectiveness or value.

In fact the brain is very good analogue of the an organisation’s development lifecycles. not least because it is the reason why we are by far the most dominant organism the world as ever seen. From the point of view of development, I would say that an organisation’s formal processes represent about half its brain – a good deal of its memory, its practical skills, its controls for perception and behaviour, quite a lot of its capacity for reasoning about cause and effect, balance and coordination, and most of its basic language and social skills. Yet in many companies development lifecycles are organised and managed like the brain of a crocodile, not a human being, and while that continues it will never evolve into an intelligent being. (The analogy is more exact than you might imagine.)

But of course, the human consumes about 20% of the body’s energy, while most development lifecycles would be lucky to receive 1% of a company's attention. Which is odd, to say the least. Any improvement in process brought about by improving the local development lifecycle would surely need to change in performance by at least, say, 4-5%. In a £20 million programme, that means spending £1 million on their lifecycle would at least be covered - yet does anyone spend so much on this crucial part of development? As for a £100 million portfolio, how many companies pend £5 million a year on maintaining their development processes, let alone the central nervous system's budget of 20%?

On the other hand, the efficiencies that could be achieved by integrating the delivery process as a whole and making it a dynamic part of real delivery are vast. Some time ago Accenture published a paper showing that training had an ROI of more than 350%, and I suspect that the same would be true of improving most companies’ development lifecycles.

Here is a quick questionnaire of the most important issues, based on about 20 years of looking at (and occasionally helping to six) the problem. Note that it does not start with the details of the lifecycle documents and products – that is the least important fact! On the other hand, in more mature organisations the issue is often no more than obsolescence and missing items follow from the lack of sustained management focus, but in all too many cases there are major gaps.

1. Is there a global management approach to the delivery process itself?
• Clear unitary and controlled ownership, management & rules of delegation of the end-to-end development process.
• Is there a development strategy?
• Is there real expertise in methodology development? Just asking PMs what they think is like asking drivers how to design a far – you’ll get some of the user requirements but nothing useful about the design.
• Is there a coherent or proportionate rollout/update process?
2. Is there a simple, intelligible presentation & access?
• A single, integrated model of delivery as a whole, including:
• Governance, management technical tasks and support functions?
• All stages, including both work selection and initiation and solution deployment/transition and work closure?
• A single, user-friendly site for accessing the delivery process as a whole?
• Effective control over authoritative versions (and withdrawal of obsolete materials)?
3. Does it cover all of your most important delivery strategies?
• Outsourcing?
• Offshoring?
• Package procurement & implementation?
• SAAS (Software As A Service – thinks like Salesforce.com)?
• Does it have enough (or anything) to say about non-development activities?
• Procurement?
• Support and maintenance?
• Technology upgrades (Oracle, SWIFT, etc.)?
4. How mature is the lifecycle itself?
• Are there proper delivery and management processes - i.e., some components exist, but they do not operate as a complete, end-to-end, Prince2-like process?
• Is there a true programme management lifecycle (most organisations are dominated by programmes now)?
• Does it include a convincing model of change management as a whole, notably:
• Business design, development, readiness & transition?
• Operational design, development, readiness & transition?
• Does it handle very small projects (which can often be managed through a single artefact)?
5. How well does the lifecycle define basic management elements?
• Roles + responsibilities – are they current, consistent and complete?
• Are there explicit criteria, rules and authorities for adaptation, scaling & exemption?
• Does it include (or at least point to) integrated stage and task-level processes & tools?
• If it is a waterfall lifecycle, does it include a risk-driven iteration model for managing individual tasks and stages?
• Are there detailed procedures for basic management tasks (risks, issues, assumptions, dependencies, change/configuration control, product/document management, impact analysis, estimating, planning, resourcing…)?
• Have standard stage/task/product level risks, assumptions, dependencies, etc. been identified and articulated?
• Does it set credible gateways, including include stage-end consolidation & validation, evaluate full project content, review performance to date or readiness for next stage, etc.?
6. Are all individual products actually inadequate?
• Are products defined by independent product descriptions?
• Are there stage, product- and task-level procedures, advice & information?
• Are there samples of good practice, including instances for each major area of usage?
• Does each item have a supporting quality checklist?
7. Is alignment with other functions well defined?
• Clear & efficient access to supporting management functions & data (resourcing, MI, finance, architecture, etc.)
• Explicit alignment with and access to related standards + policies?
8. Is the lifecycle actively supported?
• Are there discipline or process owners, with clear roles & responsibilities, a proper management cycle and allocated time to do the job?
• Are there SMEs, with clear requirements and channels for feeding their experience into the organisation (e.g., a central lessons learned system or training/briefing programme)?
• Is there an R&D process (minimally to drive innovation, capture and socialise training and disseminate new joiners’ knowledge & experience)?
9. Is there a training programme covering all processes, roles, tools & techniques?
• Is there a specialised SDLC training function & system.
• Is there a training programme for staff, consultants, outsourcers, offshore & contractors?
• Are there self-training packages for key activities – individual products, reviews, testing, requirements management, etc. – so users can refresh their knowledge independently and as and when needed?

Without a Yes to at least most of these questions, you have the methodological equivalent of a crocodile's brain, and it will be all but impossible to make substantial and sustainable progress to real intelligence.

When are your requirements ready?

A very common failing in all sorts of projects is being stuck with what are in fact quite inadequate requirements.

The fact is, most organisations are pretty bad at explaining exactly what it is they want a project to accomplish. There are lots of good reasons for this - the situation at the start of a project is often fluid or unclear, there are too many options to be precise, it's hard to define a truly innovative idea in detail until you have tried it out, conditions and priorities change as the project proceeds, better ideas surface, and so on.

But that's not the same as simply being bad at requirements. The above issues relate mainly to the content of requirements, which is very hard to nail down definitively; what I am talking about here is their quality - a different issue. Badly defined requirements area major cause of problems for projects and businesses alike, causing the routine delivery of the wrong thing and lots of unhappy stakeholders. Fortunately there are ways of ensuring that the requirements - such as they are - are at least defined well enough for the project to proceed reasonably comfortably.

The issue I have in mind is the requirements review process - how requirements get signed off. There are lots of things you can do to make this fairly robust, but one technique I have seldom seem defined clearly enough is that of asking the requirements' principle users to confirm that they are fit for purpose.

There are three key groups of people who have an interest in how well requirements are stated:
- the analysts who will have to translate the requirements into a functional solution.
- the (user and operational) acceptance testers who will have to check that the requirements have been met.
- the operations personnel who will have to convert the requirements into SLAs and OLAs.

These people are not generally very interested in the requirements contents. But give them poorly quality requirements - too vague, imprecise, unanalysed, inconsistent, with key areas missing, and so on - and they simply won't be able to do their job. Which means that the solution the project delivers is all but bound to leave everyone with a nasty taste in their mouths.

All I am advocating here is that requirements be signed off by these groups. As far as I am aware, although many organisations ask their analysts to approve requirements, most don't ask testers or operations staff, and this may be a major cause for project failure. It's not hard to arrange, and it should certainly be welcomed by the groups in question. If, on the other hand, you cannot get their approval, maybe you should be looking to the users to define what they want a little more clearly - and so avoid storing up trouble for the future.

Of course, there's also a case for weak requirements. Well, not exactly weak, but at least requirements that recognise that they may not be the last word. Organisations and businesses change. So do markets and so does good practise and the opportunity the requirement was originally designed to address. So if you're embarking on an 18-month project to deliver x, it's probably not too smart to try to nail down your definition of x too soon. But that is not to say that you should not try to meet the above test from the start - it's just that your project should also make a strong allowance for change. That could mean a large tolerance, but it should also mean setting the right expectations from the start. For example, the business and users should expect to have to re-validate their initial requirements at regular intervals, you might want to prioritise items that are pretty safe from future fluctuations (e.g., stable regulatory requirements or generic interface components), design for flexibility (loose coupling, modularity, etc.) and so on.

This is the case for agile, of course - but of that more than enough has already been said by anyone and everyone, including me!

Defining RAG statuses

In my work I come across a lot of attempts to explain in simple, one-word ways what the status of a piece of work – usually a project – really is. Much the most popular is the ‘traffic light’ system, or ‘RAG report’ as it’s known around across the IT industry. It’s a great idea: simple, clear and apparently quiet unequivocal.

The only problem seems to that in many organisatons the very definition of Red, Amber and Green is usually, frankly, quite irrational.

A fairly representative set of answers to the question ‘What do RAG statuses actually mean?’ can be found here: http://www.linkedin.com/answers/management/planning/MGM_PLN/186467-1517184. However, there is surprisingly little on the web about this topic, so here are my current thoughts on defining red, amber and green. Nothing radical, but a little more consistent and logical than some of the ideas I have seen floated, especially in the companies I have worked in.

The first question that needs to be answered is what exactly ARG reports are for. If you look at most organisations’ RAG criteria, they are generally defined in terms of percentages or absolute numbers. For example, if a project budget looks like going over by 20%, it’s a Red. I don’t understand this approach, especially in a project-based organisation. One of the basic features of any intelligent project governance approach is to define project-specific tolerances t reflect the project-specific circumstances, known risks, and so on – not to treat them all as though they were peas in a pod.

So when project A is 20% over budget, that may indeed be disastrous, because its agreed budget tolerance is 10%. But project B, which has always been expected to need more re-financing at some point, has a tolerance of 30% (yes, such projects do exist), so overspending by 20% is not, by itself, cause for concern, and certainly not cause for trumpeting a disaster from the rooftops.

So what – or, more precisely, who - are RAG reports for? First and foremost, they are not for everyone. By and large, they are for people who a) know the basic parameters of the project, including its tolerances for budget, delivery, and so on; and b) are in some sense accountable for the project’s success, or at least need to understand its prospects for success. In other words, RAG reports are aimed at people like the project board, quality managers, your PMO and so on.

So what do they need to know about a project that can be usefully and meaningfully communicated in something as simple as a single colour? Really it’s very simple: Do you (the report’s audience) need to do about this work?

So the message the RAG status needs to convey to the reader:

• Green: Everything’s fine, you have more pressing things to worry about, go away.
• Amber: I have problems, but I’m pretty sure I can fix them with what I have available. So nothing to actually worry about yet, but you probably need to keep an eye on what happens next.
• Red: I have real problems and I can’t solve them with what I have available. YOU NEED TO DO SOMETHING.

Hence the RAG definitions I would recommend:

• Green: All aspects of the project are fully under the PM's control using only the project's authorised plan & arrangements (e.g., budget, dependencies, resources, etc.).
• Amber: Additional actions are required, but can be successfully managed within the project's authorised capabilities & tolerances.
• Red: Cannot be resolved within the project's authorised capabilities & tolerances. Requires escalation.

These are very general descriptions, of course, though none the worse for that. By being so general, they make it easier to achieve the consistency that is the basis of good quality management. But at the same time they are probably too abstract (I would not say vague) to be easily sued by busy project managers who aren’t much inclined to debate the finer subtleties of their project’s status.
So in addition to these high-level definitions, here are a few definitions of more detailed RAG statuses, as they relate to particular areas of project management. They are pretty useful tests of the overall status, but always bear in mind that the ultimate test is the above core definitions.
Finance
Green:

• The project's current budget is sufficient for the project, and is expected to remain so.

Amber:

• There are outstanding changes that have yet to be budgeted for.
• The PM does not maintain a record of expenditures.
• Actual and forecast expenditure have not been reviewed/reconciled since the last report.
• The authorised budget is currently being challenged.
• The project is forecast to overspend (including tolerance) but there is a credible path to recovery.

Red:

• The project is forecast to overspend (including tolerance) and there is no credible path to recovery.
• Finances were Amber in the last report and no recovery plan has yet been agreed to return those particular problems to Green.
• The project (or stage) has mobilised without budget authorisation.
• The project is overspent (including tolerance).

Scope & governance
Green:

• The authorised scope is correct, is authorised, meets stakeholder expectations, and is expected to remain so.
• The current governance meets the project's needs, is within our governance framework, and is expected to remain adequate.

Amber:

• The project is not explicitly aligned with an authorised business goal and/or has moved from baseline scope.
• There is at least on open & unauthorised change request (CR).
• Cumulative impact of CRs exceeds original tolerance.
• The authorised scope is forecast to become invalid (e.g., known change in business strategy) but there is a credible path to recovery.

Red:

• The authorised scope is forecast to become invalid and there is no credible path to recovery.
• Scope & Governance was Amber in the last report and no recovery plan has yet been agreed to return those particular problems to Green.
• The project has started without an authorised scope.
• There is no Project Sponsor/Senior supplier/Senior user on your project board.
• The authorised scope is no longer valid.

Schedule
Green:

• The currently authorised plan and arrangements are sufficient to assure the successful delivery of the project as a whole.

Amber:

• Plan updates are needed to reflect expected changes in activity, scope, CRs etc.
• The project plan has not been revised since the last report.
• A critical path product/milestone has slipped/is forecast to slip, but there is a credible path to recovery.

Red:

• A critical path product/milestone has slipped/is forecast to slip, without a credible path to recovery.
• Schedule was Amber in the last report and no recovery plan has yet been agreed to return those particular problems to Green.
• The project (or stage) has started without an approved plan.
• Unfinished work that should already have been complete has yet to be rescheduled.
• Work is underway that is not on the authorised plan.

Resources
Green:

• The current stage has named, agreed resources and the resource requirements for the project as a whole are agreed.

Amber:

• The plan includes over-committed resources.
• The project lacks (or is forecast to lack) resources needed for successful delivery, but there is a credible path to recovery.

Red:

• The project lacks (or is forecast to lack) resources needed for successful delivery, and there is no credible path to recovery.
• Resources were Amber in the last report and no recovery plan has yet been agreed to return those particular problems to Green.
• Plan contains tasks without predecessors or successors.
• Plan contains tasks in the current stage without assigned resources.
• The plan for the current stage is not fully resourced.
• The plan for the project as a whole does not identify at least resource types.

Risks & issues
Green:

• All known risks and issues can be managed within the current project arrangements & capabilities.

Amber:

• At least one severe risk/issue is unlikely to be resolved as planned.
• The project has escalated at least one risk.

Red:

• The project has no effective risk/issue log.
• Risks & Issues were Amber in the last report and no recovery plan has yet been agreed to return those particular problems to Green.
• The risk/issue log has not been reviewed since the last report.
• At least one severe risk/issue is unlikely to be resolved as planned.

Dependencies
Green:

• All dependencies for the project as a whole have been formally defined and agreed.

Amber:

• Not all dependencies for the project as a whole have been formally defined.
• Not all dependencies for the project as a whole have been formally agreed on both sides.
• An external dependency on the critical path has slipped (or is forecast to slip), but there is a credible path to recovery.

Red:

• Dependencies were Amber in the last report and no recovery plan has yet been agreed to return those particular problems to Green.
• An external dependency on the critical path has slipped (or is forecast to slip), and there is no credible path to recovery.
• Not all dependencies for the current stage have been formally identified and agreed with the responsible managers.
• The project plan does not identify all external dependencies and deliveries.

This multiplicity of criteria raises a key point: how many RAG statuses should your project have? Personally, I would strongly advocate using several RAG indicators at once. This not only gives your readers some idea of what questions they should be asking next, but they also allow organisations such as quality management or your PMO to compare reports from all their projects to identify hotspots and bottlenecks in the existing process that would perhaps benefit from a little company-wide improvement.

Exactly which areas you chose to RAG is up to you, of course. But what ever they are, they should be the areas you regard as the best indicators of project success and failure. That’s why I tend to start with the set above: in my experience, it is because dependencies and resourcing and all the rest tend to be the areas that drag a project under. You should chose your own, and test them every six months or so to see whether trends in individual RAG statuses did indeed predict success and failure. A few quick statistical tests using Excel is all you need (though what you use to replace unhelpful tests or unexplained failures is more speculative – a bit of an experiment).

Of course, this leaves a very important point unclear. If one particular facet of my work – the dependencies, for example, or the risks – is red but the rest are green, how do I calculate the overall status?

It is very tempting to fudge things here. If it’s mostly Green with just one Red, can’t we take a sort of average and call it Amber? No, we can’t – and the reason is simple. All the RAG criteria suggested above are individually capable of wrecking your project. Or if they aren’t they should not be on your list of questions. So if any of them is Red, the project as a whole is Red too.

One more detail. All the above RAGs are based on objective information (though no information in business is safe from manipulation). But there is one area of subjecting knowledge this leaves out: the manager’s own expectations of success. This is an important factor: a project manager faced with reds and ambers but who still expects to deliver as planned either has something interesting to tell you or needs to re-learn the basics of project management. Either way, I always include a ‘deliverability’ RAG – the PM’s assessment of how likely they are to succeed. The basic definitions of each colour are the same, and here are a few things PMs should ask themselves when setting their deliverability RAG:
Deliverability
Green:

• You are confident that the project will deliver as planned and authorised, without disproportionate risks.

Amber:

• You are not confident that the project will deliver as planned and authorised, but there are viable methods for recovering from this.

Red:

• You are not confident that the project will deliver as planned and authorised, and there are no viable methods for recovering from this.
• Deliverability was Amber in the last report and no recovery plan has yet been agreed to return those particular problems to Green.

I have found this a very useful and workable system, even down to the fact that it makes the supporting tools really easy to build. It takes practically no knowledge of spreadsheets to calculate the overall RAG status of a piece of work based on this system. No complex look-ups of percentages of values or conditions: if it’s red down below, it’s Red on top. Simple, effective, and above all else it tells its audience exactly what they want to know – What do I need to do?

Level 0 context diagrams

I have long been interested in context diagrams - which is to say, single diagrams of the Big Picture within which a methodology (SDLC, etc.) operates. As a result of a succession of methodology-related engagements, here is my current top-level version:

Click on the picture to expand it.

The idea is to identify all that factors that explain what, ultimately, the methodology is trying to accomplish, how it is governed, how project and programme goals, objectives and targets are set, what support is available, who controls the overall approach (e.g., the core methodologies), and so on. In my experience, most organisations address this issue in a very piecemeal manner, with occasional and very ad hoc references to the details scattered all across the methodology and in surrounding structures (e.g., PMO rules, local standards, and so on).

This is unfortunate, as it invites conflict, makes its hard to understand the whole, makes compliance with the methodology much harder to justify, all but ensures that major errors and omissions will exist, and so on. It also makes it hard to identify who to go to when the methodology does not actually answer a question. Of course, defining all this will demand a vast amount of information that is typically either widely scattered, hard to find or simply missing. But at the very least, for each box you will need to know:

• Overview of purpose in the organisation as a whole
• Role in delivery (eg, direction, prioritisation, project governance, and so on)
• Specific dependencies
• Process/standards
• Contacts
• Organisation
• Ownership
• Management cycles

You can find an editable PowerPoint version here. I'd be interested in comments, and eventually plan to create a fully-fledged presentation explaining each item in the model in detail.

Free stuff - no, really

After a long absence, here we are again. Part of the time away has been spent creating a new website, which will be of little interest to most people except for the Downloads section.

Unlike most such pages, this one really is designed to give you free stuff, not just adverts for myself. Knowing full well that this is good stuff (well, good enough for other people to pay me for it) and not wanting to let it fall into oblivion, I thought I’d just give it away. Really.

Right now it has tools and training materials for lessons learnt systems, stakeholder management, various aspects of methodology, and so on.

I plan to add to it occasionally. The main areas will be methodology, quality, and governance, but I have a good deal else. And if you have any requests, I may have something I could post just for you.

Agile: Guidelines or methodology?

Another discussion on LinkedIn, this time about whether Agile is a methodology or just guidelines. The consensus seems to be guidelines, which seems to reflect the spirit of Agile better.

But at the same time, this view seems only to address Agile in the abstract, not Agile (or any other delivery model) as implemented in any real organisation. Which is a pity, because it is at that point that the strains will start to be felt if Agile remains no more than guidelines. On the other hand, to convert it to a formal corporate methodology would not only defeat much of its underlying philosophy and approach but also lead to Agile being ossified in the same way that waterfall – which was never inherently rigid or bureaucratic – was ossified by immature implementations and too much top-down corporate management freakery.

And of course, there always been legitimate management reasons why Agile cannot be left to go its own way, any more than any other management tool. There will be inescapable (and entirely reasonable) reporting requirements, stakeholders will often want to know what progress is being made in non-Agile terms, and so on.

So, for any organisation that does not want to see something as promising as Agile degenerate into either paperwork or making it up as you go along, it is necessary to be rather more specific about how either a Agile-as-methodology or Agile-as-guidelines is implemented.

So even if Agile is to be forced into the mould of a corporate methodology, any self-respecting implementation can features that prevent it from becoming a rigid, inappropriate and self-defeating. For example:

• It will be specifically tailored to the type of work that is really being done.
• It will be abstract enough to permit considerable leeway for professional judgement.
• It will be fully scalable (no, not just big, medium and small).
• It will include user-friendly mechanisms for granting exceptions and waivers.
• It will include wide-ranging but rigorous (the very opposite of rigid) ranges of meaningful options.
• It will be implemented through training and tools, techniques and templates that make explicit the team’s authority to vary, depart from or just plain ignore the ‘rules.

And so on.

I don’t see that any of this gets in the way of Agile, or how any complex organisation could safely or profitably implement it without at least a few of these quite standard methodology components.

How good is DSDM?

I recently initiated a discussion on LinkedIn entitled How good is DSDM? Although there was (unsurprisingly) a consensus that DSDM was A Good Thing, we were collectively unable to come up with much hard data until Jennifer Stapleton – a past Technical Director of the DSDM Consortium – kindly offered me some data from Xansa (bought by Steria in 2007) and British Airways. Xansa’s data is especially interesting, as it covers a number of clients (including BT) and Xansa was, at that time, the world’s largest DSDM practice.

The gist of the data is that DSDM offers huge improvements in productivity, team size, delivery time and project quality. Here are the basic graphs (not as contemporary as I'd like, but still solid data):

What, ultimately, is Agile about?

There is an interesting discussion of Agile going on at LinkedIn at the moment. The topic under review is 'Transitioning from command and control to a servant based style of leadership'. personally I think the idea of 'servant leadership' is both misconceived and redundant, as the answer (so far as I understand the issue) was provided by the German sociologist Max Weber about a century ago.

In brief, at least as far as successful Agile projects are concerned, I suspect that this change in the way organisations work under Agile is closely connected to the distinction between being a professional and an employee.

• An employee is someone you pay to be able to tell them what to do, and is best suited to command and control.
• A professional is someone you pay so that they will tell you what to do, and so works better in a collaborative environment - which Agile is designed to create.

Conversely, I suspect that whether an Agile initiative is successful depends heavily on the extent to which truly professional capabilities and a professional culture exist. In that respect it would be very interesting to hear from people for whom Agile had not worked as to why it had failed.

Note the causal direction. If companies insist on command and control, they get employees – i.e., people who need to be told what to do. If you give people opportunity and responsibility (and a non-trivial amount of skill), you will get professionals.

On the other hand, the training and coaching needed to get people who were previously treated as employees to operate as professionals (and therefore suite to Agile) can be very great. The transition is not easy or straightforward, not least because the skills required are by no means solely technical. There are personal and social capabilities that are also required to succeed at Agile. But they are encompassed by the concept of professionalism.

This can be exemplified by a major cultural problem Agile implementaitons often seem to face, namely empowering staff saying no to their boss (eg, the Agile PM). Managers need technical training (i.e., how to do Agile) but other team members need the social and personal ability to insist on their own professional perspective. Few organisations cultivate this attitude (though I have known a few), but I would say that it is crucial to making a success of Agile.

The same point applies at the other end – to business stakeholders. They also frequently need a change of culture – to become involved, to own the project, to participate effectively, to accept an incremental approach and to be able to change their minds without embarrassment or political penalty.

Why prefer DSDM to Scrum?

Ultimately practitioners know that there is no need to choose between Scrum and DSDM, of course: they can be integrated into a hybrid that suits your specific requirements. However, it is helpful to have a clear idea of what the different flavours are and what they are capable of, because it is not only purists who want to know exactly what you are doing: so will the people who are shelling out the cash to pay for the change. Being able to give them a clear choice between clerly defined options, plus clear criteria for choosing, is essential to selling any form of agile.

So, here are some reasons for preferring DSDM to Scrum, in direct proportion to the size, complexity and innovativeness of the project and the difficulty of the technical and compliance requirements.

1. The Foundation stage makes sure that everyone ‘gets’ what you are up to. Without this, something very bad is likely to happen to your project. Adding a 'Sprint 0' isn’t the answer unless it looks very like DSDM’s Foundations anyway.


2. DSDM is a true end-to-end process, not just the middle, development snippet. Since most of the mistakes in IT are made before the ink is dry on the contract (internal or commercial), this is a crucial consideration.


3. Defining the roles as completely as DSDM does can be extremely helpful in any but the most trivial of projects. After decades of IT neglecting this issue, it is refreshing that DSDM includes it. Very hard to do well (and the sell to the business and operations alike), but absolutely essential.


4. Defining a few basic products – BAD, SAD etc. - is also very helpful. Given the complexity of most real projects, it can be impractical to rely too heavily on prototypes and the authority of empowered teams, powerful though they are.

As this list suggests, DSDM is much more suited to a corporate environment, where the playfulness, autonomy and spontaneity of Scrum clashes rather sharply with the demand for formal definition, accountability and predictability. It's hard to get the full benefits of Scrum while adhering to corporate formalities dictates concerning governance and accountability, investment prioritisation, intelligibility and visibility to a wider group of stakeholders, quality management, reporting and architecture.

Anders Larson has kindly pointed me to this good comparison of DSDM and Scrum from the DSDM Consortium site. Its author, Andrew Craddock, summarises the position very well: after listing the principles of the Manifesto for Agile Software Development, to which both Scrum and DSDM practitioners contributed and subscribe -

• People and Interactions over Processes and Tools
• Working Software over Comprehensive Documentation
• Customer Collaboration over Contract Negotiation
• Responding to Change over Following a Plan

- he notes that 'DSDM recognizes value in the items on the right of the Manifesto statements more than Scrum does, whilst still putting the highest value on the items on the left. This allows DSDM to fit more comfortably with the normality of larger organisations and gives rise to some of the differences between these two Agile approaches.'

The business case for DSDM (and Agile generally)

Currently I am participating in a very interesting discussion about the business case for DSDM with the LinkedIn site’s DSDM Group, particularly with David Winders.

There seems to be very little hard evidence that DSDM is significantly more efficient than a waterfall approach at doing the work, which would seem to kill the business case stone dead. After all, why incur the costs of migration if the grass really isn’t any greener on the other side?

But as the discussion has evolved, it has become clear that this is to misread what Agile methods like DSDM (or Scrum) are offering. (And perhaps this should have been obvious from the start – I only really noticed this point once the discussion was well under way).

To boil down what is (for the time being) the final argument, there is indeed a narrowly economic argument for DSDM. Unlike a waterfall, DSDM is never going to waste stakeholder time and money by delivering (say) 100 function points that were agreed a year ago, when they were almost certainly based on premature and immature judgements. We probably don’t want exactly those things any more: our thinking has evolved, our goals have evolved, and the business situation has evolved. Instead it (DSDM or any other agile methodology) will deliver 100 function points and for much the same price per FP, but they will all be FPs we know we want.

Add to this DSDM’s far closer control over the delivery process, which makes it far less likely that the whole project will go over budget, schedule but under scope, and the business case for DSDM is clear: even if DSDM projects don’t cost less per FP, they represent better value for money, because they are all FPs you want.

Another interesting point was made by John Isgrove of Collaborative Consulting -

What we found was that DSDM completed the same development in 66% less time. It did not complete in less effort however as with agile projects they tend to be shorter but fatter resource profiles i.e. work for less time but more people involved at the same time across the time period. Overall effort was about the same with with higher proportions than the waterfall project being spent by the analysts and testers.

Moving to Agile - Identifying the truly essential documentation

A crucial feature of migrating to Agile is identifying the irreducible documentation needed to support the process. For many Agile practitioners the very notion of a fixed documentation suite will raise their hackles, but there are many stakeholders, not all of whom are not directly involved in the delivery process, but all have interests that must be taken into account.

To name only the most obvious, there are three general classes of stakeholders, each with their own distinct information needs:

1. The delivery team itself
2. The business
3. Operations, compliance and support

These groups are already often poorly served by waterfall processes; the risk is that Agile’s understandable and largely justified desire to take an axe to the great mass of pointless documentation will result in them being served still worse. So…

The delivery team

Well, obviously. And generally speaking the Agile assumption that there is, by default at least, no need for internal project documentation, holds good. Why would I need to write down what I can tell everyone in a tenth of the time? Why write down anything at all if its lifetime is likely to average 15 days?

But less obvious, is it in fact the case that they need nothing? In very simple projects, yes, it usually is, but in a more complex programme – for example, a dozen Agile workstreams flowing into a common integration/release cycle – they may well need a good deal more. The PM is likely to be unavailable (doing programme-level duties at meetings and forums) and when they return to base, unable to convey everything by word of mouth.

Then the standard criterion applies – fitness for purpose. But in a project of any significance (size or importance), that rule is unlikely to result in their being no documentation at all. Add to that the other reason why documents exist – because the project is innovative, organisationally complex, heavy with technical or regulatory content, and ‘pure’ Agile starts to look a bit thin. Absolutely never create more documentation than are strictly needed, but don’t assume that the minimum set is zero. That is usually wishful thinking.

Business stakeholders

Yes, a properly empowered business rep should allow you to dispense with almost all external review and approval, but even in the most benign environment, there are likely to remain yet higher level reports. Much as most of us would like to dispense with this too (why don’t they just have faith in us?) but ultimately projects exist solely because they serve the interests of the organisation as a whole, and someone up there really does need to know what you are up to. So there is all but bound to be some form of reporting.

However, taking a constructive approach to this can also encourage those to whom reports are due – PMOs, line managers, HR, finance, direct reports to senior stakeholders, etc. – that they don’t really need the detail they are probably accustomed to. Reporting strictly by exception is the starting point, as it is that, rather than no reporting at all, which is the true corollary of empowerment. Using the move to Agile to rationalise reporting isn’t a bad idea either.

Operations, compliance and support

Now we come to the groups who are most in need to solid documentation and most likely to be neglected. In many organisations the needs of operations, service transition and support teams are already poorly met, and Agile is unlikely to do them any favours. Nevertheless, it is crucial that their needs are met, and these go far beyond touching base with them or even having them represented on the project. They will spend far more on the delivered system than the developers, and the better equipped they are to manage the delivered solution, the better for everyone.

Operations, compliance and support will make up the great majority of all but the most superficial IT projects (e.g., throwaway web pages or transient rate changes), and their needs are not only substantial but also penetrate deeply into the heart of what is being developed. They all need to be consulted about the original requirements (e.g., to define SLAs) and strategy (e.g., to ensure sustainability), they all need early warning of what, functionally and technically, is coming down the track, they all need to know what exceptions have arisen as the original requirements/backlog is progressively shaped and re-shaped, and they all need to be aware of the (now multiple) schedules for release.

Conclusion

So what does this all add up to?

1. There is a very great deal of documentation that genuinely can be thrown away – and good riddance. It adds nothing but dead weight to the project and should be discarded whenever possible. Which is remarkably often.
2. Quite a lot of other stuff needs to be transformed. It is very likely that groups such as administrative functions (PMOs, HR, finance) will have to change the way they think about projects and how they are reported on, but it is unlikely that they will eschew reporting altogether. Nor should they. Likewise for business stakeholders – they need to assign real power to their business representatives within the project, but they are unlikely abandon all visibility. And given their responsibilities, how wise would it be?
3. Finally, there are those who will need what they have always needed. But if you don’t give operations and support what they need, they probably won’t notice much: by and large they aren’t getting it now either.

Moving to Agile - Critical business factors

Some years ago I implemented DSDM (then the preferred flavour of Agile) at Churchill Insurance. As far as the core processes were concerned, there was no great difficulty (admittedly we threw everything at it), but the business was a problem for two key reasons: empowerment and availability.

The nub of the empowerment issue was asking the business to allow their own representatives on the project enough authority to approve of what was going on on the spot (e.g., prototypes, changes, reprioritisations, etc.), without constantly referring to senor management. In short, they could not let go of the strings. As anyone who has worked in IT projects for any length of time will know, there is a ludicrous irony in this, because quite a lot of the problem with delivering successful IT projects of any kind is the business’s ambivalent attitude the ownership. In all too many companies, the business want to be in control of the project (i.e., able to make make-or-break calls about it) without taking responsibility for its delivery. The implicit question is always, do you prefer delivery to control?, and all too often the effective answer is ‘control’. This is as much a problem for Agile as it ever was for waterfall projects.

The second problem was getting the required effort from the business representatives to the project. These individuals were necessarily very experienced and valuable people (who else would you empower?) but that was precisely what made them hard to replace in their day jobs. So we needed them to spend about three days a week on the project, but they still needed five (usually very long) days for their BAU work. No surprise, then, that project work soon started the back seat to day job, the quality and speed of decision-making plummeted and the project started to look pretty wonky.

So although the business was keen on an Agile approach, they could not participate effectively. It was a long struggle – only partially successful – to deal with this problem.

Moving to Agile - Creating an Agile environment

In my experience a crucial precondition for achieving agility is a certain level of maturity. Although the core processes, roles and tools are critical, so is the presence of quite a wide range of highly standardised core mechanisms of which the agile operations can take advantage.

The classic example, familiar from IT development (whence agility sprang, of course) is test automation. Without this the development team is unlikely to be able check its day’s labours swiftly enough to move on confidently the next day. But test automation itself can only be adopted by an organisation that already has standardised test classes, a well established test process, a clear understanding of the basic mechanisms of test scripting, and so on. Without all of these (and much more), test automation will fall flat on its face – becoming either ineffectual or rigid – and quickly start to turn agility into paralysis.

Of course, in a vey small, simple project or activity, the preconditions for agility are very limited. But in a more complex situation, such as BAU operations or a large-scale programme, agility can be achieved, but only ensuring that a full ‘agile environment’ is also in place. The elements of this environment can themselves be agile, but they certainly must be present and specifically geared to allowing other areas to take them completely for granted – it is, after all, the most basic basis of agility that the would-be agile activity can either omit or take for granted that everything in their environment.

Hence one of the key task – perhaps the single most important task – when implementing agile methods is to investigate what the organisation’s ways of working can offer to the agile area – and what they demand from it too.

A few (there are many more - this is just a flavour) of the areas you need to get right include:

1. A governance system that allows for rapid validation and approvals of many incremental releases.
2. Business and operational organisations and processes that are capable of assimilating frequent change.
3. Office arrangements that support closely collocated teams.
4. Extremely slick mechanisms for remote groups – vendors, outsourcers, other offices, and so on.
5. System architectures that support rapid change.

And so on - an enormous range of factors, many of which are typically intensively embedded in the wider organisation. it isn’t easy getting this sort of thing right.

The value of validation

I just attended a workshop on testing procedures. It was a bit worrying – not that anything was wrong with the procedures themselves (in fact they were well thought through) , but it was a bit shocking that testers still need telling.

One topic the workshop didn’t address was the distinction between verification and validation. These terms are used in different ways in different environments, so I should start by saying what I mean by each. Verification is making sure that a production meets its spec. Validation is checking that, even if it does, it will also fulfil the original requirement it is intended, to meet. Not at all the same thing, not only in the obvious sense but also in the sense that step-by-step verification from the requirements to (say) the code you are reviewing would not be equivalent to validating the code against the original requirement. There is just no substitute for asking which requirement a piece of code contributes to – and how each requirement is realised in the code.

There are lots of reasons for this, of which the fallibility of previous checking it one. But there is (a mathematician friend tells me) a much more compelling one that should convince even the most hardened reviewer and tester. This is that it is (apparently) possible to prove mathematically that no two languages can be translated into one another such that the semantics is exactly correct.

This may seem an abstruse point, so here is a practical example I have used in training courses. There is a well known saying in English that, if you translate it correctly into Russian (again, I am told) and then re-translate it back into a possible but correct English phrase. One of the possible outcomes is the following:

The vodka is acceptable but the meat is off.

So what was the original English phrase? Give yourself a few seconds before you look at the answer, which is at the foot of this post.

Now look. Not quite the same, is it? Now it is essential to recognise that both the translations, from English to Russian and from Russian to English, were both 100% correct. Just like a model may correctly represent a requirement, a design a model and code a design. And yet it is clear that if you came up with the second English phrase (the code, as it were) rather than the first (the requirement), it would leave something to be desired. The problem is, requirements, models, design and code are all in different languages (in every sense) and not two languages (let alone four) are exactly equivalent.

Hence the critical value of validation as well verification. Your just have to do it, not because your verification (reviews, testing, static analysis, etc.) isn’t good enough but because it does a different job.

Not that you should validate everything. It’s expensive, and like verification itself, not always the most productive thing you could be doing with your resources. Like everything else in good management, what you look at should be determined by the risk it represents. So only validate the items that represent a real threat if they are wrong. By and large, focus on the critical, the complex (at any level), the novel (to you). After that, either an error won’t matter much or you should be able to fix it relatively easily.

Answer: The spirit is willing but the flesh is weak. Or, more completely, ‘Watch and pray, that ye enter not into temptation: the spirit indeed is willing, but the flesh is weak’ (St Matthew 26:41).

Implementing a methodology - do's and don'ts

An old friend sends me the outline of an IT methodology he is being expected to implement by a big client. It’s the usual bureaucratic behemoth and he asks me what can be done to make it work decently. My reply:

"Thanks for this. I read as much as I could bear and skimmed the rest. I can see why they’d hate it!

Going by this description, I had a similar problem at a large client financial services where I spent 2½ years implementing a global consultancy’s not very lovely methodology. They hated that too, but we eventually got grudging support and even commitment.

I guess that you could summarise what I would do as follows:

1. Insist on taking training very seriously – train everyone from top to bottom of the company, tailor the training to their exact needs, and as far as the delivery teams are concerned, don’t let anyone tell you can do this in less than a day for the introduction and a day for each major development stage (or discipline, perhaps). I personally trained more than 800 people in methodology at one client and they grudgingly agreed that it was money well spent. I think this which as because the training included lots of ‘whys’ as well as ‘hows’. That way people were constantly given the message that there really is a compelling reason to do this stuff, that this really is for your own good – as engineers, as professionals, and as people who don’t want to waste their own time or other people’s. The training has to be full of useful nuggets (eg, 40% of all software engineering is waste and rework, primarily for lack of decent processes), war stories, realistic exercises (ideally one big case study) and methods for finding that magic 80/20 position.

2. Encourage PMs to create tailored versions of the methodology for themselves. This is easy and reasonable and builds ownership. Given that it means cutting out waste and rework and building in the uniqueness of local areas, your client should want it too. After all, if the generic methodology includes stuff (as it always does) that doesn’t make sense in a given project context, they shouldn’t have to do it. And make sure that you build the process for tailoring the process into the main process (e.g., as a project initiation activity), make it a high priority item in training (for senior management too), and reward managers for being insightful and innovative.

3. Scale the methodology thoroughly – with a two-page checklist for truly tiny pieces of work, and a serious approach to low-risk work that really does require only a light touch. But never say that the process is optional. It is never optional. It is simply adaptable. Build in get-out clauses for compliance, take the maintenance people’s problems with project-size methodologies seriously, create massively simplified tools appropriate to very low risk situations. Make sure everything is driven by a clear sense that real risks are being managed rather than a formal procedure being complied with. But never let them do nothing – that’s just the thin end of the wedge.

4. Make the waiver/exception process as simple as possible. Quick, clear lines of authority (ideally as local as possible), 5 questions maximum, rapid response guaranteed. I would suggest simple answers to questions like:

• What do you want an exception/waiver from?
• Why do you want it?
• What will you do instead?
• Why is that better than the standard process?
• What residual risks does that create?

5. Ensure that the methodology is properly owned, so that there is someone to go to for a decision on what is really meant or needed by a specific item, or to approve an exception. This is crucial if the system is to continuously improve itself – someone has to have it as a high-priority responsibility to actually improve it.

6. Support users constantly by training a methodology expert or two (one of my clients had 6-7 for 600 engineers) providing training, internal consultancy, project management consultancy, explanations and ideas for quick wins. They also provide a powerful conduit for communicating new ideas between groups.

7. Build an decent wiki (not a fixed website) that provides high-level process flow models to remind people what to do, and which can be drilled down for more details, online forms, etc. This paper you sent me is a prime example of a format people just won’t read – even I felt ill just looking at the endless levels of heading number. On the other hand, it’s not a bad script for a training course, so it’s not wasted. Even something as simple as online PowerPoint presentations are very effective, although they tend to offend web purists! I have a couple I built for Citibank, Churchill Insurance, Amex, Accenture, etc., if you’d like the see them.

8. Build an effective lessons learnt system that completes the loop from project experience to the methodology and back (through rollouts and training) back to projects.

9. Finally, pure stick: Tie their bonuses to compliance with the methodology, as confirmed by an independent assessor. Brutal, unpopular, initially counter-cultural in many companies but amazingly effective. It provides a somewhat perverse way of dealing with the inevitable feeling on the software engineer’s part that they have little interest in complying other than simple obedience to a very dubious corporate rule – if all else fails, fine them for non-compliance. Having dealt with thousands of IT people, I can only say that it has its place in the methodology implementation toolkit.

I dare say that some or all of this could be sold to anyone who a) hated methodology enough; b) had no choice about complying; and c) could afford the likes of me to make it work!"

The objectives of maturity management

The purpose of maturity management achieve the following major objectives:

• To define a strategy for creating revolutionary change by means of evolutionary steps.
• To free leaders from the limitations of corporate management systems by creating management systems that enable leadership rather than constraining it.
• To define a truly manageable management system capable of supporting fundamental, strategic change.

Surprisingly, these are not stated objectives of other management models such as the Software Engineering Institute’s well known Capability Maturity Model or the Project Management Institute’s standards. Nor are they made any easier to achieve by the approach those standards adopt, which is basically pragmatic, eclectic and bound by convention.

These objectives are described in more detail below.

Objective 1: Revolution by evolution

The primary objective of maturity management is to deliver radical, even revolutionary change. That means not merely re-invigorating moribund management systems and staunching the haemorrhages caused by poor management practice, but creating genuinely world-class organisations.

But how is that objective to be achieved? Most approaches to organisational change share at least one assumption: that radical results could be delivered in a single heroic step. Maturity management is based on a quite different assumption: that realistically, radical change can only take place in well-defined, incremental steps, quite probably extending over many years and certainly requiring many discrete developmental steps.

Hence its first objective: to define a sequence of discrete, manageable stages through which radical change can be brought about. Revolution by evolution, in fact.

Objective 2: Freeing leadership from management

One way of conceptualising how maturity management works is in terms of the distinction many authors have drawn between management and leadership. To quote Stephen Covey's Seven Habits of Highly Effective People:

Management is efficiency in climbing the ladder of success; leadership determines whether the ladder is leaning against the right wall.

Other commentators have expressed similar sentiments in different ways, but it is striking that they all insist on this difference and on the importance of leading organisations rather than merely managing them. Leaders bring vision, inspiration and direction, and without it an organisation loses its impetus, its cultural integrity and its ability to take decisive action.

Yet many organisations seem determined to encumber their leaders with unnecessary or subordinate management tasks, even actively disabling them by failing to provide the basic information and decisions real leadership demands.

Of course, no organisation could succeed by completely replacing management by leadership. Conversely, where leadership is not supported by robust management, the ‘leadership’ and ‘empowerment’ routinely degenerates into senior management abdicating responsibility for the actions, accomplishments and performance of their subordinates, backed up by the usual blame and recrimination when things go wrong.

So a balance must be struck – but only the right balance:

• The ability to manage is quite commonplace, whereas leadership is notoriously rare.
• The ability of leaders to delivery results depends on the presence of management systems (including competent and empowered managers) capable of implementing their vision.
• Unbridled, universal ‘leadership’, if not backed up with clear control of the whole, will soon degenerate into chaos, and the whole becomes a great deal less than the sum of its parts.
• Once they have been applied to a range of assignments, many leadership skills can be translated into reliable methods, tools and techniques that can be taught to less inspired individuals.

Hence another aspect of maturity management: by continually upgrading management systems, activities that previously required that rare combination of inspiration and perspiration that defines genius can be done almost as effectively by any modestly capable individual who has been trained to use the appropriate methods, tools and techniques and is supported by the necessary flow of information and decisions. Indeed, the whole history of management consists very largely of the creation of management systems to do things that were previously done only by great leaders. That is one of the main reasons why great organisations – nations, teams, businesses and so on – can exist at all.

On the other hand, where will future leaders acquire the vision on which leadership so crucially depends? Where will they get that spark of insight leavened by sound practical experience? Surely the answer is, yet again, from the management systems in and through which they work. If these systems are bad, then any manager’s experience will be less than illuminating. If, on the other hand, the management systems they use are well designed, effective and properly directed and maintained, their experience of their work, the organisation and its goals will be clear, well-structured and informative. Its purposes, methods and underlying philosophy will be clear and reinforced throughout. Conversely, the better structured the system, the easier it will be to spot any residual problems. But most importantly of all from the point of view of inculcating leadership, the values, purpose and opportunities it faces will be clear.

Hence the maturity management approach: wherever possible it replaces leadership by management. This is not because we should prefer management to leadership after all, but because we should reserve the special talents involved in leadership for tasks where they are really needed. If some leadership skills can be made so straightforward that they happen as a matter of course and the same results can be reliably achieved by the routine use of a management system, this can only strengthen an organisation, and release its true leaders to focus on areas that demand real leadership.

To summarise the whole above argument in terms of a contemporary management buzz phrase, the trick is not to rely on those who can ‘think outside the box’, but to learn from them, and so make the box the rest of us work in bigger. Much, much bigger.

Objective 3: A manageable management structure

If the purpose of maturity management is to achieve radical change by incremental steps, and its principle instrument is the conversion of leadership into management, it is clear that its next objective must be to define a management system that drives change. More precisely, maturity management must tell us:

• To define why the assignment exists, and so ensure that the assignment contributes to strategic goals.
• To define what the assignment will do, and so keep the assignment is kept on track to its goals.
• To define how the assignment will do this, and so ensure that an effective technical solution is delivered.
• To handle all the data and decisions needed to reach the above goals.

To achieve this, a maturity management methodology defines a complete, generic management system consisting of three core components:

• A generic management task model.
• A generic management system model.
• A generic management maturity programme model.

Defining generic management components makes it much easier to define management in terms of discrete units of management activity that are easily understood, easy to implement and use, and easy to revise or replace in the face of new problems and changing circumstances. It also provides the bedrock of the principles of recursion and iteration. Furthermore, by breaking the implementation process into short-, medium- and long-term changes and by embedding the components in a well defined hierarchy of maturity levels, systems and tasks, it is easy to adapt the generic components to local needs and the most appropriate methods, tools and techniques.

Documenting V-model-based methodologies

I am often involved in methodology development projects, usually based on the standard V-model. Because any serious solution delivery process now greatly exceeds this simple model, yet it is hard to diagram the extra tasks without creating a complete mess that no–one can understand, this raises the question of what functions, processes and tasks need to be shown outside the main V.

In response, the first question I would ask is what exactly your organisation’s ‘standard’ approach is. That is, what is the approach that you usually take to delivery, that occupies 80% of your people 80% of the time, and so on? That then tells you what should lie in the main V, and what not.

In any organisation whose maid delivery process is bespoke development (as opposed to, say, buying in and adapting packages), I would suggest that all else should be pushed off the main V. This would normally mean that the following are all put offline from the main diagram:

• Procurement.
• Service delivery.
• Testing.
• Project management.
• Business change management.

I would expect each of these to develop its own process (frequently its own V) and to be accessible directly from the methodology home page. Essentially, the reasons why each of these functions needs pulling out are that:

• It’s a minor variant of the major process - the main V should show only the major process (e.g., in this case, procurement).
• It is logically asynchronous with the chosen logical model of solution delivery – in most cases, stage-based development (e.g., test preparation of various kinds).
• Its logical structure is non-linear (e.g., project management).
• It may be invoked at any point (e.g., change control, defect management, risk management, and so on).
• It has a specialised audience, so most people don’t need to know how it is done (all lower-level technical processes).

So procurement should excluded from our V-model only because it is not the usual method for doing delivering a solution, and may not happen at all for many projects. If, however, you are predominantly procurement-driven, then procurement should move straight into the V and bespoke development be put off-line.

Hence also my exclusion of project management from the V, which will probably strike most people as off. But most project management activity (governance, planning, task assignment + tracking, risks and issues, reporting, etc.) is either ad hoc, repetitious or cyclical, and does not fit the linear structure of a V.

On the other hand, to make sure that everything stays aligned and everyone knows what they are supposed to be doing, the main V should ideally include not only everything in the standard delivery sequence path but also the touch-points with each of these other functions. This should indicate the points at which show not only where they provide their respective ‘services’ but also the points at which they take their ‘feed’ from the main process. This can make the main diagram physically or logically complex, but I think that that would be ideal.

For example, the main links between a bespoke development V and service planning are probably:

• During Initiation, where service planning needs to indicate the contributions it will need to make to the project.
• During Requirements, where service planning needs to identify the non-functional requirements it they need to define SLAs, shape the environmental design, and so on.
• During Design, service planning generally need to be involved in environmental + infrastructural design.
• During Build, service planning may need to be involved in unit + integration testing where these relate to infrastructure and environments, and in non-functional testing where this is will bear on SLAs.
• During system testing, service planning will be interested not only in non-functional testing but also in identifying any work-arounds + FAQs (for the Help Desk), plus starting to collect any residual defects that are likely to affect the working solution.
• Finally, during the Deployment stage, service planning will be involved in Operational Acceptance Testing and a range of deployment planning and activity.

There may be other lesser dependencies, such as where various flavours of readiness need to be checked. But by and large, the definition of all these tasks, plus any supporting techniques, tools and templates, guidelines, process maps, and so on, can all lie outside the main V. Only the touch-points need to be identified there.