Moving to Agile - Creating an Agile environment

In my experience a crucial precondition for achieving agility is a certain level of maturity. Although the core processes, roles and tools are critical, so is the presence of quite a wide range of highly standardised core mechanisms of which the agile operations can take advantage.

The classic example, familiar from IT development (whence agility sprang, of course) is test automation. Without this the development team is unlikely to be able check its day’s labours swiftly enough to move on confidently the next day. But test automation itself can only be adopted by an organisation that already has standardised test classes, a well established test process, a clear understanding of the basic mechanisms of test scripting, and so on. Without all of these (and much more), test automation will fall flat on its face – becoming either ineffectual or rigid – and quickly start to turn agility into paralysis.

Of course, in a vey small, simple project or activity, the preconditions for agility are very limited. But in a more complex situation, such as BAU operations or a large-scale programme, agility can be achieved, but only ensuring that a full ‘agile environment’ is also in place. The elements of this environment can themselves be agile, but they certainly must be present and specifically geared to allowing other areas to take them completely for granted – it is, after all, the most basic basis of agility that the would-be agile activity can either omit or take for granted that everything in their environment.

Hence one of the key task – perhaps the single most important task – when implementing agile methods is to investigate what the organisation’s ways of working can offer to the agile area – and what they demand from it too.

A few (there are many more - this is just a flavour) of the areas you need to get right include:

1. A governance system that allows for rapid validation and approvals of many incremental releases.
2. Business and operational organisations and processes that are capable of assimilating frequent change.
3. Office arrangements that support closely collocated teams.
4. Extremely slick mechanisms for remote groups – vendors, outsourcers, other offices, and so on.
5. System architectures that support rapid change.

And so on - an enormous range of factors, many of which are typically intensively embedded in the wider organisation. it isn’t easy getting this sort of thing right.

How many maturity models are there, dammit?

Looking through my files on maturity management this morning, I came across the following list - and I don't think it's even nearly complete!

• Agile Maturity Model
• Architecture Maturity Models
• Assessment Maturity Model
• Automated Software Testing Maturity Model
• Brand Maturity Model
• Business Archive Management (thanks to Cole Sandau)
• Business Continuity Maturity Model
• Business Intelligence Maturity Model
• Capability Assessment Tool (CAT)
• Capability Maturity Model for Software (retired)
• Capability Maturity Model Integration (CMMI)
• Change Management Maturity Model
• Change Proficiency Maturity Model
• Configuration Management Maturity Model
• Customer Experience Maturity Model
• Data Center Infrastructure Maturity Model
• Data Governance Maturity Model
• DITA Maturity Model
• Earned Value Management Maturity Model
• E-Government Maturity Model
• e-Learning Maturity Model
• Enterprise Architecture Maturity Model
• Extended Enterprise Architecture Maturity Model (E2AMM) v2.0
• Green Enterprise Maturity Model
• Information lifecycle management maturity model
• Information Maturity Model
• Information Process Maturity Model
• Information Security Management Maturity Model
• Gartners' Infrastructure Maturity Model
• Integrated Product Development Capability Maturity Model
• Internet Maturity Model
• IT Architecture Maturity Model
• IT Maturity Model
• IT Service Capability Maturity Model
• Knowledge Management Maturity Model
• Leadership Maturity Model (LMM)
• Learning Management Maturity Model
• Localization Maturity Model
• Managed Care Maturity Model
• Medicaid Information Technology Architecture
• Gartner's Network Maturity Model
• Open Source Maturity Model
• Operations Maturity Model
• Organisational Capability Maturity Assessment (CMA)
• Organizational Project Management Maturity Model
• Organisational Project Management Maturity Model (OPM3)
• Outsourcing Maturity Model
• People Capability Maturity Model
• Performance Engineering Maturity Model
• PM2 Maturity Model
• Portfolio, Programme and Project Management Maturity Model
• PRINCE2 Maturity Model (P2MM)
• Process Maturity Model (PMM)
• Product Development Capability Maturity Model
• Programme Management Maturity Model (PMMM)
• Project Management Maturity Model
• Property Asset Management Maturity Model
• R+D Maturity Model
• Resource-Oriented Architecture Maturity Model
• Reuse Maturity Model
• Risk Maturity Model
• SaaS Architecture Maturity Model
• SaaS Simple Maturity Model
• Security Maturity Model
• Service Desk aturity Model
• Service Integration Maturity Model
• Services Maturity Model, Self-Assessment Maturity Model
• SOA Maturity Model
• Software Acquisition Capability Maturity Model
• Software Engineering Capability Maturity Model
• Software Maintenance Maturity Model
• Software Reliability Engineering Maturity Model
• Stakeholder Relationship Management Maturity
• Systems Security Engineering Capability Maturity Model
• Talent Management Maturity Model
• Testing Maturity Model
• Threading Maturity Model (ThMM)
• Training Management Maturity Model
• Usability Maturity Model
• Web 2.0 Maturity Model
• Web Services Maturity Model
• Website Maturity Model

... and so on. And on. And on.

I also found a rather nice 'Maturity Maturity Model' and even a splendid Capability Im-Maturity Model!

Given that maturity models are basically a good idea - at least they get us away from the silly idea that radical change can be accomplished in a single step - it's a pity that so many of them are based on the chronically immature SEI CMM model. This, I have always thought, is more like a list of things the DoD finds it hard to do, in approximate order of difficulty.

I have had quite a few goes at maturity models (not to mention basing a complete book on the large-scale structure of human history on an analogous idea), including my 'Lattice Methodology', which is designed to direct strategic transformation programmes by maturity management methods, and a methodology maturity model. I may post either or both here, though I wouldn't get your hopes up just yet.

Anyone got any more? And if someone can find the URLs, I'd be happy to put them in.

The objectives of maturity management

The purpose of maturity management achieve the following major objectives:

• To define a strategy for creating revolutionary change by means of evolutionary steps.
• To free leaders from the limitations of corporate management systems by creating management systems that enable leadership rather than constraining it.
• To define a truly manageable management system capable of supporting fundamental, strategic change.

Surprisingly, these are not stated objectives of other management models such as the Software Engineering Institute’s well known Capability Maturity Model or the Project Management Institute’s standards. Nor are they made any easier to achieve by the approach those standards adopt, which is basically pragmatic, eclectic and bound by convention.

These objectives are described in more detail below.

Objective 1: Revolution by evolution

The primary objective of maturity management is to deliver radical, even revolutionary change. That means not merely re-invigorating moribund management systems and staunching the haemorrhages caused by poor management practice, but creating genuinely world-class organisations.

But how is that objective to be achieved? Most approaches to organisational change share at least one assumption: that radical results could be delivered in a single heroic step. Maturity management is based on a quite different assumption: that realistically, radical change can only take place in well-defined, incremental steps, quite probably extending over many years and certainly requiring many discrete developmental steps.

Hence its first objective: to define a sequence of discrete, manageable stages through which radical change can be brought about. Revolution by evolution, in fact.

Objective 2: Freeing leadership from management

One way of conceptualising how maturity management works is in terms of the distinction many authors have drawn between management and leadership. To quote Stephen Covey's Seven Habits of Highly Effective People:

Management is efficiency in climbing the ladder of success; leadership determines whether the ladder is leaning against the right wall.

Other commentators have expressed similar sentiments in different ways, but it is striking that they all insist on this difference and on the importance of leading organisations rather than merely managing them. Leaders bring vision, inspiration and direction, and without it an organisation loses its impetus, its cultural integrity and its ability to take decisive action.

Yet many organisations seem determined to encumber their leaders with unnecessary or subordinate management tasks, even actively disabling them by failing to provide the basic information and decisions real leadership demands.

Of course, no organisation could succeed by completely replacing management by leadership. Conversely, where leadership is not supported by robust management, the ‘leadership’ and ‘empowerment’ routinely degenerates into senior management abdicating responsibility for the actions, accomplishments and performance of their subordinates, backed up by the usual blame and recrimination when things go wrong.

So a balance must be struck – but only the right balance:

• The ability to manage is quite commonplace, whereas leadership is notoriously rare.
• The ability of leaders to delivery results depends on the presence of management systems (including competent and empowered managers) capable of implementing their vision.
• Unbridled, universal ‘leadership’, if not backed up with clear control of the whole, will soon degenerate into chaos, and the whole becomes a great deal less than the sum of its parts.
• Once they have been applied to a range of assignments, many leadership skills can be translated into reliable methods, tools and techniques that can be taught to less inspired individuals.

Hence another aspect of maturity management: by continually upgrading management systems, activities that previously required that rare combination of inspiration and perspiration that defines genius can be done almost as effectively by any modestly capable individual who has been trained to use the appropriate methods, tools and techniques and is supported by the necessary flow of information and decisions. Indeed, the whole history of management consists very largely of the creation of management systems to do things that were previously done only by great leaders. That is one of the main reasons why great organisations – nations, teams, businesses and so on – can exist at all.

On the other hand, where will future leaders acquire the vision on which leadership so crucially depends? Where will they get that spark of insight leavened by sound practical experience? Surely the answer is, yet again, from the management systems in and through which they work. If these systems are bad, then any manager’s experience will be less than illuminating. If, on the other hand, the management systems they use are well designed, effective and properly directed and maintained, their experience of their work, the organisation and its goals will be clear, well-structured and informative. Its purposes, methods and underlying philosophy will be clear and reinforced throughout. Conversely, the better structured the system, the easier it will be to spot any residual problems. But most importantly of all from the point of view of inculcating leadership, the values, purpose and opportunities it faces will be clear.

Hence the maturity management approach: wherever possible it replaces leadership by management. This is not because we should prefer management to leadership after all, but because we should reserve the special talents involved in leadership for tasks where they are really needed. If some leadership skills can be made so straightforward that they happen as a matter of course and the same results can be reliably achieved by the routine use of a management system, this can only strengthen an organisation, and release its true leaders to focus on areas that demand real leadership.

To summarise the whole above argument in terms of a contemporary management buzz phrase, the trick is not to rely on those who can ‘think outside the box’, but to learn from them, and so make the box the rest of us work in bigger. Much, much bigger.

Objective 3: A manageable management structure

If the purpose of maturity management is to achieve radical change by incremental steps, and its principle instrument is the conversion of leadership into management, it is clear that its next objective must be to define a management system that drives change. More precisely, maturity management must tell us:

• To define why the assignment exists, and so ensure that the assignment contributes to strategic goals.
• To define what the assignment will do, and so keep the assignment is kept on track to its goals.
• To define how the assignment will do this, and so ensure that an effective technical solution is delivered.
• To handle all the data and decisions needed to reach the above goals.

To achieve this, a maturity management methodology defines a complete, generic management system consisting of three core components:

• A generic management task model.
• A generic management system model.
• A generic management maturity programme model.

Defining generic management components makes it much easier to define management in terms of discrete units of management activity that are easily understood, easy to implement and use, and easy to revise or replace in the face of new problems and changing circumstances. It also provides the bedrock of the principles of recursion and iteration. Furthermore, by breaking the implementation process into short-, medium- and long-term changes and by embedding the components in a well defined hierarchy of maturity levels, systems and tasks, it is easy to adapt the generic components to local needs and the most appropriate methods, tools and techniques.

Even if you can measure, maybe you still can’t manage

It has always been assumed that the use of metrics is one of the more convincing indicators that business management is approaching maturity. There is an alternative view, however, which is that measurement is used because there is some mileage in it, and it is easier than being serious about understanding management.

This is all encapsulated in the famous consultant’s dictum that if you can’t measure you can’t manage. This in turn seems to derive from a remark by Lord Kelvin:

When you can measure what you are talking about and can express it in numbers,
you know something about it; but when you cannot measure it, when you cannot
express it in numbers, your knowledge is of a meagre and unsatisfactory
kind.

Now Lord Kelvin was one of the truly giant figures of science, so he probably shouldn’t be contradicted without good reason. But there is another figure, altogether more relevant to management and business in general and, unusually for that field, of equal standing even to Kelvin. This is John Maynard Keynes, who not only invented large chunks of twentieth century economics and shaped how the world worked for half a century but was also a great authority on statistical methods. So his opinion is certainly worth considering. And while his opinion does not contradict Kelvin’s, it certainly undermines quite a lot of modern business measurement programmes.

Am I right in thinking that ... the statistical method ... essentially
depends on ... having furnished, not merely a list of the significant causes,
which is correct so far as it goes, but a complete list? For example, suppose
three factors are taken into account, it is not enough that these should be in
fact verae causae [true causes]; there must be no other significant factor. If there
is a further factor, not taken account of, then the method is not able to
discover the relative quantitative importance of the first three. If so, this
means that the method is only applicable where [one] is able to
provide beforehand a correct and indubitably complete analysis of the
significant factors. The method is neither one of discovery nor of
criticism.

In other words, measurement is only valid where the underlying model of what you’re measuring is:

• Coherent.
• Consistent.
• Complete.
• Correct.
• Current.

… and probably a lot of other things beginning with ‘C’.

Now, I’d like to think that modern management and business systems were based on a clear conceptual framework, but my sense of humour is not quite so surreal. It is true that areas like manufacturing, logistics and mass commodities measurement is alive and very healthy, but I suspect that that is mostly because these areas are closer to technology than business. But as far as the day-to-day management of less mechanical things – such as people and projects - is concerned, it is simply contrary to the management culture of most companies to manage in the objective terms that measurement either assumes or supports.

I think I can honestly say that every people- or project-based company I have ever worked in was racked with opportunism, pragmatism (no, not a good thing, especially in this context) and the horizons of a whelk. In fact the entire history of management often seems to consist of one insight/revolution/fad after another.

This is completely anathema to the entire ethos of measurement, and the persistence of this attitude strongly implies that:

• We don’t really know how business and management work - otherwise we would not let consultants sell us a new toy every ten minutes.
• We don’t really care how business and management work - otherwise we have recognised how pointless much of what management does really is decades ago, anda come up with more intelligent models.
• The widespread introduction of measurement and metrics isn’t noticeably improving our understanding of either.

Certainly we are nowhere near the point where Kelvin’s advice (which was pretty weak on a few other things) can be sensibly applied.

But does it matter? Not much, as it happens. You only have to go back to the source of the ‘if you can’t measure you can’t manage’ philosophy to see why. Lord Kelvin was a physicist. He studied atoms. In particular he studied how things move and use energy when you bash them about. As I have argued elsewhere, as a model for anything except physics, physics fluctuates between the unhelpful and the disastrous, and other people who have taken Lord Kelvin’s a bit too seriously whilst he was banging on about non-physical matters seem to have come unstuck. Even Charles Darwin was seriously disturbed by Lord Kelvin’s ignorant but highly effective assault on evolutionary theory, which was based solely on a model of matter that left out radioactivity and which I have written about in more detail elsewhere. But this was a perfect example of perfectly accurate numbers leading to completely spurious conclusions because the underlying model was wrong.

But metrics are a limited (though not useless) basis for managing business for a different reason. As far as intelligent beings are concerned, metrics make sense only if you can assume that the basis on which they act is not changed by the fact of being measured. But we know from a dozen different sources that this is exactly what is not true about human beings. Indeed, it cannot be, for the simple reason that human beings are not atoms. Rather, as conscious beings, if they become aware that they are being watched, this fact alone is enough to create a new ‘factor’ in the ‘system’, and so to undermine the very assumption that the observers know what they are looking at!

Take, for example, the well-known Hawthorne Effect. Between 1924 and 1932, experiments observed workers’ performance as they changed various conditions – lighting, group incentives, and so on. They found that it was not the changes themselves that caused increases in performance so much as the workers’ consciousness that they were being watched (as demonstrated by the constantly changing conditions). In other words, measuring their performance changed their performance.

Hence another well-known phenomenon in business and management, which also tends to undermine measurement, namely the fact that people tend to manage so as to optimise what they are being measured on. Again, the Keynes effect – the system is changed by the fact of being measured.

Is this a problem for management? Yes – if you think that human beings are simply assets you buy and sell and will do as they are told in a completely mechanical manner. And in some businesses, perhaps what the organisation needs really are such drones. It must be a very crude, basic industry if it is.

Yet we should still celebrate the fact that human beings are never so lifeless that they can be expected to behave like things instead of people. For this simply ignores where the real ‘value’ in employing intelligent beings comes from – from their insight, their creativity, their enthusiasm, their combination of love of doing great things with the uniquely human quality of responsibility to do it properly.

When we can reduce that to numbers, we will be either gods or in big, big trouble.

Capability management cycles

I have spent a long time studying capability management - the active creation of a integrated approach to continuous improvement that goes beyond localised incremental change.

This diagram summarises my current view of this process.

(Click on the picture to expand.)

Taking quality back from the bureaucrats

I have spent a good deal of my career around quality management. Be it pure ISO 9000- and TQM-style quality management, more tangential activities such as methodology and process architecture or innovation and research management, I think I have done pretty much everything in this field.

One persistent theme in all this experience – and in most organisations this seems to be as true today as it has ever been - is the tendency of quality to degenerate into bureaucracy. For serious professionals quality should be the sexiest thing on earth, but in fact most people find that quality management is a tiresome chore of jumping hurdles, filling in forms, being subjected to irksome and apparently pointless audits, and being admonished to do better by tiers of senior management who plainly haven’t a clue about what your working environment is really like.

Hence the collapse into bureaucracy. If I can’t/won’t join up all the dots, then some of them will have to remain mysteries and I can only get you to do what I want by simply ordering you to do it. It doesn’t have to be a direct order – I can just create a system of bureaucratic controls and reports and you will do as you re told anyway.

Well, that’s the theory. But as every real manager knows, it just doesn’t work like that. In fact, if I wanted to design a system for stifling commitment, creativity, and passion, that’s how I’d do it – replace a real understanding of quality with a bureaucracy.

So how to take back quality from the bureaucrats? Not easy. Perhaps, ultimately, not possible. But here are three steps that should take you some way towards that goal.

Step One – Define quality as excellence

If you want passion, commitment and creativity, then your quality management system absolutely must nurture professional, managerial and operational excellence. If you want to create a culture in which everyone wants to do their job and then some­ - the basis of every truly triumphant organisation – then there is no alternative.

But is this not already the norm? Although there are many preliminary definitions of quality (e.g., as compliance with specification, as fitness for purpose, as customer satisfaction, and so on) surely everyone in quality management would probably like to define quality as ultimately about excellence?

True so far as it goes, but as so often, it’s not far enough. The problem is that these definitions tend to stay on the rhetorical level. In the absence of a genuine culture of quality, it is extremely difficult to move quality on any further. So what is needed to make that move? What is needed is a practical definition of what excellence is. I would suggest the following key elements that, if accepted, provide a basis for practical change. In a sense this is quite simple and already very familiar.

For example:

• Excellence must be the explicit, overriding goal. Not conformity or standards or even customer satisfaction.
• Your organisation can only excel as a whole. Privileging supposedly ‘key’ functions is self-defeating.
• Excellence can only be known through practical results. These results must be defined in terms of achievements, not activity.
• Excellence must be driven. Leadership should be the goal at every level – corporate, functional, individual. To achieve this, innovation must be actively pursued and external research, benchmarks and resources vigorously exploited.
• Excellence is suffocated by ignorance, distraction, trivia and noise. Excellent people need excellent leadership, processes, systems, skills, tools, information, and support.
• Excellence is founded on a personal desire to excel, not a bureaucratic procedure. Translating a corporate plan for excellence into a personal desire for demand recognition and reward, for contribution to every level.

All quite familiar, really. But how many of these are embedded in management systems that facilitate accomplishment, development and leadership? In my experience, very few. And they are almost invariably confounded by the presence of countervailing forces, such as the standard ‘Not in this financial year’. ‘Not in my backyard’ and ‘Not invented here’ syndromes. There is seldom any investment in standing back and looking at the organisation as a whole or in looking to what other have achieved, and little reward for creating change.

Step Two – Mapping your management system onto your definition of excellence

Hence step two – creating the management system that will actually communicate, support and track this translation of corporate goals into local objectives. Here are some basic elements of such a system:

• Quality is defined and measured by intrinsic values – delivery, satisfaction, ROI, etc., not formally correct ‘quality metrics’.
• Quality management is recognised as a key investment, not just an overhead or cost.
• Quality is a cultural value. Not only does everyone routinely ask ‘How can we do this better?’ but there are regularly meetings and improvement systems for making sure that they really do get better.
• Your management systems are intrinsically adaptable – and regularly adapted. That is to say, they are rigorous but not rigid, and driven by explicit objectives that are traceable to real corporate goals.
• The Quality System itself is driven exclusively by identifiable risks, not by formal models or methodologies. If you can’t say what risk your management system is responding to when it makes me follow some formal routine, then what is the point in my doing so?

The management system is owned by its users and stakeholders, and no one else. The ultimate test of this is that your management systems are valued by its users, not imposed from without.

Again, not exactly startling. At least, I hope not, after a couple of decades of consultants like me ranting on about this sort of thing.

But again, how much of this is real in your organisation? Have you ever looked? Have your ever gone about your organisation, in disguise like a medieval prince, to see what life is really like for the teeming masses? You don’t believe your own subordinates’ and HR department’s protestations that all is well, do you? You don’t read Dilbert? Ye gods…

Step Three – Make it happen

Finally, some basic mechanics of a management system that will deliver all this. There are endless things that could be done, but I always feel that these are the most directly useful.

• Insist on black-box management – it's the ideal way to combine empowerment with responsibility. But make sure that you define all four 'outside' sides of a black box - the expected output, the standard inputs, the underlying infrastructure and information sources and the overarching context information that tells your team what they are contributing to - but also the 'inside' side - the basic standards, frameworks and tools for doing the job. Otherwise you will end up with everyone making it up as they go along.
• Promote your best to managing the management system itself. Respected individuals with vision and imagination who really get things done.
• Make a stint in some form of quality management a precondition of promotion. You might be astonished how must there is to learn from a simple exercise like conducting a quality audit or building processes.
• Build systems that explicitly connect your strategic goals and tactical activities together. And make sure that they work both ways - top-down and bottom-up.