Can non-technical reviewers review technical products?

In many (perhaps most) of the organisations I have worked in, the question of who reviews what has been either contentious, ambiguous or been given a quite unnatural answer. The reason for this unhappy situation is that it is often difficult for a non-technical reviewer to evaluate a project, especially its technical content.

In my own specialist area – IT – this might manifest itself in a pained question such as ‘How can the business approve a change in a database design?’ But there are similar questions in all complex management situations – can techies contribute usefully to business cases, for example?

Good question – and in my experience, people only say ‘good question’ when they mean that there’s no good answer. But in this case, there is an answer, and what is more once the answer is understood it leads to a more robust approach to reviewing generally.

The basic problem is to decide what objective reviewing is trying to achieve, and so to decide whether non-technical (non-business, etc.) reviewers have any role to play in achieving it. To put it concisely, the purpose of reviewing is to decide whether the item under review is meeting its requirements. I don’t mean this in the technical sense of ‘requirement’ – i.e., something the work is supposed to achieve for it to be considered a success. I just mean does it do what it is supposed to do? This might well mean ‘does it fulfil its requirements?’, but it could also mean ‘does it comply with this specification’ or ‘if we follow this plan, will we succeed?’, or any number of other things.

From that point of view, the right reviewers are the people who can – and need to – make that call. But that still doesn’t mean that they are technically capable of understanding the item they are reviewing. Or are they? In what sense do they need a precise technical understanding of the content of the item – for example, a design document - to be able to evaluate it? To put my complete argument in a nutshell, what I am getting at is the idea that reviewing is based not on what it says so much as on what it means.

To go back to my problem about business people reviewing a change to a database design. Can they understand what the change says? Probably not, if by that you mean their grasp of namespaces, indexing and denormalisation issues, and their opinion of such things, in strictly technical terms, is probably worthless.

But that isn’t necessarily all that the review is for. For behind every such technical change, there is a pyramid of managerial and business implications that non-technical reviewers can not only understand perfectly well but are probably better judged by non-technical people.


This is illustrated in the following diagram (click to expand):

Hopefully it is clear what the diagram implies. At the lowest level, where the database change itself occurs, there is probably little benefit to be had from asking non-technical people what they think of the change from a purely technical point of view. ‘Who knows, and who cares?’ is probably the right answer. But as soon as the wider implications of the change – the non-technical elements of what the change means rather than the details of what the change documents say – start coming to the fore, both their interest and their ability to judge should start to grow rapidly.

For example, assume that the database change in question is to move from a distributed to a centralise structure. Although the technical issues will be beyond the business’ grasp, and so will most of the implementation and operational issues, not much else should be beyond them. Looking at the diagram again, what are the changes in test requirements this database change will call for? To have all your database testers in one team, located centrally, rather than separate teams all around the business? What does that entail? Much lower costs? Great, we’ll have it. And a simplified roll-out that can now happen three months earlier? Even better. But what is the downside? The changes in platform mean that we will need to recruit a whole new database team? How long will that take? What will it cost? Oh... not such a no-brainer then. And there’s a small chance that we won’t be able to meet our delivery timescales after all? But at least the total development cost will be well down? Great! But the operating cost will in fact go up? Damn...

It’s a complicated business, as anyone who has been in such a situation will testify. But perhaps it should be – and perhaps excluding the business (and other non-technical people) from reviews on the grounds that they ‘won’t understand’ what they are reviewing is not only a very narrow interpretation of what ‘understanding’ means in such a situation but positively counter-productive. After all, if you don’t ask them now, when will you? When it’s too late?

Of course, it’s not easy to make sure a review like this is successfully executed. It’s very hard to work out the real implications of as subtle a thing as a database change. But if you are the project manager and you can’t tell your customers what the consequences of your project really are, perhaps you should be finding out. After all, it’s not as though they will never find out. But the alternative to telling them in an orderly and systematic manner like the above can only be finding out through missed milestones and blown budgets.

In a way none of this should need saying – anyone who does a change request nowadays will perform an impact analysis that covers most of these issues. But as so often in project management, this simple lesson simply has not spread in the systematic manner to areas like reviewing (product or project) as one would have hoped.