Here is a startling account of how organisations resist criticism and change.

The following is an excerpt from Burt Perrin's review of DfID evaluations (p.17), which was commissioned by the Independent Advisory Committee on Development Impact.

A key issue regarding the role of committees and DFID managers concerns their authority with respect to commenting upon and approving deliverables, in particular drafts of final reports. Frequently, management and/or steering groups will provide very detailed comments, sometimes using Track Changes, to ‘suggest’ changes or even rewriting parts of drafts, deleting some critical comments and replacing these with other more positive statements. These ‘comments’ are sometimes strident and very directive.

In my view, such ‘feedback’ provided to almost all the evaluations reviewed represents a clear threat to their independence. In most cases, it appears that evaluation teams were able to deal with requests for inappropriate modifications in a responsible fashion. But this can be difficult, where even the expectation of negative reaction to critical comments can lead to self censorship. In my view, there are two studies, the Private Sector Infrastructure evaluation and the Pakistan country programme evaluation, which clearly crossed the line such that their independence was compromised...

In the former case, very strong demands for change to drafts were made, including indicating what the evaluation ‘needs’ to say. Important considerations were omitted from consideration or changed from evaluation questions to assumptions, including questions raised in the terms of reference and the preliminary literature review (whether or not infrastructure support necessarily contributes to poverty reduction, and the value of a facility approach itself).

In the case of Pakistan, some documentation was withheld from the evaluation team on the grounds of confidentiality. The evaluators also were very clearly and strongly told that they just could not say certain things that the Government of Pakistan at the time might find objectionable, out of concern for ‘sensitivity’, even though this included reference to a published article in a UK newspaper (e.g. ‘We cannot allow the sort of judgement in this sentence in what will, effectively, become a public document … The regime and media here will not make the distinction between us and our consultants.’).

For these and other reasons, it was not possible for the evaluation report to speak of the actual reasons for some of the reported findings. There were also separate internal reports that call into question the transparency and integrity of the formal published evaluation report and the management response.