NAO or never

I've been given a copy of an unpublished National Audit Office report into NGOs' responses to the South Asia earthquake in 2005. It a damning analysis of aspects of the relief effort, for instance:

• Single person summer tents were airlifted from the UK even though they were unsuitable for a Pakistani winter
• Temporary shelters were distributed without technical advice being given on their construction - causing injury and death as a result
  
• NGOs were largely unco-ordinated and failed to plug into the 'cluster' system

But most serious is the following assessment of the charities' evaluation of their impact:

While financial reporting systems enable accurate reporting of inputs and costs, it is more difficult to assess impact. Reports submitted to donors did not always show clearly what had been achieved specifically with the donors' funds as opposed to others sources of funding...The problem was exacerbated by apparent inaccuracies in reports to DFID and DEC. For example:

• With £300,000 of funding, one NGO claimed to have produced 15,000 temporary shelters and a second NGO 3,300.
• 3,750 blankets cost one NGO £78,000 while another purchased 19,800 with £54,000.

In addition, above it was stated that some NGOs had underestimated family size in planning. At least two increased the average used when reporting against objectives; thereby appearing to increase the numbers reached. For example; one NGO stated that it had met its objective for beneficiaries reached. However, as it had had to distribute more tents per family, while it distributed the planned number of tents, it achieved only 82 per cent coverage for provision of winterised shelter in the target area.

Without assurance that reported figures for purchases and beneficaries are accurate it is not possible to determine where there are errors in reporting and where there are actual variations in performance. Robust analysis of what agencies have achieved with funding is vital for DFID to be able to assess the performance of the agencies it is funding and could also provide an opportunity for agencies to identify areas for improvement in their systems. Agencies should ensure that they have good systems to record outputs and outcomes by donor, and DFID and DEC should review reports for inconsistencies and undertake analysis of the figures.

Only one thing to add: The reports should be published so that there can be public scrutiny of where the NGOs were effective - and where they weren't.