The DfID response to my latest FOI request says that there are over 3,000 DfID projects which have had their performance assessed and recorded on the database.

They say they won't publish it because it "supports the operational management" of the projects (rather than being "key project content").

I've asked them for a data dump of all the records relating to these hidden fields. Once it's released, it should be possible to do a mash-up with the already-published information in the PPI.

DfID divulge data

DfID have today released the first version of their Project Information Database.

There is a wealth of information there but it is important to ask straight away - what other information does DfID have which they have decided not to publish?

In my most recent FOI request I obtained a full list of the project information stored on DfID's databases.

This includes the following fields:

Performance Review
Is an Annual Review Req
Ann Review Exempt Just.
Annual Review Due Date
Annual Review Prompt Date
Annual Review Authorised
Summary Review Authorised
Is a PCR Required?
PCR Exempt Justification
PCR Due Date
PCR Prompt Date
PCR Deferral?
Deferral Justification
PCR Authorised

Output Scoring
Outputs
Impact Weight %
Output Performanc
Impact Weighted Score
Risk

Project Scoring
Review Date
Total Impact Score
Output Risk
Project Purpose Score
Purpose Justification
Overall Risk score

Method of Scoring
Sources of Information

Scoring Responsibility
Partners Involved

Knowledge Sharing/Lessons
Date
Lesson Category
Notes

Conditionality Review
Disbursement Suspended
Cause
Date Suspended
Consequences

As far as I can make out these fields have not yet been included in the online database. Instead, under 'Aid effectiveness' they have included the following, rather irrelevant, indicators:

Use of partner country public financial management systems
Are partner country budget systems used?
Are partner country financial reporting systems used?
Are partner country audit systems used?
Are all 3 partner country public financial management systems used?

Are partner country procurement systems used?

Use of programme based approaches
Does the contribution fit within a single programme/budget framework?
Is the contribution led by the host country/organisation?
Is there a process for co-ordinating donors?
Are at least 2 partner country systems being used?
Is this a Programme Based Approach?

As mentioned last time, I have another FOI to determine the criteria the department used when selecting which indicators to be published. By coincidence the deadline for replying to that is tomorrow! I wonder if they were concerned I might steal their thunder...

This is an interesting paper on different approaches to impact evaluation, published by the increasingly active 3ie.

It includes contributions from big thinkers such as Robert Chambers and Martin Ravallion - who offer widely different visions of both the philosophy and implementation of evaluation.

Self-regulation

The One World Trust have found that there are at least 50 civil society self-regulatory initiatives on development. They are listed in a new database all about self-regulation, which claims to be the first of its kind.

FOI failures

DfID have responded to my FOI request on their plans to publish aid information from their database.

Disappointingly they won't give out information about Ministerial decisions on the matter, but hopefully my latest FOI attempt will prove more fruitful.

Meanwhile, here's another tantalising glimpse of what's to come:

Publishing Project Information (PPI): As part of our commitment to improving transparency, DFID will publish summary project information on its website from early 2009. Information published will include project descriptions, dates, purposes, locations, sectors, aid effectiveness, summary financial data and whether or not conditions are attached. These items will subsequently be made available as a feed to AiDA and other external stakeholders for use in their analysis and information systems. We are also working on publishing more detailed project documentation later in 2009.

Mashing with DfID

Check out this transcript of the 1st meeting of Open Knowledge Foundations' Working Group on Open Knowledge in Development. (I'm andeggs)

We're going to mash around with DfID data that's due to be released on August 10th. I've got an FOI about it here.

One Word: Radicalism

The Tories have published their ideas for international development, called 'One World Conservatism'.

Its got a very encouraging bent towards transparency and evaluation. Here are some of the highlights:

• We will ensure the impartial and objective analysis of the effectiveness of British aid through an Independent Aid Watchdog.
  
• We will commit up to £10 million per year to the 3ie to commission impact evaluations of programmes supported by multilateral aid projects that DFID gives money to.We will volunteer bilateral DFID projects to be evaluated by the 3ie, and actively encourage all multilateral projects to which DFID gives money to have their programmes evaluated by the 3ie.
  
• We will publish full information about all of DFID’s projects and programmes – including the results of impact evaluations – on its website, and have them translated into local languages. This information will be published in a standardised format so that it can be freely used on third-party websites.
  
• We will also work for greater transparency across the whole aid system. We will require all bodies receiving DFID funds, whether NGOs, multilateral organisations or governments, to move towards greater openness and transparency.

Why are the Conservatives so far ahead of Labour on this one?

Last August, a consultancy called Agulhas had a look at DfID's progress on aid effectiveness. Overall it rated the department highly, although this paragraph is telling:

"the evaluation finds that DFID (like most donors) likes to project a positive image of its own performance, and tends to gloss over imperfections in the aid process. (Development NGOs are also guilty of overselling the impact of aid in their ‘Make Poverty History’ campaign.) In the view of a number of internal and external respondents to the evaluation, DFID is over-sensitive to external criticism, and unwilling to engage the public in a frank debate on the effectiveness of external assistance."

White flag

This paragraph is buried deep in DfID's new white paper on development:

"Part of making international aid more transparent is each donor publishing information of the projects they fund. The UK will make information about the projects we fund available to the public both here in the UK, and in the countries we work, through the launch of a new searchable database on the DFID website." (para 7.21)

Now it's up to everyone else to follow suit.

Bah humbug!

Just finished reading William Easterly's A White Man's Burden. In it he describes the unspoken system failure at the heart of the aid merry-go-round:

"I feel like a kind of Scrooge pointing out the ... tragedy [of ineffectual aid] when there is so much goodwill and compassion among so many people to help the poor ... the foreign aid bureaucracy has never quite gotten it - the central problem is that the poor are orphans: they have no money or political voice to communicate their needs or motivate others to meet those needs".

When you buy something in a market or make a vote in a democracy you are sending feedback to the seller or politican. This makes them accountable and pressures them into making decisions whic are your interest, rather than theirs. As the aid system is largely unaccountable the organisations the poor receive a dreadful service.

Since there is no direct line from the poor to the people making decisions, there is a principal-agent problem: aid agencies have too many objectives and each objective is the responsibility of several aid agencies. That is, no one faces the chop when the thing goes wrong.

Having described an all-too-familiar programme failure (this time by CIDA), Easterly writes:

Aid agency watches should tough on such disasters, if only with the aim of strengthening the accountability lobby in foreign aid... The way forward is politically difficult: truly independent evaluation of specific aid efforts... only outside political pressure on aid agencies is likely to create the incentives to do these evaluations.

In the end Easterly sees the way forward as being reliant on pressure from the people who stump up the cash for international develompent: taxpaying citizens.

Mobilize the altruistic people in rich countries to put heat on the agencies to make their money actually reach the poor, and to get angry when the aid does not reach the poor.

We're angry all right!

PS. Check out Easterly's ranking of aid agencies here.

Look harder, Lockhart

June's Prospect carried an article by Clare Lockhart, co-author of Fixing Failed States. She argues that the aid system has failed:

• It's wasteful since so much is spent on overheads
• Recipients cannot hold NGOs to account
• Projects are not thought through with business rigour
• Aid can undermine reconstruction through, for example, distortionary wages

In place of NGO bumph, she argues for greater funding of developing country government activities: "At the heart of the problem lies the failure of the aid system to invest in the restoration - or establishment - of a functioning government, market and civil society."

She's right the system has failed. But her analysis of why it's failed is wrong.

The heart of the problem is actually that the aid system is over-ambitious, unaccountable, opaque, unco-ordinated and not incentivised to succeed. Rather than striving to remake society, aid agencies should try to be effective in achieving specific well-defined goals (such as "Build a road from A to B").

Clare's approach is statist - she calls for "sustained commitments" to governments for "ten to 15 years or more". But, as she notes, aid currently failes to complete even relatively small-scale projects, like building schools (in Afghanistan only 6 out of 140 that had been promised delivered on time). how can it be expected to achieve something so much more ambitious?

What's her suggestion for keeping these externally-funded governments in check? Aid agencies to be made more accountable to beneficiaries.

But this will never happen as long as someone else is paying their wages. Instead, aid agencies must be made more accountable to donors and tax-payers - only we can really demand more.

Disasters at DEC

According to the FT, the Disasters Emergency Committee have stopped commissioning evaluations of how the money is spent. According to the article this "potentially raising questions over whether millions of pounds in donations are being used effectively." It certainly does.

NAO it is available

The NAO report mentioned in the post below is available here.

NAO or never

I've been given a copy of an unpublished National Audit Office report into NGOs' responses to the South Asia earthquake in 2005. It a damning analysis of aspects of the relief effort, for instance:

• Single person summer tents were airlifted from the UK even though they were unsuitable for a Pakistani winter
• Temporary shelters were distributed without technical advice being given on their construction - causing injury and death as a result
  
• NGOs were largely unco-ordinated and failed to plug into the 'cluster' system

But most serious is the following assessment of the charities' evaluation of their impact:

While financial reporting systems enable accurate reporting of inputs and costs, it is more difficult to assess impact. Reports submitted to donors did not always show clearly what had been achieved specifically with the donors' funds as opposed to others sources of funding...The problem was exacerbated by apparent inaccuracies in reports to DFID and DEC. For example:

• With £300,000 of funding, one NGO claimed to have produced 15,000 temporary shelters and a second NGO 3,300.
• 3,750 blankets cost one NGO £78,000 while another purchased 19,800 with £54,000.

In addition, above it was stated that some NGOs had underestimated family size in planning. At least two increased the average used when reporting against objectives; thereby appearing to increase the numbers reached. For example; one NGO stated that it had met its objective for beneficiaries reached. However, as it had had to distribute more tents per family, while it distributed the planned number of tents, it achieved only 82 per cent coverage for provision of winterised shelter in the target area.

Without assurance that reported figures for purchases and beneficaries are accurate it is not possible to determine where there are errors in reporting and where there are actual variations in performance. Robust analysis of what agencies have achieved with funding is vital for DFID to be able to assess the performance of the agencies it is funding and could also provide an opportunity for agencies to identify areas for improvement in their systems. Agencies should ensure that they have good systems to record outputs and outcomes by donor, and DFID and DEC should review reports for inconsistencies and undertake analysis of the figures.

Only one thing to add: The reports should be published so that there can be public scrutiny of where the NGOs were effective - and where they weren't.

Not a load of CODswallop

Here are the presentations and papers from a recent Brookings conference on aid. Most of them relate to the macro v. micro initiative debate, recently stirred up by Prof. Banerjee from MIT.

However the most interesting thing is a set of slides from Nancy Birsall's Center for Global Development on an idea for development programmes delivered through 'Cash on Delivery' (COD).

The idea is that donors would only get paid for their outcomes (not inputs). For example, a donor working on education would receive $100 for every child who completed a year of primary school. This would obviously put much greater pressure on donors to perform.

Marking up

Development Initiatives have announced an ambitious and exciting project called 'aidinfo' which seeks to make aid information more transparent and accessible to the world:

People in rich countries will be able to find out more quickly exactly where and how their money has been spent – and the impact it has had. They will be able to find out what people think of the services. They will be able to hold their own governments to account for whether aid money has been well used. And if they see that aid is spent wisely, they may be willing to provide more in future.

Their memorandum to the Commons Committee on Co-ordination for Aid Effectiveness, suggests the standard may be based on the International Development Markup Language.

A word of advice to DI: concentrate more on the political hurdles than the technical ones. There is widespread and entrenched resistance to transparency.

It is important to me

In May the Charity Commission published a survey of 1,000 Brits, in which the following unequivocal views were expressed:

• 96% of respondents agreed that "It is important to me that charities provide the public with information about how they spend their money"
• 90% of respondents agreed that "It is important to me that charities explain in a published annual report what they actually achieved"

So why don't they do it?

In a word

At an initial cost of at least €143,000, a group of civil society organisations (CSOs/NGOs) have belatedly begun to examine how their effectiveness can be improved. The process is called 'Framing and Promoting the Effectiveness of CSOs as Development Actors' and kicks off with a meeting on 29th June run by CONCORD.

An interesting tension that emerges from the concept note and FAQ paper is that between 'aid effectiveness' and 'development effectiveness'. Governments signed up to the former as part of the Paris Declaration, but CSOs have agreed that they would rather the latter:

The concept framing the process is “development effectiveness”, which goes beyond the concept of aid effectiveness enshrined in the Paris Declaration. While the latter is perceived by civil society as being a narrow and technical agenda designed to govern official aid, “development effectiveness” is much broader in scope and frames effectiveness in terms of the impact development has on poor and marginalised communities and territories. It therefore reflects more properly the diverse roles CSOs play in development. Aid effectiveness can be conceived as a part of development effectiveness.

Of course civil society organisations are not the same as governments and it is not unreasonable to imagine them adhering to a different set of accountability principles. But this paragraph reads suspiciously like CSOs are preparing for a cheeky manoeuvre.

They would be well advised not to try and slide from a tightly defined concept (aid effectiveness) to a more loosely defined one (development effectiveness) for the sake of convenience. Donors are watching!

Tailored suits, tax-free salaries, white Land Cruisers and Geneva

Newspaper commentators are not often worth reading but this sustained attack on internationalism by Simon Jenkins is. Here's the best bit:

We are all still hardwired to treat international as a good thing. In the process we have abandoned the constitutionalism and accountability that should govern any form of government if it is not to run amok. The one facet of neoconservative America that I share is frustration with the UN and related organisations' inability to walk the talk.

Charismatic compulsion

Paul Collier, author of the Bottom Billion, recently spoke at TED. He points out that Angola's annual oil revenues were $50bn, dwarfing the entire $34bn aid to the poorest billion people; and argues a set of international standards on good governance in extracting commodities.

Sure, fine, but as the previous post pointed out net private resource flows to the developing world from US citizens and companies exceeded $130bn last year. It is cross-border links which are really growing; and which aid organisations must exploit.