The End of the Beginning

I've decided to close this blog.

A visceral disgust in development NGOs' unwillingness to be open with their donors prompted me to start back in 2007. I was shocked that these do-goody organisations feared transient negative publicity more than long-term ineffectiveness.

Along the way I've been pleasantly surprised at the power that can result from good arguments and a means to communicate with the world. When I started I had no idea that what I wrote would be heard outside of the echo chamber, let alone persuade DfID, Oxfam and others to open up.

2011 saw a sea change in attitudes towards open data in development and there are now major organisations such as Publish What You Fund, aidinfo and IATI working towards these ends. I feel we've turned a corner now and it's time for this blogger to hang up his hat.

Thanks for following - and please do stay in touch through my profile.

Oxfam commits

Here's an interesting post about Oxfam's commitment to publish to the IATI standard.

It's fantastic to see how seriously they are taking the responsibility. They've come a long way since 2007, when their reluctance to publish first inspired this blog.

Kevin Starr at IIT Design Research Conference 2010

Kevin Starr at IIT Design Research Conference 2010 from IIT Institute of Design on Vimeo.

In response to the Information Commissioner's decision notice about my FOI request for information in DfID's ARIES database, DfID sent me a CD with this Excel file.

It contains internal performance scores for about 2,800 projects conducted by DfID. Most of the scores were recorded either as part of a project's annual review or when a project finished.

I have conducted a very preliminary exploration of the data, which shows the variation in the performance of DfID's programmes (called 'Budget centres'):

[This table is based on the most recent (non-zero) output scores for each project that had them. 'Budget centres' with only 1 project are not shown]

Even this initial examination of the data poses some interesting questions about what is being funded and makes one curious to know more. Why have projects in Liberia done so well while those in Rwanda have floundered?

The dataset also contains information on projects which have had their funding suspended, information about the perceived risk of each project and detailed information about the desired outputs from each project. I hope others will help integrate this dataset with other datasets to reveal new patterns.

The letter I received from DfID made the point that under the UK Aid Transparency Guarantee "from 2011/12 onwards, project scores will be published as part of ARs [Annual Reviews] and PCRs [Post Completion Reports] and as a separate dataset". This is particularly encouraging as it would mean that the publication of performance information would be standard and regular, rather than only in response to ad hoc requests like this.

The full text of the decision notice from the ICO is now available for those who are interested.

Decision time

Yesterday I received the Decision Notice from the Information Commissioner's Office (ICO) about my FOI request for the project information in DfID's ARIES database.

Almost 2 years after I originally made the request, the ICO has found partly in my favour: DfID must publish 64 fields relating to the 8,000 projects in the database.

However, 6 of the fields - those which contain comments relating to project performance - have been withheld under the argument that their disclosure would be likely to inhibit civil servants' ability to freely and frankly exchange views and advice.

According to the letter (which will be published on the ICO's website), DfID must publish the information within '35 calendar days' of 23 June 2011.

What's going to be released? The Decision Notice only shares the titles of the fields but together with this user guide to the ARIES database (PDF), we can guess at what is likely to be most interesting:

• 'Total impact score' - a measure of how effectively the project achieved each of its outputs
• 'Output risk' - a measure of the risk of project failure
• 'Disbursement suspended' - information relating to the suspension of disbursement
• 'Method of scoring' - whether reviews were conducted by DfID staff, consultants, etc

It is really exciting that so much information will be released and I look forward to examining it and mashing it up with other datasets. Hopefully the data publication won't be a one-off: it would be great if the fields were regularly published on DfID's website.

What's been refused? The fields which won't be released contain comments made by DFID staff and partners in relation to the projects under review. The ICO argues (para 88) that because some of these comments "could potentially embarrass foreign governments", their disclosure could make civil servants "more circumspect in expressing their views about the performance of a project" - and that this would therefore make them less able to provide free and frank advice.

This makes sense. But here's a question: wouldn't it be possible for each of the records to be assessed to determine which records are sensitive and which are not? Not according to the Decision Notice: DfID explained to the ICO that doing so would place "an impossible burden on the department", particularly since the expertise required to do such a review is spread right across the organisation. It is obviously not right to place such a burden on DfID and I accept the Commissioner's decision on this.

Stepping back from this particular FOI request, we are still left with the problem that rich information about what works (and doesn't work) in development will still be withheld from the public domain. Perhaps the problem could be approached in a different way? Rather than attempting to review the entire dataset at one fell swoop, a team from DfID or ICAI could annually review a subset of the comments (say, for projects completed 1 year previously) and determine which information could be released and what should be redacted. At least then there would be a mechanism for releasing the raw data - and ensuring it isn't lost to the sands of time.

It is clear that a lot of people worked hard to respond to this request. My thanks go to everyone at DfID and the ICO who were involved.

DfID's muscle

According to this FOI response I received from DfID, from next financial year NGOs which are funded through a Programme Partnership Arrangement (PPA) will be required to publish information about DfID-funded projects in line with IATI standards.

The requirement to publish data in the IATI standard by 2012/13 forms part of the Memorandum of Understanding between DfID and the organisations (para 19).

According to the accompanying letter:

"Each PPA partner is required to submit an annual report to DFID for appraisal. The format of this report has still to be finalised but it is intended that it will include a requirement for all PPA partners to show the progress they are making towards becoming IATI compliant."

DfID should publish these annual reports automatically (but if they don't I expect they'll be obtainable under FOI).

About 40 of the largest development NGOs receive funds from DfID through the PPA. Once each NGO has set up the mechanisms for publishing DfID-funded projects in IATI format, there won't be any excuse for not publishing information about all their other projects too.

Make Aid Transparent

Great video from Publish What You Fund to launch their new Make Aid Transparent campaign.



Their website encourages people to sign a petition asking governments to provide better information about the aid they distribute.

The recent AidWatch 2011 report scored European donor countries according to their transparency. As a rough calculation, if we divide each country's aid spend by its transparency score then we can identify the countries where improvements would have the greatest effect.

France and Germany come bottom of the list. They are each responsible for about €9 billion of aid (2009 ODA figures) but scored dreadfully on transparency across the board.

Let's hope this campaign concentrates on these two laggards.

unique ideas about unique ids

I recently came across opencorporates.com, a madly ambitious project from the prolific Chris Taggart.

The website started in Dec 2010 as an open database of all companies in the UK - based on scraping the Companies House website. It has since gone global: by recruiting other scrapers, opencorporates now contains details of companies in over 20 jurisdictions.

It looks like it could soon become the world's central database of corporations.

The power of a database like this comes when it is connected to other databases. Like this list of all organisations funded by DfID for example. (Most will be registered as a corporation, even if they are also registered as a charity or NGO).

If the databases were connected then we could receive updates every time information about any of the organisations funded by DfID changed - for example every time a Director left, an annual report was submitted or the company changed registered address. In time, if other datasets were also connected, we could conceivably get updates every time any of the organisations was taken to court - or every time a press release was written about them.

This brave new world of linked data is coming fast. For the International Aid Transparency Initiative (IATI), it it is crucial that donors use unique identifiers (based on their registered company number) when referring to other organisations, so that this data can be connected to other datasets.

Although DfID should be applauded for being the first donor to publish data in the IATI format, not including unique identifiers for recipient organisations is a big omission.

If DfID know the company numbers for organisations they fund, then they should include them in their IATI data release; if they don't know them then they should start collecting them.

New ICO update

Another update from the ICO on my FOI request. Don't hold your breath!

"The draft decision is now with the designated signatory awaiting the final review before it is issued. We are nearly there now and hopefully a decision will be made fairly soon.

I apologise that it has taken us longer than expected to issue a decision. I can assure you that it is primarily to do with the complicated nature of the information under consideration rather than any avoidable delays on our part."

ICO update

Just heard from the Information Commissioner's Office on their handling of my Freedom of Information complaint. The officer handling the complaint emailed me:

"The Department responded to a number of my queries between September and November last year. I had to then seek input from our policy team. I am currently in the process of setting up a meeting with one of our signatories and other managers to further discuss some of the issues raised by the Department’s response.

Although I cannot specifically indicate when the investigation would be completed, I should point out that we are aiming to close all freedom information cases which were created before April 2010 by the end of March 2011. Your complaint was processed on 24 February 2010."

Which I think means they are still working on it.

Pro-ability

I've just finished Ayn Rand's Atlas Shrugged, whose radical individualist philosophy is provocative and challenging.

At its heart, the book is an argument for the human mind: she argues that science, medicine, industry - progress - result from brilliant people applying reason to the problems they face.

However much ordinary folk toil in labour, she says, it is only the creative, intelligent, fearless few who invent and lead, so producing value in the capitalist system and driving forward humanity.

How many people could have discovered electricity? How many people could have produced the ipad?

Perhaps there is a rather unpalatable fact that some people, by their genes, or upbringing or disposition, are always going to be more productive than others.

It may not be fair and it may not be the way we would have wished it to be. But if it is an historical fact that the effort of a few individuals has been central to human progress, then shouldn't this fact be reflected in development policy?

What would a 'pro-ability', rather than a 'pro-poor' policy look like? What would we change if we wanted to maximise achievement rather than fairness?

Here are some ideas:

• Maths, science and technology olympiads for a country's brightest students
• Overseas university scholarships for promising students (with a monetary incentive to live in-country after graduation)
• Seed funding for entrepreneurs who have shown early signs of success

txteagle

I've just learned about txteagle, a company that connects organisations to the millions of mobile phone users in East Africa.

Nathan Eagle, CEO, suggests that western firms break their tedious tasks down into SMS-sized chunks and pay a few pennies for each one that's completed. It's Amazon's Mechanical Turk taken to extreme.

Couldn't txteagle also be used to gather feedback on development projects? The service is big in Kenya. People on the ground could tell us how this is getting on for example.

Chartjunk

Earlier this week Publish What You Fund produced an 'interactive map' of UK aid spending in conjunction with Open Knowledge Foundation.

I have a lot of time for these two organisations but unfortunately the tool they've produced is pretty lame (in their defence they say it's "work in progress"). Here's what's wrong with chartjunk like this:

• Circles are misleading graphical elements since it is more difficult for the human eye to compare areas than to compare lengths (as you would with e.g. a bar chart).
• It is difficult to compare one year to the next, since the circles move position and many of them are not labelled.
• The provenance and accuracy of the data is not made clear and the 'More info...' button doesn't ever fulfil its promise.
• It is not immediately apparent which circles are subsets of other circles (i.e. 'India' is in 'South' and 'Asia' but 'Bangladesh' is not in 'India')
• There is no comparison with any other dataset so there it is difficult to make any new insights.

What would be a better way to explore the data provided DfID?

I've put the dataset on UK aid spending together with this data on time required to start a business, collected by the World Bank.

It's not the sort of comparison you usually see. Hopefully the resulting chart is interesting (look out for the log scale on the y-axis to spread out the data points).

Quibbling

Last week NPC published an excellent report calling for charities to be more willing to publish evidence of their impact. The researchers write:

When surveyed, donors consistently say that the two most important factors in trusting charities are how the money is spent and what it achieves. For ‘informed donors’, annual reports, annual reviews, impact reports and charity websites will be their first port of call to find out what they want to know. If charities are not communicating their impact in these materials, donors will look elsewhere for those that are.

The report analyses the annual reports of 20 large charities (including Salvation Army) and shows how they largely fail to communicate the difference they have made to people's lives (their outcomes) - even though most are good at describing what they do (their outputs).

I have only one quibble with the report (there always has to be a quibble): while the researchers always name specific charities when they highlight 'examples of good practice', they couch criticism in general terms so that specific poor performers aren't identified.

Maybe they felt that naming and shaming organisations would be unfair since they were randomly chosen? Or that it would be pointless because people generally learn from good - not bad - practice?

But the mistake of this approach is to believe that organisations are willing to be transparent, if only they were shown how.

The truth is that charities choose to remain opaque because it is risky - in terms of both income and reputation - to tell the truth. If it were to name and shame, NPC would help rebalance the equation by making it more costly for charities to avoid publishing their outcomes.

PS Check out OpenCharities - hopefully the seed of a new platform for collaborating about charity outcomes.

Haddads and Haddad-nots

The Institute of Development Studies today released a report based on a survey of people in the UK.

The press release seems bland enough but IDS Director Lawrence Haddad's quote contains slippery logic:

This survey suggests development charities and the Government need to take a fresh approach to engaging with the public about aid. We need to hear more from the people whose lives have been changed by aid. We should do more to understand what UK taxpayers' need to hear to be convinced that aid works. And we need to be honest about what works and what doesn't, so we can learn for the future.

I actually think the opposite is true:

We need to hear more from the people whose lives were not changed by aid. UK taxpayers should do more to understand what the aid sector needs to hear to be convinced that there is a transparency problem. We need to demand information about what works and what doesn't, so that we ensure that the aid sector learns from the past.

Won't somebody please think of the children?

After every high there is a comedown.

Yesterday I uploaded Oxfam's spending data but today I received a reply from Save the Children, repeating their refusal to publish what they fund.

Frankly I consider their argument to be absurd:

I have investigated this further for you and for security reasons we wouldn't be able to give you the names and locations of our overseas projects. If we were to make this information public it could put our overseas staff and the children within our projects at risk.

The problem with making the 'security argument' is that lots of things could put people at risk. Inevitably we have to draw the line and say that the likely benefits of action outweigh the potential risks.

In any case it would be reasonable for Save the Children to choose not to disclose details about a minority of projects if they were particularly sensitive (this is what Oxfam have done).

One can't help but think that there are more risks to the NGO's reputation than there are to children's lives. In fact, it's more likely that children's lives would be improved if aid agencies embraced transparency, rather than cowered from it.

Information wants to be free

Oxfam has broken ranks.

In response to my formal complaint that the NGO was breaching its 'Open Information Policy', Oxfam have sent me the details of every overseas project they funded in 2009-10.

They admitted there had "been a breakdown in our internal communications" and that the information should have been made available immediately. Thanks to Joss Saunders (Company Secretary) for doing the right thing!

The spreadsheet of all 1017 projects is now online. It contains the title, location and allocated funds for every project in 2009-10. I had to scan in the list (the originals are here) so some of the titles have come out a bit dodgy (but I typed in the expenditures so they should be kosher).

As far as I'm aware this is the first time a major British international development NGO has published their spending to this level of detail. It's a really exciting time for the open aid data movement.

Some of the projects have laughable names: "Social dialogue and pedogic strategy on racism in Guatemala [sic]" and "Promoting active citizenship for the Right to the City in Cochabamba". But let's not judge a project by its title. Instead we should encourage Oxfam to release more information about the purpose and evaluation of these projects so that they can be properly judged.

One interesting thing I've noticed is that the funding for the projects follows a log-normal distribution (see the chart here). So the small projects are tiny compared to the big ones. If we look more closely at the data we can see that Oxfam is active in about 60 countries, but in about half of these it spends less than a £1m (at that level of funding a country office can't do much more than maintain a physical presence). Wouldn't it be better if Oxfam concentrated on making a measurable difference in just a few places?

There's obviously a lot more analysis to be done. What's interesting about working with this data is that as soon as you have it you want to know more: 'what is this project?', 'why was so much spent on that?', 'were any of them effective?' Hopefully the pressure on Oxfam (and others) to publish aid data will grow - and not diminish - as a result of them taking this first tentative step towards greater transparency.

Share your failures on FAILfaire.

Déjà vu all over again

In 2008 I asked 8 large international development charities to provide a detailed breakdown of their spending that year. None were able to.

In July 2010 I repeated the exercise with the same charities to see if anything has improved. It hasn't.

Here's what I requested, for each overseas project funded in 2009-10: 1) the name of the project 2) the location of the project 3) the total expenditure on the project in 2009-10. You can read the original emails here.

• ActionAid - directed to Annual Report
• British Red Cross - directed to Annual Report
• Cafod - directed to Annual Report
• Care International - no response
• Christian Aid - directed to Annual Report
• Oxfam - directed to Annual Report
• Plan International - no response
• Save the Children - directed to Annual Report
• Tearfund - directed to Annual Report

(NB. None of the Annual Reports has the list of all funded projects that I was after)

All the organisations claim to support transparency. ActionAid, Plan, Save the Children and Oxfam are signatories to the INGO Accountability Charter. This states that "By signing this Charter we seek to promote further the values of transparency and accountability that we stand for, and commit our INGO to respecting its provisions." What's the point if they won't answer a simple query about spending?

Others have produced 'Open Information Policies' (lite versions of Freedom of Information) - ActionAid, Save the Children, Christian Aid and Oxfam all promise to release information in response to these sort of requests. But when a request is actually made, they balk.

So it seems that we are in an odd situation where all of the organisations agree with the transparency argument and understand its importance for donors, taxpayers, partner organisations and aid recipients. But they don't know how to change their organisations so that a standard request for information is responded to properly.

PS I'm going to submit formal complaints to the 4 organisations with Open Information Policies - I'll let you know how I get on!