Interoperability Judo in the Aid Sector
Photo Courtesy of Yayasan Damai Olahraga Bali
Ten years ago I nearly set an Italian hotel on fire. I’d plugged an American fan into a European electrical socket, and after about 30 seconds I had a shower of sparks landing on the curtains. What I’d forgotten to account for, of course, was the difference between the 120 volt standard that the fan was expecting and the 240 volts that the outlet was producing. Just because the connection worked physically didn’t mean it would work practically. Contrast this with last month, when I’d brought my laptop and charger on a trip to India but again forgot a power converter. Thankfully, Apple has an elegant solution to the problem of different electrical standards. Rather than trying to convince every country to use the same voltage in its wall sockets, they’ve just built a charger that can handle a range of inputs. They accept the complexity that happens when large groups of people try to collaborate and work with it, not against it. They’ve taken an obstacle and turned it into an opportunity. It’s design judo.
When it comes to financial flows in the aid sector, standards are more complicated than deciding what plug to use. With so many governments, organizations, and companies sending billions of dollars to support global development, communicating the details of these relationships and transactions in a shared framework becomes a herculean task. The International Aid Transparency Initiative (IATI) has made progress in establishing a standard for the sector to describe funding and implementing relationships consistently. The list of 480+ entities that publish IATI-compliant data understates the standard’s reach. Most of the 29 member countries of the Development Assistance Committee (DAC) report IATI data about their aid spending, and the funds sent by these governments represent about 95% of total DAC expenditures. It’s hard to estimate an exact number, but it’s safe to say that the IATI standard describes a significant majority of the world’s aid dollars.
Still, there are some challenges to using IATI-compliant data to get a precise understanding of how the aid sector is actually organized. Despite IATI’s thoroughness, organizations still interpret the requirements differently, leading to the same data fields containing multiple types of information. This can make seemingly simple tasks, like identifying a unique organization consistently, very difficult in practice. Similarly, there aren’t strict validations or requirements preventing organizations from omitting data or inadvertently hiding important outcome data in a pages-long list of transactions. Organizations that don’t share their data are left out entirely, even if they’re mentioned frequently by organizations that do report. All this can make it hard for aid professionals like funders, program implementers, or researchers to extract useful conclusions from IATI data.
So what should the sector do about this? One approach might be to double-down on the rules associated with our data standards and try to force everyone to provide clear, accessible, and organized data. This would be similar to convincing all countries to share the same voltage standard; it’s not a practical option. The alternative is the judo method: work with the challenges inherent in the IATI standard instead of trying to regulate them away. Some friends and I recently tried to do just that as our capstone project for the UC Berkeley Masters of Information and Data Science degree.
The end result is AidSight, a platform that provides easy-to-use tools for the aid sector to search IATI data, explore relationships between organizations (including those that don’t report their data directly), and validate the likely usefulness of their results. For example, imagine you’re an aid agency that needs to report on the current state of the water sector in Ghana. First, AidSight enables you to query all IATI data in plain english instead of a complex requiring search interface or a code-heavy API call. Your results appear as network diagram that maps the relationships between the organizations that meet your search criteria, whether they report to the IATI standard or not. Here’s our result for the Ghanian water sector – note that we’re mapping the just the organizations and relationships, not their real-world locations or relative sizes:
The green dots represent organizations that report data to IATI directly, the red dots are organizations that are implied in the data that other organizations report, and the width of the lines connecting them indicates the strength of the relationship. This approach takes the data reported by 484 organizations and turns it into results for tens of thousands. In this example, there are two “hubs” of reporting organizations on the right side of the map that work with 5-7 non-reporting organizations at varying levels of connection. In contrast, there’s another hub organization (GlobalGiving itself) towards the bottom left that works with many more organizations, but in the same way with all of them. Using this method, users are quickly able to spot the key players in any sector and explore the strength of their collaborations instantly.
Understanding these connections is important, but what if the report needs more granular results? Before downloading and analyzing the raw data, you’d want to know if you’re likely to be able to draw meaningful conclusions from the results we’ve found. To make this easy, AidSight contains a data quality dashboard that uses heuristics to estimate how useful each organization’s data is likely to be and summarizes it with a simple letter grade.
Now, anyone at an aid agency can measure IATI data quality with a few clicks and save their data science teams to focus on only the most useful datasets. We can also use this approach to establish valuable benchmarks for the aid sector as a whole. The average grade of C- suggests that there’s lots to be done to improve the quality of development data reporting, but having a framework to measure progress makes it possible to consider how we might get there.
Currently, AidSight is a minimum viable product, so there are many improvements to make. Still, solutions that focus on data interoperability without trying to fight the natural complexity of the aid sector represent exciting opportunities for us to bring enhanced accessibility and understanding to our work in a democratic way. Taking the judo approach to development data means that a growing number of inventive, creative, and driven users will be able to discover new solutions to the aid world’s challenges.
Special thanks to the other members of the AidSight team: Natarajan Krishnaswami, Minhchau Dang, and Glenn Dunmire, as well as Marc Maxmeister for his feedback on this work. Explore IATI data yourself at aidsight.org or download the open source code on Github.
About Author
