Ideas into Impact, Ep. 02: Data you can act on
The second episode examines the distance between collecting data and being able to act on it, and argues that the problem is usually the question rather than the data. The series is in production.
Institutions in Somalia and across the region collect a great deal of data. Programmes run baseline surveys, monitoring systems record activity continuously, and ministries hold administrative records that nobody has looked at. Very little of it changes a decision. The second episode of the series takes up why, and argues that the constraint is rarely collection capacity. It is that the data was gathered without a decision in mind, so there is no decision it can settle.
What the episode discusses
The conversation covers the practical craft: designing an instrument backwards from the decision it is meant to inform, deciding in advance what result would change the answer, and being explicit about what a sample can and cannot support. It deals with the specific difficulties of field research in this environment, where the sample that can be reached is shaped by access and security rather than by the sampling frame, and where the honest response is to publish the difference rather than present the achieved sample as the intended one.
The episode covers instrument design that starts from the decision, and the discipline of publishing what a sample cannot support.
It also takes up administrative data, which is the largest underused resource most institutions hold. Registration records, payment logs, clinic and school registers and customs entries are generated as a by-product of operations, cost nothing further to obtain, and are usually stored in a form nobody can query. The episode discusses what it takes to make that material usable: agreeing definitions across the offices that generate it, cleaning it once rather than repeatedly, and deciding who is responsible for it when the programme that created it ends.
The argument
The case made in the episode is that defensibility comes from provenance and stated limits, not from volume or precision. A leader presenting a figure needs to be able to say where it came from, when it was collected, who collected it and what it excludes. A number that cannot answer those questions will not survive its first challenge in a public meeting, however carefully it was calculated.
The episode also argues against a habit common in programme reporting: the confident single figure produced from thin evidence because a template required a number in the box. A range with its assumptions stated is more useful to a decision-maker than a point estimate that cannot be interrogated, and it is more honest about what was actually established. The cost of the alternative is cumulative. Once a figure has been repeated three times it acquires a life independent of its source, and correcting it later is treated as a controversy rather than as ordinary practice.
The conversation ends on the discipline that makes the rest possible: writing down, before collection begins, what decision the data is for and what result would change it. Teams that do this find they need less data than they thought. Teams that do not tend to collect a great deal and use none of it.
Who it is for
The episode is aimed at monitoring and evaluation staff, ministry planning and statistics officials, programme managers who commission research and have to use it, and the analysts who present figures to people who will be held to them.
Status
The Veynuus podcast is in production. No episode has been released, and the description above sets out the intended content of the second episode rather than a summary of a recorded conversation. To be notified when the series launches, or to propose a guest or a subject, write to [email protected].