Evidence checker tool
Authors: Prisca Benelli and Ehtisham Ul Hassan
Even though the breadth and quality of evidence on interventions in humanitarian crises is limited, as has been shown by, for instance, Blanchet et al, 2017 for health and Williamson et al, 2017 for child protection, there is a growing recognition of the need for evidence-based interventions to help improve the effectiveness and efficiency of humanitarian responses.
Humanitarian actors are becoming more and more aware of this, and devising strategies to meet the need. This includes work in Save the Children to create evidence-based “Common Approaches” and an “Evidence Checker tool” to spot check our use evidence in humanitarian programmes.
“Common Approaches” are comprehensive sets of tools and programmatic approaches that, with appropriate adaptation, can be replicated across a range of contexts and in different countries. By reviewing and evaluating existing evidence and pre-assembling sets of standard operating procedures based on evidence of what works, we free our teams in a variety of humanitarian responses from having to start from scratch when they approach a problem. This gives them more time for innovation and for finding ways to adapt, learn and refine solutions relevant to the context they find themselves in.
The seeds for the Evidence Checker tool were sown in 2015 when Save the Children UK decided to include the generation and promotion of reliable evidence in our 3-year strategy, as a way to improve the quality of our programmes. This required the development of a key performance indicator to measure our own generation of evidence, which revealed that, within an organisation whose main aim is the delivery of aid, doing new studies is perhaps less important than using evidence that already exists. It was decided to complement it with a measurement of the use of evidence in proposals and we devised a tool to assess this when developing aid programmes. More specifically, we wanted to measure how evidence is used to justify needs, analyse problems and design interventions in proposals submitted to donors, with a specific set of adaptations for humanitarian programmes.
This “Evidence Checker tool” was designed in consultation with both those who write proposals and those who implement projects. It was recently launched internally, and will facilitate the assessment of proposals by scoring their various elements. The cumulative score can then be used to determine whether the proposal contains sufficient use of evidence. Among the points considered are whether the needs assessment is robust enough to justify the problem analysis; whether the proposal cites evidence to justify the approach suggested (and, if so, whether the majority of external sources are properly cited and findable); and whether there is explicit discussion of limitations, uncertainties and contradictions in the evidence base. This discussion of the limitations of the external evidence base should also help us to improve behaviors and practices across our programmes.
We now plan to use the tool to assess a sample of proposals from different humanitarian responses and different sectors, to identify which sectors are making good use of available evidence and to target support to those which need to improve. Right now, in November 2018, the tool has just been approved so it’s too early to comment on its utility and effectiveness. However, given the importance of practicing what one preaches, we are pleased to be applying an evidence-based approach to measure the use of evidence!
If you’d like to know more, please contact Prisca Benelli at Save the Children.
Blanchet, Karl, Anita Ramesh, Severine Frison, Emily Warren, Mazeda Hossain, James Smith, Abigail Knight et al. “Evidence on public health interventions in humanitarian crises.” The Lancet (2017).
Williamson, Katharine Debbie Landis, Harry Shannon, Priya Gupta, Leigh-Anne Gillespie, “The Impact of Protection Interventions on Unaccompanied and Separated Children in Humanitarian Crises” Feinstein International Famine Center (2017)
About the authors:
Ehtisham Ul Hassan is currently the Deputy Head of Impact Innovation and Evidence Team at Save the Children UK. He leads work on implementing systematic approach of evidence and learning agenda within SCUK and across the movement.