Evidence in humanitarian emergencies: what does it look like?
By Jeremy Shoham and Marie McGrath, ENN
Over 20 years, ENN has published Field Exchange to help achieve our purpose of strengthening the evidence and know-how for effective nutrition interventions in countries prone to crisis and high levels of malnutrition. It provides the experiences of those implementing nutrition programmes in acute and protracted emergencies. Most of these experiences and lessons learnt would not have been captured or disseminated without this type of non-peer reviewed publication. Yet, for many, Field Exchange, and equivalent publications are the ‘grey’ literature, often seen as a source of findings that are ‘plausible’ but not evidenced. Hence the question, what is evidence and what does it look like – especially in humanitarian contexts?
The few reviews of evidence in humanitarian nutrition programming show that there is very little ‘probabilistic’ evidence out there. Randomised trials that are held up as the “gold standard” for assessing the effects of interventions are particularly difficult to mount in humanitarian contexts. They require foresight and investment by donors; early collaboration between pragmatic, creative academics and operational agencies; and long term commitment to plan, deliver and publish. And, many of them do not come cheap. Even where randomised trials are carried out (mostly in secure settings), they don’t necessarily tell us whether something will work or not in the complex environment and ‘ever shifting sands’ of an emergency or, critically, how it works. This is a key uncertainty for programmers looking to adapt and respond to the needs of specific populations and contexts.
There are also challenges around the global coordination of ‘robust’ research. Research agendas are not shared between research institutions and there is competition for scarce resources – even (shamefully) amongst research groups in the same research consortium. The culture around research is often ‘secretive’ and conflict of interests are not always apparent.
However, the fact that there is a gap in the evidence doesn’t stop programming. Agencies on the ground still need to respond, innovate and adapt, guided by what they know or suspect works, influenced by agency strengths, sometimes driven by donor interests and sadly, often underpinned by the bureaucratic imperative that the implementing agency must, itself, survive. The intention though is always worthy – to alleviate suffering.
On the positive side, ENN has witnessed (and, through Field Exchange, has captured and disseminated) innovation, programme development and learning. Seen as a non-governmental organisation with no vested interest other than to reflect learning, multiple agencies have shared programming experiences though Field Exchange, documenting perceived and measured successes and failures. This provides a collective memory and exchange – and evidence of sorts – of what works and what doesn’t work. The process of experience capture is cathartic for many. It helps them unpack and reflect, enabling both personal and agency reflection and learning. It has also helped identify where urgent ‘robust’ research is needed. The impact on programming and research of this collective experience has been substantial, not least on ENN’s research and reviews.
Should we aspire to more? Absolutely. We need more randomised trials, complemented by theories of change, to help explain how and when interventions are likely to impact nutrition. We also need institutional changes which allow for more of these studies to be done in challenging contexts. The current architecture makes such research very difficult. At the same time, we need to continue to capture the kind of ‘evidence’ provided by practitioners, which is critical to shining a light on programme performance, and to identify where greater institutional coherence and joined up thinking is needed.
A great example is the special edition of Field Exchange on the Syria nutrition response in 2014 that documented detailed case studies of more than 60 programmes. This single publication has had a significant impact on international guidance around infant feeding in emergencies, the need for policies on non-communicable diseases (NCDs) in emergencies and generated a considerable research focus on how to address high levels of stunting in protracted crises.
In conclusion, evidence is not just generated by academic researchers, statisticians and the like but also by those at the sharp end of programming. Many of those at the sharp end may well need to brush up on their epidemiology, just as many professional researchers may need to familiarise themselves with the complex circumstances of humanitarian crises and the unique insights of the implementers on how the programming they are intimately involved in is playing out on the ground.
About the authors
Jeremy Shoham is a Technical Director with ENN and a founding Director of the organisation. He is co-editor of ENNs publication, Field Exchange, with research interests in cash programming in emergencies and moderate acute malnutrition management. Additional areas of work include knowledge management support to the SUN Movement.
Marie McGrath is a Technical Director with ENN and co-editor of Field Exchange. Research interests within ENN are on management of acute malnutrition in infants under 6 months (MAMI). Additional areas of work in include guidance development on infant feeding in emergencies.
Links to ENN resources
Dolan C, McGrath M, Shoham J (2015). ENN’s perspective on the nutrition response in the Syria crisis. Field Exchange 48, November 2014. p2.
Navarro-Colorado C, Mason F, Shoham J (2008). Measuring the effectiveness of Supplementary Feeding Programmes in emergencies. HPN Network Paper. Number 63, September 2008.
Hall A, Blankson B, Shoham J (2011). The impact and effectiveness of emergency nutrition and nutrition-related interventions: a review of published evidence 2004-2010. Emergency Nutrition Network.
Grijalva-Eternod CS, Wells JCK, Cortina-Borja M, Salse-Ubach N, Tondeur M, Dolan C, Meziani C, Wilkinson C, Spiegel P, Seal AJ (2012). The double burden of obesity and malnutrition in a protracted emergency setting: a cross-sectional study of Western Sahara refugees. PLoS Medicine 9(10): e1001320.
Fenn B, Noura G, Sibson V, Dolan C, Shoham J, (2015). The role of unconditional cash transfers during a nutritional emergency in Maradi region, Niger: a pre-post intervention observational study. Public Health Nutrition 18(2): 343-51.
Angood C, McGrath M, Mehta S, Mwangome M, Lung’aho M, Roberfroid D, et al. (2015) Research priorities to improve the management of acute malnutrition in infants aged less than six months (MAMI). PLoS Medicine 12(4): e1001812.
Angood C, Khara T, Dolan C, Berkley JA, WaSt Technical Interest Group (2016). Research priorities on the relationship between wasting and stunting. PLoS ONE 11(5): e0153221.
Dolan, C., Khara, T., Mejia Acosta, A., & Shoham, J. Government experiences of scale-up of community-based management of acute malnutrition (CMAM): A synthesis of lessons. (2012).
Khara T, Dolan C (2014). Technical Briefing Paper: The relationship between wasting and stunting: policy, programming and research implications.
Shoham J, Dolan C, Gostelow L (2013). The management of acute malnutrition at scale: A review of donor and financing arrangements.