Sometimes researching the connections between agriculture, food systems, nutrition and health feels like trying to untangle a ball of string which has been left with a kitten for too long. Just when you think you are beginning to sort out one area, another knot of factors comes up demanding attention. This is partly because the underlying determinants of nutrition and health outcomes are rooted across many aspects and interactions of life. There is not a taut line between food production and those outcomes. Complex connections are rarely simple to measure, and rarely have simple solutions.
In the Innovative Methods and Metrics for Agriculture and Nutrition Actions (IMMANA) Programme, we focus not just on the research questions at the heart of these complex pathways, but the methods by which we ascertain answers to those questions. Since our ‘ball of string’ runs across disciplines, subject matter, and purpose, inventive and novel approaches are often required. IMMANA aims to promote thinking and application along these lines.
When the programme was being considered for another five years of funding, we were asked to take stock of the field, where we had contributed, and what the most promising areas of growth might be if we were to continue. Since we are researchers at heart, and since we work in such a cross-cutting field, we needed to apply a similar level of innovation just to approach answering this query.
We chose to undertake an Evidence and Gap Map (EGM) of research tools, metrics and methods used in agriculture, nutrition and health (ANH) to address our aims. Evidence and Gap Maps are tools developed through systematic search strategies to capture a comprehensive set of evidence that exists or doesn’t exist, mapped against a conceptual framework. Our manuscript will explain and discuss our findings, and in tandem with the publication I reflect on this particular exercise in evidence synthesis, and evidence synthesis in general.
Finding a theoretical grounding
Figure 1. Agriculture, Nutrition and health Rube Goldberg machine.
First of all, we had to adapt the traditional fixed framework used to map effectiveness studies into categories applicable to metrics and methods. After lots of playing around, we used established theoretical pathways – the many-arrowed diagrams created by experts to identify and situate the multiple factors at play determining the causal route between agriculture, food systems, nutrition and health. We took these theoretical Rube Goldberg machines and flattened and compressed them into twelve measly categories (Figure 1). Through our expert consultation, we found that everyone with a stake in the game (especially our advisory group) wanted something different – animal source food experts wanted to disaggregate from plant food production; the gender specialists wanted a separate gender row, and so on. But when we tried to map onto increasingly narrow and niche categories (or those inherently cross-cutting ones), every reported methodological innovation seemed to fit in every box, or too many of them, making it impossible to differentiate categories. By choosing only twelve, we found a way to visualise interdisciplinarity without losing the forest for the trees.
Figure 2. Heatmap of number of reports by thematic domain (rows), against innovations in types of tools, metrics, and methods (columns). Supplemental Table 2 is a list of the number of reports for each code, and Supplemental Table 3 is a list of unique tools, metrics, and methods represented by reports.
Secondly, innovation, while itself acts as a catch-all buzzword, is almost impossible to define. We focused on the last decade of research (since 2008, ten years prior to our search date and corresponding to the global food price crisis). This approach assumes that newness is a proxy for innovation. To be fair, during this period there was a strategic shift and explicit call for different ways to think about how the agriculture and food systems shaped nutrition outcomes, especially in light of slow progress on improving child growth, micronutrient status and protecting the vulnerable from shocks. This led to several waves of investment and renewed focus that spurred advancement in the field.
But is borrowing from another field - for instance, applying a type of network analysis from the social sciences to nutrition programme analysis - “innovative” in itself? We thought so, but it turns out that to identify these advances systematically is… difficult. We did our best by defining innovation in three ways:
- something that is completely new;
- something with significant modifications or revisions since 2008; and
- novel applications of tools, metrics and methods from other disciplines applied to ANH research.
We stuck to these criteria as best we could discern (what is ‘significant revision’ anyway?), but in doing so the team basically had to become experts in everything in order to screen reports – and we had over 50,000 of them. The nuances of this decision warrant a longer conversation over a cuppa.
The nature of evidence synthesis
A third and final thought is about the nature of evidence synthesis itself, and how this project fits in. Conventional systematic reviews that focus on a clear question and answer (usually around causality) have their place. You can be sure reports aren’t cherry-picked, they follow a strict process and appraise the evidence, and they save you from digging for and reading many, many articles on the same topic. However, they are often very narrow, and almost always conclude with the trope: ‘more, better evidence is needed’. Their necessarily narrow scope also doesn’t allow for a holistic or user-driven sense of what’s out there.
In this regard, EGMs are great – although they don’t seek to tell you what the evidence concludes overall - they are systematic, but much broader, and present a final product that is interactive and customisable for end users. For example, in our EGM - all those cross-cutting themes and characteristics, such as a focus on aquaculture, gender, children, climate, participatory methods, or technology like wearable cameras were coded and made into selectable filters for each user to include or exclude. That means that when the main stakeholder question is: What have you contributed to the field, and where should you contribute in the future?, we were able to map the field, layer onto it contributions arising from or linked to our activities, and identify where there was an absence of work done.
In this sense, we hope to have provided a resource that goes beyond our own research question, which can be used to strategise across research, programme implementation and policy-making. It certainly isn’t a genie that will tell you an answer to a specific question. We’ve learnt that new things aren’t always better – there is merit and strategy from borrowing and applying rather than constantly seeking to reinvent the wheel. We ask those applying for IMMANA grants to discuss and situate their proposal in the context of the EGM, hoping that it will lead to more impactful work. More and more, we see these mapping exercises as a tool to visualise and digest an interconnected area of research so that we can walk the talk of resource-conscious, informed and evidence-based decision-making at every level.