Monitoring and Evaluation in Health and Social Development Interpretive and Ethnographic Perspectives
Edited by Stephen Bell and Peter Aggleton
The Reality Check Approach has proven to be a powerful companion to quantitative research in the design and evaluation of programs. In Chapter 12 of the new publication Monitoring and Evaluation in Health and Social Development: Interpretive and Ethnographic Perspectives, RCA practitioner Dee Jupp discusses examples from Ghana and Nepal where RCA studies added invaluable insights to quantitative research which may have otherwise gone unnoticed or inadequately interpreted. In Nepal for example, a RCA study preceded the evaluation of a DfID-supported Rural Access Programme to help develop a theory of change constructed from people’s perspectives and to enhance the design of household surveys. Whereas the initial project theory of change believed that greater access would decrease outward migration, the RCA study showed the exact opposite - people perceived that outward migration would increase. To assist with the households survey, the RCA study noted that many people were using private providers for healthcare, which had not been initially included as a response option in the surveys.
In Ghana, RCA is being used in a mixed method process as part of the Millennium Village Project. The RCA study helped substantiate baseline survey data which indicated that food expenditure was increasing at the same level as increases in total expenditure. Typically, food expenditure decreases as a proportion of total expenditure as expenditure increases, so the survey team was ready to dismiss this as a measurement error prior to listening to the RCA experience in the field. It also helped to explain why the incidence of malaria remained high despite nearly 90 percent of households surveyed as having at least one mosquito net. Basically, the nets were unused for most of the year because it was too hot to sleep with them and many people did not actually understand the connection between using the nets and malaria prevention.
The Reality Check Approach was also employed as part of a multiple methods evaluation looking at 40 years of development interventions in East Nepal. While existing government statistics still had many areas of the Koshi hills area of East Nepal classified as poor, the RCA study found that this was incorrect, and that significant change and development had occurred in this area. The RCA study uncovered that the introduction of cardamom cultivation through people’s own efforts rather than external development programmes was, in fact, the main driver that had turned this region from a cash poor area to relatively cash rich.
Methodological Considerations in Evaluating Long-Term Systems Change
Written by Andrew Koleros, Dee Jupp, Sean Kirwan, Meeta S. Pradhan, Pushkar K. Pradhan, David Seddon, and Ansu Tumbahangfe
Published by American Journal of Evaluation 1-17 aje.sagepub.com
RCA participated in a multiple methods evaluation study to look at the long-term impact of development interventions in the Koshi Hills region of eastern Nepal. Evaluations typically assess the linear linkages between specific interventions and intended program outcomes, but this is likely to attribute any changes to a particular program(s) while obscuring the wider social and economic factors contributing to changes. This evaluation hoped to limit the normative assumptions that come from a project focus by using multiple methods and by focusing on a much longer period of time not tied to any individual project. With the help of insights gathered from the RCA study in the Koshi Hills region, the evaluation narrative was based around the perspective of the people in that area (their Theory of Change). This helped to limit normative assumptions while adding rigour and explanation to the findings. The RCA study also helped uncover a key driver of significant change in the Koshi Hills region which wasn’t reflected in existing data, namely the introduction of cardamom cultivation. This was a particularly important discovery because the cardamom was introduced to the region through people’s own efforts rather than through external development programmes.
In this article in the American Journal of Evaluation, the evaluation team, including RCA practitioners Dee Jupp and Ansu Tumbahangfe, discuss the methods used in this unique long-term evaluation along with the lessons learned and implications for future studies.
Us in Their Shoes: 27 stories everyday lives in Indonesia
Written by RCA team, 2015
Usually, the Reality Check Approach (RCA) team writes up their field experiences from particular RCA studies in conventional report format. However, the team felt that over this last year, they had much more to tell about the everyday lives of the families who allowed us to stay with them for several days and nights than these reports could contain. ‘Us in their shoes’ is an anthology of stories of just a few of the more than 150 ordinary families across Indonesia with whom the RCA team has stayed this year. The stories are put together from the personal diaries made by the RCA team members during RCA thematic studies. They reflect the closeness achieved in the relationship between the families and the researchers and highlight the insights gathered by actively joining in their everyday lives. Each one of the stories has been written because the author felt moved by their ‘family’- their optimism, their wisdom, their humour, their warmth and sometimes their sadness. They are stories of resilience, endurance and aspirations but also ordinary human shortcomings and foibles. The intention is to share some of the multiple and contemporary perspectives of people living in poverty, or as most of them prefer to say, living simply.