Monitoring and Evaluation – CARE Gender Toolkit

Gender Monitoring, Evaluation and Impact Measurement

Tools selection must be rooted firmly in the research framework and key questions for the study. For guidance on design and planning, see pages in the "Preparation" section. This page provides 3 sections on gender monitoring, evaluation, assessment and learning:

Guidance on Measuring Gender-Transformative Change
Tools for Measuring Change in Gender Relations and Women's Empowerment / Gender Transformative measurement, evaluation, and learning systems
Indicators for Gender-Transformative Change

1. Guidance on Measuring Gender-Transformative Change

Understanding change in women’s experiences and views toward empowerment is complex and can present us with a number of challenges. Some key challenges in measuring women's empowerment are highlighted in greater detail within:

Applying feminist principles to program monitoring, evaluation, accountability and learning. This paper shares reflections on how to apply feminist principles to monitoring, evaluation, accountability and learning (MEAL) practice
A practical guide to measuring women’s and girls’ empowerment in impact evaluations: Pulling insights from diverse disciplines and the experience of J-PAL affiliated researchers around the world, this guide offers practical tips for capturing changes in empowerment, that are tailored to the local context, and that minimize reporting bias.
Measuring gender-transformative change: A review of literature and promising practices.This paper from CARE, World Fish, and the CGIAR Research Program on Aquatic Agricultural Systems explores ways to measure gender transformative change. Gender-transformative approaches aim to move beyond individual self-improvement among women and toward transforming the power dynamics and structures that serve to reinforce gendered inequalities. As defined by the CGIAR Research Program on Aquatic Agricultural Systems (AAS), a gender-transformative approach to development goes beyond the “symptoms” of gender inequality to address “the social norms, attitudes, behaviors, and social systems that underlie them” (AAS 2012, 3). This approach entails engaging groups in critically examining, challenging and questioning gender norms and power relations (Rottach et al. 2009) that underlie visible gender gaps.
Strategic Impact Inquiry on Women's Empowerment Global Research Framework - Describes how CARE approached impact analysis on women's empowerment and outlines a framework for analysis as well as important considerations for research on women's empowerment.
Social norms measurement page - tools and resources on measuring social norms change.

Lessons in Measuring Empowerment

These papers highlight the need to:

Work more reflectively to try to understand complex change. What else is happening in the broader environment ?
Consider what women’s empowerment might look like in various contexts over time.
Ask not only what has changed, but how have things changed? For whom (and at what level)? How has the change been perceived or experienced by diverse groups? Who led the change, and who feels ownership over shifting norms?
Measure changes in agency, structures and relations: How have changes in gender relations affected structures and relationships among and between various groups?
Situate changes in the broader historical, social and political concept of groups.
Consider dimensions of change that are not so easy to see or count. Monitoring tends to focus on change that can be easily measured, such as economic change. Social and political changes are harder to discern, and psychosocial dimensions of change are under-represented. How have women's and men's aspirations been changing? Or sense of confidence and capability? Or recognition from others?
How have changes taken place compared to previous conditions and relationships? How has a group's status changed relative to others?
Acknowledge diversity among different groups, and how interventions affect multiple aspects of a person's life (see the Good Practices Framework for Gender Analysis)

To measure changes in gender equality and women's empowerment, we must be aware of what is measured, who measures it, and how. Situating changes within this broader context and perspective, we can see a qualitative difference between interventions that show we are making the world more fundamentally equal.

Why Measure Change

Before thinking about HOW we measure change, we must be clear on WHY we measure changes in gender relations and women's empowerment. Monitoring or exploring change for CARE is important to provide:

Accountability - to communities, impact groups, donors, partners and other stakeholders to assess our work. Monitoring is critical to ensure interventions Do No Harm.
Shaping Programming - monitoring change can help CARE understand what it takes to advance our mission and vision, and identify the best role for CARE to contribute to social justice.
Enhancing Program Quality - beyond shaping the direction and form of CARE's work, monitoring and assessment also improve the quality of programming by ensuring that lessons learned from one intervention can influence others, and by identifying key promising practice models for gender equality.
Informing Advocacy - to promote broader change and influence policy, distribution of resources and other key decisions that affect the lives of communities, and the status of groups with whom we work.

Within CARE, we also have the responsibility to ensure that any analysis reflects organizational principles and ethics. Individuals, communities and key stakeholders can mutually analyze findings and develop concrete actions to advance gender equity and equal human rights for women and men.

2. Tools for Gender-Transformative Measurement, Evaluation, and Learning Systems

A myriad of approaches can complement one another in studies looking to understand what changes have occurred and why over the life of a program or project.

Organizational Knowledge

Measuring gender-transformative change indicates adopting approaches and systems that take a critical examination of an institution's practices, which helps an institution to identify how its own research practices can contribute to (or are currently impeding) empowerment and broader social change processes. A number of approaches look at the knowledge through documentation and staff experiences to understand change:

Social Analysis

To understand people’s own perspectives on the changes that have occurred in their lives since participation in a project, and their experiences and interpretations of these changes, teams can engage methods in analyzing the context and understanding women's empowerment - how communities define it, as well as where women situate themselves in relation to it - across time.

In addition, this section highlights further exercises with communities to gain their perspectives and analyze change.

Comparison across Groups and Time

As highlighted throughout this collection, the work of CARE is only one of multiple factors influencing women’s lives.

To extract how CARE’s work has contributed to changes in women's lives, some teams used as a basis for comparison across different groups or time periods, to illustrate CARE's contributions to change:

Anthropometric and Health Status Measures
Focus Group Discussion
Field Observation
Knowledge and Achievement Tests
Interviews / Surveys
Ranking exercises
Analysis of Existing Data

Situational Analysis Tools

To understand change, teams have also repeated situational analysis tools before, during and after a project cycle to track changes in gender relations.

3. Indicators and Discussion on Women's Empowerment Impact Measurement

The Gender Equality Women’s Voice (GEWV) indicators were developed with the aim of allowing CARE to capture, measure, and track the changes occurring as a result of the dedicated gender approach across development and humanitarian programming. The resulting data should enable CARE to systematically build a picture of the changes in agency, structures, and relations taking place in the communities in which we work. The aim of doing so is to:

Understand the results of taking a gender approach within the complex process of gendered change;
Test our assumptions and theory of change, and measure progress against our gender aims;
Capture and tell the story of gender transformation;
Learn how to better improve our work to address inequalities.

CARE Gender Equality and Women's Voice Measurement Resources

External Women's Empowerment Measurement Frameworks and Guidance:

AWID (2019). Feminist Realities: Our Power in Action. An Exploratory Toolkit.
S Batliwala (2006). “Measuring Social Change: Assumptions, Myths and Realities.” Alliance Magazine, 11(1): pp.12-14.
Srilatha Batliwala and Alexandra Pittman (2011). Capturing Change in Women's Realities.
D Jupp, S Ibn Ali and C Barahona (2010). Measuring Empowerment? Ask Them. Sida studies in Evaluation.
Naila Kabeer (1999).The Conditions and Consequences of Choice: Reflections on the Measurement of Women's Empowerment.
A Malholtra, SR Schuler and C Boender (2002). Measuring Women's Empowerment as a Variable of International Development. World Bank.
USAID/East Africa (2009). Violence against Women and Girls: a Compendium of monitoring and evaluation indicators.

References

From the SII:

CARE India (unpublished). Strategic Impact Inquiry 2005 Annex of Tools.
E Martinez (2008). Heart of the Challenge: Women’s Empowerment Global Final Synthesis Presentation. Available at the Overview page of CARE’s Women’s Empowerment Strategic Impact Inquiry Library.

Methods Compendium:

CARE USA (2008). Gender, Sex and the Power to Survive: the impact and implications of empowering women at risk of HIV/AIDS: A Global Research Design Framework Appendices.
CARE (2007). Promising Practices Inquiry on Women’s Empowerment: Resources.
K de Boodt (2007). Empowerment Approaches for Understanding Empowerment: Learning from Practice – Appreciative Inquiry (Dialogue Valorisant) for Positive Change. CARE Burundi.
V Vaughn (2007). Promising Practices Inquiry on Women’s Empowerment: A Practical Guide to the Process. CARE USA.
CARE Mali (2005). Mali SII Tools.
K Glenzer and J Rugh (2005). Women’s Empowerment Strategic Impact Inquiry Meta-Evaluation Analytic Guide. CARE USA.
K Glenzer (2005). Starter Kit for Conducting a Desk Review. CARE USA.
M Drinkwater (2004). Reflective Practice. CARE.
M Drinkwater, A Singh, G Hora (2004). Change from the Heart: Unlocking the Potential – Report on the Chhattisgarh Reflective Practice Exercise Summary. CARE.
CARE Burundi (2006). La Piste du Voyage: L’Approche valorisante synthese de methodologie.

Regional Pages:

CARE Ecuador (2007). Experiential Methodologies: a proposal for developing qualitative research.
CARE Niger (2006). Strategic Impact Inquiry Research CARE Niger: Meta-Evaluation of a decade of VSL.
D Koenig (2006) Meta-Evaluation: Mata Masu Dubara Projects in CARE Niger. CARE Niger.
CARE Mali (2005). Strategic Impact Inquiry: Some Lessons Learned.
SII Global Advisory Committee (2005) CARE Niger Strategic Impact Inquiry Phase I: Preliminary Plans from CARE SII Global Advisory Committee.

Other Resources

J Buter (2008). Reflexion sur le genre et la diversité sur le lieu de travail à CARE Burundi. MDF Training and Consultancy.
R Davies and J Dart (2004). The Most Significant Change Technique: A Guide to Its Use. Available at the Rick’s Methods page of Monitoring and Evaluation News.
Price and K Hawkins (2004). “Researching Sexual and Reproductive Behavior: A Peer Ethnographic Approach.” Social Science & Medicine, 55(2002): pp. 1325-1336.
Options and the University of Swansea Center for Development Studies (undated). The PEER Method.
P Stephenson with S Gourley and G Miles (2004). Resourcing Organisations with Opportunities for Transformation and Sharing: Child Participation. Teddington: Tearfund.
G Ashford and S Patkar (2001). The Positive Path: Using Appreciative Inquiry in Rural Indian Communities. The International Institute for Sustainable Development (IISD).
W Knightly, D Whitelock and J Colder (2001) ‘Education and its effects on Self Esteem: Monitoring mature women students’ movement along this measure.’ The Gender and Education 3rd International Conference, The Centre for Research and Education on Gender, Institute of Education London, UK pp.293-301.
D VanBelle-Prouty and H Sey (1998). Girls' Participatory Learning Activities in the Classroom Environment (GirlsPLACE) A View to the Experiences of Girls Institute for International Research.
C Wang and M Burris (1996). “Photovoice as a Tool for Participatory Evaluation: The Community's View of Process and Impact.” Journal of Contemporary Health, 4: 47-49.

From the SII Library:

Methods Compendium:

T Klouda (2008). Links between ‘Empowerment’ and HIV Risk. CARE USA.
CARE (2007). Gender, Sex and the Power to Survive: the impact and implications of empowering women at risk of HIV and AIDS, Global Research Design Framework.
F Wamara (2006). Strategic Impact Inquiry Quantitative Field Survey: Magu District’s VSLA Groups Field Completion Report. CARE International – Tanzania.
F Maiga (2006). Etude SII – Questionnaire Individuel Femme. CARE International – Mali.

Regional Pages:

CARE Ethiopia (2006). Strategic Impact Inquiry on Women’s Empowerment in Relation to FGC Elimination Project in Awash.
CARE India (2006). Strategic Impact Inquiry: Sustainable Tribal Empowerment Project.
N Kanji, B Bode and A Haq (2006): Strategic Impact Inquiry: CARE Bangladesh Nijeder Janyia Nijera.

External Resources

M Bamberger, J Rugh and L Mabry (2006). RealWorld Evaluation. Thousand Oaks, CA: Sage Publications.
R Chambers (2005). Participation and Numbers. Q-Squared Working Paper, No. 13. Toronto: Center for International Studies.
Caldwell, Sprechman and Rugh (1997). DME Workshop Facilitator's Manual.
C Robson (1993). Real World Research: a resource for social scientists and practitioner-researchers. Maiden, MA: Blackwell Publishing