on how companies should think about fairness when engaging in measurement, learning, and evaluation (MLE) practices.
In an ideal situation, MLE can help generate social impact and challenge deeply held and outdated core beliefs, helping organizations stay agile and adapt to new challenges so they can align with key results that serve their mission statements. But data itself has limitations: without looking at evaluation work through an equity lens, we run the risk of using the information we glean to accidentally reinforce systemic and structural barriers that already exist. and limiting opportunities for marginalized communities.
In the aftermath of the event, NationSwell asked the three panelists – Dr. Albertina Lopez from the Center for Evaluation Innovation, Dr. Daniela Pineda from Informed Insight and Dr. Amber Banks from the Center for Trust and Transformation – to dive deeper into how organizations can meaningfully track changes at a systemic level and build trust within their communities through compassionate and equitable MLE practices.
Here are some key insights from the panelists:
NationSwell: Why do you think MLE is important for change work?
Dr. Albertina Lopez, Senior Associate at the Center for Evaluation Innovation: Evaluation is a problem-solving tool, and problem-solving is necessary for change work. Assessment – with the accompanying quantitative and qualitative measures – helps us learn systematically and intentionally rather than defaulting to our brain’s natural and biased way of working, which is usually to provide information aligned with our existing beliefs and beliefs. rejecting what isn’t – things like confirmation bias. Evaluation can help us see a broader set of strategies so that we can select those that meet the demands of justice. We can use evaluation to unpack the answers to questions such as, “How well does our work align with how the community defines problems and solutions?” to help us plan forward-looking strategic questions such as: “How can we dedicate more resources in 2022 to the racial and gender justice work that communities are doing in the places we touch?”
Dr. Amber Banks, Founder and CEO of the Center for Trust and Transformation: Reflective practices are essential to social change. The essence of measurement, learning and evaluation activities is to create space to understand if we are progressing towards our goals. Learning and evaluation can help changemakers determine whether their efforts are having the desired impact and adjust course to accelerate or strengthen impact. Justice also demands an evolution in the field/practice of evaluation. Assessment must also move towards asset-based frameworks that honor the truth and abundance of those most affected by injustices and not centered on those already in power. Accountability centered on the responsibility of institutions to move towards more liberating practices is essential.
Dr. Daniela Pineda, Founder and CEO of Informed Insights: Whether you’re working on a data dashboard, facilitating a learning session, or conducting an assessment, MLE tools and practices are powerful resources to help us hold each other accountable. They clarify issues and help define what success looks like.
When we are ‘doing the work’, taking the time to collect data or participate in an evaluation may seem like a luxury we don’t have. But I invite you to think again. MLE tools can be used to tell our stories, quantify and describe systemic inequalities, and describe success. We cannot afford to do without these activities. Being able to name the changes we seek is powerful because it helps focus on what matters.
NS: What does it look like to practice MLE in a way that serves fairness and justice?
LOPEZ: At the Center for Evaluation Innovation, we partner with philanthropy in strategy, learning, and evaluation efforts to advance racial equity and justice. We largely work with people who advance policy and system change, and therefore practice using evaluation to challenge power and help shift it to those who have been historically oppressed by structural racism. In practice, this is like applying the principles of the Fair Assessment Framework™ in how we plan and implement work and using the Empowerment Framework to refine what we assess.
BANKS: When we think of MLE, we think of processes and methodologies, but the most important part of learning and assessment is relationships. Centered relationships can provide a foundation for listening, co-creating, shared ownership, and building trust. It is also important to honor the negative experiences that many communities/individuals have had, as evaluation is an extractive and punitive process. The intentional design and implementation of humanizing approaches rooted in asset-based frameworks and creating opportunities for multiple voices and perspectives is essential for equity and justice. Ultimately, generative learning is built on trust and the ability to be vulnerable to what works and what doesn’t. It also requires recognizing that assessment/data has historically been used to harm communities of color and communities affected by poverty. Serving equity and justice means reversing power imbalances in evaluation and expanding the invitation for co-creation, multiple truths and reorientation around the purpose and value of learning for impact .
PINEDA: The good news here is that there is no single manual or toolkit that will tell you how to “do” a fair MLE. And the even better news is that there are so many entry points for each of us to use MEL tools and practices in the service of equity and justice.
In my practice, I have had the privilege of working with organizations that are taking steps to make their evaluation practice more equitable. Centering equity and justice is about our values as practitioners, how we partner with others, and what we want to accomplish with these partnerships. I’ve partnered with organizations that have gone back to the drawing board to discuss what they mean by community engagement. Serving equity in these conversations was as much about being realistic about the kind of input they sought from community partners as it was about addressing their internal barriers to sharing control with community members. As a learning partner, I have designed processes that consider power dynamics, bring together input from different types of experts, and illuminate how the myth of objectivity can privilege methods of research that, at worst, reifies structural inequalities. All of this work is ongoing because serving equity is not like flipping a switch; it is a practice.
NS: What is your vision for the future of MLE?
LOPEZ: My vision is justice. Justice is why we measure, evaluate and learn, and how we work. What does it look like to have justice as both a principle and a standard? First, we must know what justice is. The Oxford Dictionary states that it is the “maintenance of what is right or just by the exercise of authority or power; the awarding of a deserved reward or punishment; give due merit. So, if we want to serve justice, we must know what power we must wield to meet its demands. We can look at our power positionally (eg, what influence does your organizational role give you?) and personally (eg, what skills and relationships do you have?). Once we have an idea of what our power is, we can ask ourselves when to use it and how to do it in the complex and unique situations that we encounter daily as agents of change. Love is the way. Justice requires love, as I learned from the influential moral philosopher Paul Tillich in his book “Love, Power and Justice”. It means that we use our positional and personal power with love, as Bell Hooks proclaims in his book “All About Love: New Visions”: “…mix[ing] various ingredients – attention, affection, recognition, respect, commitment and trust, as well as honest and open communication. In our future, MLE serves justice and we love in our power.
BANKS: My vision is that we will center learning on assessment. This learning will value stories and not just quantitative data. Numbers only tell part of the story and transformative change requires embracing stories, complexity and tension. We will honor that the narratives included in assessments belong to the communities served and that deficit-based narratives are a form of violence against communities of color and those affected by poverty that perpetuate inequalities. We will focus on how evaluators and others in positions of power relate to community partners and make it clear why we are collecting data, what the data will be used for and who will define success. Ultimately, “assessment” as we know it will be driven by those closest to the job and centered on asset-based learning, reflection, and frameworks.
PINEDA: The future is bright. In the past few years alone, we have seen a huge shift in the field as more and more changemakers demand and expect their MLE partners to center equity in their work. I also see more colleagues working to rethink the way they work. I take it for granted that all researchers own their biases and have abandoned the myth of objectivity. In the near future, I want MLE practitioners to redefine what it means to conduct rigorous research and evaluation; privileging marginalized voices; and to value the knowledge of those who are hardest hit by structural inequalities over those who have more privilege and hold power.