Integrating DEI into Evaluation

When I began my own consulting firm, I was excited to fully claim antiracism as a value and equity as the lens for my work. I wondered, What projects might I support that help create stronger communities for all? Participating in evaluation work was not on the list. As someone who has primarily focused on strategy development and programs, I could not have imagined myself in the quantitative wonderland of raw data and coding. But an opportunity came up that shifted my perspective on the role that a diversity, equity, and inclusion lens can play on an evaluation team, and I wanted to share that with you.

A few years ago, when my business was still a side-gig, I had the opportunity to take on First Steps Kent (FSK) as a client. FSK administers $5.9 million taxpayer dollars annually to fund 39 early childhood programs in Kent County, Michigan with the ultimate goal of improving the health, school readiness, and wellbeing of young children. My work with them was focused on building team capacity through leadership support, as well as diversity, equity, and inclusion (DEI) strategy and development. This past fall, when I launched Advancing Communities for Equity, FSK asked that I work with the external evaluation team that had been selected through a Request for Proposal process (RFP) as an adviser on DEI. The team would be evaluating the impact of taxpayer investment by examining the impact of those 39 programs on early childhood outcomes in Kent County.

As someone who had worked with strong evaluation teams as a client, I wasn’t exactly sure what my role would entail, but as the scope took shape, I could see the alignment with my “disruptive” nature. I would be holding space for an equity lens in the evaluation process, activities, methods, and questions by asking the questions that pervade my worldview, and data-driven consulting work:

  • How is power operating?
  • Whose participation, needs, and input are reflected? Who is missing?
  • What might be the root causes for why certain groups are not reflected?
  • What structural, systemic, and institutional barriers might be at play, and how can these be addressed or reduced?

I came to this equity lens from a combination of lived and chosen experiences. As a first-generation American, I walked through childhood as both an insider and an outsider. I can recall the first time I saw my father – who seemed as big as the whole world – diminished by a racist comment in front of me at the grocery store, as he stammered through broken English to communicate what he needed from the clerk. We never talked about it, but I could see his face hot with shame and anger as he avoided my gaze. The interaction was a formative childhood memory. As a resident of the Deep South and typically one of a handful of Latinos in our small community and in my schools, I often identified with the experience of the Black community, despite my fair skin. I didn’t often see myself reflected in history lessons, novels, or media, let alone teachers or leaders. But I could relate to feeling marginalized, and could see the cost of stripping people of their dignity. Social justice work in high school sharpened my lens, and self-awareness.

I cannot claim to be an expert in DEI. I have done a lot of reading and research, participated in a three-year fellowship with the Kellogg Foundation, a certificate program at Northwestern, and have learned a lot. But to work in this space is to be fully present with your biases as a human being first, to be open to examining them, and to commit to removing them. It is a lifelong journey. To commit to antiracism, at least for me, is to recognize the privilege of being in spaces where I have a voice to call out inequity at play. The DEI advisor role requires me to bring historically marginalized communities, people with disabilities, families in rural parts of the county, into the room.

The First Steps Kent team truly lives their values, and are pioneers in the early childhood landscape (my area of expertise) so I couldn’t pass up the opportunity. Inspired by Linda Tuhiwai Smith, author of Decolonizing Methodologies, I wondered if I might be able to contribute in some small way, to enhance the process to “reduce racist practices and attitudes, ethnocentric assumptions and exploitative research.” It has been a valuable process that has taught me a great deal.

I have had the opportunity to work with a progressive team of evaluation experts at Basis Policy Research for this process. Here are my three big takeaways so far:
  1. Commit to diversity, equity, and inclusion staff people, and professional development for your evaluation team. Most organizations are working to build these muscles, and each individual begins at their own starting point, loaded with personal context and stories as in the brief example I shared with you earlier. Commit to DEI by prioritizing staff on your team who can support the professional development of the team, and integrate equity into organizational policy and practice, and into the external work that is produced. The evaluation proposal should reflect a DEI lens on the front-end, including the research questions, process, methods, and outputs. If it doesn’t, bring in some help.
  2. Be flexible and open, co-creating with the community of stakeholders. I was brought in to support the work of Basis Policy Research led by Senior Researcher, Dr. Kiel McQueen, who carries himself with the true spirit of a scientist: curious and objective. He and the Basis team have been open to my questions and suggestions, sometimes those which may upend a well-established practice. For example: Evaluation work often concerns itself with the experiences of those participating. However, equity concerns itself with who may be excluded. How do we approach the process, to identify and learn why families may not be participating?
  3. Begin and end with community. Evaluation work can reinforce inequities when it does not take into account what participants want from the process. Historically marginalized communities are often the subject of transactional engagement, in the form of webinars, surveys, and focus groups. These brief encounters are often translated into new ‘knowledge,’ where we may be assuming to tell a community about themselves, and then move on. An equitable evaluation process begins and ends with community. How can we co-lead with them, the true stakeholders of the work, who are best positioned to tell us what they need? Importantly, the First Steps Kent team has a community advisory board, consisting of leaders, parents, and service recipients, who have been thought partners along the way. We can’t lose sight of the context experts.
A few resources come to mind in planning for a collaborative evaluation process:
  • I regularly refer to the engagement spectrum model from the International Institute for Public Participation that shows what could be possible when engaging with stakeholders from a place of shared power (although I would replace the word empower with co-lead).
  • Community Science’s guides for Doing Evaluation in Service of Racial Equity are fantastic, and if you can find recordings of the webinars, check them out.
  • Our friends at Basis Policy Research have found this guide from MDRC to be helpful, and this model is helping us with a second project that I am supporting with their team.

As someone new in the role of DEI advising, I have wondered about the value of my perspective. Dr. McQueen has let me know that it has sharpened this project and others that he leads. He writes:

"Working with Advancing Communities for Equity has helped our evaluation team become more intentional about how we employ culturally responsive evaluation practices. We have been pushed to consider how we prioritize and embed stakeholders’ voice, expertise, and cultural traditions in all phases of evaluation. This process has helped our evaluation team honor the needs of the populations we serve and consider how we are designing activities accessible to stakeholders from all backgrounds. Emmy has helped us grow as personally and professionally, which in turn has yielded improved evaluations for our partners."
Dr. Kiel McQueen
Senior Researcher, First Steps Kent

The true measure of success will be how the evaluation results are received by the families of Kent County, and how the information is used to better reflect their needs and desires in optimizing child outcomes. I look forward to continuing to learn and grow with them.