Miles Wilson is a philanthropic professional with nearly 30 years of experience supporting the U.S. social sector as well as past efforts in Northern Ireland, the Netherlands, and South Africa. Miles’ work has covered a broad spectrum of core social sector activities, and he currently serves as the Deputy Director of Education Grantmaking at Ascendium Education Group. Miles was most recently a Senior Fellow with the Aspen Institute Forum for Community Solutions. This feature, Social Justice and a Relevant Philanthropic Sector, is the fourth in a six-part series of blog posts about his experiences in philanthropy. A version of this blog series is running on the Center for Effective Philanthropy website. Access the entire six-part series here.
In the last blog post, I discussed the issue with logic models. Fortunately, there are other evaluation options out there. Among the most promising evaluation approaches includes Developmental Evaluation which is particularly appropriate for social innovation and systems change initiatives. Celebrated evaluation expert Dr. Michael Quinn Patton says the following about Developmental Evaluation: “Traditional evaluation aims to control and predict, to bring order to chaos. Developmental evaluation accepts such turbulence and the way the world of social innovation unfolds in the face of complexity. Developmental evaluation adapts to the realities of complex nonlinear dynamics rather than trying to impose order and certainty on a disorderly and uncertain world.” Another is also Adaptive Evaluation which also recognizes the complexity of real life circumstances and operating across contexts and populations.
What I currently find most exciting is the increased promotion and use (albeit small at this point) of goal-free evaluation. This is an approach that pushes back on the constant focus on measuring impact on predetermined sets of goals. Goal-free evaluation allows organizations to respond to the broad programmatic interests of a funder to conduct its long-term work. At the same time, an external evaluator, independent of both the nonprofit and the funder, “attempts to observe and measure all actual outcomes, effects, or impacts, intended or unintended, all without being cued to the program’s intentions.” It’s an unbiased view and wealth of information regarding what is actually happening and where the greatest value appears to lie. In their Stanford Social Innovation Review article, “Ten Reasons Not to Measure Impact and What to Do Instead,” authors Mary Kay Gugerty and Dean Karlan offer a clear caution about the focus on impact measurement.
The push for more and more impact measurement can not only lead to poor studies and wasted money, but also distract and take resources from collecting data that can actually help improve the performance of an effort…To create a right-fit evidence system, we need to consider not only when to measure impact, but when not to measure impact.
If DEI is to mean anything, evaluation must become fully incorporated into everything that happens in the work of philanthropy, and if done properly, evaluation can be a valuable tool to advance DEI in individual foundations and throughout the work of the field
Again, this pushes back against the foundation’s orientation to grant recipients and the often self-imposed pressure to demonstrate impact in nearly all circumstances, but the potential value provides opportunities for learning on both sides and a much deeper and objective understanding of how and why programs might succeed or fail. While Goal-Free Evaluation is not intended to be a stand-alone evaluation, it lends itself to great promise for becoming stand alone with greater experience of use, and when paired with appropriate interim grant reporting. I also think that this form of evaluation is an excellent fit for learning and accountability for funders who provide their grant recipients with long-term general operating support, funding the nonprofits mission rather than projects.
If DEI is to mean anything, evaluation must become fully incorporated into everything that happens in the work of philanthropy, and if done properly, evaluation can be a valuable tool to advance DEI in individual foundations and throughout the work of the field. To this end, there is a wonderfully promising effort called the Equitable Evaluation Initiative. The Initiative is a five-year effort that attempts “to align evaluation practices with an equity approach—and even more powerfully, to use evaluation as a tool for advancing equity.” The approach of the initiative is based upon four important principles of diversity of teams, cultural appropriateness, use of evaluation to reveal structural inequity, and advancing the community role in shaping evaluation. This initiative is an important element for establishing critical infrastructure for imbedding and assessing DEI efforts throughout the field and should be supported and embraced by foundations and field infrastructure organizations alike.
At the end of the day, the field needs to recognize that a “one-size fits all” approach to evaluation doesn’t benefit foundations, nonprofits, or the constituents nonprofits attempt to serve. It’s also reasonable to say that small nonprofits and those operating in poorer regions such as much of the South, Southwest, Appalachia, and communities of color in literally every state are often negatively impacted by broad one-size fits all approaches to evaluation. It’s important for the field to remember of the nation’s estimated 1.5 million nonprofit organizations, 72% have budgets of $500,000 or less and of those 61% have budgets of $100,000 or less. Ultimately, size and access to resources should impact considerations of the type of evaluation deployed.
By 2003, I began to see a noticeable uptick in conversation at the Council on Foundations as well as among the various Regional Associations of Grantmakers regarding issues of building greater DEI within organized philanthropy. In later years I was excited to take part in some of those discussions and watch as the field produced a vast number of papers, guides, handbooks, and even DEI initiatives to advance the work. We spoke not only of greater representation of people of color in leadership roles in philanthropy, but also about greater grant dollars going to communities of color including an emphasis on organizations led by people of color – serving people of color. Seventeen years later, I am deeply disappointed that very little meaningful action has taken place to grow DEI in philanthropy in any substantial or comprehensive way. In fact, I fear the field has actually moved backward on this important issue.