October 2021
By Maryfrances Porter & Alison Nagel
Trying to do the Impossible
Since 2000 we’ve seen nonprofits try to do the impossible – and it isn’t breaking the cycle of intergenerational poverty. It isn’t trying to retain talented staff. It isn’t the endless chore of applying for grants and wooing donors so your organization stays sustainable and diversified (for brilliant commentary on this Sisyphean task you should read Nonprofit AF!). It’s none of these. The impossible task we are talking about program evaluation.
Program evaluation. An enigmatic necessity. Another thing to please funders and create headaches for you. Or: something you’re potentially really into, but you only have a vanishing idea (and vanishing time!) about how to approach. Or: maybe you’re drowning in data, but you’re not even sure it’s the right data.
We are here to tell you that program evaluation IS impossible for small- to medium-sized nonprofits. It’s not your mission and it’s not your job. However, you can still meaningfully track and understand the impact your organization is making for the people served – and that’s more than enough!
What’s that?!? Here’s more…
Goldilocks and the Three Errs
At its roots, program evaluation is a scientific pursuit. Scientists get together, often partnering with a nonprofit or school, and (1) design a program and then (2) test that program to see if it leads to important outcomes for the people served. So many wonderful programs have come out of program evaluation science including the Abecedarian Project, the Gang Resistance and Education Program, and so many more (you can find more in our Resource Library, click the tag for Solid Programming).
This is obviously important work, but not the important work of nonprofits. Nonprofits leaders aren’t scientists, and they don’t have millions of dollars of federal research funding to design and test program effectiveness. This model isn’t right.
Program evaluation can also be a strategic investment in a one-time process where a nonprofit hires an external (aka objective) program evaluator (aka a scientist) to help them take a good, close look at what they are doing and how it’s working. That program evaluator embeds in the nonprofit, asks questions, pokes around, measures, understands, and returns valuable information to the nonprofit. And this can be super valuable and help nonprofits get funding and improve programming for a year or two. But, what about ongoing understanding of impact and efficacy? This model isn’t right either.
Some people believe program evaluation is helping nonprofits retrofit and integrate as many scientific methods into programming as possible, create a “quasi-experimental” design where a comparison group is also tracked to test the “null hypothesis” that the program has no outcome, put scientific surveys in place, use statistics to determine if the program worked. We don’t know about reading that but just typing it was stressful! So, this model isn’t right either.
When we took our fancy-pants PhD’s and started consulting with nonprofits, trying to help them do program evaluation, we noticed immediately that what we learned in graduate school was valuable – but the science method just didn’t fit the budget, the mission, or the needs of nonprofits in the messy, day-to-day work of providing critical services. What we were taught to do wouldn’t give nonprofits real-time access to key data for monitoring client progress, staff performance, and program efficacy. And it certainly didn’t make it easy to tell clear, confident, and convincing stories to funders and the public.
Ahhh…Just Right
It’s taken years, but we’ve boiled down not only what nonprofits should do, but what they can do! It’s based in science, it’s informed and smart, but it talks and walks “nonprofit.” Some might call it a version of Program Evaluation, but we call it getting real results in real life.
Here’s what nonprofits can do:
- Nonprofits can be offering core programs that have already proven to work in the long-term (e.g., students earn a high school diploma (program) -> they are adequately prepared for the workforce -> they earn more money than students who do not earn a diploma (long-term impact): Learn more, earn more). If a proven program isn’t available, then programming that would be expected to work is a fine substitute, even though it can be a little harder to talk about.
- Nonprofit staff can be adequately trained and supervised in strategies and techniques for providing services that have been proven to work (e.g., patient-centered care, creating environments that promote positive youth development).
- Nonprofits can ask clients what they got from programming and how they liked participating. If nothing more, this is a simple matter of equity. People seeking services from nonprofits aren’t exercising a choice, they are struggling to meet a need. People who can’t vote with their wallet, still deserve to be asked what they think. Nonprofits have a responsibility to make sure the people served have meaningful means of providing actionable feedback. BONUS – you need this information to show you’re doing something that works for people.
- Nonprofits can clearly, confidently, and convincingly use client feedback to continuously improve programming and service provision, recruit new clients, build their reputation, and – yes – even let funders know their dollars are actually improving lives. This is masterful storytelling with data.
The ImpactStory™ Strategy
We created this strategy as a tool for small- and medium-sized nonprofits in telling clear, confident, and convincing stories about the impact they have on the people they serve.
There are three steps to ImpactStory™:
- Map Out Your Strategy – Be clear about what you do. Identify the reasons what you do is expected to work. Articulate the connections between your programs and the knowledge, skills, abilities, motivation, and confidence people get from working with your organization.
- Gather the Pieces – Be strategic and savvy about collecting only data that truly helps you understand your clients’ experiences and the ways their lives have been impacted by your organization. Create smart surveys and use simple, easy-to-understand data analysis.
- Tell Your Story – Use powerful data viz of key data points paired with and powerful narratives, woven together to tell the specific storis you know needs to be told. This might be a deep dive report for you and your program directors to pour over and use in planning, a dashboard for the Board to monitor, a glossy annual report for funders, or a curated story for a grant or social media.
ImpactStory™ Coaching
At Partnerships for Strategic Impact®, we focus on coaching, training, and supporting small- and medium-sized nonprofits in telling powerful impact stories. You’ve worked hard to develop honest, trusting relationships with all your stakeholders and funders, and you have the powerful testimonials highlighting how you’ve changed lives. But we all know numbers still matter.
You really can tell the whole story of how your organization is improving people’s lives. You really can talk about how people can take what they learn and make lasting change. And you really can talk about how your work is done in service of improving community conditions for everyone.
This is within reach. Your funders and the people being served deserve to know more about the value your organization delivers.
We Get It
Your work is value driven. Its value lies in the impact of your work on the people served. Let us help you tell your impact story. Reach out, answering questions is our specialty.
— Maryfrances & Alison
ImpactStory™ Coaches
Want more? Check out our Blogs and Videos.
We also curate and annotate resources just for small- and medium-sized nonprofits. Check out our Resource Library.
Looking for excellent consultants to help you with your next project? Search (or contribute to!) our growing directory of Recommended Consultants – recommended by and for small- and medium- sized nonprofits.
Here are some upcoming training opportunities from some of our favorite consultants named Ann.
- Evaluation and Collaboration Consulting and Podcast with Ann Price at Community Evaluation Solutions
- Data Visualization Training with Ann K Emery at Depict Data Studio