Charities should drop “flaky” measurement systems as they try to show their impact, a think tank boss has said.
Dan Corry, chief executive of NPC, told his organisation’s annual conference that a focus on social return on investment, through which charities attach a cash value to the outcomes achieved thanks to investment in their projects, “isn’t really getting us anywhere”.
He also warned charities not to “caricature, dismiss and hurl abuse” at one another amid disagreements about the most equitable ways to collect and use evidence.
Corry was speaking on the final day of NPC’s Ignites conference, where he outlined details of its new approach to charity evaluation, which he said would balance service-user-led insights with traditional ‘top down’ data.
He said: “We have, for instance, virtually dropped doing simple social return on investment calculations, at best trying to guide those who ask for them to do break-even analysis instead [to measure when costs and revenue are equal].
“The thought sequence that goes into SROIs should be on everyone’s mind, but a ‘one pound in, fifteen back’ isn’t really getting us anywhere. The numbers are almost always flaky and based on too many assumptions.
“No wonder we now see what some academics have called SROI inflation.”
A more balanced approach to evaluation should draw on the experience and expertise of charities’ service users while keeping some of the biggest evaluation tools, Corry argued.
He said: “It’s not just charities and governments who should be asking, and be asked, what works, but individuals and communities as well.
“Data matters, but the crucial thing is that users, the community, are involved in framing the questions that we use that data to answer and are an integral part of the evaluation work.”
Corry said that it was important to “shift power in our sector” but argued that charities should not ditch “the impact agenda” along the way.
He said that this would be “fundamentally wrong and even dangerous”, adding: “Yes, things do need to change, and they are changing. But to truly shift power in our sector we need to blend the top-down with the bottom-up – to develop balanced evaluation – not destroy the very concept of assessing impact.
“Right now, we’re in danger of the two strands splitting. As we get stuck in opposing camps, we caricature, dismiss, and hurl abuse.
“That we must and will resist.
“We all lose if we don’t, and the people we want to help will lose out the most.”