Impact measurement in the spotlight

The pros and cons of measuring impact using techniques such as Social Return on Investment (SROI) continue to be explored across the sector.

Are we ready for SROI?

This week, Demos published a study that said most charities are not ready to use the SROI technique. The report, Measuring Social Value, highlights a perceived gap between the aspirations of policy makers for quantifiable measures of social value, and the ability of voluntary sector organisations to measure and capture basic social outcomes. It argues that the sector as a whole should achieve a basic and universal standard of outcome measurement before attempting to implement more complex models such as SROI, which presents social value in monetary terms.

Pros and cons

Members of the Voluntary Sector Studies Network (VSSN) have also had lively exchanges in the last few weeks about the merits of demonstrating social impact in economic terms.  Some commentators feel that provision of this type of information is crucial for making the case for their work.  Others question why so many organisations feel they now have to justify their work in terms of returns to the public purse, particularly when benefits of much of the sector’s work is already often very obvious. 

Another concern expressed in the VSSN debate is the danger of presenting the financial worth of a service as a set figure, when in reality the calculations to produce that figure are based on many varied assumptions.  Some participants in the discussion also felt that basing decisions about the importance of services purely on these types of economic proxies could mean simply avoiding responsibility for tough choices: instead of deciding which services are most needed, market forces dictate which are funded and poor results can then be blamed on the bureaucracy of the ‘system’.

One participant queried: "If we monetise the social good of services, what happens if we then find there are no 'economic benefits' from, say, getting mentally ill people into work?  Do we stop?"

Solutions

Responses to this dilemma suggested by network members included:

• check whether a financial figure is really required – for which audiences and why
• use techniques such as SROI only to choose between programmes before implementation, not to prove their impact once the work has been delivered – as the decision that this work is worthwhile has already been taken
• suggest alternative, perhaps more meaningful ways of demonstrating impact
• provide a cost-benefit analysis using a financial figure for the costs of delivery but expressing the impact in different terms eg "for every XXX pound spent we were able to bring XXX people into full employment."

Measurement: tools or fools gold?

Meanwhile, a recent presentation at a joint Birkbeck University/IVAR seminar on the theory and practice of impact called for a good dose of common sense when attempting to measure impact. Read some notes of the presentation.

Measuring outcomes for public services

The findings of a major study, Measuring Outcomes for Public Service Users, were published last month by the Office for National Statistics (ONS).  NCVO was a key partner in the project, which was based on a three year study into the quality and outcomes of adult social care services and early years education. It also had a particular focus on the performance of the voluntary and community sector in delivering such public services.

The report highlights the need to measure outcomes and value for money in the widest possible way, by focusing not just on the benefit of the service delivery to the individual end user but on the wider social value created as well.

Commenting on the findings, Sir Stuart Etherington, Chief Executive, NCVO said:

"Effective commissioning must focus on outcomes.  Only then will we achieve the high quality, value for money public services that people deserve.  All too often, statutory funding for front line services adopts narrow performance measurement models.  It should be no surprise that if you define, commission and manage services in the same way you always have, you will get the same outcomes, regardless of who is delivering them."

Etherington continued:

"Commissioning public service delivery by outcomes gives voluntary and community organisations the opportunity to demonstrate the full value they give to individuals and communities.  It ensures that hard pressed public funding is used to best effect in the wider public interest."

Read more about the methodology used in the adult social care study to assess the results for adult social care.

Read the full Measuring Outcomes for Public Service Users report.