Case study - Pilotlight Focus on Impact

Dan Ritman, Evaluation Manager at Pilotlight, shares his experience of demonstrating the impact of the work of this infrastructure organisation.

About Pilotlight

Pilotlight brings together teams of senior business people to provide free capacity building support to charities and social enterprises. With support and facilitation provided by Pilotlight staff the business volunteers coach senior charity staff and trustees through a strategic planning process.

"Pilotlight helps ambitious organisations who are tackling disadvantage, to reach more people, more effectively."

Why focus on impact?

Pilotlight recognised that by looking at the impact of their work they could learn what was most effective about their approach and improve it.

Understanding their results means they can also make a strong case for funding and other support, as well as motivate their staff and business volunteers.

"Understanding our impact makes us a more effective, more sustainable organisation."

How they got started

"We began with a blank piece of paper!"

Pilotlight simply asked their partner charities (those that are provided with the support) about their experience of working with Pilotlight and what changed for them. In time, they found common themes emerging which they then began to use as a framework to structure their information.

Common themes and a journey

They recognised that their partner charities typically strengthen several areas of their work, during and after the Pilotlight relationship. These include:

• Capabilities in planning, financial management, communication and staff and trustee performance
• Resources (turnover, number of staff, breadth of services)
• People helped by the charity
• Changes in lives and communities.

"We planned out these changes in a service-user journey. This helps us to understand and explain the changes that we're looking for."

View the Pilotlight service-user journey (PDF 62Kb).

Evidence collection

Pilotlight monitors these kinds of changes, and any feedback on the Pilotlight process, through:

• Collection of baseline data at the start of the Pilotlight process (such as turnover, services and people helped by the partner charities)
• Reports at the end of the Pilotlight process based on interviews with all participants.
• Annual updates (for up to three years) covering significant changes in the partner charity. This helps to show the lasting impact.

Pilotlight accept that attribution of change to their work is a difficult issue.

"We often ask our partner charities what they feel the change can be attributed to. We don't try to claim too much but we also don't let it put us off showing the possible links between our work and the change in the partner charities."

Using the information

Individual reports and updates are then distributed to all participants (Pilotlight staff, business volunteers and partner charities.)

For other external audiences Pilotlight makes simplified versions of these reports and updates (e.g. case studies and key figures) for presentation. View an example of a Pilotlight case study (PDF 14kb).

Annually Pilotlight aggregate information from the reports and updates to produce overall impact figures for Pilotlight and emerging themes and lessons. Aggregate figures and written summaries of observations from individual projects are distributed internally. The lessons from these serve as a basis for discussion about modifying their approach and informing their relationships.

"Stories and figures are also presented at internal meetings to motivate ourselves."

Full Value of the work

Over the last year or so Pilotlight have started looking at the impact the work has on the business volunteers who carry it out. They have found that it is not just the partner charities that benefit from the work. Often the business volunteers also find that it gives them refreshed and improved skills and helps them to be more effective in their professional work.

Have a look at their business volunteer impact report (PDF 28kb)

Challenges

"This takes time! But as a result we have seen improvements to the Pilotlight process and have built a strong, confident case for communicating the difference we make."

Pilotlight recognise that there are particular challenges for an Infrastructure organisation in demonstrating their impact. Whilst they can look at their impact in the same way as any organisation, the additional layer(s) between them and ultimate service-users means it is harder to present their impact in terms of the current interest in social impact.

They address this challenge by telling the story of the ultimate change in lives and communities in their case studies. View an example of a Pilotlight case study (PDF 14kb). They do not try to claim responsibility for this impact but it helps to explain the ultimate purpose of their work.

Their service-user journey also shows the link between Pilotlight's activites and the impact of the charities they work with. View the service-user journey (PDF 62kb).

Plans for the future

Pilotlight are now thinking more about incorporating their impact measurement into the capacity-building process they use. They feel that this will make it more valuable for participants, more empowering and turn it into a learning process for all involved.

Advice for others

And finally, Dan's advice for anyone who is thinking about demonstrating their impact:

"Keep it simple. You know what difference you make – when you have a good day; think about the simplest ways to demonstrate that. Don't worry about whether it's an output or an outcome or an impact…!"

For further information on any of Pilotlight's impact work please contact Dan Ritman or visit Pilotlight's website.