Collecting data

When you are deciding how you will judge the impact of your work, you will need to choose what to assess. You will also need to identify how to collect the information or data that will tell you about your progress.

Choosing the right sources of data

The two main sources of data are:

• documents, eg published reports, unpublished documents, and information from databases
• people, eg through questionnaires, interviews over the phone or face-to face, discussion groups, diaries (paper, photo, video), or observation.

Collecting data

The collection methods you choose will depend on the type of information you want and the resources at your disposal. There are a range of impact tools and techniques which can help to capture evidence, including information about soft outcomes.

Reliable data

Try to make each data collection method:

• valid (the data is relevant to the question you are asking)
• reliable (the data is consistent even if it is collected in different places and at different times) 
• unbiased .

Relying on just one source of data for all your information will increase the possibility that the information is inaccurate

Also, consider using more than one source of data to provide information about a topic. Sometimes referred to as triangulation, this can give you richer data and better judgements. Relying on just one source of data (eg user surveys) for all your information will increase the possibility that the information is inaccurate.

Processing data

Data is usually either numerical or textual.

Numerical data can be entered and analysed in a computer spreadsheet or database. Most people will find they can ask many questions of the data fairly easily, such as 'what percentage of users rated our advice service as very good?' If you want to ask complex questions, such as 'which element - age, gender, education or income - most affects the way a user responds to our counselling service?' you could consider getting advice from an expert in statistics.

Textual data are usually analysed by looking for common themes or patterns in the material. Do certain words or ideas keep coming up? You can categorise sections of text into the main themes and then ask what these themes tell you. For example, you may notice that first-time users of your day centre keep complaining about the reception area, but other users don't. Is the apparent drop in demand for your services actually caused by the reception area putting off new users?

Honesty is vital when collecting data.

Ethics

Credibility and accountability are two cornerstones of monitoring and evaluation, especially self-evaluation. Honesty is vital when collecting data. You need to ensure it is done fairly and accurately and that people who give you data are trusted and treated with respect. Fabricated data is almost always discovered and risks destroying an organisation's entire credibility, and its future. Honest results demonstrate integrity, openness and an organisation that is capable of learning. Read about developing an improvement culture .

It is also important to consider confidentiality in collecting data. Ensuring secure ways of collecting, storing and analysing data is essential if you are to maintain the trust of those providing the data, and to stay on the right side of data protection laws. The Information Commissioner's Office has information for organisations about their data protection responsibilities.

How can you use the data you collect to judge your impact?