Setting NCVO's data free

A little while ago I promised to release some of NCVO's data, to put some of the principles of open data into action.

The first dataset is our work on classifying charities.

We undertook this piece of work about 2 years ago, in order to better understand the nature of the voluntary sector. While we had some classification information (such as the classification system used by the Charity Commission), the nature of these meant that producing stats that covered the whole sector was difficult. The Commission's system is useful for understanding what an individual charity does - it is multi-dimensional and charities can select more than one category - but we wanted to be able to look at and compare discreet groups of charities.

So we undertook quite a long process to classify every charity on the register. This involved a variety of different methods - outlined in this document - including keyword searches, matching to other registers and looking at individual sources.

We classified organisations using the International Classification of Nonprofit Organisations [pdf] (ICNPO). This system was designed by the Center for Civil Society Studies at Johns Hopkins University in US as part of efforts to draw up a UN Satellite Account for the nonprofit sector.

The ICNPO is not a perfect fit for the UK VCS - We had to add a couple of categories for large groups of organisations (such as scout groups and nurseries), and had to look carefully at the definitions we need.

The classification isn't perfect - a lot was done automatically and the quality is therefore dependent on the keywords used, and even for manual classifications some organisations will firmly disagree with the category in which they've been placed. The size and variety of activities in the VCS in England and Wales means these sort of exercise will always be difficult, and involve some square pegs and round holes. The data presented here is a classification of charities, it doesn't claim to be the classification.

Also included in the data is the "general charities" classification. This is something we talk about a lot, and is used to exclude some registered charities from the data we present in the Almanac, in order to give a picture more akin to what the "man on the street" might consider to be a charity. This means excluding independent schools, government-owned charities, faith groups and a few other groups. There's more information about general charities here.

So, the data. I've uploaded it to Google Fusion Tables, a useful service that Google provides for hosting large amounts of data. We're releasing it under an Open Database License (attribution, share-alike) so anyone can use it, so long as they attribute NCVO as the source and share any adapted versions of it. You can also download a CSV version of the data [csv, 5mb] - please note that this file has over 280,000 rows and so will not open in some programmes (older versions of Excel for example), and may need specialist expertise to handle.

The data has three columns:

• regno - the Charity Commission registration number of each charity
• ICNPO - the ICNPO classification category assigned to the charity. The list of possible categories can be found here. Some very new charities will not have a category assigned, and some old charities that have been removed from the register. Each charity is assigned to one category.
• general_charities - The General Charity Exclusion category assigned to the charity. The list of possible categories can be found here. Many charities do not have a general charity classification - again these are mostly very old or very new charities.

We're interested in your thoughts on the data - both as a general set and for specific data items. I should again emphasise that the nature of the data means that not every charity will be in the right category, and that in many cases there is no right category as classification is a subjective exercise. You can use the comment boxes on Fusion Tables to add your thoughts on the data, and use the filter function to find charities you are interested in. I also hope to keep this data up to date as we classify more charities, and also add SIC and COFOG codes to organisations.

• The data: Google Fusion Tables | CSV
• ICNPO categories used: google spreadsheet | XML
• General charities categories used
• The classification method

Update: fixed a couple of broken links. Tim (see below) has also added the dataset to CKAN.

Update2: ICNPO classifications now in XML format