Research for specific purposes

2. Research for specific purposes

This section looks at the specific circumstances that might lead to you doing some research and how you can apply some of the content in the next sections.

We look at two perspectives:

Part A Research to help you assess what you have done in the past

Part B Research to help you plan for the future

A. Assessing what you have done

At some stage, most organisations will need to do some sort of assessment of how well you are doing what you set out to do. This can either be as an internal process or one that is required of you by a funder. This could involve:

Monitoring – keeping count of how many people use your services, what sort of people they are (age, where they live, etc.) and what type of service they use
Evaluation – assessing how well you have met the needs of the people who use your services
Impact measurement being able to tell what changed for people as a result of your services

Monitoring

Monitoring entails regularly and systematically keeping a record of how your services are being used. This can be as simple as doing a daily headcount of the people in your building and noting it in a diary. If you want to know more about the people using your services you may ask individual staff or volunteers to keep records of the people they are working with, so that you can also tell, for example, how many children were in the crèche, or how many older people attended the lunch club. You can do all of this without troubling people by asking them questions. You should only do this with characteristics that don’t require you to make assumptions about people, such as their specific age or their ethnicity.

However, you might also do more formal monitoring where you ask people to sign in to a training session and fill out a form saying how old they are, their gender, their ethnicity, where they live, what transport they used to get to you, etc.

This information builds up a picture over time, so regardless of why you are collecting it – it could be a condition of a funder – you should always keep systematic records of what you have collected and use the data to review your services. Having this data will be vital when it comes to using the research techniques we discuss in this toolkit.

Your monitoring will produce quantitative data and this can be used to produce your outputs

You need to make sure that the information you collect is relevant to what you are doing and that you keep the information securely – especially if it is of a sensitive nature. You should check that what you are collecting and how it is stored is compliant with GDPR. Sometimes there can be conflict between what you are asked to collect by funders and what it is reasonable to collect in practice.

Community research toolkit

Introduction to the toolkit

Planning research template

Research design template

Glossary

About ARVAC

Feedback form

Example: A voluntary organisation subcontracted to deliver part of a large return to work programme was required to ask all the participants about their sexual orientation. Many of their clients were adults with learning difficulties and the project workers found that several of these participants did not understand the question. The workers started to explain what the question meant and realised they were discussing sexual matters with people who were vulnerable and subject to safeguarding rules. The participants and the workers found this very upsetting and in the end the project manager advised the workers not to ask the question but to tick the box ‘prefer not to say’ in each case.

Evaluation

Evaluation entails finding out how your services are used and viewed and using what you find out to plan, develop and improve your services. The three basic questions an evaluation sets out to answer are

What are we achieving?
Are we doing things competently?
What effects are we having?

Evaluation provides clear feedback to everyone involved in the organisation: users, staff, management, funders and the media.

Overview of process

The steps for an evaluation are much like those for any research project:

Agree who the evaluation is for.
Plan the starting points of the evaluation.
Agree what information is needed and how to collect it; establish baseline information.
Review the data and analyse what the information shows.
Tell people the results.
Evaluate the evaluation – what would you do differently next time? Was any data missing or hard to obtain? Could it be made more user-friendly?

Who it’s for

The first stage of an evaluation is to establish who and what the evaluation is for, as this will guide how it is to be carried out. Is it for internal use only? Or will it be used externally, in an annual report or as part of a progress report? How will its findings feed into the planning process? Will your organisation carry out the work itself, use a facilitator to help you or employ a consultant to carry it out for you?

Starting points

Clear and realistic goals for the project or service being evaluated. These need to be defined from the outset and have measurable objectives. It sometimes helps to have a ‘theory of change’ or a ‘logic chain’.
Enough time and money for the evaluation. The cost of an evaluation is linked to the size of the project. Allow about 5-10% of the annual budget for an ongoing service and up to 25% of the budget for a pilot scheme.
A self–critical approach and a willingness to change the way the service is delivered, acting on the results of the evaluation. You could use an approach such as action research to build reflection and planning change into your research project.
As with other types of research project, think carefully about the questions you want your evaluation to answer. You are unlikely to be able to evaluate every aspect of your service. Here are some common questions asked in an evaluation. We have given the technical names of each kind of evaluation but more important is selecting the purpose that is most appropriate for you.

Are we doing it right? How can we improve service delivery? How are our programmes and policies designed and implemented?

How do they relate to our aims and objectives? This includes looking at structure, decision-making, working relations and is known as a process evaluation.

While we are carrying out our work, how can we learn from experience and adjust the work accordingly? Which strategy works best? This is known as a formative evaluation.

In the end, what did and did not work and why? What improvements could be made? An evaluation carried out the end of a project, asking about the overall picture, is known as a summative evaluation.

What did we achieve? What outcomes and culture changed? What changes in behaviour were triggered? These questions are referred to as an impact evaluation. (See more information below)

How does the organisation itself work? This is an organisational evaluation.

You will also find these concepts and questions useful.

What goes into the provision of service/ activity? What resources (staff, skills, knowledge, budgets, equipment and the influences of policy or legislation) are needed to formulate and execute a policy, programme or project? These are inputs.

Did the service meet its specified objectives and agreed targets? This refers to your monitoring information and your outputs.

What happened as a result of the outputs? For example how many more people were able to claim benefits, more people able to seek work, improved housing conditions? These are outcomes.

Many funders expect some kind of outcome monitoring, a way of measuring intermediate or long-term outcomes that demonstrate to them that you are helping to achieve their objectives and goals.

Impact measurement

Impact measurement is the assessment of the changes that occurred as a result of the activities you have provided. This could include changes in the participants’ knowledge, attitudes, values, skills, behaviour, condition or status. The overall goal is to determine that a programme or service has made a difference. Measuring the impact of your work is a valuable exercise as it enables your organisation to assess whether its activities are achieving the ends that it set out to achieve. It also enables you to tell your own story to funders and to place a value on the things that you and your users think are important. The Public Services (Social Value) Act 2012 introduced the option for public service organisations to commission services based on the social value they can achieve as well as on cost. This is being used inconsistently across the country but it is enabling service providers to push for more weight to be given to social value in commissioning services.

See the Social Value Act introductory guide

Funders are increasingly requesting that impact measurement is undertaken and sometimes voluntary and community groups struggle to prove impact because it can be quite complicated to do. The references below provide a good introduction to the impact measurement process. Sometimes funders, particularly in the public sector, are not really interested in the overall impact of your work but are just looking for a ‘cost benefit analysis’ that proves how much money they are saving elsewhere.

For example, a voluntary organisation that supports older people is funded to install basic home improvements, such as handrails, that prevent people having falls. The funder will want to know the number of handrails that have been installed and could infer that this is the number of falls prevented. The voluntary organisation might want to also produce information such as feedback from the people who have received the service about how this has made them feel happier and more confident that they will be able to remain living in their own homes. It is very hard to prove what didn’t happen (how many people didn’t fall), but the voluntary organisation has also produced a research study undertaken by a university that has evidence of how early prevention actions such as installing handrails can lead to savings of £x for the NHS. An impact report would capture all this information: numerical data (outputs), quality of life changes (outcomes), and use of external data to support the case for the impact that the project is making.

There are lots of tools available to help measure your impact but you should make sure you pick an approach which is proportionate to what you are measuring and is something that you understand and can do well. Some tools are extremely complex to use and if you select one of these it is advisable to bring in someone with experience to help you.

Choosing your collection methods

Inspiring Impact resource hub

Inspiring impact diagnostic tools

Guidance and standards

B. Planning for the future

Lots of voluntary and community groups have a high demand on their services and as long as people are coming through the door and there is money in the bank, they do not stop and think about the future. After a period in which there have been major changes in society in regard to what services and support will be funded from the public purse and what the role is of voluntary bodies, most organisations would be advised to stop every now and then and think about the future.

Research to provide evidence to help you plan for the future could include:

Needs assessment – to find out from your users or potential users what needs they have.
Mapping – to find out who is providing what services in your area.
Feasibility study – to find out if a new service could generate income for your organisation.

A needs assessment can explore or prove specific needs and to identify areas of unmet need within your target groups. You may have some anecdotal evidence of the need for your existence and the services you offer but have no substantiated evidence. Or you may be aware of needs within your community but you may not know what those specific needs are, or how they should be met. Research can play a role in both instances by helping you to identify, clarify, measure and prove need whilst also identifying relevant and appropriate ways of meeting these needs. You could include in your assessment a summary of what people are saying they need and also use evidence from existing research – some useful sources are listed in the resources section 10

Specifically, you could do some consultation with current or potential service users to ask them what they want and what they would like to get from your organisation. Increasingly funders want to see evidence that you have consulted with beneficiaries about their needs when you are bidding for their funding and that the findings have shaped how you are planning to deliver your service.

Mapping existing services in the locality will provide information on unmet needs when incorporated into the needs assessment. Finding out what provision and services already exist is crucial for building partnerships, signposting and ensuring your organisation is not duplicating other services.

Feasibility studies are undertaken when you want to develop a new service or plan an expansion of an existing service. There may be a demand for your service but can you either find funding for it or get people to pay for it? If you are asking people to pay out of their own pockets then you should be asking them about what they think is acceptable to pay. This can be difficult if you want to charge for a service that people have previously received free of charge. You can’t assume that people will continue to use a service if they have to pay for it

Example: A lunch club for older people in Norfolk had its local authority funding reduced and increased the contribution from users in order to keep open. The immediate effect was that people who had previously attended three or four times a week, started only coming once a week. The organisers were concerned about their loss of income but also that people who had benefited from the service were no longer getting access to it.

When to undertake this research:

In the start-up phase of your organisation you will need to provide evidence of the need for your organisation and its activities to potential funders, the local communities you seek to serve and to local authorities. It will also contribute to demonstrating that the activities you are developing will be taken up and are relevant and appropriate and that you are not duplicating existing services.
When you have ideas to grow your organisation.
In response to change when something happens in your area or a piece of legislation will have an impact on your users or you lose a source of funding.
On an on-going basis to ensure that, for example, your existing services meet the needs of all sections of the community, rather than only one particular group of people?

Are your existing services or areas of work still relevant or are they out of touch? Are there any barriers to some people accessing your services? If any of these questions are relevant, your organisation may not have kept in touch with the communities it serves. Research will help to establish new needs your organisation could be meeting.

Example: A new organisation, gained funding to deliver a service to homeless people in a London Borough. Part of the funding was earmarked to identify what type of homelessness service was needed in the area. The organisation conducted 50 interviews with homeless people across the Borough. The research established that there was a need among those interviewed for more information and advice services. There was also a need for more accommodation and they wanted a better service from the Borough Council. In response the organisation set up their own advice and information service in the Borough, an area where there were few services for homeless people, but a significant number of homeless people. They also established a homelessness forum, bringing together all agencies working in homelessness in the Borough. This helped to deliver a more co-ordinated service. As part of the research they compiled a directory of existing services, which ensured their new service was not duplicating others, created a useful signposting tool and helped to increase the take up of existing services. The directory was a practical outcome of their research that could be used by organisations, including the council, throughout the Borough.

It’s your choice

Whatever the purpose of your enquiry, you will need to use the processes and methods described within this toolkit – the important thing is to choose the approach that works best in your specific circumstances.

1. Why do community research?

3. Who is doing the research?