Benchmarking resident satisfaction data

Question set and guidance, July 2021.

Introduction

Understanding resident or customer views is a key element of assessing the effectiveness of an authority, alongside cost and performance information. Furthermore, understanding resident satisfaction and being able to make informed comparisons can strengthen local accountability and be a key part of the sector’s approach to managing its own performance.

The LGA conducts regular national telephone polling of resident’s satisfaction with local government, three times a year. In addition, regional level results are available on a yearly basis. This guidance outlines a set of resident satisfaction questions that councils can use in their local surveys, and guidance on the conditions that must be met, to enable the results to be benchmarked against these national and regional results.

This is a flexible approach, designed to suit local circumstances and minimise the cost to councils of collecting the data. It allows councils to use only those questions that are of interest locally and fit these questions in with existing survey plans, whilst also helping to maximise the benefit and insight that can be gained from this data by making it broadly comparable.

The questions were developed in consultation with the sector and with assistance from Ipsos Mori, including cognitive testing. In addition, the document contains additional questions which are not part of the benchmarking, but which councils may find useful to use in their own local surveys.

Purpose and scope of the guidance

This guidance has two elements:

  • A ready to use question set which is cognitively tested, developed in consultation with the sector and with advice from Ipsos Mori. These questions can be used by any council using any survey method, regardless of whether they would like to benchmark these or not.
  • For councils conducting general population telephone surveys and who wish to compare their results, a set of criteria that would need to be met to ensure that these questions can be robustly benchmarked against the LGA’s national and regional comparator figures.

It is important to note that this is a guide to benchmarking and not a guide to conducting a resident survey. Councils should follow standard recognised good practice when carrying out surveys locally.

Overview of the questions

The full set of questions can be found below. They cover the following areas:

  • satisfaction with the local area as a place to live
  • satisfaction with the way the council runs things
  • value for money
  • council responsiveness
  • how well informed the council keeps residents
  • advocacy
  • community identity
  • community safety
  • trust in the council
  • satisfaction with services
  • community cohesion
  • anti-social behaviour.

The role of benchmarking

Benchmarking of these results should be part of a wider approach to understanding and responding to local communities. Benchmarking provides context but is only one element of this approach and helps raise lines of enquiry rather than providing answers.

Councils should seek to include additional questions in their local surveys that might help diagnose what is driving their results locally. For example, questions on what needs improving in the area might be of limited use for benchmarking but could usefully be included locally to identify areas for improvement.

Benchmarking criteria

The criteria laid out below describe the conditions that would need to be met for a council to be able to make a meaningful comparison of its survey results against the national and regional LGA polling figures.

These are based on the findings of a review by Ipsos MORI, which identified a set of conditions that would need to be met to ensure that any comparisons being made are robust, fair and valid, and that any data is of a good quality that will stand up to scrutiny. The criteria were then refined based on an extensive consultation with local authorities. 

Note: The question set and guidance were initially developed as part of a project to enable council to council benchmarking (called ‘Are You Being Served’). The following document summarises and addresses the key issues raised by consultees and explains the changes that were made to the guidance as a result of the consultation: Residents' satisfaction: consultation response. Whilst the primary purpose was to look at council to council benchmarking, most of the document remains relevant to national and regional benchmarking, and it also gives background on how the questions were developed. 

It will be important to refer to this guidance right at the outset of planning any local survey activity, as some of the requirements will influence early decisions about issues such as sampling and data collection method.

Data collection methods

The LGA’s national and regional figures are obtained using a telephone poll. As the mode of data collection can have a marked impact on results, intra–mode comparisons are not desirable. Therefore, we recommend that only councils using a telephone survey method should benchmark against the LGA figures.

Sampling

Only surveys that use all adult residents as the sample frame are appropriate for benchmarking (i.e. general population surveys). Service, topic or area specific surveys that only draw a sample from a subset of the general population will not be appropriate for benchmarking.

Two types of sampling will be suitable for benchmarking, in addition to a census approach:

  • Random sampling (whereby all population members/households have a random one in ‘n’ chance of being selected) is favoured for all methods of data collection.
  • Quota sampling is also acceptable.

Authorities that have obtained satisfaction data through residents’ panels will not be able to benchmark this. This is because there is strong evidence to suggest a considerable response bias among panel members, which makes it inadvisable to compare data collected via this method with general population data.

Weighting

Where a random sampling or census approach has been used, data should then be weighted to the known profile of the local population. Variables that are typically used for weighting in residents’ surveys include age, gender and social grade (or work status as a proxy for social grade). 

This guidance does not specify exactly which variables the data should be weighted on. The purpose of weighting is to make a sample representative if it is not already. What is appropriate will depend very much on the way the survey was sampled and on local circumstances. 

Don’t know and refuse to answer

For benchmarking, it is important that the ‘don’t know’ and ‘refuse to answer’ options for each question are dealt with in the same way between surveys. Therefore councils who wish to compare their survey results to the LGA figures will need to take the following approach:

  • Do not include an explicit reference to ‘don’t know’ or ‘refuse to answer’ in the answer option list.
  • The interviewer can however code these answers if they are given spontaneously.

The questions

Question ordering

Testing has identified that respondents’ answers to certain perception-based questions may be influenced by questions asked earlier in a survey (this is known as ‘context’ and ‘position’ effects). For example, if a question about general satisfaction with an area is asked after a series of questions about crime, this is likely to elicit a more negative response than for a different authority that preceded this with questions about parks and leisure. Therefore, for each question, advice on question ordering is given for councils who wish to compare to the national and regional polling results.

Full question text

This section outlines the full text and answer options of the questions, as well as guidance on the modifications that would be needed in two tier areas and any definitions that should be included. For each question, it is also stated whether an LGA benchmark figure (national and regional) is available.

Definition of local area

Many of the questions in this set ask respondents about their ‘local area’. Please include the following text at the start of your survey to ensure that this is being interpreted consistently:

“Throughout this survey we ask you to think about ‘your local area’. When answering, please consider your local area to be the area within 15 – 20 minutes walking distance from your home.”

Question 1: Satisfaction with the local area

Question

Overall, how satisfied or dissatisfied are you with your local area as a place to live?

Very satisfied
Fairly satisfied
Neither satisfied nor dissatisfied
Fairly dissatisfied
Very dissatisfied
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

Yes

Question ordering

Questions 1 - 3 should be placed as the first questions at the beginning of the survey and in the order they appear here.

 

Introducing the council

The following preamble should be inserted before the first question that asks specifically about a named council. A slightly adjusted preamble will be used depending on which scenario your survey fits:

  • In the case of a questionnaire designed to understand perceptions of a single tier council:

“Your local area receives services from [name of council]. [Name of council] is responsible for a range of services such as refuse collection, street cleaning, planning, education, social care services and road maintenance.”

  • In the case of a questionnaire in a two-tier area, designed to understand perceptions of a district only, or a county only:

“Your local area receives services from two councils, [name of district council] and [name of county council]. EITHER This survey asks about [name of district council], which is responsible for services such as refuse collection, street cleaning and planning OR This survey asks about [name of county council] which is responsible for services such as education, social care services and road maintenance.”

  • In the case of a questionnaire designed to understand perceptions of both the district and county:

“Your local area receives services from two councils, [name of district council] and [name of county council]. [Name of district council] is responsible for services such as refuse collection, street cleaning and planning. [Name of county council] is responsible for services such as education, social care services and road maintenance.”

Question 2: Satisfaction with the local authority

Question

Overall, how satisfied or dissatisfied are you with the way [name of council] runs things?

Very satisfied
Fairly satisfied
Neither satisfied nor dissatisfied
Fairly dissatisfied
Very dissatisfied
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

Yes

Question ordering

Questions 1 – 3 should be placed as the first questions at the beginning of the survey and in the order they appear here.

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question.

Question 3: Value for money

Question

In considering the next question, please think about the range of services [name of council] provides to the community as a whole, as well as the services your household uses. It does not matter if you do not know all of the services [name of council] provides to the community. We would like your general opinion.

To what extent do you agree or disagree that [name of council] provides value for money?

Strongly agree
Tend to agree
Neither agree nor disagree
Tend to disagree
Strongly disagree
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

Yes

Question ordering

Questions 1 - 3 should be placed as the first questions at the beginning of the survey and in the order they appear here.

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, please use this slightly different introductory text:

In considering the following two questions, please think about the range of services that both [name of district council] and [name of county council] provide to the community as a whole, as well as the services your household uses. It does not matter if you do not know all of the services [name of district council] and [name of county council] provide to the community. We would like your general opinion.

Then ask ‘To what extent do you agree or disagree that [name of council] provides value for money?’ twice.

Question 4: Council responsiveness

Question

To what extent do you think [name of council] acts on the concerns of local residents?

A great deal
A fair amount
Not very much
Not at all
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

Yes

Question ordering

This question should be subject to as little bias as possible. Therefore, when an authority decides to include this question, it should be placed as the fourth question in the questionnaire.

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question.

Question 5: Informed about the council

Question

Overall, how well informed do you think [name of council] keeps residents about the services and benefits it provides?

Very well informed
Fairly well informed
Not very well informed
Not well informed at all
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

Yes

Question ordering

This question should precede any other questions about council communications and be placed as early as possible after questions 1 – 4.

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question

Question 6: Advocacy

Question

On balance, which of the following statements comes closest to how you feel about [name of council]?

I speak positively of the council without being asked
I speak positively of the council if I am asked about it
I have no views one way or another
I speak negatively about the council if I am asked about it
I speak negatively about the council without being asked
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

No

Question ordering

No specific requirement

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question.

Question 7: Community identity

Question

How strongly do you feel you belong to your local area?

Very strongly
Fairly strongly
Not very strongly
Not at all strongly
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

No

Question ordering

No specific requirement

 

Question 8a: Community safety after dark

Question

How safe or unsafe do you feel when outside in your local area after dark?

Very safe
Fairly safe
Neither safe nor unsafe
Fairly unsafe
Very unsafe
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

Yes

Question ordering

If there are any other questions about community safety in the survey, this question should precede these.

 

Question 8b: Community safety during the day

Question

How safe or unsafe do you feel when outside in your local area during the day?

Very safe
Fairly safe
Neither safe nor unsafe
Fairly unsafe
Very unsafe
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

Yes

Question ordering

If there are any other questions about community safety in the survey, this question should precede these. It should however go after the ‘after dark’ community safety question above (question 8a).

 

Question 9: Trust in the council

Question

How much do you trust [name of council]?

A great deal
A fair amount
Not very much
Not at all
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

Yes

Question ordering

This should precede any more detailed questions about trust in the council.

Two tier areas: If this questionnaire is being conducted by a district and county in partnership, repeat the question.

Question 10: Satisfaction with services

Note: This question was not included in the original ‘Are You Being Served’ technical review but is now included in the LGA polling.

Question

I am going to read out a number of different types of services that are provided by [name of council]. I would like you to tell me how satisfied or dissatisfied you are overall with the council’s…

Waste collection
Street cleaning
Road maintenance
Pavement maintenance
Library services
Sport and leisure services
Services and support for older people
Services and support for children and young people
Parks and green spaces

Response scale:

Very satisfied
Fairly satisfied
Neither satisfied nor dissatisfied
Fairly dissatisfied
Very dissatisfied
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

Yes

Question ordering

No specific requirement

 

Question 11: Community cohesion (ethnicity)

Question

To what extent do you agree or disagree that your local area is a place where people from different ethnic backgrounds get on well together?

By getting on well together, we mean treating each other with respect.

Definitely agree
Tend to agree
Neither agree nor disagree
Tend to disagree
Definitely disagree
Don’t know (Do not read out, but interviewer can code)
Too few people in local area (This should be treated the same as ‘don’t know’ both when asking the question and when reporting results)
All the same ethnic background (Also treated the same as ‘don’t know’)

LGA Benchmark available?

No

Question ordering

No specific requirement

 

Question 12: Community cohesion

Question

To what extent would you agree or disagree that people in this local area pull together to improve the local area?

Definitely agree
Tend to agree
Neither agree nor disagree
Tend to disagree
Definitely disagree
Nothing needs improving (This should be treated the same as ‘don’t know’)
Don’t know (Do not read out, but interviewer can code)

LGA Benchmark available?

No

Question ordering

No specific requirement

 

Question 13: Anti-social behaviour

Question

Thinking about this local area, how much of a problem do you think each of the following are….

a) Noisy neighbours or loud parties
b) Rubbish or litter lying around
c) Vandalism, graffiti and other deliberate damage to property or vehicles
d)People using or dealing drugs
e) People being drunk or rowdy in public places
f) Groups hanging around the streets

A very big problem
A fairly big problem
Not a very big problem
Not a problem at all
Don’t know/No opinion (Do not read out, but interviewer can code)

LGA Benchmark available?

No

Question ordering

No specific requirement

Further information and support

If you have a query on any aspect of this guidance, please email research@local.gov.uk.

Councils might want to get involved with their regional Local Area Research and Intelligence Association (LARIA) group, to provide mutual support and discuss the technical issues associated with conducting resident satisfaction surveys: https://laria.org.uk/. LARIA also has a Knowledge Hub (community of practice) group: https://khub.net/group/lariagroup.