Search tips
Search criteria 


Logo of brjgenpracRCGP homepageJ R Coll Gen Pract at PubMed CentralBJGP at RCGPBJGP at RCGP
Br J Gen Pract. 2009 August 1; 59(565): e267–e272.
Published online 2009 June 15. doi:  10.3399/bjgp09X453792
PMCID: PMC2714801

Do the Quality and Outcomes Framework patient experience indicators reward practices that offer improved access?

Richard Baker, MD, FRCGP, Head of Department, M John Bankart, MSc, PhD, Lecturer in Medical Statistics, and Ged M Murtagh, MPhil, PhD, Research Fellow



The Quality and Outcomes Framework (QOF) includes indicators for patient experience, but there has been little research on whether the indicators identify practices that deliver good patient access.


To determine whether practices that achieved high QOF patient experience points in 2005/2006 or 2006/2007 also delivered good patient access.

Design of study

Use of publicly available data to investigate two hypotheses: practices with more positive access survey findings in 2006/2007 will be more likely to have achieved maximum QOF patient experience points in the same year; and practices with maximum QOF patient experience points in 2005/2006 will have higher access survey findings in 2006/2007.


Two-hundred and twenty-four East Midlands general practices.


For hypothesis one, binary logistic regression was used, with achievement of maximum QOF points as the dependent variable, and access survey findings, responder variables, and practice variables as independent variables. For hypothesis two, general linear models were used, with access survey findings as the independent variables, and achievement of maximum QOF points and the responder and practice variables as dependent variables.


The findings did not support the first hypothesis. For the second hypothesis, achievement of maximum QOF points was only significantly associated with patient satisfaction with opening hours (positive correlation). QOF points were not associated with any other aspect of access.


The QOF patient experience indicators do not reward practices that offer good patient access. A standard patient survey with financial incentive may be more effective in identifying and rewarding practices that offer better access, including opportunity to book appointments with a particular doctor.

Keywords: healthcare quality, access and evaluation; patient-centred care; primary health care


The Quality and Outcomes Framework (QOF) is designed to reward practices that deliver high-quality care. Quality is measured by 146 indicators in four key domains: clinical, organisational, patient experience, and additional services. Practices are allocated points for targets achieved, and the number of points gained is translated into financial reward.1 The majority of studies investigating the impact of the QOF on quality of care have concentrated on the clinical indicators that account for most of the points that determine the financial reward. Although improvements among the clinical indicators have been reported,25 few studies have investigated the impact of the patient experience indicators.

In 2005–2006 and 2006–2007, four indicators of patient experience were included in the QOF, although two of the 2005–2006 indicators were modified for 2006–2007 (Table 1).6 A total of 100 points was allocated to patient experience in 2005/2006, and 108 points in 2006/2007. The indicator PE5 is being dropped for 2008/2009, and PE2 and PE6 are being dropped for 2009/2010.7 Although the indicators do not deal specifically with access or continuity, in both years they required practices to undertake a patient survey using one of two approved instruments, these being the General Practice Assessment Questionnaire (GPAQ) and the Improving Practice Questionnaire (IPQ). The GPAQ includes questions on practice opening hours, the number of days' wait for an appointment with a particular doctor or with any doctor, aspects of the consultation, and being able to get through to the practice by telephone.8,9 The IPQ includes questions on access, continuity, availability of information, and the patient's experience of the doctor's interpersonal skills.10 Thus, practices using these instruments that achieve maximum scores in the patient experience domain will have undertaken a survey, considered access and continuity, reflected on the results, and enacted changes or a detailed action plan.

Table 1
The QOF indicators for the years 2005/2006 and 2006/2007.

In 2006/2007, a new financial incentive (called a directed enhanced service or DES) was introduced in England to improve patient access to primary care.11 This provided a reward to practices that offered patients the opportunity to consult a GP within 2 working days, the opportunity to book appointments in advance, ease of telephone access, and opportunity for the patient to consult their practitioner of preference.12 One-third of the reward was offered to practices that aspired to these goals, and two-thirds awarded according to the findings of a new survey of patient access.13

How this fits in

The Quality and Outcomes Framework is a major initiative intended to transform the delivery of primary care services. Some clinical indicators have improved following introduction of the Framework, but concerns remain about the value for money of the scheme. This study suggests that current patient experience indicators do not identify practices that provide good patient experience of access.

This paper reports a study of the patient experience QOF points and patient access survey findings. It was hypothesised that:

  • practices with more positive patient access survey findings in 2006/2007 will be more likely to have achieved maximum QOF patient experience points in the same year (that is, QOF patient experience points and access survey results are associated); and
  • that practices with maximum QOF patient experience points in 2005/2006 will have a higher access survey score in 2006/2007 (that is, practices that undertake their own patient survey and enact appropriate changes if indicated will provide improved access in the following year).


All practices in three local primary care trusts (PCTs: Leicester City, Leicestershire County and Rutland, and Northamptonshire) that had completed the access survey in 2007 were included. QOF data for the years 2005/2006 and 2006/2007 were obtained from the NHS Information Centre.14

The patient access survey was administered by MORI to around 5.2 million people registered with 8241 practices between January and March 2007.15 Approximately 2.3 million people responded (44% response rate). The survey questionnaire was sent to systematic samples of patients aged ≥18 years who were registered with each practice, the sample size being adjusted according to practice size. The samples were drawn from either general practice computer systems or the registration details on a central NHS database (National Health Application and Infrastructure Services). The average sample size was 432 patients per practice (699 in those practices in which computer records could not be used to eliminate patients who had not had an appointment in the past 6 months).15 The questionnaire asked five questions about aspects of access: whether they had been able to get an appointment within 2 days, able to book in advance (more than 2 days in advance), and able to get an appointment with a particular GP, and satisfaction with (a) ease of getting through to the practice on the telephone, and (b) opening hours. The questionnaire also sought information on responders' sex, age, and ethnic group, and numbers of consultations in the past year. The data are publicly available.16 Data were also obtained on numbers of patients registered with each practice and a measure of the level of socioeconomic deprivation of the practice population (the Index of Multiple Deprivation (IMD) 2007).17

Statistical analysis

Following descriptive reporting of the data, regression models were undertaken for each hypothesis. For the first hypothesis, binary logistic regression was undertaken, with QOF points in 2006/2007 as the dependent variable, grouped into two categories: maximum points for patient experience and less than maximum points. The total patient experience points was the sum of PE2, PE5, and PE6; PE1 being omitted since it related to length of consultations and not access. The independent variables were the five access questions, access survey responder variables: age group (percentage of responders aged 18–44, 45–64 and ≥65 years); percentage of responders who were male; percentage giving their ethnicity as white (white British, white Irish, or other white category); percentage response rate for the practice; the percentage of responders reporting different numbers of appointments with their practice in the past year (categorised into 0, 1–4, and ≥5), and practice variables: number of patients in the practice; the practice IMD score; and the PCT of the practice. For the second set of hypotheses, the five access questions were the dependent variables (each dependent variable was arcsine transformed) in five general linear models using backward stepwise regression, the independent variables being achievement of maximum QOF score in 2005/2006, plus the responder and practice variables. Analyses were undertaken with SAS (version 9.1).


Of the 230 practices in the three PCTs, 224 registered for the enhanced access scheme patient surveys. Maximum QOF patient experience points were achieved by 183 practices across both years, with 201 just in 2005/2006 and 188 just in 2006/2007. Practice characteristics are shown in Table 2, and the percentages of responders answering positively to the five access survey questions are shown in Table 3, according to whether practices did or did not achieve maximum QOF patient experience points in 2005/2006.

Table 2
Practice characteristics (n = 224).
Table 3
Access survey results, 224 practices, according to achievement of maximum QOF patient experience points 2005/2006.

The regression analysis did not support the first hypothesis; there was no association between QOF points and the access survey findings. The only variable associated with achievement of maximum QOF patient experience was the number of patients registered with the practice (odds ratio 1.0002, 95% confidence interval = 1.0001 to 1.0003, P = 0.001, Hosmer and Lemeshow test of goodness of fit, non-significant [P = 0.23]). Figure 1 shows the fitted probability of achieving maximum points with increasing numbers of patients.

Figure 1
Plot of total patients against probability of practices achieving maximum QOF points, 2006/2007.

Table 4 presents the results of the regression models for each of the five questions of the access survey, showing which independent variables were associated with increased satisfaction in response to the five access questions. The analysis failed to support the second hypothesis that conducting a patient survey and enacting changes in response to the findings as required by the 2005/2006 QOF indicators was associated with improved access survey findings in the following year. Only in the case of satisfaction with opening hours was there a significant association between QOF patient experience points and experience of access. With respect to satisfaction with opening hours, Table 4 shows that in practices that achieved maximum QOF patient experience points, more patients tended to report satisfaction; satisfaction tended to be lower in practices with larger patient list sizes, and lower if a greater proportion of patients reported having one to four appointments in the past year rather than five or more appointments, or no appointments; satisfaction was higher in practices with a higher IMD score, higher when the proportion of white responders was higher, and higher as the response rate increased.

Table 4
Prediction of access survey results in 2006/2007 by QOF patient experience points in 2005/2006 (general linear models, backward stepwise regression used to eliminate non-significant predictors).

Increasing size of practice was associated with declines in patient experience of all aspects of access (opening hours, being able to see a particular doctor, telephone access, being able to see a GP in 48 hours, and being able to book an appointment in advance). People who were white, younger, and male tended to experience better access. Improved access was also associated with a higher practice response rate to the survey. This suggests that the data were not missing at random, and that there may be a bias operating.


Summary of main findings

No association was found between achievement of maximum QOF patient experience points and patient experience of access in 2006/2007 (hypothesis one). Furthermore, there was an association between QOF patient experience points in 2005/2006 and patient-reported access in 2006/2007, but only for satisfaction with opening hours (hypothesis two); for all four other aspects of access there was no significant association. In contrast, larger practices were more likely to achieve maximum QOF patient experience points,18 but smaller practices were more likely to have better patient-reported access.

Strengths and limitations of the study

This study is the first to the authors' knowledge to bring together QOF indicators and findings of the national patient access survey. It was undertaken in an area with an ethnically and socioeconomically diverse patient population, and since almost all the practices took part in the access survey, the majority of practices were included. There are some limitations to the access survey data. The published data give information on percentages of responders in different age, sex, and ethnic groups, but not the numbers. Information on practice non-responders is not available. While it was possible to include a wide variety of potentially explanatory variables relating to the responders and practices, it is likely that other variables explaining either QOF points or access survey results were omitted. It should also be pointed out that the response of patients to surveys may not change quickly following changes in service provision, a factor that might explain some of the discrepancy between the QOF patient experience indicators and access survey findings. Nevertheless, the study raises questions about the ability of the QOF patient experience indicators to identify practices that offer better patient experience of access.

Comparison with existing literature

In addition to questions on access, the national patient survey has questions relevant to aspects of continuity: being able to book an appointment in advance and being able to consult a preferred practitioner.19 Concern has been expressed about the impact of the QOF on continuity,20 and although the present study is unable to determine whether the QOF has impaired continuity, it does indicate that the QOF patient experience indicators are failing to encourage practices to improve these two aspects of continuity. This could have to do with the design of the QOF, which may encourage a more clinical or disease-orientated type of care that can sometimes conflict with the patient's own agenda.21 It has been established previously that patients tend to report that larger practices offer worse access in comparison with smaller practices.22,23 The present findings indicate that larger practices still have difficulty ensuring high levels of access and continuity. The large new primary care providers being established in some cities will need to pay particular attention to access and continuity.

Implications for clinical practice

Improvements in some clinical indicators following the introduction of the QOF have been established,25 but questions continue to be raised about the value for money of the scheme. A recent parliamentary report concluded that the QOF had not led to improved access to general practice,24 and two patient experience indicators are being removed from the QOF.7 The present study supports the view that the patient experience indicators have not been successful in promoting improved access or continuity. The study does not show why the QOF has not been successful. It is possible, for example, that practices have been attempting for many years to improve access and have already reached the best attainable level of performance. Other explanations include competing priorities for practices within the QOF, and the failure of the QOF indicators to target activities that have an impact on patient experience. Since most practices achieved maximum points for the patient experience indicators, it may be argued that the indicators are not effective in discriminating between different levels of performance. An independent patient survey that directly investigates patients' experiences of access, coupled to a financial incentive, may be a more effective means of promoting improved access and continuity.


Funding body

No external funding was used in undertaking this study

Ethics committee

Ethics approval for this study was not required since it was based on publicly available data

Competing interests

Professor Baker was chair of the expert advisory group on the QOF patient experience indicators, 2006–2008. There are no other potential conflicts of interest to report.

Discuss this article

Contribute and read comments about this article on the Discussion Forum:


1. Smith P, York N. Quality incentives: the case of UK general practitioners. Health Aff. 2004;23(3):112–118. [PubMed]
2. Doran T, Fullwood C, Gravelle H, et al. Pay-for-performance programmes in family practice in the United Kingdom. N Engl J Med. 2006;355(17):375–384. [PubMed]
3. Tahrani AA, McCarthy M, Godson J, et al. Diabetes care and the new contract: the evidence for a whole county. Br J Gen Pract. 2007;57(539):483–485. [PMC free article] [PubMed]
4. Steel N, Maisey S, Clark A, et al. Quality of clinical primary care and targeted incentive payments: an observational study. Br J Gen Pract. 2007;57(539):449–454. [PMC free article] [PubMed]
5. Gravelle H, Sutton M, Ma A. Doctor behaviour under a pay for performance contract: evidence from the Quality and Outcomes Framework. Centre for Health Economics, York: University of York; 2007. CHE Research Paper 28.
6. BMA and NHS Employers. Revisions to the GMS contract, 2006/07. London: NHS Confederation; 2006.
7. NHS Employers. Quality Outcomes Framework. (accessed 3 Feb 2009)
8. General Practice Assessment Questionnaire. (accessed 3 Feb 2009)
9. Mead N, Bower P, Roland M. The general practice assessment questionnaire (GPAQ) — development and psychometric characteristics. BMC Fam Pract. 2008;9:13. [PMC free article] [PubMed]
10. Greco M, Powell R, Sweeney K. The improving practice questionnaire (IPQ): a practical tool for general practices seeking patient views. Education for Primary Care. 2003;14:440–448.
11. Department of Health. GP patient survey: your doctor, your experience, your say. Guidance for strategic health authorities, primary care trusts and GP practices. London: Department of Health; 2006.
12. Department of Health. GP patient survey: your doctor, your experience, your say. Further guidance for strategic health authorities, primary care trusts and GP practices. London: Department of Health; 2007.
13. NHS Employers. Quality and Outcomes Framework guidance GMS contract 2008/9. London: NHS Employers; 2008.
14. NHS Information Centre. (accessed 3 Feb 2009)
15. Department of Health. 2006/2007 GP patient survey. Access survey. Final technical report. London: Department of Health; 2007.
16. Department of Health. GP patient surveys. (accessed 3 Feb 2009)
17. Department of Communities and Local Government. The English indices of deprivation 2007. London: HMSO; 2008.
18. Wang Y, O'Donnell CA, Mackay DF, Watt GCM. Practice size and quality attainment under the new GMS contract: a cross-sectional analysis. Br J Gen Pract. 2006;56(532):830–835. [PMC free article] [PubMed]
19. Baker R, Boulton M, Windridge K, et al. Interpersonal continuity of care: a cross-sectional survey of primary care patients' preferences and their experiences. Br J Gen Pract. 2007;57(537):283–290. [PMC free article] [PubMed]
20. Freeman G. Up close and personal? Continuing pressure on the doctor–patient relationship in the QOF era. Br J Gen Pract. 2006;56(528):483–484. [PMC free article] [PubMed]
21. Checkland K, McDonald R, Harrison S. Ticking boxes and changing the social world: data collection and the new UK general practice contract. Soc Policy Adm. 2007;41:693–710.
22. Baker R, Streatfield J. What type of general practice do patients prefer? Exploration of practice characteristics influencing patient satisfaction. Br J Gen Pract. 1995;45(401):654–659. [PMC free article] [PubMed]
23. Campbell JL, Ramsay J, Green J. Practice size: impact on consultation length, workload, and patient assessment of care. Br J Gen Pract. 2001;51(469):644–650. [PMC free article] [PubMed]
24. House of Commons Committee of Public Accounts. NHS pay modernisation: new contracts for general practice services in England. Forty-first report of session 2007–8. London: House of Commons; 2008.

Articles from The British Journal of General Practice are provided here courtesy of Royal College of General Practitioners