PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of qualsafetyQuality and Safety in Health CareCurrent TOCInstructions for authors
 
Qual Saf Health Care. 2007 December; 16(6): 450–455.
PMCID: PMC2653182

Comparison of patients' assessments of the quality of stroke care with audit findings

Abstract

Objective

To determine the extent of correlation between stroke patients' experiences of hospital care with the quality of services assessed in a national audit.

Methods

Patients' assessments of their care derived from survey data were linked to data obtained in the National Sentinel Stroke Audit 2004 for 670 patients in 51 English NHS trusts. A measure of patients' experience of hospital stroke care was derived by summing responses to 31 survey items and grouping these into three broad concept domains: quality of care; information; and relationships with staff. Audit data were extracted from hospital admissions data and management information to assess the organisation of services, and obtained retrospectively from patient records to evaluate the delivery of care. Patient survey responses were compared with audit measures of organisation of care and compliance with clinical process standards.

Results

Patient experience scores were positively correlated with clinicians' assessment of the organisational quality of stroke care, but were largely unrelated to clinical process standards. Responses to individual questions regarding communication about diagnosis revealed a discrepancy between clinicians' and patients' reports.

Conclusions

Better organised stroke care is associated with more positive patient experiences. Examining areas of disparity between patients' and clinicians' reports is important for understanding the complex nature of healthcare and for identifying areas for quality improvement. Future evaluations of the quality of stroke services should include a validated patient experience survey in addition to audit of clinical records.

Stroke is one of the main causes of death and disability in England, and is a major source of expenditure for the National Health Service (NHS) and the wider economy.1 Development of stroke services and improvement in the quality of care is increasingly recognised as a priority. The National Service Framework (NSF) for Older People set out standards to ensure that those who have had a stroke have prompt access to integrated stroke care services.2 The National Clinical Guidelines for Stroke, prepared by the Intercollegiate Stroke Working Party (ICWP), provide a detailed evidence‐based framework for delivering the best care for stroke patients in hospital and in the community.3 The quality of hospital stroke services has been evaluated on a 2‐year cycle since 1998 using clinical audit methods developed by the ICWP.4 Results of the national audits have enabled trusts to assess the quality of their stroke services and monitor progress against standards set out in the NSF and the National Clinical Guidelines for Stroke. Performance is assessed based on the provision of stroke services featuring at least four of five key features identified by the Stroke Unit Trialists' Collaboration as defining a specialist stroke unit5,6:

  • Having staff with a specialist interest in stroke or rehabilitation
  • Involving carers in the rehabilitation process
  • Multidisciplinary team meetings taking place at least weekly
  • Provision of information to patients and carers
  • Regular education and training for staff.

Besides evaluating the quality of care through clinical audit, there is a growing interest in assessing patients' experiences of care. Patient satisfaction surveys have traditionally been the main approach used to obtain their views of healthcare. However, these surveys have been criticised because the concept of satisfaction is ill‐defined and multidimensional,7 and because such surveys typically elicit high undifferentiated levels of satisfaction.7,8,9 Measuring patients' experiences of care by asking them to provide reports of “what happened” during a specific episode of care is considered to be a more valid approach. Responses to such “report” style questions are intended to be factual rather than evaluative and provide results that are more useful for quality improvement.10,11 As part of the Healthcare Commission's national patient survey programme, a survey was conducted in England to evaluate stroke patients' experiences of care in hospital and in the period immediately after discharge.12

There is a growing body of research on patients' experiences of stroke care,13,14,15,16,17,18 yet few studies have directly compared clinical audit with patients' own assessments of the quality of stroke care. Research undertaken in the USA showed that the process of stroke care—measured by compliance with stroke guidelines—was positively and highly associated with greater patient satisfaction.19 To assess hospital‐based stroke rehabilitation, Tyson and Turner20 combined a process audit with surveys of patient and staff opinions, arguing that by comprehensively examining many aspects of the service together, inadequacies in the quality of care could be identified more readily. Similarly, Durieux and colleagues21 compared patients' and healthcare professionals' views of standards of care and concluded that both patients' and healthcare professionals' evaluations must be considered if high quality care is to be achieved.

Methods

The 2004 national sentinel stroke audit had two parts: the first audited the organisation of care and the second audited the care of up to 40 patients per site consecutively admitted to hospital between April and June 2004 based on reports from clinical records. Reliability analyses, where the first five cases per site were audited by two different auditors, gave good levels of agreement across all the audit data items, with κ values above 0.60 dominating.4 Further details of audit forms and the methods can be found at the Royal College of Physicians website (www.rcplondon.ac.uk/pubs/books/strokeaudit/).

The questionnaire used in the patient survey was based on published qualitative studies of stroke patients' and carers' experiences of care, the NSF for older people and the National Clinical Guidelines for Stroke. Content validity and sampling issues were considered by an advisory group of experts in stroke. Cognitive interviews with 29 stroke patients supported the face validity of the questionnaire, which was further tested in a pilot study involving 187 patients in three trusts. Details about the development of the questionnaire are available online (http://www.nhssurveys.org/docs/Stroke_Development_Report.pdf).

All NHS acute trusts in England participated in the audit. Approximately a third (n = 51) of the acute trusts were randomly selected for participation in the patient survey. We asked trusts to sample patients admitted between April and June 2004 with a primary diagnosis of stroke. Between October and December 2004, the surviving sample patients were sent a questionnaire and up to two reminder letters. Linkage between the survey data and the audit data was undertaken by matching on the basis of NHS trust, gender, year of birth, and dates of admission and discharge.

Data analysis

We calculated an overall patient experience score for each respondent on the basis of their responses to 31 questions. Individual responses reflecting quality of care on each question were scored in equal increments on a scale of 0 to 100, with higher scores reflecting better experience. The overall score was a simple average of all applicable responses. We chose to derive a simple mean score in this way in order to recognise the fact that many different elements of care could potentially affect the quality of patients' experiences overall. Although differences in the overall quality of organisation within trusts might manifest in different ways, we considered this approach to remain sensitive to such disparities by broadly including all measured aspects of the patient experience. In addition, the 31 survey items were grouped into three broad concept domains, with mean scores calculated for “quality of care” (12 items), “information” (11 items) and “relationships with staff” (8 items) to have a more focused way of assessing certain key areas of organisational quality.

Data for the organisational audit were extracted from hospital admissions data and management information. We used an a priori algorithm developed by the ICWP to derive nine organisational domain scores, and from these an overall organisation score. These domains covered a broad spectrum of the organisation of care including the interdisciplinary nature of services both on and off the stroke unit, staff knowledge and skills, team working on records, meetings and agreed measures of assessment, availability of information, and communication with patients and carers. The clinical process of care audit was based on information obtained retrospectively from patient records. Eleven “yes/no” indicators were analysed, which had been identified in the 2001 national stroke audit to best represent the total clinical process.4

The distribution of organisation of care and overall patient experience scores were negatively skewed. Relationships between hospital‐level audit data on the organisation of care and patients' reports were tested with the Spearman correlation test, whereas the association between these and clinical process standards was assessed with the χ2 test. Associations of each indicator with overall patient experience scores were assessed using the Mann–Whitney test.

Results

Selection

A total of 2729 patients were sampled for inclusion in the experience survey from the 51 trusts, of whom 1042 were also included in the audit (table 11 (A)). (Another 57 who were initially included in the patient survey were excluded from analysis as they were subsequently found to have died since their stroke.) Completed questionnaires were obtained from 670 (64%) of the 1042 patients. Audit patients in the survey had similar baseline characteristics to audit patients not in the survey, though they did have slightly better health status at discharge and a shorter stay in hospital (table 11 (A, B)). Audit patients from trusts participating in the survey were similar to audit patients from other trusts although they were slightly more likely to have been treated in a stroke unit (table 11 (A/B, D)). Survey responses from audit and non‐audit patients were similar (results not shown). However, the survey sample was not completely representative of all stroke patients with some demographic groups being under‐represented.12

Table thumbnail
Table 1 Characteristics of the sample

Organisation of services

Patient experience scores for those sites with better organisation were higher both overall and within all three domains (table 22).). Of the 58 sites that had a specialist unit, 38 had all five basic features of organisation that define a stroke unit. Two sites had two features, one had three features and 17 sites had four features. Patients in sites with all five organisational features were more positive about their experience of stroke care (table 22).

Table thumbnail
Table 2 Patients' experiences of care compared with audit assessments of organisational quality of stroke services

Process of care

The overall patient experience score was significantly higher for patients who, according to the audit, had (1) stayed in a stroke unit for more than half of their stay; (2) been screened for swallowing difficulties; (3) been assessed by a physiotherapist within 72 h of admission; and (4) been assessed by an occupational therapist within 7 days of admission (table 33;; see table for p values). Similar patterns were seen for each of the three patient domains (results not shown). More specific cross‐matching of survey and audit questions showed weak trends in anticipated directions or no clear trends at all (table 44).). Relationships between audit and survey responses were observed for the provision of information on secondary prevention.

Table thumbnail
Table 3 Patients' experiences of care compared with audit data on clinical process standards
Table thumbnail
Table 4 Comparison between specific patient experience and clinical audit questions

Discussion

Our analysis shows areas of both correlation and disparity between audit and patient survey data. The patient‐reported experience of hospital care correlated with audit assessments of the organisational quality of stroke services, showing that well‐organised stroke services are important for managing patients effectively3 and for improving their experiences. Furthermore, the finding that patients are more likely to report better experiences in sites with all five features considered essential for effective stroke unit organisation, emphasises the importance of implementing these organisational standards. This is consistent with previous research showing that some of the dimensions of care evaluated in the service organisational audit relate closely to patients' satisfaction with care, such as: doctor–patient communication; information provision; staffing levels; and service coordination.13,14,15,16,17,18

There was less correlation between patients' experiences of hospital care and the clinical process indicators. Only four of the 11 indicators were significantly associated with a more positive experience. There are a number of possible reasons why certain indicators correlated with patients' reports of their experiences and others did not. The four standards of care that did correlate may have been more visible or salient to patients: assessments by a physiotherapist and/or an occupational therapist represent additional contact from health services, demonstrating to patients that a multidisciplinary approach is being taken to their care. Being cared for on a stroke unit—one of the most positively correlated indicators—is associated with many clinical and organisational standards that probably have a positive influence on patients' experiences of care. A previous study also found that it was the dimensions of stroke care most visible to the patient which correlated most highly with their overall satisfaction.19 In contrast, some indicators may not have correlated with positive experiences due to their being less salient, from the patient's perspective, to the quality of care: patients may not always be cognisant of the value of being weighed or having their mood assessed. Other process of care standards may simply not be visible or memorable to patients, particularly if they take place immediately following admission, such as a brain scan, when patients might be less likely to subsequently recollect them. The overall patient experience score did not include questions on care following discharge, which might explain why the performance of a home visit was not seen to be significantly associated with a more positive patient experience.

Some studies in other areas of healthcare have reported a lack of correlation between patients' assessments of care and clinical‐based measures.22,23 A limitation of these studies, however, was their use of global rating scales instead of patient reports to assess patients' satisfaction with care. By using patients' reports of what actually happened during their hospital stay, it should be possible to gain a greater insight into the causes of satisfaction or dissatisfaction, and to identify specific events that did or did not occur in the care process. A difficulty in comparing audit and patient survey data is that many of the dimensions of care that are important to patients, and therefore covered in the questionnaire, are either not assessed or are measured differently in the audit. For instance, evidence that the patients' rehabilitation goals have been agreed by the multidisciplinary team—one of the key process standards—was not significantly correlated with patients' overall satisfaction with care. It is possible that if the audit had recorded whether the patients' own needs and wishes had been taken into account when setting such goals, this would have been associated with a positive patient experience. We should not, therefore, be surprised when patient reports differ from those recorded through clinical audit because the reports present two different methods of assessing the quality of care. Audit has an important role in assuring adherence to clinical standards, yet a patient‐centred system of healthcare is equally dependent on the ability to systematically collect and use feedback from patients on their views of the process: human elements of care such as effective communication simply cannot be quantified in any other way. This study demonstrates, then, that neither audit nor survey data is sufficient in isolation to present a comprehensive overview of quality of care.

Where it was possible to make direct comparisons between specific survey and audit questions, very little correlation was found. The provision of information on smoking cessation was the only survey question significantly correlated with a recording in the audit that risk factors had been discussed with the patient, although information provision on other aspects of secondary prevention (diet and physical exercise) were approaching statistical significance. Notable differences between clinicians' and patients' reports were evident with regard to communication about diagnosis. For almost three‐quarters of respondents who reported that their diagnosis had not been discussed with them, the audit recorded that such a discussion had taken place. This type of discrepancy has been found in other studies,20 and a likely cause is that recording whether a diagnosis was discussed with the patient does not tell us anything about the quality of that interaction. A number of explanations for why information might not be effectively communicated to patients are cited in the literature: health professionals may feel they lack the time; the explanations given are too complicated; information is provided at the wrong time (such as when the patients' comprehension or recall was impaired); or it is too general and does not address the patients' own concerns.25,26,27,28 A further reason given is that health professionals avoid giving detailed explanations to patients due to the risk of encouraging over‐optimistic expectations of recovery following stroke.25,28,29 Identifying areas where patients' and clinicians' reports do not concur is important for a better understanding of how the quality of care can be improved, and if this research were conducted at the level of the individual clinician or clinical team, it could be a very effective teaching aid for communication skills.

We believe this study is the first to have compared findings from a clinical audit and a patient survey at a national level. The ability to link directly the experiences of patients to clinical records on several dimensions of stroke care within the acute setting was a clear strength of the study. There were, however, some limitations. As previously stated, comparing responses from the patient survey to audit data was difficult because there was not an exact match in the dimensions of care being measured. Also, the survey results may have been biased in a positive direction because patients with poorer self‐reported health status were less likely to have responded.12 It is known that patients with poorer health tend to give more negative assessments of their care.24 Similarly, our analysis did not control for other factors that could have influenced patients' reports on the quality of care, such as stroke severity or comorbidity.

Conclusion

This study has shown that patients cared for in better organised stroke units, as measured by a clinical audit, reported a more positive experience of care in the patient survey. This strengthens the rationale for ensuring that all the recognised organisational standards are implemented in stroke units. In some areas disparities were found between patients and clinicians' reports on the processes of care, and this emphasises the intrinsic differences between the two sources of feedback. Indeed, an important reason for conducting patient experience surveys is that they measure standards that cannot be reliably measured by other means. Future evaluations of stroke services should therefore undertake both an audit of clinical records and a patient survey for a complete assessment of care.

Acknowledgements

The patients who responded to the survey and the 51 acute NHS trusts.

Abbreviations

ICWP - Intercollegiate Stroke Working Party

NSF - National Service Framework

Footnotes

Funding: The Healthcare Commission funded both the 2004 National Sentinel Audit of Stroke and the stroke patient survey.

Competing interests: None.

References

1. National Audit Office Reducing brain damage: faster access to better stroke care. London: The Stationery Office, 2005
2. Department of Health National service framework for older people. London: Department of Health, 2001
3. Royal College of Physicians National clinical guidelines for stroke, 2nd edn. Prepared by the Intercollegiate Stroke Working Party. London: RCP, 2004
4. Royal College of Physicians National Sentinel Stroke Audit 2004 (concise report). Prepared on behalf of the Intercollegiate Stroke Working Party. London: RCP, 2005
5. Stroke Unit Trialists' Collaboration Organised inpatient (stroke unit) care for stroke (Cochrane Review). In: Cochrane Library. Issue 4. Oxford: Update Software, 2002
6. Langhorne P, Dennis M S. Stroke units: the next ten years. Lancet 2004. 363834–835.835 [PubMed]
7. Williams C, Coyle J, Healy D. The meaning of patient satisfaction: an explanation of high reported levels. Soc Sci Med 1998. 471351–1359.1359 [PubMed]
8. Fitzpatrick R, Hopkins A. Problems in the conceptual framework of patient satisfaction research: an empirical exploration. Sociol Health Illness 1983. 5297–311.311 [PubMed]
9. Wilkin D, Hallam L, Doggett M A. Measures of need and outcome for primary health care. Oxford: Oxford Medical Publications, 1992
10. Cleary P, Edgman‐Levitan S, McMullen W. et al The relationship between reported problems and summary evaluations of hospital care. Qual Rev Bull 1992. Feb53–59.59 [PubMed]
11. Jenkinson C, Coulter A, Bruster S. The Picker Patient Experience Questionnaire: development and validation using data from in‐patient surveys in five countries. Int J Qual Health Care 2002. 14353–358.358 [PubMed]
12. Healthcare Commission Stroke: survey of patients. London: Healthcare Commission, 2005, www.healthcarecommission.org.uk/_db/_documents/04018503.pdf (accessed July 2005)
13. Pound P, Gompertz P, Ebrahim S. Patients' satisfaction with stroke services. Clin Rehabil 1994. 87–17.17
14. Thomas C, Parry A. Research on users' views about stroke services: towards an empowerment research paradigm or more of the same? Physiotherapy 1996. 826–12.12
15. Kelson M, Ford C, Rigg M. Stroke rehabilitation: Patient and carer views. London: College of Health, Royal College of Physicians, 1998
16. Macduff C. Stroke patients' perceptions of hospital nursing care. J Clin Nurs 1998. 7442–450.450 [PubMed]
17. Pound P, Tilling K, Rudd A G. et al Does patient satisfaction reflect differences in care received after stroke. Stroke 1999. 3049–55.55 [PubMed]
18. McKevitt C, Wolfe C. Community support after stroke: patient and carer views. Br J Ther Rehabil 2000. 76–10.10
19. Reker D, Duncan P W, Horner R D. et al Postacute stroke guideline compliance is associated with greater patient satisfaction. Arch Phys Med Rehabil 2002. 83750–756.756 [PubMed]
20. Tyson S, Turner G. The process of stroke rehabilitation: what happens and why. Clin Rehabil 1999. 13322–332.332 [PubMed]
21. Durieux P, Bissery A, Dubois S. et al Comparison of health care professionals' self‐assessments of standards of care and patients' opinions on the care they received in hospital: observational study. Qual Saf Health Care 2004. 13198–202.202 [PMC free article] [PubMed]
22. Rao M, Clarke A, Sanderson C. et al Patients' own assessments of quality of primary care compared with objective records based measures of technical quality of care: cross sectional study. BMJ 2006. 33319–23.23 [PMC free article] [PubMed]
23. Chang J T, Hays R D, Shekelle P G. et al Patients' global ratings of their health care are not associated with the technical quality of their care. Ann Intern Med 2006. 144665–672.672 [PubMed]
24. Cleary P D, McNeil B J. Patient satisfaction as an indicator of quality care. Inquiry 1988. 2525–36.36 [PubMed]
25. Wiles R, Pain H, Buckland S. et al Providing appropriate information to patients and carers following a stroke. J Adv Nurs 1998. 28794–801.801 [PubMed]
26. Rodgers H, Atkinson C, Bond S. et al Randomized controlled trial of a comprehensive stroke education program for patients and caregivers. Stroke 1999. 302585–2591.2591 [PubMed]
27. Rodgers H, Bond S, Curless R. Inadequacies in the provision of information to stroke patients and their families. Age Ageing 2001. 30129–133.133 [PubMed]
28. Anderson S, Marlett N J. Communication in stroke: the overlooked rehabilitation tool. Age Ageing 2004. 33440–443.443 [PubMed]
29. Wiles R, Ashburn A, Payne S. et al Patients' expectations of recovery following stroke: a qualitative study. Disabil Rehabil 2002. 24841–850.850 [PubMed]

Articles from Quality & Safety in Health Care are provided here courtesy of BMJ Publishing Group