PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of ijicareInternational Journal of Integrated Care
 
Int J Integr Care. 2016 Jul-Sep; 16(3): 13.
PMCID: PMC5350641

Development and Validation of an Instrument for Assessing Patient Experience of Chronic Illness Care

Abstract

Introduction:

The experience of chronic patients with the care they receive, fuelled by the focus on patient-centeredness and the increasing evidence on its positive relation with other dimensions of quality, is being acknowledged as a key element in improving the quality of care. There are a dearth of accepted tools and metrics to assess patient experience from the patient’s perspective that have been adapted to the new chronic care context: continued, systemic, with multidisciplinary teams and new technologies.

Methods:

Development and validation of a scale conducting a literature review, expert panel, pilot and field studies with 356 chronic primary care patients, to assess content and face validities and reliability.

Results:

IEXPAC is an 11+1 item scale with adequate metric properties measured by Alpha Chronbach, Goodness of fit index, and satisfactory convergence validity around three factors named: productive interactions, new relational model and person’s self-management.

Conclusions:

IEXPAC allows measurement of the patient experience of chronic illness care. Together with other indicators, IEXPAC can determine the quality of care provided according to the Triple Aim framework, facilitating health systems reorientation towards integrated patient-centred care.

Keywords: patient-centred care, integrated care, chronic illness, measurement, patient engagement

Introduction

The prevalence of chronic conditions and multimorbidity is rising worldwide [1,2]. This growth has placed increasing demands on existing acute-oriented healthcare systems and has resulted in poor quality of care and deficient patient experiences as a consequence of the fragmentation and lack of coordination in the organisation and delivery of care for people living with chronic diseases [3].

Although many advances have been made in the treatment available to chronically ill patients, most patients are multimorbid with diverse clusters of chronic diseases, and current or potential complex health and social care needs. Management of these needs requires integral and proactive action and, in many cases, these complex patients do not receive the care recommended by evidence-based clinical practice guidelines [4].

Furthermore, there is also a lack of patients’ involvement and collaboration in the design and co-creation of health services, especially for those with chronic illnesses. As a consequence, the delivery of effective, high-quality chronic care requires a systemic transformation [5] that goes beyond merely adding new isolated interventions to the existing acute-focused healthcare system [6]. It requires patient engagement, widespread use of quality improvement methods and innovations in chronic care.

Two decades ago, Wagner et al. developed the Chronic Care Model [7], a framework for delivering care to patients with chronic conditions and for guiding quality improvement in chronic care. This model is focused on providing proactive, planned, integrated and patient-centred care. There is evidence that the Chronic Care Model improves clinical outcomes and experiences of chronically ill patients receiving care [8,9,10].

Cramm and collaborators have demonstrated that, over time, quality of care and changes therein translate into more positive experiences for patients with chronic conditions [11]. Therefore, patient experience can, if appropriately measured, indicate the quality of chronic illness care and can provide important information to improve quality of care, patient safety and clinical effectiveness [12].

Consistent with the Chronic Care Model, Glasgow et al. [13] developed the Patient Assessment of Chronic Illness Care scale (PACIC) to assess patient experience with chronic care delivery. This scale has been used internationally among patients with a variety of chronic health conditions, and has been adapted and validated in many countries. In a systematic review, Vrijhoef et al [14] identified it as the most applicable and relevant questionnaire for measuring the quality of integrated chronic care from the patient’s perspective. Recently, Singer et al [15,16] developed the Patient Perceptions of Integrated Care survey (PPIC), assessing a six-dimension model of integrated care. PACIC and PPIC focus on the experience between patients and the doctors and nurses who regularly provide their care. These instruments do not incorporate elements related to ICT developments in chronic care and do not directly assess the coordination between health care and social care providers.

Many new integrated care models are built to incorporate a patient’s narrative of needs, preferences and expectations [17,18], acknowledging the essential role they have in their own care and the need for truly patient-centred care [19]. However, there is a dearth of accepted metrics [20].

Our group previously developed IEMAC-ARCHO, a self-assessment tool of readiness for chronicity in healthcare organisations [21,22]. Subsequently, the need to develop an instrument to assess patient experience of chronic integrated care was identified for the following reasons:

  • – To incorporate new theories, frameworks and trends in health care, such as the Triple Aim [23], the narratives of ‘person-centred coordinated care’ [24] or coproduction approaches [25] that are emphasising the importance of patient experience;
  • – To incorporate a broad notion of integrated care, including social care and patient self-management;
  • – To include increasingly popular technological innovations that are transforming the interaction between patients and the system of care;
  • – To consider the epidemiological situation, characterised by high prevalence of chronic conditions and multimorbidity [26].
  • – To take into account the interaction with a team (or network) of providers instead of focusing on separate professionals (interactions with doctors, nurses, etc.) [27];
  • – To specifically address the concept of the “patient experience”, separating it from that of patient satisfaction considering the approaches and outcomes obtained by Michelle Beattie [28], Cramm [29,30] and Wensing [31]
  • – To complement the aforementioned tool with another that incorporates the patient perspective.

Therefore, the purpose of this paper is to describe the process of development and validation of a new tool to measure self-reported patient experience of integrated chronic care.

Theory and methods

This is a design and validation study of a new instrument to assess the experience of patients with chronic conditions, who, because of their health status, have continuous interactions with social and health care professionals and services. The new tool is theoretically based on the Chronic Care Model and is inspired by patient-centred integrated care approaches.

The tool is intended to be used routinely to assess the patients experience of chronic illness care. For this reason, the following characteristics have been prioritised in its design [28]: small size (affordable), focused on the areas which patients consider important (appropriate), elements that support the processes of transformation and attention to chronicity (sensitive), orientation to what happens during the interaction with professionals (relevant), easy to understand (simple), a limited selection of elements (feasibility), suitable in any context (adaptable), and well-founded to ensure its psychometric properties (valid and reliable).

In this study, patient experience is defined as the information that the person facilitates on what has happened (to her) in her continued interaction with the health and social care professionals and services and on how she has lived that interaction and its outcomes. Meanwhile integrated care was conceptualised according to the Chronic Care Model and its subsequent adaptation by the WHO [3,7].

The steps followed in this study are shown in Figure Figure11.

Figure 1
Summarises the steps followed in this study.

Literature search: Characteristics that should be analysed

The literature was reviewed to identify existing tools and their characteristics to evaluate patient experience with integrated chronic care. A scoping review was carried out using MEDLINE and Web-of-Knowledge. Only studies in English or Spanish published in the last ten years (until January 2015) were included considering the availability of a previous high quality systematic review conducted by Vrijhoef et al [14]. “Patient Experience” and “Patient Perceptions” with “Integrated Care” or “Chronic Care” were used as search terms. References from retrieved articles were examined to locate further studies. A total of 58 articles were found, their abstracts were revised and 18 of them were fully reviewed and incorporated as relevant sources of information. The results were initially analysed, structured and made available to all members of the research team.

Selection and formulation of reactive items

Based on the previous results, the research team developed, by consensus, a pool of 28 reactive items. This set included a minimum of two items for each characteristic of care identified as relevant. Items were elaborated by the research team in successive work-team sessions considering: patient experience dimensions identified in the literature, as well as IEMAC/ARCHO [21,22] dimensions and interventions.

Content validity: Expert panel

An expert panel was carried out from September to November 2014 using an online survey involving 15 professionals from primary care, public health, social services, management, quality and safety boards and research institutions. The selection of the participants was based on their knowledge and expertise, each having at least 15 years of experience in clinical or managerial positions. They were recruited by personal contact.

Experts evaluated: content validity (redundancies, absences, misleading questions), face validity (understanding, friendliness, adequacy, ordinal structure), relevance to justify items’ inclusion, adequacy of the type of response scale and the instrument as a whole. The results of their answers prompted some changes in the reactive items to be explored in this new instrument.

Pre-test

Two pilot groups, each of 18 patients with chronic conditions, were conducted in December 2014. Patients were recruited through patient organisations for a variety of health conditions and had different characteristics (age, gender, socioeconomic). These patients evaluated the 28 reactive items of the questionnaire, regarding appropriateness, readability, acceptability and necessary time of response for each item, as well as two possible types of response scales. These patients considered the comprehension of this 28-item questionnaire as very good (4.8/5) as well as appropriate (4.6/5). The average time of response was 15 minutes. As a result, the wording of seven items was modified to improve understanding.

Reliability and validity: Field study

To establish the psychometric properties of the instrument and select those items with the best behaviour, a field study was performed in April-May 2015 with the participation of 350 patients (sample size calculated for a p=q=0.50, 5% error and a level of significance for two queues of 0.05). These were patients older than 16 years of age, with at least one chronic disease, who visited general medicine or nursing consultants at 11 primary care centres of four regional health services in Spain (Catalonia, Madrid, Basque Country and Valencia). Among those who met the inclusion criteria, patients were recruited by the interviewer by random systematic cluster sampling with proportional allocation (K=3). The questionnaire was self-administered, and only at the request of the patient, applied by means of a personal interview. Patients interviewed were informed of the purpose of the study and informed consent was obtained. The demographic and clinical variables of the study were collected in a booklet of data collection (BDC) designed for this study. The interviewers received a briefing on the selection procedure of the patients to be interviewed and on the correct application of the questionnaire. Twelve patients declined to answer.

In the analysis, the ceiling-floor effects and the correlation of each item with the scale total score were considered, where values above 0.30 were acceptable [32]. To establish the construct validity, a preliminary study to determine the unidimensionality of the factors was conducted through the use of an exploratory factorial analysis (EFA) using principal components, with Varimax rotation of the resulting array. To remove factors, criteria were applied using Eigenvalues equal to 1 (calculating previously the statisticians Kaiser-Meyer-Olkin and the Bartlett’s test to determine the appropriateness of performed EFA). Factorial loads higher than 0.50 were considered as acceptable [33]. The internal consistency reliability of a first version of the instrument was calculated using the statistical Cronbach’s Alpha, assuming acceptable values equal to or greater than 0.70 [32,33].

Subsequently, a confirmatory factorial analysis (CFA) was carried out, using all data and a random selection (N=115) of data from patients receiving health care in different health services, to confirm the hypotheses concerning the underlying structure generated by the exploratory factorial analysis and to rate the ‘goodness of fit’. This analysis was performed using the three-factor model that was derived from EFA to verify that the isolated factors finally had not changed their structure and that statistics employed in the exploratory analysis remained satisfactory. To check the measurement model validity, the standardized root mean square residual (SRMR), the Jöreskog-Sörbom goodness of fit index (GFI), the normed fix index (NFI), and the comparative fix index (CFI) were used. Pearson correlations between factors were also calculated to check the factors’ independence.

The quality criteria for measurement properties of health status instruments proposed by Terwee et al. [34] were considered to assess the acceptability of the questionnaire’s elements.

The ability of the questionnaire to discriminate between isolated dimensions was tested by performing t-test, ANOVA or Chi-Square, because differences in scores were expected based on differences in care delivery.

Ethical approval

The protocol of the study was approved by the Ethics Committee of the University Miguel Hernández, the institution who coordinated the study, and the Madrid Health Service Research Central Commission.

Results

Participants

Three hundred thirty-eight patients responded to the questionnaire (response rate 96.6%). Table Table11 depicts their characteristics.

Table 1
Descriptive Characteristics Display.

Items analysis

Seven items were ruled out for their ceiling-floor effects. The remaining 21 were included in the subsequent analysis, after verifying that there was acceptable variability in the answers of the patients.

The values of the Alpha of Cronbach were calculated by eliminating each item in an individualised way. No items were ruled out considering these data. The values of the correlations item-total ranged between 0.18 and 0.61. Three items with correlations inferior to 0.30 were ruled out before applying the technique of the exploratory factorial analysis.

Explorative analysis of dimensionality and reliability: Exploratory factor analysis (EFA)

A first factorial solution, with 15 items, joined together in five first order factors, with a principal factor explaining the 51.6% of the variance, and four items with significant saturations in more than one factor. In the following exploratory factorial analysis, 11 items were included, each with factorial saturations higher than 0.5 and commonalities between 0.43 and 0.70. This factorial solution converged in three factors, explaining the 57.5% of the common variance (Table (Table2).2). Based on our observation, Factor 1, named “Productive Interactions”, refers to the characteristics and content of interactions between patients and professionals oriented to improve outcomes, for example the professionals who care for me listen to me and ask me about my needs/habits and preferences and they are concerned with my quality of life. Factor 2, named “New Relational Model”, refers to new forms of patient interaction with the health care system, through the internet or with peers. Factor 3, named “Patient Self-Management”, refers to the ability of individuals to manage their own care and improve their wellbeing based on professional-mediated interventions.

Table 2
Exploratory factorial analysis.

Confirmatory analysis of dimensionality and analysis of reliability: Confirmatory factorial analysis (CFA)

The confirmatory factor analysis (CFA), in the second stage of the study, indicated an acceptable fit to the data. The estimates of the parameters and factor loadings of the model are shown in Table Table33 and Figure Figure2.2. This figure also shows the optimised model of the questionnaire factorial structure based on confirmatory factor analysis.

Table 3
Confirmatory factorial index.
Figure 2
Optimised model of the questionnaire factorial structure based on confirmatory factor analysis from the validation study carried out.

Composite reliability and convergent validity

The analysis of the convergent validity was satisfactory. All standardised loads were found to be significant for the respective factor and to be greater than 0.6. The average variance extracted was greater than 0.5 [35]. The composite reliability indexes were greater than 0.7 [36], indicating acceptable reliability for all factors (Table (Table44).

Table 4
Reliability, dimensionality, and convergent validity of each factor of the questionnaire.

Discriminant analysis

In the discriminant analysis, Table Table55 shows the inter-correlations between the three factors identified in the analyses. The factors showed acceptable independence to each other. PI with PSM showed higher values.

Table 5
Inter-correlations between the latent factors.

Internal consistency

The value of the Alpha Chronbach was 0.76 for the whole scale (0.79 in factor 1, 0.56 in factor 2 and 0.63 in factor 3).

Scale scores

The average punctuation in the IEXPAC-11 items was 3.1 points (SD 0.7, IC95% 3.0-3.2). The average punctuation in each factor was 4.0 points (SD 0.9) in factor one, 1.7 (SD 0.9) in factor two and 3.7 (SD 0.9) in factor three. Table Table66 shows the results based on a set of variables.

Table 6
IEXPAC factors scores.

The new tool was named IEXPAC, Instrument for Evaluation of the Experience of Chronic Patients (available online at http://www.iemac.es/iexpac/). Additionally, an item was included for specific cases when a patient is hospitalised, to check the continuity of care during discharge and once the patient returns home. This item (number 12) is not included in the scale aggregate rating. The average score for this item was 2.5 (SD 1.7).

Discussion

The developed scale is an instrument to obtain a reliable and valid measure of the experience of patients with chronic conditions during their interaction with the system of care. IEXPAC has a condensed set of measured items so that it can be used routinely and systematically in care services for assessing whether patients perceive is receiving integrated care, have a positive relationship with the set of professionals which usually interacts with, feels abler to look after their health and, are involved in new ways of non-face-to face interactions.

Specifically, IEXPAC assesses patient experience in accordance with the Triple Aim framework. Triple Aim shows the importance of the “experience of care”, an inconsistently measured dimension. Most health systems do not regularly assess the experience of care. This instrument could be used online (http://iexpac.es) paper-and-pencil or phone to assess patient experience of care in several contexts such as a health centre, health district or health service.

Patient experience represents a unique encompassing dimension that is challenging to measure. Lacking a widely accepted definition [37], we have rooted the development of this scale in the Chronic Care Model theoretical framework, enriched with approaches coming from patient-centred care [17,18] and service coproduction theories [25]. The relationship between patient experience and quality of care is not consistent in the literature. Major studies have rendered different, even opposite, results [12,38,39], whose likely explanation has been analysed by Manary [40]. In many contexts, the established patient satisfaction terminology is being substituted for the rather new ‘patient experience’, as if they can be used interchangeably. Patient experience with chronic care, as captured with the IEXPAC scale, however, differs significantly from the traditional patient satisfaction with episodes of care. This is why, in developing IEXPAC, we put emphasis on clarifying and delimiting the concept we want to measure, as formulated in our definition of patient experience. In the past, the concept of satisfaction has been used in a bilateral way, capturing the interaction of a patient with a single professional or organization. New measures should be focused on gauging experience with more comprehensive and complex provision models where different organizations are working collaboratively to provide patient-centred care.

The IEXPAC scale has several strengths from an integrated care point of view. First, it assesses experience of care beyond punctual contact, episode or specific setting. It is expected to capture a continuous experience over six months. Second, it considers the health professionals as a team, not solely focused on individual or isolated interaction with physicians, nurses or other staff. In countries like Spain, care delivery is not only focused on “doctors”, but on a comprehensive team, where different primary care professionals interact with patients and different providers from hospitals, mental health networks, long term care facilities or social care organisations. Other authors, such as Walker et al have also acknowledged that patients “highly value a sense of all members of the care team being on the same page”. Third, the scale captures the relationship between the patient and a system of care, where self-management and social care are also relevant. Fourth, the scale incorporates an active patient role, having a clear orientation towards improving outcomes by means of patients and professionals working together (coproduction). Fifth, IEXPAC is aligned with new evaluative frameworks of population health management based on the Triple Aim vision.

The psychometric analysis of IEXPAC renders three independent factors with items that converge around concepts: productive interactions, new relational model and patient self-management, all with literature supporting their adequacy and soundness [41].

This instrument also has some limitations that should be considered. There is still no data to support whether improvements in the scale ratings are related to better clinical or health-related quality of life outcomes. Furthermore, as patients with different chronic diseases or patients in different settings have distinct experiences with chronic illness care, there may be a need for specific scales for certain chronic diseases or types of complex chronic patients, such as those in home care programmes and those who are assisted by caregivers. Finally, the way the scale is formulated does not allow to attribute responsibilities to a specific care provider at individual level, only to a team of providers or “system of care”.

To the best of our knowledge, most national health systems are not capturing the integrated care experience of patients with chronic conditions in a regular, holistic and systematic way. Most countries have a range of measures related to health outcomes and cost or utilization of services, typically included in most national or regional outcome frameworks. There is appropriate to also develop experience of care measures and incorporate them into national and regional-level integrated care models. There is a promising future for these metrics through the commissioning by health and social care authorities and through performance assessments. Relevant initiatives are expected to appear in the coming years in this field of knowledge [42,43]. For example, the area 4 (Ensuring that people have a positive experience of care) in the English NHS Outcome Framework 2015/2016 [44], may include the assessment of experience of care from an integrated care perspective. Tools like IEXPAC should contribute to this aim.

Conclusions

IEXPAC scale measures the experience of patients with chronic conditions in their continued interaction with health professionals and services in regular practice. It can ascertain the quality of care experienced by patients, contributing to the ‘experience of care’ axis of the Triple Aim, and facilitate the adoption of patient-centred care approaches by health and social care organisations.

Measurement of patient experience may also facilitate the reorientation towards patient-centred care. Presently, with numerous processes of service integration being deployed, this might be of particular importance. It is necessary to generate results that consolidate this type of measurement, showing its correlation with other outcome indicators whose relevance and usefulness are widely accepted in the literature.

Although IEXPAC has yet to prove it can be used in regular practice, it seems that new metrics like this will be welcomed and possibly incorporated into regular performance assessments or commissioning processes of health and social care.

Acknowledgements

We thank Itziar Pérez-Irazusta (Basque Health Service), Ana Miquel (Madrid Health Service), Rafael Ruiz-Riera (Catalonian Health Service), Eduardo Zafra (Valencian Health Service), Alberto Asencio (Valencian Health Service), and Manuel Sánchez (Valencian Health Service) for facilitating the field study in their Regional Health Services; Ester Sarquella (PIAISS, Catalonian Integrated Health and Social Services Plan), Sage Show and Elena Urízar (Deusto Business School Health) for their insightful comments.

Reviewers

Two anonymous reviewers.

Competing Interests

IEXPAC is provided free for non-profit organizations that want to improve chronic patient care. PF works at Merck Sharp & Dohme Spain. This study has been funded by Merck Sharp & Dohme Spain.

Authors’ contributions

JJM, RN, MG, OS designed the study. MG and OS conducted qualitative techniques. MG, OS, JCC, AG, MR and the IEXPAC development group enrolled centres and patients. OS coordinated the field study. JJM and MG performed statistical analysis. JJM, PF and RN drafted this original. All authors reviewed information sources, participated in reviewing and interpreting data and read and approved the final manuscript.

References

1. Murray CJ, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease study 2010. Lancet. 2012;380(9859):2197–2223. doi: 10.1016/S0140-6736(12)61689-4. [PubMed] [Cross Ref]
2. Tinetti ME, Fried TR, Boyd CM. Designing health care for the most common chronic condition-multimorbidity. The Journal of the American Medical Association. 2012;307(23):2493–2494. doi: 10.1001/jama.2012.5265. [PMC free article] [PubMed] [Cross Ref]
3. Nuño R, Coleman K, Bengoa R, Sauto R. Integrated care for chronic conditions: the contribution of the ICCC Framework. Health Policy. 2012;105(1):55–64. doi: 10.1016/j.healthpol.2011.10.006. [PubMed] [Cross Ref]
4. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of health care delivered to adults in the United States. The New England Journal of Medicine. 2003;348(26):2635–2645. doi: 10.1056/NEJMsa022615. [PubMed] [Cross Ref]
5. Bengoa R. Transforming health care: an approach to system-wide implementation. International Journal of Integrated Care. 2013;13(3) doi: 10.5334/ijic.1206. [PMC free article] [PubMed] [Cross Ref]
6. Ham C. The ten characteristics of the high-performing chronic care system. Health Economics, Policy and Law. 2010;5(Pt 1):71–90. doi: 10.1017/S1744133109990120. [PubMed] [Cross Ref]
7. Wagner EH, Austin BT, Von Korff M. Organizing care for patients with chronic illness. The Milbank Quarterly. 1996;74(4):511–544. doi: 10.2307/3350391. [PubMed] [Cross Ref]
8. Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the Chronic Care Model in the new millennium. Health Affairs (Millwood) 2009;28(1):75–85. doi: 10.1377/hlthaff.28.1.75. [PMC free article] [PubMed] [Cross Ref]
9. Martínez-González NA, Berchtold P, Ullman K, Busato A, Egger M. Integrated care programmes for adults with chronic conditions: a meta-review. International Journal for Quality in Health Care. 2014;26(5):561–570. doi: 10.1093/intqhc/mzu071. [PMC free article] [PubMed] [Cross Ref]
10. Davy C, Bleasel J, Liu H, Tchan M, Ponniah S, Brown A. Effectiveness of chronic care models: opportunities for improving healthcare practice and health outcomes: a systematic review. BMC Health Services Research. 2015;15(15) doi: 10.1186/s12913-015-0854-8. [PMC free article] [PubMed] [Cross Ref]
11. Cramm JM, Nieboer AP. High-quality chronic care delivery improves experiences of chronically ill patients receiving care. International Journal for Quality in Health Care. 2013;25(6):689–695. doi: 10.1093/intqhc/mzt065. [PMC free article] [PubMed] [Cross Ref]
12. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1) doi: 10.1136/bmjopen-2012-001570. [PMC free article] [PubMed] [Cross Ref]
13. Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM. Development and validation of the Patient Assessment of Chronic Illness Care (PACIC) Medical Care. 2005;43(5):436–444. doi: 10.1097/01.mlr.0000160375.47920.8c. [PubMed] [Cross Ref]
14. Vrijhoef HJ, Berbee R, Wagner EH, Steuten LM. Quality of integrated chronic care measured by patient survey: identification, selection and application of most appropriate instruments. Health Expectations. 2009;12(4):417–429. doi: 10.1111/j.1369-7625.2009.00557.x. [PMC free article] [PubMed] [Cross Ref]
15. Singer SJ, Friedberg MW, Kiang MV, Dunn T, Kuhn DM. Development and validation of the Patient Perceptions of Integrated Care Survey. Medical Care Research and Review. 2012;70(2):143–164. doi: 10.1177/1077558712465654. [PubMed] [Cross Ref]
16. Singer SJ, Burgers J, Friedberg M, Rosenthal MB, Leape L, Scheneider E. Defining and measuring integrated patient care: promoting the next frontier in health care delivery. Medical Care Research and Review. 2011;68(1):112–127. doi: 10.1177/1077558710371485. [PubMed] [Cross Ref]
17. National Quality Board . London: NHS; 2015. Jan, Improving experience of care: Our shared understanding and ambition. [cited 2016 31 Jan]. Available from: https://www.england.nhs.uk/wp-content/uploads/2015/01/improving-experiences-of-care.pdf .
18. World Health Organization . Geneva: WHO; 2015. Mar, WHO global strategy on people-centred and integrated health services. [cited 2016 31 Jan]. Available from: http://www.who.int/servicedeliverysafety/areas/people-centred-care/global-strategy/en/
19. Coulter A. After Bristol: putting patients at the centre. BMJ. 2002;324(7338):648–651. doi: 10.1136/bmj.324.7338.648. [PMC free article] [PubMed] [Cross Ref]
20. Walker KO, Labat A, Choi J, Schmittdiel J, Stewart AL, Grumbach K. Patient perceptions of integrated care: confused by the term, clear on the concept. International Journal of Integrated Care. 2013;13(1):e004. doi: 10.5334/ijic.985. [PMC free article] [PubMed] [Cross Ref]
21. Nuño-Solinís R, Fernández-Cano P, Mira-Solves JJ, Toro-Polanco N, Contel JC, Guilabert-Mora M, et al. Development of an instrument for the assessment of chronic care models. Gaceta Sanitaria. 2013;27(2):128–134. doi: 10.1016/j.gaceta.2012.05.012. [in Spanish] [PubMed] [Cross Ref]
22. Mira JJ, Nuño-Solinís R, Fernández-Cano P, Toro-Polanco N, Contel JC, Guilabert-Mora M, et al. Readiness to tackle chronicity in Spanish health care organisations: a two-year experience with the Instrumento de Evaluación de Modelos de Atención ante la Cronicidad/Assessment of Readiness for Chronicity in Health Care Organisations instrument. International Journal of Integrated Care. 2015;15(4) doi: 10.5334/ijic.1849. [PMC free article] [PubMed] [Cross Ref]
23. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Affairs (Millwood) 2008;27(3):759–769. doi: 10.1377/hlthaff.27.3.759. [PubMed] [Cross Ref]
24. The Evidence Centre for National Voices . London: National Voices; 2014. Mar, Enhancing experience. Summarising evidence from systematic reviews. [cited 2016 31 Jan]. Available from: http://www.nationalvoices.org.uk/node/1335 .
25. Batalden M, Batalden P, Margolis P, Seid M, Armstrong G, Opipari-Arrigan L, et al. Coproduction of healthcare service. BMJ Quality & Safety. 2016;25(7):509–517. doi: 10.1136/bmjqs-2015-004315. [PMC free article] [PubMed] [Cross Ref]
26. Orueta JF, Garcia-Alvarez A, Garcia-Goni M, Paolucci F, Nuno-Solinis R. Prevalence and costs of multimorbidity by deprivation levels in the Basque Country: a population based study using health administrative databases. PloS One. 2014;9(2) doi: 10.1371/journal.pone.0089787. [PMC free article] [PubMed] [Cross Ref]
27. Drewes H, van Til-de Jong J, Struijs J, Baan C, Tekle F, Meijboom B, et al. Measuring chronic care management experience of patients with diabetes: PACIC and PACIC+ validation. International Journal of Integrated Care. 2012;12(8) doi: 10.5334/ijic.862. [PMC free article] [PubMed] [Cross Ref]
28. Beattie M, Murphy DJ, Atherton I, Lauder W. Instruments to measure patient experience of healthcare quality in hospitals: a systematic review. Systematic Reviews. 2015;4:97. doi: 10.1186/s13643-015-0089-0. [PMC free article] [PubMed] [Cross Ref]
29. Cramm JM, Nieboer AP. The importance of productive patient–professional interaction for the well-being of chronically ill patients. Quality of Life Research. 2015;24(4):897–903. doi: 10.1007/s11136-014-0813-6. [PMC free article] [PubMed] [Cross Ref]
30. Cramm JM, Jolani S, van Buuren S, Nieboer AP. Better experiences with quality of care predict well-being of patients with chronic obstructive pulmonary disease in the Netherlands. International Journal of Integrated Care. 2015;15(2) doi: 10.5334/ijic.1587. [PMC free article] [PubMed] [Cross Ref]
31. Wensing M, Vingerhoets E, Grol R. Feedback based on patient evaluations: a tool for quality improvement? Patient Education and Counseling. 2003;51(2):149–153. doi: 10.1016/S0738-3991(02)00199-4. [PubMed] [Cross Ref]
32. McDonald RP. The dimensionality of tests and items. British Journal of Mathematical Statistical Psychology. 1981;34(1):100–117. doi: 10.1111/j.2044-8317.1981.tb00621.x. [Cross Ref]
33. Hair JF, Anderson RE, Tatham RL, Black WC. 5th edition. New York: Prentice Hall International; 1998. Multivariate data analysis.
34. Terwee CB, Bot SD, de Boer M, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status instruments. Journal of Clinical Epidemiology. 2007;60(1):34–42. doi: 10.1016/j.jclinepi.2006.03.012. [PubMed] [Cross Ref]
35. Bagozzi RP, Yi Y. On the evaluation of structural equation models. Journal of the Academy of Marketing Science. 1988;16(1):74–94. doi: 10.1007/BF02723327. [Cross Ref]
36. Nunnally JC, Bernstein H. 3rd edition. New York: McGraw-Hill; 1994. Psychometric theory.
37. LaVela SL, Gallan AS. Evaluation and measurement of patient experience. Patient Experience Journal. 2014;1(1):28–36. Available from: http://pxjournal.org/journal/vol1/iss1/5 .
38. Jha AK, Orav EJ, Zheng J, Epstein AM. Patients’ Perception of Hospital Care in the United States. The New England Journal of Medicine. 2008;359:1921–1931. doi: 10.1056/NEJMsa0804116. [PubMed] [Cross Ref]
39. Groene O, Arah OA, Klazinga NS, Wagner C, Bartels PD, Kristensen S, et al. Patient Experience Shows Little Relationship with Hospital Quality Management Strategies. PLoS One. 2015;10(7) doi: 10.1371/journal.pone.0131805. [PMC free article] [PubMed] [Cross Ref]
40. Manary MP, Boulding W, Staelin R, Glickman S. The Patient Experience and Health Outcomes. The New England Journal of Medicine. 2013;368(3):201–203. doi: 10.1056/NEJMp1211775. [PubMed] [Cross Ref]
41. Nuño-Solinís R. [Development and implementation of the Chronicity Strategy for the Basque Country (Spain): lessons learned] Gaceta Sanitaria. 2016 doi: 10.1016/j.gaceta.2016.03.011. [in Spanish] [PubMed] [Cross Ref]
42. Staniszewska S, Churchill N. Patients’ experiences in the UK: Future strategic directions. Patient Experience Journal. 2014;1(1):140–143. Available from: http://pxjournal.org/journal/vol1/iss1/18 .
43. Coulter A, Locock L, Ziebland S, Calabrese J. Collecting data on patient experience is not enough: they must be used to improve care. BMJ. 2014;348 doi: 10.1136/bmj.g2225. [PubMed] [Cross Ref]
44. Department of Health . London: NHS; 2014. Dec, The NHS Outcomes Framework 2015/2016. [cited 2016 31 Jan]. Available from: https://www.gov.uk/government/publications/nhs-outcomes-framework-2015-to-2016 .

Articles from International Journal of Integrated Care are provided here courtesy of Ubiquity Press