The operational definition of relational continuity is “a therapeutic relationship between a patient and one or more providers that spans various healthcare events and results in accumulated knowledge of the patient and care consistent with the patient's needs.”
To examine how well relational continuity is measured in validated instruments that evaluate primary healthcare from the patient's perspective.
645 adults with at least one healthcare contact in the previous 12 months responded to six instruments that evaluate primary healthcare. Five subscales map to relational continuity: the Primary Care Assessment Survey (PCAS, two subscales), the Primary Care Assessment Tool – Short Form (PCAT-S) and the Components of Primary Care Index (CPCI, two subscales). Scores were normalized for descriptive comparison. Exploratory and confirmatory (structural equation modelling) factor analysis examined fit to operational definition, and item response theory analysis examined item performance on common constructs.
All subscales load reasonably well on a single factor, presumed to be relational continuity, but the best model has two underlying factors corresponding to (1) accumulated knowledge of the patient and (2) relationship that spans healthcare events. Some items were problematic even in the best model. The PCAS Contextual Knowledge subscale discriminates best between different levels of accumulated knowledge, but this dimension is also captured well by the CPCI Accumulated Knowledge subscale and most items in the PCAT-S Ongoing Care subscale. For relationship-spanning events, the items' content captures concentration of care in one doctor; this is captured best by the CPCI Preference for Regular Provider subscale and, to a lesser extent, by the PCAS Visit-Based Continuity subscale and one relevant item in the PCAT-S Ongoing Care subscale. But this dimension correlates only modestly with percentage of reported visits to the personal doctor. The items function as yes/no rather than ordinal options, and are especially informative for poor concentration of care.
These subscales perform well for key elements of relational continuity, but do not capture consistency of care. They are more informative for poor relational continuity.
Coping strategies are among the psychosocial factors hypothesized to contribute to the development of chronic musculoskeletal disability. The Chronic Pain Coping Inventory (CPCI) was developed to assess eight behavioral coping strategies targeted in multidisciplinary pain treatment (Guarding, Resting, Asking for Assistance, Task Persistence, Relaxation, Exercise/Stretch, Coping Self-Statements and Seeking Social Support). The present study had two objectives. First, it aimed at measuring the internal consistency and the construct validity of the French version of the CPCI. Second, it aimed to verify if, as suggested by the CPCI authors, the scales of this instrument can be grouped according to the following coping families: Illness-focused coping and Wellness-focused coping.
The CPCI was translated into French with the forward and backward translation procedure. To evaluate internal consistency, Cronbach's alphas were computed. Construct validity of the inventory was estimated through confirmatory factor analysis (CFA) in two samples: a group of 439 Quebecois workers on sick leave in the sub-acute stage of low back pain (less than 84 days after the work accident) and a group of 388 French chronic pain patients seen in a pain clinic. A CFA was also performed to evaluate if the CPCI scales were grouped into two coping families (i.e. Wellness-focused and Illness-focused coping).
The French version of the CPCI had adequate internal consistency in both samples. The CFA confirmed the eight-scale structure of the CPCI. A series of second-order CFA confirmed the composition of the Illness-focused family of coping (Guarding, Resting and Asking for Assistance). However, the composition of the Wellness-focused family of coping (Relaxation, Exercise/Stretch, Coping Self-Statements and Seeking Social Support) was different than the one proposed by the authors of the CPCI. Also, a positive correlation was observed between Illness and Wellness coping families.
The present study indicates that the internal consistency and construct validity of the French version of the CPCI were adequate, but the grouping and labeling of the CPCI families of coping are debatable and deserve further analysis in the context of musculoskeletal and pain rehabilitation.
Comprehensiveness relates both to scope of services offered and to a whole-person clinical approach. Comprehensive services are defined as “the provision, either directly or indirectly, of a full range of services to meet most patients' healthcare needs”; whole-person care is “the extent to which a provider elicits and considers the physical, emotional and social aspects of a patient's health and considers the community context in their care.” Among instruments that evaluate primary healthcare, two had subscales that mapped to comprehensive services and to the community component of whole-person care: the Primary Care Assessment Tool – Short Form (PCAT-S) and the Components of Primary Care Index (CPCI, a limited measure of whole-person care).
To examine how well comprehensiveness is captured in validated instruments that evaluate primary healthcare from the patient's perspective.
645 adults with at least one healthcare contact in the previous 12 months responded to six instruments that evaluate primary healthcare. Scores were normalized for descriptive comparison. Exploratory and confirmatory (structural equation modelling) factor analysis examined fit to operational definition, and item response theory analysis examined item performance on common constructs.
Over one-quarter of respondents had missing responses on services offered or doctor's knowledge of the community. The subscales did not load on a single factor; comprehensive services and community orientation were examined separately. The community orientation subscales did not perform satisfactorily. The three comprehensive services subscales fit very modestly onto two factors: (1) most healthcare needs (from one provider) (CPCI Comprehensive Care, PCAT-S First-Contact Utilization) and (2) range of services (PCAT-S Comprehensive Services Available). Individual item performance revealed several problems.
Measurement of comprehensiveness is problematic, making this attribute a priority for measure development. Range of services offered is best obtained from providers. Whole-person care is not addressed as a separate construct, but some dimensions are covered by attributes such as interpersonal communication and relational continuity.
The development of a pain-management program tailored to the specific needs of patients with chronic low back pain (CLBP) requires the proper assessment of psychosocial factors affecting each individual. The Chronic Pain Coping Inventory-42 (CPCI-42) refers to coping strategies, which are commonly defined as the cognitive and behavioral techniques an individual may resort to in stressful or demanding situations. Evidence from a number of sources suggests that differences in pain coping strategies may significantly affect how an individual deals with chronic pain. We aimed to adapt the CPCI-42 to Polish cultural conditions (PL-CPCI-42) and then verify its psychometric properties based on a group of patients treated surgically due to herniated lumbar discs and coexisting spondylotic changes.
The average age of the study participants (n=90) was 43.47 years (SD 10.21). The average duration of chronic low back pain (CLBP) was 49.37 months (SD 64.71). Lumbosacral spine X-rays and magnetic resonance imaging scans were performed and all patients completed the PL-CPCI-42 and the Polish versions of the Numeric Pain Rating Scale (NPRS-PL) twice. Internal consistency of the PL-CPCI-42, floor and ceiling effects, test-retest reliability, and criterion validity were analyzed.
Resting, guarding, and coping self-statements were frequently used as coping strategies both in the test and in the retest, in contrast to relaxation and exercise/stretch. The NPRS-PL result was 5.70 cm in the test and 5.66 in the retest. Cronbach’s alpha values were recorded for the asking for assistance, coping self-statements, and seeking social support domains (0.83, 0.80, 0.83, respectively). Test-retest reliability of the PL-CPCI-42 varied from 0.53 (relaxation domain) to 0.84 (asking for assistance and coping self-statements domains).
The present study provides evidence of the validity of the PL-CPCI-42 and supports its usefulness in assessing chronic pain coping strategies, which are especially important to pain adjustment and in the creation of multidisciplinary pain management programs for patients with severe CLBP.
Validation Study; CPCI-42; Chronic Pain Coping Styles; Lumbar Disc Herniation; Chronic Pain - prevention & control
Patient evaluations are an important part of monitoring primary healthcare reforms, but there is little comparative information available to guide evaluators in the choice of instruments or to determine their relevance for Canada.
To compare values and the psychometric performances of validated instruments thought to be most pertinent to the Canadian context for evaluating core attributes of primary healthcare.
Among validated instruments in the public domain, we selected six: the Primary Care Assessment Survey (PCAS); the Primary Care Assessment Tool – Short Form (PCAT-S); the Components of Primary Care Index (CPCI); the first version of the EUROPEP (EUROPEP-I); the Interpersonal Processes of Care Survey, version II (IPC-II); and part of the Veterans Affairs National Outpatient Customer Satisfaction Survey (VANOCSS). We mapped subscales to operational definitions of attributes. All were administered to a sample of adult service users balanced by English/French language (in Nova Scotia and Quebec, respectively), urban/rural residency, high/low education and overall care experience. The sample was recruited from previous survey respondents, newspaper advertisements and community posters. We used common factor analysis to compare our factor resolution for each instrument to that of the developers.
Our sample of 645 respondents was approximately balanced by design variables, but considerable effort was required to recruit low-education and poor-experience respondents. Subscale scores are statistically different by excellent, average and poor overall experience, but interpersonal communication and respectfulness scores were the most discriminating of overall experience. We found fewer factors than did the developers, but when constrained to the number of expected factors, our item loadings were largely similar to those found by developers. Subscale reliability was equivalent to or higher than that reported by developers.
These instruments perform similarly in the Canadian context to their original development context, and can be used with confidence. Interpersonal and respectfulness scores are most discriminating of excellent, average or poor overall experience and are crucial dimensions of patient evaluations.
Management continuity, operationally defined as “the extent to which services delivered by different providers are timely and complementary such that care is experienced as connected and coherent,” is a core attribute of primary healthcare. Continuity, as experienced by the patient, is the result of good care coordination or integration.
To provide insight into how well management continuity is measured in validated coordination or integration subscales of primary healthcare instruments.
Relevant subscales from the Primary Care Assessment Survey (PCAS), the Primary Care Assessment Tool – Short Form (PCAT-S), the Components of Primary Care Instrument (CPCI) and the Veterans Affairs National Outpatient Customer Satisfaction Survey (VANOCSS) were administered to 432 adult respondents who had at least one healthcare contact with a provider other than their family physician in the previous 12 months. Subscales were examined descriptively, by correlation and factor analysis and item response theory analysis. Because the VANOCSS elicits coordination problems and is scored dichotomously, we used logistic regression to examine how evaluative subscales relate to reported problems.
Most responses to the PCAS, PCAT-S and CPCI subscales were positive, yet 83% of respondents reported having one or more problems on the VANOCSS Overall Coordination subscale and 41% on the VANOCSS Specialist Access subscale. Exploratory factor analysis suggests two distinct factors. The first (eigenvalue=6.98) is coordination actions by the primary care physician in transitioning patient care to other providers (PCAS Integration subscale and most of the PCAT-S Coordination subscale). The second (eigenvalue=1.20) is efforts by the primary care physician to create coherence between different visits both within and outside the regular doctor's office (CPCI Coordination subscale). The PCAS Integration subscale was most strongly associated with lower likelihood of problems reported on the VANOCSS subscales.
Ratings of management continuity correspond only modestly to reporting of coordination problems, possibly because they rate only the primary care physician, whereas patients experience problems across the entire system. The subscales were developed as measures of integration and provider coordination and do not capture the patient's experience of connectedness and coherence.
The operational definition of interpersonal communication is “the ability of the provider to elicit and understand patient concerns, to explain healthcare issues and to engage in shared decision-making if desired.”
To examine how well interpersonal communication is captured in validated instruments that evaluate primary healthcare from the patient's perspective.
645 adults with at least one healthcare contact in the previous 12 months responded to instruments that evaluate primary healthcare. Eight subscales measure interpersonal communication: the Primary Care Assessment Survey (PCAS, two subscales); the Components of Primary Care Index (CPCI, one subscale); the first version of the EUROPEP (EUROPEP-I); and the Interpersonal Processes of Care Survey, version II (IPC-II, four subscales). Scores were normalized for descriptive comparison. Exploratory and confirmatory (structural equation) factor analysis examined fit to operational definition, and item response theory analysis examined item performance.
Items not pertaining to interpersonal communication were removed from the EUROPEP-I. Most subscales are skewed positively. Normalized mean scores are similar across subscales except for IPC-II Patient-Centred Decision-Making and IPC-II Hurried Communication. All subscales load reasonably well on a single factor, presumed to be interpersonal communication. The best model has three underlying factors corresponding to eliciting (eigenvalue = 26.56), explaining (eigenvalue = 2.45) and decision-making (eigenvalue = 1.34). Both the PCAS Communication and the EUROPEP-I Clinical Behaviour subscales capture all three dimensions. Individual subscales within IPC-II measure each sub-dimension.
The operational definition is well reflected in the available measures, although shared decision-making is poorly represented. These subscales can be used with confidence in the Canadian context to measure this crucial aspect of patient-centred care.
Respectfulness is one measurable and core element of healthcare responsiveness. The operational definition of respectfulness is “the extent to which health professionals and support staff meet users' expectations about interpersonal treatment, demonstrate respect for the dignity of patients and provide adequate privacy.”
To examine how well respectfulness is captured in validated instruments that evaluate primary healthcare from the patient's perspective, whether or not their developers had envisaged these as representing respectfulness.
645 adults with at least one healthcare contact with their own regular doctor or clinic in the previous 12 months responded to six instruments, two subscales that mapped to respectfulness: the Interpersonal Processes of Care, version II (IPC-II, two subscales) and the Primary Care Assessment Survey (PCAS). Additionally, there were individual respectfulness items in subscales measuring other attributes in the Components of Primary Care Index (CPCI) and the first version of the EUROPEP (EUROPEP-I). Scores were normalized for descriptive comparison. Exploratory and confirmatory (structural equation modelling) factor analyses examined fit to operational definition.
Respectfulness scales correlate highly with one another and with interpersonal communication. All items load adequately on a single factor, presumed to be respectfulness, but the best model has three underlying factors corresponding to (1) physician's interpersonal treatment (eigenvalue=13.99), (2) interpersonal treatment by office staff (eigenvalue=2.13) and (3) respect for the dignity of the person (eigenvalue=1.16). Most items capture physician's interpersonal treatment (IPC-II Compassionate, Respectful Interpersonal Style, IPC-II Hurried Communication and PCAS Interpersonal Treatment). The IPC-II Interpersonal Style (Disrespectful Office Staff) captures treatment by staff, but only three items capture dignity.
Various items or subscales seem to measure respectfulness among currently available validated instruments. However, many of these items related to other constructs, such as interpersonal communication. Further studies should aim at developing more refined measures – especially for privacy and dignity – and assess the relevance of the broader concept of responsiveness.
Effective patient-centred health care requires internal participation, which is defined as interprofessional patient-centred teamwork. Many scales are designed for measuring teamwork from the perspective of one type of health care professional (e.g. physician or nurse), rather than for the use for all health care professionals as well as patients. Hence, this paper’s purpose is to develop a scale for measuring internal participation from all relevant perspectives and to check its psychometric properties.
In a multicentre cross-sectional study, a 6-item Internal Participation Scale (IPS) was developed and administered to 661 health care professionals (staff) and 1419 patients in 15 rehabilitation clinics to test item characteristics, acceptance, reliability (internal consistency) and construct validity. Additionally, we performed an exploratory factor analysis (EFA) to determine the factorial structure and explained variance. Confirmatory factor analysis (CFA) was used to verify the theoretically assumed one-dimensional factorial structure.
A total of 275 health care professionals and 662 patients participated, and the complete data sets of 272 staff members and 536 patients were included in the final analysis. The discrimination index was above .4 for all items in both samples. Internal consistency was very good, with Cronbach’s alpha equalling .87 for the staff and .88 for the patient sample. EFA supported a one-dimensional structure of the instrument (explained variance: 61.1% (staff) and 62.3% (patients)). CFA verified the factorial structure, with the factor loadings exceeding .4 for five of six items in both samples. Global goodness-of-fit indices indicated a good model fit, with a Tucker-Lewis index (TLI) of .974 (staff) and .976 (patients) and a comparative fit index (CFI) of .988 (staff) and .989 (patients). The root mean square error of approximation (RMSEA) amounted to .068 for the patient sample and .069 for the staff sample. There is evidence of construct validity for both populations.
The analysis of the scale’s psychometric properties resulted in good values. The scale is a promising instrument to assess internal participation from the perspective of both patients and staff. Further research should investigate the scale’s psychometric properties in other interprofessional health care settings to examine its generalizability as well as its sensitivity to change.
Internal participation; Interprofessional collaboration; Team; Psychometrics; Scale development; Healthcare; Germany
Improving patient safety has become a major focus of clinical care and research over the past two decades. An institution’s patient safety climate represents an essential component of ensuring a safe environment and thereby can be vital to the prevention of adverse events. Covering six patient safety related factors, the Safety Attitudes Questionnaire (SAQ) is a validated and widely used instrument to measure the patient safety climate in clinical areas. The objective of this study was to assess the psychometric properties of the German language version of the SAQ.
A survey was carried out in two University Hospitals in Switzerland in autumn 2009 where the SAQ was distributed to a sample of 406 nurses and physicians in medical and surgical wards. Following the American Educational Research Association guidelines, we tested the questionnaire validity by levels of evidence: content validity, internal structure and relations to other variables. Confirmatory factor analysis was used to examine factor structure. Cronbach’s alphas and inter-item correlations were calculated to examine internal consistency reliability.
A total of 319 questionnaires were completed representing an overall response rate of 78.6%. For three items, the item content validity index was <0.75. Confirmatory factor analysis showed acceptable model fit (RMSEA = 0.045; CFI = 0.944) for the six-factor model. Additional exploratory factor analysis could not identify a better factor model. SAQ factor scores showed positive correlations with the Safety Organizing Scale (r = .56 - .72). The SAQ German version showed moderate to strong internal consistency reliability indices (Cronbach alpha = .65 - .83).
The German language version of the SAQ demonstrated acceptable to good psychometric properties and therefore shows promise to be a sound instrument to measure patient safety climate in Swiss hospital wards. However, the low item content validity and large number of missing responses for several items suggest that improvements and adaptations in translation are required for select items, especially within the perception of management scale. Following these revisions, psychometric properties should reassessed in a randomly selected sample and hospitals and departments prior to use in Swiss hospital settings.
Safety attitudes questionnaire (SAQ); Patient safety; Psychometrics; Swiss hospital setting
Patient safety can be increased by improving the organization of care. A tool that evaluates the actual organization of care, as perceived by multidisciplinary teams, is the Care Process Self-Evaluation Tool (CPSET). CPSET was developed in 2007 and includes 29 items in five subscales: (a) patient-focused organization, (b) coordination of the care process, (c) collaboration with primary care, (d) communication with patients and family, and (e) follow-up of the care process. The goal of the present study was to further evaluate the psychometric properties of the CPSET at the team and hospital levels and to compile a cutoff score table.
The psychometric properties of the CPSET were assessed in a multicenter study in Belgium and the Netherlands. In total, 3139 team members from 114 hospitals participated. Psychometric properties were evaluated by using confirmatory factor analysis (CFA), Cronbach’s alpha, interclass correlation coefficients (ICCs), Kruskall-Wallis test, and Mann–Whitney test. For the cutoff score table, percentiles were used. Demographic variables were also evaluated.
CFA showed a good model fit: a normed fit index of 0.93, a comparative fit index of 0.94, an adjusted goodness-of-fit index of 0.87, and a root mean square error of approximation of 0.06. Cronbach’s alpha values were between 0.869 and 0.950. The team-level ICCs varied between 0.127 and 0.232 and were higher than those at the hospital level (0.071-0.151). Male team members scored significantly higher than females on 2 of the 5 subscales and on the overall CPSET. There were also significant differences among age groups. Medical doctors scored significantly higher on 4 of the 5 subscales and on the overall CPSET. Coordinators of care processes scored significantly lower on 2 of the 5 subscales and on the overall CPSET. Cutoff scores for all subscales and the overall CPSET were calculated.
The CPSET is a valid and reliable instrument for health care teams to measure the extent care processes are organized. The cutoff table permits teams to compare how they perceive the organization of their care process relative to other teams.
Psychometric properties; Care process; Organization of care; Validity; Reliability; Health care teams; CPSET; Multidisciplinary teams; Multicenter study
The Center for Epidemiologic Studies Depression Scale (CES-D) is a commonly used instrument to measure depressive symptomatology. Despite this, the evidence for its psychometric properties remains poorly established in Chinese populations. The aim of this study was to validate the use of the CES-D in Chinese primary care patients by examining factor structure, construct validity, reliability, sensitivity and responsiveness.
Methods and Results
The psychometric properties were assessed amongst a sample of 3686 Chinese adult primary care patients in Hong Kong. Three competing factor structure models were examined using confirmatory factor analysis. The original CES-D four-structure model had adequate fit, however the data was better fit into a bi-factor model. For the internal construct validity, corrected item-total correlations were 0.4 for most items. The convergent validity was assessed by examining the correlations between the CES-D, the Patient Health Questionnaire 9 (PHQ-9) and the Short Form-12 Health Survey (version 2) Mental Component Summary (SF-12 v2 MCS). The CES-D had a strong correlation with the PHQ-9 (coefficient: 0.78) and SF-12 v2 MCS (coefficient: -0.75). Internal consistency was assessed by McDonald’s omega hierarchical (ωH). The ωH value for the general depression factor was 0.855. The ωH values for “somatic”, “depressed affect”, “positive affect” and “interpersonal problems” were 0.434, 0.038, 0.738 and 0.730, respectively. For the two-week test-retest reliability, the intraclass correlation coefficient was 0.91. The CES-D was sensitive in detecting differences between known groups, with the AUC >0.7. Internal responsiveness of the CES-D to detect positive and negative changes was satisfactory (with p value <0.01 and all effect size statistics >0.2). The CES-D was externally responsive, with the AUC>0.7.
The CES-D appears to be a valid, reliable, sensitive and responsive instrument for screening and monitoring depressive symptoms in adult Chinese primary care patients. In its original four-factor and bi-factor structure, the CES-D is supported for cross-cultural comparisons of depression in multi-center studies.
End stage renal disease (ESRD) profoundly impacts the lives of patients. Kidney transplantation provides the greatest health-related quality of life (HRQOL) improvement. Its measurement has become an important outcome parameter and a very important criterion in the evaluation of any type of medical treatment, especially in the field of renal transplantation.
In 2007, a specific self-administered questionnaire for renal transplant recipients was developed in the French language: the ReTransQol (RTQ).
After 5 years of use, the properties of the RTQ needed to be re-evaluated in a larger sample.
This paper describes the analysis of the ReTransQol and its adaptation to achieve an improved and revised version.
The study design included three analysis phases for two samples of adult renal transplant recipients which came from two cross-sectional multicenter studies carried out in France in 2007 and 2012. Psychometrics properties like construct validity, acceptability and feasibility, reliability and convergent validity were evaluated and every analysis resulted in a new version of the questionnaire: the RTQ V2. The construct validity of the new RTQ was assessed with a Confirmatory Factor Analysis on a large sample of patients.
The study samples included 1,059 patients and 1,591 patients, respectively.
After a principal component analysis, item reduction was performed and a total of 13 items were deleted. A final version of the RTQ V2 was created and comprised of 32 items describing 5 domains: Physical Health, Social Functioning, Medical Care, Treatment and Fear of Losing Graft.
The explained variance between the first and second RTQ versions improved from 46.3% to 53.1%. All psychometric properties of RTQ V2 were satisfactory: IIC >0.4, IDV (%) of 100% and Cronbach’s Alpha >0.7 in every dimension. The confirmatory analysis showed that the overall scalability of the RTQ V2 was satisfactory; all items showed a good fit to the Rasch model within each dimension, and showed INFIT statistics inside the acceptable range.
Psychometric properties allow this new version of the questionnaire to be used to assess different specific dimensions for the renal transplant population, more effectively than previously possible.
Quality of life; Medical outcomes; Specific questionnaire; Renal transplantation; RETRANSQOL
One of the most widely used self-reporting tools assessing diabetes self-management in English is the Summary of Diabetes Self-Care Activities (SDSCA) measure. To date there is no psychometric validated instrument in German to assess self-management in patients with diabetes mellitus. Therefore, this study aimed to translate the SDSCA into German and examine its psychometric properties.
The English version of the SDSCA was translated into German following the guidelines for cultural adaptation. The German version of the SDSCA (SDSCA-G) was administered to a random sample of 315 patients with diabetes mellitus type 2. Reliability was analyzed using Cronbach’s alpha coefficient and item characteristics were assessed. Exploratory and confirmatory factor analysis (EFA and CFA) were carried out to explore the construct validity. A multivariable linear regression model was used to identify the influence of predictor variables on the SDSCA-G sum score.
The Cronbach’s alpha for the SDSCA-G (all items) was α = 0.618 and an acceptable correlation between the SDSCA-G and Self-management Diabetes Mellitus-Questionnaire (SDQ) (ρ = 0.664) was identified. The EFA suggested a four factor construct as did the postulated model. The CFA showed the goodness of fit of the SDSCA-G. However, item 4 was found to be problematic regarding the analysis of psychometric properties. The omission of item 4 yielded an increase in Cronbach’s alpha (α = 0.631) and improvements of the factor structure and model fit. No statistically significant influences of predictor variables on the SDSCA-G sum score were observed.
The revised German version of the SDSCA (SDSCA-G) is a reliable and valid tool assessing self-management in adults with type 2 diabetes in Germany.
Electronic supplementary material
The online version of this article (doi:10.1186/s12955-014-0185-1) contains supplementary material, which is available to authorized users.
Diabetes mellitus type 2; Self-management; Self-care; Outcome measurement; German; Validation
Tens of millions of patients worldwide suffer from avoidable disabling injuries and death every year. Measuring the safety climate in health care is an important step in improving patient safety. The most commonly used instrument to measure safety climate is the Safety Attitudes Questionnaire (SAQ). The aim of the present study was to establish the validity and reliability of the translated version of the SAQ.
The SAQ was translated and adapted to the Swedish context. The survey was then carried out with 374 respondents in the operating room (OR) setting. Data was received from three hospitals, a total of 237 responses. Cronbach’s alpha and confirmatory factor analysis (CFA) was used to evaluate the reliability and validity of the instrument.
The Cronbach’s alpha values for each of the factors of the SAQ ranged between 0.59 and 0.83. The CFA and its goodness-of-fit indices (SRMR 0.055, RMSEA 0.043, CFI 0.98) showed good model fit. Intercorrelations between the factors safety climate, teamwork climate, job satisfaction, perceptions of management, and working conditions showed moderate to high correlation with each other. The factor stress recognition had no significant correlation with teamwork climate, perception of management, or job satisfaction.
Therefore, the Swedish translation and psychometric testing of the SAQ (OR version) has good construct validity. However, the reliability analysis suggested that some of the items need further refinement to establish sound internal consistency. As suggested by previous research, the SAQ is potentially a useful tool for evaluating safety climate. However, further psychometric testing is required with larger samples to establish the psychometric properties of the instrument for use in Sweden.
Patient safety; Operating room; Safety climate; Psychometrics; Translation; Safety attitudes questionnaire
Boredom, which is a common problem in the general population, has been associated with several psychiatric disorders. The Multidimensional State Boredom Scale (MSBS) was developed, based on a theoretically and empirically grounded definition of boredom, to assess this construct. The aim of the present study was to assess the psychometric properties of the Spanish validated version of the MSBS in a multi-age sample recruited from the general population.
The patients (N = 303) were recruited from primary care settings. In addition to the sociodemographic variables and the MSBS, the General Health Questionnaire 28 items (GHQ-28), Positive and Negative Affect Scale (PANAS), Negative subscale and the Mindful Attention Awareness Scale (MAAS) were administered. We used confirmatory factor analysis (CFA) to analyse the dimensionality of the MSBS. Cronbach’s α coefficient was used to analyse the internal consistency of the scale. The consistency of the MSBS over time (test-retest reliability) was assessed using the intra-class correlation coefficient. The construct validity was examined by calculating Pearson’s r correlations between the MSBS with theoretically related and unrelated constructs. Cronbach’s α for MSBS was 0.89 (95 % CI, 0.87–0.92), ranging from 0.75 to 0.83 for the 5 subscales.
The characteristics of the final sample (N = 303) were that the participants were primarily female (66.77 %) with a mean age of 49.32 years (SD, 11.46) and primarily European (94.71 %). The CFA of the MSBS confirmed that the original five-factor model showed good fit indices: CFI = .96; GFI = .94; SRMR = .05; and RMSEA = .06 [.05–.08]. Cronbach’s α for MSBS was 0.89 (95 % CI, 0.87–0.92), ranging from 0.75 to 0.83 for the 5 subscales. The MSBS showed a test-retest coefficient measured with an ICC of 0.90 (95 % CI, 0.88–0.92). The ICC for the 5 subscales ranged from 0.81 to 0.89. The MSBS showed a significant negative correlation with MAAS and a significant positive correlation with the GHQ (total score and subscales) and PANAS-Negative Affect.
The Spanish version of the MSBS has been validated as a reliable instrument for measuring boredom in the general population. This study will facilitate the assessment of boredom for clinical and research purposes in Spanish-speaking populations.
Boredom; Questionnaire; Validation; Multidimensional State Boredom Scale
A pediatric chronic health condition not only influences a child's life, but also has impacts on parent health-related quality of life (HRQOL) and family functioning. To provide care and social support to these families, a psychometrically well-developed instrument for measuring these impacts is of great importance. The present study is aimed to evaluate the psychometric properties of the Chinese version of the PedsQL™ Family Impact Module.
The cross-cultural adaptation of the PedsQL™ Family Impact Module was performed following the PedsQL™ Measurement Model Translation Methodology. The Chinese version of the PedsQL™ Family Impact Module was administered to 136 parents of children with asthma and 264 parents of children with heart disease from four Triple A hospitals. The psychometric properties such as feasibility, internal consistency reliability, item-subscale correlations and construct validity were evaluated.
The percentage of missing item responses was less than 0.1% for both asthma and heart disease sample groups. The Chinese version of the PedsQL™ Family Impact Module showed ceiling effects but had acceptable reliability (Cronbach's Alpha Coefficients were higher than 0.7 in all the subscales except "Daily Activities" in the asthma sample group). There were higher correlation coefficients between items and their hypothesized subscales than those with other subscales. The asthma sample group reported higher parent HRQOL and family functioning than the heart disease sample group. In the heart disease sample group, parents of outpatients reported higher parent HRQOL and family functioning than parents of inpatients. Confirmatory factor analysis showed that the instrument had marginally acceptable construct validity with some Goodness-of-Fit indices not reaching the standard indicating acceptable model fit.
The Chinese version of the PedsQL™ Family Impact Module has adequate psychometric properties and could be used to assess the impacts of pediatric asthma or pediatric heart disease on parent HRQOL and family functioning in China. This instrument should be field tested on parents of children with other chronic medical conditions in other areas. Construct validity tested by confirmatory factor analysis and test-retest reliability should be further assessed.
Health-promoting lifestyle choices of adolescents are closely related to current and subsequent health status. However, parsimonious yet reliable and valid screening tools are scarce. The original 40-item adolescent health promotion (AHP) scale was developed by our research team and has been applied to measure adolescent health-promoting behaviors worldwide. The aim of our study was to examine the psychometric properties of a newly developed short-form version of the AHP (AHP-SF) including tests of its reliability and validity.
The study was conducted in nine middle and high schools in southern Taiwan. Participants were 814 adolescents randomly divided into two subgroups with equal size and homogeneity of baseline characteristics. The first subsample (calibration sample) was used to modify and shorten the factorial model while the second subsample (validation sample) was utilized to validate the result obtained from the first one. The psychometric testing of the AHP-SF included internal reliability of McDonald’s omega and Cronbach's alpha, convergent validity, discriminant validity, and construct validity with confirmatory factor analysis (CFA).
The results of the CFA supported a six-factor model and 21 items were retained in the AHP-SF with acceptable model fit. For the discriminant validity test, results indicated that adolescents with lower AHP-SF scores were more likely to be overweight or obese, skip breakfast, and spend more time watching TV and playing computer games. The AHP-SF also showed excellent internal consistency with a McDonald’s omega of 0.904 (Cronbach’s alpha 0.905) in the calibration group.
The current findings suggest that the AHP-SF is a valid and reliable instrument for the evaluation of adolescent health-promoting behaviors. Primary health care providers and clinicians can use the AHP-SF to assess these behaviors and evaluate the outcome of health promotion programs in the adolescent population.
Electronic supplementary material
The online version of this article (doi:10.1186/1471-2458-14-1106) contains supplementary material, which is available to authorized users.
Instrument development; Adolescent; Health promotion; Adolescent health promotion scale
The Internet Addiction Test (IAT) by Kimberly Young is one of the most utilized diagnostic instruments for Internet addiction. Although many studies have documented psychometric properties of the IAT, consensus on the optimal overall structure of the instrument has yet to emerge since previous analyses yielded markedly different factor analytic results.
The objective of this study was to evaluate the psychometric properties of the Italian version of the IAT, specifically testing the factor structure stability across cultures.
In order to determine the dimensional structure underlying the questionnaire, both exploratory and confirmatory factor analyses were performed. The reliability of the questionnaire was computed by the Cronbach alpha coefficient.
Data analyses were conducted on a sample of 485 college students (32.3%, 157/485 males and 67.7%, 328/485 females) with a mean age of 24.05 years (SD 7.3, range 17-47). Results showed 176/485 (36.3%) participants with IAT score from 40 to 69, revealing excessive Internet use, and 11/485 (1.9%) participants with IAT score from 70 to 100, suggesting significant problems because of Internet use. The IAT Italian version showed good psychometric properties, in terms of internal consistency and factorial validity. Alpha values were satisfactory for both the one-factor solution (Cronbach alpha=.91), and the two-factor solution (Cronbach alpha=.88 and Cronbach alpha=.79). The one-factor solution comprised 20 items, explaining 36.18% of the variance. The two-factor solution, accounting for 42.15% of the variance, showed 11 items loading on Factor 1 (Emotional and Cognitive Preoccupation with the Internet) and 7 items on Factor 2 (Loss of Control and Interference with Daily Life). Goodness-of-fit indexes (NNFI: Non-Normed Fit Index; CFI: Comparative Fit Index; RMSEA: Root Mean Square Error of Approximation; SRMR: Standardized Root Mean Square Residual) from confirmatory factor analyses conducted on a random half subsample of participants (n=243) were satisfactory in both factorial solutions: two-factor model (χ2
132= 354.17, P<.001, χ2/df=2.68, NNFI=.99, CFI=.99, RMSEA=.02 [90% CI 0.000-0.038], and SRMR=.07), and one-factor model (χ2
169=483.79, P<.001, χ2/df=2.86, NNFI=.98, CFI=.99, RMSEA=.02 [90% CI 0.000-0.039], and SRMR=.07).
Our study was aimed at determining the most parsimonious and veridical representation of the structure of Internet addiction as measured by the IAT. Based on our findings, support was provided for both single and two-factor models, with slightly strong support for the bidimensionality of the instrument. Given the inconsistency of the factor analytic literature of the IAT, researchers should exercise caution when using the instrument, dividing the scale into factors or subscales. Additional research examining the cross-cultural stability of factor solutions is still needed.
IAT; Internet; addiction; factorial structure; psychometric properties; structural validity
Despite the benefits of mother-newborn skin-to-skin contact immediately after birth, it has not been universally implemented as routine care for healthy term neonates. Midwifes are the first person to contact the neonate after birth. However, there is evidence that many midwives do not perform mother-newborn skin-to-skin contact. The aim of this study was to develop and psychometrically evaluate an instrument for measuring factors associated with mother-newborn skin-to-skin contact (MSSCQ) based on the PRECEDE-PROCEED model.
This was a two-phase qualitative and quantitative study. It was conducted during 2010 to 2012 in Tehran, Iran. In the qualitative part, 150 midwives working in labor room participated in 19 focus group discussions in order to generate a preliminary item pool. Then, content and face validity were performed to provide a pre-final version of the questionnaire. In the quantitative phase, reliability (internal consistency and test-retest analysis), validity and factor analysis (both exploratory and confirmatory) were performed to assess psychometric properties of the instrument.
A 120-item questionnaire was developed through the qualitative phase. It was reduced to an 83-item after content validity. The exploratory factor analysis loaded fifteen-factors and three constructs (predisposing, enabling and reinforcing) containing 82 items (38, 18, and 26 statements, respectively) that jointly accounted for 60.61% of observed variance. The Confirmatory factors analysis determined a model with appropriate fitness for the data. The Cronbach’s alpha coefficient showed excellent internal consistency (alpha = 0.92), and test-retest of the scale with 2-week intervals indicated an appropriate stability for the MSSCQ (ICC = 0.94).
The Mother-Newborn Skin-to-Skin Contact Questionnaire (MSSCQ) is a reliable and valid theory-based measurement and now can be used in clinical practice, midwifery and nursing studies.
Patient safety culture is how leader and staff interaction, attitudes, routines and practices protect patients from adverse events in healthcare. The Safety Attitudes Questionnaire is the most widely used instrument to measure safety attitudes among health care providers. The instrument may identify possible weaknesses in clinical settings, and motivate and guide quality improvement interventions and reductions in medical errors. The Safety Attitudes Questionnaire – Ambulatory Version was developed for measuring safety culture in the primary care setting. The original version includes six major patient safety factors: Teamwork climate, Safety climate, Job satisfaction, Perceptions of management, Working conditions and Stress recognition. We describe the results of a validation study using the Norwegian translation of the questionnaire in the primary care setting, and present the psychometric properties of this version.
The study was done in seven Out-of-hours casualty clinics and 17 regular GP practices employing a total of 510 primary health care providers (194 nurses and 316 medical doctors). In October and November 2012, the translated Safety Attitudes Questionnaire – Ambulatory Version was distributed by e-mail. Data were collected electronically using the program QuestBack, whereby the participants responded anonymously. SPSS was used to estimate the Cronbach’s alphas, item-to-own-factor correlations, intercorrelations of factors and item-descriptive statistics. The confirmatory factor analysis was done by AMOS.
Of the 510 invited health care providers, 266 (52%) answered the questionnaire - 72% of the registered nurses (n = 139) and 39% of the medical doctors (n = 124). In the confirmatory factor analysis, the following five factor model was shown to have acceptable goodness-of-fit values in the Norwegian primary care setting: Teamwork climate, Safety climate, Job satisfaction, Working conditions and Perceptions of management.
The results of our study indicate that the Norwegian translated version of the Safety Attitudes Questionnaire – Ambulatory Version, with the five confirmed factors, might be a useful tool for measuring several aspects of patient safety culture in the primary care setting. Further research should investigate whether there is an association between patient safety culture in primary care, as measured by the Safety Attitudes Questionnaire – Ambulatory Version, and occurrence of medical errors and negative patient outcome.
Adverse events; General practice; Medical errors; Out-of-hours; Patient safety culture; Primary care; Quality improvement; Safety attitudes questionnaire
The purpose of this study was to develop a vulvovaginal symptoms questionnaire (VSQ) to study symptoms, emotions, life-impact, and sexual-impact of vulvovaginal symptoms in postmenopausal women.
We developed questionnaire focused on vulvovaginal symptoms based on modifications to the Skindex-16, a validated questionnaire to measure the impact of skin disease. We then recruited postmenopausal women seeking routine gynecologic care to test the psychometric properties of the VSQ. Test-retest reliability was assessed 2 to 4 weeks after their initial recruitment and measured utilizing intra-class coefficients. Four distinct a priori scales of the VSQ were developed: symptoms, emotions, life-impact, and sexual-impact. Confirmatory factor analysis was performed to verify the four a priori scales by evaluating the goodness-of-fit of a final confirmatory factor analysis model. The internal consistency of the scales was assessed through the calculation of Cronbach’s α coefficient.
The VSQ is a 21-item written questionnaire with four scales, symptoms, emotions, life-impact, and sexual impact. One hundred twenty postmenopausal women participated in the psychometric validation of the VSQ. The test-retest reliability the four scales measured by intra-class coefficients were 0.75, 0.60, 0.55, and 0.65 for symptoms, emotions, life-impact and sexual-impact. The goodness-of –fit of the confirmatory factor response model was confirmed. Cronbach’s α coefficients were 0.76, 0.87, 0.83, and 0.82 for the scales.
The VSQ is a reliable and internal consistent instrument to measure vulvovaginal symptoms in postmenopausal women.
atrophic vaginitis; menopause; vulvovaginal symptoms
The Chronic Care Model (CCM) has achieved widespread acceptance and reflects the core elements of patient-centred care in chronic diseases such as cardiovascular diseases (CVD). In the Netherlands the extent to which CVD patients receive care congruent with the CCM is unknown. The main objectives of this study were to validate the 20-item Patient Assessment of Chronic Illness Care (PACIC) and the 11-item (PACIC-S) in the Netherlands among CVD patients and investigate the validity, reliability, and sensitivity to change of both instruments.
The Dutch version of the PACIC and PACIC-S were tested among 1484 CVD patients (out of 2760; response rate 54%) enrolled in Disease Management Programmes (DMPs) at T0 and 1167 respondents (out of 2545; response rate = 46%) at T1. Five hundred-eighty-five CVD patients filled in the questionnaire at both T0 and T1. We tested the instrument by means of structural equation modeling, and examined its construct validity, reliability and sensitivity to change. Reliability of the instrument was assessed by determining the statistical coherence of the scaled items. Internal consistency of the subscales was assessed by calculating Cronbach’s alphas and correlations between the PACIC and PACIC-S. We investigated the sensitivity to change of the original PACIC and the PACIC-S with paired t-tests among CVD patients in DMPs who filled in the questionnaire at both T0 and T1 (N = 585).
The confirmatory factor analyses revealed good indices of fit with the PACIC and PACIC-S. Internal consistency as represented by Cronbach’s alphas were also good. Correlations between the PACIC and PACIC-S subscales were excellent: 0.98 at both T0 and T1. Paired t-tests results show that the PACIC and PACIC-S improved significantly over time (p < 0.01).
The psychometric properties of the Dutch PACIC and PACIC-S were satisfactory and it is sensitive to change, rendering it a valid and reliable instrument for assessing chronic illness care among CVD patients.
Disease management; Chronic care model; Chronic illness care; Quality; Primary care; Cardiovascular diseases
Emotional distress is an important dimension in diabetes, and several instruments have been developed to measure this aspect. The Problem Areas in Diabetes (PAID) scale is one such instrument which has demonstrated validity and reliability in Western populations, but its psychometric properties in Asian populations have not been examined.
This was a secondary analysis of data from patients with Type 2 diabetes mellitus recruited through convenience sampling from a diabetes specialist outpatient clinic in Singapore. The following psychometric properties were assessed: Construct validity through confirmatory factor analysis (CFA) and Rasch analysis, concurrent validity through correlation with related scales (Kessler Psychological Distress Scale, Diabetes Health Profile—psychological distress, Audit of Diabetes Dependent Quality of Life), reliability through assessment of internal consistency and floor and ceiling effects, and sensitivity by estimating effect sizes for known clinical and social functioning groups.
203 patients with mean age of 45±12 years were analysed. None of the previously published model structures achieved a good fit on CFA. On Rasch analysis, four items showed poor fit and were removed. The abridged 16-item PAID mapped to a single latent trait, with a high degree of internal consistency (Cronbach ɑ 0.95), but significant floor effect (24.6% scoring at floor). Both 20-item and 16-item PAID scores were moderately correlated with scores of related scales, and sensitive to differences in clinical and social functioning groups, with large effect sizes for glycemic control and diabetes related complications, nephropathy and neuropathy.
The abridged 16-item PAID measures a single latent trait of emotional distress due to diabetes whereas the 20-item PAID appears to measures more than one latent trait. However, both the 16-item and 20-item PAID versions are valid, reliable and sensitive for use among Singaporean patients with diabetes.
There is a lack of validated instruments to measure the level of burden of Alzheimer’s disease (AD) on caregivers. The Impact of Alzheimer’s Disease on Caregiver Questionnaire (IADCQ) is a 12-item instrument with a seven-day recall period that measures AD caregiver’s burden across emotional, physical, social, financial, sleep, and time aspects. Primary objectives of this study were to evaluate psychometric properties of IADCQ administered on the Web and to determine most appropriate scoring algorithm.
A national sample of 200 unpaid AD caregivers participated in this study by completing the Web-based version of IADCQ and Short Form-12 Health Survey Version 2 (SF-12v2™). The SF-12v2 was used to measure convergent validity of IADCQ scores and to provide an understanding of the overall health-related quality of life of sampled AD caregivers.
The IADCQ survey was also completed four weeks later by a randomly selected subgroup of 50 participants to assess test-retest reliability. Confirmatory factor analysis (CFA) was implemented to test the dimensionality of the IADCQ items. Classical item-level and scale-level psychometric analyses were conducted to estimate psychometric characteristics of the instrument. Test-retest reliability was performed to evaluate the instrument’s stability and consistency over time.
Virtually none (2%) of the respondents had either floor or ceiling effects, indicating the IADCQ covers an ideal range of burden. A single-factor model obtained appropriate goodness of fit and provided evidence that a simple sum score of the 12 items of IADCQ can be used to measure AD caregiver’s burden. Scales-level reliability was supported with a coefficient alpha of 0.93 and an intra-class correlation coefficient (for test-retest reliability) of 0.68 (95% CI: 0.50–0.80). Low-moderate negative correlations were observed between the IADCQ and scales of the SF-12v2.
The study findings suggest the IADCQ has appropriate psychometric characteristics as a unidimensional, Web-based measure of AD caregiver burden and is supported by strong model fit statistics from CFA, high degree of item-level reliability, good internal consistency, moderate test-retest reliability, and moderate convergent validity. Additional validation of the IADCQ is warranted to ensure invariance between the paper-based and Web-based administration and to determine an appropriate responder definition.
Alzheimer’s disease; Caregivers; Burden; Psychometrics; Questionnaire