Search tips
Search criteria

Results 1-25 (214460)

Clipboard (0)

Related Articles

1.  Pre-validation methods for developing a patient reported outcome instrument 
Measures that reflect patients' assessment of their health are of increasing importance as outcome measures in randomised controlled trials. The methodological approach used in the pre-validation development of new instruments (item generation, item reduction and question formatting) should be robust and transparent. The totality of the content of existing PRO instruments for a specific condition provides a valuable resource (pool of items) that can be utilised to develop new instruments. Such 'top down' approaches are common, but the explicit pre-validation methods are often poorly reported. This paper presents a systematic and generalisable 5-step pre-validation PRO instrument methodology.
The method is illustrated using the example of the Aberdeen Glaucoma Questionnaire (AGQ). The five steps are: 1) Generation of a pool of items; 2) Item de-duplication (three phases); 3) Item reduction (two phases); 4) Assessment of the remaining items' content coverage against a pre-existing theoretical framework appropriate to the objectives of the instrument and the target population (e.g. ICF); and 5) qualitative exploration of the target populations' views of the new instrument and the items it contains.
The AGQ 'item pool' contained 725 items. Three de-duplication phases resulted in reduction of 91, 225 and 48 items respectively. The item reduction phases discarded 70 items and 208 items respectively. The draft AGQ contained 83 items with good content coverage. The qualitative exploration ('think aloud' study) resulted in removal of a further 15 items and refinement to the wording of others. The resultant draft AGQ contained 68 items.
This study presents a novel methodology for developing a PRO instrument, based on three sources: literature reporting what is important to patient; theoretically coherent framework; and patients' experience of completing the instrument. By systematically accounting for all items dropped after the item generation phase, our method ensures that the AGQ is developed in a transparent, replicable manner and is fit for validation. We recommend this method to enhance the likelihood that new PRO instruments will be appropriate to the research context in which they are used, acceptable to research participants and likely to generate valid data.
PMCID: PMC3225127  PMID: 21827689
2.  Evaluation of Item Candidates: The PROMIS Qualitative Item Review 
Medical care  2007;45(5 Suppl 1):S12-S21.
One of the PROMIS (Patient-Reported Outcome Measurement Information System) network's primary goals is the development of a comprehensive item bank for patient-reported outcomes of chronic diseases. For its first set of item banks, PROMIS chose to focus on pain, fatigue, emotional distress, physical function, and social function. An essential step for the development of an item pool is the identification, evaluation, and revision of extant questionnaire items for the core item pool. In this work, we also describe the systematic process wherein items are classified for subsequent statistical processing by the PROMIS investigators. Six phases of item development are documented: identification of extant items, item classification and selection, item review and revision, focus group input on domain coverage, cognitive interviews with individual items, and final revision before field testing. Identification of items refers to the systematic search for existing items in currently available scales. Expert item review and revision was conducted by trained professionals who reviewed the wording of each item and revised as appropriate for conventions adopted by the PROMIS network. Focus groups were used to confirm domain definitions and to identify new areas of item development for future PROMIS item banks. Cognitive interviews were used to examine individual items. Items successfully screened through this process were sent to field testing and will be subjected to innovative scale construction procedures.
PMCID: PMC2810630  PMID: 17443114
patient-reported outcomes; cognitive interviews; qualitative methods; questionnaire development
3.  Development and assessment of floor and ceiling items for the PROMIS physical function item bank 
Arthritis Research & Therapy  2013;15(5):R144.
Disability and Physical Function (PF) outcome assessment has had limited ability to measure functional status at the floor (very poor functional abilities) or the ceiling (very high functional abilities). We sought to identify, develop and evaluate new floor and ceiling items to enable broader and more precise assessment of PF outcomes for the NIH Patient-Reported-Outcomes Measurement Information System (PROMIS).
We conducted two cross-sectional studies using NIH PROMIS item improvement protocols with expert review, participant survey and focus group methods. In Study 1, respondents with low PF abilities evaluated new floor items, and those with high PF abilities evaluated new ceiling items for clarity, importance and relevance. In Study 2, we compared difficulty ratings of new floor items by low functioning respondents and ceiling items by high functioning respondents to reference PROMIS PF-10 items. We used frequencies, percentages, means and standard deviations to analyze the data.
In Study 1, low (n = 84) and high (n = 90) functioning respondents were mostly White, women, 70 years old, with some college, and disability scores of 0.62 and 0.30. More than 90% of the 31 new floor and 31 new ceiling items were rated as clear, important and relevant, leaving 26 ceiling and 30 floor items for Study 2. Low (n = 246) and high (n = 637) functioning Study 2 respondents were mostly White, women, 70 years old, with some college, and Health Assessment Questionnaire (HAQ) scores of 1.62 and 0.003. Compared to difficulty ratings of reference items, ceiling items were rated to be 10% more to greater than 40% more difficult to do, and floor items were rated to be about 12% to nearly 90% less difficult to do.
These new floor and ceiling items considerably extend the measurable range of physical function at either extreme. They will help improve instrument performance in populations with broad functional ranges and those concentrated at one or the other extreme ends of functioning. Optimal use of these new items will be assisted by computerized adaptive testing (CAT), reducing questionnaire burden and insuring item administration to appropriate individuals.
PMCID: PMC3978724  PMID: 24286166
4.  Misrepresentation of Randomized Controlled Trials in Press Releases and News Coverage: A Cohort Study 
PLoS Medicine  2012;9(9):e1001308.
A study conducted by Amélie Yavchitz and colleagues examines the factors associated with “spin” (specific reporting strategies, intentional or unintentional, that emphasize the beneficial effect of treatments) in press releases of clinical trials.
Previous studies indicate that in published reports, trial results can be distorted by the use of “spin” (specific reporting strategies, intentional or unintentional, emphasizing the beneficial effect of the experimental treatment). We aimed to (1) evaluate the presence of “spin” in press releases and associated media coverage; and (2) evaluate whether findings of randomized controlled trials (RCTs) based on press releases and media coverage are misinterpreted.
Methods and Findings
We systematically searched for all press releases indexed in the EurekAlert! database between December 2009 and March 2010. Of the 498 press releases retrieved and screened, we included press releases for all two-arm, parallel-group RCTs (n = 70). We obtained a copy of the scientific article to which the press release related and we systematically searched for related news items using Lexis Nexis.
“Spin,” defined as specific reporting strategies (intentional or unintentional) emphasizing the beneficial effect of the experimental treatment, was identified in 28 (40%) scientific article abstract conclusions and in 33 (47%) press releases. From bivariate and multivariable analysis assessing the journal type, funding source, sample size, type of treatment (drug or other), results of the primary outcomes (all nonstatistically significant versus other), author of the press release, and the presence of “spin” in the abstract conclusion, the only factor associated, with “spin” in the press release was “spin” in the article abstract conclusions (relative risk [RR] 5.6, [95% CI 2.8–11.1], p<0.001). Findings of RCTs based on press releases were overestimated for 19 (27%) reports. News items were identified for 41 RCTs; 21 (51%) were reported with “spin,” mainly the same type of “spin” as those identified in the press release and article abstract conclusion. Findings of RCTs based on the news item was overestimated for ten (24%) reports.
“Spin” was identified in about half of press releases and media coverage. In multivariable analysis, the main factor associated with “spin” in press releases was the presence of “spin” in the article abstract conclusion.
Editors' Summary
The mass media play an important role in disseminating the results of medical research. Every day, news items in newspapers and magazines and on the television, radio, and internet provide the general public with information about the latest clinical studies. Such news items are written by journalists and are often based on information in “press releases.” These short communications, which are posted on online databases such as EurekAlert! and sent directly to journalists, are prepared by researchers or more often by the drug companies, funding bodies, or institutions supporting the clinical research and are designed to attract favorable media attention to newly published research results. Press releases provide journalists with the information they need to develop and publish a news story, including a link to the peer-reviewed journal (a scholarly periodical containing articles that have been judged by independent experts) in which the research results appear.
Why Was This Study Done?
In an ideal world, journal articles, press releases, and news stories would all accurately reflect the results of health research. Unfortunately, the findings of randomized controlled trials (RCTs—studies that compare the outcomes of patients randomly assigned to receive alternative interventions), which are the best way to evaluate new treatments, are sometimes distorted in peer-reviewed journals by the use of “spin”—reporting that emphasizes the beneficial effects of the experimental (new) treatment. For example, a journal article may interpret nonstatistically significant differences as showing the equivalence of two treatments although such results actually indicate a lack of evidence for the superiority of either treatment. “Spin” can distort the transposition of research into clinical practice and, when reproduced in the mass media, it can give patients unrealistic expectations about new treatments. It is important, therefore, to know where “spin” occurs and to understand the effects of that “spin”. In this study, the researchers evaluate the presence of “spin” in press releases and associated media coverage and analyze whether the interpretation of RCT results based on press releases and associated news items could lead to the misinterpretation of RCT results.
What Did the Researchers Do and Find?
The researchers identified 70 press releases indexed in EurekAlert! over a 4-month period that described two-arm, parallel-group RCTs. They used Lexis Nexis, a database of news reports from around the world, to identify associated news items for 41 of these press releases and then analyzed the press releases, news items, and abstracts of the scientific articles related to each press release for “spin”. Finally, they interpreted the results of the RCTs using each source of information independently. Nearly half the press releases and article abstract conclusions contained “spin” and, importantly, “spin” in the press releases was associated with “spin” in the article abstracts. The researchers overestimated the benefits of the experimental treatment from the press release as compared to the full-text peer-reviewed article for 27% of reports. Factors that were associated with this overestimation of treatment benefits included publication in a specialized journal and having “spin” in the press release. Of the news items related to press releases, half contained “spin”, usually of the same type as identified in the press release and article abstract. Finally, the researchers overestimated the benefit of the experimental treatment from the news item as compared to the full-text peer-reviewed article in 24% of cases.
What Do These Findings Mean?
These findings show that “spin” in press releases and news reports is related to the presence of “spin” in the abstract of peer-reviewed reports of RCTs and suggest that the interpretation of RCT results based solely on press releases or media coverage could distort the interpretation of research findings in a way that favors experimental treatments. This interpretation shift is probably related to the presence of “spin” in peer-reviewed article abstracts, press releases, and news items and may be partly responsible for a mismatch between the perceived and real beneficial effects of new treatments among the general public. Overall, these findings highlight the important role that journal reviewers and editors play in disseminating research findings. These individuals, the researchers conclude, have a responsibility to ensure that the conclusions reported in the abstracts of peer-reviewed articles are appropriate and do not over-interpret the results of clinical research.
Additional Information
Please access these Web sites via the online version of this summary at
The PLOS Hub for Clinical Trials, which collects PLOS journals relating to clinical trials, includes some other articles on “spin” in clinical trial reports
EurekAlert is an online free database for science press releases
The UK National Health Service Choices website includes Beyond the Headlines, a resource that provides an unbiased and evidence-based analysis of health stories that make the news for both the public and health professionals
The US-based organization HealthNewsReview, a project supported by the Foundation for Informed Medical Decision Making, also provides expert reviews of news stories
PMCID: PMC3439420  PMID: 22984354
5.  The Development of Medical Record Items: a User-centered, Bottom-up Approach 
Clinical documents (CDs) have evolved from traditional paper documents containing narrative text information into the electronic record sheets composed of itemized records, where each record is expressed as an item with a specific value. We defined medical record (MR) items to be information entities with a specific value. These entities were then used to compile form-based clinical documents as part of an electronic health record system (EHR-s).
We took a reusable bottom-up developmental approach for the MR items, which provided three things: efficient incorporation of the local needs and requirements of the medical professionals from various departments in the hospital, comprehensive inclusion of the essential concepts of the basic elements required in clinical documents, and the provision of a structured means for meaningful data entry and retrieval. This paper delineates our experiences in developing and managing medical records at a large tertiary university hospital in Korea.
We collected 63,232 MR items from paper records scanned into 962 CDs. The MR item database was constructed using 13,287 MR items after removing redundant items. During the first year of service users requested changes to be made to 235 (1.8%) attributes of the MR items and also requested the additional 9,572 new MR items. In the second year, the attributes of 70 (0.5%) of the existing MR items were changed and 3,704 new items were added. The number of registered MR items increased by 72.0% in the first year and 27.9% in the second year.
The MR item concept provides an easier and more structured means of data entry within an EHR-s. By using these MR items, various kinds of clinical documents can be easily constructed and allows for medical information to be reused and retrieved as data. The success of the use of MR items in a large tertiary university hospital system provides evidence that verifies our approach as being an efficient means of user-oriented and structured data entry, enabling the easy reuse of medical records.
PMCID: PMC3324750  PMID: 22509469
Medical Record; Electronic Health Record
6.  The consumer quality index anthroposophic healthcare: a construction and validation study 
Accounting for the patients’ perspective on quality of care has become increasingly important in the development of Evidence Based Medicine as well as in governmental policies. In the Netherlands the Consumer Quality (CQ) Index has been developed to measure the quality of care from the patients’ perspective in different healthcare sectors in a standardized manner. Although the scientific accountability of anthroposophic healthcare as a form of integrative medicine is growing, patient experiences with anthroposophic healthcare have not been measured systematically. In addition, the specific anthroposophic aspects are not measured by means of existing CQ Indexes. To enable accountability of quality of the anthroposophic healthcare from the patients’ perspective the aim of this study is the construction and validation of a CQ Index for anthroposophic healthcare.
Construction in three phases: Phase 1. Determining anthroposophic quality aspects: literature study and focus groups. Phase 2. Adding new questions and validating the new questionnaire. Research population: random sample from 7910 patients of 22 anthroposophic GPs. Data collection: survey, mixed mode by means of the Dillman method. Measuring instrument: experience questionnaire: CQ Index General Practice (56 items), added with 27 new anthroposophic items added and an item-importance questionnaire (anthroposophic items only). Statistical analyses: Factor analysis, scale construction, internal consistency (Chronbach’s Alpha), inter-item-correlation, discriminative ability (Intra Class Correlation) and inter-factor-correlations. Phase 3. Modulation and selection of new questions based on results. Criteria of retaining items: general: a limited amount of items, statistical: part of a reliable scale and inter-item-correlation <0,7, and theoretical.
Phase 1. 27 anthroposophic items. Phase 2. Two new anthroposophic scales: Scale AntroposophicTreatmentGP: seven items, Alpha=0,832, ICC=4,2 Inter-factor-correlation with existing GP-scales range from r=0,24 (Accessibility) to r=0,56 (TailoredCare). Scale InteractionalStyleGP: five items, Alpha=0,810, ICC=5,8, Inter-factor-correlation with existing GP-scales range from r=0,32 (Accessibility) to r=0,76 (TailoredCare). Inter-factor-correlation between new scales: r=0,50. Phase 3: Adding both scales and four single items. Removing eleven items and reformulating two items.
The CQ Index Anthroposophic Healthcare measures patient experiences with anthroposophic GP’s validly and reliably. Regarding the inter-factor-correlations anthroposophic quality aspects from the patients’ perspective are mostly associated with individually tailored care and patient centeredness.
PMCID: PMC4230405  PMID: 24694288
Quality of care; Patient experiences; CQ-index; Anthroposophic healthcare; Validation of measuring instrument; Psychometric quality; Discriminating ability; Patient-centered care
7.  Toward a More Systematic Assessment of Smoking: Development of a Smoking Module for PROMIS® 
Addictive Behaviors  2012;37(11):1278-1284.
The aim of the PROMIS® Smoking Initiative is to develop, evaluate, and standardize item banks to assess cigarette smoking behavior and biopsychosocial constructs associated with smoking for both daily and non-daily smokers.
We used qualitative methods to develop the item pool (following the PROMIS® approach: e.g., literature search, “binning and winnowing” of items, and focus groups and cognitive interviews to finalize wording and format), and quantitative methods (e.g., factor analysis) to develop the item banks.
We considered a total of 1622 extant items, and 44 new items for inclusion in the smoking item banks. A final set of 277 items representing 11 conceptual domains was selected for field testing in a national sample of smokers. Using data from 3021 daily smokers in the field test, an iterative series of exploratory factor analyses and project team discussions resulted in six item banks: Positive Consequences of Smoking (40 items), Smoking Dependence/Craving (55 items), Health Consequences of Smoking (26 items), Psychosocial Consequences of Smoking (37 items), Coping Aspects of Smoking (30 items), and Social Factors of Smoking (23 items).
Inclusion of a smoking domain in the PROMIS® framework will standardize measurement of key smoking constructs using state-of-the-art psychometric methods, and make them widely accessible to health care providers, smoking researchers and the large community of researchers using PROMIS® who might not otherwise include an assessment of smoking in their design. Next steps include reducing the number of items in each domain, conducting confirmatory analyses, and duplicating the process for non-daily smokers.
PMCID: PMC3405537  PMID: 22770824
Smoking; assessment; IRT; item banks; PROMIS; mixed methods
8.  Measuring the ICF components of impairment, activity limitation and participation restriction: an item analysis using classical test theory and item response theory 
The International Classification of Functioning, Disability and Health (ICF) proposes three main health outcomes, Impairment (I), Activity Limitation (A) and Participation Restriction (P), but good measures of these constructs are needed The aim of this study was to use both Classical Test Theory (CTT) and Item Response Theory (IRT) methods to carry out an item analysis to improve measurement of these three components in patients having joint replacement surgery mainly for osteoarthritis (OA).
A geographical cohort of patients about to undergo lower limb joint replacement was invited to participate. Five hundred and twenty four patients completed ICF items that had been previously identified as measuring only a single ICF construct in patients with osteoarthritis. There were 13 I, 26 A and 20 P items. The SF-36 was used to explore the construct validity of the resultant I, A and P measures. The CTT and IRT analyses were run separately to identify items for inclusion or exclusion in the measurement of each construct. The results from both analyses were compared and contrasted.
Overall, the item analysis resulted in the removal of 4 I items, 9 A items and 11 P items. CTT and IRT identified the same 14 items for removal, with CTT additionally excluding 3 items, and IRT a further 7 items. In a preliminary exploration of reliability and validity, the new measures appeared acceptable.
New measures were developed that reflect the ICF components of Impairment, Activity Limitation and Participation Restriction for patients with advanced arthritis. The resulting Aberdeen IAP measures (Ab-IAP) comprising I (Ab-I, 9 items), A (Ab-A, 17 items), and P (Ab-P, 9 items) met the criteria of conventional psychometric (CTT) analyses and the additional criteria (information and discrimination) of IRT. The use of both methods was more informative than the use of only one of these methods. Thus combining CTT and IRT appears to be a valuable tool in the development of measures.
PMCID: PMC2696439  PMID: 19422677
9.  Role of religion and spirituality in medical patients: Confirmatory results with the SpREUK questionnaire 
Spirituality has become a subject of interest in health care as it is was recognized to have the potential to prevent, heal or cope with illness. There is less doubt that values and goals are important contributors to life satisfaction, physical and psychological health, and that goals are what gives meaning and purpose to people's lives. However, there is as yet but limited understanding of how patients themselves view the impact of spirituality on their health and well-being, and whether they are convinced that their illness may have "meaning" to them. To raise these questions and to more precisely survey the basic attitudes of patients with severe diseases towards spirituality/religiosity (SpR) and their adjustment to their illness, we developed the SpREUK questionnaire.
In order to re-validate our previously described SpREUK instrument, reliability and factor analysis of the new inventory (Version 1.1) were performed according to the standard procedures. The test sample contained 257 German subjects (53.3 ± 13.4 years) with cancer (51%), multiple sclerosis (24%), other chronic diseases (16%) and patients with acute diseases (7%).
As some items of the SpREUK construct require a positive attitude towards SpR, these items (item pool 2) were separated from the others (item pool 1). The reliability of the 15-item the construct derived from the item pool 1 respectively the 14-item construct which refers to the item pool 2 both had a good quality (Cronbach's alpha = 0.9065 resp. 0.9525). Factor analysis of item pool 1 resulted in a 3-factor solution (i.e. the 6-item sub-scale 1: "Search for meaningful support"; the 6-item sub-scale 2: "Positive interpretation of disease"; and the 3-item sub-scale 3: "Trust in external guidance") which explains 53.8% of variance. Factor analysis of item pool 2 pointed to a 2-factor solution (i.e. the 10-item sub-scale 4: "Support in relations with the External life through SpR" and the 4-item sub-scale 5: "Support of the Internality through SpR") which explains 58.8% of variance. Generally, women had significantly higher SpREUK scores than male patients. Univariate variance analyses revealed significant associations between the sub-scales and SpR attitude and the educational level.
The current re-evaluation of the SpREUK 1.1 questionnaire indicates that it is a reliable, valid measure of distinct topics of SpR that may be especially useful of assessing the role of SpR in health related research. The instrument appears to be a good choice for assessing a patients interest in spiritual concerns which is not biased for or against a particular religious commitment. Moreover it addresses the topic of "positive reinterpretation of disease" which seems to be of outstanding importance for patients with life-changing diseases.
PMCID: PMC550666  PMID: 15705195
Questionnaires; Religion and Medicine; Spirituality and Religion; coping; chronic disease, cancer
10.  Measuring prescribing: the shortcomings of the item. 
BMJ : British Medical Journal  1994;308(6929):637-640.
OBJECTIVES--To assess the validity of the item as a measure of the volume of a drug prescribed; and to investigate the possibility that higher quantities per item are prescribed for patients who are not exempt from the prescription charge. DESIGN--Five substudies. For the first, a frequency distribution was derived of the different quantities per item of 10 commonly used drugs prescribed by 20 randomly selected practices in each of five family health service authority areas. For the second, the variation in average quantity per item for the same drugs in the same practices was calculated. For the third and fourth, variation in average quantity per item for 90 commonly used drugs was calculated for all 90 family health service authorities and for all 14 regional health authorities in England. For the fifth, the average quantity per item for each of the 90 drugs was regressed on the percentage of items exempt from the prescription charge, at family health service authority level, and the percentage of variation explained by the regression found. MAIN OUTCOME MEASURE--Distribution of quantity per item; variation in average quantity per item between the practices, between family health service authorities, and between regions; and percentage of variation between family health service authorities accounted for by exemption from the prescription charge. RESULTS--Wide variation was found in the quantities per item prescribed by the practices, and in the average quantity per item between practices and between family health service authorities. No family health service authority was consistently high or low in quantity per item across the 90 drugs. Variation in average quantity per item was less at regional than at family health service authority level, though still high for many of the drugs. The proportion of variation accounted for by exemption from prescription charges ranged from 0% to 49% across the 90 drugs. CONCLUSIONS--The item is unsuitable as a measure of prescribing volume, even at regional level: a new measure, based on standard daily dosages, is needed. The percentage of the variation in quantity per item accounted for by exemption is inconsistent, and in over half the 90 drugs it was below 20%--therefore it is not a useful predictor.
PMCID: PMC2539693  PMID: 8148715
11.  Cross-cultural development of an item list for computer-adaptive testing of fatigue in oncological patients 
Within an ongoing project of the EORTC Quality of Life Group, we are developing computerized adaptive test (CAT) measures for the QLQ-C30 scales. These new CAT measures are conceptualised to reflect the same constructs as the QLQ-C30 scales. Accordingly, the Fatigue-CAT is intended to capture physical and general fatigue.
The EORTC approach to CAT development comprises four phases (literature search, operationalisation, pre-testing, and field testing). Phases I-III are described in detail in this paper. A literature search for fatigue items was performed in major medical databases. After refinement through several expert panels, the remaining items were used as the basis for adapting items and/or formulating new items fitting the EORTC item style. To obtain feedback from patients with cancer, these English items were translated into Danish, French, German, and Spanish and tested in the respective countries.
Based on the literature search a list containing 588 items was generated. After a comprehensive item selection procedure focusing on content, redundancy, item clarity and item difficulty a list of 44 fatigue items was generated. Patient interviews (n = 52) resulted in 12 revisions of wording and translations.
The item list developed in phases I-III will be further investigated within a field-testing phase (IV) to examine psychometric characteristics and to fit an item response theory model. The Fatigue CAT based on this item bank will provide scores that are backward-compatible to the original QLQ-C30 fatigue scale.
PMCID: PMC3078839  PMID: 21447160
12.  Construction of the Pediatric Asthma Impact Scale (PAIS) for the Patient-Reported Outcomes Measurement Information System (PROMIS) 
Recently, the National Institutes of Health Roadmap for Medical Research initiative led a large-scale effort to develop the Patient-Reported Outcomes Measurement Information System (PROMIS). PROMIS’s main goal was to develop a set of item banks and computerized adaptive tests for the clinical research community. Asthma, as the most common chronic childhood disease, was chosen for a disease-specific pediatric item bank.
The primary objective of this research is to present the details of the psychometric analyses of the asthma domain items.
Item response theory (IRT) analyses were conducted on a 34–asthma item bank. Test forms containing PROMIS Pediatric Asthma domain items were completed by 622 children ages 8 to 12. Items were subsequently evaluated for local dependence, scale dimensionality, and differential item functioning.
A 17-item pool and an 8-item short form for the new PROMIS Pediatric Asthma Impact Scale (PAIS) were generated using IRT. The recommended 8-item short form contains the item set that provides the maximum test information at the mean (50) on the T-score metric. If more score precision is required, the complete 17-item pool is recommended and may be used in toto or as the basis of a computerized adaptive test (CAT). A shorter test form can also be created and scored on the same scale.
The present study presents the PROMIS Pediatric Asthma Impact Scale (PAIS) developed with IRT, and provides the initial calibration data for the items.
PMCID: PMC3098738  PMID: 20394514
asthma; pediatric; quality-of-life; item response theory
13.  Measuring social health in the patient-reported outcomes measurement information system (PROMIS): item bank development and testing 
To develop a social health measurement framework, to test items in diverse populations and to develop item response theory (IRT) item banks.
A literature review guided framework development of Social Function and Social Relationships sub-domains. Items were revised based on patient feedback, and Social Function items were field-tested. Analyses included exploratory factor analysis (EFA), confirmatory factor analysis (CFA), two-parameter IRT modeling and evaluation of differential item functioning (DIF).
The analytic sample included 956 general population respondents who answered 56 Ability to Participate and 56 Satisfaction with Participation items. EFA and CFA identified three Ability to Participate sub-domains. However, because of positive and negative wording, and content redundancy, many items did not fit the IRT model, so item banks do not yet exist. EFA, CFA and IRT identified two preliminary Satisfaction item banks. One item exhibited trivial age DIF.
After extensive item preparation and review, EFA-, CFA- and IRT-guided item banks help provide increased measurement precision and flexibility. Two Satisfaction short forms are available for use in research and clinical practice. This initial validation study resulted in revised item pools that are currently undergoing testing in new clinical samples and populations.
PMCID: PMC3138729  PMID: 20419503
Patient-reported outcomes; Social health; Social function; Social relationships; Item banks
14.  What proportion of prescription items dispensed in community pharmacies are eligible for the New Medicine Service? 
The payment structure for the New Medicine Service (NMS) in England is based on the assumption that 0.5% of prescription items dispensed in community pharmacies are eligible for the service. This assumption is based on a theoretical calculation. This study aimed to find out the actual proportion of prescription items eligible for the NMS dispensed in community pharmacies in order to compare this with the theoretical assumption. The study also aimed to investigate whether the proportion of prescription items eligible for the NMS is affected by pharmacies’ proximity to GP practices.
The study collected data from eight pharmacies in Nottingham belonging to the same large chain of pharmacies. Pharmacies were grouped by distance from the nearest GP practice and sampled to reflect the distribution by distance of all pharmacies in Nottingham. Data on one thousand consecutive prescription items were collected from each pharmacy and the number of NMS eligible items recorded. All NHS prescriptions were included in the sample. Data were analysed and proportions calculated with 95% confidence intervals used to compare the study results against the theoretical figure of 0.5% of prescription items being eligible for the NMS.
A total of 8005 prescription items were collected (a minimum of 1000 items per pharmacy) of which 17 items were eligible to receive the service. The study found that 0.25% (95% confidence intervals: 0.14% to 0.36%) of prescription items were eligible for the NMS which differs significantly from the theoretical assumption of 0.5%. The opportunity rate for the service was lower, 0.21% (95% confidence intervals: 0.10% to 0.32%) of items, as some items eligible for the NMS did not translate into opportunities to offer the service. Of all the prescription items collected in the pharmacies, 28% were collected by patient representatives.
The results of this study show that the proportion of items eligible for the NMS dispensed in community pharmacies is lower than the Department of Health assumption of 0.5%. This study did not find a significant difference in the rate of NMS opportunities between pharmacies located close to GP practices compared to those further away.
PMCID: PMC3975589  PMID: 24606863
New Medicine Service; England; Prescription items; Pharmacy
15.  Satisfaction with care among patients with non-metastatic breast cancer: development and first steps of validation of the REPERES-60 questionnaire 
BMC Cancer  2007;7:129.
The care itinerary for cancer involves difficulties that occur in several different areas, whether in the diagnostic procedures, in surgery, or in adjuvant treatment. The aim of this work was to obtain a valid instrument measuring satisfaction among patients with breast cancer and exploring their care itinerary overall.
Development phase: Patient focus groups were implemented in two French regions in order to identify areas of satisfaction in relation to the different phases of care provision in breast cancer. On the basis of the literature and the themes and wordings derived from the focus groups, the patients identified several areas of satisfaction, which they found to be partially covered in an American satisfaction measure that has been validated in the French general population (the Consumer Satisfaction Survey in its French version, CSS-VF, 39 items). The patient focus groups suggested adaptation of certain dimensions of this instrument to the potential care providers (37 items) and produced 45 new items in six areas.
Validation phase: Using a large sample of patients (cohort of 820 women with invasive non-metastatic breast cancer) approached one month after treatment, this phase selected items that were comprehensible (non-response rate < 10%), non-redundant (r < 0.80) and reproducible (test-retest conducted on a sub-sample of 166 patients). The dimensions were identified by factor analysis on the selected items. Divergent and discriminant validity were assessed (relationships with quality of life questionnaire, comparisons between extreme groups).
Results were in favour of not inserting additional broken-down items into the CSS-VF and retaining 21 new items. The factor analysis found the initial structure of the CSS-VF (39 items in 9 dimensions) and the 21 new items divide up into four dimensions (listening abilities and information provided by doctors, organisation and follow-up of medical care provision, psychological support, material environment). No redundancy was observed between new items and CSS-VF items. Internal consistency was high. Divergent and discriminant validity were satisfactory.
Adding four new dimensions to the CSS-VF yielded a valid 60-item instrument for assessment of care provided in breast cancer. These promising results now require further investigations of its responsiveness and its robustness in other linguistic, cultural and healthcare settings.
PMCID: PMC1933545  PMID: 17634110
16.  Linguistic validation of the Alberta Context Tool and two measures of research use, for German residential long term care 
BMC Research Notes  2014;7:67.
To study the association between organizational context and research utilization in German residential long term care (LTC), we translated three Canadian assessment instruments: the Alberta Context Tool (ACT), Estabrooks’ Kinds of Research Utilization (RU) items and the Conceptual Research Utilization Scale. Target groups for the tools were health care aides (HCAs), registered nurses (RNs), allied health professionals (AHPs), clinical specialists and care managers. Through a cognitive debriefing process, we assessed response processes validity–an initial stage of validity, necessary before more advanced validity assessment.
We included 39 participants (16 HCAs, 5 RNs, 7 AHPs, 5 specialists and 6 managers) from five residential LTC facilities. We created lists of questionnaire items containing problematic items plus items randomly selected from the pool of remaining items. After participants completed the questionnaires, we conducted individual semi-structured cognitive interviews using verbal probing. We asked participants to reflect on their answers for list items in detail. Participants’ answers were compared to concept maps defining the instrument concepts in detail. If at least two participants gave answers not matching concept map definitions, items were revised and re-tested with new target group participants.
Cognitive debriefings started with HCAs. Based on the first round, we modified 4 of 58 ACT items, 1 ACT item stem and all 8 items of the RU tools. All items were understood by participants after another two rounds. We included revised HCA ACT items in the questionnaires for the other provider groups. In the RU tools for the other provider groups, we used different wording than the HCA version, as was done in the original English instruments. Only one cognitive debriefing round was needed with each of the other provider groups.
Cognitive debriefing is essential to detect and respond to problematic instrument items, particularly when translating instruments for heterogeneous, less well educated provider groups such as HCAs. Cognitive debriefing is an important step in research tool development and a vital component of establishing response process validity evidence. Publishing cognitive debriefing results helps researchers to determine potentially critical elements of the translated tools and assists with interpreting scores.
PMCID: PMC3913324  PMID: 24479645
Cognitive debriefing; Translation; Alberta context tool; Estabrooks’ kind of research utilization items; Conceptual research utilization scale; Organizational context; Research utilization; Residential long term care
17.  Internal construct validity of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS): a Rasch analysis using data from the Scottish Health Education Population Survey 
The Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) was developed to meet demand for instruments to measure mental well-being. It comprises 14 positively phrased Likert-style items and fulfils classic criteria for scale development. We report here the internal construct validity of WEMWBS from the perspective of the Rasch measurement model.
The model was applied to data collected from 779 respondents in Wave 12 (Autumn 2006) of the Scottish Health Education Population Survey. Respondents were aged 16–74 (average 41.9) yrs.
Initial fit to model expectations was poor. The items 'I've been feeling good about myself', 'I've been interested in new things' and 'I've been feeling cheerful' all showed significant misfit to model expectations, and were deleted. This led to a marginal improvement in fit to the model. After further analysis, more items were deleted and a strict unidimensional seven item scale (the Short Warwick Edinburgh Mental Well-Being Scale (SWEMWBS)) was resolved. Many items deleted because of misfit with model expectations showed considerable bias for gender. Two retained items also demonstrated bias for gender but, at the scale level, cancelled out. One further retained item 'I've been feeling optimistic about the future' showed bias for age. The correlation between the 14 item and 7 item versions was 0.954.
Given fit to the Rasch model, and strict unidimensionality, SWEMWBS provides an interval scale estimate of mental well-being.
A short 7 item version of WEMWBS was found to satisfy the strict unidimensionality expectations of the Rasch model, and be largely free of bias. This scale, SWEMWBS, provides a raw score-interval scale transformation for use in parametric procedures. In terms of face validity, SWEMWBS presents a more restricted view of mental well-being than the 14 item WEMWBS, with most items representing aspects of psychological and eudemonic well-being, and few covering hedonic well-being or affect. However, robust measurement properties combined with brevity make SWEMWBS preferable to WEMWBS at present for monitoring mental well-being in populations. Where face validity is an issue there remain arguments for continuing to collect data on the full 14 item WEMWBS.
PMCID: PMC2669062  PMID: 19228398
18.  An Item Response Theory (IRT) Analysis of the Short Inventory of Problems-Alcohol and Drugs (SIP-AD) among non-treatment seeking Men-Who-Have-Sex-With-Men: Evidence for a shortened 10-item SIP-AD 
Addictive behaviors  2009;34(11):948-954.
The Short Inventory of Problems-Alcohol and Drugs (SIP-AD) is a 15-item measure that assesses concurrently negative consequences associated with alcohol and illicit drug use. Current psychometric evaluation has been limited to classical test theory (CTT) statistics, and it has not been validated among non-treatment seeking men-who-have-sex-with-men (MSM). Methods from Item Response Theory (IRT) can improve upon CTT by providing an in-depth analysis of how each item performs across the underlying latent trait that it is purported to measure. The present study examined the psychometric properties of the SIP-AD using methods from both IRT and CTT among a non-treatment seeking MSM sample (N = 469). Participants were recruited from the New York City area and were asked to participate in a series of studies examining club drug use. Results indicated that five items on the SIP-AD demonstrated poor item misfit or significant differential item functioning (DIF) across race/ethnicity and HIV status. These five items were dropped and two-parameter IRT analyses were conducted on the remaining 10 items, which indicated a restricted range of item location parameters (−.15 to −.99) plotted at the lower end of the latent negative consequences severity continuum, and reasonably high discrimination parameters (1.30 to 2.22). Additional CTT statistics were compared between the original 15-item SIP-AD and the refined 10-item SIP-AD and suggest that the differences were negligible with the refined 10-item SIP-AD indicating a high degree of reliability and validity. Findings suggest the SIP-AD can be shortened to 10 items and appears to be a non-biased reliable and valid measure among non-treatment seeking MSM.
PMCID: PMC2726268  PMID: 19564078
Item Response Theory; Reliability; Validity; Alcohol Use; Drug Use
19.  Comparison of the uptake of health assessment items for Aboriginal and Torres Strait Islander people and other Australians: Implications for policy 
Health Assessment (HA) items were introduced in 1999 for Aboriginal and Torres Strait Islander people aged at least 55 years and all Australians aged over 75 years. In 2004 a new item was introduced for HAs among adult Aboriginal and Torres Strait Islander people aged 15–54 years. The new item has been applauded as a major policy innovation however this enthusiasm has been tempered with concern about potential barriers to its uptake. In this study we aim to determine whether there are disparities in uptake of HA items for Aboriginal and Torres Strait Islander people compared to other Australians.
The analysis was based on Health Insurance Commission data. Indigenous status was ascertained based on the item number used. Logistic regression was used to compare uptake of HA items for older people among Aboriginal and Torres Strait Islander people compared to other Australians. Adjustments were made for dual eligibility. Uptake of the HA items for older people was compared to the uptake of the new item for Aboriginal and Torres Strait Islander people aged 15–44 years.
Our analyses suggest a significant and persistent disparity in the uptake of items for older patients among Aboriginal and Torres Strait Islander people compared to other Australians. A similar disparity appears to exist in the uptake of the new adult Aboriginal and Torres Strait Islander HA item.
Further engagement of primary care providers and the community around the uptake of the new HA items may be required to ensure that the anticipated health benefits eventuate.
PMCID: PMC1239906  PMID: 16150154
20.  Development and preliminary psychometric testing of a new OA pain measure – an OARSI/OMERACT initiative 
Osteoarthritis and Cartilage  2008;16(4):409-414.
To evaluate the measurement properties of a new osteoarthritis (OA) pain measure.
The new tool, comprised of 12 questions on constant vs intermittent pain was administered by phone to 100 subjects aged 40+ years with hip or knee OA, followed by three global hip/knee questions, the Western Ontario and McMaster Universities (WOMAC) pain subscale, the symptom subscales of the Hip Disability and OA Outcome Score (HOOS) or Knee Injury and OA Outcome Score (KOOS), and the limitation dimension of the Late Life Function and Disability Instrument (LLFDI). Test-retest reliability was assessed by re-administration after 48–96 h. Item response distributions, inter-item correlations, item-total correlations and Cronbach's alpha were assessed. Principle component analysis was performed and test-retest reliability was assessed by intra-class correlation coefficient (ICC).
There was good distribution of response options across all items. The mean intensity was higher for intermittent vs constant pain, indicating subjects could distinguish the two concepts. Inter-item correlations ranged from 0.37 to 0.76 indicating no item redundancy. One item, predictability of pain, was removed from subsequent analyses as correlations with other items and item-total correlations were low. The 11-item scale had a corrected inter-item correlation range of 0.54–0.81 with Cronbach's alpha of 0.93 for the combined sample. Principle components analysis demonstrated factorial complexity. As such, scoring was based on the summing of individual items. Test-retest reliability was excellent (ICC 0.85). The measure was significantly correlated with each of the other measures [Spearman correlations −0.60 (KOOS symptoms) to 0.81 (WOMAC pain scale)], except the LLFDI, where correlations were low.
Preliminary psychometric testing suggests this OA pain measure is reliable and valid.
PMCID: PMC3268063  PMID: 18381179
Osteoarthritis; Hip; Knee; Pain; Outcome measure; Validation; Instrument development
21.  Scoring and psychometric properties of the Eye-Drop Satisfaction Questionnaire (EDSQ), an instrument to assess satisfaction and compliance with glaucoma treatment 
BMC Ophthalmology  2010;10:1.
The objective of this study was to ascertain the scoring and assess the psychometric properties of the Eye-Drop Satisfaction Questionnaire (EDSQ), a 43-item Patient-Reported Outcome instrument developed to assess patients' satisfaction and compliance with glaucoma treatment.
The EDSQ was administered during an observational, retrospective study to 184 French patients treated for glaucoma. The hypothesized structure, including six dimensions (patient-clinician relationship; patient experience; patient-treatment interaction; apprehension; patient knowledge; travel), was tested by assessing the internal consistency reliability (Cronbach's alpha) and construct-related validity (item convergent and discriminant validity). As unsatisfactory results were demonstrated, another structure was defined using a principal component analysis (PCA) combined with content of items. Psychometric properties of this new structure were assessed. Scores were compared between low, moderate and high compliance profile groups defined using data collected with the Travalert electronic device.
Analyses were performed with the 169 patients who completed at least half of the EDSQ items. The hypothesized structure showed a Cronbach's alpha lower than 0.70 for four dimensions out of six and an overall poor construct-related validity (range of item-scale correlations: 0.00-0.68). The new structure obtained with the PCA included six dimensions: concern about treatment (five items); concern about disease (two items); satisfaction with patient-clinician relationship (five items); positive beliefs (three items); treatment convenience (three items); and self-declared compliance (three items). A score ranging from 0 to 100 was calculated for each dimension, with higher scores indicating more of the attribute referred to in the dimension. Internal consistency reliability was good (Cronbach's alpha greater than 0.70 for five dimensions). The structure offered good construct-related validity (range of item-scale correlations: 0.36-0.82). Ceiling effects of 21% and 49%, were observed for the satisfaction with patient-clinician relationship and self-declared compliance scores. Patients in low compliance profile group reported the lowest score for the satisfaction with patient-clinician relationship, positive beliefs, treatment convenience and self-declared compliance dimensions, and the highest score for the concern about treatment dimension.
The scoring of the EDSQ was developed and the questionnaire proved to have satisfactory psychometric properties. EDSQ scores showed a promising relationship to compliance profiles. The EDSQ could be used in future studies.
PMCID: PMC2832632  PMID: 20122146
22.  Print media coverage of California's smokefree bar law 
Tobacco Control  2001;10(2):154-160.
OBJECTIVE—To assess the print media coverage of California's smokefree bar law in the state of California.
DESIGN—Content analysis of newspaper, trade journal, and magazine items.
SUBJECTS—Items regarding the smokefree bar law published seven months before and one year following the implementation of the smokefree bar law (June 1997 to December 1998). Items consisted of news articles (n = 446), opinion editorials (n = 31), editorials (n = 104), letters to the editor (n = 240), and cartoons (n = 10).
MAIN OUTCOME MEASURES—Number and timing of publication of items, presence of tobacco industry arguments or public health arguments regarding law, positive, negative, and neutral views of opinion items published.
RESULTS—53% of items published concerning the smokefree bar law were news articles, 47% were opinion items. 45% of items regarding the smokefree bar law were published during the first month of implementation. The tobacco industry dominated coverage in most categories (economics, choice, enforcement, ventilation, legislation, individual quotes), except for categories public health used the most frequently (government role, tactics, organisational quotes). Anti-law editorials and letters to the editor were published more than pro-law editorials and letters. Region of the state, paper size, presence of local clean indoor air legislation, and voting on tobacco related ballot initiatives did not have an impact on the presence of opinion items.
CONCLUSIONS—The tobacco industry succeeded in obtaining more coverage of the smokefree bar law, both in news items and opinion items. The tobacco industry used historical arguments of restricting freedom of choice and economic ramifications in fighting the smokefree bar law, while public health groups focused on the worker protection issue, and exposed tobacco industry tactics. Despite the skewed coverage, public health groups obtained adequate attention to their arguments to keep the law in effect.

Keywords: content analysis; politics; passive smoking; smokefree bar law; California
PMCID: PMC1747550  PMID: 11387536
23.  Some Memories are Odder than Others: Judgments of Episodic Oddity Violate Known Decision Rules 
Journal of memory and language  2011;64(4):299-315.
Current decision models of recognition memory are based almost entirely on one paradigm, single item old/new judgments accompanied by confidence ratings. This task results in receiver operating characteristics (ROCs) that are well fit by both signal-detection and dual-process models. Here we examine an entirely new recognition task, the judgment of episodic oddity, whereby participants select the mnemonically odd members of triplets (e.g., a new item hidden among two studied items). Using the only two known signal-detection rules of oddity judgment derived from the sensory perception literature, the unequal variance signal-detection model predicted that an old item among two new items would be easier to discover than a new item among two old items. In contrast, four separate empirical studies demonstrated the reverse pattern: triplets with two old items were the easiest to resolve. This finding was anticipated by the dual-process approach as the presence of two old items affords the greatest opportunity for recollection. Furthermore, a bootstrap-fed Monte Carlo procedure using two independent datasets demonstrated that the dual-process parameters typically observed during single item recognition correctly predict the current oddity findings, whereas unequal variance signal-detection parameters do not. Episodic oddity judgments represent a case where dual- and single-process predictions qualitatively diverge and the findings demonstrate that novelty is “odder” than familiarity.
PMCID: PMC3402237  PMID: 22833695
Episodic Memory; Recognition; Cognitive Models
24.  PROMIS® Parent Proxy Report Scales: an item response theory analysis of the parent proxy report item banks 
The objective of the present study is to describe the item response theory (IRT) analysis of the National Institutes of Health (NIH) Patient Reported Out-comes Measurement Information System (PROMIS®) pediatric parent proxy-report item banks and the measurement properties of the new PROMIS® Parent Proxy Report Scales for ages 8–17 years.
Parent proxy-report items were written to parallel the pediatric self-report items. Test forms containing the items were completed by 1,548 parent–child pairs. CCFA and IRT analyses of scale dimensionality and item local dependence, and IRT analyses of differential item functioning were conducted.
Parent proxy-report item banks were developed and IRT parameters are provided. The recommended unidimensional short forms for the PROMIS® Parent Proxy Report Scales are item sets that are subsets of the pediatric self-report short forms, setting aside items for which parent responses exhibit local dependence. Parent proxy-report demonstrated moderate to low agreement with pediatric self-report.
The study provides initial calibrations of the PROMIS® parent proxy-report item banks and the creation of the PROMIS® Parent Proxy-Report Scales. It is anticipated that these new scales will have application for pediatric populations in which pediatric self-report is not feasible.
PMCID: PMC3791923  PMID: 21971875
PROMIS®; Parent proxy report; Item response theory
25.  Initial Adult Health Item Banks and First Wave Testing of the Patient-Reported Outcomes Measurement Information System (PROMIS™) Network: 2005–2008 
Journal of clinical epidemiology  2010;63(11):1179-1194.
Patient-reported outcomes (PROs) are essential when evaluating many new treatments in health care, yet current measures have been limited by a lack of precision, standardization and comparability of scores across studies and diseases. The Patient-Reported Outcomes Measurement Information System (PROMIS™) provides item banks that offer the potential for PRO measurement that is efficient (minimizes item number without compromising reliability) flexible (enables optional use of interchangeable items), and precise (has minimal error in estimate) measurement of commonly-studied PROs. We report results from the first large-scale testing of PROMIS items.
Study Design and Setting
Fourteen item pools were tested in the U.S. general population and clinical groups using an online panel and clinic recruitment. A scale-setting sub-sample was created reflecting demographics proportional to the 2000 U.S. census.
Using item response theory (graded response model), 11 item banks were calibrated on a sample of 21,133, measuring components of self-reported physical, mental and social health, along with a 10-item global health scale. Short forms from each bank were developed and compared to the overall bank as well as with other well-validated and widely accepted (“legacy”) measures. All item banks demonstrated good reliability across the majority of the score distributions. Construct validity was supported by moderate to strong correlations with legacy measures.
PROMIS item banks and their short forms provide evidence they are reliable and precise measures of generic symptoms and functional reports comparable to legacy instruments. Further testing will continue to validate and test PROMIS items and banks in diverse clinical populations.
PMCID: PMC2965562  PMID: 20685078
Outcome Measures; Quality of life; Chronic disease

Results 1-25 (214460)