|Home | About | Journals | Submit | Contact Us | Français|
No treatments for axonal peripheral neuropathy are approved by the United States Food and Drug Administration (FDA). Although patient- and clinician-reported outcomes are central to evaluating neuropathy symptoms, they can be difficult to assess accurately. The inability to identify efficacious treatments for peripheral neuropathies could be due to invalid or inadequate outcome measures.
This systematic review examined the content validity of symptom-based measures of diabetic peripheral neuropathy, HIV neuropathy, and chemotherapy-induced peripheral neuropathy.
Use of all FDA-recommended methods to establish content validity was only reported for 2 of 18 measures. Multiple sensory and motor symptoms were included in measures for all 3 conditions; these included numbness, tingling, pain, allodynia, difficulty walking, and cramping. Autonomic symptoms were less frequently included.
Given significant overlap in symptoms between neuropathy etiologies, a measure with content validity for multiple neuropathies with supplemental disease-specific modules could be of great value in the development of disease-modifying treatments for peripheral neuropathies.
Even within a single etiology, peripheral neuropathy presents as a diverse array of sensory, motor, and autonomic symptoms of varying severity. The heterogeneity of these signs and symptoms increases when neuropathies of multiple etiologies are considered, therefore, peripheral neuropathy can be difficult to evaluate quantitatively in clinical trials. No treatments for axonal peripheral neuropathy are approved by the U.S. Food and Drug Administration (FDA) or Europe-an Medicines Agency (EMA), with the exception of tafamidis, which is approved for familial amyloid neuropathy, an uncommon polyneuropathy. The failure to identify disease-modifying treatments for peripheral neuropathy may be due in part to limitations of available outcome measures.1–4
The FDA held a public workshop on “Clinical Development Programs for Disease-Modifying Agents for Peripheral Neuropathy” in February 2013 that included presentations and discussions of outcome measures that could be used to assess the efficacy of disease-modifying agents. There was general agreement that the evaluation of disease modification in peripheral neuropathy can be improved, and this review was undertaken to identify next steps to modify existing peripheral neuropathy measures or develop novel measures. Although many measures of peripheral neuropathy exist, no systematic evaluation of the content validity (i.e., the extent to which an instrument measures the concept of interest)5 has been performed for these measures. Published references outline well-specified criteria to demonstrate that a self-report measure has at least achieved minimal standards to be used in research, clinical practice, and clinical trials.6 The FDA has published a guidance that describes the process for developing and subsequently qualifying patient- or clinician-reported outcome (PRO or CRO) measures that outlines steps to be taken to satisfy the above-mentioned criteria. Although the guidance outlines a development process that would be used specifically as a basis for regulatory approval and labeling,5,7 this guidance could also serve as a framework for developing valid and reliable measures for research outside the purview of the regulatory process. To ensure that a measure adequately captures the intended concept of use, the initial stages of development should include a comprehensive assessment of content validity in the intended population of use that is based on, review of available literature, expert input, and contributions from patients.5
The goals of this review are to summarize the published evidence of content validity for existing symptom measures in 3 prevalent conditions for which outcome measures have been developed and to compare and contrast the content and format of the items in disease-specific neuropathy measures. Achieving these goals will support the appropriate selection of currently available PRO and CRO measures of patient-reported symptoms. These data can serve as a foundation for development of novel symptom measures and refinement of existing tools for use in clinical trials in patients who have diverse peripheral neuropathies.
Measures used to evaluate the symptoms of diabetic peripheral neuropathy (DPN), HIV neuropathy, and chemotherapy-induced peripheral neuropathy (CIPN) were identified by searching PubMed (search criteria: [neuropathy] AND [diabetes OR HIV OR chemotherapy] AND [measure OR scale OR patient reported outcome]) and by asking all authors to review the compiled list to determine whether there were any missing measures. The content validity of clinician-based assessments using objective measures of peripheral neuropathy, such as nerve conduction studies, quantitative sensory testing, autonomic testing, and nerve or skin biopsy and of observations made by others in patients who cannot respond for themselves (e.g., parents or caregivers) is beyond the scope of this review. The following 4 searches in PubMed were subsequently performed to identify publications that evaluated content validity for each of the measures identified using the aforementioned methods: (1) “measure name or nickname AND focus group”; (2) “measure name or nickname AND content validity”; (3) “measure name or nickname AND interview”, (4) “measure name or nickname AND development”.
The format used for each measure was identified as: (1) verbal descriptor scales, (2) dichotomous (yes / no) questions, (3) numeric rating scales, or (4) some combination. The body regions assessed by the measures were identified (lower extremity only, upper extremity only, both lower and upper extremities), and a determination was made as to whether they were assessed in the same questions (e.g., numbness in hands and feet is 1 question) or separate questions (e.g., numbness in the hands is 1 question, and numbness in the feet is another). In this review, we refer to the specific questions included in the measures as the “items” and the content that was covered within each item as the “construct.” The constructs of the items from each measure were conceptually categorized into 1 of 3 domains associated with peripheral neuropathy: sensory, motor, or autonomic. The sensory domain included positive and negative symptoms associated with pain, touch, and thermal perception. The motor domain included items that assessed weakness, cramping, or difficulty manipulating objects due to symptoms. The autonomic domain included items that assessed orthostatic intolerance, urogenital dysfunction, and gastrointestinal disturbances.
For the measures that separated the items into sensory, motor, or autonomic domains, the categorization of the original developers of the measure was followed when assigning items to these domains. For measures that did not specify domains, or whose domains were different than the 3 that were adopted for this review, we categorized the items into the 3 organizing domains. Items that were worded differently, but appeared to cover similar constructs, were grouped together. For example, 1 measure8 assessed difficulty manipulating small objects and others (e.g., Shimozuma et al.9) listed specific activities such as fastening buttons or inserting contact lenses. These 2 sets of items were grouped into the same construct labeled “difficulty with manipulating small objects or fine finger movements”. Conversely, when 2 constructs were compounded in a single item, this item was characterized under both constructs. For example, 1 measure10 assessed numbness and tingling in a single question. This item was included under both the numbness and tingling constructs.
Some items included in the measures covered social participation or activities of daily living; these items were not included in this summary. These items occurred in 1 DPN measure11 and 4 CIPN measures.9,12–15 Venn diagrams (Figs. 1–3) were used to depict the overlap of content assessed by measures developed for each of the 3 conditions within each of the 3 domains, respectively. Only items that are assessed in at least 2 measures (regardless of condition) were included in the Venn diagrams. Supplementary Tables S1 and S2, available online, were used to present the individual items that occur in each measure, including those that appeared in only 1 measure.
A total of 18 measures were identified by the search methods and author input (Table 1). Supplementary Table S1 presents the number of citations that were identified by the content validity literature search, the number subsequently excluded because they did not address content validity, and the number for each measure included in this review. We identified reports of research that investigated and described methods that provided support for the content validity of 11 of the 18 measures (Table 1). Of the 18 measures, 9, 6, and 2 were designed for and primarily used in CIPN, DPN, and HIV-neuropathy, respectively. The Total Neuropathy Score16 is commonly used for both DPN and CIPN, but it was included in the DPN category for purposes of the Venn diagrams because it was first developed for DPN patients (Figs. 1–3). Of the 6 measures developed for use in DPN, published information for 3 describe at least 1 method to establish content validity. We found no published information regarding content validity for the 2 measures developed specifically for HIV-neuropathy. Eight of the 9 measures developed for CIPN report at least 1 method to establish content validity (Table 1).
The majority of the measures examined (67%) use verbal descriptor scale items (Table 2). Neuropathy symptoms are most commonly assessed in separate questions for both extremities (50%). Five measures (28%) assess neuropathy in only the lower extremities, and none focus solely on the upper extremities. The remaining measures either assess symptoms in both extremities within the same questions or do not indicate what extremity should be considered when reporting the symptoms (Table 3). Sixty-seven percent of measures group the items into domains or subscales. All of the measures contain items that were identified as sensory items in this review; all but 1 HIV-neuropathy measure includes items that were identified as motor items. Sixty-one percent of measures include items that were identified as those that address autonomic symptoms.
Venn diagrams are provided to depict the number of measures that assess each construct and the overlap of the constructs assessed among the 3 conditions [see Figs. 1, ,2,2, and and33 for sensory, motor, and autonomic domains, respectively]. These diagrams also illustrate which constructs are evaluated in measures designed for 1, 2, or all 3 conditions. Numbness, tingling, pain, allodynia (pain from increased sensitivity to light touch from items such as bed covers or activities such as putting on gloves), altered warm and cold perception, difficulty feeling the feet (when walking), burning pain, sharp pain, and burning (not specified as painful) were the most common sensory symptoms and are evaluated in at least 1 measure for each of the 3 conditions. Difficulty or weakness when walking and cramping were the most common motor symptoms evaluated and are included in scales for all 3 conditions. Supplementary Table S2 contains all items included in the individual measures.
We found that researchers reported using at least 1 of the methods for establishing content validity recommended by the FDA (literature reviews or expert or patient input) in 11 of the 18 identified instruments.5,7 However, patient input, which is emphasized most strongly in the FDA guidance,5 was only obtained for 8 measures. The explicit use of all 3 sources of input was only reported for 2 measures.8,17 Furthermore, although attempts to establish content validity were often mentioned in the Materials and Methods section of articles that described the development of PRO measures for peripheral neuropathy, the available literature does not clearly indicate that patients were consulted on the clarity of the final sets of items included in these measures. Finally, none of the measures were originally developed based on contributions from >1 disease-specific patient population, although several have subsequently been used in studies of additional conditions. This general low level of effort to establish content validity that adheres to current FDA standards suggests that existing measures may not have adequate assay sensitivity to detect modest, yet clinically relevant, disease-modifying treatment effects. Minor problems with outcome measures would not be likely to greatly limit the ability of a trial to detect the effect of a highly effective treatment. However, considering the current dearth of available disease-modifying treatments for axonal peripheral neuropathy, even treatments with modest effects would be valuable.
Our results suggest that it could be advantageous to use available measures that include the highest number of symptom constructs that were most commonly included in measures for all 3 conditions, especially when studying conditions for which no dedicated measures have been developed. Items classified as related to autonomic neuropathy were less common, and none of these items are included in at least 1 measure designed for each of the 3 conditions.
The frequent overlap in content between the measures designed specifically for the 3 different conditions (see Figs. 1–3) suggests that a single “generic” measure of peripheral neuropathy consisting of material in common accompanied by disease-specific modules could be a valuable approach. Combining a general module with disease-specific add-on modules is a potentially efficient method to approach FDA qualification of peripheral neuropathy outcome measures that prioritizes the importance of disease-specific measurements that are validated for the appropriate concepts of interest.
Eventual qualification and inclusion of such a measure in the FDA Clinical Outcome Assessment Compendium18 will require that the measure be validated as described in FDA guidances on development and qualification of PRO measures5 and drug development tools.7 For example, the constructs common to scales from all 3 conditions identified in this review could provide a starting point to establish a conceptual framework for peripheral neuropathy. Outlining the exact constructs to be included, and the optimum wording of the questions, in such a “generic” measure would need to be determined using input from focus groups or interviews with demographically and clinically diverse patients with peripheral neuropathy of each etiology for which the measure would be used in future studies. The constructs that were assessed in measures developed for only a single condition could serve as the basis for disease-specific modules; for example, various autonomic items have specifically been included in measures developed for CIPN (e.g., head rush upon standing) and DPN (e.g., problems with vaginal dryness and bloating/vomiting after meals).
The majority of the measures assess symptoms in the lower and upper extremities separately. Evaluation of both upper and lower extremities is important when assessing peripheral neuropathy, because symptoms are often worse in the lower extremities. Any future measure of peripheral neuropathy symptoms should incorporate the separate extremity format. Other formatting issues should be carefully considered, such as the scale used for the questions. The majority of the existing measures use a verbal descriptor scale. In considering the development of a new measure, patient acceptance and understanding of, and satisfaction with verbal descriptor scales should be examined in focus groups and compared with other formats including the 0–10 numeric rating scales that are commonly used to assess pain symptoms.19
Once a list of potential PRO items has been compiled, the set of questions can be administered in a pilot study of patients who experience peripheral neuropathy and can be evaluated using psychometric methods, such as factor analysis and item response theory (IRT) modeling, including Rasch analyses.20,21 Content experts and psychometricians can refine the concept of interest by identifying the items that are most pertinent to a set of sample patients and create valid and reliable scales for assessing peripheral neuropathy.5 After implementation of these measure-refining methods, the refined content and instructions should be evaluated again by patients and experts for content validity. However, confirmation of content validity is only the first step in demonstrating measure validity. Longitudinal clinical validation studies should be conducted to evaluate the test–retest reliability, construct validity (including convergent and discriminant validity), and the responsiveness (sensitivity to change) of the PRO measure.5,6 These methods should be considered in the development of new measures of general peripheral neuropathy that would be designed to address limitations of existing approaches and that could include condition-specific modules.
We have focused our systematic review on DPN, HIV-neuropathy, and CIPN, 3 prevalent distal, symmetric, sensorimotor axonal peripheral neuropathies. There are, of course, many other axonal polyneuropathies for which outcome measures have been developed, but we considered them to be beyond the scope of this review. One important example involves transthyretin familial amyloid polyneuropathy (TTR-FAP).22 The results of recent randomized clinical trials of tafamidis23 and diflunisal24 showed benefits in patients with TTR-FAP on ClinRO and PRO measures, for example, the Norfolk QOL-DN25; in addition, Rasch-built symptom and disability measures have been developed for use in future clinical trials.26 These recent studies of TTR-FAP have the potential to be used as models to address challenging issues in clinical trial design27 and the development of novel PROs5 for disease-modifying treatments for the peripheral neuropathies we have discussed.
This study is limited in that we were only able to assess the published material regarding the content validity of the measures we reviewed. Further efforts may have actually been taken to establish content validity but are not reported in the literature. In addition, the content identified by overlap of items in the various measures is dependent on choices made by previous researchers. For this study, we did not conduct focus groups with experts and patients to identify symptoms that should be included in assessments of peripheral neuropathy as a basis for comparing existing measures and evaluating their content validity. However, inclusion of items by multiple independent researchers indicates that the content is likely important and, thus is a valuable starting point upon which to develop a new measure or modify an existing one. The protocol of our study, that is, following the categorization of the original developers of a measure in assigning an item to a domain, may have resulted in inclusion or categorization of items that some investigators might consider outside the concept or domain of interest. Qualitative research based on these results should be performed to determine which of the items should be retained in future, optimized measures. Finally, we did not review other types of validity, such as construct validity or responsiveness, which are important when evaluating the overall value of individual measures.
In summary, we reviewed and summarized published information on the content validity of existing measures of DPN, HIV-neuropathy, and CIPN and compared the specific symptom constructs assessed in each of the measures. By determining the overlap in their content, we identified the set of symptoms that have been considered as most important in the development of 18 different measures. This information can be used to help inform decisions regarding which of the existing measures to use in a particular context, and can also serve as a basis for determining what symptoms should be included in novel measures of peripheral neuropathy that show content validity across multiple etiologies, perhaps with disease specific modules. Publication of the results of future efforts to develop measures should report how the content validity of the instrument was addressed.
We thank Drs. Sharon H. Hertz, Allison H. Lin, and Rigoberto Roca for their contributions to the ACTTION meeting on evaluation of peripheral neuropathy.
Funding: Financial support for this project was provided by the ACTTION public-private partnership, which has received research contracts, grants, or other revenue from the FDA, multiple pharmaceutical and device companies, and other sources. J.W.R.’s contribution was supported in part by the Department of Veterans Affairs (Biomedical and Laboratory Research Service and Rehabilitation Research and Development, 101RX001030), Baltimore Geriatric Research, Education, and Clinical Center (GRECC).
Manuscript Review/Approval: This manuscript was reviewed and approved by the Executive Committee of the ACTTION public-private partnership with the United States Food and Drug Administration.
Author Contributions: J.S. Gewandter was involved in the conception, design, analysis, and interpretation of the study. She drafted and revised the manuscript. R. Free-man and R.H. Dworkin were involved in the conception, design, and interpretation of the study and contributed to drafting and revising the manuscript. All other authors contributed to the conception of the study and intellectual revisions of the manuscript.
Potential Conflicts of Interest: The views expressed in this article are those of the authors and no official endorsement by the Food and Drug Administration (FDA), the U.S. Army, or the pharmaceutical and device companies that provided unrestricted grants to support the activities of the ACTTION public-private partnership should be inferred. B.B. Reeve was involved in developing the Patient-Reported Outcomes Measurement Information System (PROMIS) Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). All other authors have no relevant conflicts of interest to disclose.
Additional supporting information may be found in the online version of this article.