|Home | About | Journals | Submit | Contact Us | Français|
To explore the routine and effective use of genetic testing for patients with intellectual disability and developmental delay (ID/DD), we conducted a prospective, randomized observational study of 231 general pediatricians (40%) and specialists (60%), using simulated patients with 9 rare pediatric genetic illnesses. Participants cared for 3 randomly assigned simulated patients, and care responses were scored against explicit evidence-based criteria. Scores were calculated as a percentage of criteria completed. Care varied widely, with a median overall score of 44.7% and interquartile range of 36.6% to 53.7%. Diagnostic accuracy was low: 27.4% of physicians identified the correct primary diagnosis. Physicians ordered chromosomal microarray analysis in 55.7% of cases. Specific gene sequence testing was used in 1.4% to 30.3% of cases. This study demonstrates that genetic testing is underutilized, even for widely available tests. Further efforts to educate physicians on the clinical utility of genetic testing may improve diagnosis and care in these patients.
Children with intellectual disability (ID), developmental delay (DD), or autism spectrum disorder (ASD) have historically been grouped by phenotypical characteristics and their associated cognitive impairments.1 Until recently, physicians who treat these patients have had limited understanding of the genetic basis for these syndromes.2,3 By one estimate, a quarter of all patients had a delay of 5 to 30 years between the onset of symptoms and receiving a definitive diagnosis.4 Early diagnoses and treatments are, thus, often problematic because telltale manifestations were difficult to identify or only appeared later in the syndrome.5
Newer genetic testing platforms have the potential to revolutionize the diagnosis, timing, and treatment of children with DDs.1,6-8 One of these tests, chromosomal microarray analysis (CMA), is a microchip-based test that automates simultaneous analysis of many pieces of DNA across multiple chromosomal regions. CMA provides higher resolution and detects many more of potentially causal deletions and duplications (copy number variants [CNVs]) involved in ID, DD, and ASDs, improving diagnosis over earlier technologies such as karyotyping and fluorescence in situ hybridization.9 Next-generation sequencing (NGS) is an even higher resolution technique that allows detection of single nucleotide variants (SNVs) across a patient’s genome but currently does not have sufficient signal uniformity to allow clinical detection of CNVs. NGS has been shown to deliver an estimated 25% clinical detection rate when used across the whole exome and, as the cost of sequencing continues to fall, promises to provide additional genetic insights into the causes of DD, ID, or ASDs when used in combination with CMA.10,11
As a result, genetic testing is now part of the practice guidelines of the American Academy of Pediatrics as of 2010, the American Academy of Neurology, and the American Academy of Child and Adolescent Psychiatry for children with DD/ID, ASDs, or multiple congenital anomalies.12-14 In 2010, the American College of Medical Genetics (ACMG) called for CMA as a first-tier test for all genetic diagnostic evaluations of these conditions.15
Beyond improving outcomes for DD, ID, and ASD patients, a recent ACMG policy statement highlighted the need for a wider definition and understanding of clinical utility of testing.16 Current definitions of clinical utility link clinical information gathered from testing to improved health outcomes of the patients. Although genetic testing for patients within DD, ID, and ASDs may not always directly affect patient outcomes, it often leads to much better information regarding causality of the health issue and also identifies medically actionable comorbidities and secondary findings.17 If used appropriately, these sophisticated genetic tests can significantly improve clinical care by positively affecting clinical decisions, enhancing diagnostic quality, and improving patient outcomes. For example, the early diagnosis of both Rett and Dravet syndromes leads to proper neuroleptic treatment, and the diagnosis of guanidinoacetate methyltransferase (GAMT) deficiency guides specific changes in diet, which can be life saving for patients. In addition, appropriate diagnosis of disease subtypes may be overlooked if based solely on clinical manifestations, and genetic testing may pave the way for more directed therapies and surveillance. For instance, Mosaic Turner syndrome with XY cell line, unlike the Classic Turner Syndrome, will alert the clinician to the risk for future ovarian malignancies—most commonly gonadoblastoma—and contemplate on gonadectomy. These risk profiles are not applicable to Turner syndrome without the XY cell line or to other types of VHL. These risk profiles are not applicable to Turner syndrome without the XY cell line or to other types of VHL.
CMA testing, however, is not consistently reimbursed, limiting its use in clinical practice and obviating the potential advantages of informative genetic results.10 Reimbursement for NGS is even less consistent, with high rates of denials because of lack of coverage or classification as experimental or investigational.18 Compared with broader CMA testing and whole exome sequencing, narrower genetic tests, such as targeted sequencing of the GAMT gene or FOXG1 gene, that look for only a single genetic abnormality can be difficult to interpret. A negative result may be a result of the fact that the patient does not have a mutation in the specific gene syndrome being evaluated but may have other genetic causes that would surface through the broader CMA- or NGS-based testing.
Despite guidelines and the clinical utility of accurate and earlier diagnosis, genetic testing is still underused.16 Whereas evidence of utility and lack of coverage by payers is one important hurdle to testing,19 there is a fundamental concern that pediatricians may not be keeping pace with the emergence and utility of these new tests.20
In this article, we explore the extent to which genetic testing is integrated into medical care and to which current genetic tests widely available on the market are routinely and effectively used for care of patients with ID/DD. We further analyze the quality of care and the utility of the diagnostic testing. Without better knowledge of the current use of molecular diagnostic testing, it has been challenging to demonstrate clinical utility, particularly given the legion of genetic conditions associated with ID, DD, and ASDs.
Between August 2014 and January 2015, we carried out an evaluation of care among general and specialty pediatricians of children with different DDs—rare diseases with specific genetic etiologies—measuring their care patterns, the quality of their care, and their adoption of genetic testing. We conducted a prospective, randomized observational study, which used simulated patients to overcome case mix variation challenges, for 9 rare pediatric genetic illnesses.
We recruited 231 board-certified physicians who worked in community-based practices. The eligibility requirements included practicing a minimum of 2 but not more than 30 years, board certified as a pediatrician or neurologist, caring for at least 30 pediatric patients per week, and a willingness to complete care for the simulated patients online. Recruitment was done by letter invitation to a subsample of 1000 providers randomly selected from lists of approximately 5000 specialists and 25000 general pediatricians. Respondents were called, screened, and invited to participate until the study sample size of 225 pediatricians was met. A total of 216 physicians ultimately participated in the study and completed 3 Clinical Performance and Value (CPV®) vignettes. Among them, 17 were lost to follow-up: 4 were eventually determined not to meet inclusion criteria; 3 asked to be withdrawn from the study; and 10 were otherwise lost to follow-up. This study was conducted in accordance with ethical standards and approved by the Chesapeake Institutional Review Board (IRB), Columbia, MD. Written informed consent was obtained in writing from all participants, and the trial was listed in clinicaltrials.gov.
Participants completed a 20-item questionnaire at the beginning of data collection, recording physician and practice characteristics as well as a self-assessment of understanding of genetic testing.
To assess physician care quality and utility for patients with a possible underlying genetic diagnosis, we used CPV vignettes to measure the practice process. The 9 CPV vignette cases are simulated patients for whom genetic testing, if obtained, would ostensibly make a difference in their care. CPVs are widely used, validated measures of actual clinical practice that have proven particularly helpful to evaluate the variability and quality of care.21,22 In this study, participants all cared for the same simulated patients, randomly assigned, obviating the impact of patient variability or ordering effects from the analyses and isolating provider practice variability. In designing the 9 cases, care was given to creating a representative range of genetic abnormalities with genetic deficiencies caused by CNVs and sequence variants (see Table 1). For analysis, we further divided the cases into 2 groups—those that would benefit from CMA testing and those that would benefit from additional NGS following an uninformative CMA result.
The pediatricians cared for the simulated patients using the CPV vignette online platform. Their responses generated the study data, with each physician completing a total of 6 randomly assigned cases via a personalized, confidential link. Vignettes mirror an actual clinical visit and the physician-patient interaction and collect data in the 5 domains of care: history taking, conducting a physical examination, ordering laboratory and/or diagnostic imaging tests, making a diagnosis, and prescribing treatment, referrals, and therapies. Physicians move through the case, responding to open-ended questions in each domain regarding the clinical care they would provide for that patient. For the laboratory and diagnostic domain sections, they receive real-time results to any tests ordered. Each case took approximately 15 minutes to complete.
The individual care responses for each completed case were scored against explicit evidence-based criteria as determined by the literature, professional medical associations, and clinical experts. The aggregated results are presented as percentage of criteria completed according to these explicit criteria. Overall scores were calculated, along with subscores for each of the 5 domains of history taking, physical exam, workup, diagnosis, and treatment and therapeutic plan. We specifically assessed items such as the frequency and type of genetic tests used for each patient (eg, karyotyping, CMA testing, specific gene sequencing tests, or NGS).
Descriptive data were prepared for the physician and practice characteristics. With the CPV vignette scores, t tests were conducted to establish the significance of average scores within the distinct physician groups. Box plots were constructed to describe variability among physicians. To assess the physician characteristics associated with ordering CMA testing or gene sequencing (for those cases where indicated), we performed multiple linear regression analysis. All data analyses were conducted using STATA 13.0 (College Station, TX).
A total of 216 physicians completed the initial questionnaire and 3 randomly assigned CPV vignettes. The participating pediatricians had a mean age of 46 years and a mean 11 years of practice experience. Participants worked in a range of small to large practices: 57% were single specialty practices (pediatrics), and 40% worked with 10 or more other physicians. By design, generalists constituted 40% of the sample, with the remaining 60% distributed among specialists such as developmental pediatricians (11%), pediatric neurologists (25%), and others (23%) such as child psychiatrists and pediatric hospitalists. The physicians in our sample were almost all employed (93%), worked 4 to 5 days a week (89%), and had a roughly even mix of Medicaid and commercially insured patients (see details in Table 2).
We assessed the level of familiarity with genetic testing as well as the participant’s load of patients with DD, ID, and ASDs. Overall, a quarter of the participants self-assessed as having an average (3 on an ordinal scale of 5) level of understanding of genetic testing. Only 35% used genetic testing frequently or very frequently for their patients with atypical phenotypes. However, 85% of the participants felt that they had a genetics expert—usually a medical geneticist—readily available for consults (see Table 2).
The care of patients with DD, ID, or ASDs varied widely overall, and this variability was especially pronounced in the evaluation and treatment of patients (see Table 3). The median overall score was 44.7%, and the 25th to 75th percentiles ranged from 36.6% to 53.7%, indicating high variability.
Overall, pediatricians indicated the correct care elements 45.5% of the time, when scored against explicit evidence-based criteria. By domain, however, there was a steady decline in the quality of care as the physician moved through clinical encounter from initial evaluation to more complex testing ordering and treatment determination. Taking the patient history and doing the physical examination had the highest scores, but the lowest scores were found in the workup, which includes test ordering, diagnosis, and treatment domains.
We grouped the patient cases into those with CNV-based abnormalities, best diagnosed by CMA (labeled as group 1) and those with SNV-based abnormalities, best diagnosed by Sanger or NGS sequencing (labeled as group 2). We observed that physicians had significantly higher overall CPV, history, and treatment scores (P < .0001) for patients with SNV sequence variants (group 2) than CNV-based abnormalities (group 1). However, treatment domain scores were low for both groups. In addition, physicians were significantly more familiar with the workup of the patients with CNV-based abnormalities (P = .002). Both groups had similarly low diagnosis scores at 38%, and the low use of genetic testing and low workup scores across both case types affects the ability to effectively diagnose patients (P < .0001; see Table 3).
In bivariate analysis, which was carried out to look at the factors linked with higher care quality, having more than 50% of patients from private payers (47.2 vs 44.3; P = .02) and being female (47.0 vs 43.5; P = .004) were associated with a higher quality score. There were no linkages found between overall quality and subspecialists versus generalists, the frequency of testing, or higher patient volumes. Ordering of genetic testing or self-reported rates of understanding genetic testing were not linked to overall quality, either. However, having ready access to both a genetic counselor and a medical geneticist was associated with higher practice quality across the 9 cases (46.7 vs 42.8; reference, no genetics expert; P = .03).
We found that the diagnostic accuracy for DD, as measured by the proportion of physicians making a correct diagnosis, was low, with a little more than a quarter of the cases (27.4%) receiving a correct primary diagnosis (Table 4). When we disaggregated diagnostic accuracy by case type, we found that mosaic Turner syndrome (66.7%) and West syndrome (51.4%) were the most common correctly diagnosed genetic conditions. In contrast, GAMT deficiency (9.3%), Bloom’s syndrome (11.8%), and Dravet syndrome (12.8%) were the least likely to be diagnosed correctly.
Across all 9 cases, we note that physicians order CMA more than half the time (55.7%), but this ranged from a low of 24.3% for X-linked early infantile epileptic encephalopathy (West Syndrome) to a high of 68% for Dravet syndrome (Table 4). Although lower diagnosis scores tended toward more CMA testing, statistical analysis yielded no significant association between the two (P = .161).
The physicians used standard platform CMAs (45.3%) rather than enhanced CMA testing (10.4%) with more probes (eg, tests that include Cytoscan HD and FirstStepDx PLUS; P < .0001). We saw differences between the 2 groups of cases (group 1 requiring CMA for diagnosis vs group 2 requiring gene sequencing), with 58.4% of physicians ordering CMA when it was required (group 1) but only 14.6% ordering gene sequencing when it was required (group 2; P < .0001; see Table 4). Fewer, but still a substantial number of pediatricians, ordered CMA tests for the group 2 cases (34.4%) as a first-line test even though those cases ultimately required gene sequencing.
Specific gene sequence testing was used even less often, from a low of 1.4% for X-linked early infantile epileptic encephalopathy (West Syndrome) and 2.9% for Hunter syndrome, to a high of 30.3% for Bloom syndrome (see Table 4). Specific gene sequencing, although an option for all cases, was required to make the diagnosis for 4 cases (6-9). Even among these 4 cases, we found that physicians ordered gene sequencing infrequently; sequencing was ordered in only 12.8% of the cases where it was required for diagnosis. For Bloom syndrome, which had the highest rates of gene sequencing use of the group 2 cases and for which either 1 of 2 gene sequencing tests would return a diagnosis, only 19.3% of the physicians pursued sequencing, and 7.9% of physicians ordered both genetic sequencing and CMA. What is worthy of note is that there did not appear to be a tendency for the group 1 cases to focus on CMA testing nor the group 2 cases to focus on sequencing. For example, the first and second most common cases for gene sequencing were Rett and Dravet syndromes—group 1 cases. Providers appeared to be ordering either test if a genetic disorder was suspected, but only 5.7% ordered both CMA and gene sequencing. In looking at rates of ordering both tests, for most cases, fewer than 10% of the physicians asked for both tests. Only for case 4—Rett syndrome—did we see a larger proportion of physicians ordering both tests (17.9%).
Genetic testing was indicated for all 9 cases. Using multivariate logistic regression, we looked at the predictors of who ordered genetic testing overall and at predictors for either CMA testing (cases 1-5, group 1) or gene sequencing (cases 6-9, group 2). We found that for overall testing (model A), being a specialist (P = .004), having a readily available genetics expert (P = .007), having a good or excellent understanding of genetic testing (P = .005), and having greater than 20 pediatric patients a week with DD/ID/ASDs (P = .039) all predicted a greater likelihood of ordering a genetic test. We also found that those who were employed were less likely to order genetic testing (P = .03).
We found that for ordering CMA testing (model B, cases 1-5) the predictors were being a specialist (P = .04), having a genetics expert readily available (P = .001), having a higher understanding of genetic testing (P < .0001), and not being employed (see Table 5).
We found that for sequencing cases (model C, cases 6-9), the main predictor of ordering a gene sequencing test was physicians who saw greater than 50 pediatric patients a week (P = .04).
Genetic testing, and CMA in particular, increases diagnostic yield in children with DDs, including autism.23 Increased diagnostic yield, in turn, can lead to better treatment, referrals for care, and better screening for potential comorbidities. Studies have shown that CMA testing leads to important changes in medical management.1,6,24 Among the changes in care observed, critical new actions include medical referrals, new diagnostic tests, and surveillance for complications.25
In this study of a representative sample of community-practicing general and specialist pediatricians, we sought to assess clinical practice patterns using standardized, online simulated patients. In particular, we aimed to assess determinants of care quality and gain insight into how physicians incorporated genetic testing into their practice to properly diagnose and treat patients with DD and ID.
Knowledge of and experience with genetic testing varied among the pediatricians in our sample. On average, participants reported having a middle-level understanding of genetic testing, and 65% reported using genetic testing for patients only sometimes, rarely, or never. However, the majority (85%) did have access to genetic testing expertise—either a medical geneticist, a genetic counselor, or both.
CPV quality scores averaged 45% across the 5 domains of care. Compared with other similarly structured CPV studies, this is a low to normal score.26,27 The relatively low quality score is in part driven by low scores in the workup, diagnosis, and treatment of these clinical cases. Bivariate analysis yielded no differences in CPV score by physician age, subspecialty versus generalist, or the frequency with which the tests were ordered.
Admittedly, individual syndromes within the category of DD/ID/ASDs tend to be rare (http://www.orpha.net), which can in part explain the low diagnosis scores. The cases presented different levels of clinical manifestations/phenotypes of these diseases. Turner and West syndromes, both of which have more recognizable clinical patterns (growth delay, horseshoe kidneys for Turner and infantile spasms with hypsarrhythmia on EEG for West), were diagnosed correctly most frequently by study physicians (67% and 51%, respectively). Rarer diseases, such as Dravet syndrome and GAMT deficiency were correctly diagnosed infrequently. GAMT deficiency (diagnosed correctly 9.3% of the time) has a less notable clinical presentation (failure to thrive, seizures, autism, behavioral disorder, and ID are all possible presentations) and is a very rare disease, with less than 100 identified cases.28,29
Extremely rare and complicated health problems, such as those that are the focus of this study, require a team approach to care. Fortunately, 85% of the participants felt that they had a genetics expert—usually a medical geneticist—on whom they could call, a reassuring number, given the importance of pretest and posttest counseling and physician-patient communication.20,30
Ultimately, however, genetic testing tools, which have become more sophisticated and more accessible in the past decade, are required for effective diagnosis and treatment of patients who present with nonspecific DD/ID/ASDs. Yet this study shows that genetic testing is underutilized, with a little more than half the cases that required CMA receiving it (CNV-based cases) and only 14.6% of the cases that required gene sequencing receiving it (SNV-based cases). We found that predictors of use of genetic testing included being a specialist, having a good understanding of genetic testing, and increased patient load.
Although part of the lack of genetic testing use has been blamed on lack of payer coverage,8 this study demonstrates that genetic testing is still underutilized even when medical coverage is not an issue. Further efforts to educate physicians about the clinical utility of genetic testing can help improve care and treatment of these patients.
Author Contribution: JP contributed to the conception and design; contributed to acquisition, analysis, and interpretation; drafted the manuscript; critically revised the manuscript; gave final approval; and agrees to be accountable for all aspects of work ensuring integrity and accuracy. LDM contributed to the design; contributed to acquisition, analysis, and interpretation; drafted the manuscript; critically revised the manuscript; gave final approval; and agrees to be accountable for all aspects of work ensuring integrity and accuracy. DT-LC contributed to conception and design; contributed to acquisition, analysis, and interpretation; critically revised the manuscript; gave final approval; and agrees to be accountable for all aspects of work ensuring integrity and accuracy. JF contributed to analysis and interpretation; critically revised the manuscript; gave final approval; and agrees to be accountable for all aspects of work ensuring integrity and accuracy. MCA contributed to acquisition, analysis, and interpretation; critically revised the manuscript; gave final approval; and agrees to be accountable for all aspects of work ensuring integrity and accuracy. TB contributed to conception and design; contributed to acquisition, analysis, and interpretation; drafted the manuscript; critically revised the manuscript; gave final approval; and agrees to be accountable for all aspects of work ensuring integrity and accuracy.
Declaration of Conflicting Interests: Dr. Peabody developed the CPV®s and is president of CPV Technologies, LLC, which owns the quality measurement tool used in the study. Otherwise, the author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by Lineagen Inc, Salt Lake City, Utah.