To exercise evidence-based medicine (EBM), physicians need access to full-fledged research reports to critically evaluate study analysis and interpretation. However, surveys dating back to the 1980s identified physicians who had a poor grasp of statistical tests and interpretation of statistical results due to a lack of formal training in biostatistics.[10
] This problem is even more explosive today in light of increased complexity of statistical methods used in the literature.[13
] In response, graduate medical educators have increased training in biostatistics throughout the expanse of medical education. Medical schools have incorporated statistics courses and Accreditation Council for Graduate Medical Education (AGCME) guidelines since residency competency stipulates that residents must have a solid basic foundation in statistical methodology as it pertains to scientific research.[14
] While residency programs address this issue through EBM curricula and journal clubs,[15
] a few, if any, programs focus on selection and interpretation of statistical results.[18
To broadly assess residents' knowledge and skills in EBM, Windish et al.,
conducted a seminal multiprogram assessment of 11 internal medicine residency programs in Connecticut.[19
] By first reviewing research articles in six leading general medical journals between January and March 2005 on the basis of statistical methods used, the researchers developed a survey instrument of questions focused on identifying and interpreting results in the most frequently occurring statistical tests. Questions were multiple-choice, centered on a clinical vignette, and required no calculations. Attitudes and confidence questions were adapted from surveys on the Assessment Resource Tools for Improving Statistical Thinking website, rated on a 5-point Likert scale. This instrument was validated and reformulated by pilot testing the questions on 5 internal medicine faculty with advanced training in biostatistics and 12 primary care internal medicine residents.
In terms of respondent characteristics, out of 277 residents, 48% were female, 60.8% aged 26–30 years with no advanced degrees (85.1%), and a modest distribution of years since medical school (35.0% <1 year, 26.8% 1–3 years, 30.1% 4–10 years). Of the foreign medical graduates in the population, 38.6% completed their medical school training outside the U.S., 68.8% had previous coursework in biostatistics [69.5% of which were during medical school (15.9% college, 3.2% residency)]. Over 50% had previous training in epidemiology and EBM, and regularly read medical journals. Interestingly, the number of residents who could correctly identify and interpret statistical results was low. Approximately 25.6% could correctly identify chi-squared analysis, 13.0% could correctly identify Cox proportional hazard regression, 11.9% could interpret a 95% CI and statistical significance, and only 10.5% could interpret Kaplan-Meier analysis results. Using a forward stepwise regression model, advanced degrees, successive years since medical school, and prior biostatistics training were all factors found to be independently associated with knowledge scores. In terms of attitudes and confidence, 95% of residents agreed that knowledge of statistics is essential to being an intelligent reader of literature and 77% indicated they would like to learn more statistics. While over 58% of residents reported using statistics in forming opinions or making clinical decisions, 75% indicated they did not fully understand the statistics reported in literature. Only 38% of residents felt confident assessing the appropriateness of statistical testing used and respondents with a higher confidence level in statistical knowledge fared better on the knowledge questions.
While their report was confined to internal medicine residents, high internal consistency, good discriminative validity, and similarity in results among different residency programs lend credibility to the illustrated problem.[19
] The authors direct the poor knowledge and understanding of biostatistics to insufficient training. A comprehensive review of biostatistics teaching indicates that 90% of medical schools taught biostatistics in preclinical years only with varying breadth and depth of education.[20
] While basic statistics were frequently addressed, advanced methods were seldom included. Another pressing issue is that senior residents performed worse than junior residents, indicating a time correlation. Most likely, loss of knowledge over time, coupled with lack of adequate reinforcement could lead to loss of statistical competency. This lack of AGCME competency comes at a great cost. If clinicians cannot evaluate appropriate statistical tests and accurately interpret results, risks could be carried over to incorrect clinical decision-making.
West and colleagues performed a similar study in 2005 on 301 medical students, internal medicine residents and faculty, about their attitudes toward biostatistics in medicine.[21
] According to their findings, 48.3% of those surveyed felt biostatistics is a difficult subject, 87.3% felt that understanding biostatistics would help their careers, and 17.6% felt their training in biostatistics was adequate for their needs. Furthermore, 23.3% of respondents could evaluate appropriateness of statistical methods used in a study, 88% felt knowledge of statistics is necessary for evaluating medical literature, and 48.5% felt that biostatistics is a necessary skill for clinicians not involved in research. In essence, the survey strongly indicated that clinicians were uncomfortable with biostatistics and even more dissatisfied with this cognizance. It is unclear why physicians are queasy regarding statistics even though they use statistics in their daily routine.
Perhaps the finding that 20% of respondents felt their biostatistics coursework was taught effectively calls into question as to how clinicians are being educated about statistics in healthcare fields. Can understanding of statistics be improved to avoid erroneous interpretation and application? Traditional teaching methods in schools employ a stepwise approach entailing formulae, data, and spoon-fed instructions. This does not relate well to patients or analysing scientific papers. Medical statistics are often taught as abstract concepts removed from clinical relevance. Bordering on a moral quandary is the question of whether expectations for the average urologist are too high. Would the urologist who is not a researcher be better suited to appraise practice guidelines, derived by experts with the necessary statistical knowledge, rather than interpret statistics? Urology is a highly competitive field that is constantly evolving and as such, expectations will continue to be shattered and stacked higher. The current consensus will most likely rest on the urologist having a strong statistical repertoire because research is an increasingly integral component of residency and fellowship programs, because guidelines can change given new information, and because treatment accountability ultimately rests with the physician's ability to evaluate evidence and make decisions.
Most of the studies examining the use of statistics and knowledge of clinicians have thus far been centered in the U.S. In urology, only major journals have been examined leaving other international journals indexed in MEDLINE, such as Brazilian Journal of Urology and Indian Journal of Urology out of the loop. It is vital to assess how these journals and how urology practitioners in these regions fare in comparison to the current data through future investigations of this nature.