Many human brain imaging studies, particularly those involving normal subjects, rely on self-reporting as a means of identifying “normal” individuals. To determine how much attention to detail is provided in the evaluation of normal subjects or controls, a literature review was undertaken. Two years of published manuscripts (2005 and 2006) from NeuroImage were reviewed to identify those that included normal subjects either as the primary participants or as controls for a reference group. A total of 474 manuscripts were reviewed. While many manuscripts were vague about these procedures, it was clear that the vast majority (~75%) either relied on self-reporting or did not mention the methods for screening these subjects at all (). 24.1% performed in-person histories and approximately 7.4% included neurological and general medical examinations of their subjects. Thus, detailed screening of subjects for experiments requiring normal controls was limited to self-reporting or not considered worthy of attention to describe methodologically. This underscores the theme of this paper, namely, that the proper selection of normal subjects and control groups is a vital part of the experimental design in neuroimaging studies and that attention to detail and validity of results must be ensured by in-person evaluation in terms of history and physical examination.
Figure 4 Results of a literature review of manuscripts from 2005 and 2006 in NeuroImage to determine what methods of screening each published study utilized to select normal subjects. As shown, 74.4% of studies relied on self-reporting or did not mention how the (more ...)
In the ICBM project, we used an extensive set of exclusion criteria and a much more carefully defined definition of the “normal” state. Such definitions are always arbitrary and typically devised through practicality and convenience, usually by consensus of the participating investigators. That was the case for this project as well. Nevertheless, an extensive list of exclusion criteria were agreed upon to better ensure that the resulting data set would include individuals who did not have previous or current medical factors that might affect brain structure or function.
Even in this more tightly defined situation, subjects who had disorders that we wanted to omit could still have been included. One example is occult hyperlipidemia. We eliminated individuals who were on cholesterol lowering agents but did not perform serum lipid profiles to identify undiagnosed subjects. The same is true for diabetes mellitus. In addition, we relied on self-reporting with regard to alcohol consumption and illicit drug use. While some of these individuals could be identified by the physical stigmata associated with long-term use of these agents (e.g., needle track marks, ascites, etc.), such physical findings would only be evident in advanced cases. It was also unlikely that we could detect undiagnosed psychiatric disease unless the patient was manifesting overt signs of the disorder during our contact with the subject. Despite these shortcomings, the battery of screening procedures employed ensures a much higher degree of valid criteria for including “normal” subjects than self-reporting would enable.
The screening process was designed to eliminate individuals with known medical, neurological, neurosurgical or psychiatric disease. It also eliminates individuals who chronically utilize prescription, over-the-counter or illicit drugs with the exception of drugs used as prophylaxis for infectious disease in travelers, altitude or motion sickness, medications for needle stick injuries or vaccinations. We also accepted subjects who were on antibiotics more than a month prior to evaluation and, under certain circumstances, non-steroidal anti-inflammatory drugs, pain medications or sedatives (). We did accept the use of contraceptive agents and hormone replacement therapy as well as multi-vitamins, aspirin and acetaminophen were acceptable if used occasionally.
Subjects were eliminated for any procedure requiring general anesthesia in the previous year, with certain limited exceptions, and any neurosurgical, cerebrovascular, oncological or cardiac surgery irregardless of the anesthesia used. These criteria plus thorough general medical and neurological examinations would eliminate almost all subjects with underlying medical, neurological or neurosurgical disorders. Potential subjects with defined headache syndromes were also eliminated because of their association with structural or functional brain abnormalities [Kruit et al., 2004
] despite the fact that the clinical significance of such changes on MRI is a matter of ongoing debate. By historical criteria we eliminated subjects with head trauma or injuries to the spinal cord and peripheral nerves as well as those with implanted metal or electronic devices that would obviate scanning. The careful measurement of blood pressure, pulse, oximetry as well as visual and auditory acuity eliminated other risk factors. This is particularly true for high blood pressure and the resultant potential for cerebral ischemic changes [Vermeer et al., 2003a
; Jeerakathil et al., 2004
Findings on neurological exam that typically excluded subjects age 60 or greater included mild parkinsonian features, tremor, peripheral neuropathy or evidence of old central nervous system injury possibly related to previous but undiagnosed cerebral infarction. Even minimal signs on neurological exam are associated with structural brain changes [Dazzan et al., 2006
]. Physical examination also provided the opportunity to eliminate subjects that had stigmata of alcohol or illicit drug use, cardiac murmurs, carotid bruits or scars from previous surgery not reported during the in-person interview.
While subjects may pass through historical and physical evaluations without detecting evidence of medical or neurological disorders, it is well accepted that this does not ensure that abnormalities will not be seen on structural brain imaging [Grossman and Bernat, 2004
; Illes et al., 2006
; Vernooij et al., 2007
; Weber and Knopf, 2006
]. In previous reports, it has been demonstrated that subjects with otherwise normal screening studies can still have arachnoid cysts, vascular abnormalities, intracranial tumors and cerebral aneurysms identified on sturctural MRI studies. When any of our subjects were excluded because of undiagnosed disorders, they were told about the historical or physical findings associated with this conclusion and recommendations were made to them so that they could seek appropriate clinical follow-up.
As is clear from the data presented here, the most unrecognized abnormality that resulted in subject exclusion in this study was high blood pressure. This was true for both men and women, although it was more common in men. While the peak rejection rate occurred for individuals in their 60's, high blood pressure was found in subjects across the entire age span and most prominently between ages 40 and 80. Since high blood pressure usually is associated with an insidious onset and a sub-clinical course until it results in significant cardiac, cerebrovascular or other events, this result is not surprising. High blood pressure is a well-known cause of cerebrovascular disease and is associated with white matter abnormalities and “silent” infarcts on MR imaging, particularly with T2 weighted or FLAIR pulse sequences [Jeerakathil et al., 2004
; Vermeer et al., 2003a
]. Eliminating subjects with high blood pressure was, thus, an important goal of our screening procedures and proved to be effective in identifying those subjects who were unaware of their problem. This served a secondary value in that these individuals were identified and made aware of the situation so that they could seek appropriate clinical evaluation and treatment.
It was surprising to us that only slightly over 31% of individuals successfully passed a detailed telephone screen, considering the original solicitation for their participation clearly stated that subjects were required to be healthy. Even more striking was the fact that nearly half of the subjects who passed a detailed telephone interview failed the in-person history and physical examinations. The reasons for this situation are noteworthy (). In some instances, subjects were unaware of their medical problems. This is clearly true for subjects we evaluated who had undiagnosed high blood pressure or mild cognitive impairment, since such subjects will only be identified if they are formally evaluated for these problems. Subjects can be confused by terminology or jargon used in solicitation advertisements or even in initial screening questions. We found this to be the case with the term “inherited disorder.” Subjects often required considerable explanation of these terms in order to clearly understand their meaning in the context of the study. As such, it is important to use the most basic lay terminology in describing criteria reflecting a subject's personal medical history, family history or past diagnoses.
Possible reasons subjects report that they are normal but are subsequently excluded by history and physical examination criteria.
Some subjects appeared to intentionally attempt to deceive the investigators. We encountered two general categories in this group. The first is the “professional subject” who obtains a significant income by participating in medical research. Motivated by financial reasons, these individuals will provide whatever answers to questions they feel will allow them to be participants in the study [Elliot and Abadie, 2008
]. An alternate reason for direct deception by subjects is to avoid embarrassment. Often subjects are recruited for imaging studies by colleagues (e.g., fellow graduate students or faculty members) and do not want to admit to their friends or co-workers that they have an underlying neurological or psychiatric disorder, take particular medications or are in need of financial remuneration. Most institutional review boards try to avoid utilization of subjects in studies who have contact with investigators in other settings so as to avoid coercion of the subject into participation. In the situation revealed in this project, an additional factor for excluding potential subjects who are known to members of the investigational team, is to avoid putting the potential subject in the awkward circumstance of revealing to someone, whom they know in another context, that they have an underlying medical problem or financial hardships.
A number of subjects reported that they had a past medical problem but did not consider it relevant or important. This was most often true if the symptoms were mild, such as individuals with infrequent migraine headaches. Subjects also vary in their personal criteria for the definition of excessive alcohol use which tended to exceed the limits employed in medical research studies.
Subjects may be unaware or minimize the significance of mild prior cardiac disease, particularly heart murmurs. They may dismiss important disorders if they are far distant in time, for example, past head trauma with loss of consciousness, cardiac disorders, or cancer that was diagnosed, treated and “cured.” However, even congenital and corrected cardiac disorders can alter brain structure and function [Miller et al., 2007
]. Spinal cord or cardiac surgery, particularly if performed during childhood were viewed by potential subjects as closed issues, no longer relevant to their current health status. All of these factors combine to lead subjects to self-report their normality when they are, in fact, not normal.
We encountered a number of individuals in the current study who had previously qualified for studies as normal subjects based on their self-report but who, upon thorough in-person history and examinations were rejected from this study because of underlying medical disorders or medication use.
Normal subjects must be chosen to fit the experimental situation. In our case, the exclusion of as many potential confounding factors as possible was the goal. In other situations, e.g., comparison with a disease group, representative normals may be more valid controls. No single definition fits all experimental requirements. Regardless of the selection criteria, the responsibility exists to verify the medical status of such subjects with objective testing. Without such an approach, the result will be an invalid description of the brain imaging phenotype. Erroneous phenotypes would then propagate incorrect conclusions about relationships between phenotype and behavior or phenotype and genotype.
As a result of this experience, not only at the UCLA site, but throughout the ICBM consortium, we have concluded that it is vital, in any brain imaging study, to have well thought out and carefully defined criteria for subject inclusion and exclusion, a face to face medical interview by a physician and careful measurements of vital signs as well as the medical and neurological examinations of the subjects. The elimination of almost nine out of ten subjects who considered themselves normal in this cohort provides some measure of the magnitude of the problem associated with self-reported normality. Individuals in society have a very vague definition of what is normal and what is not, from a medical research perspective. To accept their vague definition will contaminate scientific data collections, analysis and the conclusions derived from such data sets.