The diagnosis, monitoring, and management of patients with systemic autoimmune diseases remain challenging, prompting a continuing search for better biomarkers. In the progression from exploratory to validated status, biomarkers must undergo careful evaluation of sources of variation [5
]. We found that fewer than one half of translational studies of biomarkers included study-design features needed for valid interpretation of clinical associations.
Gene-expression profiles were the most common potential biomarkers. If the patient and control groups are not comparable in biologic factors that might influence gene expression, the association of a particular set of genes with disease may be mistaken, whereas an unrecognized set of genes may be the true disease-associated set. Two large-scale surveys of variation in gene expression in healthy subjects showed substantial differences within individuals over time and among individuals by age and sex [16
]. Immune-related genes, including those for immunoglobulins and genes that are regulated by interferons, were among the genes most often found to be differentially expressed by age and sex [16
]. This is particularly relevant because patterns of expression of specific type 1 interferon-regulated genes, so-called "interferon signatures," have been proposed as useful biomarkers in SLE and inflammatory myopathies [18
]. Moreover, if gene expression is differentially regulated by medications, misidentification of treatment-responsive genes as disease-related signatures may occur. Similar effects can occur for other categories of biomarkers.
Studies of biomarkers of disease activity have an additional limitation if they are examined only cross-sectionally. Cross-sectional studies compare expression of the biomarker between patients with active and inactive disease, without evidence that expression can change as disease activity changes. Longitudinal studies of patients who experience changes in activity provide a more valid design [5
]. Fewer than one half of studies included a longitudinal design.
Most studies showed the potential biomarker to be positively associated with the disease investigated, despite often-limited attention to confounding. Of greater concern is the possibility that other studies failed to detect associations because of inattention to these issues. The failure to consider matching, treatment effects, and other study-design features might have led investigators to conclude that no association was present, and therefore, to abandon potentially promising biomarkers. False-negative results may be common, but because negative studies may not be reported, the extent of this problem is difficult to assess.
Studies based on histologic specimens and studies of RA were less likely to address important study-design features than were studies based on serum or other sources and studies of SLE or other diseases, respectively. Although studies of histologic specimens were generally of smaller size, and most often were studies of the synovium in RA, the associations were independent of both sample size and disease. Attention to study-design features may be less prevalent among these studies because histologic samples are more difficult to obtain, and specimens are used when they become available. In studies of animal models, age, sex, and genetic background are considered important factors to be controlled. Human studies should similarly consider these factors. Larger studies, likely representing biomarkers at a more advanced stage of evaluation, more often reported important study-design features.
Our results reflect existing community standards for studies in systemic autoimmune diseases. The deficiencies may be due in part to incomplete reporting rather than to omissions in the study design. Adoption of uniform reporting criteria would be one remedy for this problem. To the extent that these deficiencies are due to omissions in study design, our results raise questions of whether attention to issues in the testing of clinical applications of the biomarker is often overshadowed by the novelty of the laboratory science.
The strengths of this study include examination of a diverse set of potential biomarkers in several diseases, many study-design features, and three clinical applications. However, our conclusions about biomarkers for prognosis are limited by the small number of studies. Inclusion of additional years or journals might have increased the number of studies, but our search provided a broad representation of recent studies. Some study-design features, such as adjustment for age, sex, and treatment effects, may be considered more important than others, but we examined a broader set of features without suggesting a hierarchy among them. The proportion of studies that included age, sex, or race adjustment was low, even though our criteria for matching were liberal. Although we examined only systemic autoimmune diseases, similar results may occur in other diseases such as osteoarthritis, spondyloarthritis, cancer, or cardiovascular disease [20