Summary

Litière et al. (2007) presented the results of simulation studies that they claimed showed that misspecification of the shape of the random effects distribution can produce marked increases in Type II error (decreases in power) of tests based on fits of generalized linear mixed models. However, the paper contains a logical fallacy that invalidates this claim. We present logically correct simulation studies that demonstrate little increase in Type II error, consistent with the earlier work that shows little effect due to misspecification.

Background

Fibrosis stages from liver biopsies reflect liver damage from hepatitis C infection, but analysis is challenging due to their ordered but non-numeric nature, infrequent measurement, misclassification, and unknown infection times.

Methods

We used a non-Markov multistate model, accounting for misclassification, with multiple imputation of unknown infection times, applied to 1062 participants of whom 159 had multiple biopsies. Odds ratios (OR) quantified the estimated effects of covariates on progression risk at any given time.

Results

Models estimated that progression risk decreased the more time participants had already spent in the current stage, African American race was protective (OR 0.75, 95% confidence interval 0.60 to 0.95, p = 0.018), and older current age increased risk (OR 1.33 per decade, 95% confidence interval 1.15 to 1.54, p = 0.0002). When controlled for current age, older age at infection did not appear to increase risk (OR 0.92 per decade, 95% confidence interval 0.47 to 1.79, p = 0.80). There was a suggestion that co-infection with human immunodeficiency virus increased risk of progression in the era of highly active antiretroviral treatment beginning in 1996 (OR 2.1, 95% confidence interval 0.97 to 4.4, p = 0.059). Other examined risk factors may influence progression risk, but evidence for or against this was weak due to wide confidence intervals. The main results were essentially unchanged using different assumed misclassification rates or imputation of age of infection.

Discussion

The analysis avoided problems inherent in simpler methods, supported the previously suspected protective effect of African American race, and suggested that current age rather than age of infection increases risk. Decreasing risk of progression with longer time already spent in a stage was also previously found for post-transplant progression. This could reflect varying disease activity, with recent progression indicating active disease and high risk, while longer time already spent in a stage indicates quiescent disease and low risk.

Multistate modeling methods are well-suited for analysis of some chronic diseases that move through distinct stages. The memoryless or Markov assumptions typically made, however, may be suspect for some diseases, such as hepatitis C, where there is interest in whether prognosis depends on history. This paper describes methods for multistate modeling where transition risk can depend on any property of past progression history, including time spent in the current stage and the time taken to reach the current stage. Analysis of 901 measurements of fibrosis in 401 patients following liver transplantation found decreasing risk of progression as time in the current stage increased, even when controlled for several fixed covariates. Longer time to reach the current stage did not appear associated with lower progression risk. Analysis of simulation scenarios based on the transplant study showed that greater misclassification of fibrosis produced more technical difficulties in fitting the models and poorer estimation of covariate effects than did less misclassification or error-free fibrosis measurement. The higher risk of progression when less time has been spent in the current stage could be due to varying disease activity over time, with recent progression indicating an “active” period and consequent higher risk of further progression.

For both clinical and research purposes, biopsies are used to classify liver damage known as fibrosis on an ordinal multi-state scale ranging from no damage to cirrhosis. Misclassification can arise from reading error (misreading of a specimen) or sampling error (the specimen does not accurately represent the liver). Studies of biopsy accuracy have not attempted to synthesize these two sources of error or to estimate actual misclassification rates from either source. Using data from two studies of reading error and two of sampling error, we find surprisingly large possible misclassification rates, including a greater than 50% chance of misclassification for one intermediate stage of fibrosis. We find that some readers tend to misclassify consistently low or consistently high, and some specimens tend to be misclassified low while others tend to be misclassified high. Non-invasive measures of liver fibrosis have generally been evaluated by comparison to simultaneous biopsy results, but biopsy appears to be too unreliable to be considered a gold standard. Non-invasive measures may therefore be more useful than such comparisons suggest. Both stochastic uncertainty and uncertainty about our model assumptions appear to be substantial. Improved studies of biopsy accuracy would include large numbers of both readers and specimens, greater effort to reduce or eliminate reading error in studies of sampling error, and careful estimation of misclassification rates rather than less useful quantities such as kappa statistics.

For both clinical and research purposes, biopsies are used to classify liver damage known as fibrosis on an ordinal multi-state scale ranging from no damage to cirrhosis. Misclassification can arise from reading error (misreading of a specimen) or sampling error (the specimen does not accurately represent the liver). Studies of biopsy accuracy have not attempted to synthesize these two sources of error or to estimate actual misclassification rates from either source. Using data from two studies of reading error and two of sampling error, we find surprisingly large possible misclassification rates, including a greater than 50% chance of misclassification for one intermediate stage of fibrosis. We find that some readers tend to misclassify consistently low or consistently high, and some specimens tend to be misclassified low while others tend to be misclassified high. Non-invasive measures of liver fibrosis have generally been evaluated by comparison to simultaneous biopsy results, but biopsy appears to be too unreliable to be considered a gold standard. Non-invasive measures may therefore be more useful than such comparisons suggest. Both stochastic uncertainty and uncertainty about our model assumptions appear to be substantial. Improved studies of biopsy accuracy would include large numbers of both readers and specimens, greater effort to reduce or eliminate reading error in studies of sampling error, and careful estimation of misclassification rates rather than less useful quantities such as kappa statistics.