We found subsequent calendar years were associated with lower reported mortality rates, increasing cancer reporting rates, increasing prescriptions per patient, and increasing encounters per patient. Our findings also demonstrated evidence of among-practice variability within a given calendar year for the same measures, and that some of our observed practice-level covariates may contribute to the variability among practices. In a sensitivity analysis, inclusion of practice-year observations from before the practice reached their AMR and practice-year observations with less than 2 years of Vision experience did not appreciably alter the results. The implication of these results is that, when using THIN data, investigators may need to consider secular trends and among-practice variability in pharmacoepidemiology analyses. While bias resulting from failure to account for among practice variability may often be small, the among practice variability can be accounted for through a variety of methods, such as matching or through the use of random effect models, as presented in this paper.
We observed that the variability among practices was reduced by inclusion of covariates describing the practice characteristics. Although not previously studied in THIN, other investigators have observed similar findings in similar populations. For example, using the GPRD, patient level morbidity as measured by the Johns Hopkins adjusted clinical group case mix system explained 57% of the variation in prescribing among practices,20
30% of the variation in referral practices,21
and 2% of the variation in home visits.22
Importantly, even adjustment at the patient level as done in these prior studies failed to fully account for among-practice variability. Variability in disease-specific outcomes has also been noted in a UK study of 78 participating practices in an audit-based educational program in primary care.23
We and other investigators have also observed secular trends for outcomes within primary care practices. The current findings for cancer rates are similar to those of our previous study.9
Our data on secular trends in mortality, prescribing, and encounter rates are all generally consistent with public health data from the United Kingdom indicating declining rates of mortality and increasing rates of prescribing and encounters over the same time period.17,24,25
As shown before for mortality10
our analysis showed an observed difference before and after 2004. Although not the specific purpose of this study, this finding supports the generalizability of THIN data to the broader United Kingdom population.
There are several potential limitations to this study. We did not require patients to have been enrolled for the full year, as this would have prevented our analysis of death (i.e., registration ends when the patient dies). One hypothesis for the observed among-practice variability is that practices with high turnover rates have fewer encounters per patient per year and fewer prescriptions per patient per year. To test this, we included a variable for the proportion of patients that were new to the practice each year. In univariate analysis, inclusion of this variable reduced the variance component for prescriptions per year by 4.1% and for encounters per year by 1.2%. However, adding this variable to the fully adjusted models had relatively little effect on the variance estimate (data not shown).
As one of our primary aims was to assess among-practice variability, the unadjusted analyses provide important insight to the magnitude of this variability. We used adjusted analyses to determine how much of the variability could be explained as a result of confounding. We selected a limited number of practice characteristics for inclusion in the adjusted models, although of course there could be other practice characteristics that were not captured. Nonetheless, after adjusting for covariates we observed a reduction in among-practice variability of approximately 40%, most of which was due to adjustment for mean age of the patients in the practice. Other practice-level factors that may influence rates of encounters and prescribing such as GP knowledge and experience and patient expectations are not easily captured in the electronic health records of THIN, and as such were not adjusted for.26–28
In addition, we did not examine patient level factors, such as measuring burden of disease by examining established comorbidity indices. Rather, our intent was to identify practice-level factors that are readily summarized across the practice-year observations. As such, it is important to interpret the clinical implications of these data with the appropriate cautions regarding the potential for residual confounding.
A challenge in pharmacoepidemiology research is to quantify the magnitude of among-practice variability in a manner that is easily interpretable. To facilitate this, we calculated the ratio of the 75th to 25th percentile observations. This metric demonstrated that there was approximately a 30% to 60% difference among the high and low rates for these measures which was generally stable across calendar periods for most of the outcomes we examined. Examining trends in among-practice variability based on the estimated random effect variance provides the same conclusion ().
We hypothesized that among-practice variation would decrease over time as practices became more accustomed to using Vision software and benefited from ongoing clinical quality improvement programs such as the implementation of Quality Outcomes Framework (QoF) (NHS Information Centre. Quality and outcomes framework www.qof.ic.nhs.uk
) measures. For example, practice recording patterns may become more uniform as the GPs familiarize themselves with the software management system, receive feedback reports, and respond to changes in the QoF. That we did not observe these trends could be in part attributable to our design. We limited our analysis to practices with two years of Vision software experience to ensure that all practices had ample time to become familiar with using the software. We also excluded practice-years prior to the practice specific AMR date. However, neither of these is likely to have contributed substantially to these results as including these practice-year observations in the sensitivity analysis did not appreciably change our results.
Likewise, some of the variation among practices could reflect variable data quality. We observed a limited number of practices having high or low rates of events under study, denoted as outliers on the box plots in . THIN provides research files for researchers to develop suitable practice level criteria to select only practices with the most complete data on selected variables of interest. Presumably, matching on practice minimizes some of the bias that could result from variable data quality, although there was little evidence of such bias, as we did not observe improved model fit with inclusion of practice as a fixed effect for any of our outcomes, which controls for such bias at the practice level.
Another potential limitation to our study is that coding schemes change over time.29
This could affect the analysis of changes over time. This would not be expected to affect the analysis of among-practice variability as these changes would affect all practices.
Although size is a major strength of studies in THIN and other large databases, it also is a potential limitation for interpreting test results. Because of the large size of the data, some of our observations could be statistically significant, while not being clinically significant. For example, we observe significant variation among practices, yet as noted above, adjustment for practice did not improve the model fit at least for the outcomes we examined.
To what extent these results extrapolate to other primary care databases, including those inside and outside of the United Kingdom, or to administrative claims data is unknown. Similarly, whether these data are generalizable to cohorts with specific diseases or exposures remains to be determined. Additional study of other primary care databases and other outcomes will be important to ensure consistency across practices and calendar time.
In conclusion, THIN data are characterized by significant secular trends and among-practice variation that should be considered in the design of pharmacoepidemiology studies. Empiric data documenting the impact of matching on practice and/or calendar year or implementing different statistical methods to account for this would be useful since there are often studies where matching results in considerable logistic challenges or reduction in sample size. Until such data are available, matching and/or adjusting for practice and calendar time remains a prudent approach in pharmacoepidemiology studies.