There are some limitations to the validity of the data that this study is based on, as a result of estimations which were necessary to allow for the comparison of different datasets. Also, patient characteristics such as age and gender might differ in the individual datasets. Consideration of these aspects in the structured analysis was not possible because of the lack of published information and due to the inhomogeneity of studies.
For the Australian National Joint Replacement Registry, dataset allocation of the reason for revision to a component of the implant was not possible and had to be estimated. It is unlikely, however, that this would have a substantial effect on the final conclusions when differences of 951% were observed. Comparative analysis of the datasets has yielded several observations that should lead to critical scrutiny.
Systematic analysis of the data published about the cementless Taperloc stem revealed a strikingly high influence of the developer's clinic: 44.53% of all cases and 39.2% of observed component years were published by this institution. However, with regard to revision rate this dataset differs considerably in outcome from the reference data in the Australian registry and developer-independent publications. Even in a conventional meta-analysis of the data, the high influence of the developer's institution has a statistically significant effect on the overall result, which has also been demonstrated in the meta-analysis by Mallory et al. (2002)
Several points should be considered in the interpretation of the data:
1. Influence of the developing hospital
Developers of implants have several particularities that might influence the outcome:
- As a rule, the hospital concerned can rely on a high degree of expertise and a fundamental understanding of the product and its handling
- High personal motivation can be assumed when it comes to the thorough investigation of potential outcome-relevant flaws in the entire course of therapy and drawing the consequences.
- The final result of a THA implantation depends on a variety of factors, such as the product, instrumentation, operating technique, patient selection, etc. Since every product is developed against a specific background and based on a specific set of experiences, the product might make particular allowance for the factors prevailing at that particular hospital
- Last but not least, there are the implant designer's and the manufacturer's own interests in the success of the product.
2. Differences in outcome and reproducibility of the results of others, i.e. the average surgeon
Drawing on the differences in revision rate among individual hospitals in Sweden for interpretation of the divergences shows that the best departments deviate from the national average by a factor of 2 to 3 (Annual Report of the Swedish Hip Register 2008, Annual Reports of the Swedish and Danish Knee Register 2009, Danish Hip Register 2009). Since one of the most important reasons for the success of the Scandinavian registries is the reduction of individual hospitals' variances in revision rates through the higher-than-average improvement of low-performing hospitals, one can assume that there is a higher variation between results in individual hospitals in the US. The extent to which the other hospitals that have published follow-up studies on the Taperloc stem differ from the developer's institute as regards expertise should be critically examined. In any case, the data available allow us to make the conclusion that the outcome data achieved by the developing hospital are not reproducible by the average user worldwide.
Thus, the high influence of the developing institution could have a relevant impact on the average results published about the product.
As a rule, the circumstances surrounding journal publication are favorable for publishing studies from the developing institution:
- It can be assumed that implant developers have a strong commitment and a high scientific interest and motivation regarding follow-up so that there is sufficient material for journal publications
- Developing hospitals are the first to have access to new implants and are thus the first to be able to publish results. These publications are of innovative quality and are therefore attractive for journals
- As a rule, development teams closely cooperate with implant manufacturers, thus having access to resources that, among other things, can be used in follow-up and publications
- Since studies are also used in the marketing of the respective implant, manufacturers and developers are much more interested in publication than is usually the case for projects based exclusively on academic motivation, or for average users.
The majority of journal publications report a lower probability of revision than registry data. In this respect, a fundamental difference in the basic data must be taken into account: clinical studies are based on samples, which implies that any transference of results involves potential confounders. For example, the vast majority of publications originate from specialized centers, which are not representative of the worldwide or national average in all aspects. As opposed to this, registry data contain almost all surgeries performed in a country and therefore comprise the entire range of treatment, thus reducing bias factors and allowing better generalization in the area covered by the registry. The transference of results to other countries naturally creates confounders due to the local background, but its influence is considerably lower than in meta-analyses of clinical studies.
For the cementless Taperloc stem, there are no sufficient data currently available from national registries other than the Australian one. For other implants, however, such comparisons have shown that deviation from the average outcome value of the respective implant by a factor of 3 is not reached in any case.
Regarding the deviations in revision rates between individual hospitals within a certain country, without specifying factors such as implant selection, surgical techniques, the profile of the department, or other confounders, and between countries regarding the outcome achieved with the same implant, deviations by a factor of 3 appear to be explicable.
Thus, publications by authors who are not directly involved in the process of development of an implant on average show results that are reproducible in the registry, even though the average revision rates are below those of the Australian registry. The average revision rates published by the developer's institution, however, do not appear to be reproducible by other users. On average, they are approximately 9.5 times higher in the Australian register, and about 5 times higher with other users in the USA. The specific circumstances of a developing hospital therefore seem to affect the results.
3. Basic characteristics of the datasets that are being compared
In any evaluation, special consideration has to be given to the particular features inherent to the various datasets involved. As mentioned before, sample-based studies are usually conducted in specialized centers and subject to several influencing factors, such as the study design, patient selection, etc. However, the different factors influencing the results of sample-based studies are frequently not or only insufficiently apparent from the publications while they may have a relevant impact on the study results and lead to a remarkable variance in outcomes. This reduces the reliability of the data for conclusions, particularly when individual centers are overrepresented.
Registry data can be a valuable supplement for evaluation of the revision rates to be expected by the scientific community. Due to their higher degree of standardization and completeness in case recording at the national level, their validity is on average superior to that of sample-based clinical studies. Optimum reference values, however, can only be obtained within the coverage area of a register since the circumstances under which the data have been collected are always reflected in the results. Setting up of a registry should therefore be aspired to in every country.
It would generally be desirable if a large number of hospitals and surgeons were to undertake structured patient follow-ups and publish the results. This could at least reduce confounders due to individual circumstances in different hospitals.