Several authors have proposed a multiphase model to evaluate medical tests, with an initial phase of laboratory testing and a final phase of randomised trials to compare outcome between groups of patients assessed with new tests or existing tests.12-15
An intermediate phase is multivariable modelling to measure whether a text provides more information than is already available to the doctor.16
We propose a model based on comparative accuracy, which compares new and existing testing pathways, and takes into account how the test is likely to be used.
A series of questions should be considered when a new test is evaluated:
- What is the existing diagnostic pathway for the identification of the target condition?
- How does the new test compare with the existing test, in accuracy and in other features?
- What is the proposed role of the new test in the existing pathway: replacement, triage, or add-on?
- Given the proposed role, what is the best measure of test performance, and how can that measure be obtained efficiently?
To determine whether a new test can serve as a replacement, triage instrument, or add-on test, we need more than a simple estimate of its sensitivity and specificity. The accuracy of the new testing strategy, as well as other relevant features, should be compared with that of the existing diagnostic pathway. We have to determine how accuracy is changed by the addition of the new test. These changes are dependent on the proposed role of the new test.
It may not always be easy to determine the existing pathway. In some cases, the prevailing diagnostic strategy may be found in practice guidelines. If a series of tests is in use, with no consensus on the optimal sequence, researchers must decide on the most appropriate comparator. This is similar to the problem of which comparator to use when intervention trials are designed against a background of substantial variation in practice.
As our understanding grows, or when circumstances change, the role of a test may change. The cost of positron emission tomography currently limits its use as an add-on test in most centres, whereas some centres have introduced this test or combined computed tomography and positron emission tomography at the beginning of the testing pathway.
Determining the likely role of a new test can also aid the critical appraisal of published study reports—for example, in judging whether the test has been evaluated in the right group of patients. Triage tests should be evaluated at the beginning of the diagnostic pathway, not in patients who tested negative with the existing tests. Purported add-on tests should be assessed after the existing diagnostic pathway. Finding out whether a test can serve its role is not exclusively based on its sensitivity and specificity, but on how the accuracy of the existing testing pathway is changed by the replacement, triage, or add-on test.
Studies of comparative accuracy evaluate how new tests compare with existing ones
New tests can have three main roles—replacement, triage, or add-on
Features of a new diagnostic test can help define its role
Knowing the likely role of new diagnostic tests can help in designing studies to evaluate the accuracy of tests and understand study results
In general, methods to evaluate tests have lagged behind techniques to evaluate other healthcare interventions, such as drugs. We hope that defining roles for new and existing tests, relative to existing diagnostic pathways, and using them to design and report research can contribute to evidence based health care.