Based on our results, the device is not yet ready for clinical use.. The rapid communication strategy was efficient in that after feedback from our research staff regarding device performance, the company did develop new prototypes that showed a much increased sensitivity (80% with both vaginal and cervical swabs), and prevented staff injury by using a “crusher” device to break the ampoules. Although device sensitivity appeared to improve using this tactic (from 38% to 80%), due to small sample sizes this increase was not statistically significant. In addition, the increased sensitivity was accompanied by a high percentage of indeterminate results and false positives that lowered specificity from 60-77% to 25-37%, which was statistically significant at p=.05. Therefore, while safety issues improved, overall device performance is not ready for larger clinical trials.
If the problems with accuracy are resolved, this device has good potential as a point-of-care test in the future. The device would be fairly easy to use in a home or non-clinical office setting, as it is self-contained and the color change for some specimens was dramatic and immediate (within minutes). This would be a clear, visual way to determine if Chlamydia infection is present at the point-of-care, a strategy that would greatly improve STI care. The goal then for the company would be to try to find a way to generate higher sensitivity without sacrificing specificity. In fact, a highly specific test with moderate sensitivity (75-80%) would be good enough to dramatically reduce the possibility that a patient would be lost to follow-up after testing.
The device test performance is not much different than what is reported for other POC tests for Chlamydia. Currently, three tests have been cleared by the United States FDA: the optical immunoassay (OIA) (Inverness [formerly Biostar], Princeton, NJ), Clearview Chlamydia (Inverness, Princeton, NJ), and QuickVue (Quidel, San Diego, CA). These three POC tests for Chlamydia have reported low sensitivities (25-65%) compared to NAAT, but good specificities (97-100%) and most testing was done using endocervical swabs [6
]. The only study that tested vaginal swabs with Clearview® found a similarly disappointing sensitivity (33%) and high specificity (100%) [8
]. The POC test available in Europe (Chlamydia Rapid Test, Diagnostics for the Real World, LTD, Cambridge, UK ) is much more accurate (sensitivity 83.5 % and specificity of 99% using vaginal swabs, compared to NAAT). However, there is no indication that the company intends to seek FDA clearance.
One limitation to our study was our small sample size. Part of the reason for this was that when we encountered problems with a prototype, we stopped recruitment until we communicated with the manufacturer and they sent us a new prototype which resolved the problem. In the case of devices A and B, this was mainly technical issues (could not determine color change, ampoules breaking, etc.). Although these were fixed with the next two prototypes, C and D, the tests were continuing to generate poor specificities, with a high number of false positive results. As these small samples demonstrated rough estimates of test performance, we believed that it was a waste of time and resources to study a larger number of women with poorly performing prototypes. However, our small sample size (n=83) results in very imprecise estimates of sensitivity and specificity as the small sample size leads to an overlapping of confidence intervals. Thus, further testing on a larger sample of adolescents is needed to determine more reliable estimates of the properties of these tests.
Another limitation is that we did not use all four prototypes on the same patient sample, which could have affected device results. For example, it is possible that certain prototypes were able to detect positive infections only if the patient had a high bacterial load present. Nor did we collect data on co-infections or symptoms which could have affected results. Future research on this POC test should examine the affect of such compounding factors on device performance.
POC testing for STIs, especially for an infection as highly prevalent as Chlamydia, enables patients to receive immediate results, care, counseling, and can help prevent reinfection. This method of testing is particularly beneficial to adolescent women, who often are disproportionally affected, tend not to come in for follow-up results, and may often even initially resist clinical testing with a pelvic examination. There is an urgent need for continued development of new devices while refining the feasibility and accuracy of existing devices. Ultimately, we wish to create easy to use, sensitive POC tests that can provide both quick results in a clinical setting and eventually allow individuals to test themselves in non-clinical settings as well.
We showed that an academic-industry collaboration with frequent communication is an important step between bench science and large clinical trials. Although preliminary results from our industry partner looked quite promising, by using the device in a clinical setting in real time, we uncovered three problems that must be resolved before large scale manufacture and trials are performed. In order to move the POC STI test device field forward, other researchers and companies may want to consider this strategy to improve the chances of developing an accurate and user-friendly device.