PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jcmPermissionsJournals.ASM.orgJournalJCM ArticleJournal InfoAuthorsReviewers
 
J Clin Microbiol. 2010 March; 48(3): 753–757.
Published online 2009 December 30. doi:  10.1128/JCM.01460-09
PMCID: PMC2832432

Italian National Survey of Blood Donors: External Quality Assessment (EQA) of Syphilis Testing[down-pointing small open triangle]

Abstract

The detection of syphilis among blood donors may reveal high-risk sexual behavior, which can go unreported at the time of donor selection and compromise the safety of the donated blood. In Italy, blood is collected, tested, and distributed by transfusion services (TSs), which also perform outpatient transfusions. Although the TSs must screen for syphilis by law, there are no indications of the specific type of method to be used, generating discrepancies in the results obtained by the different TSs. To determine the proficiency of the TSs in screening for syphilis, we performed an external quality assessment (EQA). The EQA was based on two shipments of serum panels; 133 and 118 of the 326 existing TSs participated in the first and second shipments, respectively. Each panel consisted of both positive and negative serum samples. The results confirmed that the use of a single nontreponemal test (the Venereal Disease Research Laboratory [VDRL] and the rapid plasma reagin [RPR] tests) is the least sensitive means of identifying samples that are positive for syphilis antibodies. We also found that the interpretation of the results of manual techniques, such as the RPR test, the VDRL test, the Treponema pallidum hemagglutination (TPHA) assay, and the T. pallidum particle agglutination (TPPA) assay, can vary greatly among different TSs and operators. Total Ig enzyme immunoassays (EIAs) are the most sensitive. However, the determination of syphilis on the basis of the results of a single test is not sufficient for an accurate screening; and all blood units should thus be assessed by two distinct treponemal tests, that is, a total Ig EIA and the TPHA or the TPPA assay.

Syphilis is a reemerging disease and is caused by the spirochete Treponema pallidum. In most cases it is sexually transmitted, although it can also be transmitted from mother to child in utero and, rarely, through blood transfusion, especially through the transfusion of fresh blood components (6, 12). Serological tests for syphilis are considered to be a milestone in syphilis control, and since as early as the 1930s they have greatly contributed to the detection of T. pallidum infection not only in the clinical setting but also in transfusional medicine. The detection of blood donors who are positive for syphilis is an important public health concern, given that testing may reveal high-risk sexual behavior, which can go unreported at the time of donor selection and compromise the safety of blood used for transfusions.

In some European countries, the occurrence of T. pallidum infection in the general population has been increasing, and this increase is reflected in its growing occurrence among blood donors. In particular, a survey performed in England has indicated that since 2001 there has been a trend toward a moderate increase in the incidence of T. pallidum infection among blood donors (2). In Germany, although the incidence of infection among donors is very low, increases have been recorded since 1991 (9). The incidence of T. pallidum infection among blood donors is also increasing in Italy. In particular, according to the Transfusion Transmitted Infections Surveillance System (7), it increased from 3.8 per 100,000 donations in 1999 to 7 per 100,000 donations in 2006 (11).

The data collected by the Transfusion Transmitted Infections Surveillance System in Italy are provided by the existing 326 transfusion services (TSs). These are hospital-based facilities where blood is collected, tested, and distributed and where, in most cases, outpatient transfusions are performed (10). In 2006, there were 1,539,454 donors, for a total of 2,402,267 donated units. Remarkably, 85% of the donors provided multiple donations (4). Although the TSs are required by law to screen for syphilis, there are no indications of the specific type of method that must be used, nor is there any confirmatory algorithm for testing on the basis of the different assays available. In fact, the laboratory assessment of syphilis is generally based on the detection of antibodies against T. pallidum antigens in blood by the use of either specific or nonspecific reagents. Methods based on the detection of specific Treponema antigens include passive agglutination, such as the T. pallidum hemagglutination (TPHA) assay or the T. pallidum particle agglutination (TPPA) assay, and indirect immunofluorescence, such as the fluorescent treponemal antibody absorbed (FTA-ABS) assay or the most sensitive assay, the enzyme immunoassay (EIA), for the detection of specific IgG and IgM or total Ig. Additional methods are based on nonspecific reagents, including nontreponemal lipid antigens (cardiolipin), and they most commonly rely on the flocculation technique. Of these, the Venereal Disease Research Laboratory (VDRL) and rapid plasma reagin (RPR) tests are the most commonly used.

The use of different assays from such a large array could generate discrepancies in the detection of syphilis among TSs, stressing the need for quality assessment. To this end, we performed an external quality assessment (EQA) of the quality and the comparability of the results obtained by the different TSs with the aim of contributing to the development of preventive and corrective measures (1, 3).

MATERIALS AND METHODS

The EQA was conducted to evaluate the performance of screening of donated blood units for syphilis by the TSs in Italy by comparing the results obtained in different laboratories. Before the implementation of the EQA program, a survey of the 326 TSs was performed to determine the methods and procedures used to screen for syphilis. Only the TSs that responded to the survey were included in the EQA. At this point, we had already realized that a good number of TSs used either the RPR or the VDRL test exclusively. To demonstrate to these TSs that their protocols needed to be improved because these methods do not detect all persons with present or past syphilis, both shipments included some positive samples that, when they were tested by the reference laboratories by the use of the RPR or the VDRL test, showed inconsistent results (two samples in the first shipment and one sample in the second shipment).

The EQA was based on two distinct shipments, which were made 3 months apart. The first shipment involved all of the TSs that participated in the survey mentioned above. The second involved only the TSs that had provided all of the information requested for the survey and had sent back the results of the proficiency testing for the first shipment. Participation was voluntary, and confidentiality was ensured by assigning an identification code to each participating TS, which was used for all communications.

The reference laboratories were those of the Section of Transfusion Methodology of the Istituto Superiore di Sanità (ISS; Italy's national institute of public health) and the Clinical Pathology and Microbiology Department of the San Gallicano Dermatological Institute.

Samples.

Two panels of serum samples were sent separately to the TSs. Each panel consisted of four serum samples obtained from donors, after having obtained informed consent. The first panel consisted of three positive samples and one negative control sample. The second panel consisted of two positive and two negative samples. The TSs were blinded to the serological status of the specimens. These samples (purchased from BIO-Development s.r.l., Parabiago, Milan, Italy) were sent to the TSs by courier and either were maintained at 4°C for testing within the next 24 h or were stored at −20°C until use.

Criteria for evaluating the results.

Before shipment, the reference laboratories performed both qualitative and quantitative confirmatory tests with each of the samples. In particular, all samples were tested for antibodies against T. pallidum by the RPR test (RPR card test; Radim S.p.a.) and the TPHA assay (Mycrosyph TPHA 200; Axis-Shield Diagnostics Limited), for IgG and IgM antibodies against T. pallidum (Enzywell Treponema IgG and Enzywell Treponema IgM; Diesse Diagnostica Senese), and by an IgG/IgM enzyme-linked immunosorbent assay (Enzywell Syphilis Screen recombinant; Diesse Diagnostica Senese). The EIAs were performed with the automatic Triturus instrument (Grifols, Spain). On the basis of the results of these assays, the samples were confirmed to be positive or negative (Table (Table11).

TABLE 1.
Serological characterization of the samples

The samples were then sent to the TSs, along with detailed instructions for standard handling and for submission of the results. The TSs were advised to use the standard procedures and methods that they commonly adopt in their routine practice and were asked to provide the data by the use the Qu@rk software (BIO-Development), which was available online.

Once we received the results of the tests performed for each shipment, a report was sent to all participating TSs. This report included the results obtained by the reference laboratories and an individual evaluation of each TS's performance; the first report was sent before the second shipment. Qu@rk software was used for data collection and analysis.

RESULTS

Participating TSs.

Of the 326 TSs in Italy, 133 (40.8%) participated in the evaluation of the samples in the first shipment, although the percentage of participating TSs differed significantly by geographic area: 52.4% in northern Italy, 24.3% in central Italy, and 46.4% in southern Italy and the islands.

Of the 133 TSs, only 118 participated in the evaluation of the samples in the second shipment.

Methods/kits used.

With regard to the different methods used to detect anti-T. pallidum antibodies, 84.9% of the participating TSs used a single screening method; 9.6% used two methods (total Ig EIA and TPHA-TPPA assay or RPR-VDRL test or FTA-ABS test); 3.6% of the TSs used three methods, and 2.0% used more than three methods (Table (Table2).2). The most commonly used assays were the total Ig EIA and the TPHA assay. When the tests used to evaluate the samples in first shipment were compared to the tests used to evaluate the samples in the second shipment, there was a 23.0% increase in the use of total Ig EIA and a simultaneous decrease in the use of agglutination tests (−26.1%) and nontreponomal tests (−17.7%). When the types of tests used were compared by the geographic area of the TSs, no significant differences were found.

TABLE 2.
Methods used by TSs for syphilis screening

Accuracy of classification. (i) First shipment.

The positive serum samples with the lowest antibody titers (samples 03103C and 03104D) were correctly identified by 110 (82.7%) and 118 (88.7%) TSs, respectively, whereas the sample with the highest antibody titer (sample 03101A) was correctly identified by 132 TSs (99.2%). The same percentage of TSs correctly identified the negative serum sample (sample 03102B).

(a) Positive sample 03101A.

Of the 133 TSs participating in the proficiency testing for the first shipment, 132 correctly identified serum sample 03101A as positive (reference laboratory characterization as total Ig EIA positive [EIA+], IgG EIA+, TPHA assay positive [TPHA assay+], RPR test positive [RPR test+, IgM EIA negative [EIA]). The TS that incorrectly identified this sample had based the test on the TPHA assay. Of the 132 TSs that had correctly identified the sample, only 2 used more than one type of assay. In one of these cases, the test for the detection of specific IgM produced a negative result, whereas the other tests (the RPR, TPHA, total Ig, specific IgG, and FTA-ABS assays) produced positive results. For the other TS that used more than one test for screening, it was not possible to evaluate the result produced by the VDRL test, and the correct data were obtained by the use of additional tests (the TPHA and FTA-ABS tests).

(b) Negative sample 03102B.

Sample 03102B was correctly classified as negative by 132/133 TSs (99.25%). The one TS that incorrectly classified this serum sample had based the analysis on the TPHA method.

(c) Positive sample 03103C.

Only 110/133 (82.7%) TSs correctly classified sample 03103C (reference laboratory characterization as total Ig EIA+, IgG EIA+, TPHA assay+, RPR test negative [RPR test], IgM EIA) as positive. Of the 23 TSs that provided incorrect results, 14 (10.5%) classified the sample as negative. Of those 23 TSs, 11 used a single test based on either the RPR or the VDRL test. Two used either the TPHA or the TPPA assay, and one used both the RPR test (negative result) and the TPHA assay (borderline result) and provided a final result of negative. Seven (5.3%) of the 23 TSs classified the sample as borderline. Six of those TSs used a single method (four used the TPHA or the TPPA assay, one used the FTA-ABS assay, and one used the total Ig assay), whereas only one of them used two different assays, which showed discordant results (negative by the RPR test and weakly positive by the TPHA assay). The remaining two TSs (1.5%) were not capable of providing a valid result. One of them used only the TPHA assay (which gave a weakly positive result), whereas the other one used as many as four different tests, but these gave discordant results (FTA-ABS assay negative, VDRL test positive, TPPA assay negative, and total Ig assay positive). Of note, although 11 TSs correctly classified the result for the sample, they erroneously interpreted as positive the results of the RPR test and/or the VDRL test, which, in the reference laboratories, produced negative results for this low-titer sample. Of these 11 laboratories, 6 used only one of these tests, whereas the remaining 5 used both tests.

(d) Positive sample 03104D.

For serum sample 03104D (reference laboratory characterization as total Ig EIA+, IgG EIA+, TPHA assay+, IgM EIA, RPR test borderline), 118 (88.7%) TSs provided a correct classification. Of the 15 TSs that obtained incorrect results, 11 identified the sample as negative (9 of them were based on the RPR or the VDRL test, 1 on the TPHA assay, and 1 on the total Ig assay); 4 of them classified it as borderline (1 performed the TPHA assay, 1 performed the RPR test, and the remaining 2 performed tests that were based on the RPR or the VDRL test [with negative results] and the TPPA or the TPHA assay [with positive results]). Also with this sample, although six TSs provided a correct result on the basis of the findings of more than one assay, they erroneously interpreted the results of the RPR or the VDRL test.

(ii) Second shipment.

For the second shipment, 118 of the 133 TSs sent back the results. The negative serum samples (samples 03105A and 03106B) were correctly classified by 115 (97.5%) and 109 (92.4%) TSs, respectively. The positive serum samples (samples 03107C and 03108D) were correctly classified by 109 (92.4%) and 116 (98.3%) TSs, respectively.

(a) Negative sample 03105A.

Although the reference laboratories obtained a borderline (indeterminate) result when they used TPHA assay to test negative serum sample 03105A, since the serum cross-reacted with the kit's control erythrocytes, thus invalidating the test, none of the participating TSs obtained a borderline result. In fact, of the 118 TSs participating in evaluation of the samples in the second shipment, 115 (97.5%) correctly classified the result for this sample. Of the three TSs that failed to correctly classify this sample, two obtained a positive result (one by using total Ig EIA and one by using the RPR test). The remaining TS obtained a borderline result (a negative result by the TPHA and FTA-ABS assays and a positive result by the RPR test).

(b) Negative sample 03106B.

Negative sample 03106B was correctly classified by 109 (92.4%) of the 118 TSs. The remaining nine TSs gave false-positive results (eight used the RPR and VDRL tests and one used the TPHA assay). Of note, only 50% of the TSs that used the VDRL or the RPR test (either alone or in combination with other tests) obtained an accurate result.

(c) Positive sample 03107C.

As was found for negative sample 03106B, 109 (92.4%) TSs gave correct results for positive sample 03107C (reference laboratory characterization as total Ig EIA+, IgG EIA+, TPHA assay+, IgM EIA, RPR test borderline). Of the remaining nine TSs, eight produced false-negative results (four by using the RPR and VDRL tests and four by using the TPHA and TPPA assays), whereas the remaining TS, which used different assays and which obtained discordant data, gave a borderline result.

(d) Positive sample 03108D.

The result for positive sample 03108D (reference laboratory characterization as total Ig EIA+, IgG EIA+, IgM EIA+, TPHA assay+, RPR test+), which had a very high antibody titer, was correctly classified by 116 (98.3%) TSs. The remaining two TSs failed to correctly classify the sample by using the VDRL test.

Figure Figure11 summarizes the results of TSs that were discordant with those of the reference laboratories for each shipment. False-positive results were mainly observed in the second shipment, whereas the incidence of false-negative and borderline results was highly reduced in that shipment.

FIG. 1.
Summary of incorrect results. The graph shows the percent discordance from the reference classifications obtained by TSs with the samples in the two shipments.

The results in Fig. Fig.22 indicate that the total Ig EIA and TPPA assay had the best performance in identifying syphilis, especially in the second shipment. The major discordances were found for the nontreponemal methods (the VDRL and RPR tests), which showed the poorest performance. The majority of the false-negative results in the first shipment and false-positive results in the second shipment were obtained with these assays. Data are expressed as the percentage of all correct results obtained by a certain method with respect to all of the results (correct and not correct) obtained by the same method.

FIG. 2.
Percentage of correctly classified samples for the two shipments, by the type of test performed and the serological status of the sample. The graphs show the accuracies of the different methods that the TSs used to test the samples in the first (light ...

DISCUSSION

The World Health Organization has estimated that there are approximately 12 million new cases of syphilis worldwide each year (14), confirming that this infection is reemerging and still represents a primary health problem. Although transmission through blood transfusion is rare, screening for syphilis is mandatory for both the prevention of infectious diseases in transfusion medicine and the identification of high-risk sexual behavior in individual donors. Thus, the implementation of an appropriate diagnostic algorithm for the identification of syphilis by detecting anti-T. pallidum antibodies in serum is of utmost importance. According to the guidelines published by the U.S. Centers for Disease Control and Prevention, the diagnosis of syphilis should be based on the results of at least two tests: one treponemal and the other nontreponemal (13). In Italy, although the screening of donated blood units for syphilis is mandatory, there are no official guidelines, and each TS can thus adopt the assay (or the set of assays) deemed most suitable on the basis of local needs.

The results of the present study confirm that the use of a single nontreponemal test is the least sensitive means of identifying samples that are positive for syphilis antibodies (15), as shown by the sensitivity of 0.55 found for the RPR test and the sensitivity of 0.72 found for the VDRL test. However, it should be noted that the interpretation of the results of manual techniques, such as the RPR, VDRL, TPHA, and TPPA assays, can vary greatly among different centers and operators. In fact, the interpretation of the agglutination assay result strongly depends on the quality of the testing and the experience of the laboratory personnel. For this reason, it is often difficult to compare results from different laboratories (8).

When the results obtained with the samples in the two shipments were compared, there was a marked difference in the percentage of correctly classified samples. For the first shipment, the percentage of correct results was lower than that for the second shipment, with a higher percentage of positive samples erroneously being classified as negative (i.e., false-negative results). A higher percentage of false-positive results and a lower percentage of false-negative results were obtained with the samples in the second shipment. One explanation for this finding could be that the TSs were influenced by the results of the first shipment. In fact, those TSs that used a single nontreponemal test which produced a false-negative result with samples in the first shipment were more cautious in evaluating the samples in the second shipment (i.e., there was a tendency to interpret the results as positive). Although these results were less accurate for negative samples, they are more reassuring in terms of the safety accuracy of pretransfusion screening; in other words, although a negative blood unit could be discarded, there is a decreased risk of transfusion of a positive unit. Moreover, some TSs used different assays to test the samples in the second shipment, and there was an increased rate of use of the total Ig EIA, which is the most sensitive assay. In fact, completely automated tests for the detection of specific anti-Treponema pallidum antibodies were developed to have improved sensitivity.

On the basis of the results presented here, we can conclude that the use of a single test is not sufficient for screening and that all blood units should thus be assessed by two distinct treponemal tests, namely, the total Ig EIA and the TPHA or the TPPA assay. This is also in agreement with the recommendations for the serological diagnosis of syphilis issued by the United Kingdom (5). For results that cannot be interpreted or for the more accurate determination of the status of the infection, it might be necessary to perform the IgG and IgM EIA and the RPR test.

Acknowledgments

We appreciate the support and advice of F. Ensoli (Clinical Pathology and Microbiology Department of the San Gallicano Dermatological Institute, Rome, Italy) for his comments on the manuscript. We also thank M. Kanieff for linguistic revision of the manuscript.

Footnotes

[down-pointing small open triangle]Published ahead of print on 30 December 2009.

REFERENCES

1. Barca, A., R. Sardelli, F. Mozzi, G. Pizzocolo, and M. Orlando. 1997. Programma nazionale di valutazione esterna della qualità dei test di screening per la ricerca di HBsAg, anti-HCV e anti-HIV presso le strutture trasfusionali: attività e risultati 1992/1996. Ligand Assay 2:224-229.
2. Brant, L. J., A. Bukasa, K. L. Davison, J. Newham, and J. A. Barbara. 2007. Increase in recently acquired syphilis infections in English, Welsh and Northern Irish blood donors. Vox Sang. 93:19-26. [PubMed]
3. Candido, A., P. Chionne, L. Milazzo, A. Dettori, E. Madonna, A. Taffon, L. A. Kondili, A. Barca, H. J. Hassan, and M. Rapicetta. 2008. Nucleic acid testing (NAT) for HCV RNA in Italian transfusion centres: an external quality assessment. J. Clin. Virol. 41:277-282. [PubMed]
4. Catalano, L., F. Abbonizio, A. Giampaolo, and H. J. Hassan. 2007. National and regional register of blood and plasma. Report 2006. Rapporti ISTISAN 07/46. Istituto Superiore di Sanità, Rome, Italy.
5. Egglestone, S. I., and A. J. Turner. 2000. Serological diagnosis of syphilis. PHLS Syphilis Serology Working Group. Commun. Dis. Public Health 3:158-162. [PubMed]
6. Golden, M. R., C. M. Marra, and K. K. Holmes. 2003. Update on syphilis: resurgence of an old problem. JAMA 290:1510-1514. [PubMed]
7. Gonzalez, M., V. Regine, V. Piccinini, F. Vulcano, A. Giampaolo, and H. J. Hassan. 2005. Residual risk of transfusion-transmitted human immunodeficiency virus, hepatitis C virus, and hepatitis B virus infections in Italy. Transfusion 45:1670-1675. [PubMed]
8. Müller, I., V. Brade, H. J. Hagedorn, E. Straube, C. Schörner, M. Frosch, H. Hlobil, G. Stanek, and K. P. Hunfeld. 2006. Is serological testing a reliable tool in laboratory diagnosis of syphilis? Meta-analysis of eight external quality control surveys performed by the German Infection Serology Proficiency Testing Program. J. Clin. Microbiol. 44:1335-1341. [PMC free article] [PubMed]
9. Offergeld, R., S. Ritter, and O. Hamouda. 2006. The risk of transfusion transmitted infections—current aspects. Transfus. Med. Hemother. 33:130-134.
10. Piccinini, V., F. Abbonizio, L. Catalano, and H. J. Hassan. 2006. Regional distribution of the Italian transfusion centres (2005 updating). Strumenti di riferimento 06/S1. Istituto Superiore di Sanità, Rome, Italy.
11. Piccinini, V., F. Vulcano, S. Palmieri, L. Catalano, A. Giampaolo, and H. J. Hassan. 2008. Surveillance of transfusion-transmissible infections in Italy: year 2006. Not. Ist. Super. Sanità 21:3-6.
12. Singh, A. E., and B. Romanowski. 1999. Syphilis: review with emphasis on clinical, epidemiologic, and some biologic features. Clin. Microbiol. Rev. 12:187-209. [PMC free article] [PubMed]
13. Workowski, K. A., and S. M. Berman. 2006. Sexually transmitted diseases treatment guidelines, 2006. MMWR Recommend. Rep. 55(RR-11):1-94. [PubMed]
14. World Health Organization. 2001. Global prevalence and incidence of selected curable sexually transmitted infections: overview and estimates. World Health Organization Geneva, Switzerland. http://www.who.int/hiv/pub/sti/en/who_hiv_aids_2001.02.pdf.
15. Young, H. 2000. Guidelines for serological testing for syphilis. Sex Transm Infect. 76:403-405. [PMC free article] [PubMed]

Articles from Journal of Clinical Microbiology are provided here courtesy of American Society for Microbiology (ASM)