Application of Bioluminescence Imaging to the Prediction of Lethality in Vaccinia Virus-Infected Mice

doi:10.1128/JVI.01296-09

J Virol. 2009 October; 83(20): 10437–10447.

Published online 2009 August 5. doi: 10.1128/JVI.01296-09

PMCID: PMC2753125

Application of Bioluminescence Imaging to the Prediction of Lethality in Vaccinia Virus-Infected Mice ^†

Marina Zaitseva,¹^* Senta M. Kapnick,¹ John Scott,¹ Lisa R. King,¹ Jody Manischewitz,¹ Lev Sirota,¹ Shantha Kodihalli,² and Hana Golding¹

Division of Viral Products, Center for Biologics Evaluation and Research, Food and Drug Administration, Bethesda, Maryland 20892,¹ Department of Clinical Research, Cangene Corporation, Winnipeg, Manitoba, Canada R3T5X8²

^*Corresponding author. Mailing address: Division of Viral Products, Center for Biologics Evaluation and Research, Food and Drug Administration, Building 29B, Room 4NN06, 8800 Rockville Pike, Bethesda, MD 20892. Phone: (301) 827-0788. Fax: (301) 496-1810. E-mail: marina.zaitseva/at/fda.hhs.gov

Received June 24, 2009; Accepted July 26, 2009.

This article has been cited by other articles in PMC.

Abstract

To find an alternative endpoint for the efficacy of antismallpox treatments, bioluminescence was measured in live BALB/c mice following lethal challenge with a recombinant WR vaccinia virus expressing luciferase. Intravenous vaccinia immunoglobulin treatments were used to confer protection on a proportion of animals. Using known lethality outcomes in 200 animals and total fluxes recorded daily in live animals, we performed univariate receiver operating characteristic (ROC) curve analysis to assess whether lethality can be predicted based on bioluminescence. Total fluxes in the spleens on day 3 and in the livers on day 5 generated accurate predictive models; the area under the ROC curve (AUC) was 0.91. Multiple logistic regression analysis utilizing a linear combination of six measurements: total flux in the liver on days 2, 3, and 5; in the spleen on days 1 and 3; and in the nasal cavity on day 4 generated the most accurate predictions (AUC = 0.96). This model predicted lethality in 90% of animals with only 10% of nonsurviving animals incorrectly predicted to survive. Compared with bioluminescence, ROC analysis with 25% and 30% weight loss as thresholds accurately predicted survival on day 5, but lethality predictions were low until day 9. Collectively, our data support the use of bioimaging for lethality prediction following vaccinia virus challenge and for gaining insight into protective mechanisms conferred by vaccines and therapeutics.

In 1980, the World Health Organization declared that the world was finally free of smallpox as an extant human disease, and routine smallpox vaccination was discontinued. However, concerns that variola virus (the causative agent of smallpox) might be reintroduced into the human population as a bioterrorist agent has intensified research targeted toward the development of novel antiviral therapies and safe and effective vaccines (3, 14). The global eradication program was accomplished using several vaccines based on attenuated replicating vaccinia virus strains, including Dryvax—a vaccine based on live vaccinia virus derived from the New York City Board of Health strain (NYCBOH) and prepared from calf lymph (Wyeth Laboratories) that was used in the Unites States and West Africa (22). For a long time, Dryvax was the only U.S. licensed vaccine for human use against the smallpox virus and was considered the “gold standard” due to the long-lasting virus-neutralizing antibodies it generated (1, 13). However, due to substantial risks of developing adverse reactions, such as progressive vaccinia, eczema vaccinatum, and severe generalized vaccinia, vaccination is contraindicated in people with compromised immune systems and individuals with eczema. In addition, the Dryvax vaccine was also shown to induce transient myopericarditis in a small proportion of healthy recipients, and therefore, it is no longer recommended for general use (2). The smallpox ACAM2000 vaccine utilizing the same NYCBOH strain of vaccinia virus and propagated in African green monkey cells (Vero cells) is manufactured by Acambis/Baxter Pharmaceuticals and was recently licensed by the Food and Drug Administration for clinical use. A total of six clinical trials (phases I to III) with ACAM2000 vaccine were performed, and the results showed that vaccine-emergent reactions were slightly reduced in the ACAM2000 recipients (11). However, concerns related to stability, inability to be diluted, and decreased numbers of takes in ACAM2000 vaccine-experienced subjects suggested that further developments are needed to ensure the maintenance of a potent and safe vaccine stockpile (11). Several alternative vaccine modalities that are expected to have better safety profiles are under development or in clinical trials, such as modified vaccinia Ankara (5), DNA plasmids, and recombinant proteins. It is conceivable that new immunotherapies are needed to ensure safe and efficient treatment of complications associated with live vaccinia virus-based vaccines and for treatment of subjects exposed to variola virus.

Intravenous vaccinia virus immunoglobulin (VIGIV; Cangene Corporation) is a new product that replaced intramuscular vaccinia virus immunoglobulin (VIGIM), which had been used for treatment of complications related to smallpox vaccinations since the 1950s. VIGIV is prepared from the purified gamma immunoglobulin (immunoglobulin G) fraction of plasma taken from healthy donors previously vaccinated with Dryvax who demonstrate high titers of vaccinia virus neutralizing antibody. This product was licensed for the treatment of patients who develop complications following Dryvax vaccination. It has been suggested that VIGIV might also be life saving for unvaccinated persons who have come into contact with people exposed to variola virus itself and may help to limit the spread of the disease (27). The efficacy of VIGIV has not been tested in many clinical situations. The clinical pharmacology and pharmacodynamics of VIGIV were tested in two phase I clinical trials, which showed that the product was well tolerated, with all adverse effects related to VIGIV being typical of those expected following intravenous administration of the protein (27). The recommended dosage of VIGIV for the management of several complications of smallpox (vaccinia) vaccinations is 6,000 U/kg body weight (4).

The development of novel vaccines against smallpox and of new pre- or postexposure therapies critically depends on the use of animal models for the initial preclinical testing. Several animal models are employed as surrogate models of variola virus infection in humans, including infection of macaques with monkeypox virus, which has been used as a tool to dissect the immune responses to poxviruses (21). Importantly, experiments with rhesus macaques demonstrated a critical role for antibodies in the protection of vaccinated animals and helped select an optimal vaccine regimen utilizing DNA-coded monkeypox virus proteins in proof-of-concept studies (6, 15). However, due to the expense and the requirement for biosafety level 3 facilities, nonhuman primate models cannot be widely used for smallpox vaccine development. The majority of preclinical testing and initial characterization of smallpox vaccines and therapies is performed with the Western Reserve (WR) strain of vaccinia virus or with ectomelia virus, which is highly lethal in mice. Various endpoints are used to follow infections in mice, including weight loss, pox lesion scoring, and viral-load measurements by plaque formation on sensitive cell lines. These endpoints are not optimal, as they cannot avoid morbidity or accurately predict lethality in individual animals. They require very large number of animals in order to determine survival rates and to quantify viral loads in internal organs. Lethality was frequently used as an endpoint in the past but is no longer acceptable.

Whole-animal bioimaging has been widely used for studies of microbial and viral pathogens in small-animal models (16, 19). This technology allows monitoring of pathogen dissemination in real time, locating pathogens residing in unexpected anatomical sites, and greatly reducing the numbers of animals required per study by providing spatial and temporal information for individual animals (16, 19). For this purpose, genes coding for luciferase enzymes are expressed in bacterial or viral pathogens, and dissemination of the recombinant pathogen is recorded by detecting light emitted from the tissues of infected live animals (19). Using bioimaging of live animals, bioluminescence of poxviruses expressing luciferase was employed to evaluate mucosal vaccinia virus vaccine (7), to dissect the roles of type I interferons and innate immunity in controlling viral replication and spread (18), and to characterize the tissue distribution of and immune responses induced by viral strains used for the development of vectored vaccines against other pathogens (9, 23). Measurements of photon fluxes have been shown to correlate in linear fashion with viral loads in internal organs in vaccinia virus-infected mice, suggesting that bioluminescence provides a direct measure of viral dissemination (19). However, whether bioluminescence signals derived from luciferase-expressing vaccinia virus can be used to predict lethality in infected mice has not yet been investigated.

To that end, we used a recombinant WR vaccinia virus expressing luciferase to record bioluminescence in healthy mice after intranasal (i.n.) infection with 1 to 5 50% lethal doses (LD₅₀). Groups of animals were immunized with Dryvax or pretreated with VIGIV. We used acquired images from surviving and nonsurviving mice to calculate total fluxes in internal organs and applied receiver operating characteristic (ROC) curve analysis to generate predictive models. Our data showed that measurements of total fluxes in the spleen and liver 3 and 5 days postinfection provided strong predictive models of lethality. The predictive power of bioimaging was further investigated when bioluminescences from several organs and multiple time points were combined in multiple logistic regression analysis.

MATERIALS AND METHODS

Virus.

A thymidine kinase-positive (TK+) recombinant WR vaccinia virus expressing the luciferase reporter gene under the control of a synthetic immediate-early promoter (WRvFire) was kindly provided by Bernard Moss (NIAID, NIH) and was used in all experiments (25). Preliminary studies established the LD₅₀ of this virus to be around 1 × 10⁵ PFU in BALB/c mice following i.n. administration.

Mice and protocols for in vivo treatments.

Female BALB/c mice (National Cancer Institute, Frederick, MD) were used at 5 weeks of age. The mice weighed 16 to 20 g at the time of infection. Immediately prior to challenge, the mice were anesthetized using 2,2,2-tribromoethanol dissolved in tertiary amyl alcohol and diluted in sterile phosphate-buffered saline (PBS) according to the manufacturer's instructions. The anesthesia was administered at 20 μl per gram body weight by intraperitoneal (i.p.) injection. Preliminary studies were performed to optimize the dose and the volume of inocula for i.n. infection of anesthetized mice. In the experiments included in the current study, mice were challenged with 10⁵ PFU in 10 μl. In some experiments, the mice were challenged with 10⁴ or with 10⁶ PFU of WRvFire.

Immunizations of mice against vaccinia virus were performed by i.p. inoculation of Dryvax vaccine at 10⁶ PFU either 2 or 3 weeks before challenge, based on an earlier study in the laboratory. For prophylactic passive immunizations, animals were given the human VIGIV at 0.3, 1.0, 3.0, 10.0, or 30.0 mg/animal (lot 173601; 56 mg/ml; 6,675 U/ml; Cangene Corporation, Winnipeg, MB, Canada) in 600 μl of PBS via the i.p. route 2 days prior to WRvFire challenge, based on earlier studies of SCID mice (8) and preliminary protection experiments with normal BALB/c mice.

The neurovirulent WR strain of vaccinia virus has been shown to induce acute lethality within 6 to 7 days following i.n. challenge (26). In addition, in preliminary experiments utilizing prophylactic VIGIV treatments of mice infected with WRvFire via the i.n. route, the animals were observed for a month and no deaths were noted beyond the 10-day time point. Therefore, in this study, the mice were followed for 14 days after challenge, at which point all surviving mice were euthanized by CO₂ asphyxiation. The handling of mice and experimental procedures were approved by the Center for Biologics Evaluation and Research animal study review committee.

In vivo measurements of luciferase activity.

Mice were injected i.p. with 150 μg/g body weight of d-luciferin potassium salt (Caliper Life Sciences, Hopkinton, MA) 10 to 15 min prior to being imaged. The mice were anesthetized in an oxygen-rich induction chamber with 2% isofluorane and were imaged in dorsal and ventral positions using an IVIS 50 cooled charge-coupled-device camera system (Caliper). To overcome a dramatic difference between bioluminescence in the internal organs and nasal cavities and to avoid oversaturation of the camera, images of mouse torsos were acquired with the heads covered with black paper. Images were collected daily between days 1 and 10 postchallenge at optimized exposure times to avoid saturation of the camera and were analyzed with Living Image 3.02 software (Caliper). To quantify the amounts of light emitted from specific organs, a single region of interest (ROI) was created for each organ and used throughout analysis. The background bioluminescence was determined using images of d-luciferin-injected animals 1 day prior to infections and was subtracted from experimental values during analysis. To normalize the values of photons registered in infected organs for the differences in image acquisition times and fields of view, the relative bioluminescence was calculated using a photon-per-second mode with normalization for the imaging area (photons/s/cm²/sr) (total flux), as recommended by the manufacturer and as previously described (19). For correlation studies, immediately after being imaged, infected mice were euthanized by CO₂ asphyxiation, and the internal organs (lungs, spleens, and livers) were collected. After the weights were recorded, the organs were processed immediately for viral-load measurements in vitro or frozen at −70°C.

Processing of organs and measurements of viral loads by plaque assay and by in vitro luciferase assay.

For plaque assay, fresh organs were placed into the wells of six-well plates (Costar) containing 1 ml of PBS. The tissues were minced, cell suspensions were prepared using the bottoms of 3-ml syringes, and serial 1:10 dilutions of the cell suspensions were prepared in PBS. One hundred thousand BSC-1 cells per well were seeded into 24-well plates (Costar) 24 h before assay. At the time of the assay, the medium was removed from the BSC-1-containing wells, and 0.1 ml of the neat or diluted (to avoid toxicity) cell suspension was added to duplicate wells. After a 2-h incubation at 37°C, Eagle's minimal essential medium containing 2% fetal calf serum was added, and the plates were incubated at 37°C for 48 h. At the end of the incubation, the medium was removed and the cell monolayer was washed once with PBS before being stained with 0.1% crystal violet solution in 20% ethanol. Plaques were counted, and the viral titers were calculated.

For the in vitro luciferase assay, each frozen organ was pulverized into a fine powder using a mortar and pestle chilled on dry ice. The powder was placed into a 1.5-ml Eppendorf vial and resuspended in 500 μl of 1× reporter lysis buffer (Promega, Madison, WI) in the presence of EDTA-free Complete protease inhibitor (Roche, Indianapolis, IN). After being vortexed for 15 min, the vials containing the resuspended powder were subjected to three freeze-thaw cycles using dry ice and a 37°C water bath and centrifuged for 3 min at 10,000 rpm on a tabletop Eppendorf centrifuge, and the supernatants were collected for storage at −70°C. An additional 500 μl of 1× reporter lysis buffer was added to the pellet, the vials were vortexed, and after the vials were spun, the supernatants were collected and stored at −70°C. Luciferase assay of the stored samples was performed using a Promega Luciferase Assay System according to the manufacturer's instructions and read on a Synergy 2 Multi-Detection Microplate Reader (Biotech Instruments Inc., Winooski, VT).

Statistical analysis.

Kaplan-Meier survival curves of time to death following infection were generated using standard GraphPad Prism V5 software. We used t tests to compare the mean total fluxes between each Dryvax regime and challenge-only groups and between surviving and nonsurviving animals using Excel. JMP 7.0 (SAS Institute Inc., Cary, NC) was used to perform ROC curve analyses to assess the power of total flux measurements each day and in each organ as predictors of a lethal outcome. In these analyses, an area under the ROC curve (AUC) of 1.0 represented a perfect test with 100% detection of lethality (sensitivity) and 100% detection of survival (specificity), whereas an area of 0.5 represented random discrimination. The 95% confidence intervals for each AUC were calculated using a bootstrap method (10).

After evaluating the predictive capacity of each total flux measurement individually, we constructed a series of multiple logistic regression models to evaluate the performance of collections of total flux measurements as multidimensional predictors of lethality using SAS version 9.2. In brief, a stepwise variable-selection procedure was used to identify daily organ total flux measurements most strongly associated with lethality, and a ROC curve was calculated for the chosen set of measurements. This procedure was repeated using synthetic variables derived from the daily total flux measurements: the mean total flux across days for each organ, the rate of change of the total flux in each organ over the 5 days postinfection (estimated as the linear regression slope), and the maximum total flux in each organ.

RESULTS

Visualization and quantitation of WRvFire virus.

Initially, it was important to investigate the sensitivity of the system and to correlate the levels of bioluminescence recorded in live animals with viral loads measured in various organs by in vitro assays. Five-week-old BALB/c mice were infected with 10⁵ PFU of WRvFire i.n. and were imaged on days 1 to 10 postinfection. Strong signals were detected at various time points in the nasal cavity, lungs, spleen, liver, and ovaries. In addition, pox development was also noted in WRvFire-infected mice (Fig. (Fig.11).

FIG. 1.

Images of 5-week-old mice infected with WRvFire. Representative images of the heads and ventral torsos (day 3) and of dorsal torsos (day 6) of BALB/c mice infected with 10⁵ PFU of WRvFire via the i.n. route are shown. Bioluminescence in the nasal cavity (more ...)

To determine whether there was a direct correlation between the light detected by the camera and the amount of luciferase present in the organ, BALB/c mice were infected with WRvFire at 10⁴, 10⁵, or 10⁶ PFU. Six days after infection, the mice were subjected to bioimaging. Bioluminescence in the lungs and livers was determined using the ROI tool as shown in Fig. Fig.1,1, and the total fluxes emitted by infected organs were calculated. The animals were sacrificed, and the lungs and livers were harvested and tested individually in the luciferase reporter gene assay ex vivo. The levels of total fluxes emitted by the lungs and livers recorded in live animals were compared with the numbers of luciferase units per gram tissues obtained from the same 16 animals (a total of 32 organs) (Fig. (Fig.2).2). Linear correlation (R² = 0.82) was observed between the values of luciferase expression detected by in vivo and ex vivo assays, suggesting that bioimaging provided an accurate evaluation of luciferase expression in individual organs.

FIG. 2.

Correlation plot analysis of WRvFire expression evaluated using in vivo bioimaging and an ex vivo luciferase reporter assay. Sixteen mice were infected with WRvFire as described in the text. Six days postinfection, the mice were subjected to bioimaging, (more ...)

To confirm that WRvFire viral loads could be quantitatively assayed by bioimaging, the traditional PFU assay was employed. Twelve BALB/c mice were infected i.n. with WRvFire at 10⁵ PFU/animal, and images of six infected mice per time point were taken 3 and 4 days postinfection before euthanasia (Fig. (Fig.3).3). The lungs and spleens were collected, and the viral loads in these organs were determined by plaque assay. The numbers of PFU per gram of tissue detected in the organs of individual mice were plotted against the values of WRvFire expression in the same organs detected by bioimaging of live animals. The data were processed by linear regression analyses. In both organs, a very strong correlation was found between bioluminescence and the traditional PFU assay. The R² values were 0.83 and 0.97 for lungs and spleens, respectively (Fig. (Fig.3).3). Together, these data demonstrated that the dynamic range and the sensitivity of the bioimaging assay were similar to those of the ex vivo viral load measurements. Therefore, bioimaging could be evaluated further for its ability to predict survival after prophylactic vaccination or therapies.

FIG. 3.

Luciferase expression detected in the lungs and spleens during bioimaging of live animals correlated well with viral loads determined by plaque assay. Twelve mice were infected with 10⁵ PFU of WRvFire. On days 3 and 4 postinfection, images of live animals (more ...)

Bioimaging of animals following vaccine-induced protection from lethal challenge.

To determine whether protection from lethality was correlated with a reduction in bioluminescence in live animals, BALB/c mice were inoculated with the Dryvax vaccine either 2 or 3 weeks prior to challenge with WRvFire. All control mice succumbed to death within 8 days postinfection, and all mice that received Dryvax vaccine i.p. were protected (Fig. (Fig.4a).4a). Images were collected daily using the IVIS instrument and were used to quantify the total flux in the nasal cavity and in the lungs of individual animals (Fig. 4b and c). In both the unimmunized and the Dryvax-immunized groups, the initial luciferase signals in the nasal cavity were on the order of 10⁸ photons/s/cm²/sr. The signals in the unimmunized animals increased about 1.5 to 2 log units during the following days and reached a plateau between days 5 and 7 (Fig. (Fig.4b).4b). In contrast, in Dryvax-immunized animals, a rapid reduction in bioluminescence was observed, with a complete loss of signals in the nasal cavity by day 4 (Fig. (Fig.4b).4b). Bioluminescence in the lungs of unimmunized mice ranged between 10⁵ and 10⁶ photons/s/cm²/sr on day 1, increased approximately 1.5 to 2 log units, and plateaued on day 5 (Fig. (Fig.4c).4c). Five mice in the group of mice immunized 3 weeks prior to challenge showed initial signals in the lungs on day 2 (5 × 10⁶ photons/s/cm²/sr) that disappeared on day 3. No bioluminescence was detected in the lungs of the remaining Dryvax-immunized animals on day 2. The values of the daily bioluminescence measurements in the nasal cavities and in the lungs in unimmunized and in Dryvax-immunized mice were subjected to t tests (Table (Table1).1). Following initial viral replication in the nasal cavity observed in both groups of mice on day 1, the differences in bioluminescence signals between the two groups of mice gradually increased, as reflected in dramatically increased t values between days 2 and 7 (Fig. (Fig.4b4b and Table Table1).1). In agreement with the observed higher variability of bioluminescence in the lungs, the t statistic for the lungs were lower than for the nasal cavity. Nevertheless, the differences between unimmunized and Dryvax-immunized groups were significant starting from day 3 (t ≥ 2) (Table (Table1).1). No bioluminescence was detected in the livers, spleens, and ovaries of protected animals (data not shown). These data showed that protection from lethality conferred by prophylactic Dryvax immunization was correlated with the absence of bioluminescence in the internal organs and with significantly lower bioluminescence in the nasal cavity and in the lungs of protected mice within 2 days postchallenge.

FIG. 4.

Complete protection from lethality in Dryvax-immunized mice was correlated with low or no bioluminescence in the lungs and nasal cavity. (a) BALB/c mice were inoculated i.p. with PBS (black curves) or with 10⁶ PFU of Dryvax vaccine at weeks (wk) −3 (more ...)

TABLE 1.

t statistic for the total fluxes in challenge-only mice compared with Dryvax-immunized mice^a

Bioimaging of animals treated with VIGIV prior to WRvFire challenge.

While active immunization with a potent vaccine such as Dryvax is likely to provide complete protection from morbidity and mortality, there are currently no global vaccination campaigns planned. In the event of smallpox outbreaks, active vaccination of all high-risk individuals, such as military personnel, designated emergency “first responders,” and health care providers, might not be feasible. Instead, preexposure treatments with anti-vaccinia virus immunoglobulin G, while not providing sterilizing immunity, could provide protection from morbidity and lethality. Therefore, it was important to determine whether bioimaging can detect differences between nonsurviving and surviving animals in WRvFire replication and dissemination to internal organs following prophylactic treatments with VIGIV. To this end, we performed six experiments in which protection from lethality was monitored in animals receiving placebo (PBS control) or VIGIV at doses ranging from 0.3 to 30 mg/animal 2 days before challenge. One representative experiment is shown in Fig. Fig.5.5. BALB/c mice were inoculated with VIGIV at the indicated concentrations, followed by i.n. challenge with 1 × 10⁵ PFU of WRvFire. The results of this experiment showed that 80% of untreated mice died between days 7 and 10 postinfection. Among the VIGIV-pretreated groups, 30 mg and 10 mg VIGIV (corresponding to 3,576 units and 1,192 units/animal, respectively) completely protected mice from lethality; VIGIV at 1 and 3 mg protected 50% of the animals, and 0.3 mg of VIGIV did not protect at all (Fig. (Fig.5).5). Six experiments with partial protection from lethality by VIGIV were performed, and in all experiments, images of infected mice were collected (Fig. (Fig.6).6). Total fluxes in the nasal cavities, lungs, livers, and spleens of 200 animals were recorded for 10 days (or prior to mortality) (Fig. 6a to d). In total, 147 animals survived and 53 died. Mean bioluminescence values reached maximum levels on day 5 in the lungs, nasal cavities, and livers and on day 3 in the spleens (Fig. (Fig.6).6). The calculated t values confirmed that the differences in the total fluxes in internal organs between surviving and nonsurviving animals were significant ( Table Table22).

FIG. 5.

Survival of WRvFire-challenged mice following prophylactic treatment with VIGIV. BALB/c mice received i.p. injections of VIGIV at the indicated concentrations 2 days before i.n. challenge with WRvFire (10⁵ PFU) and were followed for lethality as described (more ...)

FIG. 6.

Total fluxes in internal organs of surviving and nonsurviving mice challenged with WRvFire and pretreated (or not) with VIGIV. Two hundred BALB/c mice in six experiments were challenged with WRvFire alone or received VIGIV at 0.3, 1.0, 3.0, 10.0, or 30.0 (more ...)

TABLE 2.

t statistic for the total fluxes in surviving (n = 147) versus nonsurviving (n = 53) animals^a

Applying statistical methods to bioluminsecence measurements to generate models predicting lethality in VIGIV-pretreated and WRvFire-challenged animals.

Statistical analysis showed that the mean levels of total fluxes recorded from the upper respiratory tract, lungs, and other internal organs were significantly different between surviving and nonsurviving animals. It was also apparent that VIGIV pretreatments did not provide complete control of virus replication in the upper respiratory tract and lungs and did not completely prevent dissemination of WRvFire to the internal organs, even in animals that survived lethal challenge. Some animals that exhibited strong bioluminescence survived, and some animals with low levels of signal died. In other words, there was a clear overlap in the bioluminescence levels between surviving and nonsurviving animals. Therefore, a direct comparison of the fluxes alone was not sufficient to predict survival or lethality with perfect accuracy.

In order to develop a reliable prediction model, we applied ROC curve analysis. To identify early markers for prediction, we used bioluminescences recorded on days 1 to 5. AUCs were determined for each organ and for each time point using total fluxes in surviving and nonsurviving animals from the experiments shown in Fig. Fig.66 (Table (Table3;3; see Fig. S1 in the supplemental material). The AUC values for days 1 to 5 were between 0.78 and 0.85, 0.72 and 0.83, 0.66 and 0.91, and 0.80 and 0.91 for the nasal cavity, lungs, liver, and spleen, respectively (Table (Table3).3). Maximum AUC values were obtained on day 5 for the nasal cavity, lungs, and liver and on day 3 for the spleen, suggesting that the greater divergence in bioluminescence between surviving and nonsurviving animals coincided with the day of maximum mean bioluminescence value for each organ (Table (Table33 and Fig. Fig.6).6). Measurements of total fluxes in the spleens on day 3 predicted lethality with 94% sensitivity (94% of the animals that died were correctly identified as nonsurviving [true positives {TP}] based on bioluminescence above the selected threshold) and with 73% specificity (73% of the animals that survived were correctly predicted to survive [true negatives {TN}] based on observed bioluminescence that was below the selected threshold) (Table (Table3).3). In the same animals, total fluxes in the liver on day 5 provided 100% sensitivity and 72% specificity, which were higher than the combined sensitivities and specificities obtained with total fluxes in the nasal cavity and lungs at any time point (Table (Table3).3). This suggested that bioluminescence in the spleen on day 3 and the liver on day 5 may provide better predictive models than total fluxes recorded in the nasal cavity or lungs.

TABLE 3.

ROC analysis of bioluminescence in the internal organs of WRvFire-infected mice recorded on days 1 to 5 postinfection

The goal of the study was to determine whether bioimaging can substitute for lethality as an endpoint for the evaluation of antismallpox vaccines and therapies. Therefore, we chose a targeted sensitivity of >90% as a primary parameter, i.e., over 90% of animals with lethal outcomes are correctly identified as TP. Assuming that the sensitivity is expected to be >90%, we sought the organ and time point that provided the highest specificity, i.e., correctly predicted surviving animals as TN if their bioluminescence was below the selected threshold and, accordingly, identified fewer surviving animals with bioluminescence above the threshold (false positives [FP]) (Table (Table4).4). In this scenario, the highest specificities (the highest percentage of identified TN) were achieved with total flux measurements from day 5 in the nasal cavity, lungs, and liver and on day 3 using fluxes in the spleen, with computed specificities between 69 and 78% (Table (Table4).4). Accordingly, the percentages of expected FP (calculated as 100 − specificity) were between 54 and 65% when total fluxes from day 1 were used for modeling and were reduced to 22 to 33% with total fluxes from days 3 (spleen) and 5 (all other organs) (Table (Table4).4). The univariate model utilizing total fluxes from the liver on day 5 had a specificity of 78% (with 22% of surviving mice identified as FP), which was the highest rate of correct identification of surviving animals compared with other organs and time points (Table (Table4).4). In summary, our data showed that if the sensitivity is set to correctly predict lethality in 90% of animals that will in fact die, measurements of total fluxes in the spleen on day 3 and the liver on day 5 postinfection will correctly predict survival in 76 to 78% of the animals that survive.

TABLE 4.

Predictability of lethality and survival outcomes using total fluxes recorded from WRvFire-infected mice

To determine which organ at which associated time point could serve as an optimal predictor of survival, we used the same models described above and set the specificity at >90%, i.e., over 90% of animals that survive exhibit bioluminescence below the selected threshold (TN) (Table (Table4).4). We then sought the organ and time point that provided the best sensitivity or most accurately identified the proportion of nonsurviving animals predicted to die (TP). Accordingly, it was expected that the same organ/time point should identify the lowest proportion of nonsurviving animals with bioluminescence below the threshold (false negatives [FN]) (Table (Table4).4). When the specificity was set at >90%, total flux in the spleen on day 3 provided a sensitivity of 74%, which was higher than the sensitivities computed with total fluxes in the liver, lungs, and nasal cavity at any time point (Table (Table4).4). At the same time, the univariate model utilizing total flux in the spleen on day 3 identified 26% of FN, which was the lowest rate of incorrect predictions (FN) compared with any other organ or time point. These data confirmed that the total flux in the spleen on day 3 may provide an optimal threshold for determination of survival in WRvFire-infected mice.

Multidimensional prediction of lethality.

Using multiple logistic regression with stepwise variable selection, we identified a collection of six total-flux measurements that each contributed significantly to predicting lethality even after the others were taken into account: flux in the liver on days 2, 3, and 5; in the spleen on days 1 and 3; and in the nasal cavity on day 4 (Fig. (Fig.7).7). The model with these six bioluminescence measurements had an AUC of 0.96 (Fig. (Fig.7a).7a). At a sensitivity of 90%, this model had a specificity of 88%, meaning that classification based on a threshold applied to a linear combination of these six variables led to only 10% of nonsurviving animals being incorrectly predicted to survive and only 12% of surviving animals being incorrectly predicted to die.

FIG. 7.

Development of highly predictive models using combinations of bioluminescence measurements in multiple organs. Total fluxes in the liver on days 2, 3, and 5 (Liv 2, 3, 5); in the spleen on days 1 and 3 (Sp 1, 3); and in the nasal cavity on day 4 (NC 4) (more ...)

We also performed a second multiple logistic regression with stepwise variable selection using synthetic variables derived from the bioluminescence measurements rather than the measurements themselves (Fig. (Fig.7b).7b). The synthetic variables considered were the mean bioluminescence across the first 5 days postinfection for each organ, the linear rate of increase in bioluminescence for each organ over 5 days (estimated as a linear regression slope), and the maximum bioluminescence across days measured in each organ. This procedure led to a more parsimonious model, with only three variables included: mean bioluminescence in the spleen and nasal cavity and the slope of bioluminescence increase in the spleen. The model yielded an AUC of 0.93 (Fig. (Fig.7b).7b). At a sensitivity of 90%, this model had a specificity of 83%, meaning that classification based on the model led to 10% of nonsurviving animals being incorrectly predicted to survive and 17% of surviving animals being incorrectly predicted to die. These data confirmed that combinations of bioluminescence recorded in several organs and at multiple time points provided the most accurate prediction model of lethality.

Weight loss is not an optimal predictor of lethality.

Infection with vaccinia virus causes rapid reduction in weight, and many studies use 25% or 30% weight loss as a threshold for sacrificing vaccinia virus-infected mice (20). We sought to develop a predictive model based on percentages of weight loss and compared it with our models that utilized bioluminescence. The weights of the same individual animals shown in Fig. Fig.66 were recorded on days 5, 7, and 9, and percentages of weight loss were calculated from control weights measured immediately prior to challenge (Fig. (Fig.8;8; see Table S2 in the supplemental material). ROC analysis of percent weight loss on days 5, 7, and 9 generated AUC values of 0.73, 0.85, and 0.91 (see Fig. S2 in the supplemental material). These data suggested that on day 5, weight losses were less predictive than bioluminescence; however, reductions in weight at later time points generated accurate prediction models.

FIG. 8.

Weight loss in surviving and nonsurviving mice following VIGIV administration and challenge with WRvFire. Weights were recorded for 182 animals prior to challenge and were followed during survival or for 10 days. Mean values for percent weight loss are (more ...)

To verify whether 25% or 30% weight loss could accurately predict lethality or survival, we calculated the specificity and sensitivity for each of these thresholds. Setting the threshold on day 5 to ≥25% weight loss provided 98% specificity, meaning that 98% of surviving animals exhibited weight loss of less than 25% and thus were correctly identified as survivors at that time point (Table (Table5).5). The same threshold provided a sensitivity of 11%: only 5 of 47 nonsurviving animals lost more than 25% of their body weight by day 5. Thus, a large number of nonsurviving animals (42 of 47) were incorrectly identified as survivors (FN), suggesting that although a ≥25% weight loss threshold may be an optimal predictor of survival, it does not accurately predict lethality at day 5 postinfection. The sensitivity of the model was greatly improved on days 7 (97%) and 9 (89%) utilizing the same ≥25% weight loss threshold (Table (Table5),5), but at the expense of specificity, meaning a large proportion of animals were incorrectly predicted to die based on weight loss (42 and 34% FP on days 7 and 9, respectively) (see Table S1 in the supplemental material). Animals used for ROC analysis reached 30% weight reduction levels only after day 5. Setting the threshold to ≥30% weight loss generated a specificity of 84% for days 7 and 9, suggesting that, as expected, a large proportion of animals that survived until that time point did not lose more than 30% of their weight (Table (Table5).5). The observed 59% sensitivity on day 7 suggested that many nonsurviving animals (41%) lost less than 30% of their weight and thus were incorrectly classified as survivors (FN) based on the selected threshold (see Table S1 in the supplemental material). These data showed that 25 or 30% weight loss thresholds provided accurate predictions of survival as early as day 5, but lethality predictions were difficult for both thresholds until day 9, when 89% sensitivity was obtained.

TABLE 5.

Prediction of lethality and survival using 25 and 30% weight loss as thresholds^a

The limited ability of the weight loss parameter to predict lethality was further supported by the simple analysis of the numbers of mice that survived although they lost greater than 25% of their weight (see Table S2 in the supplemental material). Of the 135 animals that survived to day 14, there were a total of 57 and 45 animals that lost more than 25% of their weight by days 7 and 10, respectively (see Table S2 in the supplemental material). The numbers of animals that lost 25 to 29.9%, 30 to 34.9%, or >35% of their weight at each observation time point were not identical, as some of the animals in these groups regained weight and other animals that survived showed weight reduction at the same time point. Importantly, the eventual survival of some mice that lost more than 25%, or even more than 35%, of their weight further supports the results of ROC analysis that demonstrated the low predictive value of weight measurements.

DISCUSSION

In a search for a quantitative endpoint indicative of protection in the vaccinia virus lethal-challenge model, we employed whole-body bioimaging to follow the dissemination of WRvFire virus in live animals. The main findings in our study were as follows. (i) Organ viral loads measured by PFU assessment correlated well with the total fluxes recorded in the same organs. (ii) Protection from lethality in Dryvax-immunized mice was correlated with a great reduction in bioluminescence in the lungs and nasal cavities 1 to 2 days postchallenge. (iii) The predictive power of bioluminescence was evaluated using 200 mice that were challenged with the lethal dose of WRvFire following VIGIV administration. ROC analysis of the total fluxes recorded in 200 animals with known survival outcomes generated predictive models with high AUC values and showed that bioluminescence in the spleen and liver on days 3 and 5, respectively, most accurately predicted lethality. (iv) A 90% accurate prediction of lethality was achieved with multidimensional analysis incorporating variables of six bioluminescence measurements recorded during the first 5 days postchallenge. (v) Comparison of the predictive power of bioimaging with the predictive power of weight loss (either 25% or 30%) confirmed that bioimaging can be used as a more accurate and earlier predictor of lethality than weight reductions in the WRvFire challenge model.

The initial sets of experiments demonstrated that bioluminescence detected a broad range of total fluxes emitted from multiple organs in live animals, which correlated well with luciferase activity detected in vitro and viral loads detected by a traditional PFU assay. In terms of sensitivity, the lowest viral titers detected in the spleens recovered from i.n.-infected mice were in the range of 10³ PFU/g tissue. These low titers were also detected by in vivo imaging in the same animals and corresponded to 2.5 × 10⁶ photons. Thus, in agreement with previous reports (18, 24), the data presented in the current study support the use of bioimaging as an alternative technique to measure poxvirus loads.

The highest mean values of total fluxes post-i.n. challenge with 1 to 5 LD₅₀ of WRvFire were detected in the nasal cavities and in the lungs. The mean bioluminescence signals in the livers were higher than in the spleen, and recorded fluxes in all organs showed large animal-to-animal variability. In comparison to other sites, bioluminescence signals in the ovaries and skin pox lesions started to appear with delayed kinetics on day 5 on average, and pox lesions were also noted in a large proportion of animals that survived the infection and therefore were not appropriate for the development of lethality prediction models.

We next tested whether bioluminescence acquired from the internal organs of WRvFire-infected mice can be used to predict lethality. Dryvax protected 100% of mice and was used as an internal positive control for the bioimaging. Protection in these mice was correlated with reduction in bioluminescence signals starting from day 3 postinfection. To create a scenario of less than 100% protection, we inoculated mice with VIGIV at doses ranging from 0.3 to 30 mg/animal 2 days before challenge, which resulted in a full spectrum of protection from 0 to 100%. We chose ROC analysis to determine whether measurements of total fluxes in VIGIV-treated and WRvFire-infected live animals can provide thresholds that accurately predict mortality versus survival outcomes. Steady increases in AUC values were observed, peaking at day 5 (nasal cavity, lungs, and liver) and between days 3 and 5 in the spleen. The highest AUC coincided with maximum mean bioluminescence signals, suggesting that optimal discrimination between surviving and nonsurviving animals can be expected when maximal viral replication is reached. From the AUC curves, it was evident that total fluxes recorded in the spleen on day 3 provided the earliest accurate indication of the infection outcome. Therefore, it is possible that the interaction between rising anti-vaccinia virus cellular immune responses shown to emerge starting from day 2 postinfection and the rate of viral replication in the spleen by day 3 can determine, to a large extent, whether an animal will survive or die (17).

Low AUC values obtained with total fluxes in the liver on days 1 and 2 suggested that at the initial infection, a low rate of viral replication in the liver does not necessarily lead to death. However, starting from day 3 and peaking on day 5, AUCs generated for the liver reached a maximal level in the univariate model, indicating that the rate of viral replication in the liver by days 4 to 5 postinfection could determine the outcome of the infection. This finding is indirectly supported by an earlier study showing that WR vaccinia virus was detected at higher titers in the liver than attenuated variant of WR vaccinia virus with mutations in the A33R or B5R genes (12). Surprisingly, predictive models utilizing total fluxes in the nasal cavity and lungs were weaker than the models generated with total fluxes from the liver and spleen. In part, this could be due to large animal-to-animal variability of viral entry into the lungs. Alternatively, it may reflect low access of the developing adaptive immune cells to the nasal cavity.

Vaccinia virus disseminates to multiple internal organs; therefore, to increase the discriminative power of our prediction models, we employed multidimensional analysis. A linear combination of six parameters from three organs provided very strong discriminative power, with an AUC of 0.96: at a sensitivity of 90%, this model incorrectly predicted survival in 10% of nonsurviving mice and death in only 12% of surviving mice. Similarly, a combination of synthetic variables, i.e., mean bioluminescence in the spleen and nasal cavity and the slope of bioluminescence increase in the spleen, generated a very strong predictive model (AUC = 0.93). Together, these data confirmed that bioluminescence in multiple organs, rather than in individual organs, was optimal for the prediction of lethality in the WRvFire lethal-challenge model.

Reduction in body weight of vaccinia virus-infected mice is routinely used as an indicator for euthanasia based on the assumption that vaccinia virus-infected mice do not recover if they lose 25 to 30% of their initial weight. ROC analysis using weight losses on day 5 generated a lower AUC value than that for bioluminescence in any organ on days 3 to 5. On days 7 and 9, the AUC values generated using percent weight loss were comparable to AUCs derived using bioluminescence on days 3 and 5. Altogether, these data confirmed that bioluminescence can accurately predict the mortality and survival of animals in the WRvFire lethal-challenge model early after infection (days 3 to 5) and before major weight loss is observed.

We then investigated the discriminative power of predictive models generated using 25% and 30% weight loss as thresholds for lethality. As expected, both thresholds accurately predicted survival. However, 25 and 30% weight loss did not work well as predictors of lethality: 25% loss was limited by low specificity even at days 7 and 9 (it incorrectly predicted death in a large proportion of animals based on the weight loss), and 30% loss reached accurate prediction only at day 9. Since animals are routinely sacrificed based on weight loss, the true efficacy of novel treatments in preventing lethality may not be accurately measured using this endpoint.

In summary, using side-by-side statistical analysis of two measurements, we showed that bioluminescence-based assessment of viral loads in internal organs can predict lethality more accurately than weight loss measurements in the vaccinia virus challenge model. Importantly, bioimaging can substitute for lethality as an endpoint in testing the efficacies of new vaccines and prophylactic interventions against poxvirus infections in the mouse model. This model could also be applied to the evaluation of postexposure treatments with immune-based therapies and antiviral drugs.

Supplementary Material

[Supplementary material]

Click here to view.

Acknowledgments

We express our gratitude to Mike Bray, Clement Meseda, and Jerry Weir for reviews of the manuscript and insightful comments.

This project was funded in part with Federal funds from the National Institute of Allergy and Infectious Diseases, National Institutes of Health, Department of Health and Human Services, under IAA 224-06-1322.

Footnotes

Published ahead of print on 5 August 2009.

^†Supplemental material for this article may be found at http://jvi.asm.org/.

REFERENCES

1. Amanna, I. J., M. K. Slifka, and S. Crotty. 2006. Immunity and immunological memory following smallpox vaccination. Immunol. Rev. 211:320-337. [PubMed]

2. Belongia, E. A., and A. L. Naleway. 2003. Smallpox vaccine: the good, the bad, and the ugly. Clin. Med. Res. 1:87-92. [PubMed]

3. Bray, M., and C. J. Roy. 2004. Antiviral prophylaxis of smallpox. J. Antimicrob. Chemother. 54:1-5. [PubMed]

4. Cangene Corporation. 2005. Vaccinia immune globulin intervenous human (VIGIV) package insert. Cangene Corporation, Winnipeg, Manitoba, Canada. http://www.fda.gov/BiologicsBloodVaccines/BloodBloodProducts/ApprovedProducts/LicensedProductsBLAS/FractionatedPlasmaProducts/ucm179513.htm.

5. Cosma, A., R. Nagaraj, C. Staib, C. Diemer, F. Wopfner, H. Schatzl, D. H. Busch, G. Sutter, F. D. Goebel, and V. Erfle. 2007. Evaluation of modified vaccinia virus Ankara as an alternative vaccine against smallpox in chronically HIV type 1-infected individuals undergoing HAART. AIDS Res. Hum Retrovir. 23:782-793. [PubMed]

6. Edghill-Smith, Y., H. Golding, J. Manischewitz, L. R. King, D. Scott, M. Bray, A. Nalca, J. W. Hooper, C. A. Whitehouse, J. E. Schmitz, K. A. Reimann, and G. Franchini. 2005. Smallpox vaccine-induced antibodies are necessary and sufficient for protection against monkeypox virus. Nat. Med. 11:740-747. [PubMed]

7. Gherardi, M. M., and M. Esteban. 1999. Mucosal and systemic immune responses induced after oral delivery of vaccinia virus recombinants. Vaccine 17:1074-1083. [PubMed]

8. Goldsmith, J. C., N. Eller, M. Mikolajczyk, J. Manischewitz, H. Golding, and D. E. Scott. 2004. Intravenous immunoglobulin products contain neutralizing antibodies to vaccinia. Vox Sang. 86:125-129. [PubMed]

9. Gomez, C. E., J. L. Najera, E. Domingo-Gil, L. Ochoa-Callejero, G. Gonzalez-Aseguinolaza, and M. Esteban. 2007. Virus distribution of the attenuated MVA and NYVAC poxvirus strains in mice. J. Gen. Virol. 88:2473-2478. [PubMed]

10. Gonen, M. 2007. Analyzing receiver operating characteristic curves using SAS. SAS Press, Cary, NC.

11. Greenberg, R. N., and J. S. Kennedy. 2008. ACAM2000: a newly licensed cell culture-based live vaccinia smallpox vaccine. Exp. Opin. Investig. Drugs 17:555-564. [PubMed]

12. Gurt, I., I. Abdalrahman, and E. Katz. 2006. Pathogenicity and immunogenicity in mice of vaccinia viruses mutated in the viral envelope proteins A33R and B5R. Antivir. Res. 69:158-164. [PubMed]

13. Hammarlund, E., M. W. Lewis, S. G. Hansen, L. I. Strelow, J. A. Nelson, G. J. Sexton, J. M. Hanifin, and M. K. Slifka. 2003. Duration of antiviral immunity after smallpox vaccination. Nat. Med. 9:1131-1137. [PubMed]

14. Harrison, S. C., B. Alberts, E. Ehrenfeld, L. Enquist, H. Fineberg, S. L. McKnight, B. Moss, M. O'Donnell, H. Ploegh, S. L. Schmid, K. P. Walter, and J. Theriot. 2004. Discovery of antivirals against smallpox. Proc. Natl. Acad. Sci. USA 101:11178-11192. [PubMed]

15. Hooper, J. W., E. Thompson, C. Wilhelmsen, M. Zimmerman, M. A. Ichou, S. E. Steffen, C. S. Schmaljohn, A. L. Schmaljohn, and P. B. Jahrling. 2004. Smallpox DNA vaccine protects nonhuman primates against lethal monkeypox. J. Virol. 78:4433-4443. [PMC free article] [PubMed]

16. Hutchens, M., and G. D. Luker. 2007. Applications of bioluminescence imaging to the study of infectious diseases. Cell Microbiol. 9:2315-2322. [PubMed]

17. Koszinowski, U., and R. Thomssen. 1975. Target cell-dependent T cell-mediated lysis of vaccinia virus-infected cells. Eur. J. Immunol. 5:245-251. [PubMed]

18. Luker, K. E., M. Hutchens, T. Schultz, A. Pekosz, and G. D. Luker. 2005. Bioluminescence imaging of vaccinia virus: effects of interferon on viral replication and spread. Virology 341:284-300. [PubMed]

19. Luker, K. E., and G. D. Luker. 2008. Applications of bioluminescence imaging to antiviral research and therapy: multiple luciferase enzymes and quantitation. Antivir. Res. 78:179-187. [PMC free article] [PubMed]

20. Meseda, C. A., A. D. Garcia, A. Kumar, A. E. Mayer, J. Manischewitz, L. R. King, H. Golding, M. Merchlinsky, and J. P. Weir. 2005. Enhanced immunogenicity and protective effect conferred by vaccination with combinations of modified vaccinia virus Ankara and licensed smallpox vaccine Dryvax in a mouse model. Virology 339:164-175. [PubMed]

21. Panchanathan, V., G. Chaudhri, and G. Karupiah. 2007. Correlates of protective immunity in poxvirus infection: where does antibody stand? Immunol. Cell Biol. 86:80-86. [PubMed]

22. Parrino, J., and B. S. Graham. 2006. Smallpox vaccines: past, present, and future. J. Allergy Clin. Immunol. 118:1320-1326. [PubMed]

23. Ramirez, J. C., M. M. Gherardi, and M. Esteban. 2000. Biology of attenuated modified vaccinia virus Ankara recombinant vector in mice: virus fate and activation of B- and T-cell immune responses in comparison with the Western Reserve strain and advantages as a vaccine. J. Virol. 74:923-933. [PMC free article] [PubMed]

24. Rodriguez, J. F., D. Rodriguez, J. R. Rodriguez, E. B. McGowan, and M. Esteban. 1988. Expression of the firefly luciferase gene in vaccinia virus: a highly sensitive gene marker to follow virus dissemination in tissues of infected animals. Proc. Natl. Acad. Sci. USA 85:1667-1671. [PubMed]

25. Townsley, A. C., A. S. Weisberg, T. R. Wagenaar, and B. Moss. 2006. Vaccinia virus entry into cells via a low-pH-dependent endosomal pathway. J. Virol. 80:8899-8908. [PMC free article] [PubMed]

26. Williamson, J. D., R. W. Reith, L. J. Jeffrey, J. R. Arrand, and M. Mackett. 1990. Biological characterization of recombinant vaccinia viruses in mice infected by the respiratory route. J. Gen. Virol. 71:2761-2767. [PubMed]

27. Wittek, R. 2006. Vaccinia immune globulin: current policies, preparedness, and product safety and efficacy. Int. J. Infect. Dis. 10:193-201. [PubMed]

Articles from Journal of Virology are provided here courtesy of
American Society for Microbiology (ASM)