Home | About | Journals | Submit | Contact Us | Français |

**|**HHS Author Manuscripts**|**PMC2860876

Formats

Article sections

- Abstract
- 1. Introduction
- 2. Mathematical Background
- 3. Methods
- 4. Results
- 5. Conclusions and Discussions
- References

Authors

Related links

Phys Med Biol. Author manuscript; available in PMC 2010 April 28.

Published in final edited form as:

Published online 2009 April 8. doi: 10.1088/0031-9155/54/9/004

PMCID: PMC2860876

NIHMSID: NIHMS196005

Gene Gindi: ude.bsynus.lim@idnig

The publisher's final edited version of this article is available at Phys Med Biol

See other articles in PMC that cite the published article.

For the medically relevant task of joint detection and localization of a signal (lesion) in an emission computed tomographic (ECT) images, it is of interest to measure the efficiency, defined as the relative task performance of a human observer vs that of an ideal observer. Low efficiency implies that improvements in reconstruction algorithms may be possible. Calculation of ideal observer performance for ECT is highly computationally complex. We can, however, compute ideal observer performance exactly using a simplified “filtered-noise” model of ECT. This model results in images whose correlation structure, due to quantum noise, background variability and regularization, is similar to that of real ECT reconstructed images. A two-alternative forced choice test is used to obtain the performance of the human observers. We compare the efficiency of our joint detection-localization task with that of a correponding signal-known-exactly (SKE) detection task. For the joint task, efficiency is low when the search tolerance is stringent. Efficiency for the joint task rises with signal intensity but is flat for the SKE task. For both tasks, efficiency peaks at a midrange level of regularization corresponding to a particular noise-resolution tradeoff.

A principled approach to assess image quality in emission computed tomography (ECT) is to use a scalar task-based figure of merit (FOM) (Barrett and Myers 2004). These FOMs can be used to compare or optimize imaging systems or reconstruction algorithms. A commonly used task is the detection of a signal (a lesion) in a noisy complex image. In ECT, Poisson (photon) noise is present in the sinogram, but an important additional source of noise is the statistical variation in the spatial distribution of uptake of radionuclide due to anatomical and other effects in the underlying object (Barrett and Myers 2004). We shall term this form of noise background variability (BV).

Task FOMs can be evaluated using human or mathematical model observers. Model observers can be designed to emulate human performance (Barrett *et al.* 1993), but a different sort of model observer, the Bayesian ideal observer is used here. These ideal observers are mathematical (model) observers that deliver the best possible performance for the task at hand. They have knowledge of all relevant probability densities related to BV, photon noise, and any other sources of uncertainty. For example, for a signal present/absent detection problem, the Bayesian ideal observer maximizes a common FOM, the area under the receiver operating characteristic (ROC) curve, AROC (Barrett and Myers 2004). A human observer is not ideal, and its performance will be less than that of the ideal observer. A measure of their relative performance is called efficiency. A plethora of studies, for example (Tanner and Birdsall 1958, Burgess *et al.* 1981, 1982, 1997, Park *et al.* 2005, 2007), have explored human efficiency for different tasks and different forms of BV. If the ideal observer performance is much greater than that of the human, there is an implication that further processing of the image might well be used to improve human performance (Rolland *et al.* 1991, Abbey *et al.* 2006). Thus it is of interest to obtain the knowledge of human efficiency relative to the ideal observer.

So far we have focused on detection tasks, but in this paper we are interested in the medically more realistic task of joint detection and localization of a signal in a noisy ECT image. This differs from the more conventional pure detection task of a known signal at a known location embedded in a noisy ECT image. Indeed we shall compare efficiency for these two tasks. Recently we have formulated an ideal observer for this joint detection-localization task (Khurd and Gindi 2005). This observer is optimal in that it maximizes the area under the localization ROC (LROC) curve (Swensson 1996).

We shall make use of the following acronyms: the “SKS (signal-known-statistically) task” shall specifically refer to the joint detection and localization of a signal in a noisy background. Here the signal is known exactly except for location. The “SKE (signal-known-exactly) task” shall specifically refer to the detection of a known signal in a noisy background. Here, the signal form and location are known exactly. We note that both the SKS and SKE tasks differ from a multiple-alternative forced choice (MAFC) task in which an observer is presented with an image containing one signal of known form but unknown location. The MAFC observer knows that the signal will be in any one of *M* precisely specified locations and his task is to try to correctly identify the signal location.

We also mention a different approach to image quality evaluation as espoused in the scan-statistics literature (Swensson 1996, Popescu and Lewitt 2006). This approach typically uses detection or joint detection and localization FOMs. A review of the scan-statistics methodology and an excellent resource of scan-statistics (and related) methods in imaging can be found in (Popescu and Lewitt 2006). In this approach, one eschews the often hard-to-obtain knowledge of probability densities related to any sources of uncertainty needed by ideal observers. Instead one takes a more practical approach, directly applying empirical (non-ideal) observers to samples of images to obtain histogram estimates of probabilities that can then be used to derive image-quality FOMs. In (Popescu and Lewitt 2006), PET versus time-of-flight PET are compared under a task context of detection of a signal at an unknown location.

We are interested in ECT, but the calculation of ideal observer performance for realistic SPECT or PET systems is highly computationally complex. To address this computational problem, we make a major simplification by using a “filtered-noise” model proposed in (Abbey and Barrett 2001). In (Abbey and Barrett 2001), the reconstructed ECT image is modelled as a stationary noise field with a possible added signal. The correlation structure of the noise field is designed to approximate the effects of: (1) propagating photon noise from a sinogram into the reconstruction via an FBP algorithm (Riederer *et al.* 1978); (2) propagating a form of BV (Rolland and Barrett 1992) into the reconstruction and (3) capturing the effects of any smoothing operations on the reconstruction. This model was used in (Abbey and Barrett 2001) not for efficiency studies, but to examine the effects of regularization, BV and photon noise on the performance of both human and human-emulating (non-ideal) model observers. These studies were also used to improve the predictive power of human-emulating observers.

The rest of the paper is organized as follows: In Section 2, we introduce the mathematical background. Experimental methods are presented in Section 3 and experimental results are given in Section 4. We conclude in Section 5 with a conclusion and discussion.

Scalars and scalar-valued functions are denoted by unbolded letters. Vectors are indicated by bolded lowercase letters. Although we use 2-D images, we use lexicographically ordered vectors to represent images, with each element containing the intensity of an image pixel. Bolded uppercase letters are used to represent matrices.

We now focus on the mathematical description of the SKS task. A test image **y** is either signal-absent or includes a signal that is present at one of *J* locations, and the objective of an observer is to determine which class the test image **y** belongs to. This task is a (*J* + 1)-class classification problem (also known as multiple hypothesis-testing problem) with classes (hypotheses) expressed as:

$$\begin{array}{l}{H}_{0}:\mathbf{y}=\mathbf{b}+\mathbf{n},\\ {H}_{j}:\mathbf{y}={\mathbf{s}}_{j}+\mathbf{b}+\mathbf{n},\phantom{\rule{0.38889em}{0ex}}(j=1,2,\cdots J),\end{array}$$

(1)

where **n** is an *M* × 1 vector of noise with mean zero. For class *H*_{0}, the test image **y** comprises a *M* × 1 background image **b** plus zero-mean noise **n**, but has no signal. For class *H _{j}*,

We assume that an observer performs the SKE task by computing a scalar test statistic *t* based on **y**. The scalar test statistic is compared to a threshold *τ* to decide signal-present if *t*(**y**) ≥ *τ* and signal absent if *t*(**y**) *< τ*. For each threshold *τ*, one calculates *P _{TP}* (

In order to introduce location uncertainty into the detection problem, we consider the localization ROC (LROC) curve (Swensson 1996). In addition to reporting whether or not the image contains a signal, the observer also needs to localize the signal within a search tolerance in a signal-present image. As shown in Figure 1, the LROC curve plots the probability of correct joint detection and localization *P _{CL}*(

This ideal observer (Khurd and Gindi 2005) searches for the signal in the test image **y** and reports whether the signal is present, and if so, reports one of *L* candidate locations. We decide the signal is correctly localized if the reported location is reasonably close to the true signal location, i.e., within a circular tolerance region surrounding the true location.

Let *P _{j}, j* = 0

$$\begin{array}{l}{t}_{\mathit{SKS}}(\mathbf{y})=\underset{l\in \{1,\cdots ,L\}}{max}\sum _{j\in T(l)}{P}_{j}\phantom{\rule{0.16667em}{0ex}}LR(\mathbf{y},{H}_{j})\\ l(\mathbf{y})=arg\underset{l\in \{1,\cdots ,L\}}{max}\sum _{j\in T(l)}{P}_{j}\phantom{\rule{0.16667em}{0ex}}LR(\mathbf{y},{H}_{j})\\ \text{Report}\phantom{\rule{0.16667em}{0ex}}\text{signal}\phantom{\rule{0.16667em}{0ex}}\text{in}\phantom{\rule{0.16667em}{0ex}}T(l(\mathbf{y}))\phantom{\rule{0.16667em}{0ex}}\text{if}\phantom{\rule{0.16667em}{0ex}}{t}_{\mathit{SKS}}(\mathbf{y})>\tau ,\phantom{\rule{0.16667em}{0ex}}\text{else}\phantom{\rule{0.16667em}{0ex}}\text{decide}\phantom{\rule{0.16667em}{0ex}}{H}_{0}\end{array}$$

(2)

where the likelihood ratio LR is defined as

$$LR(\mathbf{y},{H}_{j})\equiv \frac{p(\mathbf{y}\mid {H}_{j})}{p(\mathbf{y}\mid {H}_{0})},\phantom{\rule{0.16667em}{0ex}}j=1,\cdots ,J$$

(3)

For the SKE task, we are given a signal with *known* location *j*. The ideal observer *t _{SKE}* = LR(

We use a two-alternative forced-choice (2AFC) test to evaluate the performance of a human observer. In a 2AFC test appropriate for the SKS task, an observer is shown many pairs of test images. Each pair, as seen in Figure 2, comprises a signal-absent image **y** and a signal-present image **y**′. The observer is then forced to choose which image contains the signal and report the location of the signal. We assume the observer forms two internal responses (test statistics), *λ*^{+} = *λ* (**y**′) and *λ*^{−} = *λ*(**y**), and chooses the image with larger response as the signal-present image. A candidate signal location *l* is also reported by the observer. Therefore, the observer correctly detects the signal-present image if *λ*^{+} ≥ *λ*^{−} and correctly localizes the signal if the true signal location is within *T*(*l*). The proportion of correctly detected and localized images, *P _{C}*, is computed after showing many pairs of images to the human observer. It can be shown that, for a human observer experiment, ALROC equals to

Consider a general ECT system where the sinogram **g** is given by

$$\mathbf{g}=\mathcal{H}\mathbf{f}+{\mathbf{n}}_{P}$$

(4)

where **n*** _{P}* is the Poisson noise, and is a system matrix, which for this case would be a digital version of the Radon transform. The object

$$\widehat{\mathbf{f}}=\mathrm{\Theta}\mathbf{g}=\mathrm{\Theta}(\mathcal{H}\mathbf{f}+{\mathbf{n}}_{P})$$

where we assume the reconstruction operator Θ is an FBP operator that may include regularization.

Take the object
$\mathbf{f}={\mathbf{s}}_{j}^{\prime}+{\mathbf{b}}^{\prime}+{\mathbf{n}}_{b}$, where
${\mathbf{s}}_{j}^{\prime}$ is the signal present in the object domain (
${\mathbf{s}}_{j}^{\prime}=0$ for a signal-absent object), **b**′ is the *deterministic* part of the background in the object domain and **n*** _{b}* is zero mean BV in the object. Then the reconstructed image can be written as

$$\widehat{\mathbf{f}}=(\mathrm{\Theta}\mathcal{H}{\mathbf{s}}_{j}^{\prime}+\mathrm{\Theta}\mathcal{H}{\mathbf{b}}^{\prime})+(\mathrm{\Theta}\mathcal{H}{\mathbf{n}}_{b}+\mathrm{\Theta}{\mathbf{n}}_{P})$$

(5)

where the first two terms are deterministic and the latter two terms random. The first term,
$\mathrm{\Theta}\mathcal{H}{\mathbf{s}}_{j}^{\prime}$, is the noiseless reconstructed signal and the second term, Θ**b**′, is the noiseless reconstructed deterministic background. The third term, Θ**n*** _{b}*, is the object background variability propagated into the reconstruction, and the fourth term, Θ

The ECT model (5) can be approximated by the filtered-noise model introduced in (Abbey and Barrett 2001). This noise model is used to emulate the tomographic reconstruction process wherein the correlation structure of the reconstructed images is a combination of propagated quantum noise and BV modulated by some form of regularization. By “emulate”, we mean that no actual reconstruction is done; instead, a noisy image is generated whose noise structure is designed to be that of a reconstruction.

We now describe the process of generating filtered noise. The filtered noise **n*** _{f}* can be generated by the process

$$\mathbf{\Lambda}\equiv \mathbf{B}{({\mathbf{N}}_{q}+{\mathbf{N}}_{a})}^{1/2}.$$

Here the background noise-power spectrum (NPS) is denoted by **N*** _{a}* which represents the fluctuations in the reconstructed images due to BV. The quantum NPS,

$${\mathbf{s}}_{f}^{j}={\mathbf{F}}^{H}\mathbf{BF}{\mathbf{s}}_{j}^{\prime},\phantom{\rule{0.38889em}{0ex}}{\mathbf{b}}_{f}={\mathbf{F}}^{H}\mathbf{BF}{\mathbf{b}}^{\prime},$$

also include the effects of regularization.

Given the filtered noise model, we can rewrite (1) as

$$\begin{array}{l}{H}_{0}:\mathbf{y}={\mathbf{b}}_{f}+{\mathbf{n}}_{f},\\ {H}_{j}:\mathbf{y}={\mathbf{s}}_{f}^{j}+{\mathbf{b}}_{f}+{\mathbf{n}}_{f},\phantom{\rule{0.16667em}{0ex}}j=1,2,\cdots J.\end{array}$$

(6)

and since **n*** _{f}* is multi-variate Gaussian, the corresponding likelihood functions can be written as

$$\begin{array}{l}p(\mathbf{y}\mid {H}_{0})=\frac{1}{{(2\pi )}^{M/2}\mid {\mathbf{K}}_{{\mathbf{n}}_{f}}{\mid}^{1/2}}exp\left(-\frac{1}{2}{(\mathbf{y}-{\mathbf{b}}_{f})}^{T}{\mathbf{K}}_{{\mathbf{n}}_{f}}^{-1}(\mathbf{y}-{\mathbf{b}}_{f})\right),\\ p(\mathbf{y}\mid {H}_{j})=\frac{1}{{(2\pi )}^{M/2}\mid {\mathbf{K}}_{{\mathbf{n}}_{f}}{\mid}^{1/2}}exp\left(-\frac{1}{2}{(\mathbf{y}-{\mathbf{b}}_{f}-{\mathbf{s}}_{f}^{j})}^{T}{\mathbf{K}}_{{\mathbf{n}}_{f}}^{-1}(\mathbf{y}-{\mathbf{b}}_{f}-{\mathbf{s}}_{f}^{j})\right),\end{array}$$

(7)

where **K**_{nf} is the covariance matrix of the filtered noise **n*** _{f}* and is derived as

$${\mathbf{K}}_{{\mathbf{n}}_{f}}={\mathbf{F}}^{H}\mathbf{\Lambda}\mathbf{F}{({\mathbf{F}}^{H}\mathbf{\Lambda}\mathbf{F})}^{H}={\mathbf{F}}^{H}{\mathbf{B}}^{2}({\mathbf{N}}_{a}+{\mathbf{N}}_{q})\mathbf{F}.$$

(8)

For the components in the discrete transfer function **Λ**, we use the same functional forms as in (Abbey and Barrett 2001). For fluctuations due to background variability, they recommend the use of inverse power law noise, whose isotropic NPS, **N*** _{a}*, is expressed as

$${[{\mathbf{N}}_{a}]}_{kk}=\frac{{W}_{a}}{1+{\left({\scriptstyle \frac{{\rho}_{k}}{{\rho}_{a}}}\right)}^{\beta}}$$

where [**X**]* _{kk}* denotes the

$${[{\mathbf{N}}_{q}]}_{kk}=\{\begin{array}{ll}{W}_{q}{\rho}_{k}\hfill & \text{if}\phantom{\rule{0.16667em}{0ex}}{\rho}_{k}\ge {\rho}_{q},\hfill \\ {W}_{q}{\rho}_{q}\hfill & \text{if}\phantom{\rule{0.16667em}{0ex}}{\rho}_{k}<{\rho}_{q}\hfill \end{array},$$

where *ρ _{q}* is a constant that imposes a small DC component near the origin for normalization purposes, and

$${[\mathbf{B}]}_{kk}=\frac{1}{1+{\left({\scriptstyle \frac{{\rho}_{k}}{{\rho}_{c}}}\right)}^{2\nu}}$$

where *ρ _{c}* is the cutoff point of the filter and the order

For the SKS task, without loss of generality, we assume the signal is uniformly distributed at all possible locations. We use a circle as the tolerance region and represent tolerance by its radius *r _{tol}*. The signal
${\mathbf{s}}_{j}^{\prime}$ is a Gaussian blob of peak intensity

To calculate ALROC in a simulation experiment, we generate *N*^{+} = 5000 signal-present images and *N*^{−} = 4000 signal-absent images. Unlike the case for real ECT, for the filtered-noise model, the likelihood ratio *LR _{f}* (

$${LR}_{f}(\mathbf{y},{H}_{j})=exp\left({(\mathbf{y}-{\mathbf{b}}_{f})}^{T}{\mathbf{K}}_{{\mathbf{n}}_{f}}^{-1}{\mathbf{s}}_{f}^{j}-\frac{1}{2}{({\mathbf{s}}_{f}^{j})}^{T}{\mathbf{K}}_{{\mathbf{n}}_{f}}^{-1}{\mathbf{s}}_{f}^{j}\right).$$

(9)

Using (2) and (9), we compute the corresponding observer responses *t*^{+} for each signal-present image and *t*^{−} for each signal-absent image and we also obtain the observer-reported location *l* for each signal-present image. If the reported location for one signal-present image leads to an incorrect localization, i.e., the actual signal location *j* *T*(*l*), we simply discard the corresponding *t*^{+}. We then determine a set of *N _{τ}* thresholds from the range of the values of

$$\begin{array}{c}{P}_{CL}(\tau )=\frac{\text{Number}\phantom{\rule{0.16667em}{0ex}}\text{of}\phantom{\rule{0.16667em}{0ex}}\text{correctly}\phantom{\rule{0.16667em}{0ex}}\text{localized}\phantom{\rule{0.16667em}{0ex}}{t}^{+}\phantom{\rule{0.16667em}{0ex}}\text{that}\phantom{\rule{0.16667em}{0ex}}\text{exceeds}\phantom{\rule{0.16667em}{0ex}}\tau}{{N}^{+}}\\ {P}_{FP}(\tau )=\frac{\text{Number}\phantom{\rule{0.16667em}{0ex}}\text{of}\phantom{\rule{0.16667em}{0ex}}{t}^{-}\phantom{\rule{0.16667em}{0ex}}\text{that}\phantom{\rule{0.16667em}{0ex}}\text{exceeds}\phantom{\rule{0.16667em}{0ex}}\tau}{{N}^{-}}\end{array}$$

Note that only those *t*^{+} with correct localization are counted in computing *P _{CL}*(

As seen in Figure 2, each observer in a 2AFC test for an SKS task is shown a pair of test images on a Sony Multiscan200ES CRT monitor. The 128 × 128 test image is magnified to 256 × 256 pixels using bilinear interpolation so that the image subtends a reasonable solid angle as viewed by the human observer. The human is free to adjust her viewing position. The image containing the signal is randomly determined to be either on the left or the right side with equal probability. A signal located at the center of an empty background is also shown to the observer. The observer is forced to choose with a mouse-click which of the two images contains the signal and where in that image the signal is located. The observer is presented with 100 image pairs for training and 300 pairs for testing. In the training session, the correctness of the observer’s answer is immediately reported. If the observer chooses the correct signal-present image and the distance between the true signal location (indicated by a “+” in Figure 2) and the observer reported-location (indicated by a “x” in Figure 2) is shorter than the radius of tolerance region, then the signal is deemed to be correctly detected and localized. In the case in Figure 2, this pair of images is counted as one correct detection and localization. In the testing session, no feedback is provided to the observer except the final report of *P _{C}*. While Figure 2 illustrates the 2AFC procedure, Figure 3 better illustrates the qualitative nature of the images viewed by the human observers.

Sample signal-present test images with different cutoff frequencies. The signal is pointed out by an arrow. From left to right: *ρ*_{c} = 0.06, 0.11, 0.165, 0.30, 0.45. These figures are typical of those seen by observers. In the first, third and fifth **...**

For the 2AFC test for the SKE case, the training and testing procedures are similar to that of the SKS case. The only difference is that the signal location is pointed out by a crosshair. The observer is forced only to choose which side he thinks the signal is on. For all SKE and SKS human experiments, performance was averaged over four observers. Error bars in performance represent 68% confidence intervals.

A photometer from Quantum Instruments Inc. model PMLX was used to calibrate the monitor. For each grey level [0–255], the photometer reading of luminance in (*cd/m*^{2}) was recorded. The plot of luminance vs. grey value was nearly flat from [0–50] and monotonically rising (approximately quadratically) from [50–255]. The histogram of grey values of all displayed images fit within this monotonic region.

A commonly used definition of efficiency for the SKE case is the squared ratio of the detectability index of the human observer, *d _{h}*, and the ideal observer,

We introduce a second definition of human efficiency which is simply the ratio of the AUC (area under curve) of the human observer and the ideal observer, i.e.,

$${e}_{2}=\frac{{\mathit{AUC}}_{\mathit{human}}}{{\mathit{AUC}}_{\mathit{ideal}}},$$

(10)

where AUC equals AROC or ALROC as appropriate.

Several experiments were implemented to compare the SKE and SKS performance of ideal and human observers for this filtered-noise ECT model. It is of interest (Park *et al.* 2005) to perform psychometric studies to see how efficiency varies with signal and background quantities, and in this study we have focused on the variation of efficiency with two important variables: signal amplitude and level of regularization. The two parameters varied were *a _{G}* to control signal intensity, and cutoff frequency

Effects of regularization on noise in the frequency domain: (a) Noise power spectrum before regularization: the spectrum comprises the background and quantum NPS; (b) Discrete transfer function Λ with different cutoff frequencies *ρ*_{c}.

Effects of *ρ*_{c} on signal spectrum: (a) Signal spectrum (before regularization); (b) Signal spectrum with different cutoff frequencies (after regularization). Note that the curves for *ρ*_{c} = 0.30 and *ρ*_{c} = 0.45 overlap.

One of our goals is to compare SKE and SKS performance. However, the results for SKS experiments will vary as the tolerance radius *r _{tol}* changes. For example, with

Experimental results investigating tolerance effects. We use a Gaussian blob signal with *a*_{G} = 65 and regularization parameter *ρ*_{c} = 0.165. (a) ALROC versus radius of tolerance region for both the ideal and human observers; (b) Efficiency of the **...**

With the tolerance issue addressed, we can now compare SKE and SKS performance for the two psychometric experiments. In Figure 7(a), we vary the signal intensity *a _{G}* from 45 to 85 and plot the SKS performance curves for both ideal and human observers. The relative efficiency curves are plotted in Figure 7(b). As expected, the performances of both ideal and human observers increase monotonically as the signal intensity increases. However, the efficiency curves have a slight dip at an intermediate intensity.

Experimental results with *ρ*_{c} = 0.165 for varous signal intensities and for SKE and SKS tasks. (a) Ideal and human observer performance; (b) Human efficiency relative to the ideal observer.

The corresponding SKE results are also shown in Figure 7, where the intensity range is lower in order to get ideal observer performance equivalent to that of the SKS case (ALROC for the highest 3 intensities matches AROC for intensities *a _{G}* = 11, 24, 37). As seen in Figure 7(a), we again observe the monotonic (with intensity) trend in performance. However, in Figure 7(b) we observe relatively flat efficiency curves in the selected intensity region. The efficiency curve for SKS rises over its intensity range since the human is better able (relative to the ideal observer) to take advantage of the increased localization ability afforded by brighter signals. For the corresponding SKE intensity range, no such rise is seen because no localization is involved in the SKE task.

To study the effects of regularization on performance, we vary the cutoff frequency *ρ _{c}* of the Butterworth filter. For the SKS case, as shown in Figure 8(a), the cutoff frequency has no effect on the performance of the ideal observer (the slight variation in the ideal observer curve is due to finite sample effects). Indeed, it is readily shown mathematically that ALROC does not vary with

Experimental results for varous cutoff frequencies and for both SKS and SKE tasks: (a) Ideal and human observer performance; (b) Human efficiency relative to the ideal observer for the SKS case; (c) Human efficiency relative to the ideal observer for **...**

The corresponding SKE results are also shown in Figure 8. The range of cutoff frequencies is the same as that of the SKS case and again, the ideal observer performance is independent of *ρ _{c}*. The signal intensity

Examination of Figures 8(a) and 8(b) shows that SKE efficiency apparently exceeds that of SKS for either definition of efficiency. However, comparison of efficiencies for these two cases is not meaningful due to the different nature of the SKE and SKS tasks.

We have, for the first time, calculated the efficiency of a human observer relative to the ideal observer for a joint detection and localization task. The effects of task, tolerance radius, signal intensity and cutoff frequency on human efficiency were analyzed in the context of a simplified filtered-noise model for ECT. For the SKS task, results showed that both the ideal and human observers performed poorly in localizing the precise signal center but performed well in localizing the rough location of the signal center. Psychometric tests with varying signal intensity showed for the SKS task a rising trend of efficiency with intensity. For the SKE task, this trend was approximately flat. Psychometric tests with varying cutoff frequency showed a similar trend for both tasks: efficiency peaked at a mid-range cutoff corresponding to a particular noise-resolution tradeoff.

While there is an extensive literature on human efficiency in visual tasks, the most relevant to our work is that in (Park *et al.* 2005, 2007). They studied human efficiency for a planar pinhole emission imaging system that delivered images containing a form of “lumpy bakcground” BV, uncorrelated photon noise, and smoothing (regularization) due to the finite size of the pinhole. Ideal observer performance and human 2AFC tests were done for an SKE detection task and an SKS task in which the observer was required to only detect - but not localize - the signal. For the psychometric intensity tests in (Park *et al.* 2005), performance for the SKS-detection-only task was better than that for SKE.

It is also relevant to mention the work of Gifford *et al* (2003, 2005) who addressed the joint detection-localization task for SPECT. However, they did not consider ideal observers and instead focused on model observers designed to emulate the performance of humans.

We have addressed task performance in ECT, but used the filtered-noise model approximation to allow exact calculation of ideal observer performance. Clearly this study needs to be extended toward more realistic ECT. If BV is excluded, then we can indeed calculate ideal observer performance exactly for realistic SPECT (Liu *et al.* 2008). If we include Gaussian BV, we can calculate approximate ideal observer performance for realistic SPECT (Zhou *et al.* 2008). In future work, we shall address computational problems in calculating good approximations for ideal observer performance for more realistic ECT models.

This work was supported by NIH NIBIB 02629.

The probability of correct detection and localization in a 2AFC test can be written as *P _{C}* =

$$\begin{array}{l}{P}_{C}=P({\lambda}^{+}\ge {\lambda}^{-})P(\text{correct}\phantom{\rule{0.16667em}{0ex}}\text{localization})\\ =\sum _{j=1}^{J}{P}_{j}{\int}_{-\infty}^{\infty}{\int}_{-\infty}^{\infty}P(j\in T(l)\mid {H}_{j})\text{step}({\lambda}^{+}-{\lambda}^{-})p({\lambda}^{+}\mid {H}_{j})p({\lambda}^{-})\text{d}{\lambda}^{-}\text{d}{\lambda}^{+}\end{array}$$

(A.1)

where

$$\text{step}(x)=\{\begin{array}{ll}1,\hfill & x\ge 0\hfill \\ 0,\hfill & \text{otherwise}\hfill \end{array}$$

Since both *λ*^{+} and *λ*^{−} are random variables through their dependence on the test images **y**′ and **y** respectively, we have

$$\begin{array}{l}p({\lambda}^{+}\mid {H}_{j})={\int}_{{\mathbf{y}}^{\prime}}p({\mathbf{y}}^{\prime}\mid {H}_{j})\delta ({\lambda}^{+}-\lambda ({\mathbf{y}}^{\prime}))\text{d}{\mathbf{y}}^{\prime},j=1,\cdots ,J\\ p({\lambda}^{-})={\int}_{\mathbf{y}}p(\mathbf{y}\mid {H}_{0})\delta ({\lambda}^{-}-\lambda (\mathbf{y}))\text{d}\mathbf{y}\end{array}$$

and (A.1) becomes

$${P}_{C}=\sum _{j=1}^{J}{P}_{j}{\int}_{-\infty}^{\infty}\text{d}{\lambda}^{+}{\int}_{-\infty}^{\infty}\text{d}{\lambda}^{-}{\int}_{{\mathbf{y}}^{\prime}}{\int}_{\mathbf{y}}P(j\in T(l({\mathbf{y}}^{\prime}))\mid {H}_{j})\text{step}({\lambda}^{+}-{\lambda}^{-})\times p({\mathbf{y}}^{\prime}\mid {H}_{j})\delta ({\lambda}^{+}-\lambda ({\mathbf{y}}^{\prime}))p(\mathbf{y}\mid {H}_{0})\delta ({\lambda}^{-}-\lambda (\mathbf{y}))\text{d}{\mathbf{y}}^{\prime}\text{d}\mathbf{y}$$

(A.2)

Taking the integrals over *λ*^{+} and *λ*^{−}, we have

$${P}_{C}=\sum _{j=1}^{J}{P}_{j}{\int}_{\mathbf{y}}{\int}_{{\mathbf{y}}^{\prime}}p(\mathbf{y}\mid {H}_{0})p({\mathbf{y}}^{\prime}\mid {H}_{j})P(j\in T(l({\mathbf{y}}^{\prime}))\mid {H}_{j})\text{step}(\lambda ({\mathbf{y}}^{\prime})-\lambda (\mathbf{y}))\text{d}\mathbf{y}\text{d}{\mathbf{y}}^{\prime}$$

(A.3)

For any observer with test statistic *t*(**y**), the ALROC is given by

$$\mathit{ALROC}={\int}_{0}^{1}{P}_{CL}(\tau )\text{d}{P}_{FP}(\tau )$$

(A.4)

where

$${P}_{FP}(\tau )=P(t>\tau \mid {H}_{0})={\int}_{\tau}^{\infty}p(t\mid {H}_{0})\text{d}t.$$

(A.5)

Since *P _{FP}* (

$$\mathit{ALROC}=-{\int}_{\infty}^{-\infty}{P}_{CL}(\tau )\frac{\text{d}}{\text{d}\tau}{P}_{FP}(\tau )\text{d}\tau $$

(A.6)

where the minus sign and the change of integration limits are due to the fact that *P _{FP}* (

From (A.5), we have

$$\frac{\text{d}}{\text{d}\tau}{P}_{FP}(\tau )=-p(\tau \mid {H}_{0}),$$

and we can further rewrite ALROC in an alternative expression as follows,

$$\begin{array}{l}\mathit{ALROC}={\int}_{-\infty}^{\infty}{P}_{CL}(y)p(y\mid {H}_{0})\text{d}y\\ ={\int}_{-\infty}^{\infty}\sum _{j=1}^{J}{P}_{j}P(x>y\mid {H}_{j})P(j\in T(l)\mid {H}_{j})p(y\mid {H}_{0})\text{d}y\\ =\sum _{j=1}^{J}{P}_{j}{\int}_{-\infty}^{\infty}{\int}_{-\infty}^{\infty}p(x\mid {H}_{j})\text{step}(x-y)P(j\in T(l)\mid {H}_{j})p(y\mid {H}_{0})\text{d}x\text{d}y\end{array}$$

(A.7)

where we replace *τ* by a new variable *y* as *τ* will only be used to represent the threshold.

Again, both *x* and *y* are random variables through their dependence on the test images and we have

$$\begin{array}{l}p(x\mid {H}_{j})={\int}_{{\mathbf{y}}^{\prime}}p({\mathbf{y}}^{\prime}\mid {H}_{j})\delta (x-t({\mathbf{y}}^{\prime}))\text{d}{\mathbf{y}}^{\prime},j=1,\cdots ,J\\ p(y\mid {H}_{0})={\int}_{\mathbf{y}}p(\mathbf{y}\mid {H}_{0})\delta (y-t(\mathbf{y}))\text{d}\mathbf{y}\end{array}$$

(A.8)

From (A.7) and (A.8), the ALROC can be written as

$$\begin{array}{l}\mathit{ALROC}=\sum _{j=1}^{J}{P}_{j}{\int}_{-\infty}^{\infty}\text{d}x{\int}_{-\infty}^{\infty}\text{d}y{\int}_{{\mathbf{y}}^{\prime}}{\int}_{\mathbf{y}}P(j\in T(l({\mathbf{y}}^{\prime}))\mid {H}_{j})\text{step}(x-y)\times p({\mathbf{y}}^{\prime}\mid {H}_{j})\delta (x-t({\mathbf{y}}^{\prime}))p(\mathbf{y}\mid {H}_{0})\delta (y-t(\mathbf{y}))\text{d}{\mathbf{y}}^{\prime}\text{d}\mathbf{y}\\ =\sum _{j=1}^{J}{P}_{j}{\int}_{\mathbf{y}}{\int}_{{\mathbf{y}}^{\prime}}p(\mathbf{y}\mid {H}_{0})p({\mathbf{y}}^{\prime}\mid {H}_{j})P(j\in T(l({\mathbf{y}}^{\prime}))\mid {H}_{j})\text{step}(t({\mathbf{y}}^{\prime})-t(\mathbf{y}))\text{d}\mathbf{y}\text{d}{\mathbf{y}}^{\prime}\end{array}$$

which is the same as *P _{C}* if the test statistics

- Abbey CK, Barrett HH. Human- and model-observer performance in ramp-spectrum noise: effect of regularization and object variability. J Opt Soc Am A. 2001;18(3):473–488. [PMC free article] [PubMed]
- Abbey CK, Zemp RJ, Liu J, Lindfors KK, Insasa MF. Observer efficiency in discrimination tasks simulating malignant and benign breast lesions imaged with ultrasound. IEEE Trans Medical Imaging. 2006;25(2):198–209. [PMC free article] [PubMed]
- Barrett HH, Myers KJ. Foundations of Image Science. Wiley Interscience; 2004.
- Barrett HH, Yao J, Rolland JP, Myers KJ. Model observers for assessment of image quality. Proc. of the National Academy of Sciences; 1993. pp. 9758–9765. [PubMed]
- Burgess AE, Li X, Abbey CK. Visual signal detectability with two noise components: anomalous masking effects. J Opt Soc Am A. 1997;14(9):2420–2442. [PubMed]
- Burgess AE, Wagner RF, Jennings RJ. Statistical efficiency: a measure of human visual signal detection performance. J Appl Photogr Eng. 1982;8:76–78.
- Burgess AE, Wagner RF, Jennings RJ, Barlow HB. Efficiency of human visual signal discrimination. Science. 1981;214:93–94. [PubMed]
- Clarkson E. Estimation receiver operating characteristic curve and ideal observers for combined detection/estimation tasks. J Opt Soc Am A. 2007;24(12):91–98. [PMC free article] [PubMed]
- Gifford HC, King MA, Pretorius PH, Wells RG. A comparison of human and model observers in multislice LROC studies. IEEE Trans Medical Imaging. 2005;24(2):160–169. [PubMed]
- Gifford HC, Pretorius PH, King MA. Comparison of human- and model-observer LROC studies. Proc. SPIE; 2003. pp. 112–122.
- Khurd P, Gindi G. Decision strategies that maximize the area under the LROC curve. IEEE Trans Medical Imaging. 2005;24(12):1626–1636. [PubMed]
- Liu B, Zhou L, Kulkarni S, Gindi G. Effects of background complexity on LROC analysis of SPECT. Conf. Rec. IEEE Nucl. Sci. Symp. Med. Imaging; 2008. p. M06-479.
- Park S, Clarkson E, Kupinski MA, Barrett HH. Efficiency of the human observer detecting random signals in random backgrounds. J Opt Soc Am A. 2005;22(1):3–16. [PMC free article] [PubMed]
- Park S, Gallas BD, Badano A, Petrick NA, Myers KJ. Efficiency of the human observer for detecting a Gaussian signal at a known location in non-Gaussian distributed lumpy backgrounds. J Opt Soc Am A. 2007;24(4):911–921. [PubMed]
- Popescu LM, Lewitt RM. Small nodule detectability evaluation using a generalized scan statistic model. Phys Med Biol. 2006;51(23):6225–6244. [PubMed]
- Riederer SJ, Pelc NJ, Chesler DA. The noise power spectrum in computed X-ray tomography. Phys Med Biol. 1978;23(3):446–454. [PubMed]
- Rolland JP, Barrett HH. Effect of random background inhomogeneity on observer detection performance. J Opt Soc Am A. 1992;9(5):649–658. [PubMed]
- Rolland JP, Barrett HH, Seeley GW. Ideal versus human observer for long-tailed point spread functions: does deconvolution help? Phys Med Biol. 1991;36(8):1091–1109. [PubMed]
- Swensson RG. Unified measurement of observer performance in detecting and localizing target objects on images. Med Phys. 1996;23(10):1709–1725. [PubMed]
- Tanner WP, Birdsall TG. Definitions of d′ and
*η*as psychophysical measures. J Acoust Soc Am. 1958;30(10):922–928. - Zhou L, Gindi G. Collimator optimization in SPECT based on an LROC ideal observer. Conf. Rec. IEEE Nucl. Sci. Symp. Med. Imaging; 2008. p. M15-2.

PubMed Central Canada is a service of the Canadian Institutes of Health Research (CIHR) working in partnership with the National Research Council's national science library in cooperation with the National Center for Biotechnology Information at the U.S. National Library of Medicine(NCBI/NLM). It includes content provided to the PubMed Central International archive by participating publishers. |