|Home | About | Journals | Submit | Contact Us | Français|
A ‘virtual histology’ can be thought of as the ‘staining’ of a digital ultrasound image via image processing techniques in order to enhance the visualisation of differences in the echotexture of different types of tissues. Several candidate image-processing algorithms for virtual histology using ultrasound images of the bovine ovary were studied. The candidate algorithms were evaluated qualitatively for the ability to enhance the visual differences in intra-ovarian structures and quantitatively, using standard texture description features, for the ability to increase statistical differences in the echotexture of different ovarian tissues. Certain algorithms were found to create textures that were representative of ovarian micro-anatomical structures that one would observe in actual histology. Quantitative analysis using standard texture description features showed that our algorithms increased the statistical differences in the echotexture of stroma regions and corpus luteum regions. This work represents a first step toward both a general algorithm for the virtual histology of ultrasound images and understanding dynamic changes in form and function of the ovary at the microscopic level in a safe, repeatable and non-invasive way.
Interpreting ultrasonographic images of mammalian ovaries is challenging, even under the best imaging conditions. In routine scanning for research or clinical purposes, the three principal structures of interest are follicles, corpora lutea (CL) and stroma. Ovarian follicles are fluid-filled sacs that contain the female gamete (oocyte) in mammals. The CL is a temporary endocrine gland that forms from the follicular tissue after ovulation. Its primary function is to produce the steroid hormone progesterone, which is necessary for maintaining pregnancy. The interfaces between these physiological entities are of the most interest as we try to interpret their biological functions over time. The interfaces between the structures are often indistinct in ultrasound images. We hypothesise that ultrasound images may be digitally ‘stained’ using a new computer enhancement technology to produce a ‘virtual histology’ in which the subtle differences in echotexture resulting from different ovarian structures are made more apparent.
Algorithms based on an image-filtering technique called sticks filtering, and some novel variations of this technique, are proposed as an initial approach to the ‘virtual histology’ of ovaries directly from digital ultrasound images. Six candidate ‘staining’ algorithms were evaluated qualitatively and quantitatively for their ability to alter the echotexture of different structures to aid in the discrimination between the physiological follicles and luteal structures and sustentacular tissues of the ovarian stroma. The techniques are demonstrated using excised bovine ovaries imaged in an in vitro laboratory setting.
To put this work into context, the topics of boundary enhancement for ultrasound images and the concept of a virtual histology are reviewed.
Ultrasonographic images are speckle images. That is, the image is formed by a noisy pattern, which, at a small scale, is speckled. Existing boundary enhancement techniques focus on suppressing speckle because speckle obfuscates edges. Despeckling techniques include median filtering (Ritenour et al. 1984), Wiener filtering (Jain 1989; Bylund et al. 2003), adaptive weighted median filtering (Loupas et al. 1987), bilateral filtering (Tomasi and Manduchi 1998), anisotropic diffusion filters (Yu and Acton 2002) and wavelet soft thresholding techniques (Gupta et al. 2004). The current research differs from these methods in that the goal is to make echotextural differences in the image more apparent, rather than the suppression of echotexture. Paradoxically, the algorithms presented herein achieve this by making use of a speckle-reduction algorithm known as the sticks filter (Czerwinski et al. 1998), which also has the property of accentuating linear features in speckle images.
Line and boundary detection in speckle images can be thought of as the problem of determining whether a line of some orientation passes through each pixel. If n discrete orientations are considered, then this becomes a ‘maximum likelihood’ problem with n + 1 hypotheses: hypotheses h1 through hn of a line in one of the n orientations plus the hypothesis that there is no line at all (the null hypothesis).
Line and boundary detection as a maximum likelihood problem has been investigated for speckle images using sticks filters (Czerwinski et al. 1998). A very simple detection rule, a sticks filter bank, was shown to be nearly optimal when the speckle noise is uncorrelated (Czerwinski et al. 1998). The detection rule performs well when the parameters of the noise are not known (Czerwinski et al. 1998). Hence, sticks filtering may be expected to perform well on ultrasonographic images by both enhancing lines and smoothing speckle, thereby enhancing edges that are not visible in the original image. Image structure can be further enhanced by colouring pixels according to the orientation of stick that was the most likely hypothesis (Czerwinski et al. 1999). Sticks filtering has been used as an edge contrast enhancement step in algorithms developed for the detection of the boundary of prostates in ultrasound images (Pathak et al. 2000; Abolmaesumi and Sirouspour 2004).
The sticks filter was used as a basis for ‘staining’ algorithms because it suppresses small-scale speckle while enhancing linear structures in textures. The length and orientation of these linear features vary with the tissue imaged. The staining algorithms proposed further exploit these features to create textures with more varied appearance, but which are consistent with respect to the imaged tissue.
The idea of a ‘virtual histology’ is not new; however, the meaning of the term varies in the literature. This nomenclature has been used to refer to acquisition of histological images of tissue in vivo using endomicroscopy (Kiesslich and Neurathman 2004). It has also been used to refer to colour-enhanced intravascular ultrasonographic images created by examining the spectra of radiofrequency backscatter (Nair et al. 2002). Others (Inoue et al. 2003; Sakashita et al. 2003) use the term to refer to unprocessed images acquired using laser-scanning confocal microscopy.
The term ‘virtual histology’ is used herein to refer to the study of the micro-anatomy of tissue through computer processing of conventionally acquired ultrasonographic images. Virtual histology images are created by exploiting statistical differences in the echotexture of different tissues by enhancing those differences via image-processing techniques so that the resulting textures have a visibly more distinct appearance when compared with each other. One can intuitively think of this process as ‘staining’ the image to bring echotextural differences to the forefront. The primary advantage of virtual histology arises from the ability to study dynamic changes in form and function at the microscopic level in a safe, repeatable and non-invasive way.
In this context, a virtual histology is distinct from the idea of an image segmentation. In a segmentation, the image is divided into regions that are homogeneous with respect to some criterion; in medical imaging, the criterion is typically that each region contain only one kind of tissue.
The objective of the present study was to explore several candidate image-processing algorithms for virtual histology using ultrasound images of the bovine ovary. The candidate algorithms were evaluated qualitatively for the ability to enhance the visual differences in intra-ovarian structures and quantitatively, using standard texture description properties, for the ability to increase statistical differences in echotexture of different ovarian tissues.
The result of a sticks filter with stick length 15 applied to a 640 × 480 pixel ultrasonographic image of a bovine ovary is demonstrated in Fig. 1. The sticks filter not only reduces speckle, but also reveals some structure, real or artificial, within the echo-texture of the ovary. This structure may be exploited to create new textures that are representative of the imaged tissue. In the following sections, the definition of the sticks filter is reviewed and some novel variants are introduced.
A sticks filter bank, as defined by Czerwinski et al. (1998), consists of a set of 2n − 2 square binary valued linear filter masks of size n × n, each responding maximally to a length n line segment of different orientation. Each mask in a sticks filter bank represents the hypothesis of a line segment in a particular orientation in the maximum likelihood formulation of the line and boundary detection problem. A possible (because there are different ways to rasterise line segments) filter bank for sticks of length 5 is shown in Fig. 2.
The sticks filter bank is applied to an image as follows. For each pixel in the image, each of the 2n − 2 masks is superimposed on the image so that the centres of the masks are aligned with the current pixel of interest. The mean intensity of pixels along each hypothesised line is computed. The mask that results in the largest mean intensity is taken as the most likely hypothesis. The resulting maximal average is assigned to the corresponding pixel in the output image. The null hypothesis can be considered by requiring that the maximal average meet or exceed a predefined threshold, although this is typically not done in sticks filtering because it introduces artificial intensity discontinuities in the output image.
For sticks of length n, the above procedure can be implemented by preparing a bank of filter masks where the non-zero entries lie along the hypothesised lines and have value . Each filter is then applied to the input image via discrete convolution (i.e. linear filtering). The output image is formed by taking the pixel-wise maximum over all the 2n − 2 filtered images. This process is formally defined as follows.
Let f : [0, W − 1] × [0, H − 1] → [0, D − 1] be a function representing the input image of width W, height H and D intensity levels; f(x, y) is the intensity of the pixel at (x, y). Let s1, s2, …, s2n−1 be the stick filter masks. The output image, g(x, y), is given by:
where * denotes discrete convolution.
Sticks filter banks for a given stick length can vary slightly depending on the way in which the lines are discretised. Sticks filters were constructed by specifying the end-points of the line segments and using a digital differential analyser algorithm (Foley et al. 1994) to draw the intermediate pixels.
The sticks filter masks defined by Czerwinski et al. (1998) are herein referred to as ‘uniform sticks’ because the contributions of pixel intensities along the stick to the response of the filter are weighted equally.
Gaussian sticks are introduced as an alternative to uniform sticks. Gaussian sticks are similar to uniform sticks, except that instead of each non-zero entry in the filter masks having equal weight, the weights obey a Gaussian distribution centred on the middle of the mask, as in Fig. 3.
The width of the Gaussian is selected such that the end-points of the stick are three standard deviations away from the centre; thus, the total weight of all mask entries is nearly 1. In general, for sticks filters of size n, a Gaussian function with a standard deviation of is required. The mask positions that comprise the stick are determined in the same way as for linear sticks; the only difference is the weighting.
Gaussian sticks are proposed in an effort to improve the contrast between linear features and background. Figure 4 illustrates why an improvement may be expected. When processing pixels that are actually part of a linear feature, such as a line edge or step edge, both uniform and Gaussian sticks respond maximally in the orientation parallel to the feature (top of Fig. 4). The response of both filters in this case is the same, because the total weight of all mask entries is 1 for both types of stick. When processing pixels that are within pixels of the linear feature, where n is the stick length, the sticks that are perpendicular to the linear feature give the maximum response (bottom of Fig. 4). However, in this situation, the maximal response of the Gaussian stick will be smaller than the maximal response of the uniform stick because the weights near the ends of the stick are comparatively small.
Figure 5 depicts a 256 × 256 pixel image that demonstrates the effects of uniform and Gaussian stick filters of length 15 on some simple lines and shapes. The blurring around objects in the middle image of Fig. 5 is caused by the ends of sticks centred on black pixels protruding into white regions, causing an increase in the mean intensity along the stick. The response of the stick filter at such a pixel is therefore of intermediate intensity. Similarly, sticks centred on white pixels that are close to black regions protrude into the black background, causing a response that is less than maximum intensity. Thus, the sharp step edges in the original image are blurred by the sticks filter. The use of Gaussian sticks of the same length, depicted on the right of Fig. 5, reduces this effect. The Gaussian weighting causes the maximal response of the filter bank to decrease more quickly with increasing distance from the linear feature. Especially for longer sticks, the blurring caused by the uniform stick may reduce contrast between linear features in close proximity. Using Gaussian sticks, blur is reduced and we may achieve improved contrast between nearby linear features and background while retaining the property of speckle noise reduction. However, a desirable feature of the uniform sticks is their ability to connect collinear line segments, such as those on the left of the image in Fig. 5, which, in a noisy image, may be part of the same linear feature. This ability is clearly lessened by the Gaussian weighting.
The other variant of the sticks filter that was considered is that of outputting, for each pixel, the variance of the responses of each stick orientation as opposed to the maximum response. This variant of the sticks filter is sensitive to step edges. Pixels in regions of homogeneous intensity (A and C in Fig. 6) will offer similar stick filter responses regardless of orientation resulting in a low variance among responses. Edge pixels (B in Fig. 6) will also offer a low variance in responses because approximately half of each stick will be on either side of the edge for almost all stick orientations. In contrast, pixels near an edge, but not on it (D in Fig. 6), will offer a higher variance in stick filter responses because some sticks cross the edge and some will not.
The technique of constructing an output image from stick variance rather than maximum stick response is independent of the stick weighting used. In the Results section, we show that the stick variance filter can create good virtual histology images for either weighting scheme.
Stick length greatly affects the output of sticks filtering. The stick lengths used in our experiments (see below) were chosen with the following guidelines in mind. Czerwinski et al. (1998) reported that stick length should be longer than the correlation length of noise, but shorter than the distance over which boundaries appear to be straight line segments. Pathak et al. (2000) and Czerwinski et al. (1999) both applied sticks filtering to ultrasound images and considered a stick length of 15 to be of ‘intermediate’ length. Longer stick lengths resulted in a greater reduction of speckle, whereas shorter stick lengths resulted in better matches with boundaries of higher curvature (Czerwinski et al. 1999).
Odd length sticks must be used so that they are symmetrical along the perpendicular bisector of the stick.
Images were selected from a dataset obtained during a previous study of bovine ovaries collected at defined times of the oestrous cycle (Singh et al. 1997, 1998). Ovaries were placed in degassed phosphate-buffered saline and imaged at 0.5 mm increments in parallel planes with a broad-band, convex-array ultrasound transducer interfaced with an ATL Ultra Mark 9 HDI ultrasound machine (Advanced Technology Laboratories, Both-ell, WA, USA). All images were 640 pixels wide and 480 pixels high, with 256 grey levels. Ovaries were collected at four specific points during the oestrous cycle corresponding to wave patterns of follicular growth (Adams and Pierson 1995). Images were collected on Day 3 (Day 0 = wave emergence) of Wave 1 (first wave of oestrous cycle; D3W1), Day 6 of Wave 1 (D6W1), Day 1 of Wave 2 (D1W2) and after onset of pro-oestrus (D ≥ 17; Singh et al. 1998).
Two animals from each of the four groups described above were chosen at random for the present study to obtain a dataset sampled relatively uniformly over the reproductive cycle. For each animal, an image of the dominant follicle and CL was selected by choosing the image slice from the ovary that contained the structure of interest at maximum diameter.
The sticks filtering techniques described above, together with additional processing steps, comprised our candidate virtual histology algorithms. Several candidate virtual histology algorithms were tested and are summarised in Table 1. The input images were processed using a sticks filter with the given weighting, output mode and additional operations for the given stick lengths. The additional operations were used to create high-contrast textures from the linear features accentuated by the sticks filters. The additional operations used were as outlined below.
In this operation, horizontal and vertical 3 × 3 Sobel convolution filter masks were applied to the images. The respective responses h and v were combined as . The result was an edge magnitude image that is an estimate of the magnitude of the gradient of the input image (Sonka et al. 1999). The convolution filtering was performed using Matlab’s (The Mathworks, Inc., Natick, MA, USA) conv2 function. This operation was applied in Algorithms 1 and 2. The Sobel operator was used to exaggerate the edges and texture in the despeckled sticks filtered image. It computes an approximation of the magnitude of the first derivative of the sticks filtered image, transforming step edges into line edges and line edges into ‘double edges’; this effect is illlustrated in Fig. 8.
Normalisation is a linear remapping of pixel intensities that improves the brightness of the image. Matlab’s imadjust function was used to linearly redistribute the intensities in an image from the range [min, max] to the range [0, 255] where min and max are the lowest and highest intensities in the input image, respectively. This step was required for Algorithms 3 and 4 because the variance output mode does not generate pixel values in the usual grayscale range [0, 255].
Classical (global) histogram equalisation is a (generally non-linear) remapping of pixel intensities that spreads the pixel intensities over the entire available dynamic range and makes use of as many different intensities as possible. However, this remapping does not take into account local variation of the intensity histogram. Adaptive histogram equalisation performs the remapping on local groups of pixels (Sonka et al. 1999). There are many ways in which this can be done. Matlab’s adapthisteq function was used to perform histogram equalisation locally on 10 × 10 non-overlapping pixel neighbourhoods. This was the final step in Algorithms 1–4. This step was applied in order to increase contrast and to remove local variations in intensity so that image texture varies only in structure. Thus, homogeneous tissue regions should appear to be of uniform texture, free of local intensity variation, whereas structural differences in texture between regions of different tissues should be more apparent.
For Algorithms 3 and 4 in Table 1, it should be noted that the Sobel filtering operation was omitted. It was hypothesised that the stick variance filters and AHE alone would sufficiently improve texture differences between tissue regions due to the effect of the stick variance filter on step edges (Fig. 7).
For Algorithms 5 and 6 in Table 1, the difference image of two of the prior rows was computed. Algorithm 5 computed the difference image of the results of Algorithms 1 and 3, whereas Algorithm 6 computed the difference image of the results of Algorithms 2 and 4. Resulting pixels with negative intensity were truncated to intensity 0. No additional steps were performed for Algorithms 5 and 6.
Stick lengths to be used in testing of the algorithms in Table 1 were chosen according to the guidelines described above. Uneven stick length intervals were used because the masks in the stick filter bank must be of odd size. Sticks of length less than 9 were not considered because the boundaries of dominant follicles do not generally have very high curvature and the degree of speckle reduction of sticks shorter than length 9 is not strong. Sticks of length greater than 35 were not considered both because there appeared to be little improvement in results between sticks of length 31 and sticks of length 35 and because processing with stick lengths larger than 35 begins to become slow. We are also justified in focusing on longer length sticks because Czerwinski et al. (1999) showed that the degradation of performance of the sticks filter is worse if the optimal stick length is underestimated compared with when it is overestimated.
Stick thickness was not considered in the present study. Thicker sticks (lines with width >1 pixel) have been observed to increase the relative brightness of broad boundaries, but blur the transitions between the regions separated by the boundary; moreover, the use of thicker sticks reduces the visibility of thin boundaries (Czerwinski et al. 1999). Comparisons of the visual qualities of each algorithm are made in the Results section.
A qualitative analysis of some sample results of the virtual histology algorithms is given in the following sections. The images can be compared with a standard haematoxylin and eosin-stained section of a bovine ovary (Fig. 9) containing a dominant follicle and a CL. All examples are processed versions of the images in Fig. 10 using sticks of length 25. Figure 10 shows the image slices of two different ovaries from different animals in which the dominant follicle had maximal diameter compared with the other image slices of the same ovary. The left image contains the dominant follicle (A) and a subordinate follicle (B). The right image contains the dominant follicle (C), the subordinate follicle (D) and also happens to exhibit a large CL (E). Processed versions of these images are discussed with reference to the real histology of the ovary shown in Fig. 9. The histology shown is not that of either ovary imaged in Fig. 10, although it is highly similar morphologically and developmentally to the right-hand image.
The follicle–fluid interfaces were sharply delineated (Fig. 11; points labelled A). The previously indistinct boundary of the small follicle in the left image was sharpened. The antrum of the small follicles in both images lost their characteristic darkness; however, the boundaries remained visible and the interiors remained smooth in texture compared with that of the stroma (B) and the CL (C).
The texture of stroma regions (B) remained distinguishable from that of the CL (C). The stroma texture exhibited strong linear features predominantly perpendicular to the axial direction of the ultrasound beam, whereas in the CL the direction of strong linear features appeared to be more varied. Overall, the textures of parts of the image with similar grey levels were strikingly different after processing. Application of the sticks filter appears to enhance the visual appreciation of the micro-anatomy of luteal structures. We measure the apparent difference in texture quantitatively below.
Results for Algorithm 2 are given in Fig. 12. Follicle–fluid interfaces were more sharply delineated (A); however, the edges created around the dominant follicles were rougher and exhibited discontinuities. The follicle–fluid interface visibility was improved v. the uniform-weighted sticks for the subordinate follicles. The improvement resulted from the centre-biased stick weighting, which reduced blur, allowing more of the dark colour of follicle antra to remain; moreover, the centre bias caused the stick to behave more like a shorter stick with uniform weight, making it more sensitive to features of higher curvature than a uniform stick of the same length.
Stroma texture was not as distinguishable from CL texture when compared with the results from uniform mean sticks. This may again be attributed to the Gaussian-weighted stick’s affinity for structures of higher curvature, suggesting that the CL contains longer linear structures than the stroma. Indeed, the luteal gland comprises a heavily trabeculated wall surrounded by a relatively dense basal lamina; it appears as though the bending of the wall in a confined space is reflected in the filtered output image (cf. Fig. 9).
Results for Algorithm 3 are given in Fig. 13. We recall from the earlier description that this filter responds highly on either side of a step edge and responds weakly on the edge itself. This was reflected by the presence of a dark ring corresponding to the follicle–fluid interface of the dominant follicle in the left image (A). Follicles in the right image exhibited dark edges along some sections of the follicle–fluid interface with low curvature (B), especially along the bottom of the subordinate follicle and the left side of the dominant follicle; higher curvature segments were not as visible. Subordinate follicle C did not exhibit any distinct boundary, nor at any other stick length tested for this algorithm; thus, the lack of a distinct boundary could not have been caused solely by high boundary curvature. We note that in the original image this follicle had a particularly indistinct step edge along the follicle–fluid interface compared with the other follicles.
The stroma regions (D) differed significantly in texture from the corpus luteum (E). The CL texture was darker and finer than that of the stroma and the pattern of luteal tissue around the periphery is reminiscent of that seen in the histological section (Fig. 9). There was also a visual impression of internal organisation: a crenelated exterior wall and trabeculated internal structure. Stroma appeared to be randomly organised without any appreciation of structure other than that imposed by the boundaries of the ovary.
Results for Algorithm 4 are given in Fig. 14. As expected, the Gaussian-weighted sticks resulted in a more detailed image with less blurring. The dark edge at the follicle–fluid interface of the dominant follicle in the left image (A) was thinner compared with uniform sticks, but also exhibited more discontinuities. The follicle–fluid interfaces in the right image (B) were barely visible at all. The appearance of the subordinate follicle in the left image (C) was cleaner primarily because there was less blurring into the antral region. Overall, the delineation of follicle–fluid interfaces was poorer for Gaussian stick variance compared with the uniform weighting for all tested stick lengths less than 25.
The CL region (E) was darker than stroma regions (D), although both tissues exhibited finer texture than the result of the uniform-weighted variance algorithm. The texture of the CL region was suggestive of tissue structure organised in a roughly circular pattern around a central core. The walls appeared to have a crenelated exterior appearance and internal structure consistent with the highly folded walls of hypoechoic endocrine tissue. This attribute sets this algorithm apart from the others for which the CL texture had a homogeneous appearance (cf. Fig. 9). The structure observed in the CL was most apparent for length 15 sticks.
In this algorithm, the output image J of Algorithm 3 was subtracted from the output image I of Algorithm 1 to obtain a difference image D. Pixels for which the result of the subtraction was negative were set to zero. That is:
The image subtraction reduced the intensity of the blurring in I on either side of a step edge (where image J is of high intensity) while having little effect on pixels in I that are on a step edge (where image J is of low intensity). The effect on the follicle–fluid interface was to retain the bright edge created in the mean stick images, whereas blur around these edges was reduced by subtracting the bright edges around the follicle–fluid interface from the stick variance images. Results for Algorithm 5 are shown in Fig. 15. Indeed, the blur around the follicle–fluid interface from Fig. 11 was reduced or removed in the vicinity of the boundary (A).
Texture of the stroma (B) was noticeably coarser than texture of the CL (C). The CL texture again exhibited the suggestion of complex internal structure highlighted by distinct regions containing a fine texture, whereas the stroma regions were more homogeneous and random.
Results for Algorithm 6 are shown in Fig. 16. The advantage gained by Algorithm 5 relied on the relatively large amount of blur introduced by the uniform-weighted sticks. The Gaussian-weighted sticks, introduced as a variation for reducing blur, were unable to reduce the blur in the stick mean images from Fig. 12 by subtracting the variance result from the mean result. Although overall contrast in the textures is improved, the bright edges of the follicle–fluid interfaces (A) were not separated as well from their neighbouring texture compared with Algorithm 5. Texture differences between stroma (B) and CL (C) were not as pronounced compared with Algorithm 5.
An experiment was conducted to determine to what extent the virtual histology algorithms increased the difference in echotexture between ovarian stroma and CL regions. Stroma and CL texture regions in images from eight pairs of ovaries from the eight different animals described above were used. In general, texture regions were hand selected, irregularly shaped regions of homogeneous stroma or CL texture.
Texture properties of these regions were measured by computing their grey level co-occurance features (Haralick et al. 1973). Co-occurance features are derived from co-occurance matrices.
The entry at row i, column j of a grey level co-occurance matrix (GLCM) M for displacement d = (x, y) is the number of times a pixel with grey level j is located at a displacement of (x, y) from a pixel with grey level j. Formally:
All pairs of pixels [(k, l), (k + x, l + y)] in an image, or region of interest, are considered. A normalised GLCM P is constructed by dividing by the sum of the entries in M:
P(x,y)(i, j) may be interpreted as the probability that a pixel of intensity j occurs at displacement (x, y) from a pixel of intensity i.
Normalised GLCMs were computed for displacements (1, 0), (1, 1), (1, 0) and (−1, 1) for each texture region of interest. The texture features of contrast, correlation, energy and homogeneity (Haralick et al. 1973) were computed from the GLCMs for the unprocessed texture regions, as well as the same regions after processing with Algorithms 1–6 from Table 1 for all stick lengths. These texture features are defined, for displacement d, as:
Intuitively, contrast measures dissimilarity in grey levels for the given displacement; off-diagonal entries in the matrix contribute more to the sum. Conversely, homogeneity measures grey level similarity for the given displacement. Large values of energy result from ‘orderly’ textures. Correlation measures the linear relationship (if any) between pixel intensities at the given displacement. This value ranges from −1 (strong negative correlation) to 1 (strong positive correlation), with a result of 0 indicating no correlation in pixel intensities.
The relative feature difference (RFD) δ between feature values for CL and stroma texture were computed for the original images and the images resulting from each algorithm and stick length combination. That is, let α be a co-occurance feature for a stroma region of an ovary and β be the feature value for the CL region of the same ovary. The relative feature difference for a given feature was defined as the ratio as follows:
In order to aid the interpretation of the results, we computed the change in RFD magnitude for each stick length in each algorithm. Let δorig denote the RFD between stroma and CL textures in the original unprocessed regions and let δi denote the RFD between stroma and CL textures after processing with one of the Algorithms 1–6 using stick length i. Then, the change in RFD magnitude, Δi, is defined as:
Intuitively, if the magnitude of δi is greater than δorig, then Δi is positive, indicating that the given algorithm with stick length i caused the RFD to move away from zero and caused the textures to become more different. Otherwise, Δi is negative, indicating a reduction in magnitude of the RFD, causing the two textures to become more similar. For example, if δorig = 1 and δi = 2, then Δi = 2 − 1 = +1, indicating an increased texture difference. Similarly, if δorig = −1 and δi = −2, then Δi = 2 − 1 = +1, indicating an increase in the negative magnitude of the RFD and, again, an increased texture difference. Conversely, if δorig = 1.5 and δi = −0.5, then Δi = 0.5 − 1.5 = −1, indicating that the magnitude of the RFD decreased, resulting in more similar textures, even though the absolute value of the RFD actually changed by −2 and changed in sign. Thus, good algorithms for improving differences between CL and stroma texture will result in consistently positive values of Δi.
Overall, for Algorithms 1 and 2, there were no texture features for which the RFD magnitude consistently increased after processing. For Algorithms 5 and 6, the RFD magnitude for the contrast feature consistently decreased; the energy feature was inconsistent, exhibiting increased RFD magnitude for some ovaries and decreasing it for others; the RFD of homogeneity was hardly changed at all, exhibiting very slight increases and decreases depending on the ovary. Algorithm 3 produced the second-best results, producing fairly consistent RFD increases in contrast, with a couple of exceptions; homogeneity increased consistently, but to a lesser degree than contrast and energy. For contrast, stick length influenced the degree of RFD increase. Results for Algorithm 3 are shown in Fig. 17. Algorithm 4 produced the best results, producing consistent increases in contrast RFD where, again, stick length influenced the magnitude of the increase; energy RFD was consistently higher for all stick lengths, with the exception of one anomalous ovary, which, on closer inspection, appeared to have been imaged without proper calibration of brightness/contrast of the ultrasound machine resulting in clipping of the intensity at the high end of the brightness scale; homogeneity was also consistently increased, although generally to a lesser degree. Results for Algorithm 4 are shown in Fig. 18.
Mean RFD magnitude change for each algorithm, averaged over all stick lengths, is given in Table 2 to provide a high-level comparison of the ability of the six algorithms to improve RFD. Contrast is consistently improved by Algorithms 3 and 4. Although the standard deviations of contrast improvement are high for all algorithms, the positive skew for Algorithms 3 and 4 indicates that the majority of this variation is in the favourable direction. Good results for energy are obtained by Algorithms 3–6 when the aforementioned anomalous results of one ovary are discounted. Homogeneity is consistently improved by Algorithms 3 and 4, with Algorithm 4 having slightly more favourable statistics.
For all algorithms, the RFD for correlation showed inconsistent small increases and decreases and is consequently deemed not to be a useful feature for stroma/CL discrimination for the co-occurance displacements considered.
There are several important aspects of the image-processing approach that bear consideration.
A comprehensive study of the effect of stick length and determination of the ‘best’ stick length is left as future work; however, we can, at this time, make the following qualitative observations.
Smaller follicles were better delineated by shorter sticks. This is not surprising given the nature of the sticks filter. The sticks filter highlights linear features by correlating line segments with the image. The amount of curvature of a feature affects the scale on which the feature appears to be linear. Lower curvature features are ‘close to linear’ over longer segments than features of high curvature; thus, it should not be unexpected that shorter sticks are needed to resolve the follicle–fluid interface of smaller follicles. This behaviour is consistent with results observed by Czerwinski et al. (1999). In general, the uniform difference algorithm produced the best delineation of the follicle–fluid interface (figs 2.35–2.42 in Eramian et al. 2006).
Stick length had an effect on the enhanced visual appreciation of micro-anatomical structure in the CL. This structure was most visible for the uniform stick variance algorithm. Sticks of length 11, 15, 21 and 25 all produced good quality images (figs 3.12–3.18 in Eramian et al. 2006). Shorter sticks resulted in finer delineation of apparent structures, whereas longer sticks offered higher contrast and more of a macroview of structure. Gaussian stick variance also offered good appreciation structure in the CL textures; however, sticks of at least length 15 were required to obtain adequate contrast (figs 3.27–3.34 in Eramian et al. 2006). The results for mean sticks (both uniform and Gaussian weighted) were much less dependent on stick length. For the uniform difference algorithm, only three stick lengths were used. Texture does not differ significantly with the stick length, although the length 25 sticks produced results with slightly higher contrast (figs 3.35–3.42 in Eramian et al. 2006). Results for the Gaussian difference were relatively independent of stick length (figs 3.43–3.50 in Eramian et al. 2006).
Even for modestly sized input images and values of n, the computation of 2n − 2 convolution operations can become time consuming. Run times for the uniform mean and uniform variance algorithms are depicted in Fig. 19. Timings were performed on an Intel SE7501 server with a 2.4GHz CPU and 2Gb of RAM running Mandriva Linux 2006. Figure 19 shows that the non-sticks filtering steps are, as expected, independent of the stick size. Total running time appears to be quadratic, which is what one would expect from theoretical analysis of the algorithm’s computational complexity. The additional time required for filtering in the uniform variance algorithm is caused by having to retain the responses for all filters and calculate the variance of those responses for each pixel. We note, however, that our implementation of this algorithm is inefficient and could be improved by incrementally computing variances for each pixel after each stick filter is applied.
There are other ways in which run time could be improved. Sticks filter masks are sparse with relatively few non-zero entries, therefore the masks can be implemented by a list of only the non-zero mask entries. This reduces the computation time for each filter mask in the filter bank by a factor of n (the stick length). Moreover, speed could be improved by implementing the procedures in a compiled language such as C. An implementation programmed in C using lists of non-zero mask entries instead of matrices was implemented and the run times for uniform mean stick filtering (performed on the same hardware) are shown in Fig. 20. This implementation does not include the non-sticks filtering steps and should be compared only with the ‘sticks filtering’ times in Fig. 19. With these optimisations, it is possible to implement all six virtual histology algorithms in time comparable with the C implementation.
Efficiency could also be improved by taking advantage of the Convolution Theorem and performing the filtering in the frequency domain instead of the space domain using precomputed frequency impulse responses of each stick filter.
Our overall hypothesis, that ‘staining’ algorithms would be able to produce images reminiscent of a virtual histology approach to study ovarian function, was supported. Although our technique does not replace human interpretation and decision making, it is a step towards acquiring near histological information in a safe, non-invasive manner.
Six candidate methods for producing a ‘virtual histology’ of bovine ovaries were tested. The staining algorithms were shown to produce sharper follicle boundaries and the CL texture produced by some algorithms was shown to be reminiscent of actual histology. The uniform difference method produced the best delineation and isolation of the follicle–fluid interface. The quantitative difference between stroma and CL echotexture was shown to be increased most consistently by Algorithms 3 and 4. The ability to isolate the boundaries of follicles was, indeed, dependent on stick length, as expected. Future work will study the accuracy of boundary location and the qualities of the edge profile that forms the boundary.
This work represents a natural first step towards understanding changes over time inherent in the CL, as well as a first step towards looking at pathological changes in the ovary based on echotexture analysis. Increased understanding of the variation of the statistical properties of echotexture in the menstrual cycle will facilitate the definition of discriminating features for automated image classification and diagnosis in vivo, as well as providing a basis on which to construct a complete segmentation of the image into regions of homogeneous tissue to complement the virtual histologies.
This research was supported by grants from the Saskatchewan Heath Research Foundation, the Natural Sciences and Engineering Research Council of Canada and the Canadian Institutes of Health Research. We wish to thank Dr J. Singh, Department of Veterinary Biomedical Sciences, University of Saskatchewan, for providing the image in Fig. 9.