Home | About | Journals | Submit | Contact Us | Français |

**|**HHS Author Manuscripts**|**PMC3273962

Formats

Article sections

Authors

Related links

Proc Int Conf Image Proc. Author manuscript; available in PMC 2012 February 7.

Published in final edited form as:

Proc Int Conf Image Proc. 2011 September 14 : 1165–1168.

PMCID: PMC3273962

NIHMSID: NIHMS293047

Shantanu H. Joshi,^{1} Antonio L. Marquina,^{2} Stanley J. Osher,^{3} Ivo Dinov,^{1} Arthur W. Toga,^{1} and John D. Van Horn^{1}

We present a novel edge preserved interpolation scheme for fast upsampling of natural images. The proposed piecewise hyperbolic operator uses a slope-limiter function that conveniently lends itself to higher-order approximations and is responsible for restricting spatial oscillations arising due to the edges and sharp details in the image. As a consequence the upsampled image not only exhibits enhanced edges, and discontinuities across boundaries, but also preserves smoothly varying features in images. Experimental results show an improvement in the PSNR compared to typical cubic, and spline-based interpolation approaches.

Interpolation is a frequently needed tool for many imaging applications ranging from image zooming, resizing, retouching, formatting, manipulation, and compositing. Aside from applications aiming to aesthetically manipulate images, one routinely needs image resampling for the purpose of image matching and registration for computer vision applications. Often times, high throughput machine vision tasks such as scene reconstruction and warping utilize simple but fast techniques such as bilinear, bicubic, and occasionally b-spline interpolation for single images. These methods implicitly assume a smoothness prior in the image resampling process. For example, bilinear interpolation assumes that the resampled intensity values arise from first order local averaging of neighboring intensities of the image, whereas higher order methods such as bicubic, and b-spline assume that local intensities are estimated by imposing smoothness constraints by fitting high-order polynomials to the intensity function of the image. While bilinear interpolation restricts signal overshoots at discontinuities, bicubic, b-spline and other higher order methods introduce ringing, and haloing artifacts in images. Furthermore, the performance of many vision applications rely on accurate preservation and detection of edges from images. Thus a compromise needs to be achieved between edge fidelity and processing latency.

There are several approaches for image upsampling, especially for preserving edges [1, 2, 3] during the interpolation process. Our approach focuses on single image upsampling, and is different from image super-resolution approaches [4] that typically involve either fusion of multiple images, or integration of example-based constraints along with sub-pixel homologies. Conventionally, the problem of image upsampling is approached by formulating a degradation model specified by a convolution kernel as well as a downsampling operator. The high resolution image is then reconstructed by solving the inverse problem of image reconstruction that assigns intensity values to the desired image. In this paper, instead of explicitly optimizing over the degradation model, we directly focus on the upsampling operator that yields a one-step interpolation of the observed image. Our resampling approach borrows from methods in fluid dynamics [5, 6] that attempt to construct total variation diminishing (TVD) solutions from high resolution numerical schemes for modeling shocks and discontinuities in fluid flows. Our interpolation method does not require iterative optimization; instead it provides a one-step up/downsampling of the original image. It relies on higher order derivatives of the image, and limits spatial oscillations at edges and discontinuities, and at the same time, satisfies the dual requirement of speed and edge accuracy.

This section formulates the upsampling problem, and introduces the edge-preserving operator based on a slope-limiter function. Before using the operator for the purpose of image upsampling, we make the following observation first. The reconstruction error for a single frame image degradation model is usually expressed as a convolution of the high resolution image *u* with a blur kernel operator, followed by a downsampling process, and can be given by

$$E={\parallel u-SD(h\ast u)\parallel}^{2},$$

(1)

where *S* is an upsampling operator, and *D* ο *S* = *I*, an identity matrix. The degraded image is confounded by both the down sampling process, and the blur operator. Assuming that the operators *D* and *h*, are fixed, and independent of the image *u* in Eq. 1, there are a variety of upsampling operators *S*, that seek to minimize the cost in Eq. 1. For example, it can be shown that for piecewise smooth images, a bilinear interpolant operator yields a lower estimate for the error *E* in equation 1 when compared with a nearest neighbor interpolant. Slope-limiter functions have previously been introduced [7, 5, 6] in fluid dynamics for clamping spurious oscillations, and improving the resolution at edge discontinuities. The idea here is to choose an appropriate slope-limiter form such that the interpolant ensures accurate spatial approximation, and prevents excessive increase in the total variation in the neighborhood of a pixel. In order to fix notation, we consider a two-dimensional *m* × *n* image, and set up a pixel *u _{jk}* : 1 ≤

$${u}_{jk}=\frac{1}{{h}_{x}{h}_{y}}{\int}_{{x}_{j}-{h}_{x}\u22152}^{{x}_{j}+{h}_{x}\u22152}{\int}_{{y}_{k}-{h}_{y}\u22152}^{{y}_{k}+{h}_{y}\u22152}g(x,y)dxdy,$$

(2)

where *h _{x}* and

Our goal is to approximate the true intensity function of the image *g*(*x, y*) in each pixel by means of an elementary function *H _{jk}*(

$${H}_{jk}={f}_{jk}+\frac{a}{x-{x}_{j}+c}+\frac{b}{y-{y}_{k}+d},$$

(3)

Our goal is to find a specific form of the above nonlinear function. We consider the following ansatz [6] in order to represent the functional form given in Eq. 3,

$${H}_{jk}(x,y)={u}_{jk}+{a}_{jk}\frac{{h}_{x}}{{\alpha}_{x}^{2}}[\mathrm{log}\left(\frac{2-{\alpha}_{x}}{2+{\alpha}_{x}}\right)-\frac{{h}_{x}}{x-{x}_{j}-\frac{{h}_{x}}{{\alpha}_{x}}}]+{b}_{jk}\frac{{h}_{y}}{{\alpha}_{y}^{2}}[\mathrm{log}\left(\frac{2-{\alpha}_{y}}{2+{\alpha}_{y}}\right)-\frac{{h}_{y}}{y-{y}_{j}-\frac{{h}_{y}}{{\alpha}_{y}}}],$$

(4)

where the parameters *a _{jk}, b_{jk}* are given by ${a}_{jk}=\text{harmod}(\frac{{u}_{jk}-{u}_{j-1k}}{{h}_{x}},\frac{{u}_{j+1k}-{u}_{jk}}{{h}_{x}})$, and ${b}_{jk}=\text{harmod}(\frac{{u}_{jk}-{u}_{jk-1}}{{h}_{y}},\frac{{u}_{jk+1}-{u}_{jk}}{{h}_{y}})$. In order to define

$$\begin{array}{c}\hfill {\alpha}_{x}=2\left[\sqrt{\frac{{e}_{x}}{{\Delta}_{-}^{x}{u}_{jk}}}-1\right],\text{if}\mid {\Delta}_{-}^{x}{u}_{jk}\mid \le \mid {\Delta}_{+}^{x}{u}_{jk}\mid \hfill \\ \hfill {\alpha}_{x}=2\left[1-\sqrt{\frac{{e}_{x}}{{\Delta}_{+}^{x}{u}_{jk}}}\right]\text{if}\mid {\Delta}_{-}^{x}{u}_{jk}\mid <\mid {\Delta}_{+}^{x}{u}_{jk}\mid \hfill \\ \hfill {\alpha}_{y}=2\left[\sqrt{\frac{{e}_{y}}{{\Delta}_{-}^{y}{u}_{jk}}}-1\right],\text{if}\mid {\Delta}_{-}^{y}{u}_{jk}\mid \le \mid {\Delta}_{+}^{y}{u}_{jk}\mid \hfill \\ \hfill {\alpha}_{y}=2\left[1-\sqrt{\frac{{e}_{y}}{{\Delta}_{+}^{y}{u}_{jk}}}\right]\text{if}\mid {\Delta}_{-}^{y}{u}_{jk}\mid <\mid {\Delta}_{+}^{y}{u}_{jk}\mid \hfill \end{array}$$

(5)

We now define the upsampling operator *S* at the center of half-size pixels in each dimension as

$$S({x}_{j}\left({\theta}_{x}\right),{y}_{k}\left({\theta}_{y}\right))={H}_{jk}({x}_{j}\left({\theta}_{x}\right),{y}_{k}\left({\theta}_{y}\right)),$$

(6)

where *x _{j}*(

In this section, we present results of upsampled images by applying the operator specified in Eq. 6. Figure 1 shows upsampled images obtained using bilinear, b-spline, and the edge-preserved upsampling operators. The respective PSNR values are quantified in Table 1. In order to calculate the PSNR, the original images were downsampled using bilinear interpolation and then upsampled using each of the upsampling operators. The edge-preserving piecewise hyperbolic operator not only outperforms bicubic, and b-spline interpolation, but the resulting upsampled images appear sharper, and show better edges. Finally, the edge-preserving operator is also used to upsample color images as shown in Fig. 2. Here, the color image was first separated into luminance (Y), and the two chrominance channels (UV), and each interpolation operator was applied separately to each channel, and then converted back to the RGB colorspace. It is observed that the edge-preserved upsampled images resolve sharpness and detail better compared to the bilinear, and spline based approaches.

Upsampled images (×2) resulting from bilinear, b-spline, and edge-preserving interpolation applied to the downsampled image.

From top: Upsampled images (×2) resulting from bilinear, b-spline, and edge-preserved interpolation applied to the downsampled original image. The same interpolation method is applied separately to each of the luminance and the two chrominance **...**

The proposed piecewise hyperbolic operator ensures that the upsampled reconstructions are smooth inside each pixel, and limits oscillations and jump discontinuities located at the pixel interfaces. From the experimental results obtained after applying our edge-preserving operator for interpolation, we observe that this reconstruction error is further lowered compared with both bilinear, bicubic, as well as b-spline interpolates. The operator is simple to implement, and executes in 0.2 s for an average image size of 256×256 pixels in MATLAB on an Intel 2.4 GHz platform. We anticipate the usefulness of this operator in routine image processing as well as computer vision tasks, where the preservation of edge quality is of primary importance.

This work is supported in part by NIH grants RC1MH088194, and P41 RR013642. Additionally, Dr. Antonio Marquina gratefully acknowledges the support from the NSF grants DMS-0312222, ACI-0321917, the NIH grant G54 RR021813, as well as DGICYT MTM2008-03597 from the Spanish Government Agency.

[1] Fattal R. Image upsampling via imposed edge statistics. ACM Transactions on Graphics. 2007 Aug;26(3):95.1–95.7.

[2] Mishiba K, Suzuki T, Ikehara M. Edge-adaptive image interpolation using constrained least squares; 17th IEEE International Conference on Image Processing (ICIP), 2010; 2010.pp. 2837–2840.

[3] Casciola G, Montefusco LB, Morigi S. Edge-driven image interpolation using adaptive anisotropic radial basis functions. J Math Imaging Vis. 2010 Jan;36(2):125–139.

[4] Glasner D, Bagon S, Irani M. Super-resolution from a single image; 12th IEEE International Conference on Computer Vision; 2009.pp. 349–356.

[5] Harten A. High resolution schemes for hyperbolic conservation laws. Journal of Computational Physics. 1983;135(2):357–393.

[6] Marquina A. Local piecewise hyperbolic reconstruction of numerical fluxes for nonlinear scalar conservation-laws. SIAM J Sci Comput. 1994 Jan;15(4):892–915.

[7] Van Leer B. Towards the ultimate conservative difference scheme II. monotonicity and conservation combined in a second order scheme. Journal of Computational Physics. 1974;14(4):361–370.

[8] Serna S, Marquina A. Power ENO methods: a fifth-order accurate weighted power ENO method. J Comput Phys. 2004 Jan;194(2):632–658.

PubMed Central Canada is a service of the Canadian Institutes of Health Research (CIHR) working in partnership with the National Research Council's national science library in cooperation with the National Center for Biotechnology Information at the U.S. National Library of Medicine(NCBI/NLM). It includes content provided to the PubMed Central International archive by participating publishers. |