|Home | About | Journals | Submit | Contact Us | Français|
We present a novel edge preserved interpolation scheme for fast upsampling of natural images. The proposed piecewise hyperbolic operator uses a slope-limiter function that conveniently lends itself to higher-order approximations and is responsible for restricting spatial oscillations arising due to the edges and sharp details in the image. As a consequence the upsampled image not only exhibits enhanced edges, and discontinuities across boundaries, but also preserves smoothly varying features in images. Experimental results show an improvement in the PSNR compared to typical cubic, and spline-based interpolation approaches.
Interpolation is a frequently needed tool for many imaging applications ranging from image zooming, resizing, retouching, formatting, manipulation, and compositing. Aside from applications aiming to aesthetically manipulate images, one routinely needs image resampling for the purpose of image matching and registration for computer vision applications. Often times, high throughput machine vision tasks such as scene reconstruction and warping utilize simple but fast techniques such as bilinear, bicubic, and occasionally b-spline interpolation for single images. These methods implicitly assume a smoothness prior in the image resampling process. For example, bilinear interpolation assumes that the resampled intensity values arise from first order local averaging of neighboring intensities of the image, whereas higher order methods such as bicubic, and b-spline assume that local intensities are estimated by imposing smoothness constraints by fitting high-order polynomials to the intensity function of the image. While bilinear interpolation restricts signal overshoots at discontinuities, bicubic, b-spline and other higher order methods introduce ringing, and haloing artifacts in images. Furthermore, the performance of many vision applications rely on accurate preservation and detection of edges from images. Thus a compromise needs to be achieved between edge fidelity and processing latency.
There are several approaches for image upsampling, especially for preserving edges [1, 2, 3] during the interpolation process. Our approach focuses on single image upsampling, and is different from image super-resolution approaches  that typically involve either fusion of multiple images, or integration of example-based constraints along with sub-pixel homologies. Conventionally, the problem of image upsampling is approached by formulating a degradation model specified by a convolution kernel as well as a downsampling operator. The high resolution image is then reconstructed by solving the inverse problem of image reconstruction that assigns intensity values to the desired image. In this paper, instead of explicitly optimizing over the degradation model, we directly focus on the upsampling operator that yields a one-step interpolation of the observed image. Our resampling approach borrows from methods in fluid dynamics [5, 6] that attempt to construct total variation diminishing (TVD) solutions from high resolution numerical schemes for modeling shocks and discontinuities in fluid flows. Our interpolation method does not require iterative optimization; instead it provides a one-step up/downsampling of the original image. It relies on higher order derivatives of the image, and limits spatial oscillations at edges and discontinuities, and at the same time, satisfies the dual requirement of speed and edge accuracy.
This section formulates the upsampling problem, and introduces the edge-preserving operator based on a slope-limiter function. Before using the operator for the purpose of image upsampling, we make the following observation first. The reconstruction error for a single frame image degradation model is usually expressed as a convolution of the high resolution image u with a blur kernel operator, followed by a downsampling process, and can be given by
where S is an upsampling operator, and D ο S = I, an identity matrix. The degraded image is confounded by both the down sampling process, and the blur operator. Assuming that the operators D and h, are fixed, and independent of the image u in Eq. 1, there are a variety of upsampling operators S, that seek to minimize the cost in Eq. 1. For example, it can be shown that for piecewise smooth images, a bilinear interpolant operator yields a lower estimate for the error E in equation 1 when compared with a nearest neighbor interpolant. Slope-limiter functions have previously been introduced [7, 5, 6] in fluid dynamics for clamping spurious oscillations, and improving the resolution at edge discontinuities. The idea here is to choose an appropriate slope-limiter form such that the interpolant ensures accurate spatial approximation, and prevents excessive increase in the total variation in the neighborhood of a pixel. In order to fix notation, we consider a two-dimensional m × n image, and set up a pixel ujk : 1 ≤ j ≤ m, 1 ≤ k ≤ n, where ujk is an average of the true signal intensity g(x, y) in the pixel, and is written as
where hx and hy are the pixel step sizes along the X and Y dimension of the image. Furthermore, we assume that the domain of the pixel function ujk, centered at (xj, yk) is given by , and denote the divided differences in the image by , and , and .
Our goal is to approximate the true intensity function of the image g(x, y) in each pixel by means of an elementary function Hjk(x, y), such that Eq. 2 is satisfied. While there are several different edge-preserving forms for the function Hjk, following Marquina  we restrict our focus to special type of functions also known as slope-limiters. There is a wide variety of such nonlinear functions  that preprocess divided differences and enforce the order of accuracy. In this paper, we propose a third order approximation to the function H(x, y) using a piecewise hyperbolic form. Additionally we will use the harmod limiter function  that uses the notion of a harmonic mean instead of an absolute mean of the divided differences. The harmod limiter operator is given by . We assume that the general class of the nonlinear interpolant functions has a general form given by
where the parameters ajk, bjk are given by , and . In order to define αx and αy, we first define , and . Then the parameters αx and βy are defined as
We now define the upsampling operator S at the center of half-size pixels in each dimension as
where xj(θx) = xj + θxhx, and yk(θy) = yk + θyhy, and θx, θy [−1/4, 1/4]. Similarly, the downsampling operator D is defined as . On account of the higher-order approximation, it is noted that the upsampling criteria is given by D ο S(u) = u + O(((hxhy)), and no longer identity. However, from a practical standpoint, the errors are negligible, and the signal is not degraded in a significant way. It is also noted that the operation of downsampling followed by upsampling is not reversible, and thus D ο S ≠ S ο D.
In this section, we present results of upsampled images by applying the operator specified in Eq. 6. Figure 1 shows upsampled images obtained using bilinear, b-spline, and the edge-preserved upsampling operators. The respective PSNR values are quantified in Table 1. In order to calculate the PSNR, the original images were downsampled using bilinear interpolation and then upsampled using each of the upsampling operators. The edge-preserving piecewise hyperbolic operator not only outperforms bicubic, and b-spline interpolation, but the resulting upsampled images appear sharper, and show better edges. Finally, the edge-preserving operator is also used to upsample color images as shown in Fig. 2. Here, the color image was first separated into luminance (Y), and the two chrominance channels (UV), and each interpolation operator was applied separately to each channel, and then converted back to the RGB colorspace. It is observed that the edge-preserved upsampled images resolve sharpness and detail better compared to the bilinear, and spline based approaches.
The proposed piecewise hyperbolic operator ensures that the upsampled reconstructions are smooth inside each pixel, and limits oscillations and jump discontinuities located at the pixel interfaces. From the experimental results obtained after applying our edge-preserving operator for interpolation, we observe that this reconstruction error is further lowered compared with both bilinear, bicubic, as well as b-spline interpolates. The operator is simple to implement, and executes in 0.2 s for an average image size of 256×256 pixels in MATLAB on an Intel 2.4 GHz platform. We anticipate the usefulness of this operator in routine image processing as well as computer vision tasks, where the preservation of edge quality is of primary importance.
This work is supported in part by NIH grants RC1MH088194, and P41 RR013642. Additionally, Dr. Antonio Marquina gratefully acknowledges the support from the NSF grants DMS-0312222, ACI-0321917, the NIH grant G54 RR021813, as well as DGICYT MTM2008-03597 from the Spanish Government Agency.