PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Proc Int Conf Image Proc. Author manuscript; available in PMC 2012 February 7.
Published in final edited form as:
Proc Int Conf Image Proc. 2011 September 14 : 1165–1168.
PMCID: PMC3273962
NIHMSID: NIHMS293047

FAST EDGE-FILTERED IMAGE UPSAMPLING

Abstract

We present a novel edge preserved interpolation scheme for fast upsampling of natural images. The proposed piecewise hyperbolic operator uses a slope-limiter function that conveniently lends itself to higher-order approximations and is responsible for restricting spatial oscillations arising due to the edges and sharp details in the image. As a consequence the upsampled image not only exhibits enhanced edges, and discontinuities across boundaries, but also preserves smoothly varying features in images. Experimental results show an improvement in the PSNR compared to typical cubic, and spline-based interpolation approaches.

Keywords: interpolation, edge-preserving, slope-limiter

1 Introduction

Interpolation is a frequently needed tool for many imaging applications ranging from image zooming, resizing, retouching, formatting, manipulation, and compositing. Aside from applications aiming to aesthetically manipulate images, one routinely needs image resampling for the purpose of image matching and registration for computer vision applications. Often times, high throughput machine vision tasks such as scene reconstruction and warping utilize simple but fast techniques such as bilinear, bicubic, and occasionally b-spline interpolation for single images. These methods implicitly assume a smoothness prior in the image resampling process. For example, bilinear interpolation assumes that the resampled intensity values arise from first order local averaging of neighboring intensities of the image, whereas higher order methods such as bicubic, and b-spline assume that local intensities are estimated by imposing smoothness constraints by fitting high-order polynomials to the intensity function of the image. While bilinear interpolation restricts signal overshoots at discontinuities, bicubic, b-spline and other higher order methods introduce ringing, and haloing artifacts in images. Furthermore, the performance of many vision applications rely on accurate preservation and detection of edges from images. Thus a compromise needs to be achieved between edge fidelity and processing latency.

There are several approaches for image upsampling, especially for preserving edges [1, 2, 3] during the interpolation process. Our approach focuses on single image upsampling, and is different from image super-resolution approaches [4] that typically involve either fusion of multiple images, or integration of example-based constraints along with sub-pixel homologies. Conventionally, the problem of image upsampling is approached by formulating a degradation model specified by a convolution kernel as well as a downsampling operator. The high resolution image is then reconstructed by solving the inverse problem of image reconstruction that assigns intensity values to the desired image. In this paper, instead of explicitly optimizing over the degradation model, we directly focus on the upsampling operator that yields a one-step interpolation of the observed image. Our resampling approach borrows from methods in fluid dynamics [5, 6] that attempt to construct total variation diminishing (TVD) solutions from high resolution numerical schemes for modeling shocks and discontinuities in fluid flows. Our interpolation method does not require iterative optimization; instead it provides a one-step up/downsampling of the original image. It relies on higher order derivatives of the image, and limits spatial oscillations at edges and discontinuities, and at the same time, satisfies the dual requirement of speed and edge accuracy.

2 Edge-Preserved Sampling

This section formulates the upsampling problem, and introduces the edge-preserving operator based on a slope-limiter function. Before using the operator for the purpose of image upsampling, we make the following observation first. The reconstruction error for a single frame image degradation model is usually expressed as a convolution of the high resolution image u with a blur kernel operator, followed by a downsampling process, and can be given by

E=uSD(hu)2,
(1)

where S is an upsampling operator, and D ο S = I, an identity matrix. The degraded image is confounded by both the down sampling process, and the blur operator. Assuming that the operators D and h, are fixed, and independent of the image u in Eq. 1, there are a variety of upsampling operators S, that seek to minimize the cost in Eq. 1. For example, it can be shown that for piecewise smooth images, a bilinear interpolant operator yields a lower estimate for the error E in equation 1 when compared with a nearest neighbor interpolant. Slope-limiter functions have previously been introduced [7, 5, 6] in fluid dynamics for clamping spurious oscillations, and improving the resolution at edge discontinuities. The idea here is to choose an appropriate slope-limiter form such that the interpolant ensures accurate spatial approximation, and prevents excessive increase in the total variation in the neighborhood of a pixel. In order to fix notation, we consider a two-dimensional m × n image, and set up a pixel ujk : 1 ≤ jm, 1 ≤ kn, where ujk is an average of the true signal intensity g(x, y) in the pixel, and is written as

ujk=1hxhyxjhx2xj+hx2ykhy2yk+hy2g(x,y)dxdy,
(2)

where hx and hy are the pixel step sizes along the X and Y dimension of the image. Furthermore, we assume that the domain of the pixel function ujk, centered at (xj, yk) is given by (xjhx2,xj+hx2)×(ykhy2,yk+hy2), and denote the divided differences in the image by Δ+xujk=uj+1kujkhx, Δxujk=ujkuj1khx and Δ+yujk=ujk+1ujkhy, and Δyujk=ujkujk1hy.

2.1 Higher-order Piecewise Hyperbolic Operator

Our goal is to approximate the true intensity function of the image g(x, y) in each pixel by means of an elementary function Hjk(x, y), such that Eq. 2 is satisfied. While there are several different edge-preserving forms for the function Hjk, following Marquina [6] we restrict our focus to special type of functions also known as slope-limiters. There is a wide variety of such nonlinear functions [8] that preprocess divided differences and enforce the order of accuracy. In this paper, we propose a third order approximation to the function H(x, y) using a piecewise hyperbolic form. Additionally we will use the harmod limiter function [6] that uses the notion of a harmonic mean instead of an absolute mean of the divided differences. The harmod limiter operator is given by harmod(α,β)=sgn(α)+sgn(β)22αβα+β. We assume that the general class of the nonlinear interpolant functions has a general form given by

Hjk=fjk+axxj+c+byyk+d,
(3)

Our goal is to find a specific form of the above nonlinear function. We consider the following ansatz [6] in order to represent the functional form given in Eq. 3,

Hjk(x,y)=ujk+ajkhxαx2[log(2αx2+αx)hxxxjhxαx]+bjkhyαy2[log(2αy2+αy)hyyyjhyαy],
(4)

where the parameters ajk, bjk are given by ajk=harmod(ujkuj1khx,uj+1kujkhx), and bjk=harmod(ujkujk1hy,ujk+1ujkhy). In order to define αx and αy, we first define ex=harmod(Δ+xujk),Δxujk), and ey=harmod(Δ+yujk),Δyujk). Then the parameters αx and βy are defined as

αx=2[exΔxujk1],ifΔxujkΔ+xujkαx=2[1exΔ+xujk]ifΔxujk<Δ+xujkαy=2[eyΔyujk1],ifΔyujkΔ+yujkαy=2[1eyΔ+yujk]ifΔyujk<Δ+yujk
(5)

We now define the upsampling operator S at the center of half-size pixels in each dimension as

S(xj(θx),yk(θy))=Hjk(xj(θx),yk(θy)),
(6)

where xj(θx) = xj + θxhx, and yk(θy) = yk + θyhy, and θx, θy [set membership] [−1/4, 1/4]. Similarly, the downsampling operator D is defined as D(v)jkl=14[v(xj(θx),yk(θy))]. On account of the higher-order approximation, it is noted that the upsampling criteria is given by D ο S(u) = u + O(((hxhy)), and no longer identity. However, from a practical standpoint, the errors are negligible, and the signal is not degraded in a significant way. It is also noted that the operation of downsampling followed by upsampling is not reversible, and thus D ο SS ο D.

3 Results

In this section, we present results of upsampled images by applying the operator specified in Eq. 6. Figure 1 shows upsampled images obtained using bilinear, b-spline, and the edge-preserved upsampling operators. The respective PSNR values are quantified in Table 1. In order to calculate the PSNR, the original images were downsampled using bilinear interpolation and then upsampled using each of the upsampling operators. The edge-preserving piecewise hyperbolic operator not only outperforms bicubic, and b-spline interpolation, but the resulting upsampled images appear sharper, and show better edges. Finally, the edge-preserving operator is also used to upsample color images as shown in Fig. 2. Here, the color image was first separated into luminance (Y), and the two chrominance channels (UV), and each interpolation operator was applied separately to each channel, and then converted back to the RGB colorspace. It is observed that the edge-preserved upsampled images resolve sharpness and detail better compared to the bilinear, and spline based approaches.

Fig. 1
Upsampled images (×2) resulting from bilinear, b-spline, and edge-preserving interpolation applied to the downsampled image.
Fig. 2
From top: Upsampled images (×2) resulting from bilinear, b-spline, and edge-preserved interpolation applied to the downsampled original image. The same interpolation method is applied separately to each of the luminance and the two chrominance ...
Table 1
Comparison of PNSR (10log(2552IupsampledIorig2)) calculated from the L2 norm between the upsampled and the original image for each of the interpolation operators.

4 Discussion

The proposed piecewise hyperbolic operator ensures that the upsampled reconstructions are smooth inside each pixel, and limits oscillations and jump discontinuities located at the pixel interfaces. From the experimental results obtained after applying our edge-preserving operator for interpolation, we observe that this reconstruction error is further lowered compared with both bilinear, bicubic, as well as b-spline interpolates. The operator is simple to implement, and executes in 0.2 s for an average image size of 256×256 pixels in MATLAB on an Intel 2.4 GHz platform. We anticipate the usefulness of this operator in routine image processing as well as computer vision tasks, where the preservation of edge quality is of primary importance.

Acknowledgments

This work is supported in part by NIH grants RC1MH088194, and P41 RR013642. Additionally, Dr. Antonio Marquina gratefully acknowledges the support from the NSF grants DMS-0312222, ACI-0321917, the NIH grant G54 RR021813, as well as DGICYT MTM2008-03597 from the Spanish Government Agency.

5 References

[1] Fattal R. Image upsampling via imposed edge statistics. ACM Transactions on Graphics. 2007 Aug;26(3):95.1–95.7.
[2] Mishiba K, Suzuki T, Ikehara M. Edge-adaptive image interpolation using constrained least squares; 17th IEEE International Conference on Image Processing (ICIP), 2010; 2010.pp. 2837–2840.
[3] Casciola G, Montefusco LB, Morigi S. Edge-driven image interpolation using adaptive anisotropic radial basis functions. J Math Imaging Vis. 2010 Jan;36(2):125–139.
[4] Glasner D, Bagon S, Irani M. Super-resolution from a single image; 12th IEEE International Conference on Computer Vision; 2009.pp. 349–356.
[5] Harten A. High resolution schemes for hyperbolic conservation laws. Journal of Computational Physics. 1983;135(2):357–393.
[6] Marquina A. Local piecewise hyperbolic reconstruction of numerical fluxes for nonlinear scalar conservation-laws. SIAM J Sci Comput. 1994 Jan;15(4):892–915.
[7] Van Leer B. Towards the ultimate conservative difference scheme II. monotonicity and conservation combined in a second order scheme. Journal of Computational Physics. 1974;14(4):361–370.
[8] Serna S, Marquina A. Power ENO methods: a fifth-order accurate weighted power ENO method. J Comput Phys. 2004 Jan;194(2):632–658.