Super-Resolution Image Reconstruction Based on an Improved Maximum a Posteriori Algorithm

Fangbiao Li, Xin He, Zhonghui Wei, Zhiya Mu and Muyu Li,2(.Department of Photoelectric Measurement and Control, Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 30033, China; 2.University of Chinese Academy of Sciences, Beijing 00049, China)

Super-resolution (SR) image reconstruction produces high resolution imagesfrom low resolution images using image enhancement techniques[1]. With the focal plane array (FPA) technology approaching the best imaging limit, SR image reconstruction plays an important role in many applications, such as video surveillance[2], criminal investigation analysis[3-4], medical image processing[5]and satellite imaging[6].

SR image reconstruction is developing rapidly and a variety of algorithms have been proposed, such as the iterative back-projection (IBP) algorithm[7], projection onto convex sets (POCS) algorithm[8], maximum likelihood (ML) algorithm[9]and maximum a posteriori (MAP) algorithm[10].The MAP algorithm has received much attention because of its advantages, such as complete theory framework, flexible spatial domain observation model, powerful inclusion of apriori-knowledge and producing superior results[11]. However, the traditional MAP algorithm cannot reconstruct a precise image due to the estimate of shifting.

In this paper, we introduce an improved MAP algorithm which applies image registration in the framework but separate it in process of iteration to minimize the influence of shifting estimate. Experimental results indicate that the proposed method can enhance image resolution efficiently.

1 Observation Model

The observation model describes the connection between a high resolution (HR) image and a LR image[12], it can help us to understand the theory of SR image reconstruction, as shown in Fig.1.

The observation model can be denoted as

y=DMBx+n

(1)

whereyis the LR image,xis the unknown HR image,Dis the down-sampling operator,Mis the motion operator,Bis the blurring function, andnis the additive noise.

Fig.1 Degradation process of imaging

2 MAP Algorithm

From the observation model, many image details are lost during degradation process; therefore, SR reconstruction is an ill-posed problem. MAP algorithm can provide a flexible and suitable way to model a priori knowledge to constrain the solution. What we should do is to maximize the posteriori probability Pr (x|y) and estimatewith the following equation as[13]

(2)

After apply the theorem of Bayesian, the estimatecan be expressed as

(3)

Because the LR imageydoesn’t affect the reconstruction of the SR image in the framework of MAP, we can get the optimization problem by substituting the logarithmic function to the expressions

=argPr (y|x)+lg Pr (x)}

(4)

where Pr (y|x) is the conditional density model that can be defined as

(5)

and Pr (x) is the prior model that can be defined as

(6)

So the optimization problem can be expressed as

(7)

When we find the solution of this minimization problem, we can get the estimate image. From the observation model of Fig.1, the processes of SR image reconstruction include three steps: image registration, non-uniform interpolation and reconstruction. We can get the parameters of shifting and blurring with the traditional MAP algorithm. Then, we use the parameters to reconstruct the SR image. However, the estimation of shifting and blurring interacts with SR image reconstruction. We can’t get a high precision solution of the shifting parameter if the HR image is unknown. Therefore, we propose an improved MAP algorithm which applies image registration in the framework but separates it in process of iteration to minimize the influence of shifting estimate. In other words, we update the shifting parameter in the process of SR image reconstruction constantly. The framework of the algorithm can be expressed as

,=argPr (y|x,s)+lg Pr (x)+
lg Pr (s)}

(8)

wheresis the shifting parameter. Based on Eq.(6), the optimization problem of the improved algorithm can be expressed as

(9)

We first assume a coarse shifting parameter, and use Eq.(6) to find a. After getting the estimated, we can find a more precision solution of the shifting parameter by use the following formula as

(10)

When we get a more precision solution, a better estimation ofcan be found by using this solution and the best image can be got by repeating the process.

3 Experimental Results and Analyses

In order to demonstrate the effects of the improved MAP image reconstruction algorithm, we compare the performance of our method with that of two other image reconstruction methods (i.e., bilinear interpolation and traditional MAP) by using two parallel experiments. The resolution of the LR images in the visible image experiment is 102×102 pixels and infrared image experiment is 64×51 pixels. The experimental results can be seen from Fig.2 and Fig.3.

Fig.2 Results of visible image by using different SR image reconstruction methods

Fig.3 Results of infrared image by using different SR image reconstruction methods

Fig.2 and Fig.3 illuminate the effects of different SR reconstruction methods for visible and infrared images. The resolutions of the reconstructed visible and infrared images are 204×204 pixels and 128×102pixels respectively.The effects of image reconstruction by different algorithms (Fig.2 and Fig.3) show that the proposed method can enhance image resolution efficiently. The edges and details of the reconstructed images are expressed faultlessly.

Furthermore, in order to fully evaluate the quality of reconstructed images, various quality metrics such as peak signal to noise ratio(PSNR) and structure similarity image measure(SSIM) would be used. PSNR evaluated gray similarity and SSIM evaluate structure similarity between two images. We can evaluate the algorithm efficiency by using PSNR and SSIM simultaneity.

From the data presented in Tab.1 and Tab.2, higher SSIM and FSIM values from the proposed approach indicate that the structure and features are recovered to a large extent. From Tab.1, the PSNR of the improved algorithm increases 1.39 dB for the visible image and 1.52 dB for the infrared image compared with that of the traditional algorithm. From Tab.2, the SSIM increases 0.019 6 and 0.026 1 respectively. The improved algorithm performed the best on the image reconstruction and can enhance image resolution efficiently.

Tab.1 PSNR(dB) of different reconstructed methods in visible and infrared images

Tab.2 SSIM of different reconstructed methods in visible and infrared images

4 Conclusion

In this paper, we present an improved MAP algorithm to reconstruct SR images. The algorithm takes full account of the relationship between image registration and reconstruction in the process of super resolution reconstruction. The method can minimize the influence of shifting estimate in the process of iteration to improve the performance of SR image reconstruction. Because the improved algorithm can replace the shifting parameter on time, the structure and features are recovered to a large extent in the reconstruction image. The improved performance of our algorithm is evident from the qualitative and analytical results. In addition, the computational cost of our algorithm is not increased compared with the traditional MAP algorithm.

[1] Whitney T, Straub J， Marsh R. Image enhancement using hierarchical bayesian image expansion super resolution[J]. Mobile Multimedia/Image Processing, Security, and Application,2015,8(17)： 9-17.

[2] Zhaoghao Karl, Chen Long, Yang Xusan, et al. Super-resolution dipole orientation mapping via polarization demodulation[J]. Light Science & Application, 2016，12(5)：1-8.

[3] Rajan D, Chaudhuri S. Simultaneous estimation of super-resolved scene and depth map from low resolution defocused observations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2003,5(25)： 1102-1112.

[4] Li Wei, Zhao Yibo, Ai Feizi, et al. Pixel super-resolution using wavelength scanning[J]. Light: Science & Application, 2016，8(5)： 1-11.

[5] Ashikaga H, Estner H L, Herzka D A, et al. Quantitative assessment of single-image super-resolution in myocardical scar imaging[J]. Medical Imaging and Diagnostic Radiology,2014,9(18)： 512-521.

[6] Droege D R, Hardie R C, Allen B S, et al. A real-time atmospheric turbulence mitigation and super-resolution solution for infrared imaging systems[J]. Infrared Imaging Systems: Design, Analysis, Modeling and Testing, 2012,8(3)：550-559.

[7] Irani M，Peleg S. Improving resolution by image registration[J]. Cvgip: Graphic Models & Image Processing,1991,5(3)： 231-242.

[8] Patti A J，Altunbasak Y. Artifact reduction for set theoretic super resolution image reconstruction with edge adaptive constrain and higher-order interpolation[J]. IEEE Transactions on Image Processing,2001,5(10)： 179-192.

[9] Cain S, Hardie R C， Armstrong E. Restoration of aliased video sequences via a maximum-likelihood approach[J]. Proc Nat Infrared Information Symposium on Passive Sensors,1996,12(5): 2517-2518.

[10] Zhang D, Li H F， Du M H. Fast MAP-based multiframe super-resolution image reconstrction[J]. Image and Vision Computing,2005,10(23)： 671-678.

[11] Panagiotopoulou A, Anastassopoulos V. Regularized super-resolution image reconstruction employing robust error norms[J]. Optical Engineering,2009,11(48)： 117004-117013.

[12] Li Fangbiao, He Xin, Wei Zhonghui, et al. Sub-pixel image registration based on super resolution reconstruction[J]. Optics and Precision Engineering,2017,2(5)：457-465.(in Chinese)

[13] Hardie R C, Barnard K J， Armstrong E E. Joint MAP registration and high-resolution image estimation using a sequence of undersampledimages[J]. IEEE Transactions on Image Processing,1997,10(6)： 1621-1628.