Dehazing algorithm using adaptive dark channel fusion and sky compensation

LU Xinxuan，YANG Yan

(School of Electronic and Information Engineering，Lanzhou Jiaotong University，Lanzhou 730070,China)

Abstract：Aiming at the inaccurate transmission estimation problem of dark channel prior image dehazing algorithm in the sudden change area of depth of field and sky area,a dehazing algorithm using adaptive dark channel fusion and sky compensation is proposed.Firstly,according to the characteristics of minimum filtering of large window scale and small window scale in the dark channel prior,the fused dark channel is obtained by weighted fusion of the approximate depth of field relationship,thus obtaining the primary transmission.Secondly,use the down-sampling to optimize the primary transmission combined with gray scale image of haze image by fast joint bilateral filtering,then restore the original image size by up-sampling,and the compensation of the Gaussian function is used in the sky area to obtain corrected transmission.Finally,the improved atmospheric light is combined with atmospheric scattering model to recover haze-free image.Experimental results show that the algorithm can recover a large amount of detailed information of the image,obtain high visibility,and effectively eliminate the halo effect.At the same time,it has a better recovery effect on bright areas such as the sky area.

Key words：dark channel prior；approximate depth of field；weighted fusion；fast joint bilateral filtering；Gaussian function compensation

0 Introduction

In outdoor conditions,imaging systems are often susceptible to scattering of suspended particles and droplets in the air,resulting in poor visibility,color distortion,loss of detail and blurring of the obtained image[1],directly affecting people’s daily life.Therefore,obtaining clear and haze-free images has important practical value,and it is a research hotspot of digital image processing[2].

The current image dehazing methods are mainly divided into three categories:one is based on the image enhancement method[3-5],which can effectively improve the contrast of haze images without considering the factor of image degradation under haze condition,and highlight the characteristics of the specific area of the scene.But it does not take into account haze imaging models,the enhanced effect is not ideal,such as Retinex algorithm[3]and other traditional image contrast enhancement algorithms[5].The second method is based on the image restoration method[6-10],which analyzes physical process of image degradation based on the physical imaging model under haze condition,and proposes hypothesis or prior information to model and analyze it.In Ref.[6],a dehazing method based on dark channel prior information was proposed.Rough transmission of haze image was estimated by dark channel prior,and repaired and optimized by soft-matting algorithm.This method has physical validity,but there are problems such as high time complexity,incomplete haze removal and color distortion in the sky area.Ref.[7] removed haze by maximizing local contrast,which could restore image detail and structure to the greatest extent,but color oversaturation usually occured in restored images.Ref.[8] assumed that the local variation of atmospheric dissipation function is gentle.The white balance processing was applied to haze image to simplify atmospheric scattering model,and median filtering was used to estimate atmospheric dissipation function,but edge information remained so poor.In Ref.[9],the transmission constraint and the constraint relationship between adjacent pixels were combined to increase constraint conditions,so image dehazing method with boundary constraints was proposed.But the algorithm complexity is high.Ref.[10] optimized dehazing by local atmospheric light and atmospheric dissipation function.This method improves the inaccuracy of global atmospheric light in the dark channel prior algorithm,but distortion occurs in some parts of images.The third method is the machine learning method[11-12],which acquires model by training sample to achieve a certain dehazing effect.The other classical algorithm is the end-to-end algorithm proposed in Ref.[11],but the effect is unstable and has certain limitations.

Aiming at the problems that dark channel prior algorithm overestimates the transmission of edge region and sky area does not satisfy dark channel prior,this paper proposes a dehazing algorithm using adaptive dark channel fusion and sky compensation.(1)Considering that the expected effects of dark channel prior at different pixel scales are inconsistent,weighted fusion is used to image dark channel of the two scale templates in order to obtain primary transmission;(2)Theoretical edge function is used to optimize transmission,and the fast joint bilateral filtering is used to eliminate texture effect.The effect is to compensate for sky area to further estimate accurate transmission;(3)The improved global atmospheric light is used as atmospheric light value to avoid the influence of bright objects in the image;(4)Restore a haze-free image by atmospheric scattering model.Experimental results show that the algorithm can effectively prevent halo effect and color distortion,and it can obtain better dehazing effect in the depth of image.

1 Dark channel prior theory

1.1 Atmospheric scattering model

In the field of computer vision and graphics,atmospheric scattering model for imaging systems under haze condition is[13]

I(x)=J(x)t(x)+A(1-t(x)),

(1)

whereI(x)is haze image;J(x)is clear image;Ais atmospheric light value;t(x)is transmission.On the right side of Eq.(1),the first termJ(x)t(x)is direct attenuation,and the second termA(1-t(x))is atmospheric dissipation function.For transmissiont(x)=e-βd(x),d(x)is depth of field,βis atmospheric scattering coefficient.

1.2 Dehazing method of dark channel prior

Dark channel prior provides a new solution to image dehazing,which is based on a large number of outdoor haze-free images.That is,in most areas of outdoor haze-free images (non-sky area),the pixel values of at least one of the three color channels (R,G,and B)are very low to get

(2)

Jdark(x)→0,

(3)

whereJc(y)is the component of haze image in three color channels;Ω(x)is square filter window centered on the pixel pointx,which is usually taken as 15×15 pixel scale.So dark channel prior takes minimum channel for haze image at the first,and then performs minimum filtering.

Assuming that atmospheric light value is known,rough transmission can be obtained based on the atmospheric scattering model and dark channel prior as

(4)

whereIdark(x)is dark channel operation of haze image;ωis constant for setting the fidelity of restored image to prevent haze from being completely removed,and usually set as 0.95.

Since dark channel prior algorithm uses minimum filtering operation and the transmission at the sudden change of depth of field is underestimated,so the obtained clear image will have significant block effect at the sudden change area of depth of field.He et al.[6]further optimized the rough transmission using soft mating algorithm to obtain better edge retention.However,soft mating algorithm is computationally intensive and cannot meet the requirements of real-time processing.It is optimized by simple and fast guided filter algorithm[14].Although the running time is greatly reduced,there is still halo effect.

At the same time,when haze imageI(x)contains bright areas such as sky region,the pixel values are higher in this area,which results in a higher dark channel valueIdark(x)in this area approaching atmospheric light valueA,so the transmission value is seriously underestimated by Eq.(4).Furthermore,the use of Eq.(1)causes the restored image to be excessively amplified,resulting in noise and color distortion in this area.

2 Proposed algorithm

For dark channel prior algorithm,there are halo effect at the sudden change area of depth of field and color distortion problem in the sky or bright area.This paper proposes a dehazing algorithm using adaptive dark channel fusion and sky compensation,which repairs overestimated transmission region according to the difference between large pixel scale and small pixel scale in dark channel prior.The algorithm flow chart is shown in Fig.1.

Fig.1 Algorithm flow chart

2.1 Dark channel fusion

In dark channel prior theory,when image is operated by large pixel scale module,the probability that the module contains dark pixels will increase.It can effectively eliminate the interference of white objects or strong light source pixels in the image,thereby estimating the relatively accurate atmospheric light value.And the recovery effect is better.However,the assumption that the transmission is locally constant will fail,and the restored image will have more severe halo effect in the depth of field.

On the contrary,when image is operated by small pixel scale module,restored image will not have too obvious halo effect.But the dark channel estimation is inaccurate,and the transmission estimation is small overall,so that the restored image may be supersaturated.At the same time,the module contains a large number of white objects.If atmospheric light value estimation method proposed in Ref.[6] is directly used,it will cause wrong estimation.

Considering the above factors,this paper proposes a method combining dark channel priors to avoid the halo effect in the sudden change area of depth of field.And the atmospheric light value is estimated more accurately,restored image has ideal brightness.In order to combine dark channel priors with two different pixel sizes to obtain more accurate transmission,a weighted fusion is used as

(5)

(6)

According to Ref.[15],the relationship between depth of field information and brightness and saturation is proposed.Here,the brightness and saturation of the image are selected to estimate the approximate depth of field information to define the weighted fusion coefficienta2(x)as

(7)

(8)

whereVs(x)is approximate depth of field;V(x)is brightness component of haze image;S(x)is saturation component of haze image;minis minimum filtering,and the filter window size is 35×35;a2(x)is Gaussian weighting coefficient;μis expected value;σis standard deviation,and the effect is best whenμandσare taken as 0.5.

For the normalized haze image,approximate depth of field histogram is shown in Fig.2(g).The parameterμcontrols the center position of Gaussion function.Due to the greater brightness and less saturation in distant areas,the pixel value distribution is usually concentrated between 0.2 and 0.6,and the part smaller than 0 inFig.2(g)is set to 0.While the close areas are the opposite,the pixel value distribution at close areas is usually between -0.9 and -0.7.

Fig.2 Weighted coefficient a2(x)analysis

In summary,using the solid Gaussian model ofμ=0.5 in Fig.2(h)and Fig.2(i)to ensure attenuation speed is appropriate,a larger weight can be selected at distant fields,so that the weighting function can be estimated.Large pixel scale dark channel is taken in places with large depth of field,small pixel scale dark channel is taken in places with small depth of field,and Gaussian function is used to approximate dark channel prior.

When the pixels are distributed in the close areas and the sudden change areas of depth of field,the value ofVs(x)is small,so the value ofa2(x)is small,and vice versa.The contrast of dark channel is shown in Fig.3.As seen from Fig.3(e),dark channel for the region of depth of field is severely underestimated,resulting in serious halo effect in the restored image of Fig.3(f).It can be seen from Fig.3(g)that edge fusion dark channel has good performance and distant view areas avoid the unreasonable situation of dark channel.However,there is slight halo effect due to the use of 5×5 minimum filtering operation at edge areas.

Assuming that atmospheric light value is known,the primary transmission can be obtained according to Eq.(1)as

(9)

Fig.3 Dark channel comparison

2.2 Optimized transmission

Because restored image still has a little halo effect via minimum filtering operation,and it is seen from Fig.3(h)that some parts are unnaturally restored.At the same time,the large pixel scale dark channel has some incompatibility to the image containing large areas of sky,as shown in Fig.4(a)and Fig.4(b).That is,the sky area is located at the distant regions,and the alternating non-sky area will have serious block effect.

To further remove halo effect and refine contour,median filtering is used to quickly estimate edges.Median filtering is a kind of nonlinear filtering,which can better preserve edge information and suppress noise.Therefore,median filtering is used to optimize contour as

(10)

(11)

whereE(x)is edge information function;θtakes 0.95;W(y)is minimum channel of haze image;medianis median filtering;ρ(x)is median filtering window size of 3×3;minis minimum filtering;max represents taking the maximum;min represents taking minimum;Ω2(x)is the same size as fusion window in the previous section;C(x)is the linear constrained minimum channel of haze image to approximate minimum channel of haze-free image.

Then,primary transmission is optimized by edge information function as

t″(x)=t′(x)-E(x),

(12)

wheret″(x)is primary transmission of edge optimization.It can be known that transmission is only related to atmospheric scattering coefficient and depth of field,and independent of detail information,which is constant in a small local area.So it needs to remove texture effect and make it smooth.

A fast optimization algorithm for joint bilateral filtering is proposed here to further improve operation speed.The gray scale image of original image is subjected to down-sampling with sampling rate of 0.2,the down-sampling images are subjected to joint bilateral filtering,and the filtered image is subjected to a double sampling operation of bicubic interpolation algorithm,and the rough transmissiontr(x)is obtained.

Gδ2(‖x-y‖)g(x),

(13)

wheret″(x)is primary transmission of down-sampling;g(x)is gray scale image of down-sampling;GδrandGδsare Gaussian functions;δrandδsare the scale of Gaussian function in value domain and the scale of Gaussian function in spatial domain,respectively;Hxis normalization coefficient,which is the sum of the weights ofΦ(x)regions centered on the pixelx.

(14)

After fast joint bilateral filtering process,the rough transmissiontr(x)is smoother,and there is no halo effect at the sudden change area of depth of field.The contrast effect is shown in Fig.4.

Fig.4 Optimized comparison

2.3 Sky compensation

In the dark channel prior,transmission is underestimated in the sky region,tr(x)is deviated from true transmission in the sky area.So a certain offset compensation is performed fortr(x)as

t(x)=tr(x)+φ(x)tr(x),

(15)

wheret(x)is accurate transmission;φ(x)is compensation function,and the range is [0,1].Whenφ(x)is used to compensate for non-sky area,t(x)will increase excessively in the non-sky area,which will result in incomplete dehazing of restored image.Whenφ(x)is used to compensate for sky area,t(x)will increase moderately in the sky area,thereby eliminating the failure of dark channel prior to sky area.Therefore,in order to prevent the phenomenon oft(x)from being incomplete in the non-sky area,the adaptive Gaussian function is adopted as

φ(x)=e-((1-W(x)/A)2/λ2),

(16)

whereW(x)is minimum channel;λis standard deviation,and the range is [0,1].After a lot of experiments,setλas 0.15;φ(x)is estimate of the degree of image compensation using the (1-W(x)/A)Gaussian function to distinguish sky region and non-sky region.In the sky area,(1-W(x)/A)is smaller,adaptively setφ(x)to a larger value.Conversely,in the non-sky area,(1-W(x)/A)is larger,adaptively setφ(x)to a value close to zero.The compensation degree of sky region and non-sky region can be distinguished,so that transmission of sky region can be adaptively compensated.As seen from Fig.5(e)and Fig.5(f),the color of the latter is more natural in the sky region.

Fig.5 Transmission before and after compensation

2.4 Atmospheric light value estimation

Atmospheric light valueAis an important parameter based on dehazing algorithm of atmospheric scattering model.Its accuracy directly determines the quality of restored image.The global atmospheric light approximates the pixel value of the most opaque area in the haze image.In Ref.[6],the first 0.1% brightest pixel in the darkest region of dark channel image was used,and the corresponding maximum pixel value in the haze image was selected as atmospheric light value.Ref.[9] selected maximum value of the three color channels as atmospheric light value.However,when the image contains a strong light source,a white object or an image whose pixel value is larger than the pixel value of the sky region,there is a certain error,thus it should not be used as atmospheric light value.

In order to solve the problem that the high-brightness pixel image causes an erroneous estimation when atmospheric light value is selected,and the restored image is relatively dim,in this paper,an improved method for the atmospheric light value selection proposed by He et al.[6]is used to improve the accuracy of atmospheric light value estimation as

(17)

Fig.6 Estimation of atmospheric light

2.5 Image restoration

According to the obtained transmissiont(x)and atmospheric light valueA,the final clear imageJ(x)can be recovered as

(18)

It is impossible to completely avoid the introduction of noise during the imaging process.To prevent noise amplification,set the lower bound of transmissiont0=0.1 to avoid distortion in the restored image.

3 Experimental results and analysis

In order to verify recovery effect of the proposed algorithm,the current classic image dehazing algorithms are compared with the proposed algorithm in subjective and objective aspects.All algorithms are implemented in Matlab language,the operating system is 64-bit Window7,the program running environment is Matlab R2016a,and the computer is configured as Intel(R)Core(TM)i5-6300HQ CPU@2.30 GHz,4 GB memory.

3.1 Subjective evaluation

Subjective evaluation can most intuitively reflect the human eye’s feelings about recovery effect.Some image dehazing algorithms are selected for comparative analysis,mainly including the dark channel prior algorithm proposed by He et al.[6],the local atmospheric light algorithm proposed by Sun et al.[10],the end-to-end system algorithm proposed by Cai et al.[11],and the linear transmission algorithm proposed by Wang et al.[16].Experimental effects are shown in Fig.7.

In this paper,representative images are selected for comparative analysis of different scene features.Fig.7 shows the comparison of dehazing effects of different algorithms.Among them,Fig.7(a)is haze image and Fig.7(b)is dehazing effect of dark channel prior algorithm.The detail of restored image is obviously increased,and the haze of image is basically removed,but the dehazing is not complete in the sudden change area of depth of field.As the image 1 and image 4 shown in Fig.7,there is obvious residual haze at the sudden change area of depth of field,and the color of restored image is dark,which is caused by inaccurate estimation of atmospheric light value.As the image 2 a nd image 4 in Fig.7,the color distortion occurs in the sky area.As the image 3 in Fig.7,the restored image is significantly darker.Fig.7(c)shows dehazing effect of local atmospheric light algorithm.The contrast of the restored image is obviously improved,and the effect of eliminating haze is also good,but color distortion phenomenon occurs in the scene containing sky area.As the image 2 and image 4 in Fig.7,cloud color is seriously distorted in the sky area,which is mainly caused by morphological operation of atmospheric dissipation function,the estimation is not accurate,and the restored image is saturated.Fig.7(d)shows dehazing effect of the end-to-end system algorithm.This method uses the convolutional neural network method to train the transmission map with the end-to-end system to achieve optimal effect.It has certain algorithm superiority and can recover certain details,but the phenomenon of incomplete dehazing exists,and the effect of dense haze image processing is not ideal,as the image 4 and image 5 shown in Fig.7.The Fig.7(e)shows dehazing effect of the linear transmission algorithm,which can restore a lot of image details,but there is still phenomenon of incomplete dehazing and color distortion for some images.

Fig.7 Comparison of experimental results

Fig.7(f)shows dehazing effect of proposed algorithm.Compared with the above four algorithms,the details of image restoration are significantly increased in proposed algorithm,the overall brightness and saturation are better,the color of close regions is natural,and the dehazing result of distant regions is thorough.At the same time,the recovery effect in the sky area is relatively good,and the textures are clear.Further,it can be seen from the comparison of Fig.8 that the image restored by the algorithm in this paper has better sense of hierarchy and visual effect.Compared with the He’s algorithm,residual haze is removed in the areas of sudden depth of field and the distortion of sky area is better handled.Compared with the Sun’s algorithm,proposed algorithm eliminates the color distortion of sky area without over saturation and has higher contrast.Compared with the Cai’s algorithm,the algorithm in this paper recovers more details of the images and removes haze more thoroughly.Compared with the Wang’s algorithm,the details of scene increase significantly while maintaining the brightness of these images.

Fig.8 Comparison of detail effect

3.2 Objective evaluation

Subjective evaluation often has some limitations,in order to further verify the effectiveness of proposed algorithm,objective evaluation is used to explain.In this paper,an image quality evaluation method without reference[17-18]is used.The number of visible edge set (d),average gradient (r),measurement of image visibility (IVM)and running time (T)[19]are taken as objective evaluation indexes.The larger value ofd,rand IVM,the better quality of restored result,and the smaller ofT,the better quality[20].

The corresponding mathematical representation is

(19)

(20)

wheren0is the number of visible edges of haze image;nis the number of visible edges of recovered clear image;riis the gradient ratio of haze-free image and haze image atPi;φris the visible edge set of haze-free image.Experimental images are in Fig.7,and experimental result data comparison is shown in Fig.9.

Fig.9 Objective evaluation

The image 1-image 5 in Fig.9 represent the haze image in Fig.7,respectively.As seen from Fig.9(a),proposed algorithm basically obtains good results,but it is slightly insufficient compared to the Sun’s algorithm.From Fig.9(b),it can be seen that the Sun’s algorithm also shows its advantages,but the overall performance of proposed algorithm is better than other algorithms.From the measurement of image visibility in Fig.9(c),proposed algorithm is better than other algorithms.From the comparison of running time of Fig.9(d),it can be seen that compared with other classical algorithms,running time of proposed algorithm has large advantage.So dehazing effect of proposed algorithm is relatively better and has certain superiority.

4 Conclusions

Aiming at the problem that dark channel prior algorithm is not accurate in the estimation of depth of field and sky area,this paper proposes a dehazing algorithm using adaptive dark channel fusion and sky compensation.The method starts from dark channel prior of two sizes,adopts approximate estimation of depth of field change,and performs weighted image fusion to correct the corresponding transmission regions.At the same time,the bilateral filtering combined with up-sampling and down-sampling fast operation optimizes the transmission and performs Gaussian compensation on the sky region.Finally,atmospheric scattering model is used to obtain a clear image.Experimental results show that proposed image restoration algorithm can effectively remove the influence of haze,and color fidelity has better performance.Image details are obviously increased,and color distortion of sky region can be effectively prevented.The objective evaluation also shows certain advantages.