SAR Tomography with Improved Non-Local Means Filtering Based on Adaptive Window

Shenglei Wang, Zhiyang Chen, Yuanhao Li, Cheng Hu

Abstract: In order to mitigate speckle noise in synthetic aperture radar (SAR) images and enhance the accuracy of SAR tomography, non-local means (NL-means) filtering has been proven to be an effective method for improving the quality of SAR interferograms.Apart from considerations like noise type and the definition of similarity, the size and shape of filtering windows are critical factors influencing the efficacy of NL-means filtering, yet there has been limited research on this aspect.This paper introduces an enhanced NL-means filtering method based on adaptive windows,allowing for the automatic adjustment of filtering window size according to the amplitude information of the SAR interferogram.Simultaneously, a directional window is incorporated to align SAR interferograms, achieving the dual objective of preserving filtering standards and retaining detailed information.Experimental results on interferogram filtering and tomography, based on TerraSAR-X data, demonstrate that the proposed method effectively reduces phase noise while maintaining texture accuracy, thereby improving tomography quality.

Keywords: NL-means filter; adaptive window; SAR interferogram filtering; SAR tomography

1 Introduction

Synthetic aperture radar tomography (TomoSAR)extends the principle of synthetic aperture to elevation direction, utilizing multiple SAR images from different observation angle to achieve threedimensional high-resolution imaging [1].It can be applied to fields such as urban three-dimensional imaging and forest parameter inversion [2, 3].The quality of 3D imaging results is significantly contingent upon the quality of the SAR images employed in the imaging process.

SAR images are often contaminated by speckle noise due to the imaging principles, which can reduce the quality of SAR images and further affect the results of tomography.Therefore,before conducting tomography, the SAR images need to be denoised first.Common filtering methods include mean filtering, median filtering,wavelet transform filtering, Goldstein filtering and so on [4, 5].

The common speckle processing methods mainly include four categories: spatial speckle suppression algorithm, transform domain speckle suppression algorithm, anisotropic diffusion algorithm and non-local means algorithm.The spatial speckle suppression algorithm is the earlier algorithm used in speckle suppression, including the classical Frost filter [6], Kuan filter [7], etc.Transform domain speckle suppression algorithms are also widely used in speckle processing,including wavelet transform based speckle suppression methods [8], Shearlet transform based speckle suppression methods [9] and so on.anisotropic diffusion (AD) filter [10] is an anisotropic diffusion filter that considers both edge-preserving and speckle suppression.After improvement, it has also been applied to SAR image speckle suppression and has a good speckle suppression effect [11, 12].Compared with the above methods, the non-local means method is a new method for SAR speckle suppression, which has better edge and detail preservation effect,and is suitable for SAR images with complex structure.

Non-local means filtering is an emerging image denoising method proposed by Buades et al.in 2005 [13].The main idea of this algorithm is to utilize the high redundancy of natural images and the decorrelation of noise.The principle of NL-means filtering is to find patches obtain the filtered patch by weighted averaging of these similar patches.The NL-means algorithm excels in preserving image edges, textures,and other features.Over recent years, it has emerged as a prominent research focus in the realm of image denoising.Through numerous refinements, this algorithm has demonstrated exceptional efficacy in enhancing image quality by reducing noise.

Due to its effective denoising capabilities,the NL-means filter is increasingly applied in SAR image denoising.In the context of SAR image denoising, the enhancements to the traditional NL-means filter primarily focus on the following directions:

1) Improved the filtering algorithm based on the noise characteristics of SAR images.For example, Zhong et al.[14] adopted a pixel classification strategy to effectively reduce the influence of the multiplicative speckle model and improve the effectiveness of similar patch search,and designed a method to search rotation invariant similar patches by using orientation information.

2) Based on the statistical characteristics of SAR images, redefine the similarity in the algorithm according to different probability density functions.For example, based on the assumption that the speckle of SAR multi-view images follows the generalized gamma distribution,Deledalle et al.[15] used the probability similarity of image patches instead of Euclidean distance, and proposed the PPB speckle reduction algorithm based on the probability similarity of image patches.Zhong et al.[16] derived a SAR image speckle reduction algorithm based on Bayesian non-local means based on Bayesian framework, and proposed a statistical distance to measure the similarity of image patches, which has certain speckle reduction effect.

3) Based on the structural characteristics of SAR images, redefine the similarity in the algorithm according to characteristic parameters.For example, Yi et al.[17] considered the structural similarity of images and introduced the structural similarity index (SSIM) into the non-local means to suppress the speckle of SAR images and realize the optimization of edge-preserving characteristics.Chen et al.[18] proposed a SAR image speckle reduction algorithm based on nonlocal means and coefficient of variation, which also has good filtering and edge preservation characteristics.

In addition to the above several improvement ideas, the size and shape of the filtering window are also important factors affecting the denoising effect, but there has not been a systematic discussion and analysis.In terms of windows,the current problems of NL-means filtering mainly include:

1) The window size was fixed, and could not be adjusted according to the image characteristics.

2) The shape of the window is a fixed square window, which cannot accurately match similar patches according to the texture information of the image.

To solve these problems, this paper proposes an improved NL-means filtering method,which can adaptively adjust the size of the filtering window according to the amplitude information of the SAR interferogram.At the same time,the direction window is introduced to match the phase fringes, so as to achieve the purpose of preserving the filtering standard while retaining the detail information.

This paper is organized as follows.Section 2 mainly introduces the basic principle of NLmeans filtering.Section 3 introduces the specific process of the improved NL-means filtering algorithm proposed in this article, as well as the confirmation method of key parameters.Section 4 shows the filtering effect of the proposed NLmeans filtering method for real SAR interferograms, which verifies the good effect of the proposed method in detail preservation of SAR interferograms.Section 5 mainly shows the application of the improved algorithm proposed in this paper to SAR tomography, which verifies that using the SAR interferogram sequence filtered by the proposed method for tomography can indeed improve the quality of 3D imaging.

2 The Principle of NL-means Filtering

The main idea of non-local means filter is to utilize the intrinsic redundancy of natural images and the decorrelation of noise.In a typical natural image, various elements such as edges, points,and other features exhibit similarity in multiple regions within the same image [19].The process of NL-means filtering is to traverse all pixels of the whole image once for each pixel to be filtered.During this traversal, the algorithm assesses and quantifies the similarity, assigns weights based on this similarity, and subsequently computes a weighted average to derive the filtered pixel.The basic expression of the algorithm is as follows:

wherev˜(x) denotes the denoised image,v(y)denotes the noisy image,w(x,y) denotes the weight, andΩxdenotes the neighborhood of the pixelx.The determination of weight is the key of the algorithm.In the traditional NL-means algorithm, the weight is related to the Euclidean distance between two image patches, which can be expressed as [13]

In practical applications, the computation involved in traversing the entire image for similarity determination is notably extensive， and there are numerous patches exhibiting low similarity throughout the image, the computational efficiency is adversely affected.Hence, it is necessary to set the search window and similarity window to reduce the range of pixel traversal.

Fig.1 is the windows diagram of NL-means filtering.The blue block represents the patch to be filtered, the green block represents the patch used to compute similarity, and the large white window is the search window.During the filtering process, the green block systematically slides,pixel by pixel, within the search window.It calculates similarity with the blue block at each position and executes a weighted average to accomplish the filtering.In practical terms, the window size significantly influences the filtering outcome.According to the non-local means filtering principle, the size of the search window dictates the distance between the similar patch and the patch to be filtered.A larger search window encompasses patches from more distant regions,resulting in a more pronounced filtering effect.Generally, the recommended sizes for the search window and similarity window are denoted by|W|=21×21 and |Δ|=7×7 respectively.This window size not only exhibits robust noise-filtering capabilities but also excels in handling intricate details and fine structures [13].

Fig.1 The windows diagram of NL-means filtering

3 Improved NL-means Filtering Method

While the original NL-means filtering method demonstrates effectiveness in optical image processing, several enhancements are imperative for SAR image processing.The key areas for improvement include: 1) The original NL-means filter employs a fixed window size, devoid of adaptability to the unique characteristics of the image.Consequently, the filtering effect may not be consistently pronounced, and in certain instances, essential details may be inadvertently filtered, especially for images with varying precision; 2) During the computation of weights based on similarity, the substantial difference between the pixel slated for filtering and those within the search window can result in weight instability,thereby compromising the overall filtering effect;3) The fixed shape of the filtering window induces transitional filtering for structures in the image with a distinct directional stripe pattern.This can lead to the loss of detailed information due to blurring.Furthermore, the noise characteristics inherent in SAR interferograms differ from the additive noise found in optical images.Consequently, directly applying the original NLmeans filtering method to SAR interferograms may not yield optimal filtering results.

Addressing the aforementioned limitations,this paper proposes an improved NL-means filtering method based on the distinctive characteristics of the interferogram itself.The objective is to implement more profound filtering in regions with low signal-to-noise ratios, preserve original detail information in high signal-to-noise ratio areas, and mitigate the interaction between pixels exhibiting significant differences.The definition of the weight in the original NL-means filter is adjusted to enhance the weight stability.Additionally, the size and shape of the search window are adaptively adjusted, ensuring the retention of intricate structural details.This adaptive approach contributes to the overall improvement in the quality of the filtered SAR interferogram.The specific process of the improved NL-means filtering method is as follows:

Step 1 Input the SAR interferogram, and calculate the SNR for the entire image using the amplitude values derived from the complex image, referencing a designated noise region.

Step 2 Identify pixels for filtering based on the calculated SNR, the pixels whichSNR ≥15 dBdo not need to be filtered.

Step 3 Compute the correlation coefficient for the identified pixels according to the SNR,and determine the size of the search window according to the correlation coefficient, that is,the numberLof pixels participating in the final weighted average process.

Step 4 Select the appropriate direction window to align with the fringe direction and texture characteristics of the interferogram.

Step 5 Calculate the similarity between each pixel within the direction window and the target pixel slated for filtering.the target pixel slated for filtering, and filter out the topLmost similar pixels for weighted average to complete the filtering of a single pixel.

Step 6 Traverse all identified pixels across the entire image, executing the complete image filtering process.

3.1 Definition of Similarity

In actual SAR interferograms, the distribution of strong and weak scatterers is frequently non-uniform.During NL-means filtering, the pixel intensities within the search window can markedly differ from the intensity of the pixel slated for filtering.This discrepancy is particularly pronounced when filtering weak scatterers, where introducing excessive information from strong scatterers is undesirable and may lead to interference.Consequently, this paper introduces a modification to the weight calculation formula in the original NL-means filter.The aim is to address the issue of elevated weight assigned to strong scatterers due to the lower intensity of the central pixel slated for filtering.To mitigate the issue of excessively high weights assigned to strong scatterers, an enhancement is introduced to the original NL-means filter weight definition.This improvement incorporates a normalization process using the intensity of the central pixel‖v(x)‖2.The refined weight calculation formula, designed to alleviate the influence of the center pixel, is expressed as follows:

3.2 Introduction of Directional Windows

In traditional NL-means filtering, the prevalent window shape is predominantly rectangular.However, due to the pronounced directional characteristics of interference fringes in interferometric phase, the conventional rectangular filtering window may disrupt the continuity of fringes,particularly in densely packed fringe regions.The interferometric phase filtering algorithm based on directional window should adopt a directional window that fits the direction of the interference fringe.The pixel values within a directional window exhibit proximity, suggesting approximate homogeneity.Consequently, employing a directional window for filtering serves the dual purpose of noise reduction without compromising the integrity of interference fringes.

In the classical Lee filter proposed by Lee et al.[20], 16 kinds of directional window templates are used for filtering, and the quality of filtered images is successfully improved.The templates of these 16 directional windows are also introduced in the improved NL-means filtering algorithm proposed in this paper, as shown in Fig.2.During the filtering process, calculations exclusively involve the white pixels within these templates.

Fig.2 Directional window templates in classical Lee filtering

The selection of the directional window can be based on the complex interferometric phase,and the specific procedure is to calculate the mean value based on 16 directional windows in the window.Subsequently, the window with the highest amplitude is selected for further processing.

3.3 Adaptive Windows Size

3.3.1 Similarity Window Size Selection

For determining the similarity window, a simulation experiment approach is employed.Firstly,construct a simulation plane scene with phase 0,and add noise with different signal-to-noise ratios.Then, filter the noise by the combination of search window and similarity window with different sizes.Next calculate the RMSE value of the filtered phase to compare the filtering effect.

Fig.3 The trend of interferogram phase RMSE with filtering window for different SNR: (a) 0 dB; (b) 5 dB; (c) 10 dB

Fig.3 shows the changes of phase RMSE after filtering in the simulation plane with three kinds of noises added.According to the above experimental results, it can be seen that in NLmeans filtering, the influence of similarity window on filtering effect is much less than that of search window, and when the similarity window is increased to 3×3, the filtering effect is no longer significantly improved.Therefore, the method proposed in this paper does not focus on the change of the similarity window, which is fixed as a 3×3 rectangular window, and the subsequent adjustment of the size of the search window can meet the requirements of adaptive filtering.

3.3.2 Search Window Size Selection

The search window is determined by the probability density distribution of the interferogram phase.The size of the search window can be deduced by the probability density function of the interferogram phase when the threshold is set.In the SAR interferogram, the probability density function of the interferometric phase can be expressed as [21]

The simulation results of the variation trend of the phase standard deviation with the correlation coefficient under different looks are shown in Fig.4.

Fig.4 Trend of phase standard deviation with correlation coefficient

It can be seen from Eq.(4) that the probability density function of the interferometric phase is related to the looksLand the correlation coefficientγ, where the looksLis a key parameter used in multi-look processing.The multi-look process is the process of averaging the range or azimuth directions of the SAR image, so the looksLcan be considered as the number of pixels involved in the average, which is similar to the search window size in NL-means filtering.In NL-means filtering, the size of the search window is the number of pixels that eventually participate in the weighted average; therefore, the looksLcan be considered equivalent to the size of the search window in NL-means filtering.

In the algorithm proposed in this paper, the function of adaptively improving the window size need to be realized by the amplitude information of the SAR interferogram.Therefore, the correlation coefficient in the above derivation needs to be associated with the SNR of the SAR image.The correlation coefficientγhas the relationship with SNR as following functional:

On the basis of the above derivation, the phase standard deviation at SNR=15 dB is used as the threshold to simulate the variation trend of the phase standard deviation with the looks in the case of different SNR, as shown in Fig.5.It can be seen that in the case of different SNR, as long as the looks is increased to a certain value,the phase standard deviation can reach the standard of SNR=15 dB.According to this feature,the looksLcan be equivalent to the size of the search window, that is, the number of pixels participating in the weighted average, and finally the purpose of adaptively determining the search window can be achieved.

Fig.5 Trend of phase standard deviation with looks at different SNR

In order to verify the equivalence between the size of the search window and the looksL,the simplest weighted average filtering method,mean filtering, is used to filter the simulation noise plane.The value of the mean filtering window is set to the value of the looksLwhen the SNR reaches the standard of SNR=15 dB, and test whether the filtering effect of noisy images with different SNR is up to the standard.

The results of the mean filtering are shown in Tab.1.The simulation shows that when SNR =15 dB, RMSE = 0.404 1.Through the experiment, it can be seen that the phase RMSE can basically reach the standard after the filtering window size is selected according to the simulation looks, and the RMSE after filtering is slightly higher than the standard value because the direction of the filtering window is fixed, butit can still be judged that the looksLis equivalent to the size of the filtering window.

Tab.1 Comparison of mean filtering results for noisy images with different SNR

3.4 Determination of Filter Coefficients h

The main parameter in filtering also includes the filtering coefficienth, which needs to be further analyzed on the basis of the improved weight calculation formula.

As we know from the previous derivation, if the intensity of the center pixel is much larger than that of the surrounding points, it will not be filtered.Therefore, this paper focuses on the case that the center pixel is much smaller than the surrounding pixels, that is, the weight calculation method whenb ≫1, and the improved weight formula can be simply written asw2=exp(-b2/h2).

Whenb ≫1, the intensity of the central pixel is much smaller than that of the surrounding pixels.In this case, the product of the corresponding weight of the surrounding pixels and the pixel should be much smaller than that of the central pixel, so as to reduce the influence of the surrounding pixels on the central pixel.The expression is as follows:

After simplification, the parametershandbactually need to meet the following conditions:

The effect of changing the parameterhon the weight calculation is shown in Fig.6.Through the simulation results of the influence ofthe change of parametershandbon the weight,it can be seen that when the parameterh≤2,the condition of Eq.(9) can be satisfied.In NLmeans filtering, the parameterhrepresents the filtering degree, therefore, the larger the value ofh, the more obvious the filtering effect.In the case of satisfying the condition, we still want to obtain more obvious filtering effect.Therefore, in this paper, the value of parameterhis determined as 2.

4 Interferogram Filtering Experiments

In order to verify the effectiveness of the proposed method for filtering SAR interferograms, in this section, a measured interferogram is filtered and the filtering results are compared.The methods adopted include the original NL-means filtering, the NL-means filtering with directional window only and the improved NL-means filtering proposed in this paper.

The original data of the experiment is SAR interferogram generated by two pre-processed(registration, deramping, phase compensation)TerraSAR-X images.The imaging area is the Regent Hotel and some surrounding buildings.The slant range resolution is 0.45 m, azimuth resolution is 0.85 m, range bandwidth is 300 MHz,and perpendicular baseline length is 21.24 m.The specific acquisition parameters are given in Tab.2.

Tab.2 TerraSAR-X acquisition parameters

4.1 Interferometric Phase Filtering Effect

The before-after comparison of the interferometric phase obtained by different filtering methods is shown in Fig.7.From the comparison of interferometric phase before and after filtering, it can be seen that the filtering effect of the original NL-means filtering is very obvious, but it will lead to the destruction of structural information.At the location of the building, the NL filter completely destroyed the details, and the results were improved after adding Lee window, but the details could not be completely preserved.On this basis, adding window filter could achieve the purpose of preserving details as much as possible under the condition of meeting the filtering effect requirements (phase RMSE of 15 dB).

Fig.7 Comparison of phase filtering effect of SAR interferogram in the vicinity of Regent Hotel: (a) before filtering; (b) original NLmeans filter (Search window 9×9); (c) NL-means filter with directional windows; (d) NL-means filter proposed in this paper

In order to more clearly show the effect of the proposed method on the interferometric phase filtering, the zoom of line feature of the interferometric phase before and after filtering are intercepted for comparison.As we can see in Fig.8, a line target is circled in the black box.The phase standard deviation of the line target is before filtering is 1.78, and which after filtering is 1.75.It can be seen that the phase continuity on the filtered line target is better, and the phase noise is effectively filtered without destroying the structural information of other surrounding targets.

4.2 Suppression of Interference from Strong Scatterers

In addition to has a better filtering effect on the interferometric phase, the proposed method can also suppress the strong scatterers point spread phenomenon.In order to verify the suppression effect of the proposed method on the interference of strong scatterers, the amplitude maps under different filtering cases were obtained for comparison.The before-after comparison of the interferogram amplitude is shown in Fig.9.As can be seen from the unfiltered amplitude, there are many strong scatterers points within the scope of the Regent Hotel building.The reason is that the building has many glass Windows on the facade, and the metal structure of the glass window frame will form the corner reflector effect, so the scattering coefficient is very high.The strong scatterer interference suppression performance of the algorithm can be evaluated by comparing the weak scatterer filtering case within the scope of the Regent Hotel building.

Fig.8 Zoom of line feature before and after filtering: (a) before filtering; (b) filter proposed in paper

By observing the SAR interferogram amplitude of the Regent Hotel before and after filtering, it can be seen that the original NL-means filtering will lead to a relatively serious diffusion effect.The addition of Lee window has a certain suppression effect on the diffusion effect, but it is also quite serious.However, the diffusion of strong scattering points is effectively suppressed in the results of the improved filtering method,which also indicates that it is effective to select filter points according to the search window.

Fig.10 shows the comparison of coherence map before and after filtering.The mean of the correlation coefficient before filtering is 0.48, and which after filtering is 0.53.It can be seen that after filtering, the correlation of all imaging areas is improved, especially the building area, which also proves the effectiveness of the filtering method proposed in this paper.

Fig.10 Coherence map before and after filtering: (a) before filtering; (b) filter proposed in paper

In order to quantitatively analyze the filtering effect of the algorithm, Tab.3 shows the comparison of the edge-preserving index (EPI)and the equivalent number of looks (ENL) in the flat region of the phase before and after filtering by different filtering methods.

Tab.3 Comparison of evaluation indexes of different filtering methods

It can be seen from the results that although the ENL in the flat region after filtering by the proposed method is lower than that of the original NL-means filtering method, the EPI of the phase is much higher than that of the original NL-means filtering method, which also reflects the optimization effect of the proposed method on phase structure preservation.

5 Comparison of Tomographic Results

In order to verify the improvement of the proposed filtering method in SAR tomography, in this section, interferometric filtering-based tomography is performed on a set of TerraSAR-X data to compare the imaging results.This set of data contains 19 interferograms processed from 20 SAR images, and the specific imaging parameters are the same as in the above section.The imaging area is the Regent Hotel Beijing and some nearby buildings.When generating the interferogram, the first SAR image is used as the master image, and the remaining images are used as slave images in turn.The acquisition time and baseline settings of the 20 SAR images are shown in Tab.4, and the spatio-temporal baseline is shown in Fig.11.

Tab.4 SAR image acquisition time and baseline

Fig.11 Spatio-temporal baseline for tomography

The proportion of pixels with different SNR in the scene and the corresponding window sizes are given in Tab.5.

Tab.5 Window sizes corresponding to different SNRs

The tomographic results before and after interferogram filtering are shown in Fig.12.From the following three sets of tomographic results, it can be seen that the use of filtered interferograms for tomographic imaging can effectively reduce the number of spurious points in the scene, can make the point cloud more concentrated in the building area, and improve the quality of tomographic imaging.However, the original NL-means filtering will cause building structures besides the main building to be filtered out, and the overall number of targets is reduced, which is effectively improved by the improved method proposed in this paper.

Fig.12 Comparison of tomographic results before and after interferogram filtering: (a) before filtering; (b) aftering filtering (Original NL-means filter); (c) after filtering (filter proposed in this paper)

In order to prove the effectiveness of the proposed method in SAR tomography, the imaging effect of the Regent Hotel area is analyzed.Tab.6 shows the points in the complete point cloud before and after filtering and the points in the part of the point cloud of the Regent Hotel building.It can be seen that although the number of points in the filtered complete point cloud and the point cloud of the Regent hotel have decreased, the proportion of the points in the hotel area in the complete point cloud has increased, which indicates that the spurious points have been successfully filtered after filtering.Although the original NL-means filter can increase the proportion of the main buildings in the whole scene, it will lead to a large reduction of the number of points in the complete point cloud, which indicates that the original NLmeans filter will filter out most of the detail information.Compared with the original NLmeans filter, the filtering method proposed in this paper retains more structural information of other buildings on the premise of increasing the proportion of the main buildings.This reflects that the filtering method proposed in this paper has improved the quality of SAR tomography.

Tab.6 Comparison of points in the point cloud before and after filtering

6 Conclusion

To enhance the quality of SAR tomography, it is essential to filter the SAR interferogram beforehand.While the original NL-means filter demonstrates commendable filtering efficacy, it grapples with challenges such as a fixed window size and suboptimal adaptability to speckle noise in SAR interferograms.In response to these limitations, this paper introduces an improved NLmeans filter, leveraging the distinctive features of SAR interferograms.Through adjustments to the similarity definition in the original NL-means filter and adaptive resizing of the search window,this approach facilitates precise and refined filtering of SAR interferograms.

Filtering experiments conducted on measured interferograms and subsequent tomography experiments with the acquired data affirm the effectiveness of the proposed method.It demonstrates not only a robust denoising capability but also adept detail retention.Nevertheless, the method presented in this paper still harbors areas for improvement.For instance, the fixed shape of the similarity window lacks adaptability to varying image features, and the directional window directly borrows the template from the Lee filter without more innovative modifications.In future research, these aspects will be subject to further exploration and enhancement.