A progressive framework for rotary motion deblurring

Jinhui Qin, Yong Ma, Jun Huang, Fan Fan, You Du

School of Electronic Information, Wuhan University, Wuhan, 430072, China

Keywords:Rotary motion deblurring Progressive framework Blur extents factor TDM-CNN

ABSTRACT The rotary motion deblurring is an inevitable procedure when the imaging seeker is mounted in the rotating missiles.Traditional rotary motion deblurring methods suffer from ringing artifacts and noise,especially for large blur extents.To solve the above problems, we propose a progressive rotary motion deblurring framework consisting of a coarse deblurring stage and a refinement stage.In the first stage,we design an adaptive blur extents factor (BE factor) to balance noise suppression and details reconstruction.And a novel deconvolution model is proposed based on BE factor.In the second stage,a triplescale deformable module CNN(TDM-CNN) is designed to reduce the ringing artifacts, which can exploit the 2D information of an image and adaptively adjust spatial sampling locations.To establish a standard evaluation benchmark,a real-world rotary motion blur dataset is proposed and released,which includes rotary blurred images and corresponding ground truth images with different blur angles.Experimental results demonstrate that the proposed method outperforms the state-of-the-art models on synthetic and real-world rotary motion blur datasets.The code and dataset are available at https://github.com/Jinhui-Qin/RotaryDeblurring.

1.Introduction

Because of its low cost and high attack precision, the imaging seeker is widely used in precision-guided missiles[1-3].But when a missile spins,the relative rotation between the camera and scenes during the exposure time leads to rotary motion blurring [4-6],making subsequent computer vision tasks difficult, such as object detection [7,8], recognition [9,10], and tracking [11,12].Therefore,reducing rotary motion blurring is a vital preprocessing step in these applications.

Unlike defocus blur [13] and video blur [14,15], the rotary motion blur is a typical space-variant blur, and the extent of the blur increases with the radius from the rotation center and the relative rotation angle [6].Since rotary parameters can be determined by the imaging system or estimated [4,6], the rotary motion deblurring methods are treated as non-blind deblurring.Generally,these methods can be divided into two main categories: model-based methods and learning-based methods.

The key idea of model-based methods is transforming the rotary motion blur into the space-invariant blur.Therefore, how to transform rotary motion blur and remove the space-invariant blur are two crucial problems.

For the first problem,Sawchuk et al.[16-18]propose coordinate transformation restoration (CTR), transforming the blurry image from the rectangular coordinate system to the polar coordinate system.However,the CTR involves two coordinate transformations and interpolations, leading to computation complexity and calculating errors.Then blurring paths methods(BPM)[4]are proposed,in which the 2D blurry image is decomposed into a series of 1D blurry sequences.

For the second problem, Yuan et al.[6] present a modified Wiener filtering(M-Wiener) to avoid overregularization in deblurring.M-Wiener divides frequency responses of blur kernel into illconditioned frequency components, which are filtered, and wellconditioned frequency components.Then, Wiener filtering is used to obtain the deblurred estimation.To preserve strong edges and suppress noise, a sparse adaptive prior (SAP) is proposed,whose prior is regulated by the images themselves [19].However,these methods suffer from ringing artifacts and noise because of the Gibbs phenomena.

The learning-based methods are still in their developing stage.As far as we know, Qiu et al.[20] first proposed an End-to-end rotary motion deblurring based on a generative adversarial network(GAN),marked RMD-GAN.To provide the GAN with rotary parameters,including rotation center and blur angle,a rotation blur field is proposed and fed into the network as prior for training,which explicitly indicates relative blur extents at every position[20].Actually, because more details are smoothed when blur extents increase, it is difficult for CNN to extract effective features.Therefore, learning-based methods cannot effectively restore the regions with large blur extents,such as locations far away from the rotation center.

To avoid artifacts and noise, this paper presents a progressive rotary motion deblurring framework composed of two stages: a coarse deblurring stage and a refinement stage.In the first stage,we first decompose the 2D blurry image into a set of 1D blurry sequences.Next,we take the radius and blur angle into account and propose a blur extents factor (BE factor), which can adjust regularization parameters to balance the noise suppression and details reconstruction.And we present a novel deconvolution model with BE factor to recover 1D deblurred sequences.Finally,we reconstruct these deblurred sequences into a 2D coarse restored image.In the refinement stage, considering that a 2D image has more information than a 1D sequence, we design a triple-scale deformable module CNN(TDM-CNN) to reduce the ringing artifacts in the coarse restored image,whose sampling locations adaptively varies with different degrees of the ringing artifacts.

The contributions of this paper are summarized.

(1) We present a progressive rotary motion deblurring framework for rotary motion deblurring, consisting of a coarse deblurring stage and a refinement stage.

(2) In the coarse deblurring stage, we propose a BE factor to balance noise reduction and details reconstruction.

(3) In the refinement stage,we design a TDM-CNN to reduce the ringing artifacts, which can adaptively regulate sampling locations by learned offsets.

(4) To the best of our knowledge, we first release a real-world rotary motion blur dataset containing the rotation center and blur angle.

2.Related work

2.1.Model-based methods

As shown in Eqs.(1)-(3), the model-based methods are summarized into three steps.Firstly,the rotary motion blurry image g is converted into a space-invariant transformed image gtthrough the space transformation function (STF)

where(x0,y0)means rotation center,θ is the relative rotation angle between the camera and scenes during the exposure time,termed blur angle [6].It should be noted that the form of gtrelies on the rotation model.

There are two types of STF:CTR and BPM.Sawchuk et al.[16-18]first focus on rotary motion blur and present CTR.CTR converts blurry images from the Cartesian coordinate system to the polar coordinate system.However, there are two coordinate transformations in the CTR technology, which increase the computational burden.At the same time, transformations of different coordinate systems involve interpolations.Thus, some calculating errors are introduced and impact restored images.

In order to solve the above problems, BPM is proposed [4-6].BPM adopts the Bresenham algorithm to construct the coordinate relationship between 2D and 1D on the radius r [4,6,19], then use the coordinate relationship to decompose the 2D blurred image into a series of 1D sequences.But BPM only uses the 1D information of the image to deblur and ignores the 2D information,limiting the performance of this algorithm.

Secondly, the intermediate estimate ^ftis recovered by an optimization algorithm and reduce the rotary motion blur.

where h is the blur kernel, determined by rotary parameters, ⊗denotes the convolution operator.ftis the transformed ideal image,λ represents the regularization parameter,and R is the image prior.

The solution of Eq.(2) is another critical question for modelbased methods.As a simple but effective method, the Wiener filter is used to obtain^ft[5]Considering the piecewise smoothness of the adjacent pixels,Hong et al.[4]propose second-order difference prior(SDP)to constrain the ideal image.But SDP often imposes an overregularization in edges.To solve this problem, M-Wiener [6]filters the zero frequency of the blur kernel and then applies Wiener filtering to restore the estimation image.Wang et al.[19]propose SAP,which can adaptively suppress small derivative values associated with noise and preserves large values associated with strong edges.But these methods involve ringing artifacts and noise,as displayed in the third and fourth columns of Fig.1.

Finally, the intermediate estimate ^ftis transformed by the inverse space transformation function (ISTF) to obtain the restored image ^f.

2.2.Learning-based methods

Currently, deep learning has achieved state-of-the-art performance in image enhancing tasks such as image de-blurring[21-23], denoising [24,25] and fusion [26-28].

Qiu et al.[20]first introduce CNN into the field of rotary motion deblurring and proposes RMD-GAN.To combine the GAN and rotary parameters, they present a rotation blur field [29], equipping the generator with the ability to deal with varying blur extents.To fuse the context information in the shallow layer and highresolution details, the feature pyramid network (FPN) [30] is used as the backbone network of the generator.However,learning-based methods fail in regions with large blur extents,as illustrated in the fifth column of Fig.1.

2.3.Deformable convolution

Conventional CNNs are limited to model geometric variations due to their fixed structures.For example, convolution layers sample fixed locations,leading to the same receptive field for every activate unit.Dai et al.[31]propose deformable convolution to cope with geometric transformations.Deformable convolution enables CNNs to change the sampling grid by offsets,which is learned from additional standard convolution.Stacking more Deformable convolutions and introducing modulation mechanisms,Zhu et al.[32]expand the ability to model various transformations.Benefiting from feature extraction for geometric variations, deformable convolution has been successfully applied in many visual tasks,such as object detection[33,34],super-resolution[35,36],and nonblind motion deblurring [37,38].

Fig.1.Selected results from different rotary motion deblurring methods at blur angle of 10.08°.

3.Methodology

Given rotary motion blurred image and rotary parameters,namely rotation center and blur angle, we present our algorithm consisting of two stages, as depicted in Fig.2.In the coarse deblurring stage,we propose a BE factor to balance noise reduction and details restoration.And a novel deconvolution model is designed to obtain the intermediate image.In the refinement stage,to reduce the ringing artifacts, we present a TDM-CNN, which can regulate sampling positions by learned offsets.

3.1.The coarse deblurring stage

In this stage, according to rotation center and blur angle, we coarsely remove the rotary motion blur,and the pipeline is shown in the course deblurring stage in Fig.2.The rotary motion blurry image is decomposed into a series of 1D blurry sequences along the radius r.And these sequences can be expressed in the form of convolution [6,19]

where the subscript i is the i-th element of the sequences.gr,hr,frrepresents the 1D blurred sequence, blur kernel, and ideal sequence.Nris the length of gr.

Eqs.(4) and (5) show that the rotary motion blur on one circle can be regarded as a 1D blur,in which blur extents are proportional to the radius and blur angle.

Eq.(4) can be rewritten in matrix form

where grand frare column vectors with Nrelements.Cris an(Nr×Nr)-square circular matrix [4].n is the additive white Gaussian noise (AWGN).

To recover the 1D blur, we assume that the first-order and second-order gradient of frsatisfies the Gaussian distribution.And then,regularization terms‖dxfr‖22and‖dxxfr‖22,in which dxand dxxare first-order and second-order derivative operators, can be obtained, denoted as FSDP.

Meanwhile,Regularization parameters play an important role in image restoration.When blur extents increase, we should raise regularization parameters to restrain noise since the details in the blurry image are smoothed, and noises dominate the highfrequency components.Otherwise, as blur extents are small and the details play a dominant role relatively in the high-frequency components, the regularization parameters ought to be lessened in order to reconstruct more details.Hence, regularization parameters are supposed to be positively associated with blur extents to balance the noise suppression and details reconstruction.We impose a stronger punishment for larger blur extents and propose a BE factor wbe.

In conclusion, the optimization algorithm constructed in this paper is

where (⬇)*represents the conjugate operator.°is the elementwise multiplication operator.^F r = F (^fr), Hr= F (hr), Dx=F (dx), Dxx=F (dxx) are the frequency domain of ^fr, hr, dx, dxx,respectively.F () is the FFT operator.

The sharp sequence can be obtained by the inverse fast Fourier transform (IFFT)

where F-1is the IFFT operator.

After calculating all clear sequences on different radiuses, we convert them to a 2D image by ISTF.Meanwhile,about 11%points in a 2D image cannot be mapped in any 1D sequences,leading to socalled ‘missing pixels’ [6,19].We interpolate these missing pixels with the median filter.

In the first stage, most blur is roughly removed, but there still are ringing artifacts.Because finite Fourier basis functions cannot model step signals,called Gibbs phenomena,these ringing artifacts are unavoidable.And the ringing artifacts become stronger as the blur extents increase [39],as displayed in the blue box in Fig.3.In the refinement stage,we try to reduce these ringing artifacts by the proposed TDM-CNN.

3.2.The refinement stage

The intermediate image is obtained after the first stage and is limited to the ringing artifacts deriving from the Gibbs phenomena.In fact, we could restrain those ringing artifacts in a 1D sequence.However,allowing for the case that there is more information in a 2D image than in a 1D sequence,we reduce the ringing artifacts in the intermediate image.Since CNN has seen great success in various computer vision fields, we utilize CNN to achieve our goal in the refinement stage.

The ringing artifacts usually occur at strong edges and vary with the extent of the blur.However, the traditional convolutional network has fixed geometric structures, which makes it challenging to learn geometric transformations of these variable ringing artifacts.Hence, we propose the TDM-CNN to reduce the ringing artifacts.The TDM-CNN adjusts the sampling positions to capture the features of the ringing artifacts by the offsets,which are learned from a standard convolution.

We regard the problem of refinement as three sub-problems at different scales.The overall network architecture is shown in the refinement stage of Fig.2, and consists of three sub-nets with the same structure, called DM-CNN (Deformable module CNN).The network takes three intermediate images I of different scales as input,as shown in Eq.(12),and generates three restored images R with the corresponding size, of which the largest output image is the final restored image.

Fig.3.Pattern diagrams for the ringing artifacts:(a) The ground truth image; (b) The rotary motion blurry image with blur angle at 10.08°; (c) The intermediate image through our coarse deblurring stage; (d) The ringing artifacts because of the Gibbs phenomena.

Fig.4.Visual comparison of Δθ=0.5 °.

Table 1 Quantitative evaluation of angle error: average PSNR/SSIM.Bold and underlining indicate best performance and second best, respectively.

Fig.5.Visual comparison of Δp = 5.

where fDM-CNNkis DM-CNN operator.The subscript k is the scale index.The smaller the k, the higher the scale is I2and I3downsampled from I1by factors of 2 and 4.↑is the upsampling operator,and some possible operators can be chosen, such as upsampling convolution, reshaping, and upsampling.In all experiments of the refinement stage, we use upsampling convolution since shared low-frequency information can be learned [40].

The structure of the DM-CNN is a stack of convolutional layers.And the DM (Deformable Module) comprises of two deformable convolutions at the front and the end of DM-CNN.N (set N = 7 in this paper) ResBlocks are used in the middle part, and batch normalizations of the original ResBlock [41] are removed, following[40,42].To reuse features and speed up network convergence,two skip-structure connections are constructed to connect the first and last layers, the ceil(N/4)-th and ceil(3 N/4)-th residual models,respectively.

In the refinement stage,we use PyTorch[43]as a deep learning framework, and trainings were performed in NVIDIA GTX 3090GPU.Some data augmentations are involved to avoid overfitting,such as randomly horizontal and vertical flips, rotation with 90°,saturation in HSV color space,and Gaussian noise.The solver in this paper is ADAM [44] with batch size 8 for training.Multistep learning rate scheduler is used with milestones at 125, 185, and 225.And The learning rate begins from 1 × 10-4.

Table 2 Quantitative evaluation of center error: average PSNR/SSIM.Bold and underlining indicate best performance and second best, respectively.

Fig.6.Schematic diagram of the image acquisition system.

Fig.7.Some Blurry-GT pairs from real-world rotary motion blur datasets.

Fig.8.Visual comparison of the proposed BE factor.

We use SSIM loss [45] in this paper.

Table 3 Comparison by incorporating proposed BE factor into different models: average PSNR/SSIM.Bold indicates best performance of each method.

Fig.9.Visualization of the learned sampling positions.The cyan points are fixed sampling positions from conventional convolution, while the red points are learned ones beyond fixed sampling positions.

where I and S denote intermediate and ground truth sharp image.wkis weight for k-th scale, and we set wk= 1.0.

4.Experiments and results

4.1.Datasets

4.1.1.Synthetic rotary motion blur datasets

We use BSD500 datasets [46] to generate our rotary motion burring datasets,based on Eq.(4).The first 400 images are chosen as training datasets,and the last 100 images are testing datasets.All images are center cropped to 320×320,meaning the maximum of radius rmaxis 160.Eqs.(14)and(15)show that the extent of the blur is identical when rθ is the same.As a result,when an interval of the blur angle Δθ is quite small, these generated blurry images are identical:

To obtain Δθ,let r be the maximum value rmax.So,any r can be satisfied in Eq.(14).

The range of blur angles in this paper is[5.04°,10.08°].Assume that the rotating speed of the low-speed spinning missile is 15-25 r/s,and the exposure time is 1 ms.Then,the blur angle is 5.4°-9.0°,indicating that the setting of blur angles meets actual needs.Consequently,there are 15 samples for each clear image by setting Δθ=0.36°, and we get 400 × 15 = 6000 training data.What's more,we test each image at 6 specific blur angles and obtain 100×6 = 600 testing data.All images are corrupted by the AWGN with strength σ = 1%.

4.1.2.Test on deblurring with parameters uncertainty

In real-world scenarios, the rotation parameters, including the rotation center and blur angle, are difficult to obtain accurately.Therefore, deblurring methods should be robust to parameters uncertainty.

Blur angel error: To evaluate the performance of different methods when the blur angle is inaccurate, we synthesize the dataset at the blur angle θ and deblur at the blur angle θ+Δθ.

Fig.4 shows the visual results of all methods at a blur angle of 5.04°and 10.08°.Because of the blur angle error, SDP and SAP are corrupted by noise and ringing artifacts.RMD-CNN falls to strong edges when blur extents are large, such as the first-row fourthcolumn.Ours can reduce these noise and artifacts and restores more texture details.

Table 4 Ablation study of DM: average PSNR/SSIM.Bold indicates best performance.

Fig.10.Ground truths of plane, fish, building and glass.

The quantitative results are demonstrated in Table 1.SAP obtains the second best when Δθ is small.However,as Δθ increases,RMDGAN outperforms SAP in most cases.We think SAP suffers from noise and artifacts because of center errors.Since the proposed TDM-CNN can effectively reduce artifacts,our method still achieves the best results.

Rotation center error:In order to obtain a blurry image with an inaccurate rotation center,we blur the image at the rotation center at (200, 200) after the GT image is expanded to 400 × 400.And then,the blurry image is center cropped to 320×320,in which the center is (200+Δp, 200).

The qualitative results of center errors are shown in Fig.5.Due to center error,traditional methods,the SDP and SAP,are degraded by noise and ringing artifacts,such as the first row first and second columns.And RMD-CNN generates some artifacts,as shown in the first-row fourth column.The proposed method restores strong edges such as the second row and fifth columns.

Table 2 presents the quantitative results.When the rotation center is not exact, SDP and SAP are corrupted by noise and artifacts.Though SDP or SAP achieves the second best in PSNR in most circumstances, RMD-GAN is superior to them in SSIM.Due to the progressive framework, our method still yields the best performance.

4.1.3.Real-world rotary motion blur datasets

The critical difficulty in generating real-world blur datasets lies in acquiring ground truth.To solve this problem,blurry images are averaged from multiple clear input images,and ground truth is one of the clear input images [21,40].According to this principle, we build an image acquisition system to collect real-world rotary motion blur datasets.As shown in Fig.6, the camera is fixedly connected to the stepper motor.The camera saves a current sharp frame when the stepper motor is rotated one step.After rotating several steps,the system captures a set of sharp images at different angles.The rotary motion blurry images are generated by averaging from those sharp images,in which the first sharp image is defined as the ground truth image.Furthermore, in our experiments, the stepper motor takes 5 k steps to rotate one round,and we capture 250 images in one scene.

The training and test datasets have 30 and 10 scenes, respectively, including cars, buildings, text, etc.Same as the Synthetic Data, the range of blur angles is [5.04°, 10.08°].On the training dataset, there are 101 Blur-GT (Ground Truth) image pairs in each scene.So, we get 101 × 30 = 3030 training pairs.We choose six specific blur angles on the test dataset and obtain 10×6=60 test pairs.Fig.7 demonstrates some Blurry-GT pairs on the training dataset.

4.2.Ablation experiments

4.2.1.Effectiveness of the BE factor

The visual comparison is illustrated in Fig.8, and in the first column BE factor is removed by setting p=0 in Eq.(8).As displayed in the blue boxes of the first and second columns,in areas far from the rotation center, where the blur extents are large, noise is effectively suppressed because the BE factor increases the regularization parameters.On the other hand,as depicted in the red boxes of the first and second columns, in the places near the rotation center,associated with small blur extents,the edges of the restored image are sharper, since the regularization parameters are decreased by BE factor.In conclusion, BE factor can balance noise reduction and details preservation.

4.2.2.Effectiveness of the DM

At places occurring the ringing artifacts, we visualize the sampling positions of the last deformable convolution from DM-CNN1,as illustrated in Fig.9.We can observe that the spatial distribution range of learned sampling positions varies with ringing artifacts,different from fixed ones.What's more, when the ringing artifacts are stronger, meaning larger blur extents, the sampling positions are more expansive, such as the red boxes of the first and second columns in Fig.9.Thus,by adaptively determining spatial sampling locations,the proposed DM reduce different degrees of the ringing artifacts.

Quantitative comparisons are shown in Table 4.To remove our DM, we replace all deformable convolution with conventional convolution.It can be observed that the PSNR and SSIM of our method are improved by 0.02-0.09 dB and 0.0005-0.0012,demonstrating the effectiveness of the proposed DM.

Fig.11.Visual comparison of plane and fish at different blur angle.

4.3.Results on rotary motion blur datasets

4.3.1.Synthetic rotary motion blur datasets

We compare the results with model-based method,SDP[4]and SAP [19] and learning-based method RMD-CNN [20] in qualitative and quantitative ways.

To demonstrate the superiority of our progressive framework,four representative scenes are selected, and ground truths are shown in Fig.10, including plane, fish, building and glass.The qualitative results are shown in Figs.11 and 12, containing blurry images in the first row and deblurring results with different blur angles.

The second and third rows show the results of SDP and SAP,respectively.As illustrated in the blue boxes of building at a blur angle of 5.04°, SDP and SAP are corrupted by the ringing artifacts and noise due to the Gibbs phenomena.What's more, in the blue box of plane at a blur angle of 10.08°,the ringing artifacts are more vital with blur extents increasing.

The results of RMD-CNN are depicted in the fourth row.We can notice that the RMD-CNN removes most blur when blur extents are small.For example,in the blue box of the building at a blur angle of 5.04°, some window details are restored.However, the performance of RMD-CNN decreases, and even artifacts are introduced.For instance,several edges of non-existence appear,as depicted in the blue box of the fish at a blur angle of 10.08°.

Fig.12.Visual comparison of building and glass at different blur angle.

Table 5 Quantitative evaluation of rotary motion deblurring methods on test dataset: average PSNR/SSIM.Bold and underlining indicate best performance and second best,respectively.

Fig.13.Ground truths of real-world rotary motion blur datasets.

Fig.14.Visual comparison of real-world rotary motion blur datasets at different blur angles.

Our results are shown in the last row.In the blue box of the plane and the red box of the fish at a blur angle of 10.08°, our method can preserve strong edges because TDM-CNN effectively reduces the ringing artifacts.Significantly,as displayed in the blue boxes of building and glass at a blur angle of 10.08°, the proposed method restores more texture details than others.

Fig.16.Quantitative comparisons of the SSIM on real-world rotary motion blur datasets.

Table 6 Quantitative evaluation of rotary motion deblurring methods on the real-world dataset:average PSNR/SSIM.Bold and underlining indicate best performance and second best,respectively.

Fig.17.Visual comparison of real-world rotary motion blur image without ground truth.

The quantitative results with different blur angles are summarized in Table 5.As the blur angle increases,the performance of all methods decreases in terms of PSNR and SSIM because the deblurring is more complicated.We also notice that the performance of RMD-GAN is even worse than that of SAP.We think the RMD-GAN can not effectively extract features from a blurry image since many details are lost during the blurring.Owing to the proposed progressive framework,our method outperforms the second best with more than 2.44 dB (PSNR) and 0.1079(SSIM).

4.3.2.Real-world rotary motion blur datasets with ground truth

On real-world rotary motion blur datasets, we choose two scenes to show visual results.Ground truths and results are shown in Figs.13 and 14, respectively.

In Fig.14,the results of SDP and SAP are illustrated in the second and third rows,respectively.We can notice that SDP and SAP suffer from more severe noise and ringing artifacts on real-world datasets than on synthetic datasets,especially at a blur angle of 10.08°.We think that three kinds of errors, which make the blurry signal and the kernel deviate from the accepted convolution theorem,lead to noise and ringing artifacts.

(1) According to Eq.(4),the convolution theorem is constructed in a 1D sequence, so the blurry 1D sequence should first be fetched along one circle.Bresenham algorithm is widely used in this process.Although Bresenham is very close to the ideal circle, there is still an error.

(2) According to Eq.(5),since the subscript i is an integer,rθ also belongs to an integer, indicating that the integer number of pixels is averaged during blur.This hypothesis is also unrealistic and leads to kernel errors.(3) There is an error between the estimated and real rotation center.As a result, the fetched 1D blurry sequence is not in the same circle.

Among them,(1)and(2)are the limitations of the model itself;

(3) is the parameter estimation error.However, the existing traditional models do not take these factors into account.

The fourth row depicts the results of RMD-CNN.Similar to the results on synthetic datasets, RMD-CNN can restore blurry images well when the blur angle is small, as shown in the fourth column.However,some uncomfortable artifacts arise when the blur extents are large, as illustrated in the third and sixth columns.

The last row shows our results.Our method preserves strong edges.For example,as depicted in the last column,the edge of the Chinese character is restored since the proposed TDM-CNN relieves the ringing artifacts.

Table 6 shows the quantitative results on real-world datasets.Figs.15 and 16 show the PSNR and SSIM of every image on realworld datasets, respectively.The performance of SDP and SAP drops rapidly as the blur angle increases,indicating that the modelbased methods are degraded by noise and ringing artifacts.Because of the refinement stage, our solution can reduce these negative effects.

4.3.3.Real-world rotary motion blur image without ground truth

To further demonstrate the effectiveness of our method on realworld blur,we rotate the camera at high angular speed,and a rotary blurred image is captured.A sharp image is taken as the reference ground truth when the camera is stationary.In this experiment,the rotary center and blur angle are determined by Ref.[6].

As shown in Fig.17, the traditional methods, namely SDP and SAP,suffer from noise and ring artifacts.Although some details are restored,RMD-GAN fails to noise and locations where blur extents are large.Compared with other methods, ours achieves the best visual result, demonstrating the superiority of the proposed method.

4.3.4.Running time

Running time.The running time of different methods is shown in Table 7.For a fair comparison, we test all methods on the Synthetic dataset and calculate the average running time.As demonstrated in Table 7 our method is faster than SAP but slower than the others.Since our first stage deals with 1D data of different lengths,which is hard to accelerate with GPU, and consumes most of the running time.

5.Conclusions

In this paper, a novel progressive framework for rotary motion blur is proposed consisting of a coarse deblurring stage and a refinement stage.In the first stage,BE factor is presented to balance the noise suppression and details preservation, which can adaptively adjust regularization parameters according to blur extents.Moreover, our BE factor is still effective when applied to other methods.In the second stage, TDM-CNN is designed to effectively reduce the different degrees of ringing artifacts by adaptively changing sampling locations.Extensive experiments show ourmethod is superior to other state-of-the-art solutions,including on synthetic and real-world rotary motion blur datasets.

Table 7 The running time of all methods.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work was supported in part by the National Natural Science Foundation of China under Grant 62075169, Grant 62003247, and Grant 62061160370; in part by the Hubei Province Key Research and Development Program under Grant 2021BBA235; and in part by the Zhuhai Basic and Applied Basic Research Foundation under Grant ZH22017003200010PWC.