Smart Computational Imaging (SCI) Lab
智能计算成像实验室

Block diagram of IRLMS

Improved IRLMS and phase correlation

图片

We are developing several simple and effective scene-based nonuniformity correction (SBNUC) techniques to adaptively compensate the FPN according to the scene information (no calibration targets). Our approaches are featured by fast convergence rate (less than 20 frames or even only use 2 frames), high steady-state accuracy, robustness to "ghosting artifacts" and computational simplicity. Those methods can be integrated with the hardwares of the infrared cameras to make them more robust to the temporal drift of  the FPN which may occur slowly over time. With those SBNUC technologies, we have successfully developed several ‘smart' infrared cameras, including two cooled thermal imager [3-5μm, HgCdTe IRFPA] with resolution 320×256 and 640×512, respectively; and two uncooled thermal imager with resolution 320×240 and 640×480 (8-14μm, VOx Microbolometer).

Our Work
The main research work and contributions are summarized as the following aspects (click picture for details):

ABUIABAEGAAgwfHirQUoqfPF_QQwkAM42gI

Interframe-registration based SBNUC

4875095

Registration method under FPN

417596

Statistical SBNUC

Interframe-registration-based SBNUC

图片

Scene-based nonuniformity correction algorithm based on interframe registration (IRLMS)

We present a simple and effective scene-based nonuniformity correction (NUC) method for infrared focal plane arrays based on interframe registration. This method estimates the global translation between two adjacent frames and minimizes the mean square error between the two properly registered images to make any two detectors with the same scene produce the same output value. In this way, the accumulation of the registration error can be avoided and the NUC can be achieved. The advantages of the proposed algorithm lie in its low computational complexity and storage requirements and ability to capture temporal drifts in the nonuniformity parameters. The performance of the proposed technique is thoroughly studied with infrared image sequences with simulated nonuniformity and infrared imagery with real nonuniformity. It shows a significantly fast and reliable fixed-pattern noise reduction and obtains an effective frame-by-frame adaptive estimation of each detector’s gain and offset.


The simplest and most efficient registration-based SBNUC!



Reference:


Mid-wave IRFPA (3-5um)      

 Long-wave IRFPA (8-14um)

图片

Improved interframe registration based nonuniformity correction for focal plane arrays  (Improved IRLMS)

We further improved the interframe registration based nonuniformity correction algorithm for focal plane arrays. The method simultaneously estimates detector parameters and carries out the nonuniformity correction by minimizing the mean square error between the two properly registered image frames. A new masked phase correlation algorithm is introduced to obtain reliable shift estimates in the presence of fixed pattern noise. The use of an outliers exclusion scheme, together with a variable step size strategy, could not only promote the correction precision considerably, but also eliminate ghosting artifacts effectively. The performance of the proposed algorithm is evaluated with clean infrared image sequences with simulated nonuniformity and real pattern noise. We also apply the method to a real-time imaging system to show how effective it is in reducing noise and the ghosting artifacts.

Reference：


   Simulated NU (compared with orignal IRLMS)          

 Complex scene and camera motion  

Real-time NUC of indoor scene

图片

Scene based nonuniformity correction based on block ergodicity for infrared focal plane arrays

We put forward a new scene based nonuniformity correction algorithm for IRFPA. This method adopts phase-correlation method for motion estimation and takes the sum of mean-square errors of the pixel brightness between several adjacent frames as the cost function when the brightness constancy assumption between two adjacent frames is satisfied. Nonuniformity correction parameters could be estimated by minimizing such cost function. In order to reduce calculation quantity, we can divide these images into several subblocks, and solve for the optimum solution of the cost function respectively in each subblock. From the analysis, it is shown that the optimum solution is of global uniqueness when all the elements in subblocks could satisfy the ergodicity condition. Then the estimated value of nonuniformity correction parameters could be deduced by minimizing the cost functions. The nonuniformity correction experiments for both infrared image sequence with simulated nonuniformity and infrared imagery with real nonuniformity show that the proposed algorithm could achieve a great correction effect by only analyzing a small number of frames.

Reference：


图片

A two-frame approach for scene-based nonuniformity correction in array sensors
This paper introduces a new scene-based technique to correct the fixed-pattern noise (FPN) in array sensors. This method registers a pair of image frames exhibiting small relative scene translation and then the noise pattern can be reconstructed using the constrained least-squares estimation. The key advantage of this technique is that the accurate estimates of the bias nonuniformity can be obtained with only two images, without imposing any assumptions on the structure of the FPN. Besides, the method works on almost static scene, and therefore does not require larger scale global motion and statistical assumptions on the scene irradiance. We test our method on synthetically generated FPN as well as with real infrared data, and experimental results demonstrate the significant reduction in FPN, validating the effectiveness of our approach. Finally, we validate the feasibility and validity of using the proposed method as a first step fostering the success of more sophisticated registration-based time-evolving correction algorithms.

The minimum (2) frames required for SBNUC(without imposing any assumptions on the structure of the FPN, like the strips or grids)



Reference：


Registration Method Under FPN

Simulated NU  

Long-wave IRFPA (8-14um)      

 Mid-wave IRFPA (3-5um)

图片

Registration method for infrared images under conditions of fixed-pattern noise
We proposes a new registration method for infrared images under conditions of fixed-pattern noise (FPN). Conventional registration techniques are susceptible to FPN and it is therefore very desirable to have a registration algorithm that is tolerant to FPN. For this purpose, we utilize the difference of the cross-power spectrum of two discrete shifted images to suppress the noise power spectrum while the shifts information is well preserved. In particular, we show that the phase of the cross-power spectrum difference is a periodic two-dimensional binary stripe signal with the exact shifts determined to subpixel accuracy by the number of periods of the phase difference along each frequency axis. Robust estimates of shifts can be obtained by transforming its discontinuities to Hough domain. Experimental results show that the proposed method exhibits robust and accurate registration performance even for the noisy images that could not be handled by conventional registration algorithms. We have also incorporated this technique to a registration-based nonuniformity correction (NUC) framework, indicating that our registration technique is able to estimatemotion parameters reliably, leading to satisfactory NUC result.


Reference:


Statistical SBNUC

图片

New temporal high-pass filter nonuniformity correction based on bilateral filter
A thorough analysis of low convergence speed and ghosting artifacts in temporal high-pass filter correction has been undertaken in this work and it has found out that the keys of these problems are the interference of a large sum of unrelated scene information in the nonuniformity correction (NUC) process. In order to overcome these drawbacks, a new scene-based NUC technique based on bilateral filter has been developed. This method separates the original input frames into two parts and it estimates the NUC parameters only by using the residuals. The experimental results have shown that it can significantly increase convergence speed and reduce ghosting artifacts.

Reference:


图片

Scene-based nonuniformity correction method using multiscale constant statistics
In scene-based nonuniformity correction (NUC) methods for infrared focal plane array cameras, the statistical approaches have been well studied because of their lower computational complexity. However, when the assumptions imposed by statistical algorithms are violated, their performance is poor. Moreover, many of these techniques, like the global constant statistics method, usually need tens of thousands of image frames to obtain a good NUC result. In this work, we introduce a new statistical NUC method called the multiscale constant statistics (MSCS). The MSCS statically considers that the spatial scale of the temporal constant distribution expands over time. Under the assumption that the nonuniformity is distributed in a higher spatial frequency domain, the spatial range for gain and offset estimates gradually expands to guarantee fast compensation for nonuniformity. Furthermore, an exponential window and a tolerance interval for the acquired data are introduced to capture the drift in nonuniformity and eliminate the ghosting artifacts. The strength of the proposed method lies in its simplicity, low computational complexity, and its good trade-off between convergence rate and correction precision. The NUC ability of the proposed method is demonstrated by using infrared video sequences with both synthetic and real nonuniformity.

Reference:


 Mid-wave IRFPA (3-5um)        

  Long-wave IRFPA (8-14um)