【THE MIRAGE】Deep Learning Enhances Precision of Single-Frame Fringe Analysis

A new publication from Opto-Electronic Advances, 10.29026/oea.2024.230034 discusseshow the synergy of traditional techniques and deep learning enablessingle-frame high-precision fringe pattern analysis.

Optical metrology, as a general-purposemetrology technique that uses light as information carriers for non-contact andnon-destructive measurement, is fundamental to manufacturing, basic research,and engineering applications. With the invention of the laser andcharge-coupled device (CCD), many optical metrology methods and instruments areemployed in state-of-the-art manufacturing processes, precision positioning,and quality assessment because of their advantages in terms of accuracy,sensitivity, repeatability, and speed. For many optical metrology techniquessuch as interferometry, digital holography, and fringe projection profilometry(FPP), fringe pattern analysis is the primary focus of research for recoveringthe underlying phase distribution from the recorded fringe patterns. Theaccuracy and efficiency of phase retrieval from fringe patterns are essentialto dynamically reconstruct various desired physical properties of the objects(the profile, distance, strain, etc.).

For structured light 3D imaging based onFPP, to minimize the number of fringes required for a single reconstruction,Prof. Qian Chen and Chao Zuo's research group at Nanjing University of Scienceand Technology established the theoretical framework for phase shiftingprofilometry and temporal phase unwrapping and developed a series of compositephase shifting methods for fast 3D measurement, including bi-frequency phaseshifting, 2+2 phase shifting, geometric constraints based composite phaseshifting, and micro Fourier transform profilometry (μFTP). These compositephase shifting methods reduce the number of fringe patterns required per 3Dreconstruction from ~10 to 5, 4, 3, or even 2, achieving high-speed 3D sensingat 10k frames per second. Nevertheless, high-accuracy 3D reconstruction usingonly one single pattern has been the ultimate goal of structured light 3Dimaging in perpetual pursuit. However, the key to the success of FTP is thatthe high-frequency fringe information modulated by the object surface can bewell separated from the background intensity in the frequency domain. As aresult, the FTP technique is limited to measuring smooth surfaces with limitedheight variations.

Recently, with the explosive growth ofavailable data and computing resources, deep learning, as a"data-driven" machine learning technique, has achieved impressivesuccess in numerous fields, such as computer vision and computational imaging. Deeplearning, which pervades almost all aspects of optical metrology, providessolutions to many challenging problems, such as fringe denoising, fringeanalysis, and digital holographic reconstruction.

However, different from traditional fringeanalysis methods, these deep learning approaches focus mainly on training a DNNto accurately identify an image-to-image transform from massive input andoutput data pairs, as the physical laws governing the image formation or otherdomain expertise pertaining to the measurement have not yet been fullyexploited in current deep learning practice. Consequently, the performance ofdeep learning approaches in solving complex physical problems relies heavily onthe underlying statistical characteristics within the dataset. In order to pushthe limits of fringe pattern analysis in terms of speed, accuracy,repeatability, and generalization, the synergy of physics-based traditionalmethods and data-driven learning approaches become to represent the generaltrend [Fig. 1(b)].

In a recent publication in Opto-ElectronicAdvances, Prof. Qian Chen and Prof. Chao Zuo's research group at NanjingUniversity of Science and Technology reported a physics-informed deep learningmethod for fringe pattern analysis (PI-FPA), which integrates a lightweight DNNwith a learning-enhanced Fourier transform profilometry (LeFTP) module [Fig.2], enabling more accurate and computationally efficient single-shot phaseretrieval. The lightweight network refines the initial phase to further improvethe phase accuracy at a low computational cost, compared with universalend-to-end image transform networks (U-Net and its derivatives).

Dynamic 360-degree 3D reconstructionresults of a workpiece model by different fringe analysis methods are shown in[Fig. 3]. For the 3-step PS method, when dynamic scenes are measured, therelative motion between the object and the phase-shifting fringe patternssequentially projected will cause motion artifacts and thus introducenon-negligible errors into 3D reconstruction results. For single-frame fringeanalysis, FTP is suitable for dynamic 3D measurement, but yields coarse 3Dresults with low quality in terms of accuracy and resolution due to thespectrum overlapping. U-Net can further improve the quality of 3Dreconstruction, but it cannot reliably retrieve the phase of the object withmetal materials which is relatively rare in the training dataset, precludingthe recovery of fine surfaces. This experiment demonstrates that the proposedPI-FPA can be applied for high-quality and efficient 3D modeling of complexstructure parts.

The proposed physics-informed deep learningtechnique for fringe pattern analysis (PI-FPA) not only learns the inherentstatistical characteristics within the dataset like traditional neuralnetworks, but also masters the physical laws describing the image formation,realizing single-frame phase reconstruction with high precision and highcomputational efficiency, while exhibiting its good generalization to raresamples never seen by the network. In the future, we will investigate the phaserecovery performance of PI-FPA for different types of fringe images, andexplore related fringe analysis applications in the fields of interferometryand digital holography in optical metrology, further pushing the limits offringe pattern analysis in speed, accuracy, repeatability, and generalization.

Public Release. This material from the originatingorganization/author(s) might be of the point-in-time nature, and edited forclarity, style and length. Mirage.News does not take institutional positions orsides, and all views, positions, and conclusions expressed herein are solelythose of the author(s).View in full here.

窗体顶端

窗体底端

Tags: university, technology, Computing, resolution, 3D, technique, Engineering, University of Science and Technology, efficiency, computervision, machine learning, 3Dimaging, metrology, deeplearning, sensitivity, neuralnetworks

复审 | 左超

终审 | 徐峰