Review and Comparison of High-Dynamic Range Three-Dimensional Shape Measurement Techniques

Abstract

In the last decade, a significant number of techniques for three-dimensional (3D) shape measurement have been proposed. There are a large number of measurement demands for metallic workpieces with shiny surfaces in industrial applications; however, such shiny surfaces cannot be directly measured using the conventional structured light method. Therefore, various techniques have been investigated to solve this problem over the last few years. Some reviews summarize the different 3D imaging techniques; however, no comprehensive review exists that provides an insight into high-dynamic range (HDR) 3D shape measurement techniques used for shiny surfaces. We present a survey of recent HDR techniques for the digitization of shiny surfaces and classify and discuss the advantages and drawbacks of different techniques with respect to each other.

Hui Lin 1; 2 and Jian Gao 1 and Guanjin Zhang 1 and Xin Chen 1 and Yunbo He 1 and Yan Liu 3

Academic Editor:Armando V. Razionale

1, Key Laboratory of Mechanical Equipment Manufacturing and Control Technology of Ministry of Education, Guangdong University of Technology, Guangzhou 510006, China

2, School of Physics and Mechatronics Engineering, Shaoguan University, Shaoguan 512005, China

3, Guangdong Provincial Key Laboratory of Optomechatronics, Shenzhen 518057, China

Received 8 November 2016; Revised 7 February 2017; Accepted 26 February 2017; 12 April 2017

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Three-dimensional (3D) shape measurement is an important topic that has been investigated to a large extent. A significant number of techniques for acquiring 3D shapes have been proposed, leading to a wide range of applications, such as range sensing, industrial inspection of manufactured parts, reverse engineering (digitization of complex, free-form surfaces), object recognition, entertainment, 3D map building, biometrics, and documentation of cultural artifacts [1, 2]. The recent release of Microsoft Kinect has enabled 3D technology to penetrate into our daily lives [3], and the combination of 3D printing technology with 3D shape measurement techniques could generate additional excitement across disciplines. These techniques cover a wide range from microscale objects, such as interferometry and confocal techniques, to several meter-sized objects, such as time-of-flight technique [4], and generally can be classified into two categories: surface contact and surface noncontact. The surface contact measurement system, such as mechanical probe-based coordinate measuring machine (CMM), has been used in the industry for many years and can measure geometric shape with high accuracy (usually more accurate than current noncontact systems). However, CMM is relatively slow and only measures a limited number of points in an object's surface. In contrast, the optical methods have an ability to provide a high-speed 3D inspection of objects in a noncontact manner. These methods include time-of-flight, stereovision, photogrammetry, laser range scanning, and structured light methods. Among these, the structured light method is one of the most widely used 3D shape measurement techniques owing to the advantages of full-field inspection, high resolution, and accuracy that it offers [5].

Typically, a structured light measurement system consists of a camera and a projector that projects a sequence of coded patterns onto object surfaces. The camera captures images of the reflected pattern deformed by the object surface from another direction. The reflected pattern has the height information of the object embedded in its phase distribution. Then, from the captured images, a specially designed algorithm, such as a phase-shifting algorithm, is employed to obtain the absolute phase map, which can be used for determining correspondences between the camera and the projector in a structured light system. Finally, the 3D coordinates of the object surface can be calculated from the absolute phase map using a phase-to-height mapping based on a calibration performed earlier over the measurement volume of interest. The measured point cloud of the object can be further compared with the computer aided design (CAD) model to evaluate the manufacturing quality. However, the use of this system still involves difficulties when dealing with nondiffuse (non-Lambertian) surfaces because the optical signal cannot be properly retrieved. Most of the optical methods assume that the surface being measured is a Lambertian surface, which has a diffuse or near-diffuse reflectance and low reflectivity variations from point to point. Therefore, it is usually very difficult for any optical method to accurately measure shiny objects or objects with a large range of reflectivity variation across the surface. For example, Figures 1(a) and 1(b) show a plastic block and the corresponding captured fringe pattern image, respectively. The camera can capture a clear image of the fringe pattern owing to the diffuse surface of the plastic block. However, for a different surface material, such as a metallic workpiece shown in Figure 2(a), we can see in Figure 2(b) that only regions with strong reflection can provide measurable fringe patterns, while other regions cannot been imaged by the camera. As we increase the exposure time, in Figure 2(c), fringe patterns reflected from regions with low reflection become visible while regions with strong reflection become so bright that image saturation occurs. Thus, insufficient information is available to obtain the 3D coordinates of the metallic workpiece. The fundamental reason is that the metallic workpiece has a much higher level of intensity variation in their reflections compared with what can be captured by the 0-255 gray-level intensity range of the camera.

Figure 1: Captured fringe pattern images of a plastic block with diffuse surfaces: (a) a plastic block and (b) a fringe pattern image of the plastic block.

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

Figure 2: Captured fringe pattern images of a metallic workpiece with shiny surfaces: (a) a metallic workpiece, (b) a fringe pattern image captured by the camera with exposure time at 16.7 ms, and (c) a fringe pattern image captured with exposure time at 100 ms.

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

(c) [figure omitted; refer to PDF]

A large number of metallic workpieces are needed to be milled in industrial applications. After milling, the workpiece surface will become very bright that the conventional structured light method cannot directly measure. One method to solve this problem is to use the CMM to measure the workpiece, and the other is to spray a thin layer of powder onto the workpiece to make its surface diffuse prior to measurement by the structured light method. However, this supplementary step is troublesome and time-consuming because the workpiece needs to be cleaned afterward. In addition, the final accuracy is often dependent on the powder thickness and its homogeneity [6]. To avoid the spraying step, various methods have been investigated over the last few years. State-of-the-art methods for 3D shiny surfaces measurement are typically referred to as high-dynamic range (HDR) techniques. HDR 3D shape measurement techniques can be classified into six categories: (1) using multiple exposures; (2) adjusting projected pattern intensities; (3) using polarizing filters; (4) using color invariants; (5) photometric stereo technique; and (6) miscellaneous. Some reviews detail the different techniques for 3D shape measurement [5, 7, 8]. However, no comprehensive review exists that provides an insight into HDR techniques used for shiny surfaces. In this paper, we review various HDR techniques that were recently developed for the measurement of shiny surfaces, discuss the advantages and drawbacks of different techniques with respect to each other, and assess the current state-of-the-art techniques in the field.

The remainder of this paper is organized as follows. Section 2 attempts to classify and discuss the advantages and drawbacks of various HDR techniques. Section 3 compares these HDR techniques based on their main characteristics, and Section 4 draws a conclusion.

2. HDR Techniques

2.1. HDR Techniques Using Multiple Exposures

In 2009, Zhang and Yau [9] proposed a technique called "high-dynamics range scanning" using multiple exposures, where a sequence of images captured at different exposures is combined into a single set of HDR images, that is, phase-shifting images, by selecting the brightest unsaturated intensities at each pixel. Typically, the best method to adjust the exposure to acquire high-quality fringe patterns is to manually adjust the exposure time instead of the lens aperture because manual adjustments of the lens aperture can cause undesired motion between system components that changes the system calibration. The images taken at low exposure times contain useful information from regions on the surface with high reflectivity, whereas the images taken at high exposure times contain useful information from regions on the surface of low reflectivity. Therefore, the specular or dark region can be properly measured without affecting the rest of the regions. Figure 3 shows the synthesis process. This technique is less sensitive to local surface reflectivity variation, benefiting from the phase-shifting algorithm, which is less sensitive to the ambient light. However, for a large range of reflectivity variation, the signal-to-noise ratio (SNR) is very small for low reflectivity regions. Thus, it is necessary to control the ambient light at a very low level in the industrial in situ environment; otherwise, the quality of measurement is hard to be ensured. Since the used exposure time is subjectively selected, it lacks quantitative manner to determine the proper exposure time. Besides, to synthesize the HDR images, a large number of exposure times are required in some cases, which is time-consuming. For example, in Zhang's experiment, they needed 23 exposures to obtain a high-quality point cloud of a China vase. Even with the three-step phase-shifting algorithm and three-frequency heterodyne phase-unwrapping algorithm, a total of 3×3×23=207 fringe images are required to synthesize the HDR images.

Figure 3: Synthesis of the HDR images from a sequence of fringe images captured at different exposures in [9].

[figure omitted; refer to PDF]

In addition to the use of multiple exposures, Liu et al. [10] used a dual-camera structured light system to acquire the 3D shape of shiny objects and fill in missing data from regions with specular highlights and diffused darkness, that is, two camera-projector monocular systems operated from different viewing angles. An image-segmentation technique was also introduced to distinguish which monocular system is suitable to reconstruct a certainly lost point accurately. This approach worked well because specular highlights are highly dependent on lighting and viewing angles, although additional complexity is required in processing for image masking and registration to merge different surface regions.

For the use of multiple exposures, the initial values of the exposure time are difficult to determine when inspecting an unknown scenario because the initial values cannot be obtained adaptively according to the measured scene. Thus, to adapt different surface reflectivity, it is necessary to use the exposure time as many as possible or rely on experience to select several exposure times. To avoid these, Feng et al. [11] used the histogram to predict one or more optimal exposure times based on the reflectivity distribution of the surface. Then, raw fringe images were captured using these optimal exposure times to synthesize the HDR images.

Due to the influence of camera blurring and random noise, the conventional intensity-based pixel saturation identification method is sometimes susceptible, which determines the brightest intensity reaching the highest quantization level, for example, 255 for an 8-bit camera. Thus, the brightest intensity recorded in a pixel can be unstable and deviates from 255, especially when the saturation of the pixel is slight. Long et al. [12] used the magnitude of a nonprincipal frequency component to identify saturated pixels, leading to more accurate and robust saturation identification and higher measurement accuracy.

Alternatively, Jiang et al. [13] proposed a technique for capturing raw fringe images by simultaneously adjusting the exposure time and the projected light intensity. This technique takes the highest modulation intensity rather than the brightest unsaturated intensity as the criterion for selecting the pixels from the raw fringe images to synthesize the HDR images so that a high SNR is obtained and the ambient light effect is minimized. In addition, a method of autoselection of parameters of the exposure time and the projected light intensity is developed. However, its implementation is relatively complicated, and it achieves the measurement by adding at least 5 times the number of fringe images compared with the conventional phase measurement. To address this time-consuming problem, the same team [14] proposed a very fast HDR fringe pattern projector with a frame rate of 700 Hz, which reduced the routine projecting time by 88%. The dynamic range of the light intensity can vary at least 9 times by using multiple homochromous LED chips and intensity modulation technique.

Similarly, for this time-consuming problem, Ekstrand and Zhang [15] presented an autoexposure technique in which the required exposure time can be predicted automatically according to the reflectivity of the measured object surface. This technique reduces the human intervention and improves the intelligence of the 3D shape measurement system. However, the single predicted exposure time does not always fit the surface with a large range of reflectivity variation, for example, the surface on the metallic workpiece shown in Figure 2. In 2014, a similar approach was presented by Zhong et al. [16], which selects an optimal exposure time as large as possible and meanwhile avoids image saturation in regions of specular reflection during the process of phase shifting. However, these techniques using a single exposure time are limited to improve the SNR in the region of weak reflection.

The advantage of these techniques is that, besides some postprocessing for the captured raw fringe images, there is no need to make too many changes based on the original technique. Adjusting the exposure time would appear to be the best option. However, the exposure time cannot be arbitrarily chosen for a structured light system, where a digital-light-processing (DLP) projector is used to project the fringe patterns. To precisely capture the projected grayscale values, the camera and the projector must be precisely synchronized, and the exposure time must be set as an integer of 1/fp s, where fp is the refresh frequency of the projector (fp is generally 60 Hz).

2.2. HDR Techniques by Adjusting Projected Pattern Intensities

Adjusting the projected fringe pattern intensities is another option. Waddington and Kofman [17-19] proposed a technique of projecting sinusoidal fringe patterns with the modified maximum input gray level (MIGL) to accommodate variable ambient illumination that would otherwise cause image saturation and measurement error in 3D shape measurement. However, this technique that only uniformly adjusts the MIGL to the projector to avoid image saturation will reduce the SNR when measuring those surface regions with low reflectivity. Then, the same team [20, 21] combined the MIGL reduction and pixel-by-pixel approaches, that is, by uniformly adjusting the projected fringe pattern intensity and selecting the brightest unsaturated pixels from raw fringe images captured at different illuminations to synthesize the HDR images. This technique can maintain a high SNR for the measurement of the surface with low reflectivity. However, similar to the techniques using multiple exposures, it also could be time-consuming. Recently, Li and Kofman [22] developed an adaptive fringe pattern projection technique by locally modifying the MIGL in projected fringe patterns, according to the local reflectivity of an object surface being measured, which is better than uniform reduction of the MIGL for saturated regions.

Babaie et al. [23] proposed a new technique for measuring 3D shape of objects with a large range of reflectivity variation. Based on the feedback from the reflected images captured by the camera, it recursively adjusts the intensities of the projected fringe pattern at pixel level. Then, the HDR images are acquired, and a four-step phase-shifting algorithm is used to obtain the absolute phase map of the object.

In 2016, Lin et al. [24] developed an adaptive digital fringe projection technique for HDR 3D shape measurement. This technique can handle the shiny surfaces and achieve improved measurement accuracy by adaptively adjusting the pixel-wise intensity of the projected fringe patterns. The adjustment is based on the camera response function according to the reflectivity of the surface and to the illumination of ambient light and surface interreflections. As a novel mathematical model was established to calculate the intensity of the ambient light and surface interreflections, the optimal intensity of each pixel in the fringe pattern to be projected is precisely calculated, which is superior to the previously proposed techniques using raw fringe images captured at different illuminations to synthesize the HDR images. In addition, this technique only needs to capture 31 images to obtain the point cloud of an aluminum alloy workpiece in their experiment. The comparison of results after using this technique is indicated in Figure 4.

Figure 4: Comparison of results after using the technique proposed by Lin et al. [24]: (a) captured fringe image of an aluminum alloy workpiece with the conventional phase-shifting method and (b) captured fringe image of the same workpiece with the technique proposed by Lin et al. [24].

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

Different from the aforementioned techniques using fringe pattern, Zhang et al. [25] utilized monochromatic white and black stripe patterns, leading to increased robustness. An adaptive intensity mask was adopted to dynamically adjust the pattern intensities to prevent overexposure in shiny regions, which is derived from point spread function and camera response function. The point spread function in this method is based on the homography matrix from the camera image plane to the projector image plane, which can be calibrated using the measurement table in advance. However, during the measurement, the position of the object being measured is difficult to ensure that it is the same as that of the checkerboard, so the coordinates after mapping by the homography matrix may not be very accurate.

An advantage of adjusting the projected pattern intensities is that the adjustment can be performed locally pixel by pixel, rather than that of the adjustment of the camera exposure time, which is global. To minimize the influence of the ambient light and surface interreflections, the lens aperture should be minimized and the used projector should provide intense light projection when applying these type of HDR techniques in the measurement.

2.3. HDR Techniques Using Polarizing Filters

To handle objects with shiny surfaces, an alternative method uses polarizing filters to eliminate the effect of highlights by changing the angle between the transmission axes of the polarizers. The basic idea is to limit the reflected light incident on the image sensor at a certain angle. As a result, the intense specular light is effectively removed; however, at the expense of reduction of captured intensity for the whole image, it may result in low SNR. The method introduced in [26, 27] used polarizing filters, which drastically reduce the output light intensity of the projector and the incident light of the camera; therefore, the dark regions of the scene are difficult to measure. On the contrary, a parallel polarizer-analyzer alignment [28, 29] can maintain good fringe quality at the dark regions, but not in the bright places. Additionally, Feng et al. [11] reported a general solution for realizing HDR 3D shape measurement based on digital fringe projection. They combined the techniques using multiple exposures and the techniques using polarizing filters. For the method by polarization, the reduced intensity by polarizers may result in low SNR for captured fringe images. For the method using multiple exposures, they may need to capture many raw fringe images to synthesize the HDR images.

In 2014, Salahieh et al. [30] proposed a multipolarization fringe projection imaging system, shown in Figure 5, in which saturated points are eliminated and the fringe contrast is enhanced by selecting the properly polarized channel measurements. The projected fringes are linearly polarized prior to incidence on the object and are captured after reflection through a camera equipped with a pixelated polarizer array of four states. However, the polarizer must be carefully adjusted for the object without knowing its optical properties, which increases the complexity of the hardware.

Figure 5: Multipolarization fringe projection system proposed by Salahieh et al. [30].

[figure omitted; refer to PDF]

2.4. HDR Techniques Using Color Invariants

This method is based on the dichromatic reflection model proposed by Shafer [31], where the color of a point is determined by the color of the object itself and the color of the light source. Diffuse reflection indicates the color information related to the object itself, whereas the specular reflection represents the color information for the light source [32]. Gevers and Smeulders [33] first proposed several color invariants based on the dichromatic reflection model. Then, Benveniste and Ünsalan [34-36] used color invariants to solve the problem in scanning the shiny surfaces. In 2011, they [37] introduced a novel color invariant to detect the red stripe, green stripe, and highlights to segment robustly the stripe from the captured image. By using this color invariant in segmentation, the effects of highlights originating from the ambient light were eliminated in scanning shiny surfaces. Recently, with the development of embedded technology, they [38] designed and implemented a novel structured light range scanner system based on the DSP evaluation module, which extracts range data to reduce the computational burden of the PC. Benefiting from color pattern projection and color invariants, their methods based on binary, ternary, and quaternary color-coded patterns can eliminate the effect of highlights and ambient light in the scanning process. An example of such patterns and color invariants is shown in Figure 6. However, this method based on color invariants is inevitably affected by the object itself if the object contains complex textures and multiple colors.

Figure 6: Binary pattern designed by Benveniste and Ünsalan [38]: (a) a sculpture under the projection of the binary pattern and (b) color-invariant image.

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

2.5. Photometric Stereo Technique

The photometric stereo technique can recover both the 3D shape and albedo of surfaces from multiple images at a fixed viewpoint under illuminations with different directions. In the earlier work of photometric stereo, the reflectance of surface was required to be Lambertian [39]. However, for non-Lambertian surface, especially for metallic surface, it is very difficult to robustly estimate the reflectance, while the specular region is present. Recently, the conventional photometric stereo techniques have been extended to specular surfaces with varying bidirectional reflectance distribution function (BRDF). Aiming at detecting specular components or separating reflecting components, many methods [40-44] are presented in later years.

The BRDF detailed in [45] is a function that defines how light is reflected at an opaque surface. For practical surfaces, the general BRDFs are complicated nonlinear equations involving seven or more parameters [46, 47]. Moreover, the reflectance and geometrical parameters vary spatially across the surfaces. Conventional techniques normally use dense images or 3D model of measured object; however, these will limit their application. In 2010, Li et al. [48] presented an efficient photometric stereo method to estimate robustly BRDF parameters without the 3D geometric model of object. Given the small number of input images, they estimate the BRDF parameters from the best window of one image by using the recovered surface normal. In [49], Goldman et al. proposed a photometric stereo method designed for surfaces with spatially varying BRDFs, including surfaces with both varying diffuse and specular properties. They modeled the surface reflectance as a linear combination of fundamental materials and recovered the shape, material BRDFs, and weight maps using the well-known Ward model [47].

In order to tackle rough specular surfaces, on which the microfacets reflect light in different directions, producing wide specular lobes under different lightings, Hin-Shun and Jiaya [50] presented a new photometric stereo method aiming to efficiently estimate BRDF parameters and reconstruct glossy surfaces. In their approach, they did not apply unreliable highlight separation and specularity estimation. Instead, an important visual cue, that is, the cast shadow silhouette of the object, was employed to optimally recover global BRDF parameters and thereby was capable of reconstructing convex object surfaces with different level of roughness. However, their method requires high computing. For example, in their experiments, it took 50 minutes in computation to reconstruct a surface of the mouse, whereas the RMS error is 0.0326 mm. Even, in [51], it used hours to produce a surface.

In 2015, Meng et al. [52] developed a gonio-plenoptic imaging system that realizes a single-shot shape measurement for specular surfaces. The system comprised of a collimated illumination source and a plenoptic camera captures the BRDF variation of the object surface in a single image in addition to the light field information from the scene. In their experiment, they oriented the illumination in 30 different directions. However, due to the fact that their system only captures about ±7° cone of light, it constrains the range of surfaces that can be reconstructed. Furthermore, their system only works for relief surfaces and will have problems to handle the surfaces with steep slopes.

2.6. Miscellaneous Techniques

To handle specular reflections that appear in isolated regions of an image, other techniques, such as multiple cameras, color light sources, color filters [53], and multiple light projection directions [54], have been used to construct the HDR image without saturation from the different captured images. Kowarschik et al. [54] used up to 15 light projection directions to compensate the influence of specular reflections or shadowed areas. Moreover, the object can be rotated, yielding other views of the object. The camera captured different patches of the object that converted into local coordinates and then synthesized to the global coordinates. However, the numerous parameters of the system make it time-consuming to perform the calibration. In addition, additional complexity in hardware, system setup, and processing of image mask and registration is required to synthesize the different surface patches.

Alternatively, Jeong et al. [55] used the spatial light modulator to control the camera exposure time at individual pixels. The final exposure setting at saturated pixels is determined from multiple iterations. In this manner of adjusting the transmittance level of the liquid crystal spatial light modulator located at the conjugate position of the CCD plane, the projected fringes are well imaged and synthesized to the HDR images. However, additional optical and control hardware is required.

Ri et al. [56] proposed an intensity extension method using a digital micromirror device camera, where the optimum exposure time for each CCD pixel can be adjusted; thus the HDR images can be acquired.

Compared with the image intensity-based sinusoidal pattern that is vulnerable to the reflective nature of shiny surfaces, the edges of binary stripe pattern are more localized and better preserved despite the specular nature of the object surfaces; therefore, it makes the decoding procedure more robust. Song et al. [57] presented a novel structured light coding strategy for shiny objects measurement, where stripe edges instead of raw image intensities are encoded in the projected patterns. An improved zero-crossing feature detector using both positive and negative stripe patterns was proposed for stripe edge localization, leading to subpixel accuracy. However, if the saturation regions in the captured image are too large to be included in a single stripe, no constructible information can be found in these regions.

More recently, Jiang et al. proposed a technique [58] that does not require the change of exposures. The fundamental idea is that, besides capturing regular fringe patterns, inverted fringe patterns are captured to complement regular fringe patterns for phase retrieval. If not all of the regular fringe patterns are saturated, inverted fringe patterns are used instead of original saturated patterns for phase retrieval. If all of the regular fringe patterns are saturated, both the original and inverted fringe patterns are all used for phase computation to reduce phase error. Though not as robust as the previously proposed HDR technique using multiple exposures, the proposed technique can substantially increase the measurement quality for high-contract surfaces in real time.

3. Discussion

Table 1 summarizes the pros and cons of the representative HDR techniques. The adaptive digital fringe projection technique developed by Lin et al. [24] provides a good and accurate 3D shape measurement of shiny surfaces and seems to have high potential for industrial application. The HDR digital fringe projection technique proposed by Jiang et al. [13, 14] is also promising, which can obtain a fast and effective measurement of objects with high-reflective surfaces under ambient light. Feng et al. [11] presented a general solution involving three concrete techniques for realizing HDR 3D shape measurement of three types of objects, which is effective. The HDR techniques using polarizing filters are accurate and fast but relatively complex in hardware compared to other techniques. Good results were also obtained by Benveniste and Ünsalan [38], but this technique is easily affected by the colors and complex textures of the object. The gonio-plenoptic imaging system developed by Meng et al. [52] can measure the specular surfaces in high accuracy. However, due to the limitation of their system's structure, only a small range of surface can be measured at a time; thus, it will take a long time to measure the whole surface. Therefore, it is not suitable for online applications. In addition, their system will have measuring problems to handle the surfaces with steep slopes.

Table 1: Pros and cons of various HDR techniques.

Technique	Ref.	Autoprediction of parameters	High accuracy	Possible on line applications	No a priori needed	No requirement of postprocessing	No additional requirement in hardware	Dealing with complex textures and colors
Using multiple exposures	[9, 10, 12]		[radical]		[radical]		[radical]	[radical]
[11, 13]	[radical]	[radical]				[radical]	[radical]
[14]	[radical]	[radical]	[radical]			[radical]	[radical]
Using an optimal exposure time	[15, 16]	[radical]		[radical]		[radical]	[radical]	[radical]
Adjusting projected pattern intensities	[17-19]			[radical]	[radical]	[radical]	[radical]	[radical]
[20, 21]		[radical]		[radical]		[radical]	[radical]
[22-24]	[radical]	[radical]	[radical]		[radical]	[radical]	[radical]
Using polarizing filters	[30]		[radical]	[radical]		[radical]		[radical]
Using color invariants	[38]			[radical]	[radical]	[radical]	[radical]
Photometric stereo	[52]		[radical]

Table 2 summarizes the complexity of the representative HDR techniques. The technique proposed by Jiang et al. [13] can provide an accurate 3D shape measurement of the standard porcelain balls with high-reflective surfaces. This technique takes the highest modulation intensity rather than the brightest unsaturated intensity as the criterion for selecting the pixels from the raw fringe images to synthesize the HDR images. As a result, it requires a large number of square root operations to calculate the modulation intensity at pixel level. Therefore, their algorithm is complicated. The adaptive digital fringe projection technique developed by Lin et al. [24] can measure the workpiece made of aluminum alloy and have a good performance with high accuracy. As their algorithm is simple, it seems to have high potential for industrial application. Additionally, Feng et al. [11] reported a general solution for realizing HDR 3D shape measurement based on digital fringe projection. For the proposed method by polarization, in their experiment of measuring a planar black/white calibration board, the used polarizers reduced intensity of the whole fringe images resulting in low SNR; thus, it reduces the measurement accuracy. Benefiting from using quaternary color-coded patterns, the algorithm proposed by Benveniste and Ünsalan [38] is the simplest; however, the resolution is limited, so the measurement accuracy is not high. In contrast, the gonio-plenoptic imaging system developed by Meng et al. [52] can achieve very high accuracyl however, their system can only measure a small range of surface at a time; thus, it will take a long time to measure the whole surface. Therefore, it is not suitable for online applications.

Table 2: Complexity of various HDR techniques.

Technique	Ref.	Number of required projections	Achieved accuracy	Complexity of calculation
Using multiple exposures	[13]	It uses 7 different parameters and needs a total of 7[low *]4[low *]3=84 projections.	The RMS error reaches 0.016 mm within 100 mm[low *]40 mm[low *]31 mm measurement volume size.	Complicated, because of a large number of square root operations needed to calculate the pixel-wise modulation intensity.
Adjusting projected pattern intensities	[24]	If four-step phase shifting algorithm and heterodyne phase unwrapping algorithm are used, it needs 7+4[low *]3+4[low *]3=31 projections.	The RMS error reaches 0.012 mm within 320 mm[low *]30 mm[low *]150 mm measurement volume size.	Simple, because it only involves the matrix inverse operations at the pixel level.
Using polarizing filters	[11]	If four-step phase shifting algorithm and heterodyne phase unwrapping algorithm are used, it needs 1+4[low *]3=13 projections, one of which is uniform white light.	The standard deviation is 0.045 mm within 300 mm[low *]30 mm[low *]250 mm measurement volume size.	More simple.
Using color invariants	[38]	If quaternary color-coded patterns are used, it needs 4 projections.	The overall average deviation is 0.506 mm within 750 mm[low *]650 mm[low *]200 mm measurement volume size.	Most simple.
Photometric stereo	[52]	It needs 30 projections in different directions.	The RMS error reaches 0.005 mm within 12.5 mm[low *]10 mm[low *]1 mm measurement volume size.	More complicated.

4. Conclusion

This paper presents a survey of recent HDR techniques for the digitization of shiny surfaces. All techniques were briefly recalled and compared with each other, with some highlights on their respective advantages and drawbacks.

The HDR techniques using multiple exposures are the first type of digital fringe projection technique used to obtain 3D data from shiny surfaces. The advantage of these techniques is that there is no need to make too many changes based on the original technique. They only require some postprocessing for the captured raw fringe images, and then a high precision 3D point cloud can be obtained. However, they may be time-consuming due to a large number of images needed to be captured in some cases. Fortunately, Jiang et al. [13, 14] improved this technique by developing a very fast HDR fringe pattern projector, thus reducing the time consumed. The HDR techniques using polarizing filters were also interesting, but the only drawback is the complexity of the hardware due to the use of the polarizer. The HDR techniques using color invariants provide good results but suffer from the colors and complex textures of the object being measured. Adjusting the projected fringe pattern intensities is another option to handle shiny surfaces. The adaptive digital fringe projection technique recently proposed by Lin et al. [24] has achieved high accuracy, and its novel mathematical model for shiny surface measurement shows some potential for industrial applications. A scanning device, which will merge these techniques, would also be interesting to investigate in the near future.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China and the Guangdong Provincial Natural Science Foundation under Grants no. 51675106, no. U1601202, and no. 2015A030312008, the Guangdong Provincial R&D Key Projects (no. 2015B010104008, no. 2016A030308016, and no. 2015A010104009), and the Young Talent Innovation Projects from Guangdong Educational Department (no. 2015KQNCX147).

References

[1] S. S. Gorthi, P. Rastogi, "Fringe projection techniques: whither we are?,", Optics and Lasers in Engineering , vol. 48, no. 2, pp. 133-140, 2010.

[2] S. Zhang, Handbook of 3D Machine Vision: Optical Metrology and Imaging , CRC Press, Boca Raton, Fla, USA, 2013.

[3] Z. Zhang, "Microsoft kinect sensor and its effect,", IEEE Multimedia , vol. 19, no. 2, pp. 4-10, 2012.

[4] P. I. Stavroulakis, R. K. Leach, "Invited review article: review of post-process optical form metrology for industrial-grade metal additive manufactured parts,", Review of Scientific Instruments , vol. 87, no. 4, 2016.

[5] F. Chen, G. M. Brown, M. Song, "Overview of three-dimensional shape measurement using optical methods,", Optical Engineering , vol. 39, no. 1, pp. 10-22, 2000.

[6] D. Palousek, M. Omasta, D. Koutny, J. Bednar, T. Koutecky, F. Dokoupil, "Effect of matte coating on 3D optical measurement accuracy,", Optical Materials , vol. 40, pp. 1-9, 2015.

[7] F. Blais, "Review of 20 years of range sensor development,", Journal of Electronic Imaging , vol. 13, no. 1, pp. 231-243, 2004.

[8] G. Sansoni, M. Trebeschi, F. Docchio, "State-of-the-art and applications of 3D imaging sensors in industry, cultural heritage, medicine, and criminal investigation,", Sensors , vol. 9, no. 1, pp. 568-601, 2009.

[9] S. Zhang, S.-T. Yau, "High dynamic range scanning technique,", Optical Engineering , vol. 48, no. 3, 2009.

[10] G.-H. Liu, X.-Y. Liu, Q.-Y. Feng, "3D shape measurement of objects with high dynamic range of surface reflectivity,", Applied Optics , vol. 50, no. 23, pp. 4557-4565, 2011.

[11] S. Feng, Y. Zhang, Q. Chen, C. Zuo, R. Li, G. Shen, "General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique,", Optics and Lasers in Engineering , vol. 59, pp. 56-71, 2014.

[12] Y. Long, S. Wang, W. Wu, K. Liu, "Accurate identification of saturated pixels for high dynamic range measurement,", Optical Engineering , vol. 54, no. 4, 2015.

[13] H. Jiang, H. Zhao, X. Li, "High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces,", Optics and Lasers in Engineering , vol. 50, no. 10, pp. 1484-1493, 2012.

[14] H. Zhao, X. Liang, X. Diao, H. Jiang, "Rapid in-situ 3D measurement of shiny object based on fast and high dynamic range digital fringe projector,", Optics and Lasers in Engineering , vol. 54, pp. 170-174, 2014.

[15] L. Ekstrand, S. Zhang, "Autoexposure for three-dimensional shape measurement using a digital-light-processing projector,", Optical Engineering , vol. 50, pp. 895-900, 2011.

[16] K. Zhong, Z. Li, X. Zhou, Y. Li, Y. Shi, C. Wang, "Enhanced phase measurement profilometry for industrial 3D inspection automation,", International Journal of Advanced Manufacturing Technology , vol. 76, no. 9-12, pp. 1563-1574, 2014.

[17] C. Waddington, J. Kofman, "Sinusoidal fringe-pattern projection for 3-D surface measurement with variable illuminance," in Proceedings of the International Symposium on Optomechatronic Technologies (ISOT '10), pp. 1-5, October 2010.

[18] C. Waddington, J. Kofman, "Analysis of measurement sensitivity to illuminance and fringe-pattern gray levels for fringe-pattern projection adaptive to ambient lighting,", Optics and Lasers in Engineering , vol. 48, no. 2, pp. 251-256, 2010.

[19] C. Waddington, J. Kofman, "Modified sinusoidal fringe-pattern projection for variable illuminance in phase-shifting three-dimensional surface-shape metrology,", Optical Engineering , vol. 53, no. 8, 2014.

[20] C. Waddington, J. Kofman, "Saturation avoidance by adaptive fringe projection in phase-shifting 3D surface-shape measurement," in Proceedings of the International Symposium on Optomechatronic Technologies (ISOT '10), pp. 1-4, IEEE, Ontario, Canada, October 2010.

[21] C. Waddington, J. Kofman, "Camera-independent saturation avoidance in measuring high-reflectivity-variation surfaces using pixel-wise composed images from projected patterns of different maximum gray level,", Optics Communications , vol. 333, pp. 32-37, 2014.

[22] D. Li, J. Kofman, "Adaptive fringe-pattern projection for image saturation avoidance in 3D surface-shape measurement,", Optics Express , vol. 22, no. 8, pp. 9887-9901, 2014.

[23] G. Babaie, M. Abolbashari, F. Farahi, "Dynamics range enhancement in digital fringe projection technique,", Precision Engineering , vol. 39, pp. 243-251, 2015.

[24] H. Lin, J. Gao, Q. Mei, Y. He, J. Liu, X. Wang, "Adaptive digital fringe projection technique for high dynamic range three-dimensional shape measurement,", Optics Express , vol. 24, no. 7, pp. 7703-7718, 2016.

[25] C. Zhang, J. Xu, N. Xi, J. Zhao, Q. Shi, "A robust surface coding method for optically challenging objects using structured light,", IEEE Transactions on Automation Science and Engineering , vol. 11, no. 3, pp. 775-788, 2014.

[26] Y. Yoshinori, M. Hiroyuki, N. Osamu, I. Tetsuo, "Shape measurement of glossy objects by range finder with polarization optical system,", Gazo Denshi Gakkai Kenkyukai Koen Yoko , vol. 200, pp. 43-50, 2003.

[27] S. Umeyama, G. Godin, "Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images,", IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 26, no. 5, pp. 639-647, 2004.

[28] T. Chen, H. P. A. Lensch, C. Fuchs, H.-P. Seidel, "Polarization and phase-shifting for 3D scanning of translucent objects," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '07), pp. 1-8, IEEE, June 2007.

[29] R. Liang, "Short wavelength and polarized phase shifting fringe projection imaging of translucent objects,", Optical Engineering , vol. 53, no. 1, 2014.

[30] B. Salahieh, Z. Chen, J. J. Rodriguez, R. Liang, "Multi-polarization fringe projection imaging for high dynamic range objects,", Optics Express , vol. 22, no. 8, pp. 10064-10071, 2014.

[31] S. A. Shafer, "Using color to separate reflection components,", Color Research & Application , vol. 10, no. 4, pp. 210-218, 1985.

[32] R. T. Tan, K. Nishino, K. Ikeuchi, "Separating reflection components based on chromaticity and noise analysis,", IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 26, no. 10, pp. 1373-1379, 2004.

[33] T. Gevers, A. W. M. Smeulders, "Color-based object recognition,", Pattern Recognition , vol. 32, no. 3, pp. 453-464, 1999.

[34] R. Benveniste, C. Ünsalan, "Single stripe projection based range scanning of shiny objects under ambient light," in Proceedings of the 24th International Symposium on Computer and Information Sciences (ISCIS '09), pp. 1-6, IEEE, September 2009.

[35] R. Benveniste, C. Ünsalan, "A color invariant based binary coded structured light range scanner for shiny objects," in Proceedings of the 20th International Conference on Pattern Recognition (ICPR '10), pp. 798-801, IEEE, Istanbul, Turkey, August 2010.

[36] R. Benveniste, C. Ünsalan, E. Gelenbe, R. Lent, G. Sakellari, A. Sacan, H. Toroslu, A. Yazici, "Binary and ternary coded structured light 3D scanner for shiny objects,", Computer and Information Sciences , vol. 62, pp. 241-244, Springer Netherlands, 2010.

[37] R. Benveniste, C. Ünsalan, "A color invariant for line stripe-based range scanners,", Computer Journal , vol. 54, no. 5, pp. 738-753, 2011.

[38] R. Benveniste, C. Ünsalan, "Nary coded structured light-based range scanners using color invariants,", Journal of Real-Time Image Processing , vol. 9, no. 2, pp. 359-377, 2014.

[39] R. J. Woodham, "Photometric method for determining surface orientation from multiple images,", Optical Engineering , vol. 19, pp. 1-22, 1992.

[40] A. Hertzmann, S. M. Seitz, "Example-based photometric stereo: shape reconstruction with general, varying BRDFs,", IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 27, no. 8, pp. 1254-1264, 2005.

[41] L. Shen, T. Machida, H. Takemura, "Efficient photometric stereo technique for three-dimensional surfaces with unknown BRDF," in Proceedings of the 5th International Conference on 3-D Digital Imaging and Modeling (3DIM '05), pp. 326-333, IEEE, Ontario, Canada, June 2005.

[42] R. T. Tan, K. Ikeuchi, "Separating reflection components of textured surfaces using a single image,", IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 27, no. 2, pp. 178-193, 2005.

[43] J. Wang, K. J. Dana, "Relief texture from specularities,", IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 28, no. 3, pp. 446-457, 2006.

[44] N. G. Alldrin, D. J. Kriegman, "Toward reconstructing surfaces with arbitrary isotropic reflectance: a stratified photometric stereo approach," in Proceedings of the IEEE 11th International Conference on Computer Vision (ICCV '07), pp. 1-8, IEEE, October 2007.

[45] F. E. Nicodemus, J. C. Richmond, J. J. Hsia, I. W. Ginsberg, T. Limperis, "Geometrical considerations and nomenclature for reflectance,", Radiometry , pp. 94-145, 1977.

[46] S. K. Nayar, K. Ikeuchi, T. Kanade, "Determining shape and reflectance of hybrid surfaces by photometric sampling,", IEEE Transactions on Robotics and Automation , vol. 6, no. 4, pp. 418-431, 1990.

[47] G. J. Ward, "Measuring and modeling anisotropic reflection,", SIGGRAPH Computer Graphics , vol. 26, no. 2, pp. 265-272, 1992.

[48] C. J. Li, Z. Zhang, T. Imamura, T. Miyaki, "An efficient BRDF acquisition for glossy surface," in Proceedings of the 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE '10), pp. V2-141-V2-145, IEEE, Chengdu, China, August 2010.

[49] D. B. Goldman, B. Curless, A. Hertzmann, S. M. Seitz, "Shape and spatially-varying BRDFs from photometric stereo,", IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 32, no. 6, pp. 1060-1071, 2010.

[50] C. Hin-Shun, J. Jiaya, "Efficient photometric stereo on glossy surfaces with wide specular lobes," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '08), pp. 1-8, IEEE, Anchorage, Alaska, USA, 2008.

[51] A. S. Georghiades, "Recovering 3-D shape and reflectance from a small number of photographs," in Proceedings of the 14th Eurographics Workshop on Rendering, pp. 230-240, ACM, Leuven, Belgium, June 2003.

[52] L. Meng, L. Lu, N. Bedard, K. Berkner, "Single-shot specular surface reconstruction with gonio-plenoptic imaging," in Proceedings of the IEEE International Conference on Computer Vision (ICCV '15), pp. 3433-3441, IEEE, Santiago, Chile, 2015.

[53] Q. Hu, K. G. Harding, X. Du, D. Hamilton, "Shiny parts measurement using color separation," in Two- and Three-Dimensional Methods for Inspection and Metrology III, vol. 6000, of Proceedings of SPIE, pp. 125-132, November 2005.

[54] R. Kowarschik, P. Kühmstedt, J. Gerber, W. Schreiber, G. Notni, "Adaptive optical three-dimensional measurement with structured light,", Optical Engineering , vol. 39, no. 1, pp. 150-158, 2000.

[55] J. Jeong, D. Hong, H. Cho, "Measurement of partially specular objects by controlling imaging range," in Optomechatronic Computer-Vision Systems II, 671808, vol. 6718, of Proceedings of SPIE, October 2007.

[56] S. Ri, M. Fujigaki, Y. Morimoto, "Intensity range extension method for three-dimensional shape measurement in phase-measuring profilometry using a digital micromirror device camera,", Applied Optics , vol. 47, no. 29, pp. 5400-5407, 2008.

[57] Z. Song, R. Chung, X.-T. Zhang, "An accurate and robust strip-edge-based structured light means for shiny surface micromeasurement in 3-D,", IEEE Transactions on Industrial Electronics , vol. 60, no. 3, pp. 1023-1032, 2013.

[58] C. Jiang, T. Bell, S. Zhang, "High dynamic range real-time 3D shape measurement,", Optics Express , vol. 24, no. 7, pp. 7337-7346, 2016.