1. Introduction
It has been attractive but extremely challenging for the in situ measurement and accurate reconstruction of translucent objects. Generally speaking, the pure direct reflection can be used for determining the 3D shape of a translucent object. However, the physical feature of translucent objects determines that the signal of the direct reflection is usually quite weak, while most parts of the illumination will go through the interface and undergo the multiple scattering processes. The surface reflectivity of the translucent object could be lower less than 5%, such as an object made of silicone rubber with the real part of the refractive index being . The optical triangular method has been considered as a popular and promising technique in this area. However, as discussed by Holroyd and Lawrence [1], it would be difficult to carry out 3D shape measurement of translucent objects using optical triangular methods, i.e., the structured-light-illumination methods, due to the subsurface scattering brings uncertainties to the reflection measurements at the object surface. The optical triangular method is simply based on the assumption that the direct reflection takes place at the object surface, which might be violated for measuring the translucent objects.
The basic approach of previous investigations for measuring the translucent objects using optical triangular methods is isolating the direct reflectance at the object surface [1,2,3,4]. The method of separating the direct and global components was initially investigated by Nayar et al. [5], which has been cited, discussed, and adopted by other works [6,7]. In the approach of Nayar et al. the global lighting effect was considered as a constant if the projected high-frequency pattern was employed, which allowed for efficient separation of the observed light intensities into direct and global light [5]. Generally speaking, it is difficult to separate the direct and global components, and the separation technique developed by Nayar et al. basically needs a lot of images, which may also be affected by the noises existed in the whole process [5]. To this end, Lockerman and co-authors proposed a method using three directions of projection [8].
The study of polarization-difference imaging [9,10] and the method combining phase-shifting and polarization filtering [11] are considered as promising approach for improving 3D surface measurement of translucent objects. Recently, Xu and co-authors proposed an approach for 3D shape measurement of translucent objects based on phase-shifting fringe projection profilometry [12]. Even though phase-shifting method can be considered to perform the separation of the direct and global components and the 3D surface measurement simultaneously [5,11], measurements using phase-shifting method certainly have a strict requirement of surrounding environment, such as that there should not be background illumination and environmental vibration. For reducing the measuring error in 3D scans of translucent objects using illuminated fringe patterns [13], a number of novel investigations have been carried out, such as the approaches based on multiple scattering [14], the photometric stereo method for measuring optically-thick translucent objects [15], the Monte Carlo simulation [16], and the Fourier single-pixel imaging technique [17]. For the binary boundary methods, exclusively high or low-frequency pattern schemes were considered robust against different global illumination effects [18]. Note that, edges in images of translucent objects are very different from edges in images of opaque objects [19], since the internal scattering within the translucent object can create a variety of image effects, basically depending on the shape and material of the observed object as well as the optical configuration of the measurement.
The existing approaches of measuring translucent objects using the optical triangular method are basically based on the phase-shifting technique and the projection of visible-light fringe patterns generated by the DMD-chip-based digital-light-processing (DLP) projector. However, the phase-shifting technique does have a rigorous requirement of the observed object and the environment, i.e., there is no environmental vibration and the observed object has to remain still during the measuring time. As far as the structured light generated by DLP projector, the intensity of projected fringe pattern is affected by the Gamma effect and defocusing, which can result in inaccuracy of image processing and reconstruction. Moreover, for using projections of visible-light fringe patterns, it is basically impossible to carry out measurements under background illumination and obtain reconstructions with better accuracy. Thus, it might be difficult to fulfill the outdoor and/or in situ measurements of translucent objects using the optical triangular method together with the phase-shifting technique and the projection of DLP-based visible-light fringe patterns due to the exist of environmental vibration and background illumination.
Therefore, for obtaining an improved solution to the challenging of separating the direct and global components of measuring the translucent object as well as overcoming the drawbacks of the phase-shifting technique with the DLP-based fringe patterns, we suggest and investigate a new method using the monochromatic structured light, techniques of optical filtering and polarization, and the image-processing algorithm based on one-shot Fourier transform profilometry.
2. Theoretical Analysis and Mathematical Description of the Proposed Method
This section presents a discussion of the physical process of the surface-shape measurement and reconstruction of a translucent object using sinusoidal fringe pattern generated by the developed optical system, which basically contains the theoretical analyses and mathematical descriptions of the generation and propagation of the sinusoidal fringe pattern in the air, the projection and reflection of the sinusoidal fringe pattern on the surface of the object being observed, and the multiple-scattering process of the transmitted sinusoidal optical signal under the surface of the translucent object and its contribution to the radiance measured by the CCD camera.
2.1. Generation and Propagation of the Sinusoidal Fringe Signal
As we discussed in the section of introduction, the proposed method of the surface-shape measurement and reconstruction of a translucent object is based on the combination of the projection of the monochromatic structured light, techniques of optical filtering and polarization, and the image-processing algorithm of Fourier transform profilometry. Profilometry based on either Fourier-transform approach or phase-shifting method relies on the projection of sinusoidal fringe patterns. The widely-used sinusoidal fringe pattern generated using DLP projector is non-monochromatic structured signal, which indicates that it is impossible to employ the techniques of optical filtering and polarization when the projected fringe pattern is generated using DLP projector. For the purpose of generating monochromatic, high contrast, and truly sinusoidal fringe patterns, we have designed and developed the laser-beam-based optical system as shown in Figure 1a. The developed optical system consists of the following parts: a CW laser source S with the wavelength nm, a rectangular grating G, a Fourier-transform positive lens L with focal length f, an adjustable spatial-frequency filter F, the observation plane P where the target T is placed, and a CCD camera C being connected to the computer. As sketched in Figure 1b, the fundamental components of the sinusoidal optical signal generator include a CW laser source S, a grating positioned at the plane , a Fourier-transform positive lens at the plane , an adjustable spatial-frequency filter located at the plane , and the observation plane is set at the plane .
To analyze the generation and propagation of the sinusoidal fringe signal, we start with the filed from a CW point laser source S, , in the form [20]
(1)
where k is the wave number in the homogeneous background medium, i.e., the air in this study, A is a complex constant related to the laser power, and . Taking for simplicity and using the paraxial approximation, the field behind the grating G at the plane is given by [21](2)
where and are the coordinate variables of , is the distance between the laser source S and grating G, and the transmission function of G. Considering that G is one-dimensional, can be described using [22](3)
where a and d are the optical parameters of grating G. H is the grating width, which is assumed, for the sake of simplicity, to be the diameter of the illumination spot of laser.Since the propagation of field from plane to plane is within Fresnel region, the field in front of the lens L at plane can be written as
(4)
where is a complex constant, and is the distance between the planes and . The field after the lens L at plane can be represented as(5)
where denotes the transmission function of the lens L.The propagating field at plane to at plane can be accurately computed using Fresnel diffraction. denotes the field in front of the spatial-frequency filter F, and is given by
(6)
where is a complex constant, and is the distance between the planes and . Combining Equations (2)–(6) as well as the expression of , we have(7)
where is the Bessel function of the first kind, with the spatial frequency at plane defined by , and is a complex constant given by .The role of the adjustable spatial-frequency filter F at plane is to select -th order spectrum described in Equation (7), and let them to pass through it, and here we take . We then get the field right behind the filter F in the following form
(8)
The propagation of field from plane to can also be analyzed using Fresnel diffraction, which let the field at plane to be given by
(9)
where is another complex constant, and is the distance between the planes and . Considering that the size of (≤4 mm in diameter) is much less than that of , i.e., the spot size of the sinusoidal fringe pattern (≥100 mm in diameter), we take an approximation that for further derivation of Equation (9). Thus, Equation (9) becomes(10)
where the complex parameter is given by .Equation (10) indicates that the field at the observation plane is an inverse Fourier transform of the field output from the spatial-frequency filter F. Combining Equations (8) and (10), we have
(11)
(12)
As indicated in Equation (12), is obviously a complex constant for determined spatial distances and , and the modulus of represents the amplitude of the generated sinusoidal signal. We see that the fringe intensity will decrease as increases since f and are generally parameters with fixed values. The part of in Equation (11) results in a definition of the range of the fringe pattern with a circle of radius . The term of is the key part of output of the designed optical system, i.e., the expression of a sinusoidal curve.
2.2. Reflection and Transmission of the Projected Sinusoidal Fringe Signal on the Surface of the Translucent Object
The proposed method of surface-shape measurement of translucent objects relies on the projection of laser-beam-based structured light, i.e., the monochromatic, high contrast, and truly sinusoidal optical signal described using Equation (11). Note that the projected fringe pattern described by Equation (11) is not affected by the defocusing issue, while the widely-used DLP projector does have the featured problem of defocusing. given in Equation (11) is coherent and linearly polarized before reaching the surface of the translucent object. As shown in Figure 2, the projected sinusoidal fringe signal undergoes the reflection and transmission on the surface of a translucent object and multiple scattering beneath the surface.
We see from Figure 2 that the total radiance measured by the detector will basically consist of , the directly-reflected radiance, , the radiance of interreflection, , the contribution from multiply-scattered photons, and , the contribution of background illuminance. In the proposed method of this work, is the only component that will be used for surface-shape reconstruction of the translucent object being observed, which will raise a challenging issue of separating from the rest parts, i.e., , and , of the totally-measured radiance. The physical characteristics of , , and can be summarized as follows.
(i) : The directly-reflected radiance, , is the reflected portion of the projected fringe pattern, which belongs to diffuse reflection. Since the surface property of the observed object is varying, some fraction of the direct reflection might be depolarized. When the energy of projection is fixed, basically depends on the real part of refraction index of the observed object in accordance with the Fresnel’s Equations for reflection and transmission. For the translucent objects used in this work, the real part of refraction index = 1.4∼1.5, which indicates that is only about 3∼4% of , the intensity of the projected fringe pattern. However, is the only part that will be picked up and used for reconstruction of the translucent objects being observed.
(ii) : Based on Fresnel’s Equations we can conclude the following two points: First, the polarization direction of is different from that of . Second, as shown in Figure 2, the dominant part of is the inter-reflections of the incident signal. However, the magnitude of is lower than the incident signal by about two orders due to the reason that the surface reflectivity of the translucent object being observed is R = 2.8∼5.3% with an assumption that the real part of the refractive index of the translucent object is in the range of = 1.4∼1.6 [23,24].
(iii) : The contribution to the measured radiance from multiply-scattered photons, , is unpolarized. Except , the rest part of the projected signal will transmit the surface, undergo a multiple-scattering process within the translucent object, and exit from the surface of the translucent object. can be estimated using diffusion approximation [25,26]. For calculating , the successful approach is a one-dimensional model based on radiative transfer theory [26]. Thus, we believe that it would be difficult to complete 3D reconstruction based on one-dimensional model of calculation.
(iv) : The background illuminance, , is unpolarized and polychromatic. It should be noted that if the projected signal is polychromatic, the surface measurement of the translucent object should be suggested to be carried out in a dark environment to keep . Otherwise, it would be very difficult or extremely time-consuming to separate from .
It should be noted that, for steady projection of the sinusoidal fringe pattern on a static translucent object being observed, and are time-independent, while is time-dependent since the background illumination is always varying. However, for measuring a dynamic translucent object, all of the values of , , and are time-dependent. However, the time-dependent values of , , and have no effects on the developed method of this investigation, which is one of the few important issues that may result in the expectation of applying the developed method to the measurement of dynamic translucent objects.
2.3. Separating from
Referring to the general definition of the global component previously suggested by Nayar and co-workers [5], the total radiance measured by the detector is given by
(13)
where and stands for the direct and the global components, respectively. Based on the proposed model discussed above, the global component contains the following three parts(14)
where is the background illumination, represents the radiance of inter-reflection, and stands for the contribution from multiply-scattered photons.Therefore, if the proposed methods can work for removing the three components of , i.e., , , and , the expected signal of can then be successfully measured, which is critically important for the 3D surface measurement and reconstruction of translucent objects.
2.3.1. Removing the Effect of Using an Optical Filter
The component related to background illumination is one important issue for in situ profilometry based on either Fourier transform or phase-shifting method, which has been the main reason that most of previous investigations have carried out experimental study in a dark environment for keeping .
In the proposed approach of this work, we are using the laser-beam-based monochromatic structured light at 532 nm as the projecting fringe patterns, which makes it possible to employ the optical filtering technique in the process of measurement. In practice, an optical bandpass filter centered at 532 nm is employed for removing the effects of background illuminations in the experimental setup of this investigation.
2.3.2. Eliminating the Effect of and Using a Polarizer
There have been some efforts that 3D reconstructions are based on the multiply-scattered photons, i.e., , such as the work by Ohtani et al. [14]. However, as we discussed above, the theoretical modeling and computation based on radiative transfer and diffusion theory for the reconstruction of the observed object must be difficult, since only one-dimensional model has been theoretically developed and experimentally validated [26], while it is almost impossible to describe the observed objects using one-dimensional model. In the work of Umeyama and Godin, a method of separating the diffuse and specular components of surface reflection using polarization as well as statistical analysis of images was proposed, but it was only validated for measuring the opaque objects [27].
In the developed method of this work, the desired useful signal is , and is then taken as noised signal to be eliminated. The component will be eliminated via employing polarization technique, which can be ensured since the incident monochromatic structured light and reflected signal from the surface of the translucent objects being observed are linearly polarized, and the part of signal composing multiply-scattered photons is completely unpolarized. When carrying out the practical measurement, a polarizer is mounted on the camera to carry out the measurement of observed object with projected fringe patterns generated using our designed laser-beam-based optical system as sketched in Figure 1.
As far as is concerned, since the magnitude of is much lower than and the polarization direction of is different from that of as discussed above, the elimination of can be accomplished when a polarizer is employed.
The angle of rotation of the polarizer is needed to be carefully controlled before taking the image or measurement of the object being observed. Note that, when having the same object and the same direction of fringe projection, the rotation of the polarizer is basically not necessary based on the discussion above, for the reason that the direction of polarization of should be unchanged. For measuring another object or having a new direction of fringe projection, the direction of polarization of will certainly be different, which will result in the requirement of rotation of the polarizer as a necessary procedure during the measurement.
3. Technical Analysis of Measurement and Reconstruction
In this investigation, the experimental setup for carrying out surface measurements of translucent objects is based on Figure 1a. From Figure 1a, we see that the parts S, G, L, and F are basically the main components of the sinusoidal optical signal generator. The laser-beam-based optical system employs a CW-laser source S with the wavelength nm, maximum output power of 200 mW, and a small divergence angle (≤1.5). A rectangular grating G illuminated by the laser beam is placed at a front focal point of a positive lens L. On the conjugate plane of S, an adjustable spatial-frequency filter F is set, which is considered that the part F plays a critical role in this system. For this spatial-frequency filter, a V-shape aperture with a width of 0.5 mm is carved with high precision, we can then choose and allow the selected order of the spatial frequency to pass through. The output of a sinusoidal fringe pattern can then be observed on the observation plane P, as indicated in Figure 1a. The design of this adjustable spatial-frequency filter F containing three accurately cut slits is actually based on our previous work of measuring modulation transfer function [28] and other recent studies on 3D surface measurements [29,30].
Note that the position of observation plane P is arbitrary, e.g., from few-ten centimeters to few-ten meters, since the projected fringe pattern is not affected by the defocusing issue as we discussed above. At any value of in Figure 1b, we have the monochromatic, high contrast, and truly sinusoidal optical signal described using Equation (11), while the only difference is the spatial frequency of the projected fringe pattern as well as the maximum intensity of the fringe that is inversely proportional to the square of .
With the proposed approach of this investigation, accurate reconstructions of translucent objects can be achieved for background illuminance up to E = 3000 Lux, which is realized using an optical bandpass filter centered at 532 nm with FWHM being 10 nm for removing the effects of background illuminations. The background illuminance E was adjusted using LED lights, and it was measured using a digital luxmeter placed beside the observation plane P. Meanwhile, as discussed above, a polarizer is employed for eliminating the part of signal composing the inter-reflected photons on the surface of the translucent object and the multiply-scattered photons exiting from the subsurface of the translucent object. Both the optical bandpass filter and the polarizer are mounted on the measuring camera.
Reconstructions of the observed translucent objects will be processed and obtained using those measurements based on Fourier transform profilometry (FTP). The method of FTP was initiated by Takeda and Mutoh [31], and it has been applied and improved in many investigations of 3D surface measurement [32,33,34,35]. However, applications of FTP have been limited to the situations with ideal conditions, while such necessary conditions may include that the surface of the measuring object must be opaque, the surface is highly reflective and diffuse, and there should be no background illumination. Thus, the quality of the measurement of surface reflectance provides a necessary basis for developing an image-processing algorithm for the retrievals of phase and 3D-surface height of the observed translucent object. The image-processing algorithm of the proposed approach in this work is based on FTP with a typical triangulation framework of 3D surface measurement, while the general framework of 3D surface measurement can be found in previous works published by Takeda and Mutoh [31], and by Maurel and co-workers [32].
4. Experimental Results and Discussion
There are two selected objects being measured in this study, including (i) a heptahedron-shape target similar to an elongated triangular dipyramid that was made of translucent room-temperature-vulcanized silicone rubber via two-component addition molding in the lab as shown in Figure 3a,b, and (ii) a translucent plastic cap made of polypropylene as shown in Figure 3c. The real part of refractive index of vulcanized silicone rubber at 532 nm is about [23], and the real part of the refractive index of the plastic cap is about = 1.47∼1.49 [24]. The designed heptahedron-shape target has a set of given geometrical parameters, which is necessary and important for validation study and determining the accuracy of the proposed approach.
4.1. Measurement and Reconstruction of a 3D Translucent Object Using Optical Filter
As discussed above, the approach proposed in this work uses the projected laser-beam-based monochromatic fringe patterns, which allows the use of an optical bandpass filter for removing the effects of background illuminations. Figure 4 shows the measurement and reconstruction of a translucent heptahedron-shape object made of vulcanized silicone rubber under the background illuminance E = 202 Lux, while the spatial frequency of the projected fringe pattern was set to 2.2 lp/mm.
All other conditions and technical parameters, such as the output power of laser, the background illuminance E, the spatial frequency of the projected fringe pattern, and the adjustable parameters of the camera, were the same for the measurements in Figure 4a,b, except that the optical filter was used in the measurement of Figure 4b, but not in the case of Figure 4a. Taking the comparison between Figure 4a,b, the background illuminance with E = 202 Lux lowered the fringe contrast and increased the noised signal, and the distorted fringes on the surface of target being measured was almost submerged by the background illumination as indicated by Figure 4a, which resulted in the failed reconstruction as indicated in Figure 4c. However, for the same environmental and experimental conditions, the measurement of Figure 4b using optical filtering technique showed greatly-improved fringe contrast with much lower noise level, which ensured the accurate reconstruction of the target, as shown in Figure 4d. Obviously, the combination of monochromaticity of projected structured signal and optical filtering technique can ensure the high contrast of the projected fringe pattern, which is critically important for in situ profilometry with background illuminations. A problem has been noticed where it seems impossible to see the fringes in Figure 4b, which can be explained with the relatively high spatial frequency (2.2 lp/mm). However, the fringes on the object surface can clearly be seen when the image of Figure 4b is enlarged by a scale factor of 5 as shown in Figure 4f.
4.2. Measurement and Reconstruction of a 3D Translucent Object Using Optical Polarizer
For verifying the polarization of the reflected signal from the surface of the translucent object, we carried out the measurements as shown in Figure 5a,b with the following conditions: (i) The background illuminance , thus the optical filter was not necessarily used. (ii) The spatial frequency of the projected fringe pattern was set to 4.0 lp/mm. The optical polarizer was not used in Figure 5a, while the measurement of Figure 5b was obtained by using optical polarizer via adjusting its direction of polarization.
The experimental condition of keeping the background illuminance being zero allowed us to study and assess the effect of multiply-scattered photons. The intensity of multiply-scattered photons basically depends on the thickness of the silicone rubber and the reflectivity of bottom surface. Due to the effect of multiply-scattered photons, the image of Figure 5a looked brighter and more blurred comparing to that of Figure 5b, which resulted in that the measurement of Figure 5b showed higher fringe contrast and yielded more accurate reconstruction of the object being observed as indicated in Figure 5d with comparing the result shown in Figure 5c.
Thus, the experimental results have proven the theoretical prediction discussed above that the effect of multiply-scattered photons can be removed using an optical polarizer, which is based on the physical basis that the projected monochromatic structured light and reflected signal from the surface of the translucent object are linearly polarized, and the signal component composing multiply-scattered photons is completely unpolarized.
4.3. Measurement and Reconstruction of 3D Translucent Objects under Different Background Illumination
Based on the experimental results of Section 4.1 and Section 4.2 above, we now present two sets of experimental measurements and reconstructions of two selected translucent objects under different background illumination, while both the optical filter and the polarizer are used simultaneously.
As shown in Figure 6, the first object was a translucent heptahedron-shape object made of vulcanized silicone, and the spatial frequency of the projected fringe pattern used in this part of the experiment was 4.0 lp/mm. We see from Figure 6a–d that the effects of background illumination and multiply-scattered photons can be mostly removed, but not completely.
The relatively high brightness of the image in Figure 6d of the measurement with background illuminance E = 3000 Lux indicates that the fringe contrast will be decreased as the background illuminance is increased. The results of Figure 6 might indicate that the accuracy of reconstruction using the measurement with background illuminance E = 196 Lux is the same as the case of E = 0 Lux, and the accuracy of reconstruction using the measurement with background illuminance E = 1000 Lux is basically satisfied. For the reconstruction using the measurement with background illuminance E = 3000 Lux as indicated in Figure 6h, the inaccuracy of reconstruction is mainly due to the decrease of the fringe contrast resulted from the very high background illuminance. Note that the fringe contrast of Figure 6h can be improved by increasing the output power of the laser source, which has been validated by preliminary experimental results.
For the second selected object, a translucent plastic cap made of polypropylene, the measurement and reconstruction were carried out with the spatial frequency of the projected fringe pattern being 1.7 lp/mm as shown in Figure 7.
The purpose of the experiment with translucent plastic cap made of polypropylene should be based on the consideration that the translucent plastic cap has different transparency, refractive index, shape, and surface smoothness. The spatial frequency of the projected fringe pattern used in Figure 7 was 1.7 lp/mm, and the intensity of the projected fringe pattern on the surface of the object was lower than that used in Figure 6.
Figure 7a,c,e were the measurements for background illuminance E = 0, 50, 180 Lux, respectively, while Figure 7b,d,f were the reconstruction using the measurements corresponding to Figure 7a,c,e, respectively. Generally speaking, the accuracy of reconstruction was accurate via comparing the reconstructed 3D shape with the geometrical parameters of the original object. Note that the we were unable to obtain accurate reconstruction if the background illuminance was increased to E = 1000 Lux as we had in Figure 6c, which might be mainly due to that the lower intensity of the projected fringe pattern on the surface of the object resulted in lower fringe contrast with E = 1000 Lux.
4.4. Analysis and Discussion of the Accuracy of Reconstruction
The analysis of the accuracy of reconstruction for the proposed method was based on the measurement and reconstruction of the designed heptahedron-shape target with given geometrical parameters. Figure 8a,b showed the measurement and reconstruction of the heptahedron-shape vulcanized silicone rubber, respectively, illuminated by monochromatic sinusoidal fringe pattern at 532 nm with the following experimental conditions: (i) The target being observed was illuminated by monochromatic sinusoidal fringe pattern at 532 nm. (ii) The spatial frequency of the projected fringe pattern is 2.2 lp/mm. (iii) The background illuminance . (iv) Both optical filter and polarizer were used for the measurement of Figure 8a.
The comparison of the retrieved height distribution on the cross section perpendicular to the ridge with actual values was described in the plot of Figure 8c, and the absolute-error distribution of retrieved surface height in accordance with Figure 8c is given in Figure 8d. The accuracy of reconstruction can be obtained using the comparison between the retrieved height distribution on the cross section perpendicular to the ridge and the actual geometrical parameters of the target being observed. We should point out that the individual point of largest deviation in Figure 8d was due to the little hump or hole on the surface of object, since the translucent heptahedron-shape object made of vulcanized silicone in our lab did not have a perfect geometrical shape as designed.
To quantitatively evaluate difference between known and calculated geometry of the object being observed, we employ the root mean square error (RMSE) for telling the accuracy of reconstruction. The lower the RMSE, the better the accuracy of the reconstruction. The calculated RMSE using the data of Figure 4e, Figure 5e, Figure 6i and Figure 8d are listed in Table 1 below. From Table 1, we see that = 0.09–0.11 (mm) can be obtained for usual indoor background illumination when both optical filter and polarizer are used in our developed method. However, when the background illumination is increased such as the case of Figure 6c,d, = 0.16–0.23 (mm) may be acceptable for many applications.
The results of Figure 8 and Table 1 may indicate that an accurate reconstruction can be obtained using 3D surface-shape measurement of the translucent object with the combination of laser-beam-based structured light and polarization technique, and the optical filtering technique should also be included and works well if the background illumination exists.
The success of the proposed method for 3D measurement and reconstruction of translucent objects is definitely based on the following items: (i) high-contrast and high-quality sinusoidal fringe projection, (ii) polarization technique, and (iii) optical filtering technique. Note that the projected structured light with the optical wave from laser source is monochromatic and generally polarized, thus both the optical filtering and polarization techniques can be used for measurement and image processing as we have proposed in our investigation. However, since the optical signal generated by a DLP-based projector is non-monochromatic and unpolarized, it is impossible to extend our approach to the methods and applications that are based on the use of structured light generated by a DLP projector.
It should be noted that the accuracy of 3D measurement and reconstruction of translucent objects is basically determined by the intensity, contrast, and spatial frequency of the projected fringes and the background illumination. The optical parameters including the spatial frequency of projected fringes should be optimized according to the measuring conditions. The intensity and contrast of the projected fringes are related to the output power of the laser beam and the background illumination, while the possibly higher output power of the laser beam might be necessary when the background illuminance is higher.
5. Conclusions
Based on theoretical analyses and experimental results of this study, we have demonstrated that reliable measurements and accurate reconstructions can be obtained for determining 3D surface shapes of translucent objects being observed with the combination of laser-beam-based fringe projection, the polarization technique, and the optical filtering technique. The rigorous theoretical analyses presented in this study have described the formation, propagation, and physical feature of the generated sinusoidal signal by the designed optical system, the reflection and transmission of the projected monochromatic fringe pattern on the surface of the translucent object, and the formation and the separation of the direct-reflection and the global components of the surface radiance of the observed object. The designed novel sinusoidal optical signal generator can generate monochromatic, high contrast, truly sinusoidal, and linearly polarized fringe patterns, which is the critical basis for this study. The polarization technique should be emphasized in the developed method of this work for eliminating the effect of multiply-scattered photons under the condition that the projected fringe pattern is generated using designed sinusoidal optical signal generator. The optical filtering technique is important and necessary for removing the effect of the background illumination since the projected fringe pattern is a monochromatic signal. It should be noted that the optical system used in this work is portable and stable, and can be used for outdoor or in situ measurement and reconstruction of translucent objects. The results of the reliable measurement and accurate reconstruction of the heptahedron-shape vulcanized silicone rubber illuminated by monochromatic sinusoidal fringe pattern at 532 nm under the background illuminance E = 0∼1000 Lux shown in Figure 6 as well as the error analysis presented in Figure 8 are capable of proving the success of the developed approach of this work. The expected applications of the developed method and result of this work may include accurate in situ shape determination of any 3D translucent objects and related target recognition under general environmental conditions.
Conceptualization, B.C. and P.S.; methodology, B.C. and P.S.; software, P.S. and H.M.; investigation, Y.W. and Y.X.; validation, Y.W. and R.W.; data curation, Y.W. and C.Z.; formal analysis, H.M. and P.C.; preparing figures, P.S.; writing—original draft preparation, B.C.; writing—review and editing, B.C. and P.S. All authors have read and agreed to the published version of the manuscript.
This work was partially supported by the Shandong Province Higher Educational Science and Technology Program under grant number J18KZ012, the National Natural Science Foundation of China under grant number 11975132, the Natural Science Foundation of Shandong Province under grant number ZR2016FB09, and the Natural Science Foundation of Shandong Province for Excellent Young Scholars under grant number ZR2019YQ01.
Not applicable.
Not applicable.
The data presented in this study are available on request from the corresponding author.
The authors declare no conflict of interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Figure 1. The developed the laser-beam-based optical system. (a) Experimental setup. S: CW laser source; G: grating; L: Fourier-transform positive lens; F: spatial-frequency filter; P: observation plane; T: target on the observation plane; C: CCD Camera connected to a computer; [Forumla omitted. See PDF.]: LED light for accurately adjusting the background illuminance. (b) Sketch of the sinusoidal optical signal generator.
Figure 2. Diagram of the projection, reflection, and transmission of the sinusoidal optical signal on the surface of a translucent object. [Forumla omitted. See PDF.] is the intensity of the projected fringe pattern; [Forumla omitted. See PDF.] stands for the directly-reflected radiance; [Forumla omitted. See PDF.] represents the radiance of interreflection; [Forumla omitted. See PDF.] denotes the contribution from multiply-scattered photons.
Figure 3. Original photos of selected translucent objects being measured. (a) Heptahedron-shape vulcanized silicone rubber. (b) The same object shown in (a) on white paper with printed letters. (c) Translucent plastic cap.
Figure 4. Measurement and reconstruction of the heptahedron-shape vulcanized silicone rubber illuminated by monochromatic sinusoidal fringe pattern at 532 nm under background illuminance E = 202 Lux. (a) Image without optical filtering. (b) Image with an optical bandpass filter centered at 532 nm with FWHM being 10 nm. (c) Reconstruction using the measurement of (a). (d) Reconstruction using the measurement of (b). (e) The absolute-error distribution of retrieved surface height on the cross section perpendicular to the ridge with actual values in accordance with data of (d). (f) The enlarged figure of the marked blue-rectangle region in (b).
Figure 5. Measurement and reconstruction of the heptahedron-shape vulcanized silicone rubber illuminated by monochromatic sinusoidal fringe pattern at 532 nm without background illumination. (a) Image without an optical polarizer. (b) Image with an optical polarizer. (c) Reconstruction using the measurement of (a). (d) Reconstruction using the measurement of (b). (e) Comparing the absolute-error distribution of retrieved surface height on the cross section perpendicular to the ridge with actual values based on the data in (c,d).
Figure 6. Measurement and reconstruction of the heptahedron-shape vulcanized silicone rubber illuminated by monochromatic sinusoidal fringe pattern at 532 nm under different background illuminance E. (a) Measurement of [Forumla omitted. See PDF.]. (b) Measurement of [Forumla omitted. See PDF.] Lux. (c) Measurement of [Forumla omitted. See PDF.] Lux. (d) Measurement of [Forumla omitted. See PDF.] Lux. (e) Reconstruction using (a). (f) Reconstruction using (b). (g) Reconstruction using (c). (h) Reconstruction using (d). (i) Comparing the absolute-error distribution of retrieved surface height on the cross section perpendicular to the ridge with actual values based on the data in (e–h).
Figure 6. Measurement and reconstruction of the heptahedron-shape vulcanized silicone rubber illuminated by monochromatic sinusoidal fringe pattern at 532 nm under different background illuminance E. (a) Measurement of [Forumla omitted. See PDF.]. (b) Measurement of [Forumla omitted. See PDF.] Lux. (c) Measurement of [Forumla omitted. See PDF.] Lux. (d) Measurement of [Forumla omitted. See PDF.] Lux. (e) Reconstruction using (a). (f) Reconstruction using (b). (g) Reconstruction using (c). (h) Reconstruction using (d). (i) Comparing the absolute-error distribution of retrieved surface height on the cross section perpendicular to the ridge with actual values based on the data in (e–h).
Figure 7. Measurement and reconstruction of the translucent plastic cap with different background illuminance E. (a) Measurement of [Forumla omitted. See PDF.]. (b) Reconstruction using (a). (c) Measurement of E = 50 Lux. (d) Reconstruction using (c). (e) Measurement of E = 180 Lux. (f) Reconstruction using (e).
Figure 8. Measurement and reconstruction of the heptahedron-shape vulcanized silicone rubber illuminated by monochromatic sinusoidal fringe pattern at 532 nm under background illuminance E = 202 Lux. (a) Image of the illuminated target. (b) Reconstruction using the measurement of (a). (c) Comparing the retrieved height distribution on the cross section perpendicular to the ridge with actual values. (d) The absolute-error distribution of retrieved surface height in accordance with (c).
Comparison of RMSE of different reconstructions.
No. of Figure | Color of Data | RMSE (mm) |
---|---|---|
red | 0.18 | |
blue | 0.09 | |
red | 0.28 | |
black | 0.10 | |
green | 0.11 | |
red | 0.16 | |
blue | 0.23 | |
red | 0.11 |
References
1. Holroyd, M.; Lawrence, J. An analysis of using high-frequency sinusoidal illumination to measure the 3D shape of translucent objects. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Colorado Springs, CO, USA, 20–25 June 2011; pp. 2985-2991. [DOI: https://dx.doi.org/10.1109/CVPR.2011.5995536]
2. Ihrke, I.; Kutulakos, K.N.; Lensch, H.P.A.; Magnor, M.; Heidrich, W. Transparent and specular object reconstruction. Comput. Graph. Forum; 2010; 29, pp. 2400-2426. [DOI: https://dx.doi.org/10.1111/j.1467-8659.2010.01753.x]
3. Park, J.; Sabharwal, A.; Veeraraghavan, A. Direct-global separation for improved imaging photoplethysmography. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops; Salt Lake City, UT, USA, 18–22 June 2018; pp. 1375-1384. [DOI: https://dx.doi.org/10.1109/CVPRW.2018.00186]
4. Subpa-Asa, A.; Fu, Y.; Zheng, Y.; Amano, T.; Sato, T. Separating the direct and global components of a single image. J. Inf. Process.; 2018; 26, pp. 755-767. [DOI: https://dx.doi.org/10.2197/ipsjjip.26.755]
5. Nayar, S.K.; Krishnan, G.; Grossberg, M.D.; Raskar, R. Fast separation of direct and global components of a scene using high frequency illumination. ACM Trans. Graph.; 2006; 25, pp. 935-944. [DOI: https://dx.doi.org/10.1145/1141911.1141977]
6. Rao, L.; Da, F. Local blur analysis and phase error correction method for fringe projection profilometry systems. Appl. Opt.; 2018; 57, pp. 4267-4276. [DOI: https://dx.doi.org/10.1364/AO.57.004267]
7. Torii, M.; Okabe, T.; Amano, T. Multispectral direct-global separation of dynamic scenes. Proceedings of the 19th IEEE Winter Conference on Applications of Computer Vision; Waikoloa Village, HI, USA, 8–10 January 2019; pp. 1923-1931. [DOI: https://dx.doi.org/10.1109/WACV.2019.00209]
8. Lockerman, Y.D.; Brenner, S.; Lanzone, J.; Doronin, A.; Rushmeier, H. Testing spatial patterns for acquiring shape and subsurface scattering properties. Electron. Imaging; 2016; 2016, pp. 1-7. [DOI: https://dx.doi.org/10.2352/ISSN.2470-1173.2016.9.MMRMA-371]
9. Rowe, M.P.; Pugh, E.N.; Tyo, J.S.; Engheta, N. Polarization-difference imaging: A biologically inspired technique for observation through scattering media. Opt. Lett.; 1995; 20, pp. 608-610. [DOI: https://dx.doi.org/10.1364/OL.20.000608]
10. Tyo, J.S.; Rowe, M.P.; Pugh, E.N., Jr.; Engheta, N. Target detection in optically scattering media by polarization-difference imaging. Appl. Opt.; 1996; 35, pp. 1855-1870. [DOI: https://dx.doi.org/10.1364/AO.35.001855] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21085310]
11. Chen, T.; Lensch, H.P.A.; Fuchs, C.; Seidel, H.-P. Polarization and phase-shifting for 3D scanning of translucent objects. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Minneapolis, MN, USA, 17–22 June 2007; pp. 1829-1836. [DOI: https://dx.doi.org/10.1109/CVPR.2007.383209]
12. Xu, Y.; Zhao, H.; Jiang, H.; Li, X. High-accuracy 3D shape measurement of translucent objects by fringe projection profilometry. Opt. Express; 2019; 27, pp. 18421-18434. [DOI: https://dx.doi.org/10.1364/OE.27.018421]
13. Lutzke, P.; Kuhmstedt, P.; Notni, G. Measuring error compensation on three-dimensional scans of translucent objects. Opt. Eng.; 2011; 50, 063601. [DOI: https://dx.doi.org/10.1117/1.3582858]
14. Ohtani, K.; Li, L.; Baba, M. Determining surface shape of translucent objects by using laser rangefinder. Proceedings of the 2013 IEEE/SICE International Symposium on System Integration; Kobe, Japan, 15–17 December 2013; pp. 454-459. [DOI: https://dx.doi.org/10.1109/SII.2013.6776744]
15. Inoshita, C.; Mukaigawa, Y.; Matsushita, Y.; Yagi, Y. Surface normal deconvolution: Photometric stereo for optically thick translucent objects. Lect. Notes Comput. Sci.; 2014; 8690, pp. 346-359. [DOI: https://dx.doi.org/10.1007/978-3-319-10605-2_23]
16. Lutzke, P.; Heist, S.; Kuhmstedt, P.; Kowarschik, R.; Notni, G. Monte Carlo simulation of three-dimensional measurements of translucent objects. Opt. Eng.; 2015; 54, 084111. [DOI: https://dx.doi.org/10.1117/1.OE.54.8.084111]
17. Jiang, H.; Zhai, H.; Xu, Y.; Li, X.; Zhao, H. 3D shape measurement of translucent objects based on Fourier single-pixel imaging in projector-camera system. Opt. Express; 2019; 27, pp. 33564-33574. [DOI: https://dx.doi.org/10.1364/OE.27.033564] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31878422]
18. Gupta, M.; Agrawal, A.; Veeraraghavan, A.; Narasimhan, S.G. A practical approach to 3D scanning in the presence of inter-reflections, subsurface scattering and defocus. Int. J. Comput. Vis.; 2013; 102, pp. 33-55. [DOI: https://dx.doi.org/10.1007/s11263-012-0554-3]
19. Gkioulekas, I.; Walter, B.; Adelson, E.H.; Bala, K.; Zickler, T. On the appearance of translucent edges. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Boston, MA, USA, 7–12 June 2015; pp. 5528-5536. [DOI: https://dx.doi.org/10.1109/CVPR.2015.7299192]
20. Goodman, J.W. Introduction to Fourier Optics; 2nd ed. McGraw-Hill: New York, NY, USA, 1996; Chapter 3
21. Reichelt, A.; Storck, E.; Wolff, U. Near field diffraction pattern behind a sinusoidal phase grating. Opt. Commun.; 1971; 3, pp. 169-172. [DOI: https://dx.doi.org/10.1016/0030-4018(71)90172-6]
22. Harvey, J.E.; Pfisterer, R.N. Understanding diffraction grating behavior: Including conical diffraction and Rayleigh anomalies from transmission gratings. Opt. Eng.; 2019; 58, 087105. [DOI: https://dx.doi.org/10.1117/1.OE.58.8.087105]
23. Smolka, F.M.; Hill, H.A. Room temperature vulcanizing silicone rubber as an optical element. Appl. Opt.; 1977; 16, pp. 292-293. [DOI: https://dx.doi.org/10.1364/AO.16.000292] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20168481]
24. Shackelford, J.F. Introduction to Materials Science for Engineers; McGraw-Hill: New York, NY, USA, 2000.
25. Ishimaru, A. Wave Propagation and Scattering in Random Media; Academic: New York, NY, USA, 1978.
26. Chen, B.; Stamnes, K.; Stamnes, J.J. Validity of the diffusion approximation in bio-optical imaging. Appl. Opt.; 2001; 40, pp. 6356-6366. [DOI: https://dx.doi.org/10.1364/AO.40.006356]
27. Umeyama, S.; Godin, G. Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images. IEEE Trans. Pattern Anal. Mach. Intell.; 2004; 26, pp. 639-647. [DOI: https://dx.doi.org/10.1109/TPAMI.2004.1273960]
28. Li, H.; Chen, B.; Feng, K.; Ma, H. Modulation transfer function measurement method for fiber optic imaging bundles. Opt. Laser Technol.; 2008; 40, pp. 415-519. [DOI: https://dx.doi.org/10.1016/j.optlastec.2007.07.004]
29. Chen, B.; Li, H.; Yue, J.; Shi, P. Fourier-transform-based surface measurement and reconstruction of human face using the projection of monochromatic structured light. Sensors; 2021; 21, 2529. [DOI: https://dx.doi.org/10.3390/s21072529]
30. Chen, B.; Gao, H.; Li, H.; Ma, H.; Gao, P.; Chu, P.; Shi, P. Indoor and outdoor surface measurement of 3D objects under different background illuminations and wind conditions using laser-beam-based sinusoidal fringe projections. Photonics; 2021; 8, 178. [DOI: https://dx.doi.org/10.3390/photonics8060178]
31. Takeda, M.; Mutoh, K. Fourier transform profilometry for the automatic measurement of 3-D object shapes. Appl. Opt.; 1983; 22, pp. 3977-3982. [DOI: https://dx.doi.org/10.1364/AO.22.003977] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18200299]
32. Maurel, A.; Cobelli, P.; Pagneux, V.; Petitjeans, P. Experimental and theoretical inspection of the phase-to-height relation in Fourier transform profilometry. Appl. Opt.; 2009; 48, pp. 380-392. [DOI: https://dx.doi.org/10.1364/AO.48.000380] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19137051]
33. Zappa, E.; Busca, G. Static and dynamic features of Fourier transform profilometry: A review. Opt. Laser Eng.; 2012; 50, pp. 1140-1151. [DOI: https://dx.doi.org/10.1016/j.optlaseng.2012.03.008]
34. Yun, H.; Li, B.; Zhang, S. Pixel-by-pixel absolute three-dimensional shape meausrement with Fourier transform profilometry. Appl. Opt.; 2017; 56, pp. 1472-1480. [DOI: https://dx.doi.org/10.1364/AO.56.001472]
35. Wang, Z.; Zhang, Z.; Gao, N.; Xiao, Y.; Gao, F.; Jiang, X. Single-shot 3D shape measurement of discontinuous objects based on a coaxial fringe projection system. Appl. Opt.; 2019; 58, pp. A169-A178. [DOI: https://dx.doi.org/10.1364/AO.58.00A169]
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Abstract
In this study, we focus on the 3D surface measurement and reconstruction of translucent objects. The proposed approach of surface-shape determination of translucent objects is based on the combination of the projected laser-beam-based sinusoidal structured light and the polarization technique. The theoretical analyses are rigorously completed in this work, including the formation, propagation, and physical features of the generated sinusoidal signal by the designed optical system, the reflection and transmission of the projected monochromatic fringe pattern on the surface of the translucent object, and the formation and the separation of the direct-reflection and the global components of the surface radiance of the observed object. The results of experimental investigation designed in accordance with our theoretical analyses have confirmed that accurate reconstructions can be obtained using the one-shot measurement based on the proposed approach of this study and Fourier transform profilometry, while the monochromaticity and the linearly-polarized characteristic of the projected sinusoidal signal can be utilized by using a polarizer and an optical filter simultaneously for removing the global component, i.e., the noised signal contributed by multiply-scattered photons and the background illuminance in the frame of our approach. Moreover, this study has also revealed that the developed method is capable of getting accurate measurements and reconstructions of translucent objects when the background illumination exists, which has been considered as a challenging issue for 3D surface measurement and reconstruction of translucent objects.
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
Details




1 Research Center for Quantum Optics and Quantum Communication, School of Science, Qingdao University of Technology, Qingdao 266525, China;
2 Office of Laboratory Management, Qingdao Agricultural University, Qingdao 266109, China;