HoloPort – design and integration of a digital holographic 3-D sensor in machine tools

Abstract

Manufacturing of high-precision components requires accuracies that even the most modern processing machines are often unable to deliver reliably. Slightly worn tools, incorrectly calibrated sensors or even different trajectories can lead to results that do not comply with the desired specifications. However, quality control is still mainly performed on randomized samples outside the machine tool in special measuring rooms. Therefore, closed-loop quality control becomes a cumbersome iterative process. With HoloPort, we present a digital holographic sensor system that is capable of measuring the complete topography of machined components with sub-micrometer precision directly inside a tooling machine. To our knowledge, HoloPort is the first wireless interferometric sensor inside a machine tool worldwide. As it is fully integrated, it features not only a multiwavelength interferometer but also a miniaturized graphics processing unit (GPU). This allows for full data evaluation directly in the sensor. A single measurement is taken and processed within 3 s during wireless operation. HoloPort is easy to integrate into a variety of machine tools. This contribution includes detailed information about the sensor architecture. Experimental results on milled parts demonstrate the performance of the system and illustrate possible inline applications as well as future perspectives for the sensor.

1 Introduction

In machining production, the measurement of component dimensions and parameters is a fundamental part of quality and efficiency assurance. Therefore, the demand for on-machine metrology is continuously increasing. It avoids time-consuming realignment operations and possible loss or damage during transport between different machining and measurement stations. Luo and Qin (2018), for example, consider “hybrid machining” as one of the main future developments in production, and they demand a measurement technology suitable for use directly at the processing location, stating the following: “However, currently there is no method to realize on-line measurement of surface integrity.” (Jiang et al., 2018). Here, we present a sensor that addresses this problem.

On the one hand, coordinate measuring machines (CMMs) as well as scanning tactile measurement systems for use on tooling machines are reliable and well-established, but they are also quite slow and incapable of capturing the surface texture with a suitable resolution (Weck and Brecher, 2006b). Optical technologies, on the other hand, allow contact-free operation but are either limited in their application (reflecting parts for deflectometry), not fast enough (white-light interferometry, confocal, focus variation) or have insufficient accuracy (laser triangulation) (Knauer et al., 2006). In addition, none of these techniques are commercially available for in-machine use.

Figure 1

System architecture: equipped with the control module, the wired sensor head HoloCut becomes the fully integrated measuring system HoloPort and operates completely wirelessly. The label numbers used in this figure also correspond to the numbers used Figs. 2a, b and 3a.

[Figure omitted. See PDF]

Featuring a simultaneous high measurement speed and accuracy, digital holography has become a versatile tool for complete 3-D inline quality control in industry (Fratz et al., 2017). The origins of digital holography date back to the 1950s (Gabór, 1948). However, only the development of electronic sensors (Goodman and Lawrence, 1967) and increasing computational power, such as that provided by graphics processing units (GPUs), has enabled the use of digital holography for industrial purposes (Schnars and Jüptner, 1994; Kuwamura and Yamaguchi, 1997; Carl et al., 2009; Fratz and Carl, 2014; Bove et al., 2005; Masuda et al., 2006; Kolenovic et al., 2003). With the introduction of small, powerful computing units originally developed for autonomous driving, a fully integrated wireless measuring system becomes possible.

This contribution presents the first wireless interferometer that allows a 3-D surface measurement of freshly machined components directly inside machine tools. In order to be capable of measuring optically rough technical surfaces, the sensor system works based on the principle of multiwavelength digital holography and achieves accuracies in the sub-micrometer range. It features a multiwavelength interferometer with three stabilized single mode, single frequency laser sources and a miniaturized GPU – all fully integrated in a compact housing and allowing wireless operation.

2 Sensor description

Integrating a surface-measuring optical sensor into a machine tool poses a number of challenges. First, the sensor has to withstand the harsh environmental conditions. Second the transmission of energy and data is a challenge: tool holders are currently not prepared for either of them. The development of such standardized interfaces has been in progress for several years, but none are currently in use (Komet Group GmbH, 2017). Thus, commercially available tactile sensors work wirelessly and are battery powered. In order to make the measuring system accessible for all machine tools, the standard interface according to DIN 69893 was chosen. Therefore, all electronics necessary for the operation of the sensor had to be integrated into the measuring system.

In Figs. 1–3, the components of the measuring system will be introduced. The numbers given in the figures correspond to the same objects in each figure.

Figure 1 depicts the system architecture: the sensor system HoloPort consists of the optical sensor head HoloCut and an integrated control system. The optical design of HoloCut (Seyler et al., 2017a) is based on the industrially proven HoloTop sensor (Fratz et al., 2017). With a focus on the form factor, the height of a standard tool was set as the mechanical boundary condition for the mechanical design. A 3-D folded beam path, as shown in Fig. 2a and b, leads to a miniaturized sensor and enables mounting in the center of gravity. The sensor head itself has a size of $\mathrm{235}\times \mathrm{140}\times \mathrm{215}$ mm${}^{\mathrm{3}}$ and weighs 7.5 kg. For further details regarding the optical and mechanical design, please refer to our previous publication (Seyler et al., 2017b).

Figure 2

Sketch of the HoloCut sensor head bottom plane (a) and top plane (b) with interfaces to the control unit: camera (2), piezo actuator (3) and rotary stage (4). The machine integration in the center of gravity of the sensor (0) is realized by a 3-D layout of the optical beam path. (c) Finite methods natural frequency simulation of the HoloCut housing: the first eigenfrequency, shown here, is at 323 Hz and well above the frequencies occurring in machine tools ($<\mathrm{300}$ Hz).

[Figure omitted. See PDF]

Figure 3

(a) CAD model of the integrated control system including a battery powered (18) embedded architecture (11–14) and a fiber switch (8) toggling a three-laser system (6). (b) Detailed photograph of the embedded system.

[Figure omitted. See PDF]

At least three temporally phase-shifted interferograms (Cai et al., 2004) are recorded with a 9 MP (megapixel) camera (2), corresponding to a lateral sampling of approximately 7 $\mathrm{\mu }$m $\times$ 7 $\mathrm{\mu }$m and a field of view of 20 mm $\times$ 20 mm. Phase shifts are introduced by a piezo actuator (3). An integrated rotary motor (4a) with mounted half-wave plate and controller (4b) allows for the manipulation of the light distribution between reference and object light; thus, the sensor can be adapted to different surface conditions. A reference target (5) in the object beam path allows for absolute measurements and stitching of single measurements using encoder information of the machine tool.

The control system has been miniaturized and directly integrated into the sensor head. Figure 3a shows a computer-aided design (CAD) model of the complete measuring system HoloPort. The intended field of application of the sensor is the inspection of technical surfaces. Due to the roughness of such surfaces, unambiguous measurement ranges of at least a few micrometers are required. To achieve this, multiple lasers are used to record interferograms at different wavelengths (Wagner et al., 2000). From these interferograms the complex wavefront for each wavelengths is calculated, i.e., the amplitude and phase for each camera pixel. The combination of these complex wavefronts enable the numerical generation of phase images at so-called “synthetic” wavelengths, which correspond to the beat frequencies. The wavelength difference determines the unambiguous height measurement range: the smaller the difference, the larger the synthetic wavelength and the unambiguous measurement range. Large measurement ranges result in less accurate measurements. The combination of more than two wavelengths allows for cascading measurement ranges, and thus increases the ratio of measurement range and accuracy (Fratz et al., 2017).

Three 15–30 mW lasers (6) with wavelengths close to 632, 633 and 640 nm, respectively, are mounted on an aluminum block (7) and thermally insulated (8). A fiber switch (9) sequentially couples (1) these lasers to the sensor head. The system runs with different laser wavelengths and provides an additional external laser input (10). The results shown in this paper were achieved using synthetic wavelengths from 60 to 88 $\mathrm{\mu }$m. The extension of the unambiguous measurement range up to several millimeters has already been shown using additional lasers (Carl et al., 2009) or specialized algorithms (Seyler et al., 2018b).

Figure 4

(a) Schematic sketch of the machine tool with its moving axes and integrated HoloCut sensor head measuring in the $z$ direction. (b) Photograph of the real setup.

[Figure omitted. See PDF]

A microcontroller (11) controls the previously described components. A specially designed circuit board (12) handles energy supply and signal distribution. Besides accuracy, the speed of a sensor system decisively determines the economic efficiency and benefits in the machine tool. HoloPort uses a fully integrated, powerful computing unit compatible with the CUDA (Compute Unified Device Architecture) programming interface: a Jetson TX2 board (13) and its carrier board (14) coupled to a separate heat sink (15) allow not only wired (16)/wireless data transfer (17) according to the state of the art but also data reduction inside the sensor itself. The resulting camera data of up to 270 MB per measurement are preprocessed internally and, thus, reduced by half – a processing time of less than 3 s is achieved for 9 million sample points. The pure measuring time is less than 1 s.

In addition, the computing power allows for in situ quality control of the acquired raw data. Interferograms disturbed by vibrations are internally evaluated for sufficient interference contrast (Seyler et al., 2018c) even before the wireless data transmission takes place. The power consumption of the whole system is approximately 20 W. A 5.2 Ah battery (18) can support the system for up to 7 h. The battery is fully charged within 75 min via a magnetic connector (19) and can be powered on by a mechanical switch (20).

3 Integration and characterization

After presenting the basic functionality and structure of HoloPort, the integration and characterization of the machine tool and the measuring system will be discussed in the following section.

3.1 Vibration analysis

As an interferometric measurement principle, digital holography is very susceptible to vibrations. Therefore, the stiffness and natural frequencies of the carrier system with the installed HoloCut sensor head were first analyzed in the machine tool. For this purpose, HoloCut was operated as a classic interferometer with a reduced field of view. Interference fringes were recorded against a mirror that was placed in the machining volume. Using a rectangular camera ROI (region of interest) of only 15 lines lead to a sampling rate of 1515 Hz, allowing for the analysis of frequencies beyond 750 Hz according to the Nyquist theorem. The sketch shown in Fig. 4a depicts the machine tool with its moving axes and integrated HoloCut sensor head measuring in the $z$ direction. Figure 4b shows a photograph of the real setup.

First, the machine tool runs in normal operating mode but with disabled spindle control. Figure 5a shows the resulting vibration measurement in the $z$ direction. With a typical relative movement of $\sim \mathrm{50}$ nm, vibrations that occur are approximately 1 order of magnitude smaller than the illumination wavelength at 633 nm. At an exemplary exposure time of 1 ms they lead to a reduction of the interference contrast of only 1.1 %. The occurring vibration at a frequency of 84 Hz results in a maximum axial movement of 26 nm during this exposure time – this movement can be reduced by lower exposure times or increased laser power. Resultant effects have already been examined by Schiller et al. (2017) and are superimposed by movements between the image acquisitions. As shown in Fig. 5 for the machine tool in emergency stop mode, the vibration amplitude reaches only half of the previous value. The previously dominant frequency of 90 Hz has now completely disappeared. As all controls are deactivated in emergency stop mode, it can be assumed that this is the frequency of the machine axis control. Once the machine tool is completely turned off, the vibrations are again reduced by approximately 50 % in amplitude. The occurring frequencies are hardly affected at all.

Figure 5

Vibrations in different operational modes: (a) dominant control loop frequency of 90 Hz in operational mode, (b) reduced vibrations in emergency stop mode and (c) further reduced vibrations when the machine is turned off. Green lines correspond to the measured values; dashed black lines show the best-fit sinusoidal function.

[Figure omitted. See PDF]

In addition to these measurements, a confocal sensor and acceleration sensors were used to characterize the highly dynamic spindle control and vibrations in the $x$ and $y$ directions with equivalent results. Only the control loop of the spindle seems to have a noticeable negative effect on the measurement accuracy with HoloPort. Enabled spindle control results in a torsional vibration, leading to micrometer displacements perpendicular to the spindle axis. However, as the sensor should not rotate at all, this control loop is not necessary and can be deactivated during measurement.

In general, displacements that occur can be addressed by a combination of lateral and axial vibration compensation (Seyler et al., 2018c, 2019).

3.2 Performance

In order to characterize the performance differences in the measurement evaluation between the embedded system and the external measuring computer, a platform-independent software was developed which automatically performs a sequence of calculations with different parameters such as number of lasers and phase steps on a data set with typical filter operations and a resolution of 3008 pixels $\times$ 3008 pixels. For each combination, 120 measured values are recorded and then reduced to the respective mean value and standard deviation. Figure 6 shows these values for both processing strategies: the transmission of raw data with subsequent external data evaluation and internal data evaluation with reduced transmission.

Figure 6

Comparison of transmission of raw data (green) with external data evaluation (blue) to internal data evaluation (orange) with reduced data transmission (red). The latter configuration pays off with at least three lasers and/or five phase steps.

[Figure omitted. See PDF]

It turns out that internal data processing with three or more lasers or a minimum of five phase steps provides speed advantages. For three lasers and five phase steps, the amount of data before transmission can be reduced by a factor of 4, so that the total evaluation time drops from almost 8 s to about 4 s. The data processing time can be reduced by a factor of 2. The measured transmission time is significantly longer than expected. A theoretical transmission time of 0.7 s was expected for the processed data. The measured transmission time of the measured data, however, is 2.5 s on average, which is clearly above the calculated value. This can be caused by the low transmission speed at the beginning of the transmission and interference, e.g., from other Wi-Fi networks. The pure measurement speed of the system is independent of the evaluation and is far below 1 s; thus, in wired operation, a total measurement time including evaluation of about 1 s is achieved.

HoloPort has a maximum power consumption of 40 W, and the typical measured energy requirement is 20 W. Therefore, the choice of the evaluation strategy has an influence on the battery life. It turns out that with internal data processing more measurements can be carried out within one battery charge (4072 compared with 3533), but the battery is discharged earlier (388 min compared with 416 min). The battery life is up to 7 h, and charging the completely discharged battery takes about 75 min. As the Jetson systems offer various performance modes, even longer battery life seems realistic with intelligent performance control.

3.3 Temperature stability

High demands are placed on the measuring system for metrological operation in a machine tool. In production environments, the ambient temperatures differ by several degrees Celsius even in stationary operation of a machine tool, depending on the installation location. The temperature in the machine is determined by the type and height of the thermal load (travel speed, spindle speed, etc.). Temperature differences of up to 10 ${}^{\circ }$C in the tool chamber are quite realistic (Weck and Brecher, 2006a).

Therefore, in addition to the internal temperature control of each individual laser, the laser system was thermally connected to the measuring head (as introduced in Sect. 2) and isolated from the rest of the electronics. Figure 7 shows the temperature curve of HoloPort under full load with lasers switched off and a more or less constant ambient temperature of 23 ${}^{\circ }$C.

Figure 7

Temperature curve of HoloPort at 23 ${}^{\circ }$C ambient temperature: the isolation of the laser chamber from the Jetson chamber fulfills its purpose and leads to 20 ${}^{\circ }$C less heating.

[Figure omitted. See PDF]

In this scenario, the heat input of the evaluation electronics (Jetson chamber) and the camera to the laser chamber can be observed. The latter only sees a small part of the high heat development of the evaluation electronics; thus, the lasers only heat up by about 5 ${}^{\circ }$C, whereas the chamber of the evaluation electronics heats up by about 25 ${}^{\circ }$C to almost 50 ${}^{\circ }$C. It is remarkable that the laser chamber faces lower heat input than the measuring head itself, which is only heated by the $<\mathrm{4.5}$ W heat input of the camera. Due to the interferometric measuring principle, slowly drifting temperatures of the HoloCut housing are not critical for the measurement of a single measuring field and only play a role in measurements with absolute references. In addition, the sensor head responds very slowly to external temperature fluctuations due to its large thermal mass. The Jetson internal temperature sensor shows a temperature of 30 ${}^{\circ }$C without load, which increases to 70 ${}^{\circ }$C under load. All observed temperatures are within the permissible operating temperatures of the individual components.

In general, there are two classes of manufactured surfaces in milling: engineered surfaces and non-engineered surfaces. Engineered surfaces such as bearings and seals are generally manufactured to provide functional properties or are subject to special safety requirements (Stout and Blunt, 2001).

Figure 8 shows the inspection of a typical non-engineered milling surface in a Hermle C32U 5-axis milling machine. HoloPort easily resolves the milling structure of the surface, revealing height differences $h$ between individual milling paths of up to 10 $\mathrm{\mu }$m. A cross section of 10 individual measurements at the same position illustrates the repeatability in the sub-micrometer range. With a measuring field of up to 20 mm $\times$ 20 mm at 9 million sample points, irregularities within the microstructure as well as machine faults quickly become apparent. In contrast to the tactile state-of-the-art sensors, the system cannot only resolve longwave components but also shortwave structures and defects of the milling operation. Due to the inherent morphological filtering of the probe head, tactile sensors, in principle, cannot detect these characteristics. The benefit of the measuring system in the optimization of milling parameters quickly becomes obvious at this point.

Figure 8

(a) Inspection of a typical milling surface by HoloPort inside a Hermle C32U 5-axis milling machine at Fraunhofer IPM. (b) Measured height differences $h$ between milling paths of a typical milled part in the range of 10 $\mathrm{\mu }$m. A cross section of 10 individual measurements at the same position demonstrates the measuring repeatability.

[Figure omitted. See PDF]

For comparison to white-light, interferometric and confocal laboratory reference 3-D measuring devices, surface parameters according to ISO 25178 were previously evaluated: all three measuring methods deliver comparable results and are able to detect both an increasing roughness of the surface due to wear of the milling cutter and a machine error (Seyler et al., 2018d).

Not only the analysis of surface parameters, but also the extensive measurement of different inspection parameters represent applications for quality monitoring (Schiller et al., 2017). Figure 9 shows a milled laser heat sink with mapped surface measurement.

Figure 9

HoloPort maps the component surface in a fraction of a second with micrometer accuracy directly in the machine tool (as in the example of a heat sink for laser assembly). Alongside the geometry, relevant surface parameters such as micro-defects and microstructures as well as the height, parallelism and inclination of surfaces can be measured simultaneously.

[Figure omitted. See PDF]

The full-field measurement shows the surface quality and additionally allows for the quantitative extraction of geometrical parameters: micro-defects and microstructures as well as the height, parallelism and inclination of surfaces can be evaluated based on the gathered measurement data.

Using the dynamic numerical control (DNC) interface of the machine tool, axis positions can be read out and used for a CAD comparison – the system becomes a full-field CMM. Even complex work pieces can be inspected completely and directly within the machine for the first time (Seyler et al., 2018a). The accuracy of a complete point cloud of the surface is limited by the accuracy of the machine tool axes, which is typically in the single micrometer range.

Further test measurements of a complex formed milling part have been carried out to demonstrate feasibility of quality inspection directly in the machine tool instead of using an additional CMM situated within a temperature-controlled measurement room. The combination of more than 30 measurements for flatness determination has been carried out with an extension of 100 mm $\times$ 60 mm, which corresponds to almost 300 million data points. Evaluation of the combined data set shows flatness information within the single micrometer range. The determination of lateral distances between reference marks located on the measured surface in the range of the lateral sampling of the sensor at 7 $\mathrm{\mu }$m appears realistic.

For additional application examples and measurement results please refer to Seyler et al. (2017a), Seyler et al. (2018a, b, c, d), Seyler (2019) and Seyler and Engler (2019).

4 Perspectives

An optical sensor measuring the surface integrity with a large field of view directly inside the machine tool opens up new possibilities for process control and optimization (Luo and Qin, 2018).

The feedback of the measurement results into the manufacturing process is a key point, especially in hybrid manufacturing. As the Jetson module was originally developed for autonomous driving purposes, the system is also capable of machine learning analyses.

Furthermore, the system is prepared for use on CMM and robots – the possibilities and limits will be determined in future studies. In addition, the measuring system will be further miniaturized to be compatible with standard tool changers.

Digital holography has even been proven to work on moving objects at speeds of several millimeters per second – simplified handling becomes possible for inline inspection, which, in turn, allows for faster production control (Schiller et al., 2017). The limits of a scanning sensor in a multiaxial system will be analyzed in future experiments.

5 Summary

Online measurements of surface integrity in machined production have been motivated. Digital holography meets the requirements for measurement accuracy in the micrometer range at simultaneous high measurement speed.

A new digital holographic sensor system for 3-D surface measurements inside a machine tool has been presented and characterized. It features 9 million 3-D samples and, to our knowledge, it is the first time that micrometer precision of such a system on an area of up to 20 mm $\times$ 20 mm has been demonstrated with various milled samples. Due to the short processing time of 3 s (pure measuring time of less than 1 s), closed-loop quality control directly inside the machine tool is feasible for the first time.

6 Outlook

Further investigations of the detection and compensation of vibrations will be carried out in the future.

Data availability

Data were been submitted on 28 January 2020: 10.24406/fordatis/51 (Seyler et al., 2020).

Author contributions

TS was responsible for the methodology, conceptualization, data curation, investigation, software, visualization, the project administration and writing the paper; JE contributed to the conceptualization and software; TB contributed to the methodology, conceptualization and software; MF was responsible for the methodology, conceptualization, funding acquisition and project administration; AB contributed to funding acquisition and supervision; and DC was responsible for funding acquisition and supervision.

Competing interests

The authors declare that they have no conflict of interest.

Special issue statement

This article is part of the special issue “Sensors and Measurement Systems 2019”. It is a result of the “Sensoren und Messsysteme 2019, 20. ITG-/GMA-Fachtagung”, Nuremberg, Germany, 25–26 June 2019.

Acknowledgements

We would like to thank the Baden-Württemberg Stiftung gGmbH for their financial support regarding the development of the HoloCut sensor. We appreciate the support and good cooperation from Hermle AG that was given during the integration of the measurement system.

Review statement

This paper was edited by Jürgen Czarske and reviewed by three anonymous referees.