A Tutorial Review on Point Cloud Registrations: Principle, Classification, Comparison, and Technology Challenges

Abstract

A point cloud as a collection of points is poised to bring about a revolution in acquiring and generating three-dimensional (3D) surface information of an object in 3D reconstruction, industrial inspection, and robotic manipulation. In this revolution, the most challenging but imperative process is point could registration, i.e., obtaining a spatial transformation that aligns and matches two point clouds acquired in two different coordinates. In this survey paper, we present the overview and basic principles, give systematical classification and comparison of various methods, and address existing technical problems in point cloud registration. This review attempts to serve as a tutorial to academic researchers and engineers outside this field and to promote discussion of a unified vision of point cloud registration. The goal is to help readers quickly get into the problems of their interests related to point could registration and to provide them with insights and guidance in finding out appropriate strategies and solutions.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

With the advent of low-cost three-dimensional (3D) imaging devices and the development of professional 3D acquisition techniques, more and more research efforts have been attracted toward the 3D point cloud from industry to academia in recent years. In contrast to a 2D image, a 3D point cloud data contain much more information of an object or a scene, and hence, it can provide a better understanding and description of the real world. Specifically, the point cloud can be used to measure the distance between objects in the current frame, achieve an accurate representation of the shape of an object, and so on. Therefore, 3D point clouds have been widely produced and processed to provide accurate and fast descriptions of the 3D geometries of objects in various applications [1], such as 3D model reconstruction, geometry quality inspection, and robotic manipulation.

However, it is impossible to obtain all the point cloud data of an object or a scene at once in practice. Therefore, we usually move the 3D acquisition device around a target object to finish a throughout scan, such that retrieving multiple point clouds at different viewpoints for better rebuilding the 3D environment or recovering the 3D shape of an object. It is critically important to capture these point clouds in such a way because (1) the Field of View (FoV) of a depth camera usually is not big enough to capture the massive scene, such as the forest and city, and (2) the quality of a single-achieved point cloud is generally not acceptable. For example, due to the low resolution of a camera, we need to fuse more frames into a global scene. In addition, to overcome the holes in the point cloud that are mainly caused by the reflection of the light on an object’s surface, multiple frames at different viewpoints are required. In short, to obtain an accurate point cloud, the practical approach is to collect multiple point clouds in an appropriate manner and integrate them together. Furthermore, it is required to know the orientation of a scanned object represented by its point cloud data in its local coordinate. We have to match a standard point cloud in the dataset with the scanned point cloud. The key technology that makes several point clouds fused into a complete point cloud with a common coordinate system is called point cloud registration, also called surface registration, in which point clouds are analyzed to represent the real-world surface of the object. It is a key step for applications and tasks, such as autonomous driving [2–5], 3D reconstruction [6–10], simultaneous localization and mapping (SLAM) [11–14], and virtual and augmented reality (VR/AR) [15–17]. In the application of point cloud registration, each point cloud acquired from different viewpoints is measured within their local coordinates. Therefore, the objective of registration is basically to calculate the transformation (translational and rotational) between point clouds to transfer these point clouds into a common mapping coordinate system. According to the number of the input data, point cloud registration can be divided into pairwise registration and multiview registration [18]. The pairwise registration particularly focuses on two point clouds and estimates the transformation between both of them, while the multiview registration, also called groupwise [19], performs more than two point clouds simultaneously, and the transformation of each scan to the reference data is calculated. According to the accuracy performance of registration, it can be divided into two stages: coarse registration and fine registration in the task of point cloud registration [20]. The coarse registration is intended to provide a rough transformation in terms of accuracy with an arbitrary initial position, and the fine registration aims to provide a relatively more accurate result for two point clouds so that the point clouds have more common regions where points become closer. Depending on whether the object of registration is deformed, the registration methods are also divided into rigid registration and nonrigid registration [21]. On the other hand, according to the types of the theoretical solutions to point cloud registration, point cloud registration can mainly be split into five categories: iterative closest point (ICP)-based methods, feature-based methods, learning-based methods, probabilistic methods, and others [22–25]. The ICP and its variants are classic solutions. They can provide an accurate and reliable transformation between two point clouds through an iterative algorithm of correspondences with a good initial position and rotation. However, the final registration results are very sensitive to the initial position and rotation [26]. Feature-based methods aim to extract the meaningful and robust local geometrical descriptors as correspondences to estimate a transformation between two point clouds. The extracted descriptors can be subdivided into local descriptors [27–29] and global descriptors [30–33]. Local descriptors are extracted from the interest part of the point cloud, and the global descriptors are generated by encoding the geometric information of the whole point cloud. Although the global descriptors have a good performance on the data with obvious geometric attributes, it is difficult to be robust on point cloud containing outlier or flat surfaces.

Learning-based methods provide a relatively more robust transformation between two arbitrarily point clouds by invariant features generated by machine-learning techniques. Thanks to machine learning, feature extraction in registration can be more invariant because the learned descriptors may contain more detailed features than geometric characteristics or other handcrafted descriptors. The probabilistic method models input point cloud as a density function, such as Gaussian mixture model, and then optimize a statistical discrepancy among probabilistic correspondences to estimate final transformation. One of the advantages of this method is that it can be applied to rigid and nonrigid registration. Other solutions include methods with auxiliary data, in which transformation information is provided from other modules and techniques, and the 4-points congruent sets algorithm based on random sample consensus (RANSAC). The registration algorithms discussed in this work could be used in the pairwise or groupwise/multiview registration and could be applied in a fine or coarse stage with respect to accuracy. Although some of them can be used in nonrigid registration, we will primarily focus on rigid registration in this paper.

Several valuable reviews so far have been published on this topic in the last decade. For instance, Pomerleau et al. [34] presented a general framework based on ICP to classify the existing solutions and algorithms during the last twenty years and mainly focused on the use cases for mobile robotics applications that cover different kinds of platforms, environments, and tasks. It also introduced several realistic situations and applications based on point cloud registration, such as search and rescue tasks, automation of inspection, shoreline monitoring, and autonomous driving. Tam et al. [35] gave a comprehensive summary on rigid and nonrigid registration of point cloud and mesh. Cheng et al. [20] reviewed the existing registration methods based on laser scanning point cloud mainly in photogrammetry and remote sensing. Li et al. [36] gave a good statement of the registration problem and presented a novel framework for globally solving the problem grounded on the Lipschitz global optimization theory with unknown point correspondences. Ben et al. [37] provided an overview of the most important rigid registration algorithm on the estimation of the optimal rigid transformation and discussed their common mathematical foundation. Zhen et al. [38] conducted a comprehensive survey on terrestrial laser scanner (TLS) point cloud registration methods and presented a benchmark dataset containing various scenes. Li et al. [39] made a comprehensive evaluation on the ICP algorithm in terms of overlap ratio, angle, distance, and noise factors. Jiang et al. [40] and Ma et al. [41] provided a comprehensive and systematic review of image registration, and briefly introduced several typical applications in which 2D image registration was discussed in more detail. Specifically, Ma introduced the existing methods of feature-based matching from handcrafted to trainable ones with a comprehensive review. Jiang presented two general frameworks for area and feature-based pipelines and briefly introduced and analyzed typical applications to reveal the significance of multimodal image matching.

Our contributions are to (1) split the point cloud registration problem into the main five categories based on the research publication in the last decades, which is beneficial for new researchers and potential users of registration; (2) systematically summarize point cloud registration techniques and analyze the similarities and differences of the currently existing methods; and (3) introduce up-to-date achievements, challenges, and further research efforts on the applications based on point cloud registration.

The structure of this paper is organized as follows: Section 2 describes the conception of point cloud registration and the issues in this area. In Section 3, the best-known solutions in ICP-based, feature-based, learning-based, probability-based methods, and others are reviewed and discussed, respectively. Finally, the conclusion and outlook on point cloud registration is given in Section 4.

2. Basic Concepts and Principles of Point Cloud Registration

Point cloud registration is a fundamental technology in computer vision and robotics and is considered as an important prerequisite in various applications that use point cloud data as the input. In this section, we first briefly introduce the source of the point cloud, namely, the point cloud acquisitions. Then, the definition of point cloud data is provided, and it is followed by the introduction of the problems on point cloud registration, and other issues and constraints in this field.

2.1. Point Cloud Acquisitions

Point clouds can be generated by a 3D/depth camera directly or calculated by photogrammetric techniques. Typically these depth cameras can be divided into two categories [42]: passive method based and active method based. A passive method obtains the depth information to generate point cloud data through computing the relationship among multiple view images or a sequential set of images taken at different viewpoints. For instance, structure from motion (SfM) and multiview stereo (MVS) [43] can be integrated to produce point cloud data with high quality in terms of resolution and accuracy [44, 45]. Given a series of images of an object captured at different views (image $i$ acquired at position $i$), the SfM can utilize these images to create a sparse point cloud based on the correspondence between two images (see Figures 1(a) and 1(b)). For a dense point cloud, the MVS technique is usually considered [44, 46], in which images are taken with the pose as the input provided by SfM, for finally producing a point cloud nearly comparable in accuracy to those generated by laser scanners. The PMVS is one of the well-known methods. It begins by feature detection and matching, and then the expansion and filtering step aims to reconstruct and remove erroneous patches repeatedly. The flowchart of PMVS is illustrated in Figure 1(c). In Figure 1, point clouds generated from SfM and PMVS are shown, respectively. Stereo-depth camera [47] is another type of camera with two or more image sensors spaced a small distance apart so that it is allowed to simulate the human binocular vision to perceive depth information by the determination of correspondence between points in images and the computation of depth values based on point correspondences [48].

[figures omitted; refer to PDF]

The primary advantage of these approaches is that they function well in most outdoor environments. Therefore, there are many outdoor applications of scanning surrounding scene and environment, such as building [49, 50], streets [51], and city [52, 53], using the passive method.

On the other hand, the active devices, such as the time-of-flight (ToF) camera, also called LiDAR, and structured light camera, usually emit or project light and utilize their properties to calculate depth information. Since the speed of light is known, the ToF cameras can calculate the distance between the camera and target objects by measuring the round trip time of a modulated light signal. The speed of generating the final point cloud data can reach 120 fps [54] so that it is widely used in real-time applications, such as autonomous self-driving [55] and gesture recognition [56]. Moreover, depending on the wavelength and power of the emitting light, the ToF camera can provide depth value at significant distances and high-resolution. A structure light camera projects the known patterns, such as phase lines, onto the surface of an object or scene to calculate depth information based on these deformed patterns [57, 58]. As illustrated in Figure 2, the vertical straight stripes are projected onto the object and be distorted because of the surface profile. Eventually, 3D shape information can be obtained by the analysis of these images. Because the structured light camera can obtain high-resolution and speed at a relatively short range [59], it is widely used in fields that require high quality of point cloud in terms of accuracy and resolution, such as 3D shape measurement field [59] and medical field [60].

[figure omitted; refer to PDF]

The basic principle and its applicable scenes of point cloud acquisitions are summarized in Table 1.

Table 1

A overview of point cloud acquisitions.

Type	Approach	Properties	Application
Passive	SfM-MVS	Take multiple view images as input to generate a sparse point cloud, and then make it dense.	Outdoor/indoor
	Stereo camera	Simulate human binocular vision to determine the position of a point to generate a point cloud.	Outdoor/indoor
Active	ToF camera	Compute a round trip time of a modulated light signal from target object’s surface to produce a point cloud.	Outdoor/indoor
	Structured light camera	Use distorted patterns projected to the surface of object to make a point cloud.	Indoor

2.2. Basics of a Point Cloud

As shown in Figure 3, a point cloud is a collection of points, i.e., $S=p_1,p_2,\dots ,p_n$, and these points describe the shape and surface of an object or scene in 3D. Generally, a point $p$ at least contains the $x$, $y$, and $z$ values that define basic geometric coordinates for the target objects, so that $S{p}_1,p_2,\dots ,p_n|p\in {ℝ}^3$. However, when some properties of a single point in the point cloud are available, such as curvature $k$, normal vector $n$, and the color information ($r,g,b$), the point $p$ could be represented by a longer vector. An example of the point cloud provided by the KITTI dataset [61] is illustrated in Figure 3.

[figure omitted; refer to PDF]

Apart from the computation of descriptors, the next step for point cloud registration is to establish correspondences between features extracted from reading and reference data. A threshold is set for finding the nearest neighbor to match point-to-point correspondence. As shown in Figure 16(a), $p_{313}$ and $p_{5210}$ are a pair with under a specific distance in feature space. In the registration experiment, a coarse registration is obtained by matching SHOT descriptors, in which a transformation is calculated based on the area of overlap where correspondences can be extracted. Then, the transformation leads into an initial guess into the ICP algorithm to achieve a fully automatic registration point cloud, as illustrated in Figure 16(b).

[figures omitted; refer to PDF]

3.2.3. Further Steps after Establishment of Correspondences

This section introduces the further steps after obtaining a large set of correspondences. To our knowledge, given a large set of correspondences, a popular strategy was proposed [146, 147] and consisted of two steps: (1) building a set of putative correspondences containing outliers and inliers; (2) removing the outliers and estimating the transformation using inliers.

For some point cloud registration tasks, the point-to-point correspondences are always established through associating each descriptor to its nearest neighbor [141] in the reference point cloud in the descriptor space. The transformation produced by these correspondences can be directly considered as a rough result and then improved by other algorithms such as ICP. However, this strategy might not be accurate enough, because the correspondences obtained from two point clouds include inliers and outliers generally, and the ICP are sensitive to noises and outliers. Additionally, the real-world data contain noises, outliers, occlusion, and even flat surfaces with few apparent features, such that the registration performance is severely degraded because the extracting of traits from the point cloud is affected.

Therefore, designing an algorithm to remove outliers and estimate transformation based on inliers efficiently is important. A key of this task is to remove mismatches from given putative feature correspondences, leaving the true matches to calculate transformation. To this end, a simple way is to reject random correspondences based on the hypothesize-and-verify framework [148]. However, the notable drawback of it is it requires a large amount of trials, especially when the inlier ratio is low and the expected confidence to find correct matches is high [149]. To eliminate the mismatches fast, a mismatch removal method was proposed [150] that preserves local neighborhood structures of feature points from thousands of correspondences in few milliseconds, resulting in a quick initialization result for registration. Ma et al. [147] considered constructing a putative set of correspondences and rejecting matches due to sufficiently different feature descriptor vectors. In addition, for nonrigid and rigid registration, a solution that robustly estimated the transformation from correspondences was proposed [151, 152] using $L_2$-minimizing estimate [153], but it is severely biased by outliers. A set of putative correspondences could be purified by semisupervised learning [154] or a mixture of densities [155]. However, these feature-based methods compute transformations solely based on a set of putative correspondences instead of the entire input data.

3.3. Learning-Based Methods

In recent years, learning-based approaches have received significant attention in many fields related to computer vision, especially object detection, classification, semantic segmentation, etc. In applications that use a point cloud as an input, traditional strategies to estimate feature descriptors heavily rely on distinctive geometric properties of the object in the point cloud, as we discussed in the last section. However, the real-world data often vary from object to object that may contain flat surfaces, outliers, and noise. Moreover, the removed mismatches usually contain useful information and can be used for transformation learning.

Learning-based techniques can be suitable to encode semantic information and be generalized in specific tasks. Specifically, most registration strategies integrated with machine-learning techniques are faster and more robust than classical methods and flexibly extended to other tasks, such as object pose estimation and object classification. Similarly, a key challenge in point cloud registration based on learning is how to extract features that are invariant to the point cloud’s spatial change and more robust to noise and outliers. In this section, we will describe and discuss several state-of-art approaches integrated with machine learning.

3.3.1. Major Methods

PointNet introduced in [156] is a deep network architecture and is fed by an unstructured point cloud ($n$ points) directly. It outputs either a global feature represented by a single vector or the local features for each point through combining the global feature with each of the point features, for the tasks of object classification and semantic segmentation separately. PointNet is composed of three parts: (1) symmetry function for unordered input; (2) local and global information aggregation; and (3) joint alignment network, as shown in Figure 17. It should be noticed that the alignment in the network is used to align both input points and point features, and this alignment can make sure that the learned representation is invariant to the geometric transformation of the point cloud. Although PointNet for 3D descriptors [157] mainly focuses on 3D classification and segmentation, they indeed provide research studies with more inspirations on the registration task because the descriptors used in segmentation task and 3D object classification bring a different perspective for correspondence establishment.

[figure omitted; refer to PDF]

As the pioneering effort that directly processes raw point cloud, PointNet++ [158] developed from the PointNet can learn local features by exploiting metric space distance. PointNet++ is a hierarchical neural network architecture that consists of a set of learning layers to combine features from multiple scales, whereas the traditional PointNet is to learn from spatial information of points and then aggregate these features into a global signature of the point cloud. This hierarchical structure includes a bundle of set abstraction levels which are composed of a sampling layer, a grouping layer, and a PointNet layer. The sampling layer focuses on selecting a set of points from input data. The grouping layer focuses on constructing local region sets for each point. The PointNet layer aims to produce feature vectors by encoding the local region pattern. A hierarchical architecture of PointNet++ is given in Figure 18, $N$ denotes the points with $d$ dimension coordinate and $C$ dimension feature, and $K$ is the number of centroid points’ neighborhood.

[figure omitted; refer to PDF]

Point Cloud Registration Network (PCRNet) [159]. It presents a framework combines PointNet features to find the transformation between point clouds. In particular, it uses five multilayer perceptrons (MLPs) which are similar to the PointNet architecture and are arranged in a Siamese architecture, to encode the point cloud as global feature vectors that contain the information on the geometry and orientation of point clouds. Followed by estimating the transformation with a single forward pass in the fully connected (FC) layers, which is different from PointNetLK [160] that uses a traditional algorithm to calculate the transformation. An architecture of PCRNet is illustrated in Figure 19. Point clouds first are fed as input into five MLPs $64,64,64,128,1024$, and the output is global features generated from symmetric max-pooling function $\varphi$. These features are concatenated and provided as input to fully connected layers, and the final result that includes translation and rotation values will be calculated in an FC with $1024,1024,512,512,256,7$. Experiments showed that iterative PCRNet is similar to Go-ICP with respect to accuracy but much faster based on ModelNet40 [161].

[figure omitted; refer to PDF]

Deep Global Registration (DGR) [24]. It consists of three modules: a 6D convolutional network for estimating the veracity of each correspondence, a weighted Procrustes solver for a closed-form rigid registration, and an optimization module that fine-tunes the registration generated from the previous step. Specifically, it first extracts features from the pairwise point cloud using fully convolutional geometric features (FCGFs) [162], and then a translation invariant convolutional network is used to identify the correct correspondences. Finally, the weighted mean squared error is minimized by the Procrustes method [163] to generate rotation $R$ and translation $T$, as shown in Figure 20. To improve the accuracy of registration, a fine-tuning module is proposed finally to minimize a robust loss function, in which the optimization space is processed smoothly by removing discontinuous. Experiments showed that it can achieve a good performance on real-world data, 3DMatch [164].

[figure omitted; refer to PDF]

Deep Closest Point (DCP) [165]. To overcome the intrinsic drawbacks of ICP, i.e., converging to spurious local optima, DCP was introduced by using learned embedding to guarantee a better marching. It has three parts: a point cloud embedding network, an attention-based module to predict a soft-matching that is generated from a probabilistic approach, and an SVD layer to estimate rigid transformation. In more detail, the raw point cloud is embedded into high-dimensional space by PointNet or dynamic graph convolutional networks (DGCNNs) [166], followed by an attention and pointer generation module to learn co-contextual information and establishment of soft correspondences. Finally, the transformation between point clouds is calculated in a differentiable SVD layer. As shown in Figure 21, ${\mathrm{ℱ}}_x$ and ${\mathrm{ℱ}}_y$ are feature embeddings, respectively, for $\mathcal{X}$ and $\mathcal{Y}$, and ${\mathrm{\Phi }}_x$ and ${\mathrm{\Phi }}_y$ are new embeddings that have contextual information. $m{x}_i,\mathcal{Y}$ is a soft pointer from each $xi$ into the points of $Y$. Finally, we can get pairs containing $x_i$ to $y_i$ over all $i$. In particular, this is a DCP-v2 integrated with an attention module while the DCP-v1 does not. Additionally, because PoinNet and DGCNN both can extract features from the point cloud, but PoinNet learns global descriptors while DGCNN learns local descriptors. In this article [165], the proposed method based on DGCNN empirically performs better than the PoinNet in terms of accuracy on the dataset ModelNet40 [161]. However, this method is not accurate enough. Thus, it can be further polished by the ICP algorithm after a better initialization provided by DCP to converge to the global optimum.

[figure omitted; refer to PDF]

Partial Registration Network (PRNet) [23]. It tackles partial correspondence problems by learning geometric priors with self-supervised learning. PRNet is composed of an appropriate geometric representation and a keypoint detector that focuses on finding common points and establishing correspondences. Moreover, the PRNet is designed to be iterative like ICP, such that it is capable of performing coarse-to-fine refinement. Specifically, PRNet first detects keypoint of point cloud $X$ and $Y$ and then predicts a mapping and rigid transformation ($m_p$) from $X$ to $Y$ based on the keypoints, as shown in Figure 22. Finally, the obtained transformation is applied to $X$, and the process is executed repeatedly with the new transformation as input. However, to be sharper and approximately differentiable than traditional methods [167], PRNet can be considered as an actor-critic method (ACM), using Gumbel-Softmax [168] to sample a matching matrix for predicting a mapping of keypoints from $X$ to $Y$. Experiments showed that PRNet is more robust to noise on the ModelNet40 [161].

[figure omitted; refer to PDF]

Robust Point Matching (RPM-Net) [169]. It builds upon the previous iterative framework RPM [170] that uses optimization technique and soft assignment strategies to solve point matching problem. RPM-Net has mainly three parts: (1) computing the hybrid features combined with spatial coordinate and geometric properties of points; (2) a differentiable Sinkhorn [171] layer and annealing algorithm [172] is applied to get soft assignments of correspondences from these features; (3) the weighted SVD is used to solve the rigid transformation. In particular, features for every point from the point cloud are extracted by PointNet and FPFH [27], followed by an annealing algorithm for soft assignment and Sinkhorn normalization that produces the final correspondences $M^i$, during which a secondary network is optional used to predict and adjust the annealing parameters ($\alpha$, $\beta$) during iteration. Finally, the rigid transformation is computed by SVD while each correspondence $\widehat{Y}$ is weighted. The overview of RPM-Net is shown in Figure 23. The result data obtained in experiments suggest that it is robust to noise data on the dataset ModelNet40 [161] based on mean isotropic rotation and translation errors.

[figure omitted; refer to PDF]

PointNetLK [173]. It builds a deep neural network framework derived from PointNet and is integrated with the Lucas–Kanade (LK) [174] algorithm which is classic techniques for image alignment. To this end, PointNetLK begins by producing $K$-dimensional vector descriptor from PointNet function $\varphi$, and then $\mathrm{\Delta }G$ as current transform is represented by an exponential map and estimated in the LK layer, estimating the alignment without costly computation of point correspondences. The final transformation $G_{\text{est}}$ is obtained by all incremental estimates computed during the iterative loop. This model is shown in Figure 24. PointNetLK evaluated on the dataset ModelNet40 [161] demonstrates more robust to noise data and is insensitive to initial position than the ICP algorithm.

[figure omitted; refer to PDF]

3DRegNet [175]. Given a set of noisy point correspondences, it focuses on classifying inliers and outliers because inliers mainly contribute to the estimation of an optimal pose. To end this, a deep neural network (DNN) for registration is designed to compute the transformation parameters ($R$ and $T$). Specifically, 3DRegNet starts by passing a set of point correspondences through a fully connected layer, during which 128 dimensional features for every correspondences are calculated and will be fed into ResNets [176] to obtain weighted $w_i\in \mathrm{0,1}$ correspondences (see Figure 25(b)). Subsequently, DNN is presented to extract and process meaningful features by pooling and context normalization. Then, the transformation is estimated at the end registration block (see Figure 25(c)). The architecture of 3DRegNet is shown in Figure 25(a). Moreover, an additional 3DRegNet can be used to compute a smaller transformation for a more precise result. A series of experiments based on the dataset ICL-NUIM [177] and SUN3D [178] suggest that a deep learning approach indeed performs better under the condition of lower quality of point correspondences.

[figures omitted; refer to PDF]

3.3.2. Comparison and Summary

In this section, the learning-based registration methods are summarized and compared in Table 3. For a better understanding, approaches mentioned in Table 3 are arranged in chronological, such that PointNetLK was introduced in 2019, whereas DGR was proposed in 2020. Thanks to the existence of various datasets, many learning-based methods can be put together and compared with each other in terms of the same indicators, such as time consumption, accuracy, and robustness to noise. However, in Table 3, the performance based on evaluation metric is our main concern because time consumption is highly dependent on the computational devices which are very diverse. Furthermore, the robustness to noise is also difficult to measure uniformly, since different noise generation distributions may affect the performance of algorithm unless the algorithm is tested for several times with different noise distributions, as in the case of Peng Li [39] applied various standard deviations of Gaussian noise to point cloud.

Table 3

Comparison among these learning-based methods.

Year	Method	Network type (for correspondence estimation and transformation estimation)	Need refinement	Tested dataset	Evaluation metric	Performance	Tested method
2020	Deep global registration (DGR)	Fully convolutional geometric features (FCGFs)/weighted Procrustes method	No	3DMatch	Translation error rotation error	Better than 3DRegNet, DCP, PRNet, and PointNetLK	—
2020	Robust point matching network (RPM-Net)	PointNet + FPFH/weighted SVD	No	ModelNet40	Mean isotropic	Better than PointNetLK and DCP-v2	—
					Rotation error
					Mean isotropic
					Translation error
2020	3D registration network (3DRegNet)	ResNet + FPFH/DNNs	No	ICL-NUIM, SUN3D	Translation error	—	Only 3DRegNet
					Rotation error
2019	Point cloud registration network (PCRNet)	PointNet/fully connected layers	No	ModelNet40	Axis-angle of rotation error	Better than PointNetLK	Iterative PCRNet
					L2-norm of translation error
					Area under the curve
2019	Partial registration network (PRNet)	DGCNN and transformer/an Actor-critic method	Yes	ModelNet40	Root mean squared error	Better than PointNetLK and DCP-v2	—
					Mean absolute error
					Coefficient of determination
2019	Deep closest point (DCP)	Dynamic graph convolutional networks (DGCNNs)/singular value decomposition (SVD) layer	Yes	ModelNet40	Root mean squared error	Better than PointNetLK	DCP-v2 (DCP with attention module)
					Mean absolute error
2019	PointNetLK	PointNet/LK algorithm	No	ModelNet40	Translation error	—	—
					Rotation error

“—” in Performance implies that this method compares with traditional solutions rather than the learning-based techniques. The Tested method refers explicitly to the solution tested.

Moreover, Network type refers to the predominant methods used to solve correspondence estimation and transformation estimation problem, though there are secondary networks applying for prediction of specific parameters, such as RPM-Net optional uses a secondary network to adjust parameters during iteration. Need refinement indicates whether this method needs other solutions to produce a higher precise result, probably because the accuracy is not enough. Eventually, Tested dataset and Evaluation metric are an important part of characteristics for learning-based methods because a uniform and standardized indicator is provided for performance.

3.4. Methods with Probability Density Function

Point cloud registration based on a probability density function (PDF), such that using a statistical model for registration, is a well-studied problem [179, 180]. The key idea in this method is to represent data by a specific probability density function, such as Gaussian mixture model (GMM) and normal distribution (ND). Then, the registration task is reformulated to the problem of aligning two corresponding distribution, followed by an objective function measuring and minimizing a statistical discrepancy between them [179]. Meanwhile, due to the representation of PDF, the point cloud can be regarded as a distribution rather than many individual points, so that it avoids the estimation of correspondences and has a good performance of robustness to noise, but in general slower than ICP-based methods [181]. In this section, methods jointly with probability density function for point cloud registration are introduced.

3.4.1. Gaussian Mixture

The point cloud registration task can be reinterpreted as a mixture density estimation and likelihood maximum problem. Therefore, a general framework, namely, mixture point matching (MPM), was introduced by Chui et al. [25]. In particular, let us take a GMM under Euclidean transformation [180] as an example, given two point clouds that were defined by $X$ and $Y$ individually. Let ${\mathrm{\Omega }}_X=x_i,{\sigma }_{ix}^2,{\varphi }_i^X|1\le i<n$ and ${\mathrm{\Omega }}_Y=y_j,{\sigma }_{jy}^2,{\varphi }_j^Y|1\le j<m$ be GMMs constructed from $X$ and $Y$, with means single point $x_i$ and $y_j$, variances ${\sigma }_{ix}^2$ and ${\sigma }_{jy}^2$, mixture weights ${\varphi }_i^X$ and ${\varphi }_j^Y$, and the number of items $n$ and $m$, respectively. Then, the $L_2$ distance transformed ${\mathrm{\Omega }}_X$ and ${\mathrm{\Omega }}_Y$ is given by$$\begin{array}{}\text{(7)}& L_{2}R,t={\displaystyle {\int }_{{ℝ}^3}{p\mathbf{p}|T{\Omega }_X,R,t-p\mathbf{p}|{\Omega }_Y}^2\text{d}\mathbf{p}},\end{array}$$where $T\mathrm{\Omega },R,t$ denotes the function that rigidly transforms $\mathrm{\Omega }$ with $R$ and $t$, and $p\mathbf{p}|\mathrm{\Omega }$ is the probability of observing a point $\mathbf{p}$ under a mixture model $\mathrm{\Omega }={\mu }_i,{\sigma }_i^2,{\varphi }_i$, defined by$$\begin{array}{}\text{(8)}& p\mathbf{p}|\Omega ={\displaystyle \sum _{i=1}{\varphi }_i\Gamma \mathbf{p}|{\mu }_i,{\sigma }_i^2},\end{array}$$where $\mathrm{\Gamma }\mathbf{p}|{\mu }_i,{\sigma }_i^2$ is the probability of Gaussian, ${p\mathbf{p}|T{\mathrm{\Omega }}_X,R,t}^2$ is invariant under rigid transformation, and ${p\mathbf{p}|{\mathrm{\Omega }}_Y}^2$ is independent of $R,t$. Then, the closed-form expression for an objective function is$$\begin{array}{}\text{(9)}& fR,t=-{\displaystyle {\int }_{{ℝ}^3}p\mathbf{p}|T{\Omega }_X,R,t-p\mathbf{p}|{\Omega }_Y\text{d}\mathbf{p}},\end{array}$$which can be minimized to produce $R,t$ through kernel density estimation (KDE), [182] expectation-maximization [183], or support vector machine (SVM) [184].

3.4.2. Coherent Point Drift

Coherent point drift (CPD) was introduced by Myronenko et al. [185] and further improved in [186]. Given two point clouds, similar to [25], CPD fits a GMM to the reference point cloud, and the Gaussian centroids are initialized from the points of reading data. The core of the CPD method is to coherently move GMM centroids as a group to preserve the topological structure of the point cloud. In CPD-based registration, a weighted GMM probability density function is defined as$$\begin{array}{}\text{(10)}& \begin{array}{c}px=w\frac{1}{N}+1-w{\displaystyle \sum _{m=1}^M\frac{1}{M}px|m},\\ px|m=\frac{1}{{2\pi {\sigma }^2}^{D/2}}\exp -\frac{{x-y_m}_2}{2{\sigma }^2},\end{array}\end{array}$$where $M$ is the number of points $Y{y}_1,y_2,\dots y_m$ in reading point cloud, $N$ denotes the number of points in reference data, and $w0\le w\le 1$ is a weight parameter that reflects distribution of the amount of noise and outliers in point cloud. It is generated by reference data comprising $X=x_1,x_2\dots ,x_n$ and capable of explicitly modeling outliers. The GMM centroid location can be adjusted by a set of transformation $\theta$ and ${\sigma }^2$ that can be estimated by minimizing the negative log-likelihood function in equation (11), under data assumption of independent and identically distributed:$$\begin{array}{}\text{(11)}& E\theta ,{\sigma }^2=-{\displaystyle \sum _{n=1}^M\log }{\displaystyle \sum _{m=1}^{M+1}pmp{x}_n|m}.\end{array}$$

It is usually solved by the EM algorithm to find $\theta$ and ${\sigma }^2$ [183]. The EM algorithm proceeds by alternating $P^{\text{new}}$ and $P^{\text{old}}$ during exception and maximum steps until convergence.

Many modifications of the original CPD algorithm were introduced. For instance, to accelerate the speed of the CPD algorithm, an accelerated CPD algorithm was presented in [187] that incorporated squared iterative expectation-maximization to converge to the global extreme and keep the same monotonically as the EM algorithm [188], and a Dual-Tree Improved Fast Gauss Transform (DT-IFGT) method was used to further accelerate CPD algorithm. To avoid setting the value of $w$ in equation (10), Peng et al. [189] presented a refined CPD algorithm using a hybrid optimization, and experiments showed that it was better than the traditional CPD algorithm in terms of robustness and accuracy. Jun et al. [190] modified the original CPD algorithm with the voxel-grid filter before processing to improve the speed of registration while maintaining the same accuracy.

3.4.3. Normal Distribution Transform

The normal distribution transform (NDT) algorithm [191, 192] regards input data as probability density function by the normal distribution of every cell (i.e., voxel box as discussed above) in the point cloud. It calculates the transformation between pairwise data as a minimization problem [193]. The probability density function is defined as$$\begin{array}{}\text{(12)}& px=\frac{1}{c}-\frac{{x-q}^T{C}^{-1}x-q}{2},\end{array}$$where $q$ and $C$ are the mean vector and covariance matrix of every cell that contains point $x$, and $c$ is a normalizing constant that can be set to one. $q$ and $C$ are calculated as$$\begin{array}{c}\text{(13)}& q=\frac{1}{n}{\displaystyle \sum _{k=1}^n{x}_k},C=\frac{1}{n-1}{\displaystyle \sum _{k=1}^n{x}_k-q{x_k-q}^T},\end{array}$$where $x_k|k\in n$ is the point contained in the cell and $n$ is the number of point in a cell. To formulate an optimization problem as a minimization problem, a score function of a pose p and transformation function $T\mathbf{p},x$ is defined as$$\begin{array}{}\text{(14)}& s\mathbf{p}=-{\displaystyle \sum _{k=1}^{n}pT\mathbf{p},x_k},\end{array}$$followed by $\mathbf{H}\mathrm{\Delta }\mathbf{p}=-\mathbf{g}$, where H and g are the Hessian and gradient of $s$. The increment $\mathrm{\Delta }\mathbf{p}$ is solved by Newton’s algorithm iteratively or Levenberg–Marquardt algorithm [191, 194, 195] and can be added to current estimate in each iteration.

NDT-based registration has many advantages, including more robust and faster than transitional ICP algorithm when processing large-scale data, supporting the multiresolution representation of point cloud [196], and having no explicit correspondences [191]. However, it is hard to achieve high accuracy compared with ICP [197]. Additionally, a key challenge in the NDT algorithm is how to choose an appropriate cell size for the point cloud because the accuracy and speed depend on the cell size. In particular, when cell size is small, it requires more computing quantity and memory, while if the call size is big, the resolution of cells is high so that it results in less accurate registration [193]. Nevertheless, due to the attractive contribution of the NDT algorithm, an NDT-based solution is still a good choice in a fine-to-coarse strategy at the coarse step [198, 199, 200]. Therefore, many modifications related to the NDT algorithm were proposed by researchers. For example, Yong et al. [201] introduced an improved normal distribution transform algorithm in a precise registration step. In this method, PDF is replaced by a mixed probability density function. Multilayered NDT [202], key-layered NDT [203], and variable size voxel NDT [204] were proposed to determine an appropriate size of cells, for converging to the optimal result and achieving high-quality registration. To avoid the discontinuity of the cost function, i.e., there are points existing at the cell boundaries, Arun et al. [205] introduced a segmented region growing NDT that generates the Gaussian clusters after removing the ground plane for accurate registration.

3.5. Others

Along with the development of the ICP-based, feature-based, learning-based solutions, and probability density function, the problem of point cloud registration also can be solved through other methods. Specifically, as we all know that the transformation between two point clouds can be easily found out by a set of correspondences. Therefore, Aiger et al. [206] focused on looking for the high confidence correspondences directly that only consist of four congruent points. Furthermore, auxiliary data generated from auxiliary approaches, such as image-assisted and Global Navigation Satellite System (GNSS), can also be used to simultaneously provide critical information for the task of point cloud registration.

3.5.1. Fast Global Registration

Fast global registration (FGR) [31] provided a fast strategy for point cloud registration where no initialization is required. Specifically, FGR operates on candidate matches that cover the surfaces of objects, and no correspondence updates or closest-point queries are performed. The special feature of this method is that a joint registration can be produced directly through a single optimization of a robust objective defined densely over the surface. However, the existing approaches to solve point cloud registration usually produce candidates or more correspondences between two point clouds and then compute and update a global result. Moreover, in the fast global registration, the correspondences are established at once in optimization and will not be estimated again in the following steps, as shown in Figure 26. Therefore, the expensive nearest-neighbor lookup is avoided to keep at a low computational cost. As a result, the line process for each correspondence and a linear system for pose estimation in the iteration step are efficient. FGR evaluates on multiple datasets such as UWA benchmark [207] and Stanford Bunny [208], compared with ICP with point-to-point and point-to-plane, and ICP variants such as the Go-ICP. Experiments showed that FGR outperforms in the presence of noise.

[figures omitted; refer to PDF]

3.5.2. 4-Points Congruent Sets Algorithm

4-points congruent sets (4PCS) [206] provided an initial transformation for reading data without starting position assumption. Generally, a rigid registration transformation between two point clouds can be uniquely defined by a pair of triplets in which one from reference data and one from reading data. However, in this method, it looks for special 4-points bases, i.e., 4-coplanar congruent points from each point cloud, by searching in small potentially sets, as illustrated in Figure 27. The best rigid transformation is solved in the largest common point set (LCP) problem. The algorithm achieves close performance when the overlap rate of the pairwise point cloud is low, and there are outliers. In order to adapt to different applications, many researchers introduced more significant work related to classics 4PCS solution [209–211]. For example, in this work [212], in contrast to the quadratic time complexity of traditional 4PCS, it presented a super-4PCS that achieved the performance with linear running time. Pascal et al. [213] proposed a keypoint-based 4-points congruent sets (K-4PCS) solution for coarse registration of point cloud through extracting 3D key points as the input that can be a discriminative representation of the reading data. Semantic-keypoint-based 4PCS proposed by Xuming [214] handles massive point cloud and improves K-4PCS in terms of accuracy and computational efficiency through semantic features for urban building scenes.

[figures omitted; refer to PDF]

3.5.3. Methods with Auxiliary Data

In various registration tasks, the problem of finding the transformation between pairwise or multiview can be transformed into estimating the spatial position of acquisitions in a common coordinate. Specifically, when retrieving point cloud data, some techniques worked with a depth camera can provide its location and orientation in real-time, such as the Global Navigation Satellite System (GNSS), inertial measurement unit (IMU), and image-based algorithms. Specifically, GNSS refers to a constellation of satellites that produce positioning and timing data to GNSS receivers. These data can facilitate the determination of location. IMU is an electronic device to measure the orientation, velocity, and gravitational forces of the body. Image-based algorithms have been widely used in simultaneous localization and mapping (SLAM) technique that aims to localize an autonomous mobile robot in an unknown environment while building a consistent and increment map around the robot [215].

In applications related to mobile mapping systems (MMSs) in which a rapid registration process is required and too large data are collected at once, a GNSS-IMU assistance system for forest mapping that can provide rough registration simultaneously is presented [216, 217]. However, the GNSS signal usually loses under some complex areas such as building occlusion and under the canopy. Marek et al. [218] proposed the use of graph-SLAM [219] to generate a local map of forest such that a stereo camera focuses on collecting data and a GPS is used to calculate an estimate of the global tree positions, wherein a 3D map is first generated by vision odometry and then improved by robust graph-SLAM. For 3D city models with color information, Bernard et al. [220] estimated the relative camera parameters from aerial image sequences as a coarse transformation for ICP algorithm refinement. Xuming et al. [221] introduced an image-driven registration for urban scenes, where a scanning network was constructed by exploiting the image information in order to guide the coarse pairwise registration task.

3.6. Applications

The point cloud registration is a critical perquisition for computer vision and robotics. However, the registration problem in computer vision usually is converted to the image matching problem (i.e., image registration) which is studied and researched in many fields, such as the medical domain [222, 223], handwritten digit recognition [224], and image stitching [225, 226].

For the point cloud registration, the robotic application is a hot topic. Pomerleau [34] provided four types of applications that consist of several good use cases covering different kinds of platforms, environments, and tasks. The first type of applications is on search and rescue tasks that need a real-time registration, in which 3D maps are required and can be used to provide the users with situation awareness and support critical decisions about risky platform motions. In this field, 3D map reconstruction is a basic and popular area of research, but it is merely a single application without more use cases. The second type of applications is on automation of inspection. Some tasks need to execute inspection or maintenance operations in environments that are difficult to access by humans due to dimensional, temperature, or air quality constraints. The third type of applications is on the development of an autonomous surface vessel in order to support environmental monitoring of freshwater bodies, in which a 3D laser was installed to complement the analysis of the ecosystem with geological information. The fourth type of applications is on an autonomous-driving or self-driving car. In addition, recently the deep learning techniques based on the point cloud are widely used [227] which take the raw point cloud as input directly.

3.7. Summary

In this section, five categories of solutions and methods for solving point cloud registration are introduced. Specifically, in the ICP-based methods, its steps, variants, and implementation details are described, and then the popular and open-source libraries or software is given. To use the ICP-based algorithm better, we need to know that ICP-based algorithms can obtain a good alignment result if providing an initial position for a point cloud to be aligned. The key is how to generate a good initial position for your applications. Therefore, for applications that need point cloud registration, a popular strategy was proposed [228, 229], which consists of two steps: (1) computing a global transformation as an initial position for the point cloud and (2) calculating a local transformation that can guarantee the result of registration. These two steps are called coarse-to-fine strategies. In this case, feature-based, learning-based, and probability-based methods are usually used to produce a global transformation, and then further improvements are made by the ICP-based algorithms. The point-to-point and point-to-plane ICP methods are widely used in real-world situations [39, 90]. To use the ICP algorithms better, many scholars considered the evaluation of ICP and its variants. For example, Peng et al. [39] took overlap ratio, angle, and distance as the influence factors of ICP and evaluated the performance on the same four datasets. In addition, Martin et al. [197] and Mouna et al. [230] compared the ICP and NDT in terms of accuracy and speed. Lachat [231] studied different software and tools to solve point cloud registration in terrestrial and airborne laser scanning. To overcome the bias on experiments, Pomerleau [232] provided datasets that cover a diverse range of challenging environments for registration algorithms.

Feature-based and learning-based methods are studied increasingly in recent years, because in general the point cloud registration problem was straightforwardly divided into several modules: feature extractor, matching, outlier rejection, and motion estimation. For the feature extractor module, in addition to the PFH and SHOT, what we describe in this section is the local-based feature. Various methods to extract the feature and description of the point cloud are also studied and summarized [140]. Feature-based methods usually extract pairwise or higher-order relationships in the histogram in a handcrafted way. However, many of these handcrafted descriptors that highly rely on distinctive geometric properties of point cloud data merely work well on real-world point clouds [233] that are noisy and low density. Learning-based methods involve regressing the transformation parameters by searching for the difference between features which generally are global-based. The state-of-art methods presented recent years are introduced, and a brief comparison of their performance and modules is provided. They are less sensitive and more robust on real-world data. To obtain a good initial transformation, the learning-based and feature-based methods can be helpful. For better using these techniques, we should pay attention to the following points: (1) a feature detection algorithm should be considered, which attains the trade-off of accuracy and efficiency. (2) A learning framework should be established, particularly for the applications containing data collection and training. In addition, probability-based methods are also used for initial pose estimation, such as NDT and CPD that are often utilized for nonrigid registration.

4. Conclusion and Outlook

Point cloud registration is a crucial step for many applications that require to build a 3D map environment and recover the 3D surface of an object or scene. However, it is impossible to retrieve all surface data at once or directly obtain the relationship between two point clouds in each coordinate system. Therefore, to place a strong emphasis on the comprehensive review of pairwise point cloud registration with various state-of-the-art algorithms, this paper begins by the basics and principles of point cloud registration, and it then goes on to divide the registration process into five main classifications: ICP-based algorithm, feature-based, machine learning-based, probability-based solution, and others. With these registration categories, we provide insight that every method has its applicable fields and disadvantages. For two completely identical point clouds with an arbitrary position, a valid transformation could be computed by traditional algorithms, such as ICP-based algorithms and feature-based solutions. For two acquired point clouds, machine-learning methods have been developed to achieve more robustness and accuracy than traditional algorithms based on experiments. Therefore, learning-based methods have attracted more and more attention for providing a new perspective and have achieved great success for solving point cloud registration. However, in real-world data, the task of registration will face uncertain situations, such as the variant of overlap, fluctuation of noise, and the huge amount of computation due to massive input data. Therefore, developing a general framework to solve the point cloud registration in the real world is still a challenging task. There is a brief guideline for those who want to use point cloud registration as a tool: when it has the apparent characteristics of the captured objects, feature-based is a good choice for your applications that need point cloud registration. However, when it is not sure what the retrieved objects are, machine-based and probability-based solutions can always provide a reliable transformation as a rough result. In this case, an ICP-based algorithm should be applied to achieve a refined registration.

With the advancement of 3D imaging technology, the resolution of point cloud data and the number of point sets will increase dramatically. For instance, a point cloud accurately representing the details on a workpiece acquired from the structured light camera (SLC) now has more than tens of millions of points. One frame data approximately describing the subway station taken by a LiDAR have at least five million points. Therefore, a technique that probably is integrated with hardware and software optimization to swiftly process and operate these unstructured point cloud data is required in the future. Additionally, as a quantitative measurement, a ubiquitous dataset containing ground-truth transformation and different environments, such as indoor and outdoor scenes, is beneficial for evaluating learning-based methods and previous strategies. As more and more public datasets are expected to be established in the future, we believe that a comparison between traditional algorithms and machine-learning methods is essential for us to choose an appropriate registration solution for a specific application.

References

[1] Q. Wang, M.-K. Kim, "Applications of 3d point cloud data in the construction industry: a fifteen-year review from 2004 to 2018," Advanced Engineering Informatics, vol. 39, pp. 306-319, DOI: 10.1016/j.aei.2019.02.007, 2019.

[2] R. W. Wolcott, R. M. Eustice, "Robust lidar localization using multiresolution Gaussian mixture maps for autonomous driving," The International Journal of Robotics Research, vol. 36 no. 3, pp. 292-319, DOI: 10.1177/0278364917696568, 2017.

[3] X. Yue, B. Wu, S. A. Seshia, K. Keutzer, A. L. Sangiovanni-Vincentelli, "A lidar point cloud generator: from a virtual world to autonomous driving," Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp. 458-464, .

[4] W. Lu, Y. Zhou, G. Wan, S. Hou, S. Song, "L3-net: towards learning based lidar localization for autonomous driving," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6389-6398, DOI: 10.1109/cvpr.2019.00655, .

[5] R. Wang, Y. Xu, M. A. Sotelo, "A robust registration method for autonomous driving pose estimation in urban dynamic environment using lidar," Electronics, vol. 8 no. 1,DOI: 10.3390/electronics8010043, 2019.

[6] M. Keller, D. Lefloch, M. Lambers, S. Izadi, T. Weyrich, A. Kolb, "Real-time 3d reconstruction in dynamic scenes using point-based fusion," Proceedings of the 2013 International Conference on 3D Vision-3DV, vol. 2013,DOI: 10.1109/3dv.2013.9, .

[7] M. Donlic, T. Petkovic, T. Pribanic, "On tablet 3d structured light reconstruction and registration," Proceedings of the The IEEE International Conference on Computer Vision (ICCV) Workshops,DOI: 10.1109/iccvw.2017.290, .

[8] B. Fang, C. Wang, "A 3d reconstruction method based on the combination of the icp and artificial potential field algorithm," Lecture Notes in Electrical Engineering, pp. 483-493, DOI: 10.1007/978-981-10-6496-8_45, 2017.

[9] P. Wu, W. Li, M. Yan, "3d scene reconstruction based on improved ICP algorithm," Microprocessors and Microsystems, 2020.

[10] S. Izadi, D. Kim, O. Hilliges, "Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera," Proceedings of the 24th annual ACM symposium on user interface software and technology, pp. 559-568, .

[11] E. Mendes, P. Koch, S. Lacroix, "Icp-based pose-graph slam," Proceedings of the 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pp. 195-200, DOI: 10.1109/ssrr.2016.7784298, .

[12] H. Cho, E. K. Kim, S. Kim, "Indoor slam application using geometric and ICP matching methods based on line features," Robotics and Autonomous Systems, vol. 100, pp. 206-224, 2018.

[13] P. Kim, J. Chen, Y. K. Cho, "Slam-driven robotic mapping and registration of 3d point clouds," Automation in Construction, vol. 89, pp. 38-48, DOI: 10.1016/j.autcon.2018.01.009, 2018.

[14] L. Han, L. Xu, D. Bobkov, E. Steinbach, L. Fang, "Real-time global registration for globally consistent rgb-d slam," IEEE Transactions on Robotics, vol. 35 no. 2, pp. 498-508, DOI: 10.1109/tro.2018.2882730, 2019.

[15] Q. Qian, Multi-camera registration for Vr: a flexible, feature-based approach, 2018. Master’s thesis

[16] D. Borrmann, A. Nuechter, T. Wiemann, "Large-scale 3d point cloud processing for mixed and augmented reality," Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), vol. 35,DOI: 10.1109/ismar-adjunct.2018.00018, .

[17] D. Ni, A. Song, X. Xu, H. Li, C. Zhu, H. Zeng, "3d-point-cloud registration and real-world dynamic modelling-based virtual environment building method for teleoperation," Robotica, vol. 35 no. 10, pp. 1958-1974, 2017.

[18] S. Xu, J. Zhu, Z. Jiang, Z. Lin, J. Lu, Z. Li, "Multi-view registration of unordered range scans by fast correspondence propagation of multi-scale descriptors," PloS One, vol. 13 no. 9,DOI: 10.1371/journal.pone.0203139, 2018.

[19] H. Zhu, B. Guo, K. Zou, "A review of point set registration: from pairwise registration to groupwise registration," Sensors, vol. 19 no. 5,DOI: 10.3390/s19051191, 2019.

[20] L. Cheng, S. Chen, X. Liu, "Registration of laser scanning point clouds: a review," Sensors, vol. 18 no. 5,DOI: 10.3390/s18051641, 2018.

[21] G. K. L. Tam, Z. Zhi-Quan, Y. Yu-Kun Lai, "Registration of 3d point clouds and meshes: a survey from rigid to nonrigid," IEEE Transactions on Visualization and Computer Graphics, vol. 19 no. 7, pp. 1199-1217, DOI: 10.1109/tvcg.2012.310, 2013.

[22] B. Yang, Z. Dong, F. Liang, Y. Liu, "Automatic registration of large-scale urban scene point clouds based on semantic feature points," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 113, pp. 43-58, DOI: 10.1016/j.isprsjprs.2015.12.005, 2016.

[23] Y. Wang, J. M. Solomon, "Prnet: self-supervised learning for partial-to-partial registration," Advances in Neural Information Processing Systems, pp. 8814-8826, 2019.

[24] C. Choy, W. Dong, V. Koltun, "Deep global registration," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2514-2523, DOI: 10.1109/cvpr42600.2020.00259, .

[25] H. Chui, A. Rangarajan, "A feature registration framework using mixture models," pp. 190-197, .

[26] Y. He, B. Liang, J. Yang, S. Li, J. He, "An iterative closest points algorithm for registration of 3d laser scanner point clouds with geometric features," Sensors, vol. 17 no. 8,DOI: 10.3390/s17081862, 2017.

[27] B. Radu, N. Blodow, M. Beetz, "Fast point feature histograms (fpfh) for 3d registration," pp. 3212-3217, .

[28] B. Radu, N. Blodow, M. Beetz, "Aligning point cloud views using persistent feature histograms," pp. 3384-3391, .

[29] J. Yang, Z. Cao, Q. Zhang, "A fast and robust local descriptor for 3d point cloud registration," Information Sciences, vol. 346-347, pp. 163-179, DOI: 10.1016/j.ins.2016.01.095, 2016.

[30] A. Gruen, D. Akca, "Least squares 3d surface and curve matching," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 59 no. 3, pp. 151-174, DOI: 10.1016/j.isprsjprs.2005.02.006, 2005.

[31] Q.-Y. Zhou, J. Park, V. Koltun, "Fast global registration, computer vision - ECCV 2016," pp. 766-782, DOI: 10.1007/978-3-319-46475-6_47, .

[32] K. Alhamzi, M. Elmogy, S. Barakat, "3d object recognition based on local and global features using point cloud library," International Journal of Advancements in Computing Technology, vol. 7 no. 3, 2015.

[33] L. F. Robert, N. Gros, Y. Noutary, L. Malleus, F. Precioso, D. Lingrand, "Keypoint-based point-pair-feature for scalable automatic global registration of large rgb-d scans," 2019. US Patent 10,217,277

[34] F. Pomerleau, F. Colas, R. Siegwart, "A review of point cloud registration algorithms for mobile robotics," Foundations and Trends in Robotics, vol. 4 no. 1, 2015.

[35] G. K. L. Tam, Z. Zhi-Quan Cheng, Y. Yu-Kun Lai, "Registration of 3d point clouds and meshes: a survey from rigid to nonrigid," Visualization and Computer Graphics, vol. 18,DOI: 10.1109/tvcg.2012.310, 2013.

[36] H. Li, R. Hartley, "The 3d-3d registration problem revisited," ,DOI: 10.1109/iccv.2007.4409077, .

[37] B. Bellekens, S. Vincent, R. Berkvens, M. Weyn, "A survey of rigid 3d pointcloud registration algorithms," .

[38] Z. Dong, F. Liang, B. Yang, "Registration of large-scale terrestrial laser scanner point clouds: a review and benchmark," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 163, pp. 327-342, DOI: 10.1016/j.isprsjprs.2020.03.013, 2020.

[39] Li Peng, R. Wang, Y. Wang, W. Tao, "Evaluation of the icp algorithm in 3d point cloud registration," IEEE Access, vol. 8, pp. 68030-68048, 2020.

[40] X. Jiang, J. Ma, G. Xiao, Z. Shao, X. Guo, "A review of multimodal image matching: methods and applications," Information Fusion, 2021.

[41] J. Ma, X. Jiang, A. Fan, J. Jiang, J. Yan, "Image matching from handcrafted to deep features: a survey," International Journal of Computer Vision, vol. 129 no. 1, pp. 23-79, DOI: 10.1007/s11263-020-01359-2, 2021.

[42] A. Zabatani, V. Surazhsky, E. Sperling, "Intel® realsense TM sr300 coded light depth camera," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.

[43] O. Ozyesil, V. Voroninski, B. Ronen, A. Singer, "A survey of structure from motion," Acta Numerica, vol. 26, pp. 305-364, 2017.

[44] Y. Furukawa, J. Ponce, "Accurate, dense, and robust multiview stereopsis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32 no. 8, pp. 1362-1376, 2009.

[45] R. Chen, S. Han, J. Xu, H. Su, "Point-based multi-view stereo network," Proceedings of the IEEE International Conference on Computer Vision, pp. 1538-1547, DOI: 10.1109/iccv.2019.00162, .

[46] Y. Furukawa, B. Curless, S. M. Seitz, R. Szeliski, "Towards internet-scale multi-view stereo," pp. 1434-1441, DOI: 10.1109/cvpr.2010.5539802, .

[47] A. Grunnet-Jepsen, J. N. Sweetser, W. Paul, A. Takagi, J. Woodfill, Projectors for Intel® realsense TM Depth Cameras D4xx. Intel Support, 2018.

[48] A. Goshtasby, W. A. Gruver, "Design of a single-lens stereo camera system," Pattern Recognition, vol. 26 no. 6, pp. 923-937, DOI: 10.1016/0031-3203(93)90058-5, 1993.

[49] A. Pezzuolo, D. Giora, L. Sartori, S. Guercini, "Automated 3d reconstruction of rural buildings from structure-from-motion (sfm) photogrammetry approach," Proceedings of the International Scientific Conference, pp. 23-25, DOI: 10.22616/erdev2018.17.n060, .

[50] S. Zhu, R. Zhang, L. Zhou, "Very large-scale global sfm by distributed motion averaging," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4568-4577, DOI: 10.1109/cvpr.2018.00480, .

[51] V. Usenko, J. Engel, Jörg Stückler, D. Cremers, "Reconstructing street-scenes in real-time from a driving car," pp. 607-614, DOI: 10.1109/3dv.2015.75, .

[52] P. Gronat, M. Havlena, J. Sivic, P. Tomas, Building Streetview Datasets for Place Recognition and City reconstruction, 2011.

[53] Y. Xu, V. John, S. Mita, H. Tehrani, K. Ishimaru, S. Nishino, "3d point cloud map based vehicle localization using stereo camera," pp. 487-492, DOI: 10.1109/ivs.2017.7995765, .

[54] Z. Gong, W. Xue, Z. Liu, "Design of a reconfigurable multi-sensor testbed for autonomous vehicles and ground robots," ,DOI: 10.1109/iscas.2019.8702610, .

[55] E. Ackerman, "Lidar that will make self-driving cars affordable [news]," IEEE Spectrum, vol. 53 no. 10,DOI: 10.1109/mspec.2016.7572525, 2016.

[56] Michael Van den Bergh and Luc Van Gool, "Combining rgb and tof cameras for real-time 3d hand gesture interaction," pp. 66-72, DOI: 10.1109/wacv.2011.5711485, .

[57] T. Bell, B. Li, S. Zhang, "Structured light techniques and applications," Electrical and Electronics Engineering, 1999.

[58] S. Zhang, "Recent progresses on real-time 3d shape measurement using digital fringe projection techniques," Optics and Lasers in Engineering, vol. 48 no. 2, pp. 149-158, DOI: 10.1016/j.optlaseng.2009.03.008, 2010.

[59] S. Zhang, "High-speed 3d shape measurement with structured light methods: a review," Optics and Lasers in Engineering, vol. 106, pp. 119-131, DOI: 10.1016/j.optlaseng.2018.02.017, 2018.

[60] D. M. McClatchy, E. J. Rizzo, J. Meganck, "Calibration and analysis of a multimodal micro-ct and structured light imaging system for the evaluation of excised breast tissue," Physics in Medicine & Biology, vol. 62 no. 23, pp. 8983-9000, DOI: 10.1088/1361-6560/aa94b6, 2017.

[61] A. Geiger, P. Lenz, C. Stiller, R. Urtasun, "Vision meets robotics: the kitti dataset," The International Journal of Robotics Research, vol. 32 no. 11, pp. 1231-1237, DOI: 10.1177/0278364913491297, 2013.

[62] A. Behl, D. Paschalidou, D. Simon, A. Geiger, "Pointflownet: learning representations for rigid motion estimation from point clouds," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),DOI: 10.1109/cvpr.2019.00815, .

[63] B. J. Brown, S. Rusinkiewicz, "Global non-rigid alignment of 3-d scans," Proceedings of the ACM SIGGRAPH 2007 papers,DOI: 10.1145/1275808.1276404, .

[64] H. Li, R. W. Sumner, M. Pauly, "Global correspondence optimization for non-rigid registration of depth scans," Computer Graphics Forum, vol. 27 no. 5, pp. 1421-1430, DOI: 10.1111/j.1467-8659.2008.01282.x, .

[65] Federica Bogo, J. Romero, G. Pons-Moll, J. Michael, "Dynamic faust: registering human bodies in motion," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), .

[66] R. A. Newcombe, D. Fox, S. M. Seitz, "Dynamicfusion: reconstruction and tracking of non-rigid scenes in real-time," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 343-352, DOI: 10.1109/cvpr.2015.7298631, .

[67] H. Chui, A. Rangarajan, "A new point matching algorithm for non-rigid registration," Computer Vision and Image Understanding, vol. 89 no. 2-3, pp. 114-141, DOI: 10.1016/s1077-3142(03)00009-2, 2003.

[68] S. Chang, C. Ahn, M. Lee, S. Oh, "Graph-matching-based correspondence search for nonrigid point cloud registration," Computer Vision and Image Understanding, vol. 192, 2020.

[69] R. A. Newcombe, S. Izadi, O. Hilliges, "Kinectfusion: real-time dense surface mapping and tracking," pp. 127-136, DOI: 10.1109/ismar.2011.6092378, .

[70] A. Ullrich, M. Pfennigbauer, "Noisy lidar point clouds: impact on information extraction in high-precision lidar surveying," vol. 28, .

[71] M. Berger, A. Tagliasacchi, S. Lee, "State of the art in surface reconstruction from point clouds," vol. 1 no. 1, pp. 161-185, .

[72] P. Schaer, S. Jan, S. Landtwing, K. Legat, "Accuracy estimation for laser point cloud including scanning geometry," Proceedings of theMobile Mapping Symposium 2007, .

[73] J. Park, H. Kim, Y. W. Tai, M. S. Brown, I. Kweon, "High quality depth map upsampling for 3d-tof cameras," pp. 1623-1630, DOI: 10.1109/iccv.2011.6126423, .

[74] Y. Wang, H.-Y. Feng, "Effects of scanning orientation on outlier formation in 3d laser scanning of reflective surfaces," Optics and Lasers in Engineering, vol. 81, pp. 35-45, DOI: 10.1016/j.optlaseng.2016.01.003, 2016.

[75] H. Chen, J. Shen, "Denoising of point cloud data for computer-aided design, engineering, and manufacturing," Engineering with Computers, vol. 34 no. 3, pp. 523-541, DOI: 10.1007/s00366-017-0556-4, 2018.

[76] S. Li, J. Wang, Z. Liang, L. Su, "Tree point clouds registration using an improved icp algorithm based on kd-tree," pp. 4545-4548, DOI: 10.1109/igarss.2016.7730186, .

[77] L. Di Angelo, L. Giaccari, "An efficient algorithm for the nearest neighbourhood search for point clouds," International Journal of Computer Science Issues (IJCSI), vol. 8 no. 5, 2011.

[78] K. Zhou, Q. Hou, R. Wang, B. Guo, "Real-time kd-tree construction on graphics hardware," ACM Transactions on Graphics, vol. 27 no. 5,DOI: 10.1145/1409060.1409079, 2008.

[79] M. Connor, P. Kumar, "Fast construction of k-nearest neighbor graphs for point clouds," IEEE Transactions on Visualization and Computer Graphics, vol. 16 no. 4, pp. 599-608, DOI: 10.1109/tvcg.2010.9, 2010.

[80] A. Nuchter, K. Lingemann, J. Hertzberg, "Cached kd tree search for icp algorithms," pp. 419-426, DOI: 10.1109/3dim.2007.15, .

[81] J. Li, B. M. Chen, G. H. Lee, "So-net: self-organizing network for point cloud analysis," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9397-9406, DOI: 10.1109/cvpr.2018.00979, .

[82] S. Rusinkiewicz, M. Levoy, "Efficient variants of the icp algorithm," pp. 145-152, .

[83] D. Holz, A. E. Ichim, F. Tombari, R. B. Rusu, S. Behnke, "Registration with the point cloud library: a modular framework for aligning in 3-d," IEEE Robotics & Automation Magazine, vol. 22 no. 4, pp. 110-124, DOI: 10.1109/mra.2015.2432331, 2015.

[84] C. Benoit, S. Prima, "An efficient em-icp algorithm for non-linear registration of large 3d point sets," Computer Vision and Image Understanding, vol. 191, 2020.

[85] T. Tamaki, M. Abe, B. Raytchev, K. Kaneda, "Softassign and em-icp on gpu," pp. 179-183, DOI: 10.1109/ic-nc.2010.60, .

[86] S. Rusinkiewicz, "Efficient variants of the icp algorithm," Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, pp. 145-152, .

[87] X.-F. Han, J. S. Jin, M.-J. Wang, W. Jiang, L. Gao, L. Xiao, "A review of algorithms for filtering the 3d point cloud," Signal Processing: Image Communication, vol. 57, pp. 103-112, DOI: 10.1016/j.image.2017.05.009, 2017.

[88] M. Yilmaz, M. Uysal, "Comparison of data reduction algorithms for LiDAR-derived digital terrain model generalisation," Area, vol. 48 no. 4, pp. 521-532, DOI: 10.1111/area.12276, 2016.

[89] S. Chen, T. Dong, F. Chen, Anthony Vetro, J. Kovačević, "Fast resampling of three-dimensional point clouds via graphs," IEEE Transactions on Signal Processing, vol. 66 no. 3, pp. 666-681, 2017.

[90] F. Pomerleau, F. Colas, R. Siegwart, S. Magnenat, "Comparing ICP variants on real-world data sets," Autonomous Robots, vol. 34 no. 3, pp. 133-148, DOI: 10.1007/s10514-013-9327-2, February 2013.

[91] R. Benjemaa, F. Schmitt, "Fast global registration of 3d sampled surfaces using a multi-z-buffer technique," Image and Vision Computing, vol. 17 no. 2, pp. 113-123, DOI: 10.1016/s0262-8856(98)00115-2, 1999.

[92] N. Ko, K. Ikeuchi, "Robust simultaneous registration of multiple range images," Digitally Archiving Cultural Objects, pp. 71-88, 2008.

[93] J. L. Bentley, "Multidimensional binary search trees used for associative searching," Communications of the ACM, vol. 18 no. 9, pp. 509-517, DOI: 10.1145/361002.361007, 1975.

[94] D. Meagher, "Geometric modeling using octree encoding," Computer Graphics and Image Processing, vol. 19 no. 2, pp. 129-147, DOI: 10.1016/0146-664x(82)90104-6, 1982.

[95] W. Guan, W. Li, Y. Ren, "Point cloud registration based on improved icp algorithm," Proceedings of the2018 Chinese Control and Decision Conference (CCDC), pp. 1461-1465, DOI: 10.1109/ccdc.2018.8407357, .

[96] J. Meng, J. Li, X. Gao, "An accelerated icp registration algorithm for 3d point cloud data," vol. 10839, .

[97] S.-Y. Park, M. Subbarao, "An accurate and fast point-to-plane registration technique," Pattern Recognition Letters, vol. 24 no. 16, pp. 2967-2976, DOI: 10.1016/s0167-8655(03)00157-0, 2003.

[98] G. Blais, M. D. Levine, "Registering multiview range data to create 3d computer objects," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17 no. 8, pp. 820-824, DOI: 10.1109/34.400574, 1995.

[99] K. Pulli, "Multiview registration for large data sets," pp. 160-168, .

[100] S.-Y. Park, S.-I. Choi, J. Kim, J. S. Chae, "Real-time 3d registration using gpu," Machine Vision and Applications, vol. 22 no. 5, pp. 837-850, DOI: 10.1007/s00138-010-0282-z, 2011.

[101] P. Yin, F. Gu, D. Li, Y. He, L. Yang, J. Han, "Gpu-based heuristic escape for outdoor large scale registration," pp. 260-265, DOI: 10.1109/rcar.2016.7784036, .

[102] S. Sun, W. Song, Y. Tian, S. Fong, "An icp-based point clouds registration method for indoor environment modeling," Advanced Multimedia and Ubiquitous Engineering, pp. 339-344, DOI: 10.1007/978-981-32-9244-4_48, 2019.

[103] T. Zinßer, J. Schmidt, N. Heinrich, "A refined icp algorithm for robust 3-d correspondence estimation," vol. 2, .

[104] T. Masuda, K. Sakaue, N. Yokoya, "Registration and integration of multiple range images for 3-d model construction," vol. 1, pp. 879-883, DOI: 10.1109/icpr.1996.546150, .

[105] H. Q. Luong, M. Vlaminck, G. Werner, W. Philips, "Consistent icp for the registration of sparse and inhomogeneous point clouds," pp. 262-267, .

[106] S. Giancola, J. Schneider, W. Peter, B. S. Ghanem, "Integration of absolute orientation measurements in the kinectfusion reconstruction pipeline," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops,DOI: 10.1109/cvprw.2018.00198, .

[107] J. H. Back, S. Kim, Yo-S. Ho, "High-precision 3d coarse registration using ransac and randomly-picked rejections," pp. 254-266, .

[108] A. Gressin, C. Mallet, N. David, "Improving 3d lidar point cloud registration using optimal neighborhood knowledge," vol. 5 no. 2,DOI: 10.5194/isprsannals-i-3-111-2012, .

[109] A. Ahmad, C. Léger, D. Mohamed, L. Roger, "3d registration using a new implementation of the icp algorithm based on a comprehensive lookup matrix: application to medical imaging," Pattern Recognition Letters, vol. 28 no. 12, pp. 1523-1533, 2007.

[110] F. Monnier, B. Vallet, N. Paparoditis, J.-P. Papelard, N. David, "Registration of terrestrial mobile laser data on 2d or 3d geographic database by use of a non-rigid icp approach," ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 5,DOI: 10.5194/isprsannals-ii-5-w2-193-2013, 2013.

[111] L. D. Wen, T. J. J. Tang, T. Drummond, W. H. Li, "Robust egomotion estimation using icp in inverse depth coordinates," .

[112] K. Naus, Ł. Marchel, "Use of a weighted icp algorithm to precisely determine usv movement parameters," Applied Sciences, vol. 9 no. 17,DOI: 10.3390/app9173530, 2019.

[113] J. Liu, M. E. Rettmann, D. R. Holmes, H. Duan, R. A. Robb, "A piecewise patch-to-model matching method for image-guided cardiac catheter ablation," Computerized Medical Imaging and Graphics, vol. 35 no. 4, pp. 324-332, DOI: 10.1016/j.compmedimag.2011.02.001, 2011.

[114] E. Héry, P. Xu, P. Bonnifait, "Lidar based relative pose and covariance estimation for communicating vehicles exchanging a polygonal model of their shape," Proceedings of the 10th Workshop on Planning, Perception and Navigation for Intelligent Vehicles, .

[115] P. J. Besl, N. D. McKay, "Method for registration of 3-d shapes," Sensor Fusion Iv: Control Paradigms And Data Structures, vol. volume 1611, pp. 586-606, 1992.

[116] K. S. Arun, T. S. Huang, S. D. Blostein, "Least-squares fitting of two 3-d point sets," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 9 no. 5, pp. 698-700, DOI: 10.1109/tpami.1987.4767965, 1987.

[117] D. W. Eggert, A. Lorusso, R. B. Fisher, "Estimating 3-d rigid body transformations: a comparison of four major algorithms," Machine Vision and Applications, vol. 9 no. 5-6, pp. 272-290, DOI: 10.1007/s001380050048, 1997.

[118] Y. Chen, G. Medioni, "Object modelling by registration of multiple range images," Image and Vision Computing, vol. 10 no. 3, pp. 145-155, DOI: 10.1016/0262-8856(92)90066-c, 1992.

[119] R. Bergevin, M. Soucy, H. Gagnon, D. Laurendeau, "Towards a general multi-view registration technique," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18 no. 5, pp. 540-547, DOI: 10.1109/34.494643, 1996.

[120] B. K. P. Horn, H. M. Hilden, S. Negahdaripour, "Closed-form solution of absolute orientation using orthonormal matrices," Journal of the Optical Society of America A, vol. 5 no. 7, pp. 1127-1135, DOI: 10.1364/josaa.5.001127, 1988.

[121] B. K. P. Horn, "Closed-form solution of absolute orientation using unit quaternions," Journal of the Optical Society of America A, vol. 4 no. 4, pp. 629-642, DOI: 10.1364/josaa.4.000629, 1987.

[122] A. W. Fitzgibbon, "Robust registration of 2d and 3d point sets," Image and Vision Computing, vol. 21 no. 13-14, pp. 1145-1153, DOI: 10.1016/j.imavis.2003.09.004, 2003.

[123] P. J. Neugebauer, "Geometrical cloning of 3d objects via simultaneous registration of multiple range images," pp. 130-139, .

[124] A. Censi, "An icp variant using a point-to-line metric," pp. 19-25, DOI: 10.1109/robot.2008.4543181, .

[125] H. Men, B. Gebre, K. Pochiraju, "Color point cloud registration with 4d icp algorithm," pp. 1511-1516, DOI: 10.1109/icra.2011.5980407, .

[126] W. Forstner, K. Khoshelham, "Efficient and accurate registration of point clouds with plane to plane correspondences," Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 2165-2173, DOI: 10.1109/iccvw.2017.253, .

[127] A. Segal, D. Haehnel, S. Thrun, "Generalized-icp," Robotics: science and systems, vol. volume 2,DOI: 10.15607/rss.2009.v.021, 2009.

[128] Q. Y. Zhou, J. Park, V. Koltun, "Open3d: a modern library for 3d data processing," 2018. http://arxiv.org/abs/1801.09847

[129] B. Radu, S. Cousins, "3d is here: point cloud library (pcl)," .

[130] F. Pomerleau, S. Magnenat, F. Colas, M. Liu, S. Roland, "Tracking a depth camera: parameter exploration for fast icp," pp. 3824-3829, DOI: 10.1109/iros.2011.6048545, .

[131] J. Yang, H. Li, Y. Jia, "Go-icp: solving 3d registration efficiently and globally optimally," Proceedings of the IEEE International Conference on Computer Vision, pp. 1457-1464, .

[132] C. Yuan, X. Yu, Z. Luo, "3d point cloud matching based on principal component analysis and iterative closest point algorithm," pp. 404-408, DOI: 10.1109/icalip.2016.7846655, .

[133] D. Avidar, D. Malah, M. Barzohar, "Local-to-global point cloud registration using a dictionary of viewpoint descriptors," pp. 891-899, DOI: 10.1109/iccv.2017.102, .

[134] K.-H. Bae, "Automated registration of unorganised point clouds from terrestrial laser scanners," 2006. PhD Thesis

[135] A. E. Johnson, S. B. Kang, "Registration and integration of textured 3d data," Image and Vision Computing, vol. 17 no. 2, pp. 135-147, 1999.

[136] D. Akca, "Matching of 3d surfaces and their intensities," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 62 no. 2, pp. 112-121, DOI: 10.1016/j.isprsjprs.2006.06.001, 2007.

[137] B. Yang, Y. Zang, Z. Dong, R. Huang, "An automated method to register airborne and terrestrial laser scanning point clouds," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 109, pp. 62-76, DOI: 10.1016/j.isprsjprs.2015.08.006, 2015.

[138] L. A. Alexandre, "3d descriptors for object and category recognition: a comparative evaluation," Proceedings of the Workshop on Color-Depth Camera Fusion in Robotics at the IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, .

[139] Y. Guo, M. Bennamoun, F. Sohel, M. Lu, J. Wan, N. M. Kwok, "A comprehensive performance evaluation of 3d local feature descriptors," International Journal of Computer Vision, vol. 116 no. 1, pp. 66-89, DOI: 10.1007/s11263-015-0824-y, 2016.

[140] X.-F. Hana, S. Jesse, J. Xie, M.-J. Wang, W. Jiang, "A comprehensive review of 3d point cloud descriptors," 2018. http://arxiv.org/abs/1802.02297

[141] Y. Ran, X. Xu, "Point cloud registration method based on sift and geometry feature," Optik, vol. 203,DOI: 10.1016/j.ijleo.2019.163902, 2020.

[142] X.-F. Han, S.-J. Sun, X.-Y. Song, G.-Q. Xiao, "3d point cloud descriptors in hand-crafted and deep learning age: state-of-the-art," 2018. http://arxiv.org/abs/1404.3978

[143] I. Hadji, G. N. De, "Local-to-global signature descriptor for 3d object recognition," pp. 570-584, .

[144] B. Radu, N. Blodow, M. Beetz, "Learning informative point classes for the acquisition of object model maps," pp. 643-650, .

[145] F. Tombari, S. Salti, L. Di Stefano, "Unique signatures of histograms for local surface description," pp. 356-369, DOI: 10.1007/978-3-642-15558-1_26, .

[146] R. Raguram, J.-M. Frahm, M. Pollefeys, "A comparative analysis of ransac techniques leading to adaptive real-time random sample consensus," Lecture Notes in Computer Science, pp. 500-513, DOI: 10.1007/978-3-540-88688-4_37, .

[147] J. Ma, J. Zhao, J. Tian, A. L. Yuille, Z. Tu, "Robust point matching via vector field consensus," IEEE Transactions on Image Processing, vol. 23 no. 4, pp. 1706-1721, DOI: 10.1109/tip.2014.2307478, 2014.

[148] M. A. Fischler, R. C. Bolles, "Random sample consensus," Communications of the ACM, vol. 24 no. 6, pp. 381-395, DOI: 10.1145/358669.358692, 1981.

[149] S. Choi, T. Kim, W. Yu, "Performance evaluation of ransac family," Journal of Computer Vision, vol. 24 no. 3, pp. 271-300, 1997.

[150] J. Ma, J. Zhao, J. Jiang, H. Zhou, X. Guo, "Locality preserving matching," International Journal of Computer Vision, vol. 127 no. 5, pp. 512-531, DOI: 10.1007/s11263-018-1117-z, 2019.

[151] J. Ma, Ji Zhao, J. Tian, Z. Tu, L. Alan, "Robust estimation of nonrigid transformation for point set registration," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2147-2154, DOI: 10.1109/cvpr.2013.279, .

[152] J. Ma, W. Qiu, J. Zhao, Y. Ma, A. L. Yuille, Z. Tu, "Robust $L_{2}E$ estimation of transformation for non-rigid registration," IEEE Transactions on Signal Processing, vol. 63 no. 5, pp. 1115-1129, DOI: 10.1109/tsp.2014.2388434, 2015.

[153] D. W. Scott, "Parametric statistical modeling by minimum integrated square error," Technometrics, vol. 43 no. 3, pp. 274-285, DOI: 10.1198/004017001316975880, 2001.

[154] J. Ma, J. Wu, Ji Zhao, J. Jiang, H. Zhou, Z. Quan, "Nonrigid point set registration with robust transformation learning under manifold regularization," IEEE Transactions on Neural Networks and Learning Systems, vol. 30 no. 12, pp. 3584-3597, 2018.

[155] J. Ma, Ji Zhao, L. Alan, "Non-rigid point set registration by preserving global and local structures," IEEE Transactions on Image Processing, vol. 25 no. 1, pp. 53-64, 2015.

[156] C. R. Qi, H. Su, K. Mo, J. Leonidas, "Pointnet: deep learning on point sets for 3d classification and segmentation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652-660, .

[157] Yi Fang, J. Xie, G. Dai, "3d deep shape descriptor," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),DOI: 10.1109/cvpr.2015.7298845, .

[158] C. R. Qi, Li Yi, H. Su, J. Leonidas, "Pointnet++: deep hierarchical feature learning on point sets in a metric space," Proceedings of the Advances in Neural Information Processing Systems, pp. 5099-5108, .

[159] V. Sarode, X. Li, H. Goforth, "Pcrnet: point cloud registration network using pointnet encoding," 2019. http://arxiv.org/abs/1908.07906

[160] Y. Aoki, H. Goforth, R. Arun Srivatsan, S. Lucey, "Pointnetlk: robust and efficient point cloud registration using pointnet," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), .

[161] Z. Wu, S. Song, A. Khosla, "3d shapenets: a deep representation for volumetric shapes," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912-1920, .

[162] C. Choy, J. Park, V. Koltun, "Fully convolutional geometric features," Proceedings of the IEEE International Conference on Computer Vision,DOI: 10.1109/iccv.2019.00905, .

[163] J. C. Gower, "Generalized procrustes analysis," Psychometrika, vol. 40 no. 1, pp. 33-51, DOI: 10.1007/bf02291478, 1975.

[164] A. Zeng, S. Song, M. Nießner, M. Fisher, J. Xiao, T. Funkhouser, "3dmatch: learning local geometric descriptors from rgb-d reconstructions," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1802-1811, .

[165] Y. Wang, J. M. Solomon, "Deep closest point: learning representations for point cloud registration," Proceedings of the IEEE International Conference on Computer Vision, pp. 3523-3532, DOI: 10.1109/iccv.2019.00362, .

[166] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, J. M. Solomon, "Dynamic graph cnn for learning on point clouds," ACM Transactions on Graphics, vol. 38 no. 5,DOI: 10.1145/3326362, 2019.

[167] C. Tang, P. Tan, "Ba-net: dense bundle adjustment network," 2018. http://arxiv.org/abs/1806.04807

[168] E. Jang, S. Gu, B. Poole, "Categorical reparameterization with gumbel-softmax," 2016. http://arxiv.org/abs/1611.01144

[169] Z. Yew, G. H. Lee, "Rpm-net: robust point matching using learned features," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11824-11833, .

[170] S. Gold, A. Rangarajan, C.-P. Lu, S. Pappu, E. Mjolsness, "New algorithms for 2D and 3D point matching," Pattern Recognition, vol. 31 no. 8, pp. 1019-1031, DOI: 10.1016/s0031-3203(98)80010-1, 1998.

[171] R. Sinkhorn, "A relationship between arbitrary positive matrices and doubly stochastic matrices," The Annals of Mathematical Statistics, vol. 35 no. 2, pp. 876-879, DOI: 10.1214/aoms/1177703591, 1964.

[172] E. Aarts, K. Jan, Simulated Annealing and Boltzmann Machines, 1988.

[173] Y. Aoki, H. Goforth, R. Arun Srivatsan, S. Lucey, "Pointnetlk: robust & efficient point cloud registration using pointnet," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7163-7172, DOI: 10.1109/cvpr.2019.00733, .

[174] D. Bruce, T. Kanade, "An iterative image registration technique with an application to stereo vision," Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674-679, .

[175] G. Dias Pais, S. Ramalingam, J. C. Nascimento, R. Chellappa, P. Miraldo, "3dregnet: a deep neural network for 3d point registration," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7193-7203, DOI: 10.1109/cvpr42600.2020.00722, .

[176] K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, DOI: 10.1109/cvpr.2016.90, .

[177] S. Choi, Q.-Y. Zhou, V. Koltun, "Robust reconstruction of indoor scenes," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5556-5565, DOI: 10.1109/cvpr.2015.7299195, .

[178] B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, A. Oliva, "Learning deep features for scene recognition using places database," Proceedings of the Advances in Neural Information Processing Systems, pp. 487-495, .

[179] B. Jian, B. C. Vemuri, "Robust point set registration using Gaussian mixture models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33 no. 8, pp. 1633-1645, 2010.

[180] D. Campbell, L. Petersson, "Gogma: globally-optimal Gaussian mixture alignment," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5685-5694, DOI: 10.1109/cvpr.2016.613, .

[181] B. Eckart, K. Kim, J. Kautz, "Fast and accurate point cloud registration using trees of gaussian mixtures," 2018. http://arxiv.org/abs/1807.02587

[182] H. Xiong, S. Szedmak, J. Piater, "A study of point cloud registration with probability product kernel functions," pp. 207-214, DOI: 10.1109/3dv.2013.35, .

[183] A. P. Dempster, N. M. Laird, D. B. Rubin, "Maximum likelihood from incomplete data via theEMAlgorithm," Journal of the Royal Statistical Society: Series B (Methodological), vol. 39 no. 1,DOI: 10.1111/j.2517-6161.1977.tb01600.x, 1977.

[184] D. Campbell, L. Petersson, "An adaptive data representation for robust point-set registration and merging," Proceedings of the IEEE International Conference on Computer Vision (ICCV),DOI: 10.1109/iccv.2015.488, .

[185] A. Myronenko, X. Song, M. A. Carreira-Perpinán, "Non-rigid point set registration: coherent point drift," Advances In Neural Information Processing Systems,DOI: 10.7551/mitpress/7503.003.0131, 2007.

[186] A. Myronenko, X. Xubo Song, "Point set registration: coherent point drift," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32 no. 12, pp. 2262-2275, DOI: 10.1109/tpami.2010.46, 2010.

[187] M. Lu, J. Zhao, Y. Guo, Y. Ma, "Accelerated coherent point drift for automatic three-dimensional point cloud registration," IEEE Geoscience and Remote Sensing Letters, vol. 13 no. 2, pp. 162-166, 2015.

[188] R. Varadhan, C. Roland, "Simple and globally convergent methods for accelerating the convergence of any em algorithm," Scandinavian Journal of Statistics, vol. 35 no. 2, pp. 335-353, DOI: 10.1111/j.1467-9469.2007.00585.x, 2008.

[189] P. Wang, P. Wang, Z. Qu, Y. Gao, Z. Shen, "A refined coherent point drift (cpd) algorithm for point set registration," Science China Information Sciences, vol. 54 no. 12, pp. 2639-2646, DOI: 10.1007/s11432-011-4465-7, 2011.

[190] J. Lu, W. Wang, Z. Fan, S. Bi, C. Guo, "Point cloud registration based on cpd algorithm," pp. 8235-8240, DOI: 10.23919/chicc.2018.8482763, .

[191] P. Biber, W. Straßer, "The normal distributions transform: a new approach to laser scan matching," vol. 3, pp. 2743-2748, .

[192] M. Martin, "The three-dimensional normal-distributions transform: an efficient representation for registration, surface analysis, and loop detection," 2009. PhD thesis

[193] M. Magnusson, A. Lilienthal, T. Duckett, "Scan registration for autonomous mining vehicles using 3d-ndt," Journal of Field Robotics, vol. 24 no. 10, pp. 803-827, DOI: 10.1002/rob.20204, 2007.

[194] J. E. Dennis, R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, 1996.

[195] H. Hong, "Probabilistic normal distributions transform representation for accurate 3d point cloud registration," pp. 3333-3338, DOI: 10.1109/iros.2017.8206170, .

[196] J. Stückler, S. Behnke, "Multi-resolution surfel maps for efficient dense 3d modeling and tracking," Journal of Visual Communication and Image Representation, vol. 25 no. 1, pp. 137-147, DOI: 10.1016/j.jvcir.2013.02.008, 2014.

[197] M. Magnusson, A. Nuchter, C. Lorken, J. Achim, J. Hertzberg, "Evaluation of 3d registration reliability and speed-a comparison of icp and ndt," pp. 3907-3912, DOI: 10.1109/robot.2009.5152538, .

[198] X. Shi, J. Peng, J. Li, P. Yan, H. Gong, "The iterative closest point registration algorithm based on the normal distribution transformation," Procedia Computer Science, vol. 147, pp. 181-190, DOI: 10.1016/j.procs.2019.01.219, 2019.

[199] M. Attia, Y. Slama, P. Laurent, H. Cao, F. Haddad, "3d point cloud coarse registration based on convex hull refined by icp and ndt," ,DOI: 10.1109/m2vip.2018.8600843, .

[200] S. Xue, G. Li, Q. Lv, X. Meng, X. Tu, "Point cloud registration method for pipeline workpieces based on ndt and improved icp algorithms," IOP Conference Series: Materials Science and Engineering, vol. 677,DOI: 10.1088/1757-899x/677/2/022131, 2019.

[201] Y. Liu, D. Kong, D. Zhao, X. Gong, G. Han, "A point cloud registration algorithm based on feature extraction and matching," Mathematical Problems in Engineering, vol. 2018,DOI: 10.1155/2018/7352691, 2018.

[202] C. Ulas, H. Temeltas, "A 3d scan matching method based on multi-layered normal distribution transform," IFAC Proceedings Volumes, vol. 44 no. 1, pp. 11602-11607, DOI: 10.3182/20110828-6-it-1002.02865, 2011.

[203] H. Hong, B. H. Lee, "Key-layered normal distributions transform for point cloud registration," Electronics Letters, vol. 51 no. 24, pp. 1986-1988, 2015.

[204] J. Lu, W. Liu, D. Dong, Q. Shao, "Point cloud registration algorithm based on ndt with variable size voxel," pp. 3707-3712, .

[205] A. Das, S. L. Waslander, "Scan registration using segmented region growing ndt," The International Journal of Robotics Research, vol. 33 no. 13, pp. 1645-1663, DOI: 10.1177/0278364914539404, 2014.

[206] D. Aiger, N. J Mitra, D. Cohen-Or, "4-points congruent sets for robust pairwise surface registration," ACM,DOI: 10.1145/1399504.1360684, 2008.

[207] A. S. Mian, M. Bennamoun, R. Owens, "Three-dimensional model-based object recognition and segmentation in cluttered scenes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28 no. 10, pp. 1584-1601, DOI: 10.1109/TPAMI.2006.213, 2006.

[208] G. Turk, M. Levoy, "Zippered polygon meshes from range images," Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, pp. 311-318, DOI: 10.1145/192161.192241, .

[209] M. Mohamad, D. Rappaport, M. Greenspan, "Generalized 4-points congruent sets for 3d registration," vol. 1, pp. 83-90, DOI: 10.1109/3dv.2014.21, .

[210] M. Mohamad, M. T. Ahmed, D. Rappaport, M. Greenspan, "Super generalized 4pcs for 3d registration," pp. 598-606, DOI: 10.1109/3dv.2015.74, .

[211] J. Huang, T.-Ho Kwok, C. Zhou, "V4pcs: volumetric 4pcs algorithm for global registration," Journal of Mechanical Design, vol. 139 no. 11,DOI: 10.1115/1.4037477, 2017.

[212] N. Mellado, D. Aiger, "Super 4pcs fast global pointcloud registration via smart indexing," Computer Graphics Forum, vol. 33 no. 5, pp. 205-215, DOI: 10.1111/cgf.12446, 2014.

[213] P. W. Theiler, J. D. Wegner, K. Schindler, "Keypoint-based 4-points congruent sets–automated marker-less registration of laser scans," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 96, pp. 149-163, 2014.

[214] X. Ge, "Automatic markerless registration of point clouds with semantic-keypoint-based 4-points congruent sets," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 130, pp. 344-357, DOI: 10.1016/j.isprsjprs.2017.06.011, 2017.

[215] S. Saeedi, M. Trentini, M. Seto, H. Li, "Multiple-robot simultaneous localization and mapping: a review," Journal of Field Robotics, vol. 33 no. 1,DOI: 10.1002/rob.21620, 2016.

[216] J. Čerňava, M. Mokroš, J. Tuček, M. Antal, Z. Slatkovská, "Processing chain for estimation of tree diameter from gnss-imu-based mobile laser scanning data," Remote Sensing, vol. 11 no. 6, 2019.

[217] P. Rönnholm, X. Liang, A. Kukko, A. Jaakkola, J. Hyyppä, "Quality analysis and correction of mobile backpack laser scanning data," ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences, vol. 3 no. 1,DOI: 10.5194/isprsannals-iii-1-41-2016, 2016.

[218] M. Pierzchała, P. Giguère, R. Astrup, "Mapping forests using an unmanned ground vehicle with 3d lidar and graph-slam," Computers and Electronics in Agriculture, vol. 145, pp. 217-225, 2018.

[219] S. Thrun, M. Montemerlo, "The graph slam algorithm with applications to large-scale mapping of urban structures," The International Journal of Robotics Research, vol. 25 no. 5-6, pp. 403-429, DOI: 10.1177/0278364906065387, 2006.

[220] B. O. Abayowa, A. Yilmaz, R. C. Hardie, "Automatic registration of optical aerial imagery to a lidar point cloud for generation of city models," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 106, pp. 68-81, DOI: 10.1016/j.isprsjprs.2015.05.006, 2015.

[221] X. Ge, H. Hu, B. Wu, "Image-guided registration of unordered terrestrial laser scanning point clouds for urban scenes," IEEE Transactions on Geoscience and Remote Sensing, vol. 57 no. 11, pp. 9264-9276, DOI: 10.1109/tgrs.2019.2925805, 2019.

[222] E. Ferrante, N. Paragios, "Slice-to-volume medical image registration: a survey," Medical Image Analysis, vol. 39, pp. 101-123, DOI: 10.1016/j.media.2017.04.010, 2017.

[223] H. Grant, U. Kruger, P. Yan, "Deep learning in medical image registration: a survey," Machine Vision and Applications, vol. 31 no. 1, 2020.

[224] C. Hubert, "Active graph based semi-supervised learning using image matching: application to handwritten digit recognition," Pattern Recognition Letters, vol. 73, pp. 76-82, 2016.

[225] R. Szeliski, "Image alignment and stitching: a tutorial," Foundations and Trends® in Computer Graphics and Vision, vol. 2 no. 1, 2006.

[226] I. Rocco, R. Arandjelović, J. Sivic, "End-to-end weakly-supervised semantic alignment," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6917-6925, DOI: 10.1109/cvpr.2018.00723, .

[227] W. Liu, J. Sun, W. Li, T. Hu, P. Wang, "Deep learning on point clouds and its application: a survey," Sensors, vol. 19 no. 19,DOI: 10.3390/s19194188, 2019.

[228] Y. Pan, B. Yang, F. Liang, Z. Dong, "Iterative global similarity points: a robust coarse-to-fine integration solution for pairwise 3d point cloud registration," pp. 180-189, DOI: 10.1109/3dv.2018.00030, .

[229] X. Huang, J. Zhang, Q. Wu, L. Fan, C. Yuan, "A coarse-to-fine algorithm for matching and registration in 3d cross-source point clouds," IEEE Transactions on Circuits and Systems for Video Technology, vol. 28 no. 10, pp. 2965-2977, 2017.

[230] M. Attia, Y. Slama, M. Amine Kamoun, "On performance evaluation of registration algorithms for 3d point clouds," pp. 45-50, DOI: 10.1109/cgiv.2016.18, .

[231] E. Lachat, T. Landes, P. Grussenmeyer, "Comparison of point cloud registration algorithms for better result assessment–towards an open-source solution," Towards Photogrammetry, vol. 42, pp. 551-558, 2020.

[232] F. Pomerleau, M. Liu, F. Colas, R. Siegwart, "Challenging data sets for point cloud registration algorithms," The International Journal of Robotics Research, vol. 31 no. 14, pp. 1705-1711, DOI: 10.1177/0278364912458814, 2012.

[233] R. Hänsch, T. Weber, O. Hellwich, "Comparison of 3d interest point detectors and descriptors for point cloud fusion," ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. II-3 no. 3, pp. 57-64, DOI: 10.5194/isprsannals-ii-3-57-2014, 2014.