Closed-Loop Control Characterization of Untethered Small-Scale Helical Device in Physiological Fluid with Dynamic Flow Rates

Abstract

Untethered small-scale helical devices (USHDs) have the potential to navigate blood vessels and treat vascular occlusive diseases. However, there are still many challenges in translating this method into clinical practice, both in terms of localization and wireless motion control. Herein, closed-loop control characterization of the USHD against and along physiological fluid inside a blood vessel phantom at different penetration depths is shown. First the dynamic flow and ultrasound images noise affecting the measurement are modeled, and control system of the USHD based on bifurcation analysis of a 1D hydrodynamic model is designed. Then a region of attraction of a USHD driven by a permanent magnet robotic (PMR) system inside a blood vessel phantom around an equilibrium point is constructed. Further, the point-to-point closed-loop control strategy is implemented based on the magnetic point-dipole approximation and kinematic control of the PMR system and ultrasound feedback inside physiological fluid, blood vessel, and soft tissue. The frequency response of the USHD is characterized against and along the flowing streams of fetal bovine serum within different flow rates in the 6–20 mm s⁻¹ range. The experimental results demonstrate the ability to navigate the USHD inside blood vessel phantoms with maximum position error of $1.99\pm 0.55$ mm.

Introduction

Untethered small-scale helical devices (USHDs) have potential in medical applications, such as targeted drug delivery,^{[ 1–3 ]} thrombus removal,^{[ 4,5 ]} and biosensing.^{[ 6,7 ]} Compared with traditional surgery, USHDs are expected to enable interventions with minimal trauma and relatively fast postoperative recovery. For medical USHDs, many proposed actuation mechanisms have been studied, such as magnetic fields,^{[ 8–10 ]} chemical fuels,^{[ 11,12 ]} light energy,^{[ 13 ]} enzymatic,^{[ 14 ]} and acoustic wave.^{[ 15 ]} Among these actuation mechanisms, USHDs driven by magnetic fields have been extensively studied for two key reasons. First, the effect of the magnetic field on the body is negligible. Second, the magnetic field is not influenced by the physical surroundings inside the body. Recently, Wang et al. proposed a closed-loop control method based on ultrasound guidance to control a colloidal microswarm in 3D space using fields generated by electromagnetic coils, and localization of the microswarm at different penetration depths has been demonstrated.^{[ 16 ]} Yan et al. proposed a control strategy through the use of integrated ultrasound and photoacoustic images to track a USHD, which allows for accurate real-time control and manipulation in optically nontransparent biological tissues.^{[ 17 ]} However, in these studies,^{[ 16,17 ]} the workspace is limited, due to the use of the Helmholtz coils surrounding the workspace. As a result, scaling up for clinical trials is not viable.

The precise control of the intravascular USHDs is critical inside the complex vascular network of the human body, which is similar to that in tissue. Lee et al. drove the USHDs to a designated lesion inside a 3D coronary artery phantom through an external magnetic field and demonstrated thrombus removal.^{[ 18 ]} Pane et al. implemented real-time visualization, position tracking, and motion control of a USHD in the lumen of a tissue-mimicking phantom using ultrasound phase analysis,^{[ 19,20 ]} both in static and in dynamic flow conditions.^{[ 21 ]} Arcese et al. implemented control of a magnetically guided microrobotic system in blood vessels, and they proposed an adaptive backstepping control law that ensures a Lyapunov stable and accurate control of USHD^{[ 22 ]}. Regardless of whether USHDs are controlled inside a complex blood vessel network or tissue, the noninvasive localization technology of USHDs is critical to constructing a corresponding closed-loop control system for medical applications. Salerno et al. presented a magnetic field triangulation algorithm for localization of the capsule inside the gastrointestinal tract.^{[ 23 ]} Son et al. developed real-time localization method using 2D arrays of monoaxial Hall-effect sensors, and this method has been used to accurately estimate the position and orientation of a USHD.^{[ 24 ]} Khalil et al. also developed the magnetic field-based localization method to localize USHDs inside an ex vivo model of a rabbit aorta.^{[ 5 ]} In addition, magnetic resonance imaging (MRI) has been used to localize nanoparticles and steer them to follow a reference path.^{[ 25 ]} Computed tomography (CT) system has also been used to scan an intestinal tract, so that USHDs can move to the target area under the influence of an external magnetic field.^{[ 26 ]}

Compared with MRI, CT, X-ray, and other imaging systems, ultrasound images are radiation free and have a wider range of applications due to its low cost and relatively high frame rate.^{[ 27 ]} Ultrasound imaging system operates at frequencies between 1 and 100 MHz and provides a spatial and temporal resolution of a millimeter to micrometers.^{[ 28 ]} Ultrasound waves are able to penetrate into and move through tissue and bodily fluid inside the body for scanning at different depths using various frequencies. For instance, a 1 MHz ultrasound wave penetrates the body to 4 cm in muscle and 15 cm in fat. Therefore, ultrasound system represents a viable option to visualize human tissue and provide visual feedback for motion control of the USHDs^{[ 29,30 ]}. However, the contrast-to-noise ratio (CNR) of ultrasound images has not been taken into account during the design of control systems, and the USHD localization within ultrasound images is crucial for constructing closed-loop control system. Therefore, we first use the CNR to quantify visibility of the USHD at different penetration depths and design a closed-loop motion control system to quantify the control characteristics of the USHD (Figure  1 ) at different penetration depths in the 15–35 mm range. Compared with existing systems,^{[ 18,20,31 ]} our permanent magnet robotic (PMR) system has an open configuration and unobstructed large workspace, as shown in Figure  2A. In the structural design of the PMR system, the collision between the ultrasound probe and the permanent magnets is avoided. We do not need to consider the issue of coil heating since the rotating magnetic field is generated by two rotating permanent magnets. Therefore, we can use the PMR system for exceptionally long periods of time. In addition, the position of the USHD converges asymptotically to the target position when the angular velocity of two rotating permanent magnets is applied as a function of the position and velocity of the USHD. Further, we achieve the following: 1) Modeling of the USHD inside a blood vessel phantom to predict the influence of the localization gap, flow rates, and CNR on the closed-loop control behavior, 2) in vitro characterization of the frequency response of the USHD against and along physiological fluid inside a blood vessel phantom with different flow rates to determine the bounded behavior of the closed-loop control system, 3) point-to-point closed-loop control using ultrasound feedback and the PMR system (Figure 2A), and 4) Characterization of the closed-loop control system inside a blood vessel phantom at different penetration depths with dynamic flow rates and motion control of the USHD inside a blood vessel phantom with a bifurcation.

Figure 1. Magnetic fields B(p) generated by two synchronously rotating permanent magnets with angular frequency of ω, which can actuate the USHD inside the blood vessel phantom of the leg under the guidance of ultrasound images. A) M1 and M2 are the magnetic dipole moments of the permanent magnets. B) The position of the USHD is determined using ultrasound images. The blue curve and red dashed curves indicate the direction of the original ultrasound waves and the reflected waves via the USHD, respectively. m is the magnetic dipole moment of the USHD.

Figure 2. Frequency response of the USHD under the PMR system. A) A USHD is controlled using a PMR system, which comprises two dipole fields ➀ fixed to two DC motors ➁. The motors are fixed on robotic bases with 3-DOF (pitch ➂, yaw ➃, and a linear motion stage ➄). The blood vessel phantom is fixed in the container ➅ of the soft-tissue phantom. Motion of the USHD is tracked using ultrasound images, and the ultrasound probe ➆ is fixed on another robotic base with 3-DOF. The motor ➇ can rotate the ultrasound probe, and two linear motion stages (➈ and ➉) can translate the ultrasound probe along the x0-axis and the y0-axis of the global coordinate system {x0,y0,z0}. The phantom frame of reference is indicated by {x,y,z}. Frequency response of a 6 mm-long USHD is characterized inside a blood vessel phantom with flow rate of 6–20 mm s−1. B) The frequency response of the USHD is tested when the USHD swims against fluid flow with different flow rates. C) The frequency response of the USHD is tested when the USHD swims along fluid flow with different flow rates. The magnitude of the magnetic field at the position of the USHD is 5 mT. Average speed of the USHD is calculated from five trials at each frequency.

The remainder of this article is organized as follows. Section 2 describes the mathematical modeling and control problem statement of the USHD to understand how experimental results are affected by the localization depth, flow rate, and CNR of the localization system. The PMR system is described, and the CNR and frequency response experimental results are presented in Section 3. Section 4 provides discussions about the limitation of experiments and potential applications of the USHD and the PMR system. Finally, Section 5 concludes and provides directions for future work.

Problem Formulation

For USHDs that swim against and along the flow inside fluid-filled blood vessels and are localized using ultrasound images, designing a control system is a critical component of motion planning. The purpose of the control system is to render a certain compact set positively invariant and asymptotically attractive under the influence of bounded fluid drag and bounded measurement noise. We consider the fluid drag as a uniformly bounded disturbance force on the USHD. We also consider that the relationship between the penetration depth of the blood vessel phantom and the CNR affects the measured output of the USHD.

System Description

The USHD is controlled using the PMR system with three degree of freedom (3-DOF), which can control the field rotation axis of the rotating magnetic field. In the PMR system, a time-periodic magnetic field is applied to the USHD, and the magnetization of the USHD ultimately aligns with the magnetic field. During experiments, the USHD translates by propulsive force combined with a magnetic pulling force, and the translation velocity of the USHD can be controlled by adjusting the angular frequency of the rotating magnetic field. When a USHD swims inside a confined environment, it is subjected to different forces, where a magnetic force and propulsive force will be produced and balanced by hydrodynamic drag force and friction force. This nonlinear system can be described by the following state–space representation.[Image Omitted. See PDF] [Image Omitted. See PDF] [Image Omitted. See PDF]where $\mathbf{x}\mathbf{=}{[x_1{x}_2]}^T$ is the USHD state vector. Further, $x_1$ and $x_2$ represent the position and velocity of the USHD, $\mathbf{y}$ is the measured output, and u is the control input. $\mathbf{A}\in {\Re }^{2\times 2}$ is coefficient matrix, $\mathbf{B}\in {\Re }^{2\times 1}$ is input distribution vector, and $\mathbf{C}\in {\Re }^{1\times 2}$ is output vector. The input d represents the exogenous signal^{[ 32 ]} caused by bounded flow rate, and ζ is the measurement noise caused by noise in ultrasound images (Figure  3A). It is crucial to precisely control the USHD to reach the desired target position under the influence of external signals and noise in ultrasound images.^{[ 33 ]}. To quantify the noise of the USHD within ultrasound images, we calculate the CNR^{[ 34 ]} of the USHD in blood vessel phantom at different penetration depths. The function ϕ includes magnetic, hydrodynamic, and mechanical dynamics of the USHD during motion inside the blood vessel phantom, as follows.[Image Omitted. See PDF]where the magnetic force along 1D blood vessel phantom scales as $f_{\mathrm{m}}\sim 3{\mu }_0\parallel \mathbf{m}\parallel \parallel \mathbf{M}\parallel /2\pi x_1^4$, ${\mu }_0$ is the permeability of free space, $\mathbf{m}$ is the magnetic dipole moment of the USHD, and $\mathbf{M}$ is the magnetic dipole moment of the permanent magnet in the PMR system.^{[ 35 ]} Note that $f_{\mathrm{m}}$ changes sign with $x_1$, and qualitatively the contribution of the magnetic force is not expected to assist propulsion when the USHD swims past the between two permanent magnets (Figure 1A). The drag force is expressed as $f_{\mathrm{d}}=\frac{1}{2}{\rho }_{\mathrm{f}}A{C}_{\mathrm{d}}{(x_2-v_{\mathrm{f}})}^2$, where $C_{\mathrm{d}}$ is the drag coefficient of the USHD, ${\rho }_{\mathrm{f}}$ is the density of fetal bovine serum (FBS), $v_{\mathrm{f}}$ is the flow rate of FBS, and A is the cross-sectional area.^{[ 29 ]} In the case of locomotion of the USHD, we calculate the Reynolds number $(R_e=\frac{{\rho }_{\mathrm{f}}{x}_2{l}_{\mathrm{t}}}{\mu })$ in the range of 1–10, where μ is the viscosity of FBS and $l_{\mathrm{t}}$ is the length of the USHD^{[ 36 ]}. At this level of $R_{\mathrm{e}}$, $C_{\mathrm{d}}$ is asymptotically proportional to $R_{\mathrm{e}}{}^{-1}$. The propulsive force along axis of rotation is given by $f_{\mathrm{p}}=k_{\mathrm{t}}\omega$, where $k_{\mathrm{t}}$ is the thrust coefficient,^{[ 31 ]} and $f_{\mathrm{p}}$ is valid below the step-out frequency of the USHD, and the step-out frequency is directly proportional to the magnetic moment and the magnetic field strength and inversely proportional to the viscosity of fluid. The friction force $f_{\mathrm{f}}={\mu }_{\mathrm{f}}{f}_{\mathrm{w}}$ is proportional to the apparent weight $f_{\mathrm{w}}=({\rho }_{\mathrm{r}}-{\rho }_{\mathrm{f}})g{V}_{\mathrm{r}}$, where ${\mu }_{\mathrm{f}}$ is the friction coefficient, ${\rho }_{\mathrm{r}}$ is the density of the USHD, $V_{\mathrm{r}}$ is the volume of the USHD, and g is acceleration due to gravity.^{[ 29 ]}

Figure 3. The closed-loop control of the USHD. A) The USHD dynamics is obtained by balancing the drag force, magnetic force, propulsive force, and friction force. R(s) and Y(s) represent the system reference input and system output, respectively. The fluid drag is modeled as a uniformly bound disturbance input D(s), and N(s) is the measurement noise due to ultrasound imaging. mt is the mass of the USHD. B) Bifurcation diagram for the real part of eigenvalues of closed-loop control system. At the equilibrium point of the closed-loop control system, the eigenvalues of Jacobian matrix can be calculated. In a small neighborhood of the equilibrium point, the closed-loop control system is stable, when the flow rate is less than a certain value. The upper bound of the disturbance caused by the fluid drag can be estimated. The positive flow rates show that the USHD swims against the flow, and the negative flow rates show that the USHD swims along the flow.

In 1D blood vessel phantom, the proportional-derivative (PD) controller $u=\gamma (\mathbf{x})=-k_{\mathrm{p}}(x_1+\zeta )-k_{\mathrm{d}}(x_2+\stackrel{.}{\zeta })$ is used so that it can control the USHD inside the blood vessel phantoms. When the USHD swims between two rotating permanent magnets, the magnetic force does not play a role because it is located in the uniform field region. By the setting ${\stackrel{.}{x}}_1={\stackrel{.}{x}}_2=0$, we can obtain the equilibrium point,^{[ 37 ]} as follows.[Image Omitted. See PDF]

Equation (5) shows the influence of the fluid flow and the measurement noise in ultrasound images at the equilibrium point. The equilibrium point can also be shifted using the gains of the control system. At the equilibrium point of the closed-loop control system, we can calculate the eigenvalues of the Jacobian matrix of the closed-loop control system, as shown in Figure 3B. Within a specific flow rate in the range of 31 mm s⁻¹, the real part of the eigenvalues of the Jacobian matrix is negative. Therefore, the equilibrium point is stable in a certain compact set. Further, we implement the closed-loop control of the USHD in physiological fluid with dynamic flow rates based on eigenvalue analysis. Note that the structure for model (1–3) includes magnetically actuated devices immersed in a fluid and the characteristics of the actuating field and the surrounding fluid are known a priori. However, the input signals d and ζ may not be known. Therefore, the response of the USHD is characterized against and along the flow to determine its bounded behavior. Likewise, the localization gap between the USHD and the ultrasound probe is varied to increase the noise in the ultrasound images.

Characterization and Motion Control Experimental Results

A PMR system is used to actuate the USHD inside the blood vessel phantoms, and a pulsation pump is utilized to provide different flow rates. In addition, the ultrasound imaging system is employed to track the USHD inside the blood vessel phantoms, and closed-loop control system is constructed based on ultrasound feedback. Characterization of the frequency response and closed-loop motion control experiments are conducted using the aforementioned systems.

Experimental Setup

Three servo motors (MX-106R Dynamixel, Robotics, South Korea) are used to control the pitch angle, yaw angle, and translational DOF of the mobile platform, as shown in Figure 2A. Two permanent magnets (NdFeB, R750F, Amazing Magnetic LLC, California, U.S.A) with diameter of 45 mm and thickness of 30 mm, are fixed to two DC motors (Maxon 47.022.022-0019-189 DC Motor, Sachseln, Switzerland). The orientation and position of the two DC motors are controlled independently to change the pitch angle and yaw angle of the USHD. The USHD (Figure 1B) with length of 6 mm and diameter of 1.5 mm is fabricated through 3D printing using polylactic acid filament (PrimaValue, 3D Printers, The Netherlands). A cylindrical NdFeB magnet with diameter of 1 mm and length of 1 mm is attached to the USHD. A transducer (SonixTouch Q+, BK Medical, Quickborn, Germany) is mounted to another robotic base with 3-DOF, which can change the position of ultrasound probe based on the centerline of blood vessel phantoms. In addition, a pulsation pump (HV-77 910-55, Masterflex, Illinois, USA) is used to provide fluid flow with different rates, and the physiological fluid is FBS (F7524–500ML, Sigma-Aldrich, USA). Demineralized water and gelatine powder (Ec Nnr: 232-554-6, Boom BV, Rabroekenweg, The Netherlands) are used to prepare the agar gel,^{[ 38 ]} and the mixture is contained inside a reservoir. Specifications of the USHD and the PMR system are described in Table  1 .

Table 1 Specification of the USHD and the PMR system. $d_{\mathrm{t}}$ and $l_{\mathrm{t}}$ are diameter and length of the USHD, respectively. $\mathbf{m}$ is the magnetic dipole moments of the USHD. $L_{\mathrm{d}}$ is the distance between the two permanent magnets, $H_{\mathrm{m}}$ is the vertical distance between the pitch joint and the two permanent magnets, and $H_{\mathrm{b}}$ is the vertical distance from the pitch joint to the mobile platform. ${\mathbf{M}}_{\mathrm{1,2}}$ is magnetic dipole moments of two permanent magnets. f is frequency of ultrasound waves, and $Gn$ is gain of ultrasound system. $d_{\mathrm{v}}$ is the diameter of the blood vessel phantom. ${\mathbf{B}}_{\mathrm{m}}$ is magnetic strength between two permanent magnets. ${\rho }_{\mathrm{f}}$ and μ are the density and viscosity of physiological fluid, respectively. $C_{\mathrm{d}}$ is the drag coefficient, and $R_{\mathrm{e}}$ is Reynolds number. $f_{\mathrm{m}}$, $f_{\mathrm{d}}$, $f_{\mathrm{p}}$, and $f_{\mathrm{f}}$ are the magnetic force, drag force, propulsive force, and friction force

Parameter	Value	Parameter	Value	Parameter	Value	Parameter	Value
$d_{\mathrm{t}}\times l_{\mathrm{t}}$ [mm]	$1.5\times 6$	$\mathbf{m}$ $\text{[}\mathrm{A}{\mathrm{m}}^2]$	$6.23\times {10}^{-4}$	Pitch [mm]	1	Helicity $[\circ ]$	71
$L_{\mathrm{d}}$ [mm]	350	$H_{\mathrm{m}}$ [mm]	164	$H_{\mathrm{b}}$ [mm]	240	${\mathbf{M}}_{\mathrm{1,2}}$ $\text{[}\mathrm{A}{\mathrm{m}}^2]$	52.26
f [ MHz]	13.3	$Gn$	42	$d_{\mathrm{v}}$ [mm]	4	${\mathbf{B}}_{\mathrm{m}}$ $[\mathrm{m}\mathrm{T}]$	5
${\rho }_{\mathrm{f}}$ $\text{[kg}{\mathrm{m}}^{-3}]$	906	μ [mPa s]	1.105	$C_{\mathrm{d}}$	0.02	$R_{\mathrm{e}}$	1–10
$f_{\mathrm{m}}$ $[N]$	$\mathcal{O}{(10}^{-6})$	$f_{\mathrm{d}}$ $[N]$	$\mathcal{O}{(10}^{-7})$	$f_{\mathrm{p}}$ $[N]$	$\mathcal{O}{(10}^{-6})$	$f_{\mathrm{f}}$ $[N]$	$\mathcal{O}{(10}^{-7})$

Frequency Response Characterization

Frequency response characterization is implemented to understand the relation between the locomotion speed and actuation frequency of the PMR system, which is influenced by fluid flow. In the frequency response characterization, flow rates of 6–20 mm s⁻¹ are tested, which is greater than the minimum flow rate in the venous system.^{[ 39 ]} In the cases of locomotion against flow rates of 6, 13, and 20 mm s⁻¹, the locomotion speed of the USHD increases linearly with the actuation angular frequency within a certain range, as shown in Figure 2B. However, the speed of the USHD will not increase when the angular frequency increases to 32π rad s⁻¹. We attribute this phenomenon to the fact that USHD does not follow the magnetic field lines at high angular frequencies. Therefore, the step-out angular frequency is 32π rad s⁻¹ when the USHD swims against flow. The similar response characterizations are observed when the USHD swims along flow rates of 6, 13, and 20 mm s⁻¹, as shown in Figure 2C. The frequency response characterization of the USHD indicates its capability to achieve bidirectional locomotion in maximum flow rate of 20 mm s⁻¹. In the closed-loop control system, the actuation angular frequency is limited below step-out frequency.

Motion Control Results

In motion control experiments, the feedback provided by the ultrasound system is used to track the USHD. The ultrasound system is set to B-mode to display a sequence of rapidly acquired images at frame rate of 18 Hz, and the gain of the ultrasound system and frequency of the propagating waves are set to 42 and 13.3 MHz in all motion control trials. The communication between the PMR system and the ultrasound system is implemented in real time using open-source library OpenIGTLink (version 3.1.0).^{[ 40 ]}

Before implementing the closed-loop motion control of the USHD, the blood vessel phantom centerline is determined so that we can plan the motion path of the USHD. We accomplish centerline registration of the blood vessel phantoms using ultrasound images, during hich the position of the ultrasound probe is controlled to scan the cross section of the blood vessel phantom, as shown in Figure  4A–C. Therefore, we can obtain the centerline based on the position of the ultrasound probe and the cross section of the blood vessel phantom with a bifurcation, as shown in Figure 4D. Further, a PD controller is used to calculate the angular speed ω of the dipole fields. The angle $\angle {\mathbf{B}}^{\mathrm{d}}(\mathbf{p})$ of the desired rotating magnetic fields is constructed using the point-dipole approximation (6) and the homogenous transformation (7), when the USHD swims inside a blood vessel phantom with a bifurcation (please refer to Appendix). Therefore, the direction of the magnetic dipole moment of two permanent magnets is constructed using the joint space coordinates. Further, we implement an OpenCV (version 3.4.9) image processing library to detect and track the USHD in each ultrasound system frame, and the template image is updated in real-time during each trial.^{[ 41 ]} The control strategy is implemented in C++ on a computer running Linux Ubuntu 16.04.

Figure 4. Acquisition of the USHD motion path. A) The ultrasound probe is used to scan the cross section of position ➀ in the blood vessel phantom. B) The ultrasound probe is used to scan the cross section of position ➁ in the blood vessel phantom. C) The ultrasound probe is used to scan the cross section of position ➂ in the blood vessel phantom. D) The motion path of the blood vessel phantom with a bifurcation is constructed using the cross section of the blood vessel phantom and the position of the ultrasound probe. All scale bars are 4 mm.

The CNR is used to quantify the USHD visibility in the ultrasound images, as shown in Figure  5A. The CNR decreased with the increase of penetration depths in the ultrasound images. Note that the high CNR is measured when high-frequency ultrasound waves are used, as shown in Figure 5B. In order to localize the position of the USHD and then construct a closed-loop control system, a high CNR is required. Therefore, a closed-loop control system is constructed to navigate the USHD in the 1D blood vessel phantom with different dynamic flow rates under the condition of penetration depths 15, 25, and 35 mm, as shown in Figure  6 . For different flow rates and different penetration depths, we observe that all states of the closed-loop control system asymptotic converge to the reference position, as shown in Figure 6A,D,G. We also observe that the position errors and the measured $x_2$-component of the closed-loop control system increase with the penetration depths, as shown in Figure 6B,E,H. The measured position error and $x_2$-component increase for different flow rates owing to the decrease of the CNR with penetration depths. In upstream and downstream fluid, Figure 6C,F,I shows motion of the USHD from starting point toward a reference position at different penetration depths. During the experiments, the ultrasound images are acquired at a rate of 18 fps, and the execution time of the template matching method for a single ultrasound image is $0.24\pm 0.09$ s.^{[ 42 ]} Therefore, we need to control the swimming velocity of the USHD by adjusting the gain of the controller, which is essential for ultrasound system tracking because an overfast navigation velocity impairs tracking accuracy. In dynamic flow conditions, experimental results demonstrate that our proposed control system can stably navigate the USHD to the reference position under the interference of external bounded flow rate and ultrasound image noise. The closed-loop control results inside the 1D blood vessel phantom are summarized in Table  2 .

Figure 5. The USHD visibility in the ultrasound images. A) At different penetration depths (Dt), the B-mode of ultrasound system is used to observe the USHD. All scale bars are 5 mm. B) The CNR is calculated for the USHD at different penetration depths and different ultrasound waves frequencies. All error bars denote the standard deviation from five trials.

Figure 6. Closed-loop control characterization of the USHD against and along the flow. A) At different penetration depths (Dt), the state of closed-loop control system is convergent when the flow rate is 6 mm s−1. B) For vf=6 mm s−1, closed-loop control system state deviation. C) For vf=6 mm s−1, the USHD swims against and along the flow toward the reference position at penetration depth of 15 mm. D) At different penetration depths, the state of closed-loop control system is convergent when the flow rate is 13 mm s−1. E) For vf=13 mm s−1, closed-loop control system state deviation. (F) For vf=13 mm s−1, the USHD swims against and along the flow toward the reference position at penetration depth of 25 mm. G) At different penetration depths, the state of closed-loop control system is convergent when the flow rate is 20 mm s−1. H) For vf=20 mm s−1, closed-loop control system state deviation. I) For vf=20 mm s−1, the USHD swims against and along the flow toward the reference position at penetration depth of 35 mm. The red arrow indicates the direction of fluid flow. All scale bars are 5 mm. Please refer to the Video S1, Supporting Information.

Table 2 At different penetration depths $(D_{\mathrm{t}})$, closed-loop control of the USHD is implemented inside a 1D blood vessel phantom against and along fluid flow of 6, 13, and 20 mm s⁻¹. The MPE in each case is calculated from five motion control trials

Flow rate	Navigation [mm]	$D_{\mathrm{t}}=15$ [mm]	$D_{\mathrm{t}}=25$ [mm]	$D_{\mathrm{t}}=35$ [mm]
6 mm s−1	Against flow MPE	$0.66\pm 0.18$	$0.92\pm 0.50$	$0.74\pm 0.56$
	Along flow MPE	$0.43\pm 0.21$	$1.26\pm 0.42$	$1.99\pm 0.55$
13 mm s−1	Against flow MPE	$0.26\pm 0.14$	$0.59\pm 0.52$	$0.85\pm 0.63$
	Along flow MPE	$0.51\pm 0.30$	$0.55\pm 0.18$	$1.13\pm 0.43$
20 mm s−1	Against flow MPE	$0.30\pm 0.21$	$0.71\pm 0.16$	$0.96\pm 0.51$
	Along flow MPE	$0.79\pm 0.43$	$1.20\pm 0.36$	$1.90\pm 0.37$

Figure  7 shows that the USHD can be controlled to navigate inside a blood vessel phantom with a bifurcation under ultrasound image noise and bounded flow rate. In the case of the flow rate of 20 mm s⁻¹ and penetration depth of 25 mm, the USHD swims upstream along paths 1 and 2, as shown in Figure 7A. For path 1, the angle of the rotating magnetic field can be changed using the joint space coordinates of the PMR system, as shown in Figure 7B, and the motion of the USHD at this moment is shown in Figure 7E. In the motion of locomotion upstream, the mean position errors (MPEs) along paths 1 and 2 are measured as $0.86\pm 0.44$ and $1.52\pm 0.17$ mm, respectively. Similar to Figure 7A, the USHD is controlled in the downstream fluid along paths 1 and 2, as shown in Figure 7C. For path 2, the angle of the rotating magnetic field can be changed when the USHD swims along flow rate inside the bifurcation blood vessel phantom, as shown in Figure 7D, and the motion of the USHD at this moment is shown in Figure 7F. In the case of motion of locomotion downstream, the MPEs along paths 1 and 2 are measured as $1.78\pm 0.19$ and $0.87\pm 0.53$ mm, respectively. With the proposed control strategy, we can overcome the disturbance and accurately navigate the USHD to the correct path under ultrasound guidance. The closed-loop control results in a blood vessel phantom with a bifurcation are summarized in Table  3 .

Figure 7. The USHD is controlled inside a blood vessel phantom with a bifurcation, when the flow rate is 20 mm s−1 and the penetration depth is 25 mm. A) In the upstream physiological fluid, the USHD swims toward the target position along paths 1 and 2, respectively. B) The PMR system configurations at two time instants during the navigation in path 1: 13 and 22 s. C) In the downstream physiological fluid, the USHD swims toward the target position along paths 1 and 2, respectively. D) The PMR system configurations at two time instants during the navigation in path 2: 9 and 20 s. E) The USHD is controlled against the flow to swim toward the target position inside path 1. F) The USHD is controlled along the flow to swim toward the target position inside path 2. The white arrow indicates the direction of motion of the USHD, and the red arrow indicates the direction of fluid flow. All scale bars are 5 mm. Please refer to the Video S1, Supporting Information.

Table 3 At penetration depths of 25 mm and flow rate of 20 mm s⁻¹, closed-loop control of the USHD is implemented inside the bifurcation blood vessel phantom against and along fluid. The MPE in each path is calculated from five motion control trials

Flow rate	Navigation	Path 1	Path 2
20 mm s−1	Against flow MPE [mm]	$0.86\pm 0.44$	$1.52\pm 0.17$
	Along flow MPE [mm]	$1.78\pm 0.19$	$0.87\pm 0.53$

Regardless of the 1D blood vessel phantom or a blood vessel phantom with a bifurcation, our experimental results demonstrate that the proposed strategy allows us to control the motion of the USHD without any additional computational burden and can be used readily to perform feedback control algorithms. By applying the proposed closed-loop motion control, we can navigate the USHD inside the blood vessel phantoms, indicating that the medical imaging that guided the USHD has great potential for performing automated delivery and targeted therapy in dynamic environments.

Discussion

Motion control of the USHDs has the potential for improved biomedical applications. In this work, we report a closed-loop control strategy to navigate the USHD in physiological fluid with dynamic flow rates using ultrasound imaging. Compared with other USHD in the blood vessel phantom,^{[ 10,18,29 ]} we model the dynamic flow as the bounded exogenous disturbance signal and quantify the ultrasound images noise using CNR, and then we analyze closed-loop control characterization of the USHD inside a blood vessel phantom at different penetration depths with dynamic flow. In addition, our PMR system has large workplace, and we don't need to consider the problem of electromagnetic coil heating, so it can be used to drive the USHD for a long time. However, motion control and the localization and biomedical applications of the USHD remains challenging for the in vivo condition. The biological hybrid USHD seems an ideal choice considering the need for biocompatibility and biodegradability for in vivo tasks. The USHD could be fabricated by combining magnetic nanoparticles and biocompatible materials, such as gelatin methacryloyl and hyaluronic acidmethacryloyl, thus showing promising potential for biomedical applications. In the venous environment, average flow rates of blood are approximately in the range from 5 to 50 mm s⁻¹, and blood flow rates of aorta and medium artery system are greater than venous blood flow rates.^{[ 39 ]} In our experiments, the maximum flow rate of a blood vessel phantom is 20 mm s⁻¹, which is four times as large as the minimum flow rate in the venous system. However, under high flow rate conditions, the propulsive thrust generated by rotating of helical body is not sufficient to move against high flow rates. Therefore, it is essential to improve the performance of the magnetic system by incorporating permanent magnets with relatively large magnetic field. A large rotation magnetic field is able to increase the torque on the USHD, thereby improving the frequency response characteristics of the USHD. In addition, the large magnet could increase the magnetic force, which could be managed to improve the propulsion of the USHD. The USHD could be designed with more magnetic material to improve its magnetic moment, further increasing the USHD's thrust against high flow rates.

In ultrasound system, the tunning of the frequency of the ultrasound waves depends on a trade-off between the penetration depth and the axial resolution. High-frequency ultrasound waves give greater axial resolution than low-frequency ultrasound waves. In contrast, high-frequency ultrasound waves do not allow adequate penetration. In our experiments, the selected penetration depths are 15, 25, and 35 mm, as shown in Figure 5, which are similar to depths of superficial veins of the legs. Their depths are estimated to vary from 2 to 30 mm in the region of the anterior thigh in adults.^{[ 43 ]} In addition, the diameter of blood vessel phantom is 4 mm, and the diameter and length of the USHD are 1.5 and 6 mm, respectively. Under this condition, 13.3 MHz ultrasound wave is used to localize a magnetic USHD in the millimeter scale. However, if we navigate the USHD inside small diameter vessels such as venules and capillaries, smaller dimensions must be used for the USHD. Therefore, the ultrasound waves with high-frequency will be utilized to detect the micrometer scale USHD. Further, the microrobot will not be able to be localized, when the penetration depth is increased to 80 mm.^{[ 16 ]} We can also observe that CNR decreases with increasing penetration depth, as shown in Figure 4. Therefore, the penetration depth and the size of the USHD could represent limitations for minimally invasive ultrasound localization techniques.

Motion control of the USHD has been validated inside blood vessel phantoms with dynamic flow rates. Therefore, in venous systems with low flow rates, the USHD has the potential to be used for medical applications. The formation of blood clots could cause local ischemia, which causes tissue to lack oxygen, and irreversible damages could occur quickly. Therefore, mechanical removal of blood clots is a promising approach using the USHD, which could provide high removal rates of blood clots compared to pharmacological thrombus dissolution.^{[ 5 ]} Additionally, targeted drug delivery using the USHD is a promising technique capable of overcoming the limitations of conventional chemotherapy that relies on body circulation.^{[ 44 ]}

Conclusions and Future Work

Closed-loop motion control system of the USHD is designed using the PMR system and an ultrasound imaging system. By means of this control system, we are able to study the closed-loop control characteristics of the USHD in a physiological fluid with dynamic flow rates and achieve asymptotic convergence inside the blood vessel phantoms. The frequency response results show that the locomotion speed of the USHD varies linearly with the actuation frequency below the step-out frequency, regardless of the direction of flow fluid. In addition, the USHD can overcome flow rate of 20 mm s⁻¹, which is four times as large as the minimum flow rate in the venous system. The experimental results demonstrate that ultrasound imaging can detect and track the USHD inside both a blood vessel phantom at different penetration depths and a blood vessel phantom with a bifurcation. In all closed-loop control experiments, the maximum position error of the USHD is $1.99\pm 0.55$ mm.

As part of future studies, the USHD will be controlled inside a 3D vascular network phantom, and ex vivo experiments will be implemented. Our closed-loop control experiments are conducted against and along fluid flow with flow rate of 6, 13, and 20 mm s⁻¹. This flow rate is greater than blood flow in small arterioles, capillaries, and venules. However, blood flow in veins with higher flow rates, the aorta, and arteries are higher than the flow rate used in this study. If the USHD can move against greater flow rates, it is necessary to improve the performance of our the PMR system by incorporating permanent magnets with relatively large magnetic field.

Appendix

The USHD is driven by two synchronized rotating dipole fields. These fields are produced by rotating permanent magnets that are position controlled using a robotic system, as shown in Figure 2A. The PMR system has 3-DOF, pitch angle (α) that describes the rotation about the intersection of the plane ($y_0=0$ mm) and the plane ($z_0=240$ mm), and yaw angle (θ) that describes the rotation about the $z_0$-axis, and these 2-DOF are fixed to a linear mobile platform that translates along the $y_0$-axis. The moving distance of the mobile platform is represented by L in the global reference frame, $\{x_0,y_0,z_0\}$. Therefore, motion of the permanent magnet is defined using the joint space coordinates, $\mathbf{q}=(\alpha ,\theta ,L{)}^{\mathrm{T}}$. Each field is modeled according to the following point-dipole approximation.^{[ 45 ]} [Image Omitted. See PDF]where $\mathbf{p}$ is the position vector of the USHD with respect to the rotating permanent magnet, $\mathbf{p}$ can be obtained by the output $\mathbf{y}$ of the system, and ${\mathbf{M}}_i$ denotes the magnetic dipole moment of ith permanent magnet.

To control the magnetic field in Equation (6), the direction of the magnetic dipole moment ${\mathbf{M}}_i$ of the permanent magnet is periodically varied and its rotation axis is controlled. The magnetic moment ${\mathbf{M}}_i$ can be constructed using the joint space variable $\mathbf{q}$ as follows.[Image Omitted. See PDF]where ${}^0{\mathbf{T}}_i^3(\mathbf{q})$ represents ith homogenous transformation matrix from the frame of reference of the ith permanent magnet to the global reference frame. Note that ${}^0{\mathbf{T}}_i^3(\mathbf{q})$ completely characterizes the position and orientation of the ith permanent magnet based on the joint variables of the system. In Equation (7), the rotation matrix of the ith permanent magnet with respect to the global reference frame is given by[Image Omitted. See PDF]

The position vector of the ith permanent magnet with respect to the global reference frame is given by[Image Omitted. See PDF]where $L_{\mathrm{d}}$ indicates the distance between the rotating permanent magnets. If the magnetic force is required to be increased to assist actuation, then this distance can be decreased. Note that decreasing this distance will also affect the magnetic torque and the step-out frequency of the USHD will be increased. In Equation (9), $H_{\mathrm{m}}$ is the vertical distance between the joint that controls the pitch and the two permanent magnets, and $H_{\mathrm{b}}$ represents the vertical distance from the joint that controls the pitch to the mobile platform. In contrast to $L_{\mathrm{d}}$ which can be used to increase the magnetic force and magnetic torque, $H_{\mathrm{m}}$ and $H_{\mathrm{b}}$ do not change the magnitude of the magnetic field and gradient in the workspace.

We assume that the magnetization of the USHD ultimately aligns with the magnetic field. Consequently, the relationship between the magnetization of the USHD and the magnetic field is as follows.[Image Omitted. See PDF]

The relationship between the applied field and the orientation of the USHD can be used to design a closed-loop control system to change field rotation axis and orients it toward the desired reference position. In this case, a prescribed trajectory along the vessel will provide waypoints. Further, a position error vector is defined as follows.[Image Omitted. See PDF]where ${\mathbf{p}}_{\text{ref}}$ is the reference position of the blood vessel phantom, and the PD controller $u=-k_{\mathrm{p}}|\mathbf{e}|-k_{\mathrm{d}}|\stackrel{\mathbf{.}}{\mathbf{e}}|$ is further used to drive the USHD. Now, we turn to the direction of the field rotation axis to achieve directional control of the USHD. According to the point-dipole approximation (6) and the direction error of the USHD inside a blood vessel phantom, the direction of the desired magnetic field is calculated using[Image Omitted. See PDF]where $\angle {\mathbf{B}}^{\mathrm{d}}(\mathbf{p})$ represents the angle of the desired magnetic field, and $e_x=|{\mathbf{p}}_{\text{ref}}-\mathbf{p}{|}_x$ and $e_y=|{\mathbf{p}}_{\text{ref}}-\mathbf{p}{|}_y$ are the position error along the x- and y-axis, respectively. We can construct desired magnetic field ${\mathbf{B}}^{\mathrm{d}}(\mathbf{p})$ based on the angle of desired magnetic field $\angle {\mathbf{B}}^{\mathrm{d}}(\mathbf{p})$ and calculate the desired magnetic dipole moment ${\mathbf{M}}_i^{\mathrm{d}}$ using Equation (6). Then, the desired magnetic moment ${\mathbf{M}}_i^{\mathrm{d}}$ can be constructed using the joint control variable $\mathbf{q}$. Since the direction of the dipole moment ${\mathbf{M}}_i$ is perpendicular to the direction of the rotation axis of the magnetic field, the rotation axis is determined. The desired control input ${({\theta }^{\mathrm{d}},{\alpha }^{\mathrm{d}},L^{\mathrm{d}})}^{\mathrm{T}}$ is controlled and updated based on the position of the waypoints along the prescribed centerline of the blood vessel phantom with a bifurcation.

Acknowledgements

This work was supported by the European Research Council (ERC) under the European Union's Horizon 2020 Research and Innovation programme under grant 866494 project-MAESTRO, and financial support from the China Scholarship Council (CSC) is acknowledged.

Conflict of Interest

The authors declare no conflict of interest.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.