Virtual Environment for Manipulating Microscopic Particles With Optical Tweezers

Headnote

In this paper, virtual reality techniques are used to define an intuitive interface to a nanoscale manipulation device. This device utilizes optical methods to focus laser light to trap and reposition nano-to-microscopic particles. The underlying physics are simulated by the use of Lagrange mechanics. A unique control method for the manipulation of the particles is also provided. The user can naturally grab and steer the particles. Behind the scene, a complex computation is performed to find the new location of the potential field induced by the laser beam that would move the particles accordingly. Haptic feedback is used to constrain the steering motion within the physical capability of the potential field.

Key words: nanoscale assembly; nanotechnology; optical tweezers; virtual reality.

Accepted: June 3, 2003

1. Introduction

The development of nanotechnology [1] has generated a great deal of scientific interest, but the promise of nanotechnology will be realized only by manufacturing commercial products from assemblies of atoms, molecules and nanoscale components. While nanoscale measurement and fabrication techniques are advancing rapidly, few effective techniques for manipulation of nanoscale objects are available. Manipulation of nanoscale objects can be achieved using a scanning probe microscope, such as an AFM (Atomic Force Microscope) or an STM (Scanning Tunneling Microscope). Guthold et al. [2] performed rolling and sliding of carbon nanotubes using an AFM. Eigler and Scheweizer [3] used an STM to position individual xenon atoms on a single-crystal nickel surface, but these tools do not yet offer very meticulous control of objects (e.g., the ability to grasp and release). Laser beams can also be used to trap and manipulate small particles. A laser apparatus, called OT (optical tweezers) [4], provides the user with a noncontact method for manipulating objects that can be applied to viruses, bacteria, living cells and synthetic micro and nanoscale particles.

Each method has strengths and weaknesses. The AFM can be used to manipulate a larger set of objects, and the relative forces required to manipulate the particles can be more easily derived. The disadvantage is that it is generally less accurate than the STM, and since the AFM can directly contact the sample, considerations of sharpness and wear of the tool tip must be accounted for. The STM, being non-contact, has little tool wear, yet manipulation is limited to conducting and semiconductor materials and typically will occur under vacuum conditions. The strength of OT is that they are well suited for manipulating biological objects and refractive and metallic particles, yet it is usually used in solution. The AFM and STM are generally limited to two dimensions with a very limited third dimension (sometimes referred to 2.5 dimensions) whilst OT can work in three dimensions. It is also interesting to note that AFM and STM can work both in imaging and manipulation mode, but OT has primarily been used for manipulation, although 3D imaging has been reported [5]. All these techniques require complex driving electronics under sophisticated computer control.

The operation of nanoscale manipulators can be done through tele-operation or automatic manipulation. In the former approach, a human operator acts as a part of the control loop and issues commands by judgments based on various forms of sensory cues. The sensory cues can be audio, visual and haptic. In the latter approach, the computer program makes autonomous decisions based on an underlying set of constraints or rules to perform the predetermined manipulation scenario. In developing these constraints and rules, one must have some understanding of nanoscale dynamics, but this is still an emerging field of research. This is why tele-operation is gaining favor for certain user interfaces. Tele-operation needs to virtually present the nanoscale environment to the operator such that the operator can be aware of the target objects and the surroundings. Various techniques in virtual reality can be used to enhance this human-in-the-loop control system by letting the user feel immersed in the environment. Among these techniques, haptic display plays a vital role for the following reason.

One of the most challenging problems in nanoscale manipulation is that real-time image sensing of nanoscale artifacts is difficult. The imaging sampling time of an AFM and STM can take seconds, or possibly minutes, depending on the scan rate and the sampling range, and cannot be done simultaneously with the manipulation. Thus the only real-time feedback occurs with the AFM by way of the force imparted on the probe when sensing nanoscale objects (contact or non-contact modes) or by the STM current fluctuations that can be correlated to height of the tip above the surface. The user feedback occurs through linking the AFM force or STM height measurement directly to a haptic device using an appropriate amplification routine. This need for real-time exploration is what attracts researchers to develop man-machine interfaces for nanoscale manipulation that use haptic display technologies. Some of the work is described below.

Hollis et al. [6] demonstrated that atomic-scale land-scapes could be explored and felt with the hand in real time by interfacing a haptic feedback device with an STM. Their haptic device is based on Lorentz magnetic levitation. Sitti and Hashimoto [7] used an AFM with a one degree of freedom haptic device to sense nanoscale forces. They demonstrated two-dimensional positioning of micrometer sized latex particles. Guthold et al. [2] used a force feedback device coupled with an AFM tip to haptically sense and manipulate carbon nanotubes, DNA and viruses. All of these research efforts used a graphical display in addition to the haptic display. The topographical data used for the graphical display of the nanoscale objects were acquired by using the probe in the imaging mode that was also used in the manipulation mode. The limitation of this method is that while manipulating the specimen, the graphical display is static and requires additional scans to see the result of the manipulation.

With OT, typical manipulation can be done through a two-dimensional pointing device such as a mouse or joystick. Usually the manipulation is done in noncontact mode, so there is no direct force that can be rendered on a haptic display. Because of the limited ability of traditional optical microscopy to resolve nanometer-scale structures, OT, unlike AFM or STM, must use other means and synthesize virtual sensory inputs to the operator. This paper presents the work on developing an intuitive interface for manipulating particles through OT with the use of VR (Virtual Reality).

2. Approach

The Lagrange equation that serves as the foundational model for the trajectory of a particle exposed to a potential field is first introduced. The potential field is an abstraction of the potential energy field that is induced by the laser beam. Note the potential field depends on both the laser and the particle. For example, if the geometry of the particle changes, the potential field is no longer applicable. Then the equation of potential field position that can be used to steer a particle is derived. Lastly, an approach in constraining the steering motion within the physical capability of the optical laser instrument through the use of haptic feedback is presented.

2.1 Modeling of Optical Tweezers Physics

2.3 Working Envelope

The working envelope (WE) is defined as the envelope of displacement vectors a particle can be moved by a potential field. The WE is dynamic in that it changes due to the velocity of the particle and the time lapse until the next control cycle. WE limits the motion of the particle.

The WE is numerically computed. By using the coordinate values at equally spaced, regular grid points as the displacement vectors of a particle with certain elapsed time and initial velocity of the particle, solutions to Eq. (7) are sought. The grid points where solutions exist form a lump. By connecting the boundary of this lump with a loop composed of line segments, one gets the WE. Feldman's algorithm [12] is used for computing the loop pruning the spurious branches. Furthermore, the entire interior set of grid points to the loop are checked to determine if they have solutions. Each point with no solution is recalculated with smaller square elements for computing the initial guess point to see if that would lead to solutions. In cases where this recursive refinement does not eventually lead to a solution, an average of neighboring solutions is used as the solution.

2.4 User Interface

The operator steers the particle with a cursor, which is a small spherical ball that is position controlled by the stylus of the haptic device. The operator moves the cursor to the particle of interest and presses the button attached to the stylus and then the particle is rigidly glued to the cursor. The system is now in the steering mode. The system then shows the WE of the particle in question. The WE depends on the time lapse between the user control cycles and the velocity of the particle. The haptic stylus shows no resistance when the particle is moved inside the WE yet encounters a virtual haptic wall when the particle is moved outside the WE. A simple linear spring model is used to render the wall.

The ellipse in Fig. 1 (a) represents the WE of the particle at the center. When the commanded displacement of the particle is within the boundary of the WE, such as the two arrows that are inside the ellipse, the operator experiences no reaction force, and the particle is moved to that position indirectly by the movement of the potential field. In Fig. 1 (b), however, the commanded displacement is outside the boundary of the WE. In this case the displacement intersects with the WE and the particle is moved up only to the intersection point. The excess amount of the displacement command (initial displacement command subtracted by the modified displacement command) is used to determine the relative magnitude of the reaction force. This way the operator would sense a virtual wall at the WE. Notice this way the particle would still move when the issuing command is physically incorrect. This maximizes the capability of the optical tweezers.

3. Results

The results are divided into three subsections. Firstly, understanding of the particle trajectory is gained with various experiments. Secondly, the numerical results in computing the laser position and WE is given. Lastly, the virtual environment is presented.

3.1 Characteristics of the Particle Movement The trajectories of particles in the potential field can be computed by Eq. (7). To gain insight into how the particles would move in the potential field, the trajectories of particles placed in grid points starting from rest is plotted. The result is illustrated in Fig. 2 (a). Following constants are used: a = 600 nm, b = 300 nm, P^sub 0^= 1.0 × 10^sup -6^ nJ, m = 1.0 × 10^sup -12^ g, x = 0 nm/µs, y = 0 nm/µs and [Delta]t= 1.0 µs. The grid spacing in Fig. 2 is 160 nm. To examine the nonlinear nature of the trajectory, the experiment is tried with multiple time steps. The result of using 10 time steps totaling one µs is given in Fig. 2 (b). It can be seen that a more accurate result can be achieved by using multiple time steps, but it requires more computation time. As mentioned before, fourth-order Runge-Kutta with a fixed time step was used for the numerical integration. The calculated results agreed with the results from the commercial numerical solver (SIMULINK1). A varied time step method was also tested in SIMULlNK giving agreeing results.

Fig. 1. Displacement of the particle (a) within WE (Working Envelope) (b) outside WE (Working Envelope).Fig. 2. Trajectories of particles in optical potential field (a) one time step of 1 µs (b) ten time steps of 0.1 µs.

The next question was how far the particles would travel in the given potential field. Figure 3 (a) is a plot of the displacement distance as a function of the particle location from the center of the potential field. Figure 3 (b) is the plot of iso-contours having the same displacement distance. The legend of the distances is shown at the top right corner. Notice that two spots symmetrical to the horizontal axis have maximal displacement. These spots can be important as they dictate the displacement limit of a given laser force field. It is also important to note that the center spot of the laser beam shows low displacement distance. The locus of maximal displacements in radial directions is given in Fig. 4 (a). The directions were sampled uniformly in increments of 3.6°.

Fig. 3. Plot of displacement distances (a) magnitude (b) iso-contours of equal distances.

Fig. 4. Locus of maximum displacements in radial directions (a) particle initially at rest (b) particle initially moving.

In realistic situations the particles would have velocities. The effect of velocities on the particle trajectory is studied. In Fig. 4 (b), the following constants are used: x = 100 nm/µs, y = 100 nm/µs. Notice the locus is changed due to the initial velocity of the particle.

In Fig. 3, the particle was initially at rest, the kinetic energy being zero. Since the potential energy is higher on the periphery than at the center, the combined total energy is higher on the periphery. This means that each (x, y) location in Fig. 3 depicts a particle in a different total energy state. One can obtain a different plot when one enforces constant total energy. Fig. 5 (a) is the coordinate system used for the plot in Fig. 5. The ellipse denotes the constant potential energy curve. The small black circle lying on the ellipse represents a particle with a velocity at a certain angle ("velocity angle") to the horizontal axis. The position on the curve is determined by the "position angle." Angles in Fig. 5 (b) and (c) are shown in radians. The particle displacement distance is plotted by varying the particle position and its velocity direction on a constant potential energy (1.0 × 10^sup -6^ nJ) curve with constant kinetic energy (5.0 × 10^sup -9^ nJ). The result is shown in Fig. 5 (b). Figure. 5 (c) is the plot of iso-contours having the same displacement distance.

3.2 Computing the Laser Position

Now comes the most important part, computing the center of the potential field such that a particle at a certain initial velocity would displace to a required position in a given amount of time. The solution of Eq. (7) can be very involved due to the highly nonlinear nature of [Delta]x and [Delta]y, especially when [Delta]t is large. The plots of [Delta]x and [Delta]y are given in Fig. 6 (a) and (b), respectively. Following parameters are used: [Delta]t = 1.3 s, x = 100 nm/µs, y = 100 nm/µs. For visual clarity the axes are moved to the bottom left. The origin is at the center. In Fig. 7, the two small loops aligned vertically at the center are the locus of points satisfying [Delta]y = -800 nm. Similarly, the biggest loop is the locus of points satisfying [Delta]x = -160 nm. The two intersection points shown in Fig. 7 are the two solution points satisfying both conditions. If this is extended spatially, and the solutions are sought, one can compute discrete points where the solutions exist or do not exist. For example, when one uses the coordinates of regular grid points as [Delta]x, [Delta]y and use small dots to represent points where there are no solutions and larger dots for points where there are, one can get Fig. 8 (a). The initial guess points were obtained with the grid distance equally spaced as shown in Fig. 8 (a). The computation took 17 s on a Pentium PC. Notice that the envelope of solvable grid points is connected with line segments. The connected segments form the WE. If desired, one can get more accurate results by using a finer grid distance for the initial guess points. Figure 8 (b) was obtained with four times finer resolution and took 288 s. Furthermore, Fig. 8 (c) was obtained with sixteen times finer resolution and took 4591 s. Notice there is very little difference between Fig. 8 (b) and Fig. 8 (c). Formal formulation for optimal grid spacing for the initial guess point is not easy to do. Therefore the results as a function of different resolutions are examined and the one where the increase in the resolution did not significantly improve the solution was chosen.

Fig. 5. Plot of displacement distances for constant energy (a) coordinate system (b) magnitude (c) iso-contours of equal distances.

Fig. 6. Solution surface (a) plot of [Delta]x (b) plot of [Delta]y.

3.3 The Virtual Environment

Figure 9 (a) illustrates the initial screen of the virtual environment (VE). The descriptions of the graphical objects are as follows. The solid sphere at the center is the particle. It is being illuminated by a dithered laser beam shooting in the direction of the plane. This is shown as the ellipse in Fig. 9 (a). The small sphere to the left of the particle is the cursor for the operator. When the haptic stylus is moved, the cursor follows accordingly. The particle can be moved only after it has been grabbed. To grab the particle, the operator moves the cursor to either touch or be inside the particle and presses the button attached to the stylus. To release the particle the operator simply releases the button. The diameter of the particle is one µm and the grid spacing distance is 2.9 µm. The physical parameters used for the potential field are a = 600 nm, b = 300 nm, p^sub 0^ = 1.0 × 10^sup -6^ nJ, m = 1.0 × 10^sup -12^ g.

Fig. 7. Solution points.

Figure 9 (b) illustrates the instant snap shot when the particle is steered to the right and the displacement amount exceeds the physical capability of the potential field. The virtual environment reacts by reacting with a force that is in the direction counter to the steering motion. The amount of force is proportional to the exceeding distance from the WE. The PHANTOM haptic device was used for the haptic output and steering. This was attached to a Windows NT with a Pentium processor and 256 MB of RAM. The descriptions of the graphical objects are as follows. The arrow direction and length shows the direction and magnitude of the reaction force applied to the haptic stylus. The slender rod with elliptical cross section denotes the dithered laser beam. Finally the closed loop shows the WE. Because the WE is too small, it was drawn with a scale factor of 100. The WE and its interior solution points are pre-computed before the execution. The bounding box region with range of -100 nm to 100 nm on each axis was used to compute the WE. The cached data depends upon the elapsed time and the initial velocity of the particle. For simplicity and the limitation of the computer memory, a constant cycle time of 0.2 µs was used. The initial velocity of the particle at each axis was varied from -80 nm/µs to 80 nm/µs in increments of 5.33 nm/µs. To introduce a viscous effect and also to narrow down the range of velocity of the particle to be considered, the velocity was reduced to one tenth of the preceding cycle at each cycle. Note that a more formal way to have done this would be to have modeled the viscous force as a function of a velocity in Eq. (6). This would have changed the dynamics described in Eq. (6) and Eq. (7). It was not done in this way for simplicity. The total processor time to compute this in a batch process using an SGI Onyx2 with four processors and 1 GB of main memory took 71 487.99 s (about 20 h).

Fig. 8. Working envelope (a) 20×20 grid points used for computing the initial guess point (b) 80×80 grid points used for computing the initial guess point (c) 320×320 grid points used for computing the initial guess point.

Fig. 9. (a) Initial screen of the virtual environment (b) the reaction force experienced by over running the particle.

4. Discussion

The discreteness of the WE introduced some jaggedness to the haptic sensation. Recall that the method of computing the WE used grid points and the smoothness of the WE is directly proportional to the grid distance. The smaller the grid distance, the smoother is the WE. However, the use efforce feedback for constraining the steering motion of the operator that exceeded the laser capability did prove to be a useful method. The reaction force from the haptic device made it natural for the operator to slow down. In one experiment with the haptic feedback off, the distance between the operator cursor and the particle reached up to 3 µm. In another experiment with the haptic feedback on, the distance between the operator cursor and the particle was kept under 0.3 µm. In order to use this VE in tele-operation, much research needs to be done to understand the interaction between the laser beam and the particle and also between the particle and the surrounding fluid. Validating the model requires deriving a more detailed mathematical model for the potential along with extensive experiments. Also the viscous effect of the surrounding fluid needs to be taken care of. When the particle is not a simple sphere, as used in this article, the orientation of the particle needs to be considered when computing the WE. This means two more variables to represent the orientation need to be added to the initial three variables (elapsed time, initial velocity in the x and y directions).

When the problem domain is expanded to three dimensions, it is presumed that most approaches used in two dimensions could be used without major changes, with several exceptions. The method for computing the initial guess point for Eq. (7) cannot be extended to three dimensions and needs a new numerical method. Also the memory requirement can be substantial in three dimensions. The cached data for the WE uses 28 MB. An Added dimension (z axis) would easily use 1 GB. For a nonspherical geometry one would also need to add two more dimensions for the orientation. Obviously, a more memory efficient representation of the WE needs to be studied.

5. Conclusion

Optical tweezers are a new device that has great potential for manipulating nanoscale objects. The nonintrusive nature of the device enables nondestructive manipulation. To better understand the underlying physical nature of the device, a virtual environment that can simulate the physics of the laser beam and particle interactions has been designed and implemented. The conversion of the system of differential equations to a system of difference equations enabled the precise control of the transient position of the particle. By putting the particle in the maximum force region, the maximum movement with the given laser condition was achieved. A new concept called WE (Working Envelope) and a complex numerical technique to compute it was also presented. It was shown that WE can prevent the user from attempting physically impossible movement of the particle.

The simulation environment could have more value when it is used in a tele-operation environment. One would have finer position control without ever losing the grip. This requires a more sophisticated potential model and experiments.

Sidebar

[J. Res. Natl. Inst. Stand. Technol. 108, 275-287 (2003)]

Sidebar

Available online: http://www.nist.gov/jres

Footnote

1 Certain commercial equipment, instruments, or materials are identified in this paper to foster understanding. Such identification does not imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are necessarily the best available for the purpose.

References

6. References

[1] National Nanotechnology Initiative: The initiative and its implementation plan, National Science and Technology Council, (2000).

[2] M. Guthold, M. R. Falvo, W. G. Matthews, S. Paulson, S. Washburn, D. S. R. Erie, F. P. Brooks, and R. M. Taylor, Controlled manipulation of molecular samples with the nanoManipulator, IEEE/ASME Trans. Mechatronics 5 (2), 189-198 (2000).

[3] D. M. Eigler and E. K. Schweizer, Positioning single atoms with a scanning tunnelling microscope, Nature 344, 524-536 (1990).

[4] Z. Ulanowski, Optical Tweezers-principles and applications, Proc. Royal Microsc. Soc. 36 (1), 7-14 (2001).

[5] C. Tischer, S. Altmann, S. Fisinger, J. K. H. Horber, E. H. K. Stelzer, and E.-E. Florin, Three-dimensional thermal noise imaging, Appl. Phys. Eett. 79 (23), 3878-3880 (2001).

[6] R. E. Hollis, S. Salcudean, and D. W. Abraham, Toward a tele-nanorobotic manipulation system with atomic scale force feedback and motion resolution, in Proc. IEEE Symposium on Micro Electro Mechanical Systems, Napa Valley, California (1990) pp. 115-119.

[7] M. Sitti and H. Hashimoto, Tcleoperated nano scale object manipulation, Recent advances on mechatronics, Springer Verlag (1999) pp. 322-335.

[8] A. Ashkin, Forces of a single-beam gradient laser trap on a dielectric sphere in the ray optics regime, Biophys. J. 61, 569-582 (1992).

[9] J. H. Ginsberg, Advanced engineering dynamics, Cambridge University Press (1995).

[10] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical recipes in C: The art of scientific computing, Cambridge University Press (1993).

[11] W. E. Lorensen and H. E. Cline, Marching cubes: a high resolution 3D surface construction algorithm, in: SIGGRAPH Computer Graphics Proceeding (1987) pp. 163-169.

[12] T. Feldman, Generating isovalue contours from a pixmap, Graphics gems III, Academic Press (1992) pp. 29-33.