Content area
Objective: To examine changes in biomechanical and motor control associated with a force-feedback computer mouse. Background: Haptic computer mice can improve the movement times for point-and-click tasks; however, changes in upper extremity biomechanics and motor control are unknown. Method: Eighteen people (ages 22-37 years) performed a point-and-click task 80 times using a force-feedback computer mouse across three different conditions: (a) no force feedback, emulating a conventional mouse; (b) a single attractive force field at the desired target that pulls the mouse to the center of the target; and (c) an attractive force field at the desired target as well as others between the two possible targets, distracting the user from the intended target. Cursor kinematics, wrist posture, and electromyographic (EMG) forearm muscle activity were recorded. Results: The point-and-click movements were 30% faster with the addition of a single force field and 3% faster with the addition of multiple force fields. The Fitts' law index of performance metrics improved from 2.9 bits/response to 4.1 bits/response for multiple attractive fields and to 6.0 bits/response for a single force field. For the distracting force fields, the cursor maximum velocities were over 50% faster. EMG amplitude values were largest for the distracting force fields. These data suggest that the operator uses increased muscle activity to accelerate the mouse through the distracting force fields. Conclusion: When implementing attractive haptic force fields, one needs to consider how to reduce these observed effects of potential distracting force fields. Application: Applications include human-computer interface design for pointing devices extensively used for the graphical user interface. [PUBLICATION ABSTRACT]
Full text
Objective: To examine changes in biomechanical and motor control associated with a force-feedback computer mouse. Background: Haptic computer mice can improve the movement times for point-and-click tasks; however, changes in upper extremity biomechanics and motor control are unknown. Method: Eighteen people (ages 22-37 years) performed a point-and-click task 80 times using a force-feedback computer mouse across three different conditions: (a) no force feedback, emulating a conventional mouse; (b) a single attractive force field at the desired target that pulls the mouse to the center of the target; and (c) an attractive force field at the desired target as well as others between the two possible targets, distracting the user from the intended target. Cursor kinematics, wrist posture, and electromyographic (EMG) forearm muscle activity were recorded. Results: The point-and-click movements were 30% faster with the addition of a single force field and 3% faster with the addition of multiple force fields. The Fitts' law index of performance metrics improved from 2.9 bits/response to 4.1 bits/response for multiple attractive fields and to 6.0 bits/response for a single force field. For the distracting force fields, the cursor maximum velocities were over 50% faster. EMG amplitude values were largest for the distracting force fields. These data suggest that the operator uses increased muscle activity to accelerate the mouse through the distracting force fields. Conclusion: When implementing attractive haptic force fields, one needs to consider how to reduce these observed effects of potential distracting force fields. Application: Applications include human-computer interface design for pointing devices extensively used for the graphical user interface.
INTRODUCTION
Various computer input devices utilize tactile feedback in order to improve human-computer interface (HCI) performance. The breakaway force-displacement characteristics of key switch design (Human Factors and Ergonomics Society, 2002) stimulate mechanoreceptors of the fingertip and finger, providing the tactile sensation of a mechanical switch. Similarly, the buttons on pointing devices provide a sensation of another mechanical switch, and in the scroll wheel the detents provide spatial information as it turns. Even the texture of the mouse surface provides confidence in holding and controlling the mouse (Mcloone, 2001). Unfortunately, most pointing devices do not provide tactile information about the location of the cursor within the graphical user interface; rather, a user relies heavily on visual and proprioceptive feedback. Active haptic devices, which provide tactile cues through programmed software and electromechanical actuators, can provide this missing link, thus improving the HCI performance metrics.
Force-feedback or haptic devices provide tactile cues through the display of forces using motors and linkages with the aim of increasing human operator performance in both virtual and telerobotic environments (Rosenberg, 1994; Sheridan, 1992). For telerobotic applications such as underwater manipulators and other robots that operate in hazardous environments, for which a human operates the remote manipulator through a kinematic replica of the manipulator in a master-slave configuration, the forces encountered by the remote manipulator (the slave) are measured through a series of sensors located on the manipulator. These forces are then applied in a scaled-down version to the human through a haptic device contained within the master control operated by the human. The haptic device is in essence a second robot that through a series of actuators and linkages can provide force feedback in various forms to the human user. This source of sensory feedback aids the human controller in determining the actual state of the robot. In virtual environments such as flight simulators and surgical trainers, a model of the physical objects within the virtual environment displays tactile cues such as simple bumps, high stiffness for virtual walls, and vibrations denoting events such as crossing a virtual boundary, all via a haptic interface device (Okamura, Cutkosky, & Dennerlein, 2001; Tendick et al., 2000).
Several studies have considered the implementation of tactile feedback in computer pointing devices. Akamatsu and MacKenzie (1996) examined a multimodal mouse that varied friction during movements and displayed a vibration when crossing boundaries of interest. Both Hasser and Goldenberg (1998) and Eberhardt, Neverov, West, and Sanders (1997) examined the effects of attractive force fields or attractive basins around target icons, which display a local force field around the target so as to pull the mouse and hence the cursor toward the center of the target icon, on the performance of a point-and-click task. For these studies, the time to complete a given task improved with the addition of tactile cues.
Dennerlein and Yang (2001) examined the perceived musculoskeletal effects associated with using these attractive force fields for a point-and-click task through use of a questionnaire. The authors reported increased comfort with a single attractive force field. A single attractive force field does not replicate a probable application, however, because on the computer desktop, for example, many possible icons or targets exist. Therefore, a realistic test for performance enhancements needs to include attractive basins within the field of the cursor movement other than that for the desired target. Dennerlein and Yang (2001 ) observed that when these distracting force fields were present, the discomfort level returned to levels associated with no force fields, yet the performance improvements remained. This study raised a question concerning how force-feedback technology affects the loading of musculoskeletal tissues and, hence, risk factors for musculoskeletal disorders of the upper extremity (Armstrong, Martin, Franzblau, Rempel, & Johnson, 1995; Silverstein, Fine, & Armstrong, 1986). Dennerlein and Yang (2001) hypothesized that the force-feedback technology changes the task constraints for the targeting task and that the distracting fields inhibit movement to the target. However, they did not measure any biomechanical parameters during their experiments that would elicit the different motor control patterns attributable to the change in task requirements (Huang, Andersson, & Thorstensson, 2003).
For these studies of point-and-click movements with attractive force fields, the only measure of performance increase has been the time to complete a task; the accuracy or precision of the task has not been explored. Fitts' law (Fitts, 1954), used to quantify the difficulty and performance of a motor task, is based on a speed versus accuracy trade-off - that is, more precise movement requires more time. Douglas, Kirkpatrick, and MacKenzie (1999) proposed that in addition to the performance measures of time and errors, experimenters should examine the effective precision of movements through observing the distribution of the target locations - that is, the click for the point-and-click task. Based on this work they proposed a modified index of performance for a targeting task that uses the actual measure of the movement precision rather than the accuracy required by the task.
The approach and goal of this study are to examine biomechanical and motor control parameters, forearm muscle activity, wrist joint kinematics, and cursor kinematics and to assess task performance and the musculoskeletal effects of force feedback during a simple point-and-click task. The experimental design compares three force-field conditions - no forces, emulating a conventional mouse; a single attractive force field around the desired target; and multiple attractive force fields within the movement space, distracting the user from the intended target. The experiments test the hypothesis that muscle activity levels and patterns as well as cursor and wrist kinematics vary across the three force-field conditions. We expect the highest muscle activity levels to occur with the presence of multiple attractive force fields. We also expect performance metrics, including movement accuracy, to be the best with the presence of a single attractive force field around the target.
MATERIALS AND METHODS
Participants and the Pointing Device Task
Eighteen adults (9 men and 9 women) with an average age of 28 years (SD = 5 years) participated in the study. The participants were recruited from the university campus and through a temporary employment agency. All protocols and consent forms were approved by the Harvard School of Public Health's Human Subjects Committee. None of the participants had prior experience with force-feedback devices and were free of upper extremity musculoskeletal disorder symptoms. All participants used their right hand for the experiment, which, in every case, was also the hand normally used by them to operate the computer mouse. The chair, table, and monitor height were adjusted for each participant in accordance with the BSR/HFES 100 standard (Human Factors and Ergonomics Society, 2002). The mouse was placed at the edge of the table and to the right of the keyboard. No forearm supports were provided.
Study participants performed the same point-and-click task for each condition. The task consisted of pointing and clicking on two alternating 30-pixel diameter circular targets. The two circular targets were aligned vertically on the screen (1024 by 768 pixels) and were separated horizontally by 500 pixels. The task software, written in C++, would highlight one target until the cursor was within the target and a button click was successfully registered. The other target was then highlighted. This alternating pattern was completed 40 cycles for 80 clicks. This pattern was chosen to elicit repeatable patterns in kinematics and muscle activity. Participants were instructed to click anywhere within the target as quickly as possible and then to move on to the next target. The participant was not allowed to proceed to the next target until the highlighted target was successfully obtained. Clicks outside of the target area were not registered; therefore, missed targets were implicitly measured through increased time to obtain the desired target. The software recorded the time and cursor location of each successful click. For 11 of the participants, the software also recorded the location of the cursor every 8 ms, providing the cursor trajectory. Digital differentiation of the cursor location estimated the cursor velocity.
Apparatus
The pointing device tasks were completed using a WingMan force-feedback mouse (Logitech, Freemont, CA). The mouse was connected to electromagnetic actuators through a two-degree-of-freedom linkage system located within the mouse platform. Through these actuators and linkage the mouse can apply forces up to 0.9 N in any direction parallel to the table top. The linkage also physically grounds the mouse such that it cannot be lifted from the table for indexing the mouse and cursor; however, its movement is easily scaled to the size of the screen. For every millimeter the mouse moved, the cursor would move approximately 30 pixels on the screen.
Ten of the participants completed the tasks with surface electromyographic (EMG) electrodes monitoring the muscle activity of the four forearm muscles that articulate the wrist: the flexor carpi radialis (FCR), the flexor carpi ulnaris (FCU), the extensor carpi ulnaris (ECU), and the extensor carpi radialis (ECR). The bipolar surface electrodes (DE-2.1 single differential electrode, Delsys, Boston, MA) were placed on top of the muscle bellies in accordance with Perotto (1994). Placements were validated through palpation and signal response to isometric test contractions. After amplification (Bagnoli-eight amplifier, Delsys, Boston, MA, with a bandwidth of 20 to 450 Hz), the EMG signals were recorded onto a personal computer at 1000 samples/s. A root mean square signal was calculated over a 0.2-s moving window. To normalize the results across participants, we collected three 5-s maximum voluntary isometric contractions (MVCs) for each muscle. Participants rested for 1 min between contractions, and the maximum value obtained during the three contractions was used as the MVC EMG value.
These same 10 participants wore a two-channel, glove-based electrogoniometry system (Wrist-system, Greenleaf Medical, Palo Alto, CA) that measured wrist posture during the tasks. The system measured wrist flexion and extension and ulnar and radial wrist deviation. The system has a resolution of 0.1°, 2° accuracy over a ±90° range, and was calibrated using a wrist jig in accordance with the methods described in Jonsson and Johnson (2001). Postures were recorded continuously by a data logger at 20 samples/s during the tasks. Digital differentiation provided estimates of the wrist joint velocities.
Experimental Design
The point-and-click tasks were completed over three force-feedback conditions (Figure 1): no force fields, emulating a conventional mouse (Figure 1a); a single attractive force field around the desired target (Figure 1b), and multiple attractive force fields between the two possible targets, distracting the user (Figure 1c). These three conditions provided the different values for the study's main independent variable. For the first condition all force fields were disabled and the mouse behaved as a conventional mouse with no force feedback.
Figure 1. The point-and-click task consisted of clicking on two alternating targets 30 pixels in diameter. The next target would be highlighted, as shown for the left target here. The task was completed for three conditions: (a) conventional mouse with no force fields, (b) a single attractive force field around the desired target (depicted here on the left target), and (c) multiple force fields located between the targets, distracting the user from the desired target. The dotted lines indicate that the distracting haptic targets were not visually displayed to the user.
The second condition was a single attractive force field that encompassed a circular area centered within and with twice the diameter of the visual target (Figure 2). Outside of this area no forces were activated. Within the area, the forces acting on the mouse were pointed toward the center of the visual target. Moving inward in this area toward the center of the target, the magnitude of the forces first increased then leveled off at 0.9 N as the cursor crossed the visual border of the target and then decreased back to no force at one half the radius of the visual target. When the cursor was within this small inner circle, no forces were applied by the mouse. Once a successful click was recorded within the target, the force field was disabled, providing no resistance to moving to the next target.
In the third condition an attractive basin was placed around the desired target, but also 12 attractive basins of the same shape and size were placed between the last target and the desired target (Figure 1c). This condition emulated having multiple targets within the movement space, similar to having multiple icons on a screen. These fields are called distracting fields because they attempt to attract the cursor to their centers, thereby distracting the user away from the desired target. The fields consisted of three rows of four attracting force fields separated by 100 pixels center to center. The middle row was vertically aligned with the two intended targets. No visual cues identifying the position of these distracting fields were provided to the participants. This pattern was selected to make it difficult for the participants to avoid the distracting force fields, which had occurred with the crosshair configuration of Dennerlein and Yang (2001). All participants practiced each condition before data collection. The order of the three conditions was randomized for each participant.
Figure 2. The attractive force field utilized for the two force-field conditions. The force field acts locally around the desired target in such a way as to pull the mouse into the center of the target. (From Dennerlein & Yang, 2001, p. 281.)
Data Analysis
The dependent variables can be grouped into three categories: cursor movement performance and trajectories, muscle activity, and joint kinematics. All these parameters were calculated for the last 40 clicks of each condition to remove any variations attributable to within-condition training. The performance measures for the point-and-click tasks were movement times, the effective width of the target, and the Fitts' law index of performance. Movement times were calculated from the cursor data recorded by the task software and were defined as the time between the successful clicks.
Index of performance was calculated using Fitts' law. Fitts (1954) proposed that movement time (MT) is related to specific task parameters, mainly the distance between the targets (A) and the accuracy or width of the targets (W), MT = a + b log^sub 2^ (A/W + c), in which the coefficients of a and b have units of time and c is dimensionless. Fitts (1954) defined a version of the logarithmic portion of the equation as the index of difficulty (ID), ID = -log^sub 2^(W/2A), which has the unit of bits per response. Given the index of difficulty Fitts (1954) defined the index of performance for a given individual and task as IP = ID/MT, which has the unit of bits per second.
This index of performance is based on the index of difficulty using the task parameters such as the distance between the targets (A) and the width of the target (W). Douglas et al. (1999) proposed that the index of performance take into account the width of targets achieved by the user rather than the task requirements. They proposed that the actual width or variation of the accuracy of the task of an individual is reflected in the distribution of the specific location of the clicks. Therefore, we calculated the effective target width as W^sub e^ = 4.133 SD, in which SD is the standard deviation of the cursor location about the mean (Douglas et al., 1999). From this new effective width we calculated an effective index of difficulty (ID^sub e^) and performance (IP^sub e^). Other cursor trajectory parameters included maximum velocity and the distance traveled by the cursor.
Summary statistics were calculated for the forearm muscle activity and the wrist joint kinematics and included the mean and standard deviation as well as the 10th, 50th, and 90th percentiles of signal amplitude, which provide a description of the range of the parameters during the experimental conditions. For EMG values the 10th percentile represents the static muscle load, whereas the 50th and 90th percentiles represent the more dynamic muscle activities associated with a task (B. Jonsson, 1988). For the postural measures the difference between the 90th and the 10th percentile provides a measure of the range of motion and the 50th percentile provides a measure of the median postural.
In addition to these summary statistics, EMG, wrist movement, and cursor movement data were separated and grouped into leftward (radial deviation) and rightward (ulnar deviation) movements. The summary statistics were calculated for data grouped into each direction and then averaged within direction, condition, and participant. To examine movement and activity patterns, we averaged the cursor trajectory and EMG data for each condition and direction within and then across participants. For the presentation of these patterns, these data were normalized with respect to time utilizing the average movement time for each condition and direction.
Statistical Analysis
Differences in dependent parameters (performance measures; EMG and postural 10th, 50th, and 90th percentile values; and cursor movements) among the three conditions (conventional, single force field, and distracting force fields) were tested individually using a repeated measures analysis of variance in JMP statistical software (SAS Institute, Cary, NC; http://www.JMPdiscovery. com/). For the repeated measures the independent variables were force-feedback condition with fixed effects and subject with random effects. For the parsed data and the calculated set of dependent parameters associated with the movement direction, direction was added to the independent variables. Significance was noted for probability of a false positive being less than 5% (i.e., alpha = .05).
RESULTS
Task performance, wrist and cursor kinematics, and wrist muscle activity differed across the three force-feedback conditions given the same point-and-click task. Performance metrics improved when attractive force fields around the desired target were present but not so when multiple attractive force fields were present (Table 1). Movement times for the individual point-and-clicks improved 30% (p < .001), from 1.12 to 0.78 s, when a single force field surrounding the desired target was present. Movement times with multiple attractive force fields were slightly improved (3%) compared with the condition that had no force field, but not significantly.
The precision of movement as defined by the effective width (Douglas et al., 1999) increased with the presence of attractive force fields (Figure 3). The effective target widths decreased 54%, from 28 pixels to 12 and 13 pixels for the single and multiple force-field conditions, respectively (Figure 3). As a result the effective index of performance increased 107% and 42% for the single and multiple force-fields conditions, respectively.
The cursor traveled faster and farther with the presence of multiple force fields than with the conventional and single force-field conditions (Table 1, Figure 4). Maximum cursor velocities increased 54% from 4.1 pixels/ms for the conventional force-field condition to 6.3 pixels/ms for the multiple force-field condition. The distance traveled by the cursor per click increased by 38% from 517 pixels for the conventional force-field condition to 715 pixels for the multiple force-field conditions. With the single force-field condition the cursor spent little time in and around the target area. For the conventional mouse approximately half of the movement time was spent with low velocity near and within the target area, whereas the time near the target areas for the single and multiple force-field conditions were less than a quarter of the movement time (Figure 4).
Figure 3. Cursor location at the point in time of the click across all three conditions (a: no force fields, b: single force field, c: multiple force fields) for one participant. The effective width decreases drastically for the force-feedback conditions, which increases task performance. The visual target's boundaries are depicted by the solid circle and are 30 pixels in diameter. The dotted-line circle represents the inner boundary of the attractive force field.
Figure 4. Temporal pattern of rightward horizontal cursor movements averaged across participants for the three force-field conditions. The circles denote the end of the movement for the three different conditions. The horizontal width of the target of 30 pixels is denoted. When multiple force fields are present, the cursor moves the fastest toward the target, often overshooting the final position.
Wrist radial and ulnar deviation followed patterns similar to those of the cursor kinematics (Table 2). Wrist velocities were 25% faster with multiple force fields, increasing from 21°/s to 28°/s. There were no postural differences across the force-field conditions (p ≥ . 13). The range of motion of ulnar deviation with distracting force fields was 2° larger than when no force fields were present (p = .02).
Forearm muscle activity increased with multiple force fields (Figure 5). The FCR and ECU EMG signals showed the most significant increases in muscle activity across the three force field scenarios. These two muscles also illustrated different activity for left and right movements, with the FCR showing more activity for leftward (radial deviation) movements (10% vs. 9% MVC) and the ECU showing more activity for rightward (ulnar deviation) movements (38% vs. 31% MVC). The FCU and the ECR illustrated increasing EMG activity levels with the increasing number of force fields present, but these trends were not statistically significant (FCU: p = .09; ECR: p = .07).
The averaged patterns of muscle activity did not change greatly across conditions (Figure 6). The FCR decreased activity for rightward (ulnar deviation) movements, but no consistent pattern was observed for leftward movements. The FCU activity decreased quickly for leftward movements (radial deviations) but was constant for rightward (ulnar deviations) movements. The ECU had an alternating pattern of activity for left and right movements, increasing quickly for rightward (ulnar deviation) movements and decreasing quickly for leftward (radial deviation) movements. The ECR increased activity for rightward movements and then decreased prior to the end of the movement. For leftward movements the ECR had no consistent pattern across participants and the three conditions.
![View Image - Figure 5. Average EMG amplitude distribution for the 10th ([open diamond]), 50th ([white square]), and 90th ([white triangle up]) percentile values across the conditions. Amplitude increases with the presence of multiple force fields. Brackets represent significant differences across the conditions. For the ECU muscle only the median value showed significance, whereas for the FCR the 10th, 50th, and 90th percentiles all showed differences across the conditions (*p < .05, **p < .001). FCR = flexor carpi radialis, FCU = flexor carpi ulnaris, ECU = extensor carpi ulnaris, ECR = extensor carpi radialis.](https://media.proquest.com/media/hms/THUM/16/ID0zS?_a=ChgyMDI2MDEyMTIxNTExMDE1MjoyODg0NzYSBzE0MTM1MzcaCk9ORV9TRUFSQ0giDjIxNi43My4yMTYuMTE0KgU0ODQ4MTIJMjE2NDU5OTYxOgtJbmxpbmVJbWFnZUIBMFIGT25saW5lWgNJTEliBFRIVU1qCjIwMDYvMDQvMDFyCjIwMDYvMDYvMzB6AIIBPFAtMTAwNzA2Ny0yMDg2MTEtQ1VTVE9NRVItMTAwMDAzNDgvMTAwMDAyNTUvMTAwMDAxNTUtNjY5MzE2NJIBBk9ubGluZcoBDlNlYXJjaFJldHJpZXZl0gESU2Nob2xhcmx5IEpvdXJuYWxzmgIHUHJlUGFpZKoCLE9TOkVNUy1UZXh0UGx1c0dyYXBoaWNzQ29udGFpbmVyLXByZXBhcmVCb2R5ygIPRmVhdHVyZXxBcnRpY2xl4gIA8gIA%2BgIBToIDA1dlYooDHENJRDoyMDI2MDEyMTIxNTEwMDYwOTozMzAzNTQ%3D&_s=UAQ%2F4oJsjW8Sh1deMdBVKYTgVdI%3D "View Image - Figure 5. Average EMG amplitude distribution for the 10th ([open diamond]), 50th ([white square]), and 90th ([white triangle up]) percentile values across the conditions. Amplitude increases with the presence of multiple force fields. Brackets represent significant differences across the conditions. For the ECU muscle only the median value showed significance, whereas for the FCR the 10th, 50th, and 90th percentiles all showed differences across the conditions (*p < .05, **p < .001). FCR = flexor carpi radialis, FCU = flexor carpi ulnaris, ECU = extensor carpi ulnaris, ECR = extensor carpi radialis.")
Figure 5. Average EMG amplitude distribution for the 10th ([open diamond]), 50th ([white square]), and 90th ([white triangle up]) percentile values across the conditions. Amplitude increases with the presence of multiple force fields. Brackets represent significant differences across the conditions. For the ECU muscle only the median value showed significance, whereas for the FCR the 10th, 50th, and 90th percentiles all showed differences across the conditions (*p < .05, **p < .001). FCR = flexor carpi radialis, FCU = flexor carpi ulnaris, ECU = extensor carpi ulnaris, ECR = extensor carpi radialis.
DISCUSSION
These results illustrate that task performance increased with the presence of force fields; however, these benefits diminished when multiple force fields were present and came with the cost of extra musculoskeletal effort to overcome these distracting force fields. The reduction in the effective target widths suggests that the force fields, in both the single and the multiple force-field conditions, increased the precision of the movements, even though the instructions to the participants were simply to click within the visual target area. Because the multiple force fields inhibited the movement to reach the final target, participants increased cursor velocity and muscle activity during the acceleration and deceleration component to overcome the effects of these distracting force fields. The large increase in muscle activity may indicate that designers need to be creative in implementing multiple force fields within the desktop environment.
Figure 6. Average EMG patterns for left (radial deviation) and right (ulnar deviation) mouse movements. The data are averaged across participants (N = 10) and normalized with respect to the average time. Some patterns vary with the direction of the movement, and the amplitude increases with the presence of distracting force fields. FCR = flexor carpi radialis, FCU = flexor carpi ulnaris, ECU = extensor carpi ulnaris, ECR = extensor carpi radialis.
The cursor kinematics illustrate that the attractive force fields around the desired target increase user performance through two effects - decreasing movement time and increasing movement precision - both through providing a physical constraint to the system (Table 1 and Figure 3). The mouse/cursor movement consists of three phases. The cursor first accelerates toward the target and then quickly decelerates as it approaches the target. For the third phase the cursor position is finely tuned to align the cursor with the target. With no force fields present, the cursor then spends approximately half of the movement cycle time in this final alignment phase, whereas in the other two conditions less than a quarter of the cycle time is spent in this phase (Figure 4). With the presence of the force field, the mouse and hence the cursor were pulled into the center of the target. As discussed in Dennerlein and Yang (2001), the force field is allowed by the user to take over the movement of the cursor during this final alignment period, resulting in faster acquisition of the target than would be the case if only fine motor control were used.
The second effect is the improved precision of the movement. Because the target's attractive force field has a dead band (in which no force is applied) within the center half of the radius of the visual target, the effective width decreases to that of this inner circle (Figure 3). The attractive force field aligns the cursor into a smaller target, allowing for more precise movements. Hence, the force field improves the capabilities of the human-machine interface by guiding the tool into the smaller target. This improved precision is observed with both the single and the distracting force-field conditions. These two effects are not mutually exclusive in this case; however, if the interface had been designed such that the force field existed only outside of the target, then the improvement in movement precision may not have been observed.
Two dominant ideas exist about the benefits of force-feedback technology: One is that it provides physical constraints to the system, and one is that the technology provides information to the end user in completing the task (Wagner, Stylopoulos, Jackson, & Howe, in press). The data here suggest that the attractive force fields provide physical constraints to the system through essentially reducing the effective width of the target. The reduction increases the precision of the user without any cognitive response on the part of the user, given that the benefits are derived on a time scale shorter than that of any possible cognitive response.
Several factors suggest that the multiple force fields affect the movement to, and not the alignment with, the desired target. First, the effective width for the target is similar to that with a single force field (Figure 3), indicating that the precision of the movements during the final alignments are similar. Second, the path of the cursor is longest with the distracting force fields, with the additional distance occurring during movement to the desired target. This may be a result of an attempt to avoid the distracting force fields, though this would be difficult because the space between the desired targets was essentially covered with distracting fields, unlike the conditions tested in Dennerlein and Yang (2001). It may also be reflecting the effects of the cursor and the mouse getting trapped into the other distracting attractive force fields. The reaction time to realize and then correct adds to the movement time.
Third, the cursor accelerated to higher maximum velocities during the movements with multiple force fields. This suggests a strategy of overcoming these distracting force fields with momentum. The higher muscle activities throughout the movement stage suggest that the wrist and forearm are creating and/or are subjected to these higher accelerations. The ECU also shows extra activity during the beginning portion of the rightward movements (ulnar deviation) with the presence of multiple force fields (Figure 6).
The addition of the cursor kinematics and muscle EMG parameters results support the suggestions of Dennerlein and Yang (2001) that users adapt a momentum strategy to "plow" through the multiple distracting force fields. Unlike the previous study, however, here the results did not show a large reduction in movement time with the multiple force fields. Differences in the two studies lie in the design of the task. In the previous study, multiple force fields were placed in a crosshair configuration and around the other visual targets with varying movement distances. Here the fixed-distance task had three rows of multiple force fields between the targets, providing many more obstacles along the paths of the movements. Therefore, the amount of performance gain with multiple force fields depends on their layout and location relative to the intended movement and desired targets.
The task of clicking between two alternating targets separated horizontally by a fixed distance was chosen to elicit repeatable patterns in muscle activity and kinematics. The kinematics showed nice, repeatable patterns. For the muscle activity, however, such distinctive patterns were not very clear. For the most part the task required constant activation of the forearm muscles. Some general patterns were observed across participants. The FCR and the ECU, with the FCU to some extent, acted as an agonist and antagonist pair for ulnar deviation movements (Figure 6). As the ECU increased activity, the FCR decreased activity during the beginning portions of the rightward movements (ulnar deviation). The ECU'S efforts are primarily that of ulnar deviation (Buchannan, Moniz, Dewald, & Rymer, 1993). During the initial portions of the leftward movements, both the FCU and the ECU decreased activity in order not to inhibit the leftward movements (radial deviation). The ECR activity increased during the rightward movement just after the beginning of the movement and then decreased just before the end. The ECR may play a role in decreasing the force on the mouse during the rightward movements or there may be cross talk with the finger extensors, which could be preventing the fingers from inadvertently activating the button.
Of course, other muscles are involved with the control of mouse movement, such as the upper arm, shoulder, and finger muscles. The activity of the muscles that articulate the wrist are most likely associated with the larger movements of the cursor, whereas much of the alignment tasks may be completed by the muscles associated with the fingers and thumb. Furthermore, there was a small, but not significant, increase of muscle activity between the conventional mouse and the single attractive force field in three of the four muscles (Figure 5). The opposite trend was expected. The increase may be an artifact from the 10th, 50th, and 90th percentile values being calculated over a smaller window and the wrist muscles being active for a larger percentage of the time within that window. If it is true that these muscles are associated with the larger movements, a larger percentage of time was spent during the gross movements for the single force-field condition.
Different motor control strategies may be utilized to complete the mousing task. For example, the left and right movements of the mouse can be achieved with radial and ulnar wrist motions or they can be achieved with internal and external rotations of the shoulder. They can even be achieved through coordinated movement of the fingers or some combination of all three approaches. The amount of wrist radial and ulnar deviation movement varied across participants from a single degree to more than 10°, suggesting that different movement strategies were indeed utilized across participants. Motor control of multiple joints can vary greatly between participants because of the redundant nature of the musculoskeletal system. Most basic motor control studies of the wrist have examined specific single joint movements (e.g., Lehman & Calhoun, 1990). The differences may limit the ability to see specific activity patterns across all participants.
Finally, these results suggest that implementing attractive force fields with multiple basins presented within the same graphical user interface, such as the desktop, will be problematic. Although performance may be dependent upon the specific configuration, subjective ratings (as reported in Dennerlein and Yang, 2001) and the increase in kinematic and muscle activity parameters indicate that the disadvantages of multiple fields outweigh the benefits. Therefore, implementations need to be creative in looking for solutions to minimize the number of attractive fields within the field of movement. Of course, going to a single field is unreasonable. However, if the software that controls the force fields turns on only a few attractive basins, then perhaps the benefits of force feedback observed with these types of force fields can be added to the desktop virtual environment. Nonetheless, the kinematic and EMG data presented here provide designers with insight into possible motor control mechanisms adapted by users of haptic interface designs as well as possible quantitative methods to evaluate usability.
CONCLUSIONS
The haptic force-feedback mouse can increase user performance through the addition of attractive force fields for a point-and-click task. These force fields change the physical constraints on the fine alignment portion of the task. When multiple fields emulating several potential targets on a computer desktop graphical user interface were added in the form of distracting force fields, extra effort, as observed through muscle activity and wrist and cursor kinematics, was needed to overcome these constraints and reach the desired target. As a result, the performance enhancements of attractive force fields may come with a cost. Therefore, when implementing attractive force fields one needs to consider how to reduce potential distracting force fields for the benefits to outweigh their musculoskeletal costs.
ACKNOWLEDGMENTS
This study was funded in part by the Liberty Mutual Harvard Program in Occupational Safety and Health and the National Institute for Occupational Safety and Health Grant 1R010H03997-01. Equipment was donated by Immersion Corporation, San Jose, CA. The authors also thank the DiMarino family, Theodore Becker, David Martin, Chris Hasser, and Chris Wagner for their contributions.
REFERENCES
Akamatsu, M., & MacKenzie, I. S. (1996). Movement characteristics using a mouse with tactile and force feedback. International Journal of Human-Computer Studies, 45, 483-493.
Armstrong, T. J., Martin, B. J., Franzblau, A., Rempel, D. M., & Johnson, P. W. ( 1995). Mouse input devices and work-related upper limb disorders. InA. Greico, G. Molteni, B. Piccoli, & E. Occhipinti (Eds.), Working with display units 94 (pp. 375-380). Amsterdam: Elsevier Science.
Buchannan, T. S., Moniz, M. 3., Dewald, J. P. A., & Rymer, W. Z. (1993). Estimation of muscle forces about the wrist joint during isometric tasks using an EMG coefficient method. Journal of Biomechanics, 26, 547-560.
Dennerlein, J. T., & Yang, M. C. (2001). Haptic force-feedback devices for the office computer: Performance and musculoskeletal loading issues. Human Factors, 43, 278-286.
Douglas, S. A., Kirkpatrick, A. E., & MacKenzie, I. S. (1999). Testing pointing device performance and user assessment with the ISO 9241, Part 9 standard. In Proceedings of the ACM Conference on Human Factors in Computing Systems- CHI '99 (pp. 215-222). New York: Association for Computing Machinery.
Eberhardt, S., Neverov, M., West, T., & Sanders, C. ( 1997 November). Force reflection for WIMPs: A button acquisition experiment. Presented at the Sixth Annual Symposium on Haptic Interfaces, International Mechanical Engineering Congress and Exposition, Dallas Texas.
Fitts, P. M. (1954). The information capacity of human motor systems in controlling the amplitude of a movement. Journal of Experimental Psychology, 47, 381-391.
Hasser, C., & Goldenberg, A. (1998 November). User performance in a GUI pointing task with a low-coat force-feedback computer mouse. Presented at the Seventh Annual Symposium on Haptic Interfaces, International Mechanical Engineering Congress and Exposition, Anaheim, CA.
Huang, Q. M., Andersson, E. A., & Thorstensson, A. (2003). Specific phase related patterns of trunk muscle activation during lateral lifting and lowering. Acta Physiologica Scandinavica, 178, 41-50.
Human Factors and Ergonomics Society. (2002). Human Factors Engineering of Computer Workstations (BSR/HFES 100). Santa Monica, CA: Author.
Jonsson, B. (1988). The static load components in muscle work. European Journal of Applied Physiology, 57, 305-310.
Jonsson, P., & Johnson, P. (2001). Comparison of measurement accuracy between two types of wrist goniometer systems. Journal of Applied Ergonomics, 32, 599-607.
Lehman, S. L., & Calhoun, B. M. (1990). An identified model for human wrist movements. Experimental Brain Research, 81, 199-208.
Mcloone, H. (2001). Touchable objects. In Proceedings of the International Conference on Affective Human Factors Design (pp. 235-241). London: Asean Academic Press.
Okamura, A. T., Cutkosky, M. R., & Dennerlein, J. T. (2001). Reality-based models for vibration feedback in virtual environments. ASME/IEEE Transactions on Mechatronics, 6, 245-253.
Perotto, A. (1994). Anatomical guide for the electromyographer: The limbs and trunk (3rd ed.). Springfield, IL: Charles C. Thomas.
Rosenberg, L. (1994). Virtual fixtures. Unpublished Ph.D. dissertation, Stanford University, 1994.
Sheridan, T. B. (1992). Telerobotics, automation and human supervisory control. Cambridge, MA: MIT Press.
Silverstein, B. A., Fine, L. J., & Armstrong, T. J. (1986). Hand wrist cumulative trauma disorders in industry. British Journal of Industrial Medicine, 43, 779-784.
Tendick, F., Downes, M., Goktekin, T., Cavusoglu, M. C., Feygin, D., Wu, X., et al. (2000). A virtual environment testbed for training laparoscopic surgical skills. Presence, 9, 236-255.
Wagner, C. R., Stylopoulos, N., Jackson, P. G., & Howe, R. D. (in press). The benefit efforce feedback in surgery: Examination of blunt dissection. Presence: Teleoperators and Virtual Environments.
Jack T. Dennerlein and Maria-Helena J. DiMarino, Harvard School of Public Health, Boston, Massachusetts
Address correspondence to Jack T. Dennerlein, Harvard School of Public Health, 665 Huntington Ave., Boston, MA 02115; [email protected]. HUMAN FACTORS, Vol. 48, No. 1, Spring 2006, pp. 130-141. Copyright © 2006, Human Factors and Ergonomics Society. All rights reserved.
Jack T. Dennerlein is an associate professor of ergonomics and safety within the School of Public Health at Harvard University. He obtained his Ph.D. in mechanical engineering from the University of California, Berkeley, in 1996.
Maria-Helena J. DiMarino was a master's student in the Department of Environmental Health at the Harvard School of Public Health. A native of Brazil, she earned her B.S. in industrial engineering in 1994 from the University of Massachusetts, Amherst. As part of her degree requirements, Ms. DiMarino completed the experimental data collection and analysis for this study before her untimely death in May 2003.
Date received: March 4, 2004
Date accepted: December 5, 2004
Copyright Human Factors and Ergonomics Society Spring 2006