User Interfaces in the OR

Abstract

Health care professionals are increasingly using computers in the OR to view, update, document, and store patient information, including x-rays and other scans or laboratory results that could affect the course of surgery. Most people are familiar with a traditional mouse and keyboard method of using a computer; however, these computer accessories were not designed specically for use in the OR, and data suggest that pathogens can be transferred to and survive on keyboards.To address this problem, some health care professionals are turning to alternative computer methods such as tablets and smartphones, which have touch screens that can be cleaned more readily or which can be used while placed in a sterile plastic bag. Soon, they may even be able to use hands-free technology, such as Microsofts KinectTM for the Xbox 360 video game system.

TAPPING INTO TECHNOLOGY

User Interfaces in the OR

ZAC WIGGY, BA

Health care professionals are increasingly using computers in the OR to view, update, document, and store patient infor

mation, including x-rays and other scans or laboratory results that could affect the course of surgery. Computers also are being used to help the surgical team perform cutting-edge procedures such as robotic-assisted surgeries1 and intraoperative magnetic resonance imaging (iMRI). The Mayo Clinic, for example, uses iMRI and reports that use of iMRI during tumor removal surgery greatly reduces the risk of damaging other parts of the brain and helps conrm successful removal of the entire tumor.2

Most people are familiar with a traditional mouse and keyboard method of using a computer; however, these computer accessories were not designed specically for use in the OR, and data suggest that pathogens can be transferred to and survive on keyboards.3,4 To address this problem, some health care professionals are turning to alternative computer methods such as tablets5 and smartphones,6 which have touch screens that can be cleaned more readily or which can be used while placed in a sterile plastic bag.5,7 Soon, they may even be able to use hands-free technology, such as Microsofts KinectTM for the Xbox 360 video game system.

During a keynote address at Microsoft TechEd 2011 in Atlanta, GeorgiaMicrosofts annual conference for information technology specialistsMicrosoft Corporate Vice President, Server and Tools Business Robert Wahbe showed a video featuring a surgeon who controlled a computer screen that displayed patient information by using gestures with his hand in the air.8

The demonstration involved a personal computer (PC) that was attached to Microsofts Kinect (Figure 1), a sensor system designed to attach to the companys Xbox 360 video game system. Microsoft wanted to show Kinects potential, but the demonstration raises questions about how health care professionals in the OR will interact with computers in the future.

MICROSOFTS KINECT

Microsoft released the Kinect sensor system in November 2010. It sits above or below a television and consists of a pair of depth sensors to track users in three dimensions, a standard color digital camera, and four microphones.9 When the system is used for playing video games, these sensors track players motions as they jump, duck, and move their arms. The system does not require players to hold or use any sort of hand-held controllerinstead, the Kinect and the Xbox 360 use the information from the sensors to keep track of where a players arms, legs, and head are and interpret the players movements into the game world. For example, one Kinect game requires players to jump when the on-screen character approaches an obstacle. When the player jumps in the real world, the camera and depth sensors see the jump and make the character jump on screen.

The games available for Kinect differ in how they handle the input from these sensors, but they generally work on a few of the same principles, one of which is skeletal tracking. The system knows the general shape of a person and uses the data from its sensors to match what it is seeing to how a person is shaped, creating a virtual skeleton of the player. If it sees forward motion in

doi: 10.1016/j.aorn.2012.01.017 526 AORN Journal April 2012 Vol 95 No 4 AORN, Inc, 2012

TAPPING INTO TECHNOLOGY www.aornjournal.org

Figure 1. Microsoft Kinect. Photograph courtesy of Microsoft Corp, Redmond, WA.

the area where it expects a players left foot to be, it will interpret that as the players left foot moving forward and remember the new location of the left foot. It keeps track of where players are whenever they are in view, and it can track more than one player, using its depth sensors and what it knows about the shape of a person.10

Kinect also employs facial and voice recognition. Facial recognition allows Kinect to take pictures of users faces and compare them with past pictures, so that users can log on to the system and access saved game data just by letting the Kinect see them. The Kinects voice recognition is similar to that present in modern PCs and smartphones, allowing users to give simple voice commands, such as telling amovie to begin playing.10

The use of surgical masks could affect the Kinects ability to recognize faces and voices, however, which could prove to be a factor in adapting this technology to the OR.

THE KINECT EFFECT

Shortly after Kinects release, hackers were hooking the device to their computers and nding new uses for it beyond gaming. Microsoft has embraced these experiments and launched a campaign called The Kinect Effect.10

Microsoft gives several examples on its Kinect Effect web site of people using the Kinect for tasks other than simple gaming. For example, the Lakeside Center for Autism in Washington uses Kinects full-body play technology for therapy with autistic children, and the Royal Berkshire Hospital in the United Kingdom uses it to help rehabilitate patients who have had strokes.10

In a surgical context, Kinects ability to control on-screen items without users touching a controller translates to surgical staff members being able to manipulate computers without touching any devices or leaving the sterile eld. A surgeon could make a certain hand gesture, moving his or her hand to the left to make an image scroll left,

See Kinect Use in Health Care

Lakeside Center for Autism uses Kinect for therapy. http://www.youtube.com/watch?v=6ZEo7vLwgf8

Web End =http:// http://www.youtube.com/watch?v=6ZEo7vLwgf8

Web End =www.youtube.com/watch?v[H11005]6ZEo7vLwgf8 .

Sunnybrook Health Sciences Centre uses Kinect in surgery. http://www.youtube.com/watch?v=f5Ep3oqicVU

Web End =http://www.youtube.com/watch?v[H11005]f5Ep3oqicVU .

Da Vinci surgery robot Kinect controls demonstrated:Gesture Surgery - Kinect Hacks. http://www.youtube.com/watch?v=i4DxNDoM6Cc

Web End =http://www.youtube.com/ http://www.youtube.com/watch?v=i4DxNDoM6Cc

Web End =watch?v[H11005]i4DxNDoM6Cc .

British news outlet ITN News discusses Kinects use in health care. http://www.youtube.com/watch?v=7-lr455xd_w

Web End =http://www.youtube.com/watch?v[H11005]7-lr455xd_w .

A demonstration of Kinects potential medical applications from the Wake Forest University School of Medicine. http://www.youtube.com/watch?v=id7OZAbFaVI

Web End =http://www.youtube.com/watch?v[H11005]id7OZAbFaVI .

Editors note: Da Vinci is a registered trademark of Intuitive Surgical, Inc, Sunnyvale, CA. Publication of this article does not imply AORN endorsement of specic products.

AORN Journal 527

April 2012 Vol 95 No 4 TAPPING INTO TECHNOLOGY

for example. Because Kinect can sense movement in three dimensions, the surgeon could move toward or away from the sensor to make an image zoom in or out.

Kinect requires special software. It is not compatible with PC software (or games) that are not specically made for use with it. Non-Microsoft software for Kinect exists, but Microsoft has not yet made available ofcial tools for developing commercial Kinect software. As of December 2011, Microsoft offered a testing version of a tool kit for developing commercial Kinect software to use with a PC. The company planned a release of these tools in early 2012.11 The hardware for such a system is simply an off-the-shelf video game accessory, but software to use the Kinect in the OR is unlikely to be widely available until the commercial software tool kit is available and software developers have had time to create products.

CONCLUSION

A computer interface that does not require surgical staff members to make physical contact with devices linked to that computer opens the door to increased computer usage in the OR. Maturing real-time body scanning technologies, robotic surgery, and the digitization of medical records provide compelling reasons for hospital administrators to consider purchasing an interface that allows scrubbed staff members to use a computer without leaving the sterile eld and rescrubbing.

After the software becomes widely available, Kinect will allow health care providers to add relatively inexpensive consumer hardware and expand their options substantially. As the technology becomes more commonplace, people

will likely nd even more uses for Kinect, many of which could nd their way into the OR.

Editors note: Kinect is a trademark and Xbox 360 is a registered trademark of Microsoft Corp, Redmond, WA. Publication of this article does not imply AORN endorsement of specic products.

References

1. Walters L, Eley S. Robotic-assisted surgery and the need for standardized pathways and clinical guidelines. AORN J. 2011;93(4):455-463.

2. Intraoperative magnetic resonance imaging (iMRI). Mayo Clinic. http://www.mayoclinic.org/intraoperative-mri/

Web End =http://www.mayoclinic.org/intraoperative http://www.mayoclinic.org/intraoperative-mri/

Web End =mri/. Accessed December 1, 2011.

3. Anaesthetists role in computer keyboard contamination in an operating room. J Hosp Infect. 2008;70(2): 148-153.

4. Hartmann B, Benson M, Junger A, et al. Computer keyboard and mouse as a reservoir of pathogens in an intensive care unit. J Clin Monit Comput. 2004;18(1): 7-12.

5. Brusco J. Tablet and e-reader technology in health care and education. AORN J. 2011;93(6):775-781.

6. Brusco J. Using smartphone applications in perioperative practice. AORN J. 2010;92(5):503-508.

7. Wodajo FM. The iPAD in the hospital and operating room. J Surg Radiol. 2010;22:44. http://www.surgisphere.com/SurgRad/issues/volume-2/1-january-2011-pages-1-112/152-column-the-ipad-in-the-hospital-and-operating-room.html

Web End =http://www.surgisphere. http://www.surgisphere.com/SurgRad/issues/volume-2/1-january-2011-pages-1-112/152-column-the-ipad-in-the-hospital-and-operating-room.html

Web End =com/SurgRad/issues/volume-2/1-january-2011-pages-1- http://www.surgisphere.com/SurgRad/issues/volume-2/1-january-2011-pages-1-112/152-column-the-ipad-in-the-hospital-and-operating-room.html

Web End =112/152-column-the-ipad-in-the-hospital-and-operating- http://www.surgisphere.com/SurgRad/issues/volume-2/1-january-2011-pages-1-112/152-column-the-ipad-in-the-hospital-and-operating-room.html

Web End =room.html . Accessed February 14, 2012.

8. Wahbe R. Keynote speech presented at: Microsoft TechEd 2011; May 16, 2011; Atlanta, GA.

9. Kinect sensor components. Xbox. http://support.xbox.com/en-US/kinect/setup-and-playspace/kinect-sensor-components

Web End =http://support.xbox. http://support.xbox.com/en-US/kinect/setup-and-playspace/kinect-sensor-components

Web End =com/en-US/kinect/setup-and-playspace/kinect-sensor- http://support.xbox.com/en-US/kinect/setup-and-playspace/kinect-sensor-components

Web End =components. Accessed December 12, 2011.

10. The Kinect Effect. Xbox. http://www.xbox.com/en-US/Kinect/Kinect-Effect

Web End =http://www.xbox.com/en-US/ http://www.xbox.com/en-US/Kinect/Kinect-Effect

Web End =Kinect/Kinect-Effect. Accessed December 1, 2011.11. FAQ. Kinect for Windows. http://kinectforwindows.org/resources/faq.aspx

Web End =http://kinectforwindows.org/ http://kinectforwindows.org/resources/faq.aspx

Web End =resources/faq.aspx. Accessed December 1, 2011.

Zac Wiggy, BA, is an assistant editor at AORN, Inc, Denver, CO. Mr Wiggy has no declared afliation that could be perceived as posing a potential conict of interest in the publication of this article.

528 AORN Journal