1. Introduction
Responsible for sustaining the human bipedal position, the spine plays a key role in posture, weight-bearing, locomotion, and protection of the spinal cord and nerve roots [1]. Spinal movements are divided into flexion and extension in the sagittal plane, right and left lateralization in the coronal plane, and rotation or circular in the longitudinal plane [2].
A recent report estimated that approximately 34.3 million adults over 18 years of age in Brazil have a chronic back problem [3]. According to the Global Burden of Disease, low back pain is the leading cause of chronic noncommunicable pain, with neck pain ranking ninth, in terms of years lived with disability [4]. These chronic spinal injuries, popularly known as “back pain”, include neck and low back pain, as well as more specific conditions such as spondylosis, radiculopathies, and intervertebral disc disorders [5].
Given the number and diversity of cases, back pain is one of the greatest demands on the health service [6]. With this high demand, physiotherapy services stand out in the health sector for the treatment of this condition. Physiotherapy for patients with back pain aims to control the pain intensity, increase range of motion, teach return to work activities, relieve tension in the affected muscles, reorganize muscle activity patterns through agonist and antagonist action, and maintain balance and spinal functions [7,8].
Since the beginning of the COVID-19 pandemic, an increase in the proportion of musculoskeletal complaints, such as back pain, has been observed, as well as a greater number of chronic cases related to the spine. Certainly, musculoskeletal disorders, including muscle weakness, fatigue, and the lifestyle imposed by social isolation, have contributed to this increase [9].
Despite the positive indication and benefits of physiotherapy, the long period required for treatment and lack of motivation to use traditional methods have been reported as reasons for abandoning physiotherapy treatment, which is one of the major causes of therapeutic failure [10,11].
Thus, this paper aimed to develop serious games, to assist health professionals in the physiotherapy of patients with spinal pain for clinical and home applications. These serious games, which are applications that integrate serious aspects with the playful and interactive elements of video games [12], using OpenPose software represent a potential tool for motivating patients during physiotherapy and reducing treatment dropout rates. To achieve this objective, this paper presents the development of serious games using the Unity 3D engine with a plugin for OpenPose, which allows tracking the movements of the human body with a camera. A laptop-type personal computer with a webcam was also used.
The use of serious games for the physical rehabilitation of back pain is an active area of research [13]. A systematic review exploring the use of virtual reality in the treatment of adults with chronic low back pain showed that virtual reality can significantly reduce pain intensity in patients with this condition [14].
The Valedo system, developed by Hocoma [15], is a commercial device used for back training with three wireless inertial sensors. The sensors transfer the trunk movements into a serious game environment and guide patients through exercises specifically designed for low back pain therapy [16]. A previous report indicated that the results of improving lumbopelvic movement control were more effective using this system than with feedback from a mirror or no feedback [17].
Reference [18] showed that training through virtual reality games with the Pro-Kin system (PK 252 N Techno body) resulted in a long-term improvement in clinical and sports performance compared to the other forms of training applied to soccer players with chronic low back pain.
A prior study developed serious games for spine rehabilitation using a Kinect device and Unity 3D. The authors revealed that the serious game was able to determine spine movement angles, with no noticeable delay [19].
In our survey, we found no papers using OpenPose for the rehabilitation of low back pain. Most literature required specific/dedicated equipment to acquire spinal movements, making implementation more difficult and increasing the costs of home care applications. This paper presents contributions for measuring the use of OpenPose in the development of serious games applied in the rehabilitation of the human spine for home care, using a simple webcam connected to a personal computer. The use of serious games described in this paper for back pain rehabilitation can bring economy, motivation, and time reduction to therapist labor.
This paper is divided into a brief presentation of serious games and OpenPose in Section 2, followed by the planning of the serious game as a function of the selected spine movements in Section 3. Section 4 presents the games developed using Unity 3D with OpenPose. Section 5 describes the validation results, and Section 6 summarizes the paper and presents the conclusions.
2. Serious Games and OpenPose Software
Serious games aim to treat patients with the exercises of a conventional physiotherapeutic intervention session, but in a more playful and interactive way, aiming at a greater effectiveness of muscle action, improvements in quality of life, and a reduction in the dropout rates of spinal rehabilitation treatment [20,21].
Notably, the use of serious games is enhanced by technologies that allow human body tracking. Technologies for body motion tracking can use specific equipment without markers, such as Kinect [22,23,24,25]; an algorithm coupled to a webcam; or markers placed on the body. Open-source software for pose estimation is an alternative that can reduce the costs associated with motion capture, as machine learning partially eliminates the need for specialized cameras and equipment [26]. In this paper, the OpenPose software was used.
Responsible for game control, OpenPose was the first open-source real-time software for estimating two-dimensional (2D) human poses in images, including body, foot, hand, and facial key points [26]. This algorithm has a plugin for Unity 3D that allows its use in game development [27].
OpenPose uses the entire image as input to a convolutional neutral network (CNN) to predict 2D confidence maps and detect body parts [26]. Figure 1 illustrates the OpenPose keypoints of the human body skeleton.
The efficiency and accuracy of OpenPose have been reported in the literature [26,27,28,29,30,31], which described the performance when using a set of 2D vector fields that encode the location and orientation of the human body over the image domain [28].
In reference [26], OpenPose was used to perform pose estimation of healthy gait kinematics and spatiotemporal gait parameters, with results that could be used in clinical and research settings when marker-based systems are not available. The hip kinematics and knee kinematics had an error of 3.7 ± 1.3 deg and 5.1 ± 2.5 deg (mean ± std), respectively, compared to the Vicon MX-T40 marker system.
In reference [29], Open Pose was used to conduct ergonomic postural assessments, and it produced better results than a Kinect-based system.
In reference [30], OpenPose was compared to a video-based human pose estimation, and the mean absolute error of the sagittal plane hip, knee, and ankle angles was 4.0°, 5.6°, and 7.4°, respectively.
In reference [31], the range of motion of the upper extremity was analyzed using OpenPose and compared with goniometry. Comparing OpenPose with manual measurements, the shoulder abduction was 0.51°, the shoulder elevation was 2.87°, the elbow flexion was 0.38°, the elbow extension was 0.65°, and the wrist extension was 0.78°. The most significant difference was found in wrist flexion, with a mean value of 8.96°.
The accuracy assessments of OpenPose reported in the literature were performed for specific situations [26,27,28,29,30,31]. No studies focusing on applications in the human spine were found.
Used in conjunction with OpenPose, Unity 3D is a well-known game development software used to create 2D and 3D games. Games developed with Unity can be used, with the proper licenses, on different platforms and even embedded in a web page [24].
Unity 3D applies the concept of object composition to a component-based architecture, to customize the game elements [32]. In this model, a game object is specified through the composition of several functionalities, which can be added (or removed). Each functionality is implemented using a component (a class that inherits from a basic component). This generic container is called the Game Object and works as a repository of functionality, or more specifically, components. In the development of games with Unity 3D, assets are used, which are representations of any object that can be used in the game. An asset can be a specific file created outside of Unity, such as a 3D model, an image, an audio file, or any other file supported by Unity 3D [32].
3. Planning and Development of Serious Games for Spine Rehabilitation
During the development of the initial test game following a patient care session, some typical spinal rehabilitation exercises could not be used, due to the limitations of OpenPose, such as the existence of only two points of articulation in the spine (Figure 1) and the technological barriers to the inclusion of these exercises in the project; for example, the use of two or more auxiliary cameras per exercise and the positioning of the cameras required by the exercises, making their application unfeasible. Exercises that could not be addressed included those requiring twisting the spine around its own axis and the cat-cow stretch pose exercise, where the patient mobilizes the entire thoracic spine, significantly increasing the thoracic kyphosis; first upwards slowly and then downwards with hands and knees on the floor.
Six physical therapy exercises were chosen based on the limitations of OpenPose: lumbar extension (McKenzie method) in the lying position (Figure 2a); lumbar extension (McKenzie method) in the upright position (Figure 2b); lumbar flexion in the upright position (Figure 2c); hip extension, lying down (Figure 2d); left lateral flexion (Figure 2e); and right lateral flexion (Figure 2f).
The McKenzie method is an advanced evidence-based physiotherapy assessment and self-treatment technique that offers conditions for the patient to treat himself quickly, safely, and effectively, without relying on medication, heat, ice, ultrasound, or surgery. This approach is known worldwide as the McKenzie Method of Mechanical Diagnosis and Therapy and uses both extension and flexion exercises [33].
The Williams exercises were developed by observing that most patients with chronic low back pain had degenerative skeletal changes secondary to intervertebral disc injuries. Therefore, the Williams exercises use spine and hip flexion as the treatment principle. To reduce pain and stabilize the trunk, the physical exercises actively develop the flexor muscles and passively stretch the lumbosacral extensor muscles. Williams emphasizes that the pelvic tilt is essential for optimal treatment results [34].
Six physical therapy exercises were chosen and divided into four serious games. The asteroids game used lumbar extension (McKenzie method) in a lying position (Figure 2a); the park day game explored lumbar extension (McKenzie method) in the upright position (Figure 2b) and lumbar flexion in the upright position (Figure 2c); the fishing game used hip extension, lying down (Figure 2d). The infinite run game used left (Figure 2e) and right lateral flexion (Figure 2f). The games were developed using free Unity 3D resources and elements built by the authors.
Modeling was carried out so that the health professional responsible for the patient and with knowledge of therapy and rehabilitation can supervise and monitor the stages of the serious games; that is, from the patient’s data entry to the end of the game.
In the main menu, the physical therapist has access to five paths: registration, calibration, games, options, and exit. In the options tab, only informative text about the game is shown, without the need for user interaction. With the exit option, the user exits the menu and games. The registration option directs the user to the second screen, the “NewPatient” scene. This screen allows the user to enter a new patient in the menu database, by inputting patient information.
With the patient’s data entered, the user’s next option is to have the patient calibrate movements intended for the current physical therapy section, where movement limits will be recorded for use in movement control in games. Such calibration is done in the “calibration” scene.
After calibrating the patient’s movements, the next option is to start the game chosen for the current physical therapy section, as previously chosen and planned by the professional. In this environment screen, the player who interacts with the game is the patient/user, represented by the main player avatar of each game. The objectives of each game are always related to the spinal movements required for the patient’s treatment, where visual effects appear in the form of awards for correct movements and staying in a position for the necessary time. This screen also displays the score and time for each game, along with a pause menu for the exclusive use of the therapist. In this menu, which is found in each game, the therapist has access to four paths: continue, main menu, save, and exit.
4. Games Developed Using Unity 3D with OpenPose Software
Through the use of Unity 3D software, it was possible to improve and facilitate the development of aesthetically pleasing, smooth, and attractive games, while maintaining the seriousness of rehabilitation. We created simple and intuitive scripts whenever possible, so the objects would be easy to reference within the game.
The patient interacts with the main character of the games through the movements captured by the camera, either a webcam or another auxiliary camera connected and configured with the computer and processed by OpenPose.
The avatar’s movement is controlled, in real-time, by OpenPose through points two (neck) and nine (central point of the hip) of the column (Figure 1). In this paper, “real-time” is defined as the generation of interactive content using computer graphics that are faster than human perception.
4.1. Asteroids Game
Figure 3 shows the default screen for the asteroids game. In this game, the player controls a character in a spaceship via a camera, in a scenario that represents a battle against asteroids in outer space. These asteroids appear on the screen over time, in predefined positions.
Figure 3 displays, at the top center, a sentence explaining what the player must do in this game as a tutorial. With the game running, it is possible to observe, in the upper right corner, the game scoreboard and its execution time, as well as one of the asteroids to be destroyed.
The avatar is the main character of this game, with the function of simulating in the virtual environment the movements of the spine of the patient user, who performs the physiotherapy exercises in front of the camera. This character is a model freely available online for download through assets. It represents a spaceship and responds to the movement of two specific points on the spine. One of these points, located on the patient’s neck, is responsible for performing the character’s movements. The other point, located in the central part of the patient’s hip, is used as a reference to verify that the patient is correctly positioned to carry out the exercise in question, which is the lumbar extension exercise (McKenzie method), performed with the patient lying on the floor with the front of the body facing the floor.
The character moves between three specific positions to fulfill the exercise. These points are the upper and lower limits of spine movement (adding to these values a positioning margin, both above and below), in which, when moving the column, the patient has visual confirmation that he is positioned correctly (Figure 4). Besides the two limit points described, there is a third point belonging to the lumbar extension exercise, which is a midpoint between the lower and upper limits. This central point is obtained through a simple mathematical formulation.
The Asteroids are the game objects to be destroyed by the player. They appear on the screen at specific and predefined points and periods of time so that the patient performs the movements that the lumbar extension exercise (McKenzie method) requires.
The asteroids are positioned on the screen in the same three positions where the spaceship moves. Each asteroid appears at a predefined time, to make the patient perform the exercise correctly. The asteroid animation gives the player the visual illusion that it is heading towards the spaceship, but it stops moving at a certain point on the screen close to where it first appeared to avoid forcing the patient to perform sudden movements, due to possible nervousness when observing an asteroid approaching very close to the spaceship.
4.2. Park Day Game
Figure 5 shows the main screen of the park day game. In this game, the player controls a character, represented by a child on the left of the screen, who encourages the animals to reach their owners. The scenario represents a wooded park with animals that interact with the user in a way that conveys a feeling of calm and tranquility.
Figure 5 shows the three animals that must be encouraged to reach their respective owners. The left part of the park illustrates the dog, the rabbit, and the cat, which must reach their owners in the respective order on the right part of the park. At the top center, a sentence explains what the player must do, working as a game tutorial. The right part shows the scoreboard for the game. The other animals in the scene intend to immerse the user even further in the context of the park, with some even having a comic nature to entertain the patient.
The avatar is the main character of the game and simulates in the virtual environment the spine movements that the patient performs in front of the camera. This character is a model asset available for free and represents a child who encourages the “pets” to run to their respective owners and responds to the movement of two specific points on the spine, one located on the patient’s neck and the other on the central part of the hip. The two exercises required of the patient are lumbar flexion (upright position) (Figure 2c) and lumbar extension (McKenzie method—upright position) (Figure 2b).
The avatar accompanies the movements made by the user in front of the camera, referring to the lumbar flexion and extension exercises. These movements respect the limits of the angle the patient can perform with their spine, as defined in the calibration scene. The character moves between three specific positions, to achieve the exercise required. Positions are obtained of the patient’s spine at two pre-established points. These points are taken as the upper and lower limits of spine movement (adding to these values a positioning margin, both more and less), with which, when moving the column, the patient has visual confirmation that he is moving correctly during the exercise. The central point is the meeting point of the two exercises and the starting point of both. The avatar’s movement is controlled in real-time using OpenPose with points two (neck) and nine (central point of the hip) of the column (Figure 1). Figure 6 shows the movement of the column controlling the avatar, which, in turn, triggers the movement of animals (secondary characters) in the game. In Figure 6a, the player performs a lumbar extension and moves the game avatar in the top position of the screen. In Figure 6b, the player performs lumbar flexion, moving the avatar in the lower position of the screen.
Points are awarded for the arrival of the pets, initially on the left part of the park, to their owners, on the right part. Each pet that reaches its respective owner adds one point to the counter.
4.3. Fishing Game
Figure 7 shows the default screen for the fishing game, illustrating the characters controllable by the player via OpenPose and the camera, which are a fisherman and his elevator platform, a flying manta ray, in a fishing scenario represented by a landscape with waterfalls, a river, trees in the background, and fish jumping in the water.
This game aims to induce the player to make the necessary movements to correctly perform hip extension in a lying position with the back facing the ground. The player needs to catch as many fish as possible, which are constantly jumping out of the water on the screen (Figure 7).
This avatar represents the aforementioned fisherman, always accompanied by his flying manta ray, and who is only responsible for simulating fishing movements and giving points to the player for catching fish. The fisherman and the manta ray respond to the movement of two specific points on the spine. One of these points, located on the patient’s neck, is responsible for performing the character’s movements. The other point, located in the central part of the patient’s hip, is used as a reference to verify if it is correctly positioned to perform the exercise in question.
This script checks the current position of the player for each frame. If it is within the limits of one of the three pre-established positions, in case the player remains in this position during the execution time of the animation (which is the fisherman making a movement to catch a fish); then a fish is caught, and a point is added to the score. Figure 8 shows the movement of the column controlling the avatar, which in turn performs the action of catching fish.
If the player remains in one of the three predefined positions during the execution time (which is the fisherman moving to catch a fish), a fish is caught and a point is added to the score.
4.4. Infinite Run Game
Figure 9 displays the default screen for the infinite run game. In this game, the character, controllable by the player via OpenPose and the camera, is a robot in a scenario composed of an infinite bridge over the open sea.
This game aims to induce the player to perform the necessary movements to correctly complete left lateral flexion and right lateral flexion exercises in an upright position (Figure 2). In this game, the player needs to collect as many coins as possible. The purpose of this game is to make the player maintain the position required by the proposed exercises for the necessary time. For that purpose, the coins alternately appear on the screen in three predefined positions.
Ten platforms were made for infinite run, and during the game’s execution, the platforms are aligned in sequence, forming an infinite bridge where the avatar runs. At the end of each platform, they are recycled to optimize the computational cost, making it lighter and run more smoothly.
The game controls and counts the distance covered by the character, estimated by converting the execution time into a proportional distance in meters, giving the player the impression of being in a race and thus increasing immersion in the game. Furthermore, there is control over the number of coins the player collects as he travels across the bridge.
Figure 10 shows the movement of the column controlling the avatar, which moves on the track, collecting the coins.
Importantly, the movement limits of the game are defined by health professionals, with a follow-up in the clinic or via video call in the event of pandemics, which lead to social isolation.
5. Results
Validation of the angles obtained by OpenPose during the execution of the serious games was carried out using a Miotec goniometer with a precision of ±0.05° and an acquisition rate of 2000 Hz.
The experiments were approved by the local ethics committee, and the participant provided written informed consent prior to testing. The test subject was a 27-year-old male with a height of 1.67 cm and a weight of 65 kg. The experimental environment to test the infinite run game and park day games was a laptop with a camera positioned on a table 0.75 m high and with the subject one meter from the laptop screen. In the fishing game, the laptop was positioned on the floor, with one meter between the screen and the subject. The laptop used for developing the serious games was a Samsung with a seventh-generation Intel Core i7 processor, 2 GB NVIDIA GeForce 940 MX graphics card, 8 GB RAM, 1 TB hard drive, and a Microsoft webcam (original factory setting).
Figure 11a illustrates the graphs obtained from OpenPose for the infinite run game, and Figure 11b shows the data obtained with the Miotec goniometer.
Table 1 depicts the data for the movements performed in the infinite run game. The data were divided according to peak and valley graphical curves.
According to Table 1, the measurements/movements from OpenPose showed repeatability (24.48° ± 1.57° peak and 23.10° ± 1.68° valley) when playing infinite run. The measurements were made by simulating a patient playing the game and performing the movements of left lateral flexion and right lateral flexion stimulated with the proposed theme. The OpenPose and goniometer measurement errors were 15.66% at the peak and 21.50% at the valley. The values given by OpenPose for the lateral flexion of the spine cannot be used directly for precision measurement but can be used to follow the evolution of the patient.
Table 2 presents the data on the movements performed in the fishing game.
According to Table 2, the OpenPose measurements showed repeatability (7.28° ± 0.92° peak and 5.85° ± 1.48° valley) when playing the fishing game. The OpenPose and goniometer measurement errors were 10.72% at the peak and 22.19% at the valley. The fishing game allowed reproduction of the proposed movements, to stimulate spinal rehabilitation exercises.
Table 3 summarizes the data for the movements performed during the park day game.
According to the findings, the OpenPose measurements showed repeatability (16.78° ± 1.74° peak and 35.91° ± 2.93° valley) when playing the park day game. The measurements were made while simulating a patient playing the game, and with the movements of lumbar flexion and extension (upright position) stimulated by the proposed theme. The OpenPose and goniometer measurement errors were 21.34% at the peak and 5.97% at the valley (Table 3). The park day game allowed the reproduction of the desired movements for spine rehabilitation, and the collected data can be used to monitor the patient’s evolution.
Figure 12 shows the respective graphs for the asteroids (Figure 12a) and infinite run (Figure 12b) games with the scores achieved. With the newly developed graphical interface for scoring, physiotherapy professionals would have quantitative tools to better evaluate human spine rehabilitation procedures.
Comparing the results of the serious games developed in this paper using OpenPose and Unity 3D, we obtained values equivalent to those obtained using Kinect [19]. The inertial sensors of the Valedo system have a better resolution [15,16,17]. No accuracy analyses for OpenPose were found in the literature applied to the assessment of human spine movements.
Besides the impossibility of using some physical therapy exercises in the games, other difficulties were noted during their development. For instance, when working with animations, the spawn method initially implemented was unfeasible, as its computational cost was very high, even causing the computer to crash and closing the software, due to a lack of response. This problem was solved using script optimization; and with less demanding cases where significant optimization was not necessary, the animation spawned and it was destroyed after it had finished. An example of optimization is the “recycling” of platforms in the infinite run game.
Another factor was the scoring method initially implemented. In some cases, this could give the player the impression that he was facing a penalty if he could not complete what was needed to score points. This problem was resolved and everything that could give this impression was removed, such as the asteroids that moved to collide with the spaceship that the player controlled if they were not destroyed in time, which could generate a sudden movement from the patient to divert the spaceship. We solved this problem by causing the asteroid to appear slowly on the screen, and even when it was not destroyed, it stopped in a position far from the spaceship, waiting for the new position of the spaceship for its destruction.
Another example is the fact that, initially, there were obstacles on the bridge of the infinite run game; even if the patient did not lose points when colliding with these obstacles, he could make sudden movements to avoid them. We solved this problem by removing the obstacles, and instead of the user receiving points for dodging them, he received points for collecting coins that appeared on the screen, as the robot avatar controlled by the user traveled across the bridge.
Motion exercise and rehabilitation during the COVID-19 pandemic are different from the containment measures during medical treatment. Normally, post-care patients who return home are required to perform physical exercises on their own, which is crucial to improve physical fitness [35]. Thus, there is a need to help patients with motion exercises and physical rehabilitation, by providing them with suitable and inexpensive equipment at home, such as personal computers with a webcam. This paper focused on back pain rehabilitation using OpenPose and Unity 3D software.
6. Conclusions
In this study, serious games were developed, to assist in the rehabilitation of the human spine. Importantly, these games are designed to improve the patients’ rehabilitation processes, especially in terms of engagement with and the continuity of treatment. The planning and choice of exercises were characterized as the main activities because, despite being an instrument of playful creation, the final objective of games was to assist in the rehabilitation treatment of patients.
Pandemics such as the COVID-19 bring new challenges to humanity, and rehabilitation procedures had to be discontinued due to the need for isolation. The COVID-19 pandemic increased the proportion of musculoskeletal complaints, such as back pain. The great difficulty of home treatment, due to the high cost of conventional equipment and the need for specialized labor to operate it, opened up new paths for developing solutions for home rehabilitation. This paper presented the development of serious games to assist health professionals in the physiotherapy of patients with spinal pain, during clinical and home applications, and to solve the isolation issues caused by the pandemic.
To create software for the proposed serious games, Unity 3D and OpenPose were used. Despite the measurement errors of the points related to the spine tracked by OpenPose and used in the serious games, the movements performed were repeatable. The values obtained using OpenPose and the score achieved could be used to monitor the patient’s evolution in spinal rehabilitation treatments.
Ultimately, the proposed games allowed the reproduction of the desired movements for the rehabilitation of the human spine, and the collected data could be used to follow the patient’s progression. However, further studies, including clinical trials with patients suffering from back pain are required, to validate these newly developed serious games.
Conceptualization, R.S.G. and R.A.P.; methodology, R.S.G. and R.A.P.; software, R.S.G. and R.A.P.; validation, R.S.G., R.A.P., G.M.V.L. and A.L.P.G.; formal analysis, R.S.G., R.A.P., G.M.V.L. and A.L.P.G.; investigation, R.S.G., R.A.P., G.M.V.L. and A.L.P.G.; resources, R.S.G., R.A.P., G.M.V.L. and A.L.P.G.; data curation, R.S.G., R.A.P., G.M.V.L. and A.L.P.G.; writing—original draft preparation, R.S.G., R.A.P., G.M.V.L. and A.L.P.G.; writing—review and editing, R.S.G., R.A.P., G.M.V.L. and A.L.P.G.; visualization, R.S.G., R.A.P., G.M.V.L. and A.L.P.G.; supervision, R.S.G. and A.L.P.G.; funding acquisition, R.S.G. and A.L.P.G. All authors have read and agreed to the published version of the manuscript.
The experiments were approved by the local ethics committee (Protocol CAAE: 51133821.4.0000.5152), and all participants provided written informed consent prior to the experiments.
Informed consent was obtained from all subjects involved in the study.
There are no additional files. All necessary data have been included in the body of the text.
The authors declare no conflict of interest.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Enlarge this image.Figure 2. Physical therapy exercises used in the games. (a) lumbar extension in the lying position; (b) lumbar extension in the upright position; (c) lumbar flexion in the upright position; (d) hip extension; (e) left lateral flexion; (f) right lateral flexion.
Enlarge this image.Figure 4. Controlling the game avatar and destroying asteroids through column movements.
Enlarge this image.Figure 6. Moving the park day game avatar by moving the column. (a) lumbar extension; (b) lumbar flexion.
Enlarge this image.Figure 6. Moving the park day game avatar by moving the column. (a) lumbar extension; (b) lumbar flexion.
Enlarge this image.Figure 11. (a) OpenPose data; (b) Miotec goniometer data for the infinite run game.
Enlarge this image.Figure 12. (a) Graph of the position over time, associated with the score obtained, A-Lying, B-Stage 1 lumbar extension, and C-Stage 2 lumbar extension in the asteroids game. (b) Graph of position and the distance traveled, as associated with the score achieved, A-Left lateral flexion, B-Center, and C-Right lateral flexion in the infinite run game.
Data acquired from the infinite run game (°).
| Peak |
Valley OpenPose | Peak |
Valley Sensor | Comparison Peak | Peak Error |
Comparison Valley | Valley |
|---|---|---|---|---|---|---|---|
| 25.1777 | −21.8261 | 34.5663 | 20.7398 | 9.3886 | 27.1612 | 1.0863 | −5.2378 |
| 27.1744 | −22.8939 | 27.6530 | 20.7398 | 0.4786 | 1.7307 | 2.1541 | −10.3865 |
| 22.0160 | −23.0193 | 27.6530 | 20.7398 | 5.6370 | 20.3848 | 2.2795 | −10.9911 |
| 23.1296 | −25.8529 | 27.6530 | 20.7398 | 4.5234 | 16.3578 | 5.1131 | −24.6537 |
| 25.1304 | −22.3809 | 20.7398 | 20.7397 | 4.3906 | 21.1700 | 1.6412 | −7.9131 |
| 24.0696 | −25.1092 | 20.7398 | 20.7397 | 3.3298 | 16.0551 | 4.3695 | −21.0683 |
| 23.8739 | −23.0177 | 27.6531 | 13.8265 | 3.7792 | 13.6664 | 9.1912 | −66.4751 |
| 25.2302 | −20.6711 | 27.6530 | 27.6530 | 2.4228 | 8.7615 | 6.9819 | −25.2484 |
Data acquired from the Fishing game (°).
| Peak |
Valley OpenPose | Peak |
Valley Sensor | Comparison Peak | Peak Error | Comparison Valley | Valley |
|---|---|---|---|---|---|---|---|
| 7.8892 | −4.3440 | 6.9133 | −6.9132 | 0.9759 | 14.1162 | −2.5692 | 37.1636 |
| 7.4072 | −5.5048 | 6.9132 | −6.9133 | 0.4940 | 7.1457 | −1.4085 | 20.3737 |
| 6.5475 | −7.3497 | 6.9132 | −6.9133 | 0.3657 | 5.2898 | 0.4364 | 6.3124 |
| 6.3751 | −6.0910 | 6.9132 | −6.9133 | 0.5381 | 7.7836 | −0.8223 | 11.8944 |
| 7.0015 | −5.4362 | 6.9132 | −6.9133 | 0.0883 | 1.2772 | −1.4771 | 21.3660 |
| 7.8486 | −6.0189 | 6.9132 | −6.9133 | 0.9354 | 13.5306 | −0.8944 | 12.9373 |
| 6.5428 | −6.2912 | 6.9132 | −6.9133 | 0.3704 | 5.3578 | −0.6221 | 8.9985 |
| 6.7035 | −5.1816 | 6.9132 | −6.9133 | 0.2097 | 3.0333 | −1.7317 | 25.0488 |
| 7.9554 | −3.7400 | 6.9132 | −6.9133 | 1.0422 | 15.0755 | −3.1733 | 45.9013 |
| 7.2640 | −5.0176 | 6.9132 | −6.9133 | 0.3508 | 5.0743 | −1.8957 | 27.4210 |
| 6.1430 | −7.3818 | 6.9132 | −6.9133 | 0.7702 | 11.1410 | 0.4685 | 6.7767 |
| 6.3507 | −9.5735 | 6.9132 | −6.9133 | 0.5625 | 8.1366 | 2.6602 | 38.4794 |
| 7.6371 | −5.9728 | 6.9132 | −6.9133 | 0.7239 | 10.4712 | −0.9405 | 13.6042 |
| 9.7088 | −3.9009 | 6.9132 | −6.9133 | 2.7956 | 40.4385 | −3.0124 | 43.5739 |
| 7.8057 | −6.0170 | 6.9132 | −6.9133 | 0.8925 | 12.9100 | −0.8963 | 12.9648 |
Data acquired from the park day game (°).
| Peak |
Valley OpenPose | Peak |
Valley Sensor | Comparison Peak | Peak Error | Comparison Valley | Valley |
|---|---|---|---|---|---|---|---|
| 17.3296 | 37.6831 | 13.8265 | 34.5663 | 3.5031 | −25.3361 | 3.1168 | −9.0169 |
| 17.5499 | 43.4869 | 13.8265 | 34.5663 | 3.7234 | −26.9294 | 8.9206 | −25.8072 |
| 16.2637 | 40.6741 | 13.8265 | 34.5663 | 2.4372 | −17.6270 | 6.1078 | −17.6698 |
| 14.9639 | 33.9152 | 13.8265 | 34.5663 | 1.1374 | −8.2262 | 0.6511 | 1.8836 |
| 15.0036 | 33.8901 | 13.8265 | 34.5663 | 1.1771 | −8.5134 | 0.6762 | 1.9562 |
| 14.8255 | 34.8205 | 13.8265 | 34.5663 | 0.9990 | −7.2253 | 0.2542 | −0.7354 |
| 19.0809 | 31.9137 | 13.8265 | 34.5663 | 5.2544 | −38.0024 | 2.6526 | 7.6739 |
| 17.8143 | 35.6055 | 13.8265 | 34.5663 | 3.9878 | −28.8417 | 1.0392 | −3.0064 |
| 16.7086 | 35.7589 | 13.8265 | 34.5663 | 2.8821 | −20.8448 | 1.1926 | −3.4502 |
| 15.2712 | 36.3465 | 13.8265 | 34.5663 | 1.4447 | −10.4488 | 1.7802 | −5.1501 |
| 18.3985 | 34.9237 | 13.8265 | 34.5663 | 4.5720 | −33.0669 | 0.3574 | −1.0340 |
| 20.0040 | 34.0739 | 13.8265 | 34.5663 | 6.1775 | −44.6787 | 0.4924 | 1.4245 |
| 18.2420 | 36.9440 | 13.8265 | 34.5663 | 4.4155 | −31.9351 | 2.3777 | −6.8787 |
| 15.9775 | 33.6312 | 13.8265 | 34.5663 | 2.1510 | −15.5571 | 0.9351 | 2.7052 |
| 14.2202 | 34.9848 | 13.8265 | 34.5663 | 0.3937 | −2.8474 | 0.4185 | −1.2107 |
References
1. Cramer, G.D.; Darby, S.A. Clinical Anatomy of the Spine, Spinal Cord, and ANS; 3rd ed. Mosby: Maryland Heights, MO, USA, 2013.
2. Saito, H.; Watanabe, Y.; Kutsuna, T.; Futohashi, T.; Kusumoto, Y.; Chiba, H.; Kubo, M.; Takasaki, H. Spinal Movement Variability Associated with Low Back Pain: A scoping Review. PLoS ONE; 2021; 16, e0252141. [DOI: https://dx.doi.org/10.1371/journal.pone.0252141] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34029347]
3. Romero, D.E.; Santana, D.; Borges, P.; Marques, A.; Castanheira, D.; Rodrigues, J.M.; Sabbadini, L. Prevalence, associated factors, and limitations related to chronic back problems in adults and elderly in Brazil. Rep. Public Health; 2018; 34, e00012817.
4. Wu, A.; March, L.; Zheng, X.; Huang, J.; Wang, X.; Zhao, J.; Blyth, F.M.; Smith, E.; Buchbinder, R.; Hoy, D. Global low back pain prevalence and years lived with disability from 1990 to 2017: Estimates from the Global Burden of Disease Study 2017. Ann. Transl. Med.; 2020; 8, 299. [DOI: https://dx.doi.org/10.21037/atm.2020.02.175] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32355743]
5. Steinmetz, A. Back pain treatment: A new perspective. Ther. Adv. Musculoskelet Dis.; 2022; 4, 1759720X221100293. [DOI: https://dx.doi.org/10.1177/1759720X221100293]
6. Beyera, G.K.; O’Brien, J.; Campbell, S. Health-care utilisation for low back pain: A systematic review and meta-analysis of population-based observational studies. Rheumatol. Int.; 2019; 39, pp. 1663-1679. [DOI: https://dx.doi.org/10.1007/s00296-019-04430-5] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31463608]
7. Gordon, R.; Bloxham, S. A Systematic Review of the Effects of Exercise and Physical Activity on Non-Specific Chronic Low Back Pain. Healthcare; 2016; 4, 22. [DOI: https://dx.doi.org/10.3390/healthcare4020022] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27417610]
8. Kim, B.; Yim, J. Core Stability and Hip Exercises Improve Physical Function and Activity in Patients with Non-Specific Low Back Pain: A Randomized Controlled Trial. Tohoku J. Exp. Med.; 2020; 251, pp. 193-206. [DOI: https://dx.doi.org/10.1620/tjem.251.193]
9. Grech, S.; Borg, J.N.; Cuschieri, S. Back pain: An aftermath of Covid-19 pandemic? A Malta perspective. Musculoskelet. Care; 2022; 20, pp. 145-150. [DOI: https://dx.doi.org/10.1002/msc.1574]
10. Holden, J.; Davidson, M.; O’Halloran, P. Motivational strategies for returning patients with low back pain to usual activities: A survey of physiotherapists working in Australia. Man. Ther.; 2015; 20, pp. 842-849. [DOI: https://dx.doi.org/10.1016/j.math.2015.04.005]
11. Shahidi, B.; Padwal, J.; Lee, E.; Xu, R.; Northway, S.; Taitano, L.; Wu, T.; Raiszadeh, K. Factors impacting adherence to an exercise-based physical therapy program for individuals with low back pain. PLoS ONE; 2022; 17, e0276326. [DOI: https://dx.doi.org/10.1371/journal.pone.0276326]
12. Alves, T.; Gonçalves, R.S.; Carbone, G. Serious Games Strategies With Cable-Driven Robots for Bimanual Rehabilitation: A Randomized Controlled Trial With Post-Stroke Patients. Front. Robot; 2022; 9, 739088. [DOI: https://dx.doi.org/10.3389/frobt.2022.739088] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35252362]
13. Adeel, T.J.; Zia, R.; Fryazinov, O. Design and Development Considerations for Serious Games to Assist in the Rehabilitation of Patients with Back Pain—An Overview. Available online: http://eprints.bournemouth.ac.uk/37891/1/EVA%20Final%20Version.pdf (accessed on 10 March 2023).
14. Brea-Gómez, B.; Torres-Sánchez, I.; Ortiz-Rubio, A.; Calvache-Mateo, A.; Cabrera-Martos, I.; López-López, L.; Valenza, M.C. Virtual Reality in the Treatment of Adults with Chronic Low Back Pain: A Systematic Review and Meta-Analysis of Randomized Clinical Trials. Int. J. Environ. Res. Public Health; 2021; 18, 11806. [DOI: https://dx.doi.org/10.3390/ijerph182211806] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34831562]
15. Mourits, B.M.P.; Vos, L.A.; Bruijn, S.M.; Dieën, J.H.v.; Prins, M.R. Sensor-based intervention to enhance movement control of the spine in low back pain: Protocol for a quasi-randomized controlled trial. Front. Sport. Act. Living; 2022; 4, 1010054. [DOI: https://dx.doi.org/10.3389/fspor.2022.1010054] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36325522]
16. Patel, S.; Park, H.; Bonato, P.; Chan, L.; Rodgers, M. A review of wearable sensors and systems with application in rehabilitation. J. NeuroEng. Rehabil.; 2012; 9, 21. [DOI: https://dx.doi.org/10.1186/1743-0003-9-21]
17. Matheve, T.; Brumagne, S.; Demoulin, C.; Timmermans, A. Sensor-based postural feedback is more effective than conventional feedback to improve lumbopelvic movement control in patients with chronic low back pain: A randomised controlled trial. J. Neuroeng. Rehabil.; 2018; 15, 85. [DOI: https://dx.doi.org/10.1186/s12984-018-0423-6]
18. Nambi, G.; Abdelbasset, W.K.; Elsayed, S.H.; Verma, A.; George, J.S.; Saleh, A.K. Clinical and Physical Efficiency of Virtual Reality Games in Soccer Players with Low Back Pain. Rev. Bras. Med. Esporte; 2021; 27, pp. 597-602. [DOI: https://dx.doi.org/10.1590/1517-8692202127062021_0034]
19. Pereira, L.B.; Barbosa, D.D.; Goncalves, R.S. Development of Games Controlled by Kinect to Spine Physical Therapy. IEEE Int. Conf. Rehabil. Robot.; 2019; 2019, pp. 193-197. [DOI: https://dx.doi.org/10.1109/ICORR.2019.8779494]
20. Matheve, T.; Claes, G.; Olivieri, E.; Timmermans, A. Serious Gaming to Support Exercise Therapy for Patients with Chronic Nonspecific Low Back Pain: A Feasibility Study. Games Health J.; 2018; 7, pp. 262-270. [DOI: https://dx.doi.org/10.1089/g4h.2017.0173]
21. Vugts, M.A.; Joosen, M.C.; Mert, A.; Zedlitz, A.M.; Vrijhoef, H.J. Effectiveness of Serious Gaming during the Multidisciplinary Rehabilitation of Patients with Complex Chronic Pain or Fatigue: Natural Quasi-Experiment. J. Med. Internet Res.; 2018; 20, e250. [DOI: https://dx.doi.org/10.2196/jmir.9739]
22. Gonçalves, R.S.; Krebs, H.I. MIT-Skywalker: Considerations on the Design of a Body Weight Support System. J. NeuroEng. Rehabil.; 2017; 14, 88. [DOI: https://dx.doi.org/10.1186/s12984-017-0302-6]
23. Rodrigues, L.A.; Gaspar, L.A.; Gonçalves, R.S. Serious Games Integrated with Rehabilitation Structures and Assist-as-Need Techniques. Handbook of Research on Promoting Economic and Social Development through Serious Games; Bernardes, O.; Amorim, V. IGI Global: Hershey, PA, USA, 2022; pp. 191-220. [DOI: https://dx.doi.org/10.4018/978-1-7998-9732-3.ch010]
24. Gonçalves, R.S.; Brito, L.S.F.; Moraes, L.P.; Carbone, G.; Ceccarelli, M. A fairly simple mechatronic device for training human wrist motion. Int. J. Adv. Robot. Syst.; 2020; 17, pp. 1-15. [DOI: https://dx.doi.org/10.1177/1729881420974286]
25. Gonçalves, R.S.; Hamilton, T.; Krebs, H.I. MIT-Skywalker: On the use of a markerless system. Proceedings of the 2017 International Conference on Rehabilitation Robotics (ICORR); London, UK, 17–20 July 2017; pp. 205-210. [DOI: https://dx.doi.org/10.1109/ICORR.2017.8009247]
26. Washabaugh, E.P.; Shanmugam, T.A.; Ranganathan, R.; Krishnan, C. Comparing the accuracy of open-source pose estimation methods for measuring gait kinematics. Gait Posture; 2022; 97, pp. 188-195. [DOI: https://dx.doi.org/10.1016/j.gaitpost.2022.08.008] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35988434]
27. Silva LJ, S.; da Silva DL, S.; Raposo, A.B.; Velho, L.; Lopes, H.C.V. Tensorpose: Real-time pose estimation for interactive applications. Comput. Graph.; 2019; 85, pp. 1-14. [DOI: https://dx.doi.org/10.1016/j.cag.2019.08.013]
28. Matínez, G.H. OpenPose: Whole-Body Pose Estimation; Technical Report Number: CMU-R1-TR-19-15 Robotics Institute, Carnegie Mellon University: Pittsburgh, PA, USA, 2019.
29. Kim, W.; Sung, J.; Saakes, D.; Huang, C.; Xiong, S. Ergonomic postural assessment using a new open-source human pose estimation technology (OpenPose). Int. J. Ind. Ergon.; 2021; 84, 103164. [DOI: https://dx.doi.org/10.1016/j.ergon.2021.103164]
30. Stenum, J.; Rossi, C.; Roemmich, R.T. Two-dimensional video-based analysis of human gait using pose estimation. PLoS Comput. Biol.; 2021; 17, e1008935. [DOI: https://dx.doi.org/10.1371/journal.pcbi.1008935] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33891585]
31. Fan, J.; Gu, F.; Lv, L.; Zhang, Z.; Zhu, C.; Qi, J.; Wang, H.; Liu, X.; Yang, J.; Zhu, Q. Reliability of a human pose tracking algorithm for measuring upper limb joints: Comparison with photography-based goniometry. BMC Musculoskelet Disord.; 2022; 23, 877. [DOI: https://dx.doi.org/10.1186/s12891-022-05826-4] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36131313]
32. Barbosa, M.B.; Rego, A.B.; Medeiros, I. Developing games with object composition: A case study using the Unity 3D platform. SBC–Proc. SBGames; 2015; 1, pp. 240-243.
33. Lam, O.T.; Strenger, D.M.; Chan-Fee, M.; Pham, P.T.; Preuss, R.A.; Robbins, S.M. Effectiveness of the McKenzie Method of Mechanical Diagnosis and Therapy for Treating Low Back Pain: Literature Review with Meta-analysis. J. Orthop. Sport. Phys. Ther.; 2018; 48, pp. 476-490. [DOI: https://dx.doi.org/10.2519/jospt.2018.7562]
34. Sukmajaya, W.P.; Alkaff, F.F.; Oen, A.; Sukmajaya, A.C. Williams Flexion Exercise for Low Back Pain: A Possible Implementation in Rural Areas. Open Access Maced. J. Med. Sci.; 2020; 8, pp. 1-5. [DOI: https://dx.doi.org/10.3889/oamjms.2020.3988]
35. Ceccarelli, M.; Bottin, M.; Russo, M.; Rosati, G.; Laribi, M.A.; Petuya, V. Requirements and Solutions for Motion Limb Assistance of COVID-19 Patients. Robotics; 2022; 11, 45. [DOI: https://dx.doi.org/10.3390/robotics11020045]
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Rodrigo Alves Prado 1 ; Guênia Mara Vieira Ladeira 1 ; Andréa Licre Pessina Gasparini 2 1 School of Mechanical Engineering, Federal University of Uberlândia, Uberlândia 38408-100, Brazil
2 Department of Applied Physical Therapy, Federal University of Triangulo Mineiro, Uberaba 38025-180, Brazil