Design of a Cultural Heritage Gesture-Based Puzzle Game and Evaluation of User Experience

Abstract

The Haptic Puzzle was a 3D gesture-based puzzle game developed to be deployed in a museum of ethnology. The Haptic Puzzle was designed according to the conceptual model for puzzle games, which was based on the four-dimensional framework. In this study, we explored the development process of the Haptic Puzzle, providing insight on the manner in which the game was designed and explaining the design choices made. Aiming to measure the experience perceived by the Haptic Puzzle’s users, we evaluated the Haptic Puzzle based on the user experience questionnaire and direct observations, with the involvement of 92 participants who were separated into groups of 9 to 12 or small groups of 2 to 3 according to their ages, which ranged from 10 to 15 years old. We discuss the evaluation results indicating that the Haptic Puzzle accomplished its purpose by engaging users in a creative activity while they experienced pleasant feelings and enjoyment. Moreover, we describe the challenges we faced and the manner in which they were confronted. The presented study provides directions for future work regarding the development and evaluation of cultural heritage gesture-based games for deployment in museums.

1. Introduction

Museums are very important, as they are excellent places for displaying and studying selected tangible and intangible assets of society and culture while they bring people together. The term “museum” comes from the Greek word “mouseion”, i.e., the temple of muses, who were the daughters of Zeus and Mnemosyne (i.e., memory). Museums, as defined by the International Council of Museums (ICOM) organization, are dedicated to the protection and dissemination of the world’s cultural heritage, as “a non-profit, permanent institution in the service of society and its development, open to the public, which acquires, conserves, researches, communicates and exhibits the tangible and intangible heritage of humanity and its environment for the purposes of education, study and enjoyment”. According to the ICOM’s definition [1], the mission of a museum is to provide education and enjoyment services to visitors, including elements that correspond to the properties of serious games (SGs), as SGs are games that have an educational purpose while they are not intended to be played primarily for enjoyment. Within this prism, it is evident that SGs are particularly suitable for enhancing the mission of museums, as they can unify learning and enjoyment aspects [2].

The study presented in this paper explores the HapticSOUND serious game (Haptic Puzzle) depicted in Figure 1, a digital three-dimensional (3D) puzzle that visitors assembled to create the traditional Cretan lute.

The HapticSOUND system is an innovative interactive virtual reality system for delivering personalized experiences and knowledge in an active, pleasant, and creative way. HapticSOUND was established at the Cretan Ethnology Museum (CEM) in Heraklion, on the Greek island of Crete. The design of the Haptic Puzzle was based on the conceptual model for the design of puzzle-based games proposed in [3], which specifies the main elements that must be taken into consideration when designing different types of educational puzzle games and the relationship between these elements. Such elements include the game’s objectives, the puzzle’s pieces and their relationship, and the levels of difficulty. Details on the objectives, the design and implementation, and the operation of HapticSOUND subsystems, including the Haptic Puzzle subsystem, are published on HapticSOUND’s official website (https://hapticsound.uom.gr/, accessed on 23 April 2023). The primary aim of the Haptic Puzzle was to foster a positive attitude of the users towards CEM and museums in general, by engaging them in a creative and pleasant activity. For this reason, the presented study focuses on the user experience (UX) of the Haptic Puzzle and its aspects (e.g., user satisfaction, understandability or perspicuity), as they are very important for the quality of such approaches [4]. Although there are many studies on interactive tangible activities for cultural heritage (e.g., refs. [5,6,7] on tangible interaction, ref. [8] on the development of a physical puzzle, ref. [9] on fracturing artefacts for 3D printable puzzles, ref. [10] on evaluating 3D printed museum artefacts), there is a lack of research regarding intangible cultural heritage, especially 3D digital puzzles, user interfaces of 3D puzzles [11], and the user experience perceived in such games. In this context, this paper aimed to present insight into how a 3D puzzle serious game for a museum was designed and developed, including the perceived UX of the game. The research question of this study focused on the efficiency of the perceived UX of the Haptic Puzzle, which was evaluated using a user experience questionnaire (UEQ) [12] with a group of 92 participants aged 10 to 15 years.

2. Background

2.1. The Puzzle Games Model

The puzzle game model (Figure 2) is leveraged on the four-dimensional framework [13] and is proposed for the development of different types of puzzle games, including puzzles with virtual and physical elements (e.g., pieces, slots).

The puzzle games model includes the following elements that must be taken into consideration when designing puzzle games:

• The location is the context where the game session takes place (indoors or outdoors) allowing situated experiences;

• The activities are the actions performed by the user during the game, depending on the role assumed by the user in the game;

• The gaming objectives are the outcomes the player achieves by performing the learning activities;

• The levels are different stages, phases, or rounds of the game that are usually associated with increasing challenge and difficulty of the game;

• The player role is the role assumed by the user during the game;

• The story associates the player’s role with the activities. It also specifies the time during which the activities of the game session are performed;

• The pieces and slots are the puzzle’s elements specified by a relationship;

• The score is the number of points assigned to the user;

• Feedback is provided to inform the user about the game’s state (e.g., when the puzzle’s elements are successfully associated);

• Hints scaffold the game process. They are materials provided to users to aid them in solving the puzzle.

The puzzle game model foresees that puzzle games can be adopted in different settings and manners and for various purposes, as there are many ways puzzle pieces can interact and be interrelated.

2.2. User Experience

UX is associated with several dynamic concepts (e.g., aesthetics, emotions, affection) and there are difficulties in formulating a commonly accepted definition [14]. UX is defined in [15] as a “person’s perceptions and responses resulting from the use and/or anticipated use of a product, system or service”. UX includes the user’s emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors, and accomplishments occurring before, during, and after the use of the product [15,16]. A different view of UX suggests the analysis of UX should be divided into distinct well-defined concepts that are measurable [17].

2.3. User Experience Evaluation

UX evaluation employs a set of methods and tools aimed at determining the perception about the use of a system or a product [12]. The evaluation of UX focuses on the participants’ feelings developed during interaction with the system [14]. UX evaluation is carried out during the implementation and testing phases of a product (e.g., game) by employing methods such as interviews, observation, and questionnaires [18]. However, the measuring and evaluation of UX is a difficult task, as there are various factors that can affect it [13], including the interaction technique employed in the product and the type of product (e.g., the type of game) [18].

Questionnaires are an effective and inexpensive way to conduct quantitative measures regarding a product’s UX. Standardized questionnaires can be used to evaluate UX, allowing users to describe their perception of the end product’s ease of use and whether it is original, clear, or confusing [12]. The survey presented in [19] revealed that 76% of researchers chose to perform UX evaluations with a single measurement, 36% of researchers performed UX evaluations during and after the experience, and 32% of researchers performed UX evaluations only after completion of the experience using manual methods of data collection (80%). Questionnaires were used by 84% of researchers to collect evaluation data. Moreover, researchers in [20] found that 96% of the reviewed studies assessed UX after interaction with the product, 65% of studies assessed UX during its usage, while 26% of studies used a standardized questionnaire for the evaluation of UX. After the analysis of 553 primary studies [12], researchers concluded that the evaluation method used by 213 of the studies was only a standardized questionnaire, with the most recognized standardized questionnaires for UX evaluation being AttraDiff (61.6%), UEQ (36.2%), and meCUE (2.2%). Researchers in [12] highlighted that although UEQ was a relatively new tool (created in 2008), its use in recent years was growing quickly and outpacing AttraDiff.

2.4. User Experience Questionnaire (UEQ)

The UEQ consisted of 26 items. Each item included a pair of terms with opposite meanings that could be rated on a seven-point Likert scale. Each response could therefore range from −3 to +3, indicating complete disagreement or complete agreement with the respective term. The positive and negative terms were listed in random order [21]. The validity and reliability of the questionnaire was established after many surveys [22] and in recent years it has been used to measure UX in many studies involving serious games [23,24,25,26,27,28]. In some cases of serious games assessed with UEQ, the game controller was the leap motion device [29,30,31].

The 26 items included in the UEQ were grouped into six different, but not independent, scales [21,23]:

• Attractiveness (impression of the product), items 1, 12, 14, 16, 24, 25;

• Perspicuity (ease of use, learning, understanding), items 2, 4, 13, 21;

• Efficiency (ease of task solving, fast-efficient interaction), items 9, 20, 22, 23;

• Dependability (user feels in control of the interaction, user feels safe when working with product), items 8, 11, 17, 19;

• Stimulation (exciting, motivation, fun to use), items 5, 6, 7, 18;

• Novelty (innovative, creative product), 3, 10, 15, 26.

Each scale included 4 items except for Attractiveness, which included 6 items (Figure 3a). The order of positive and negative items occurred randomly and half of them started with a positive term while the others started with a negative term [27].

According to the authors of [22], the total time needed to read the instructions and complete the questionnaire was between 3–5 min, and the questionnaire was available in 17 different languages.

3. Haptic Puzzle Game

3.1. Design

According to the classification of [32], the Haptic Puzzle was an assembly-type puzzle in which users assembled a set of parts of the traditional Cretan lute into the whole instrument. The selection of a puzzle-type game is suitable for deployment in a museum, as puzzle games have simple game mechanics [11] that are independent of the game’s content and underlying technology [3], thus users do not need familiarization time to learn how to play the game. Nevertheless, puzzle games stimulate users’ creativity, as they foster problem-solving, analytical, and memory skills [33]. The Haptic Puzzle was a gesture-based game; it utilized hand gesture recognition technology that allowed users to interact with the game’s elements with their bare hands without handling a controller. The utilization of gesture recognition technology increased the intuitiveness and naturalness of the interaction, improved the efficacy of the Haptic Puzzle, and was expected to improve the usability of the game [34].

The design of the Haptic Puzzle was based on the conceptual model for the design of puzzle-based games (puzzle games model) proposed in [3]. The remainder of this section explores the design of the Haptic Puzzle, presenting the most important elements of the puzzle game model and how they were realized in the design of the Haptic Puzzle. The elements of the model were analyzed from gaming, learning, and instructional perspectives. The gaming perspective embraced the activities performed by the user as a gamer, the learning perspective contained the activities performed by the user as a learner, and the instructional perspective included the instructions provided with the game in order to provide help, guidance, and feedback to the user.

• Location: The context in which the Haptic Puzzle was deployed was the Cretan Ethnology Museum in Heraklion, on the Greek island of Crete. The game activities took place indoors on the platform of the HapticSOUND system, which contains a single-user interface. One user was allowed to engage with the game at a time, and the presence of an instructor, curator, or museum staff was optional.

• Objectives: The gaming objective of the Haptic Puzzle was for the user to assemble the traditional Cretan lute. The learning objective of Haptic Puzzle was to make users identify the parts of the lute and to reflect on the lute’s structural model. However, the primary learning objective of the Haptic Puzzle was attitude based: it aimed to provide a motive for the users to act and engage in a creative activity while experiencing pleasant feelings and enjoyment. Thus, the Haptic Puzzle was expected to foster a positive attitude of the users towards CEM and museums in general. For this reason, the design of the Haptic Puzzle particularly focused on the UX aspect of the game and especially on its attractiveness and usability.

• Activities: The Haptic Puzzle user performed the gaming activity of moving and pairing 3D parts of the lute to align them and properly combine them to assemble the traditional Cretan lute. From the learning perspective, users reflected on the lute demonstrated in the CEM’s collection to visualize it, identify the parts of the lute among parts of other musical instruments (e.g., the traditional Cretan lyra and the violin), combine the lute parts in the appropriate manner and order, receive feedback, and verify that they remembered and comprehended the structural model of the lute.

• Story and levels: As only a single user can use the HapticSOUND system at a time, the Haptic Puzzle was constrained to be brief with an easy to follow narrative. The user assumed the role of a CEM curator who is a manufacturer of musical instruments and is responsible for reassembling a traditional Cretan lute that has been disassembled into pieces. The story contained three different difficulty levels (i.e., novice, moderate, and professional), which all shared the same gaming objective and activities (i.e., the reassembling of the lute), but with different conditions. In the moderate level, the user had to identify the lute parts among parts of other musical instruments, whereas in the professional level, the user had to identify the lute parts and perform his/her activity under the pressure of time. The user earned points each time he/she assembled a piece of the lute, and the higher level of difficulty of the game resulted in the user earning a higher number of points.

• Pieces: The Haptic Puzzle contained virtual 3D pieces representing the parts of a lute depicted in Figure 4.

• Each piece contained one or more puzzle joints invisible to the user, which were of tab (male) or blank (female, which served as puzzle slots) type. Each tab was related to a specific blank (and vice versa), forming pairs of puzzle joints, which were implemented as the attachment mechanisms of the puzzle. The pairs of joints were associated with a precedence order. Once the user identified a match, he/she brought together two lute pieces so that their joints collided, and if the joints were a pair with the appropriate precedence order, then the game magnetized these two pieces, which were joined by placing one joint on the position of the other along with the pieces of the lute.

• Scaffolding: The Haptic Puzzle contained a scaffolding feature that could be activated by the user. The Haptic Puzzle’s scaffolding feature included the presentation of instructions to the user and exhibition of a model of a miniature lute to aid the user’s activities.

• Feedback: The Haptic Puzzle involved a multimodal feedback feature including impressive animations, sounds, color emissions, and score assignments.

3.2. Architecture

Figure 5 depicts the main components of the Haptic Puzzle, which are described in the remainder of the section.

• Game Manager was the core component of the game:

  • ○. orchestrates the creation of the game session;

  • ○. manages the settings of the game (e.g., the levels of difficulty);

  • ○. manages the data generated during a game session;

  • ○. communicates with the database to insert and query data.

• Database stored data related to the game’s sessions (e.g., date of the session, duration, pieces attached successfully, scores), the nickname proposed by the user, the feedback provided by the user, and the configuration options managed by the game’s administrator.

• User Interface contained a gesture recognition device (i.e., the leap motion controller) and the game’s menu. The leap motion controller is a compact computer peripheral device that detects the position of hands and fingers at a high rate and precision and identifies the user’s gestures in real time. Thus, the user interface captured and interpreted the gestures performed by the user in the space in front of the leap motion controller, and it allowed the user to perform the game activities and access the components listed in the remainder of this section.

• Puzzle Pieces referred to a set of 3D objects representing the parts of the lute. The puzzle pieces included the puzzle joints (i.e., the tabs and blanks presented in Section 3.1), and the C# scripts for manipulation of the pieces and the triggering of collisions when the joints met.

• User Feedback allowed the user to submit feedback related to his/her experience with the Haptic Puzzle. Users could grade their experience in the game on a scale of 1 to 5 and enter comments by utilizing the game’s virtual keyboard (Figure 6). Feedback was stored in the Database component.

• Leaderboard presented the scores and nicknames of the users who performed well in the game in the last day, month, year, and of all-time.

• Feedback was continually provided to the user to help him/her realize the results of the performed actions and the state of the game. For example, the menu options changed color and position when they were pressed, and the lute’s parts emitted colors when they were grabbed or touched (Figure 7).

• Help provided support to users by presenting instructions and a miniature lute. When the user selected a puzzle piece that corresponded to a lute’s part (e.g., the lute’s block or bridge depicted in Figure 4), the corresponding part of the miniature lute was highlighted (Figure 8) to provide a hint to the user that the selected lute’s part could be combined with the other lute parts. Additionally, through the help component, users could activate multiple cameras on the game scene, which allowed them to have multiple views of the puzzle pieces, resulting in a better appreciation of the lute.

• Configuration allowed the user to choose a nickname and difficulty level.

• Administrator was accessible through the mouse and keyboard and allowed the administrator of the game (e.g., museum employees) to review and delete the data stored in the back-end (e.g., user comments, scores) and configure the game (e.g., change the number of pieces of the Puzzle Pieces component).

4. Implementation

The Haptic Puzzle was implemented using the Unity 3D game development engine with C# as the programming language. The Haptic Puzzle included a back-end storage facility (i.e., SQLite database), which stored usernames, statistics, and feedback provided by the users. The Haptic Puzzle’s implementation included the elaboration of a realistic 3D environment, creation of the user interface, development of several C# scripts, and generation of 3D models representing the parts of the lute. For the generation of the 3D models, the workflow presented in [11] was followed, which included 3D scanning of the museum’s lute, editing of the lute’s 3D model in Blender software, and generation of the 3D lute’s parts. Blender software was used to reduce the total number of polygons (i.e., the primary shapes of the 3D model’s surface) of the 3D lute’s parts for marking the seams (depicted in Figure 9) and unwrapping the faces of every piece. Finally, the puzzle pieces were completed with the attachment of puzzle joints on the 3D lute’s parts.

The environment of the Haptic Puzzle was developed as an extension of the CEM building, i.e., the CEM warehouse. The look and feel of the environment was designed with respect to the museum’s interior appearance. The warehouse contained various objects and artefacts, which were not demonstrated in the museum’s main collection. In this context, the user played the role of the museum’s curator who had to reassemble a disassembled lute. To play the Haptic Puzzle, the user accessed the main menu, initiated a new game session, and chose a nickname and difficulty level. Through the main menu (Figure 10), the user could also read the game’s narrative, see the scoreboards (Leaderboards component), provide feedback regarding his/her experience in the game (User’s Feedback component), and read instructions about how the game is played (Help component). While the user chose items from the main menu, the game’s camera moved in the warehouse and provided the impression to the user that he/she was exploring the warehouse. In such a way, the user explored the 3D graphics of the game.

During the game session, the user could use their hands to select the lute’s parts and move and rotate the lute’s parts in order to assemble the lute on its base. When the user grabbed a piece, the color of the piece changed to indicate that the specific part of the lute was selected. The user had to assemble the pieces in the appropriate order, which reflected the order in which a real lute is assembled during its manufacture. For example, initially, the back (or bowl) must be placed on the lute’s base, and the neck has to be attached on the bowl before the chords are attached on the neck. When a piece was attached on its paired piece, an animation and sound were played to notify the user that the attachment was successfully performed. Then, the user earned points with respect to the difficulty level of the game session. During the game session, the user could access the game menu and reset or end the session, reconfigure the game, and turn on the scaffolding features of the game (presented in Section 3.1). The game session ended when the user assembled the lute, the time provided to the user ended, or the game was idle for more than 3 min.

5. Evaluation

Researchers in [35] presented a systematic literature review in which they identified three types of serious games evaluation: simple, pre/post, and pre/post/post. They observed that the simple procedure was the most used. Usually, this procedure starts with a game session followed by an evaluation mechanism provided to the players.

For the evaluation of the Haptic Puzzle, the simple serious game evaluation type was adopted and the user experience questionnaire (UEQ) was implemented [21]. A setting was created in accordance with the specifications of the HapticSOUND system. Specifically, a large display was used in a distance of 2.5 m from the users and 1.5 m from the ground. The leap motion controller was placed at a height of 80 cm from the ground. A total of 92 participants volunteered to evaluate the Haptic Puzzle, including 45 boys and 47 girls with ages ranging from 10 to 15 years. The evaluation involved 92 game sessions that were conducted in 10 evaluation sessions. Each session contained a group of participants of similar age and a researcher who instructed and observed the session and ensured its smooth operation. The participants’ groups were either large (9 to 12 participants) or small (2 to 3 participants). The evaluation was narrowed to the ages of 10 to 15, as CEM is very often visited by primary and secondary schools, and researchers aimed at uniformity in the ages of the participants.

In the beginning of the evaluation sessions, participants observed the traditional Cretan lute and familiarized themselves with the game’s interface. Then, they initiated a game session in the novel difficulty level, and they tried to assemble the lute in the shortest time possible. The rest of the participants watched the game from a short distance, just as would happen in the real conditions of the museum. During the evaluation sessions, researchers watched the game sessions and recorded their observations focusing on the difficulties faced by the participants. At the end of an evaluation session, participants were provided with the Greek version of the UEQ in an online single page Google form and they were instructed to answer the UEQ, which contained 26 seven-point Likert questions scaled from −3 (horribly bad) to +3 (extremely good) (Figure 3). Researchers had to provide clarifications regarding the meaning of some UEQ terms, especially to young participants (10 to 12 years old). The UEQ terms that required clarification included the word “inferior” in question 5, the words “inventive” and “conventional” in question 10, and the word “obstructive” in question 11. Thereafter, researchers combined and grouped the observations they recorded during the evaluation sessions and classified them into categories. These categories included the location and orientation of the navigation menus, the assembling process, the feedback players received, and finally, some general comments.

The evaluation sessions were performed over a period of two weeks, apart from the first session which preceded the rest of the evaluation sessions by 10 days. This was because the first evaluation session was a pilot session, in which researchers reviewed the evaluation process, identified critical issues of the Haptic Puzzle, and reviewed the game’s aspects (discussed in Section 7) to optimize the perceived UX. During the evaluation sessions, 82 responses were collected and analyzed using the UEQ Excel tool (XX), which aided in the visualization and interpretation of the collected data. The responses collected during the pilot session were excluded from the analysis, as the version of the Haptic Puzzle used in the pilot session was not final.

6. Results

Figures presented in this section are derived from the UEQ Excel tool presenting the evaluation results of the six UX aspects that were measured according to the UEQ. According to the UEQ tool, 21 out of the 82 responses were invalid, and thus they were excluded from the study. The evaluation results in Figure 11 show that all UX aspects of the Haptic Puzzle had a mean value above the neutral value of 0.8. The benchmark of the UEQ (Figure 12a) showed that the Attractiveness aspect was excellent, the Stimulation and Novelty aspects were good, the Perspicuity and Efficiency aspects were above average, and the Dependability aspect was below average. The Cronbach’s alpha coefficient (Figure 12b) showed good internal consistency (i.e., consistency from 0.8 to 0.9) for Attractiveness; acceptable internal consistency (i.e., consistency from 0.7 to 0.8) for Perspicuity, Efficiency, and Stimulation; and poor internal consistency (i.e., consistency from 0.5 to 0.6) for Dependability and Novelty.

On average, each participant spent approximately 3 min (179 s) playing the Haptic Puzzle, with a standard deviation of approximately 1.5 min (95 s), while 98% of players managed to solve the puzzle.

7. Discussion

The evaluation results of the Haptic Puzzle’s UX showed that the game encouraged users to engage in a puzzle-solving activity while it fostered pleasant feelings and enjoyment. Specifically, participants of the evaluation sessions stated in the UEQ that they had an excellent overall impression of the Haptic Puzzle (related to the Attractiveness UX aspect), while they indicated it as an interesting, creative, exciting, and motivating game (related to the Stimulation and Novelty UX aspects). Nevertheless, it was also stated that the Haptic Puzzle had sufficient perspicuity and efficiency associated with the understandability and practicality of the game (e.g., ease of task solving, fast and efficient interaction). On the other hand, participants stated that the Dependability UX aspect, associated with the predictability and supportiveness of the game, was below average.

The evaluation results complied with the researchers’ observations. Participants seemed to enjoy and were stimulated by the Haptic Puzzle, especially by the gesture-based interface, the look and feel, and the 3D model of the game. More importantly, it was evident that participants were excited to use the emerging technology of gesture recognition in a game, especially by using a compact device such as the leap motion controller.

An issue that was very likely to have raised the number of inconsistent answers to the UEQ questions was that approximately 25% of the participants (i.e., mainly the younger participants aged 10 to 12 years old) had difficulties in comprehending a few terms in the UEQ (stated in Section 5). Thus, in such cases, the UEQ terms should be explained to the participants prior to the evaluation sessions or a kid-friendly version of UEQ should be elaborated and used.

7.1. Gesture Recognition Technology Considerations

Since participants were not familiar with the gesture recognition technology, they had difficulties in grabbing and pinching the lute’s parts especially in the beginning of the game sessions. The researchers observed that many of the participants did not place their hands in the proper range above the leap motion controller or they moved their hands very fast, and thus the controller could not successfully interpret the users’ gestures. However, most participants realized the proper usage of the controller after a short period of time, and they managed to solve the puzzle. Approximately 30% of participants, especially users who performed the first game sessions during the evaluation sessions, struggled in making the appropriate choices in the Haptic Puzzle’s menu and in a few cases, they accidentally chose a different menu option than the one they intended to make.

To move a lute’s part, participants had to touch it, perform a grab or pinch gesture, and move their hand. However, in many cases, participants did not realize when a lute’s part was touched in order to initiate a grab or pinch gesture. For this reason, an additional feedback feature had to be added: the lute’s parts had an additional color emission activated when the user touched a lute’s part (Figure 13). In such a way, the user was notified by the game about the exact time he/she could initiate a grab or pinch gesture, which significantly improved the dependability of the game.

As participants in some cases struggled to make the touch and grab actions, colliders of the lute’s parts were significantly increased to make the interaction easier (i.e., the green box depicted in Figure 14). Colliders in Unity 3D provide collision detection, i.e., detection when the object of the user’s hand touched the object of the lute’s part.

7.2. User Interface Design Considerations

In the pilot evaluation session, researchers observed that the Haptic Puzzle menu could be accessed by touching the “show menu” button (menu button) with one hand. The menu button was placed in the bottom center of the screen to be easily accessible for left-handed users, right-handed users, or kids that were shorter than the average user. However, participants unintentionally activated the game menu several times when they accidentally placed their hands on the menu button, and they interrupted the game flow. For this reason, the researchers made the menu button pressable only by two hands, which solved the issue.

Likewise, the layout of the buttons on the game menu had to change from grid to arc (Figure 15) because participants in their effort to activate the options located at the top of the menu inadvertently activated one of the options located in the bottom row.

8. Conclusions

Serious games are particularly suitable for enhancing the mission of cultural heritage museums, as they can ideally unify learning and enjoyment aspects. Puzzle games are suitable to be deployed in museums, as they are independent of the underlying technology and users do not require time to familiarize themselves with the game rules. In this study, we examined the manner in which the Haptic Puzzle was designed and implemented. The Haptic Puzzle was a 3D gesture-based puzzle game in which the user assembles a traditional musical instrument (i.e., the Cretan lute). The Haptic Puzzle was deployed in the Cretan Ethnology Museum as a part of the HapticSOUND virtual reality system. To design the Haptic Puzzle, we adopted the puzzle game model (a conceptual model proposed in [3]) as a guide, and we presented how the model’s elements were interpreted in the structure of the game. Additionally, we presented the evaluation of the Haptic Puzzle user experience, which required the organization of multiple evaluation sessions, direct observation of the sessions, and utilization of the UEQ questionnaire. The results of the Haptic Puzzle’s UX evaluation showed that the game accomplished its mission of encouraging users to engage in a creative activity, as most of the participants who took part in the evaluation solved the puzzle while exercising their problem-solving and analytical skills. More importantly, the evaluation results showed that the Haptic Puzzle accomplished its objective of evoking pleasant feelings, excitement, and enjoyment in users while they performed a productive activity. By reflecting on our experience during the development of the Haptic Puzzle and by interpreting the evaluation results and the observations made during the evaluation sessions, we presented and discussed aspects of the Haptic Puzzle’s design and implementation, which are important considerations for the development of gesture-based and puzzle games. These considerations aim to optimize the perceived UX of these games with respect to users’ interactions with the games’ objects and interfaces and the feedback such games provide to the users.

Author Contributions

Conceptualization, M.N.K., A.M. and I.M.; Methodology, M.N.K., A.M. and I.M.; Software, M.N.K. and A.M.; Validation, M.N.K. and A.M.; Writing—original draft, M.N.K., A.M. and I.M.; Writing—review & editing, M.N.K., A.M. and I.M.; Visualization, M.N.K. and A.M.; Supervision, I.M. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures

Figure 1. The Haptic Puzzle.

Figure 2. Graphical representation of the conceptual model for puzzle game design [3]. Asterisk (*) represents multiplicity.

Figure 3. The UEQ questionnaire: (a) in English; (b) in Greek.

Figure 4. The lute’s pieces: 1—left pegs, 2—right pegs, 3—neck, 4—frets, 5—strings, 6—back or bowl, 7—top, table, or soundboard, 8—rosette, 9—block or bridge, and 10—endclasp.

Figure 5. Haptic Puzzle’s architecture.

Figure 6. Haptic Puzzle’s virtual keyboard.

Figure 7. Color emission on grab.

Figure 8. Color emission on grab.

Figure 9. Seams of the lute’s neck.

Figure 10. Main menu.

Figure 11. UEQ Results.

Figure 12. (a) UEQ benchmarks; (b) Cronbach’s alpha coefficient values.

Figure 13. Color emission (a) on touch; (b) on grab.

Figure 14. Increased box colliders of the top part of the lute.

Figure 15. The main menu: (a) grid layout; (b) arc layout.

References

1. Desvallées, A.; Mairesse, F. Key Concepts of Museology, International Councils of Museums, Armand Colin. 2010; Available online: https://icofom.mini.icom.museum/wp-content/uploads/sites/18/2022/01/2010_key_concepts_of_museology_english.pdf (accessed on 27 March 2023).

2. Doukianou, S.; Daylamani-Zad, D.; Paraskevopoulos, I. Beyond Virtual Museums: Adopting Serious Games and Extended Reality (XR) for User-Centred Cultural Experiences. Visual Computing for Cultural Heritage; Liarokapis, F.; Voulodimos, A.; Doulamis, N.; Doulamis, A. Springer Series on Cultural Computing; Springer International Publishing: Cham, Switzerland, 2020; pp. 283-299. ISBN 978-3-030-37191-3

3. Melero, J.; Hernández-Leo, D. A Model for the Design of Puzzle-Based Games Including Virtual and Physical Objects. J. Educ. Technol. Soc.; 2014; 17, pp. 192-207.

4. Abdellatif, A.J.; McCollum, B.; McMullan, P. Serious Games: Quality Characteristics Evaluation Framework and Case Study. Proceedings of the 2018 IEEE Integrated STEM Education Conference (ISEC); Princeton, NJ, USA, 10 March 2018; pp. 112-119.

5. Hornecker, E.; Buur, J. Getting a grip on tangible interaction: A framework on physical space and social interaction. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; Montréal, QC, Canada, 22–27 April 2006; pp. 437-446.

6. Cafaro, F.; Lyons, L.; Antle, A.N. Framed guessability: Improving the discoverability of gestures and body movements for full-body interaction. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems; Montreal, QC, Canada, 21–26 April 2018; pp. 1-12.

7. Claisse, C.; Ciolfi, L.; Petrelli, D. Containers of Stories: Using co-design and digital augmentation to empower the museum community and create novel experiences of heritage at a house museum. Des. J.; 2017; 20, pp. S2906-S2918. [DOI: https://dx.doi.org/10.1080/14606925.2017.1352801]

8. Ponimin, M. Improving Children’s Love to Local Culture Through Culture-Themed “APE” Puzzle Product. Proceedings of the International Conference on Learning Innovation (ICLI 2017); Malang, Indonesia, 4–5 October 2017; Atlantis Press: Amsterdam, The Netherlands, 2017; pp. 161-166.

9. Echavarria, K.R.; Samaroudi, M.; Weyrich, T. Fracturing artefacts into 3D printable puzzles to enhance audience engagement with heritage collections. J. Comput. Cult. Herit. (JOCCH); 2020; 13, pp. 1-22. [DOI: https://dx.doi.org/10.1145/3351343]

10. Wilson, P.F.; Stott, J.; Warnett, J.M.; Attridge, A.; Smith, M.P.; Williams, M.A. Evaluation of touchable 3D-printed replicas in museums. Mus. J.; 2017; 60, pp. 445-465. [DOI: https://dx.doi.org/10.1111/cura.12244]

11. Echavarria, K.R.; Samaroudi, M.; Lloyd, J.; Weyrich, T. Interactive 3D Artefact Puzzles to Support Engagement Beyond the Museum Environment. Eurographics Workshop on Graphics and Cultural Heritage; The Eurographics Association: Reims, France, 2021.

12. Díaz-Oreiro, I.; López, G.; Quesada, L.; Guerrero, L.A. Standardized Questionnaires for User Experience Evaluation: A Systematic Literature Review. Proceedings; 2019; 31, 14. [DOI: https://dx.doi.org/10.3390/proceedings2019031014]

13. de Freitas, S.; Oliver, M. How Can Exploratory Learning with Games and Simulations within the Curriculum Be Most Effectively Evaluated?. Comput. Educ.; 2006; 46, pp. 249-264. [DOI: https://dx.doi.org/10.1016/j.compedu.2005.11.007]

14. Vermeeren, A.P.O.S.; Law, E.L.-C.; Roto, V.; Obrist, M.; Hoonhout, J.; Väänänen-Vainio-Mattila, K. User Experience Evaluation Methods: Current State and Development Needs. Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries; Reykjavik, Iceland, 16–20 October 2010; Association for Computing Machinery: New York, NY, USA, 2010; pp. 521-530.

15. ISO. ISO 9241-210:2019. Available online: https://www.iso.org/standard/77520.html (accessed on 27 March 2023).

16. Law, E.L.-C.; Roto, V.; Hassenzahl, M.; Vermeeren, A.P.O.S.; Kort, J. Understanding, Scoping and Defining User Experience: A Survey Approach. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; Boston, MA, USA, 4–9 April 2009; Association for Computing Machinery: New York, NY, USA, 2009; pp. 719-728.

17. Schrepp, M.; Hinderks, A.; Thomaschewski, J. Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios. Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience; Marcus, A. Springer International Publishing: Cham, Switzerland, 2014; pp. 383-392.

18. Bernhaupt, R. User Experience Evaluation in Entertainment. Evaluating User Experience in Games; Springer: London, UK, 2010; pp. 3-7. ISBN 978-1-84882-962-6

19. Maia, C.L.B.; Furtado, E.S. A Systematic Review about User Experience Evaluation; Marcus, A. Springer International Publishing: Cham, Switzerland, 2016; 9746, pp. 445-455.

20. Pettersson, I.; Lachner, F.; Frison, A.-K.; Riener, A.; Butz, A. A Bermuda Triangle? A Review of Method Application and Triangulation in User Experience Evaluation. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems; Montreal, QC, Canada, 21–26 April 2018; Association for Computing Machinery: New York, NY, USA, 2018; pp. 1-16.

21. Schrepp, M.; Hinderks, A.; Thomaschewski, J. Construction of a Benchmark for the User Experience Questionnaire (UEQ). Int. J. Interact. Multimed. Artif. Intell.; 2017; 4, pp. 40-44. [DOI: https://dx.doi.org/10.9781/ijimai.2017.445]

22. Schulze, K.; Krömker, H. A Framework to Measure User Experience of Interactive Online Products. Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research; Eindhoven, The Netherlands, 24–27 August 2010; Association for Computing Machinery: New York, NY, USA, 2010; pp. 1-5.

23. Ali, Q.; Heldal, I.; Helgesen, C.G.; Dæhlen, A. Serious Games for Vision Training Exercises with Eye-Tracking Technologies: Lessons from Developing a Prototype. Information; 2022; 13, 569. [DOI: https://dx.doi.org/10.3390/info13120569]

24. Ferreira, B.; Menezes, P. An Adaptive Virtual Reality-Based Serious Game for Therapeutic Rehabilitation. Int. J. Online Biomed. Eng. IJOE; 2020; 16, pp. 63-71. [DOI: https://dx.doi.org/10.3991/ijoe.v16i04.11923]

25. Heldal, I.; Helgesen, C.; Ali, Q.; Patel, D.; Geitung, A.B.; Pettersen, H. Supporting School Aged Children to Train Their Vision by Using Serious Games. Computers; 2021; 10, 53. [DOI: https://dx.doi.org/10.3390/computers10040053]

26. Maskeliunas, R.; Damasevicius, R.; Paulauskas, A.; Ceravolo, M.G.; Charalambous, M.; Kambanaros, M.; Pampoulou, E.; Barbabella, F.; Poli, A.; Carvalho, C.V. Deep Reinforcement Learning-Based ITrain Serious Game for Caregivers Dealing with Post-Stroke Patients. Information; 2022; 13, 564. [DOI: https://dx.doi.org/10.3390/info13120564]

27. Torres, A.; Kapralos, B.; Da Silva, C.; Peisachovich, E.; Dubrowski, A. Moirai: A No-Code Virtual Serious Game Authoring Platform. Virtual Worlds; 2022; 1, pp. 147-171. [DOI: https://dx.doi.org/10.3390/virtualworlds1020009]

28. Baluz, R.; Teles, A.; Fontenele, J.E.; Moreira, R.; Fialho, R.; Azevedo, P.; Sousa, D.; Santos, F.; Bastos, V.H.; Teixeira, S. Motor Rehabilitation of Upper Limbs Using a Gesture-Based Serious Game: Evaluation of Usability and User Experience. Games Health J.; 2022; 11, pp. 177-185. [DOI: https://dx.doi.org/10.1089/g4h.2022.0005] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35294849]

29. Peng, C.; Hansberger, J.; Shanthakumar, V.A.; Meacham, S.; Blakley, V.; Cao, L. A Case Study of User Experience on Hand-Gesture Video Games. Proceedings of the 2018 IEEE Games, Entertainment, Media Conference (GEM); New Haven, CT, USA, 18–21 June 2019; pp. 453-457.

30. Hoppe, A.H.; Otto, L.; van de Camp, F.; Stiefelhagen, R.; Unmüßig, G. QVRty: Virtual Keyboard with a Haptic, Real-World Representation. HCI International 2018—Posters’ Extended Abstracts; Stephanidis, C. Springer International Publishing: Cham, Switzerland, 2018; pp. 266-272.

31. Viola, E.; Solari, F.; Chessa, M. Small Objects Manipulation in Immersive Virtual Reality. Proceedings of the 6th International Conference on Human Computer Interaction Theory and Applications; Online, 6–8 February 2022; pp. 233-240.

32. De Kegel, B.; Haahr, M. Procedural Puzzle Generation: A Survey. IEEE Trans. Games; 2020; 12, pp. 21-40. [DOI: https://dx.doi.org/10.1109/TG.2019.2917792]

33. Huang, O.W.S.; Cheng, H.N.H.; Chan, T.-W. Number Jigsaw Puzzle: A Mathematical Puzzle Game for Facilitating Players’ Problem-Solving Strategies. Proceedings of the 2007 First IEEE International Workshop on Digital Game and Intelligent Toy Enhanced Learning (DIGI℡’07); Banff, AB, Canada, 26–28 March 2007; pp. 130-134.

34. Masurovsky, A.; Chojecki, P.; Runde, D.; Lafci, M.; Przewozny, D.; Gaebler, M. Controller-Free Hand Tracking for Grab-and-Place Tasks in Immersive Virtual Reality: Design Elements and Their Empirical Study. Multimodal Technol. Interact.; 2020; 4, 91. [DOI: https://dx.doi.org/10.3390/mti4040091]

35. Calderón, A.; Ruiz, M. A Systematic Literature Review on Serious Games Evaluation: An Application to Software Project Management. Comput. Educ.; 2015; 87, pp. 396-422. [DOI: https://dx.doi.org/10.1016/j.compedu.2015.07.011]