Methodology of 3D Scanning of Intangible Cultural Heritage—The Example of Lazgi Dance

Abstract

Traditional dance is one of the key elements of Intangible Culture Heritage (ICH). Many scientific papers concern analysis of dance sequences, classification and recognition of movements, making ICH data public, creating and visualising 3D models or software solutions for learning folklore dances. These works make it possible to preserve this disappearing art. The aim of this article is to propose a methodology for scanning folklore dances. The methodology was developed on the basis of capturing 3D data via an optical motion capture system with a full body Plug-in Gait model that allows for kinematic and kinetic analysis of motion sequences. An additional element of this research was the development of a hand model with which it is possible to precisely analyse the fingers, which play a significant role in many dances. The present methodology was verified on the basis of the Lazgi dance, included in the UNESCO ICH list. The obtained results of movement biomechanics for the dance sequence and the angles of the fingers indicate that it is universal and can be applied to dances that involve the upper and lower body parts, including hand movements.

1. Introduction

According to UNESCO definition, performing arts, including dance, are an example of Intangible Culture Heritage (ICH), among such other activities as: oral traditions, expressions (e.g., language, storytelling), social practices, rituals, festive events and traditional crafts [1].

Folk dances were created over the centuries based on national tradition and ideology. They represent the dominant and distinctive musical and movement characteristics. Through the influence of the national culture, dance reflects awareness of national distinctiveness. In other words, dance maintains social cohesion along with cultural diversity [2]. In the modern world the art of folk dancing is disappearing. The style of dance passing from generation to generation is fragile, even haphazard [3]. That is why the preservation of folk dances plays such an important role in ICH. The accurate recognition of these moves is a challenging task.

The dynamic development of three-dimensional (3D) technology allows both to capture and protect this type of threatened heritage. 3D dance scanning is the process of capturing real-world dance in order to create a 3D model reflecting its shape and moves. Nowadays there are many sophisticated devices that allow dance acquisition, e.g., motion capture systems. The optical ones based on retro-reflecting markers attached directly to the human body or to a special suit allow capturing a dancer’s silhouette. A similar way of movement acquisition can be done with the use of active-markers based on LED technology that enable capturing movements indoors and outdoors. The capturing of dancers may also be obtained using various sensors, such as Kinect or Xsens. The 3D data collected is further used for detailed analysis or displaying the moves in various software (e.g., for developing Virtual Reality). This process is also gaining further understanding from ICH.

Lazgi is an ancient dance created by the inhabitants of the downstream areas of the Amu-Daria River. Its history is closely related to the ancient land of Khorezm, which covers the area south of the Aral Sea. There is an opinion that the ancient meaning of the word “lazgi” is trembling, characterised by vigorous body movement and facial expression [4], which very vividly reflects the characteristics of the dance. There are many legends and myths associated with the rise of the dance [4]. The most widespread is the legend of the soul-body connection thanks to the rhythms of Lazgi. After the act of creating the human body, it could not be brought to life because of the soul’s reluctance to enter the body. The soul’s attitude changed when it heard the melody of the Lazgi dance and began to slowly penetrate the body. Therefore, Lazgi starts with a leisurely introduction, then the dancer takes a position with a hand raised to the sun. Next begins a sequence of movements involving the fingers, wrists, arms, torso and legs. The whole body starts to move by a combination of turns and swings and the rhythm of the dance gradually speeds up [5]. Despite the emergence of numerous wars, natural changes, disasters and social unrest, the Lazgi dance has survived for thousands of years. In 2019 it was inscribed on the Representative List of the Intangible Cultural Heritage of Humanity UNESCO [6].

The aim of the paper is to develop a universal dance scanning method using the motion capture system with the biomechanical Plug-in Model. The retro-reflecting markers are attached to a special suit according to this model in order to capture the dancer’s movements. Additionally, the markers were attached to the dancer’s hands in order to capture hand gesticulation. The created hand model is dedicated for capturing a type of dance where the analysis of hand and finger movements is essential. The proposed methodology consists of the following steps: planning the research, motion capture session preparation and implementation and, finally, data post-processing. The analysis of the dance, which verified the proposed methodology, involved indicating the trajectories for the selected parts of the upper and lower body and fingers, as well as changes in angles.

2. Traditional Dance Studies—Literature Overview

There are many traditional dances in the literature study that have been analysed in various aspects of ICH research. The detailed results were gathered in Table 1.

The most popular methods for 3D capturing are divided into two main categories: active methods (laser scanners, range finders, structured light projectors) and passive methods (stereo vision and visual hulls) [22]. The technologies for traditional dance capturing are presented in Table 2.

Five main aspects can be distinguished in the research: analysis of selected dance sequences, classification and pattern recognition, visualization and software solutions, description and verification of methodology and creation databases.

2.1. Analysis of Selected Dance Sequences

The first type of research concerns the analysis of trajectories obtained from motion capture systems, which show the dance rhythm and body kinematics and kinetics parameters. It allows for a better understanding of performance of the dance [28].

In [14] the studies concerning choreographic sequences analysis were performed based on 3D motion capture data. They involved trajectories of hip and foot joints showing the rhythm and periodicity of Greek dance. The average Silhouette value over all frames were also indicated for the particular dancers. In [16] the analysis of hip, knee and ankle joint angles was performed. Male dancers had narrower angles in all the mentioned joints of all the six steps than female dancers during the implementation of the Greek Syrtos dance. Additionally, in [15], the dance sequences were analysed in terms of average precision and recall values for different frequency. The identification of the Syrtos dance based on rhythm is also presented in [17]. The paper in [39] highlights a scientific approach to capture and analyse the biomechanical data of authentic traditional dance motions with the aim to digitally preserve their disappearing inherent rhythmic dying features for future training and coaching purposes. Inertial Measurement Unit (IMU)-based sensors were used to capture three-dimensional movement patterns of a professional traditional Ayala dancer. In [18], the choreography of the same Greek dance was analysed. Optimal key points were distinguished from 3D trajectories that best represented the choreography. These few frames are suitable for modelling an entire choreography, which significantly reduced the amount of information required for processing and storage. It is a particularly important aspect in modelling, analysing and storing this type of data. A Bayesian optimised, bi-directional Long Short-Term Memory (LSTM) network for pose characterisation of a choreography for Greek dance was proposed in [20]. In [21], the identification of dance choreographies based on skeleton data points was used for data obtained from heterogeneous motion capture systems (Vicon motion capture system and Kinect sensor). Finger movements based on gyroscopes and accelerometers attached to the joints for Korean dance were studied in [32]. An example analysis of traditional Hungarian dance analysis considering lower body kinematics, kinetics, biomechanical parameters (Centre of Pressure) and balancing ability through dance movements was described in [28]. The kinematic parameters, such as: Range of Motion (ROM) of knee flexion angle, ROM of hip flexion angle, and ROM of pelvis tilt angle were taken into the consideration. The hip and knee flexion-extension kinetic parameters were calculated for the successive dance steps. In [23], a deep stacked auto-encoder scheme was followed by a Hierarchical Sparse Modelling for Representative Selection (H-SMRS) summarisation algorithm for performing accurate synopses of three Greek dance sequences. Two dancers performed the following dances simultaneously. The analysis concerned human joint points from 3D data. In [25], the Tsamiko dance motion patterns from skeletal animation data captured by multiple Kinect sensors were performed. The skeletal data were split into five different body parts (torso, left hand, right hand, left leg and right leg), which were then transformed to allow invariant posture recognition.

2.2. Classification and Pattern Recognition

The second type of research concerns the classification and pattern recognition aspects. A clustering-based method for the selection of the basic primitives of a choreography, and a kinematics-based method that generates meaningful summaries at hierarchical levels of granularity were applied. It was also stated that the sex of the dancer may be recognised using 3D trajectories of toe markers and the kinematic data of joint angles [16]. The assessment of the Greek dances choreography at different representational levels was presented in [19]. The coarse levels showed the main steps of a Greek dance while the detailed levels provided an assessment per dance frame. The whole framework was created on the basis of artificial intelligence and pose identification (including bidirectional LSTM model with 3D kinematic data like joint velocity and acceleration). The effective classification and recognition of eight traditional Indian dances (accuracy score of 0.911) was presented in [29]. Deep Convolutional Neural Networks (DCNN), using ResNet50 as the base model by fine-tuning the last 14 layers was applied. Detection of basic traditional Walloon dance patterns was described in article [8]. The use of hidden Markov models to identify such steps as: Maclotte Base, Passepied Base, Passepied Fleuret and Back Step were characterised. Kinect was used as a movement registration tool. Three-dimensional position of 68 markers corresponding to 20 joints of the human body was used for classification. The classification efficiency was over 95%. As a result, it was decided to create a game that allows to learn Walloon dance. In [10], the use of motion capture technology and motion analysis method for learning traditional Chinese Agogo dance was presented. The system consists of 3D graphics, motion matching, motion database, and motion capture system. The user’s movements, which were obtained by the motion capture system, were compared with the motions in the motion database through the motion-matching component. Additionally a 3D graphics component visualises the movements by the virtual teacher and user. To start learning, it was necessary to watch a 3D animation demonstrating the basic steps by professional dancers. Then the movement was captured in real time and rendered with a virtual representative, which was displayed next in a form of cylinders representing the body segments. The colour of a cylinder showed whether the position of the body segment was correct or not. In [11] the problem of dance video recommendation by exploring intrinsic motion components in dance videos as well as appearance information were investigated. The authors present the framework for a dance video recommendation method which consisted of several items: natural dance database, convolutional networks, random forests and more. The method was tested using the HIT Dance database containing 6 different types of dance videos, including Ballet, Hip-Hop, Chinese Classical Dance, Chinese Dai Folk Dance, Chinese Mongolian Folk Dance and Chinese Uygur Folk Dance.

2.3. Visualisation and Software Solutions

Visualisation and software solutions, such as Virtual Reality (VR), Augmented Reality (AR) and software platforms are the third type of research.

Visualisation in the form of application of Slovak folk dances recorded by using motion capture was presented in [34]. The effectiveness of learning steps from recorded dances as well as testing the overall usability of the application were verified. The game for teaching young generations of Cypriot folk dances as a way of promoting and preserving ICH was described in [12]. It was created based on 3D motion capture data. The motion of the user was captured in real time via Kinect. The game included different Skeletal Structure, body proportions and comparing motions. Another approach to the transmission of the traditional Greek Tsamiko dance in a form of game-based learning application was described in [24]. It consisted of several exercises, aiming to teach different variations of that dance. In [7] Krebspolka dance was recorded and further 3D models were used to create a mobile AR application. It provided a new way of learning and facilitate the process of learning a folk dance. This process was verified by motion capture sessions. The effect of transcranial Direct Current Stimulation (tDCS) on motor learning was analysed. It was stated that students with received tDCS perform dancing significantly better. In [13] a mobile AR application for assisting the process of learning folk dances based on 3D data was presented. Avatar representations (of either male or female) were synchronised with the digital representation of the dance. A great advantage of this system was the possibility of simultaneously learning to dance by males and females, because the dance was performed in pairs, male participants had to learn the male part of the dance and a male avatar was shown to them during the learning phase, while female participants had to learn the female version by observing a female avatar. In [27], the authors compared the movements of two avatars taking into consideration not only posture matching (meaning the physical geometry of the avatar) but also style, including the required effort, shape and interaction of the performer with the environment. Avatars were created based on 3D motion data. The presented algorithm focused on certain body parts, like upper/lower body and left/right side, as well as the whole body. Chinese performances of the Taiko drumming and lion dance recorded by smartphone were presented in [9]. This tool proved to be appropriate to register their sound. In [2] the CHROMATA platform is presented. It enabled the safeguarding of traditional dances as a part of ICH. It included analysis for 3D pose estimation, folklore dance recognition and textual analysis. Additionally, in [26], the Tsamiko dance was the topic of research where a novel methodology for dance learning and evaluation using multi-sensor and 3D gaming technology was described. Dance practitioner movements were continuously visualised with an avatar thanks to fused input from multiple sensors. The proposed system allowed to compare the quality of the student’s movements with those of an expert due to the use of motion analysis and two-level Fuzzy Inference System. In [31] a method of acquiring torso movements during the traditional Japanese dance Musume-doujouji was presented. Previous studies focused primarily on the movements of the head, hands and feet, marginalising the movement of other parts of the body. The results of research using motion capture technology showed significant differences in the way of movements of the torso while recording the movements of professionals and amateurs constituted the basis of learning for beginners.

2.4. Description and Verification of Methodology

The fourth type of research concerns the methodology of dance registering.

The methodology of capturing Joao Fiadeiro’s choreographic based on three Kinects and three video cameras was presented in [33]. The sequences for dance were also obtained. This method allowed for visualising the data by manipulating colour, time, and spatial position. The Labanotation system used to recode the gesture of Thai dance was presented in [35,36,37,38]. This tool was used to translate the notation scores of Thai dance movements into 3D animation. The methodology including choreographic analysis, 3D data capturing, 3D modelling of static and moving objects and symbolic representations for various traditional dances for the Terpsichore project was described in [22].

2.5. Creation Databases

The final type of research involved databases. In [30] the acquisition and processing methods of Bedhaya Ketawang—the traditional Indonesian dance—was presented. The authors proposed building knowledge for gesture 3D Modelling in Javanese Dance using a motion capture technique. As a result, a database of 3D model skeletons and gestures for building an interactive learning media of traditional Javanese dance was made. To achieve the research goal, a system consisting of a Kinect sensor and a set of animation tools was constructed. A very interesting concept of a database of dance movement sequences and their multi-faceted analysis is presented in [40]. The Dance Motion Capture Database (DMCD) discussed is defined by metadata in five main categories: descriptive, structural, administrative, reference and statistical. The DMCD contains text data about dance and multimedia. The text data describing the dance includes [40]: dance description, story-telling, country/region of origin, date when the dance first became known, type of the dance (e.g., solo or group) and its history. The multimedia collection primarily contains multimedia data of individual dance performers, such as the motion capture data, the video recordings and music. In addition to the multimedia records of individual dance performances, the DMCD stores the artist’s data, the place where the records are made and the props used (i.e., clothes, additional items, etc.) The DMCD is used for the comparative analysis of dances and the search for similar sequences of movements in different dances. For this purpose, the concept of the Bag of Words was used [40]. This concept is widely used in machine learning, language processing and computer vision. This method is based on the representation of the text in the form of a multiset (bag) of words. In computer vision, the Bag-of-Visual-Words model is used to image classification [41]. By analogy, in [40], the Bag-of-Motifs (BoMs) motion model is used to represent the sequence of motion in dance for movements classification and searching for similar sequences. The 16 most informative joints with their relative joint angles analysed in 0.66 s long time windows were gathered to define motion words in [40]. The motion words were then grouped into clusters representing motion motifs and creating BoMs for dance. The database describing the dances can be used for further analysis of the dances.

3. Methodology of 3D Scanning of Dances

Dance is a complex combination of many different elements. It consists of choreography, music, clothes and other artefacts, scenography and others. Choreography is a sequence of dance movements performed by dancers ordered over time. It is characteristic of a given dance. The methodology of 3D dance scanning presents in the paper is limited to 3D digitisation of choreography, i.e., the movements of the dancer’s body. One of the technologies that could be used is the Motion Capture (MC) technique with the use of optical systems. The developed methodology of 3D scanning of a dancer’s movements and post-processing of the obtained results is presented in Table 3.

The aim of the first stage of the work (Table 3) is to develop a detailed action plan. For this purpose, it is necessary to acquire knowledge about the dance and background music used in its implementation. Then it is necessary to plan activities in terms of the potential availability of the laboratory, equipment and technical staff. The work also includes detailed planning of the MC session. During the second stage of the methodology, the MC sessions are prepared. A dancer is commissioned, props are prepared (suitably sized costume, markers, etc.) A laboratory and technical staff are also booked during this stage. The MC session is carried out as the third stage of the methodology. During it, after the dancer is prepared (wearing a costume, placing markers in the right places), a session of recording 3D data is carried out. Videos from reference cameras are recorded simultaneously. During a break in the MC of the session, the quality of the acquired data is pre-checked and, if necessary, the selected dance sequences are repeated. After acquiring the 3D data, the dancer changes the costume to the national costume appropriate for the dance, and a video of the dance is recorded. During this stage, the right music is used, selected in the first stage. After collecting raw data, the stage of their post-processing follows—the fourth stage of the methodology (Table 3). Raw data are cleaned and processed (mainly in order to properly maintain the trajectories of the markers covered by the dancer). A 3D choreography model is then created from multiple movement sequences. This model is used for further analysis in order to search for elementary movement sequences.

4. Materials and Methods

4.1. Motion Capture System

In this study an optical motion capture Vicon Nexus system (Oxford Metrics Ltd., Oxford, UK) was applied. It consisted of eight T40S cameras operating in the near infrared, two Bonita reference video cameras, a Giganet hub collecting data and a desktop computer. The T40S cameras were mounted two on each wall, so that the central part of the room was clearly visible. The system records the movement of the markers placed on the special suit of the dancer. In order to register 3D marker position, it must be seen by at least two cameras. The reference cameras record video, which is used both for data processing and to generate video files containing integrated video and a biomechanical model of a dancer. A great advantage of the presented systems is the temporal integration of all the recorded data, both analogue and three-dimensional. The equipment is supplied with Vicon’s Nexus software, 2.0 used to calibrate the system, record data and data processing.

4.2. Plug-in Gait Model

The motion capture system was integrated with the full body Plug-in Gait model [42], which determines the precise placement of 39 markers all over the human body. Its placement on the participant’s body is presented in Figure 1 on the left. The model uses both upper and lower body for movement analysis. The model outputs calculate kinematic and kinetic data such as: joint angles, forces, moments and powers. The first three are given in three anatomical planes: sagittal, frontal and coronal. The outputs are calculated both for the right and left side. This model gives the possibility to calculate the biomechanics of the captured dance moves [43].

4.3. Hand Model

Hand movement plays an important role in various dances [32,35]. The Plug-in Gait model does not contain markers on hands. Therefore, for the purposes of this research, a hand model was created based on the model [44]. It is presented in Figure 2. It consisted of 13 markers attached to the joints and tips of the fingers: two on the thumb (marked as TR), three on the pinky finger (marked as PR) and three on other fingers (marked as FR) except the marker (LFIN) which belongs to the Plug-in Gait model. Marker names started with L for the left hand and R for the right hand.

4.4. Dancer Preparation

The dancer wore a special suit that consisted of a sweatshirt, trousers and a cap. It was made of Velcro material, so that 14 mm retro-reflexive markers could be attached to it according to the Plug-in Gait model (Figure 1 on the right). In order to properly use this model, the participant had to be measured. The following dancer data had to be input to Vicon Nexus: height, weight, left and right leg length, right and left shoulder offset, left and right knee width, left and right ankle width, left and right elbow width, left and right wrist width and finally left and right hand width. They are all needed for calibration process and the proper model output calculations.

In order to perform the calibration, the static pose of the participant should be recorded. After indicating the labels for individual markers, all markers were verified if they were correctly attached. Calibration is the process which generates the skeleton for the captured person. Thanks to this, it will be possible to calculate outputs of the biomechanical model used.

4.5. Movement Capturing

Registration of movement was carried out with the T40S camera’s resolution set to 100 Hz. The proper connection of all the components of the system was ensured by the use of the Nexus software. A series of proper tests was preceded by calibrating the system and setting the data management hierarchy. Movement acquisition was carried out automatically by the system and consisted in recording the subject’s motion from start to finish (the moment of starting the recording depends on the operator). Movement was recorded during the playback of the Lazgi dance music. All the obtained recordings were linked to a specific person (subject). An important factor influencing the quality of the recording was the elimination of all reflective elements from the cameras’ sight. Accidental glare resulting from elements of the subject’s or the experimenter’s outfit could introduce additional artefacts in the recordings and make subsequent processing more difficult. In this study the Lazgi dance was registered.

4.6. Data Post-Processing

Captured 3D data needed post-processing, which consisted of four main steps: labelling all markers, filling-in gaps, cleaning the recording and calculating the outputs for the biomechanical model. The first step assigned the proper label to a particular marker. It might be performed automatically or manually. The next step was to calculate the interpolated positions of markers based on the previous and subsequent ones. The continuous trajectory was obtained. The third step cleared the recording of additional unintended markers, such as instances of glare. Finally, for data prepared in this way, the outputs from the Plug-in Gait model were calculated. The obtained 3D data were ready for further analysis. The post-processed model of a dancer with the Plug-in Gait model applied, together with a hand model, is presented in Figure 3.

Due to the length of the recording of the entire dance, which took almost 4 min, post-processing was very time-consuming. A better solution would be to capture shorter dance sequences, or to divide the dance into sequences immediately after capturing and then perform the post-processing of the successive recordings.

5. Motion Analysis

The methodology presented in this paper made it possible to conduct research, the results of which are both 3D silhouette data, 3D detailed hand data and angles for selected parts of body. The silhouette data relate to the position of 3D markers attached to the dancer’s body (39 according to Plug-in Gait). The hand data correspond to 3D positions of 13 markers attached according the defined hand model. From the recorded Lazgi dance data, several sequences were obtained for various type of movements. For the purpose of motion analysis two dance sequences were selected for detailed analysis, including body and hand movements.

In the first sequence, the dancer bowed. The second sequence involved dynamic hand movements.

5.1. Body Movements

The analysis of the dancer’s body movements concerns both the upper and lower parts. The obtained results include trajectories for selected markers, as well as kinematic analysis showing changes in angles for individual body parts. Trajectories presented were prepared in two planes: XY (transverse) and YZ (sagittal). The trajectories for upper body parts including shoulders (right and left) and wrists (right and left) from the thumb side indicted by the attached markers are presented in Figure 4.

The trajectories for lower body parts concerning hips (right and left) and knees (right and left) are presented in Figure 5.

For the second dance sequence the trajectories for the upper body parts including shoulders (right and left) and elbows (right and left) are presented in Figure 6.

For the second dance sequence the trajectories for the lower body parts concerning hips (right and left) and knees (right and left) are presented in Figure 7.

The angles for the lower body parts concerning ankles (right and left), knees (right and left) and pelvis (right and left) are presented in Figure 8. Please note that the lines for pelvis angles in Figure 8 are identical.

The angles for upper body parts concerning wrists (right and left), elbows (right and left) and shoulders (right and left) are presented in Figure 9.

5.2. Hand Movements

A dynamic dance sequence was selected for the analysis of hand movements (for four fingers, without thumb). The trajectories for the markers (LFR8 and RFR8) from the hand model are shown for the left and right hand, respectively. Two planes were analysed: XY and YZ. The results are presented in Figure 10.

The bend angles of the four fingers (ring, middle, index and little finger) were also calculated for each hand. The angles were calculated between the middle phalanges and the proximal phalanges. The results are presented in Figure 11.

6. Discussion and Conclusions

The article concerns the first attempt of registering Lazgi, an Uzbek national dance, using an optical motion capture system. For the purpose of this study, a methodology for capturing the movements of this dance was created. Due to the specificity of the Lazgi dance movements performed, the registration of the entire dancer’s body using the full Plug-in Gait model was taken into account. This allowed to perform both analysis based on 3D data, represented by each marker attached to the dancer’s body, and a kinematic analysis based on the generated outputs of the model used. This set of three-dimensional data gives the possibility of performing a sophisticated analysis of the dancers biomechanics of movements for the successive sequences, rhythm, velocities and others. In order to register the hand movements, which are a crucial element of this dance, a model was developed with which it is possible to analyse the movements of individual dances in detail.

The described methodology was verified by recording the Lazgi dance in a motion capture session. From the obtained data, two dance sequences were selected and analysed. The trajectories of selected markers were determined for both the upper and lower body parts. The left and right movements of the dancer were compared. Additionally, the angles of the upper and lower body parts for the right and left sides were analysed. The first sequence, the bend, is performed with little dynamism. Correctly made, it should reflect the symmetry of movements in two planes. The results presented in Figure 4 and Figure 5 are almost symmetrical with respect to the vertical axis of the body, which proves that the movements were performed correctly. Although the second movements were more dynamic, the axes of symmetry on which the movements were performed can also be seen (Figure 6 and Figure 7). Angle changes for the ankle, knee, pelvis, wrist, elbow and arm were computed for the first sequence. It can clearly be seen that the angles are similar (Figure 8 and Figure 9) and even identical (Figure 8). The presented results of hand movements (Figure 10), including the angles of the fingers (Figure 11), allow to conclude that the model is suitable for the analysis of the Lazgi dance. For a dynamic sequence, the finger movements vary from 0 to 40 degrees (Figure 11).

The obtained results confirm that the proposed methodology is correct for this dance. Moreover, it is universal and can be used both for dances in which the movements performed involve the lower parts of the body (e.g., Greek dances) and the upper parts. In addition, the presented hand model allows to analyse the movements of individual fingers.

Some problems with the presented method should be mentioned. A large number of markers placed on the hands disturbs the course of motion capture by obscuring them. This significantly increases the labour intensity of post-processing.

The obtained data can be used in five aspects: (i) dance analysis (kinematics, kinetics, sequences, rhythm and others), (ii) classification and recognition of movements, (iii) making ICH data public (e.g., in the form of a database), (iv) creating and visualising 3D models and (v) developing software solutions (e.g., VR or AR).

Additionally, the acquired data and the developed dance movement models will be eternally stored and made available on the Internet. This will contribute to the preservation and popularisation of the intangible cultural heritage.

Author Contributions

Conceptualization, M.S.-P., M.M. and E.L.; methodology, M.M., E.M.; software, J.S.; validation, J.S.; formal analysis, J.S., M.S.-P.; investigation, J.S., M.S.-P., E.L., P.P. and D.M.; resources, M.S.-P., P.P., E.M. and M.M.; data curation, J.S., M.S.-P., P.P. and E.L.; writing—original draft preparation, M.S.-P., P.P. and M.M.; writing—review and editing, M.S.-P., P.P. and M.M.; visualization, J.S., M.S.-P.; supervision, M.S.-P., E.L. and M.M. All authors have read and agreed to the published version of the manuscript.

Funding

This article has been partially supported by the Polish National Agency for Academic Exchange under Grant No. PPI/APM/2019/1/00004 titled “3D DIGITAL SILK ROAD”.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Commission for Research Ethics of Lublin University of Technology, No. 2/2016 dated 8 April 2016.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The dataset created during the study can be found on the website: https://cs.pollub.pl/zasoby/?lang=en (accessed on 23 November 2021).

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. Full body Plug-in Gait marker placement [42] (on the left) and its real implementation (on the right).

Figure 2. Left hand markers placement.

Figure 3. Model of the dancer after post-processing T-pose (on the left) and while performing the dance (on the right).

Figure 4. Trajectories for upper body parts for the first sequence.

Figure 5. Trajectories for lower body parts for the first sequence.

Figure 6. Trajectories for upper body parts for the second sequence.

Figure 7. Trajectories for lower body parts for the second sequence.

Figure 8. Angles for lower body parts for the first sequence.

Figure 9. Angles for upper body parts for the first sequence.

Figure 10. Trajectories for fingers for the second sequence.

Figure 11. Angles for hands movement for the second sequence.

Table 1

Traditional dances in the selected countries.

Country	Dance
Austria	Krebspolka [7]
Belgium	Walloon [8]
China	Lion Dance [9]
	Agogo [10]
	Dai, Uygur [11]
Cyprus	Zeimpekiko [12]
Czech Republic	Pašovská Sedlcká [13]
Greece	Laisios or Lagisios dance at a Greek Thracian wedding, Gikna [2],
	Syrtos [14,15,16,17,18,19,20,21,22,23],
	Syrtos Makedonikos [17,21,22,23]
	Syrtos Kalamatianos [14,15,17,19,20,21,22],
	Trehatos [14,15,19,20,21]
	Enteka [19,20,21,22]
	Tsamiko [24,25,26]
	Zeibekiko [27]
	Antikristos [27]
Hungary	Kalocsai mars [28]
India	Bharatanatyam  [29]
	Kathak [29]
	Kuchipudi [29]
	Odissi [29]
	Kathakali [29]
	Sattriya [29]
	Manipuri [29]
	Mohiniyattam [29]
Indonesia	Bedhaya Ketawang [30]
Japan	Musume-doujouji [31]
Korea	Didim [32]
Macedonia	Trexatos [22]
Portugal	Joao Fiadeiro’s choreographic [33]
Slovakia	Horehronie, Abov, Podpoľanie and Horné, Považie [34]
Thailand	[35,36,37,38]
United Arab Emirates	Ayala [39]

Table 2

Technologies used to capture traditional dance data.

Data Collection Methods	Papers
3D visualization, 3D poses estimation from a video	[2]
Vicon motion capture system	[10,14,15,16,17,18,21,23,31]
OptiTrack motion capture system	[7,13,28,34]
Phasespace4 Impulse X2 motion capture system	[12,27]
Microsoft Kinect	[8,19,20,21,22,24,25,26,30,33]
Gyroscopes and accelerometers	[32]
Smartphones	[9]
Video camera	[11,29,33]
Labanotation system	[35,36,37,38]
IMU sensors	[39]

Table 3

Methodology of 3D digitisation of dance choreography.

Name of the Stage	Sub-Stages	Results
1. Planning	1.1. Obtaining knowledge aboutthe dance1.2. Selecting a music track1.3. Technical planningMC session scheduling	Detailed plan of activities
2. MC sessionpreparation	2.1. Acquiring dancers2.2. Preparation of props2.3. Booking a laboratory	Everything is ready forimplementation of theMC session
3. MC sessionimplementation	3.1. Participant preparation3.2. Implementation ofthe session—data recording3.3. Preliminary checking of thequality of the obtained data3.4. Implementation of correctiverecordings (if any)3.5. Video recording of the dance	Raw data
4. Data post-processing	4.1. Post-processing anddata cleaning4.2. Creating a 3Dchoreography model4.3. Indicating dance sequences4.4. Analysis of the dancer’smovements	3D model of the danceSequence of elementarymovements

References

1. Intangible Heritage Domains in the 2003 Convention. Available online: https://ich.unesco.org/en/intangible-heritage-domains-00052 (accessed on 12 October 2021).

2. Pistola, T.; Diplaris, S.; Stentoumis, C.; Stathopoulos, E.A.; Loupas, G.; Mandilaras, T.; Kalantzis, G.; Kalisperakis, I.; Tellios, A.; Zavraka, D. et al. Creating immersive experiences based on intangible cultural heritage. Proceedings of the 2021 IEEE International Conference on Intelligent Reality (ICIR); Virtual Event, 12–13 May 2021; pp. 17-24.

3. Lack, K.J. Capturing Dance: The Art of Documentation (An Exploration of Distilling the Body in Motion). Available online: https://ro.ecu.edu.au/theses_hons/77 (accessed on 12 October 2021).

4. Khorezm Lazgi: The Sunniest Dance on Earth. Available online: https://voicesoncentralasia.org/khorezm-lazgi-the-sunniest-dance-on-earth/ (accessed on 12 October 2021).

5. Rakhimov, B. The Eternity of Lazgi. Eurasian Music Sci. J.; 2020; 2, 22. [DOI: https://dx.doi.org/10.52847/EAMSJ/vol_2020_issue_2/A3]

6. Khorazm Dance, Lazgi. Available online: https://ich.unesco.org/en/RL/khorazm-dance-lazgi-01364 (accessed on 12 October 2021).

7. Kico, I.; Liarokapis, F. Enhancing the learning process of folk dances using augmented reality and non-invasive brain stimulation. Entertain. Comput.; 2022; 40, 100455. [DOI: https://dx.doi.org/10.1016/j.entcom.2021.100455]

8. Laraba, S.; Tilmanne, J. Dance performance evaluation using hidden Markov models. Comp. Anim. Virtual Worlds; 2016; 27, pp. 321-329. [DOI: https://dx.doi.org/10.1002/cav.1715]

9. Lee, H.M.; Garg, S.; Lim, K.M.; Lee, H.P. The sound of drumming for traditional Taiko drumming and lion dance. Appl. Acoust.; 2022; 185, 108434. [DOI: https://dx.doi.org/10.1016/j.apacoust.2021.108434]

10. Chan, J.C.P.; Leung, H.; Tang, J.K.T.; Komura, T. A Virtual Reality Dance Training System Using Motion Capture Technology. IEEE Trans. Learn. Technol.; 2011; 4, pp. 187-195. [DOI: https://dx.doi.org/10.1109/TLT.2010.27]

11. Han, T.; Yao, H.; Xu, C.; Sun, X.; Zhang, Y.; Corso, J.J. Dancelets Mining for Video Recommendation Based on Dance Styles. IEEE Trans. Multimed.; 2017; 19, pp. 712-724. [DOI: https://dx.doi.org/10.1109/TMM.2016.2631881]

12. Stavrakis, E.; Aristidou, A.; Savva, M.; Himona, S.L.; Chrysanthou, Y. Digitization of Cypriot Folk Dances. Progress in Cultural Heritage Preservation; Ioannides, M.; Fritsch, D.; Leissner, J.; Davies, R.; Remondino, F.; Caffo, R. Springer: Limassol, Cyprus, 2012; Volume 7616 LNCS, pp. 404-413.

13. Kico, I.; Liarokapis, F. Investigating the Learning Process of Folk Dances Using Mobile Augmented Reality. Appl. Sci.; 2020; 10, 599. [DOI: https://dx.doi.org/10.3390/app10020599]

14. Voulodimos, A.; Rallis, I.; Doulamis, N. Physics-based keyframe selection for human motion summarization. Multimed. Tools Appl.; 2020; 79, pp. 3243-3259. [DOI: https://dx.doi.org/10.1007/s11042-018-6935-z]

15. Voulodimos, A.; Doulamis, N.; Doulamis, A.; Rallis, I. Kinematics-based Extraction of Salient 3D Human Motion Data for Summarization of Choreographic Sequences. Proceedings of the International Conference on Pattern Recognition; Beijing, China, 20–24 August 2018; pp. 3013-3018.

16. Douka, S.; Ziagkas, E.; Zilidou, V.; Loukovitis, A.; Tsiatsos, T. The 3D Motion Capturing Process of Greek Traditional Dance “Syrtos in Three” and a Proposed Method for the Sex Identification of Performing Dancers. Adv. Intell. Syst. Comput.; 2021; 1231, pp. 366-374.

17. Ziagkas, E.; Zilidou, V.; Loukovitis, A.; Douka, S.; Tsiatsos, T. Greek Traditional Dances Capturing and a Kinematic Analysis Approach of the Greek Traditional Dance “Syrtos” (Terpsichore Project). Adv. Intell. Syst. Comput.; 2021; 1192, pp. 514-523.

18. Rallis, I.; Georgoulas, I.; Doulamis, N.; Voulodimos, A.; Terzopoulos, P. Extraction of key postures from 3D human motion data for choreography summarization. Proceedings of the 9th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games 2017; Athens, Greece, 6–8 September 2017; pp. 94-101.

19. Rallis, I.; Bakalos, N.; Doulamis, N.; Doulamis, A.; Voulodimos, A. Bidirectional long short-term memory networks and sparse hierarchical modeling for scalable educational learning of dance choreographies. Vis. Comput.; 2021; 37, pp. 47-62. [DOI: https://dx.doi.org/10.1007/s00371-019-01741-3]

20. Rallis, I.; Bakalos, N.; Doulamis, N.; Voulodimos, A.; Doulamis, A.; Protopapadakis, E. Learning Choreographic Primitives Through A Bayesian Optimized Bi-Directional LSTM Model. Proceedings of the International Conference on Image Processing; ICIP, Taipei, Taiwan, 22–29 September 2019; pp. 1940-1944.

21. Rallis, I.; Protopapadakis, E.; Voulodimos, A.; Doulamis, N.; Doulamis, A.; Bardis, G. Choreographic Pattern Analysis from Heterogeneous Motion Capture Systems Using Dynamic Time Warping. Technologies; 2019; 7, 56. [DOI: https://dx.doi.org/10.3390/technologies7030056]

22. Doulamis, A.; Voulodimos, A.; Doulamis, N.; Soile, S.; Lampropoulos, A. Transforming intangible Folkloric performing arts into tangible choreographic digital objects: The Terpsichore approach. Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications; Porto, Portugal, 27 February–1 March 2017; Volume 5, pp. 451-460.

23. Protopapadakis, E.; Rallis, I.; Doulamis, A.; Doulamis, N.; Voulodimos, A. Unsupervised 3D Motion Summarization Using Stacked Auto-Encoders. Appl. Sci.; 2020; 10, 8226. [DOI: https://dx.doi.org/10.3390/app10228226]

24. Kitsikidis, A.; Dimitropoulos, K.; Uğurca, D.; Bayçay, C.; Yilmaz, E.; Tsalakanidou, F.; Douka, S.; Grammalidis, N. A game-like application for dance learning using a natural human computer interface. International Conference on Universal Access in Human-Computer Interaction; Springer: Cham, Switzerland, 2015; pp. 472-482.

25. Kitsikidis, A.; Dimitropoulos, K.; Douka, S.; Grammalidis, N. Dance analysis using multiple Kinect sensors. Proceedings of the 2014 International Conference on Computer Vision Theory and Applications, VISAPP 2014; Lisbon, Portugal, 5–8 January 2014; pp. 789-795.

26. Kitsikidis, A.; Dimitropoulos, K.; Yilmaz, E.; Douka, S.; Grammalidis, N. Multi-sensor Technology and Fuzzy Logic for Dancer’s Motion Analysis and Performance Evaluation within a 3D Virtual Environment. Proceedings of the 2014 International Conference on Universal Access in Human-Computer Interaction, UAHCI 2014; Crete, Greece, 22–27 June 2014; pp. 379-390.

27. Aristidou, P.; Stavrakis, E.; Charalambous, P.; Chrysanthou, Y.; Himona, S.L. Folk dance evaluation using laban movement analysis. J. Comput. Cult. Herit.; 2015; 8, 20. [DOI: https://dx.doi.org/10.1145/2755566]

28. Molnár, C.; Pálya, Z.; Kiss, R.M. Static Balancing Ability and Lower Body Kinematics Examination of Hungarian Folk Dancers: A Pilot Study Investigating the “Kalocsai Mars” Dance Sequence. Appl. Sci.; 2021; 11, 8789. [DOI: https://dx.doi.org/10.3390/app11188789]

29. Jain, N.; Bansal, V.; Virmani, D.; Gupta, V.; Salas-Morera, L.; Garcia-Hernandez, L. An Enhanced Deep Convolutional Neural Network for Classifying Indian Classical Dance Forms. Appl. Sci.; 2021; 11, 6253. [DOI: https://dx.doi.org/10.3390/app11146253]

30. Nurindiyani, A.; Pramadihanto, D.; Afifah, R. Motion Modeling of Traditional Javanese Dance: Introduction of Javanese Dancer Gesture with 3D Models. Proceedings of the 2019 International Electronics Symposium (IES); Surabaya, East Java, Indonesia, 27–28 September 2019.

31. Utsugi, A.; Masaya, T.; Takaoka, H. Analysis of Movements of Body Trunk in Japanese Traditional Dance. Proceedings of the 2017 International Conference on Culture and Computing; Kyoto, Japan, 10–12 September 2017.

32. Lim, Y.K.; Park, J.W. The way to preserve korean intangible cultural assets. Proceedings of the HCI International 2018—Posters’ Extended Abstracts; Las Vegas, NV, USA, 15–20 July 2018; pp. 192-195.

33. Ribeiro, C.; Dos Anjos, R.K.; Fernandes, C. Capturing and documenting creative processes in contemporary dance. ACM Int. Conf. Proc. Ser.; 2017; 2017, 3078041.

34. Hajdin, M.; Kico, I.; Dolezal, M.; Chmelik, J.; Doulamis, A.; Liarokapis, F. Digitization and Visualization of Movements of Slovak Folk Dances. Adv. Intell. Syst. Comput.; 2019; 917, pp. 245-256.

35. Tongpaeng, Y.; Rattanakhum, M.; Sureephong, P.; Wicha, S. Implementing a tool for translating dance notation to display in 3D animation: A case study of traditional Thai dance. Proceedings of the 30th International Conference on Industrial, Engineering, and Other Applications of Applied Intelligent Systems, IEA/AIE 201; Arras, France, 27–30 June 2017; pp. 22-26.

36. Tongpaeng, Y.; Sureephong, P.; Rattanakhum, M.; Yu, H. Thai dance knowledge archive framework based on Labanotation represented in 3D animation. Proceedings of the 2nd Joint International Conference on Digital Arts, Media and Technology 2017: Digital Economy for Sustainable Growth, ICDAMT 2017; Chiang Mai, Thailand, 1–4 March 2017; pp. 66-70.

37. Tongpaeng, Y.; Maphon, P.; Sureephong, P. Improving ThaiLabanXML standard for the automatic Thai laban drag drop system. Proceedings of the ECTI DAMT-NCON 2019—4th International Conference on Digital Arts, Media and Technology and 2nd ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering; Nan, Thailand, 30 January–2 February 2019; pp. 168-173.

38. Tongpaeng, Y.; Nantapatsaran, K.; Sureephong, P. Interpreting Thai laban notation into ThaiDanceXML. Proceedings of the 2018 International Conference on Digital Arts, Media and Technology (ICDAMT); Chiangrai, Thailand, 25–28 February 2018; pp. 173-178.

39. Udawatta, L.; Daniels, B.; Hissem, S.; Wivou, J.; Pathirana, P.N.; Nguyen, D.K.; Nguyen, N.D. Motion Capturing of Biomechanical Systems in Traditional Dance using IMU Sensors. Proceedings of the 2017 International Conference on Electrical and Computing Technologies and Applications; Icecta, Aurak, United Arab Emirates, 21–23 November 2017; pp. 1-5.

40. Aristidou, A.; Shamir, A.; Chrysanthou, Y. Dance Ethnography: Organizing Large Dance Collections. J. Comput. Cult. Herit.; 2019; 12, pp. 1-27. [DOI: https://dx.doi.org/10.1145/3344383]

41. Sivic, J.; Zisserman, A. Video Google: A Text Retrieval Approach to Object Matching in Videos. Proceedings of the ICCV 2003—9th IEEE International Conference on Computer Vision; Nice, France, 13–16 October 2003; pp. 1-8.

42. Plug-in Gait Reference Guide. Available online: https://docs.vicon.com/display/Nexus210/Plug-in+Gait+Reference+Guide (accessed on 20 October 2021).

43. Hernandez-Barraza, L.; Yeow, C.-H.; Varela, M. The Biomechanics of Character Types in Javanese Dance. J. Dance Med. Sci.; 2019; 23, pp. 104-111. [DOI: https://dx.doi.org/10.12678/1089-313X.23.3.104] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31500692]

44. Cook, J.R.; Baker, N.A.; Cham, R.; Hale, E.; Redfern, M.S. Measurements of Wrist and Finger Postures: A Comparison of Goniometric and Motion Capture Techniques. J. Appl. Biomech.; 2007; 23, pp. 70-78. [DOI: https://dx.doi.org/10.1123/jab.23.1.70] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17585179]