Detecting hand washing activity among activities of daily living and classification of WHO hand washing techniques using wearable devices and machine learning algorithms

Abstract

During COVID‐19, awareness of proper hand washing has increased significantly. It is critical that people learn the correct hand washing techniques and adopt good hand washing habits. Hence, this study proposes using wearable devices to detect hand washing activity among other daily living activities (ADLs) and classify steps proposed by the World Health Organization (WHO). Two experiments were conducted with 16 participants, aged from 20 to 31. The first experiment was hand washing following WHO regulation (ten participants), and the second experiment was performing eight ADLs (eight participants). All participants wore two wearable devices equipped with accelerometers and gyroscopes; one on each wrist. Four machine learning classifiers were compared in classifying hand washing steps in the leave‐one‐subject‐out (LOSO) mode. The SVM model with Gaussian kernel achieved the best performance in classifying 11 washing hands steps, with an average F1‐score of 0.8501. When detected among the other ADLs, hand washing following WHO regulation obtained the F1‐score of 0.9871. The study demonstrates that wearable devices are feasible to detect hand washing activity and the hand washing techniques as well. The classification results of getting the soap and rubbing thumbs are low, which will be the main focus in the future study.

INTRODUCTION

Appropriate hand washing is crucial to prevent the transmission of bacteria, viruses and parasites [] and reducing the rate of infections []. In particular, during the period of the COVID-19 pandemic, the importance of hand washing has been emphasised []. There are two important aspects of hand washing: the habit (when to wash hands) and the techniques (how to wash hands). Ataee et al. mentioned ten occasions when hand washing is necessary []. However, there are still cultural differences in the hand washing habit. For example, in an international survey among 63 countries in 2015, it appeared that around 77% of people did not wash their hands after using the toilet, and even in the developed countries, the rate was high, such as 50% in Netherlands and 43% in Italy []. Regarding the hand washing quality, the World Health Organization (WHO) proposes the regulation of hand washing techniques with six major steps (steps 2–7) as shown in Figure  whereby hand washing lasts for 40–60 s []. Thus, the proper hand washing techniques must be learned, along with forming a good hand washing habit. To monitor these, sensor technology and machine learning algorithms can be used to identify different classes of the hand washing techniques proposed by the WHO and detect hand washing among other daily living activities (ADLs).

[IMAGE OMITTED. SEE PDF]

Multiple types of sensors have been investigated previously, such as internet-of-things (IoT) [], audio sensor [], and camera []. Nevertheless, these sensors still have limitations since they need to be set up close to the activity area. People wash their hands in various places at home, such as the bathroom or kitchen. Hence, IoT sensors need to be installed in all these places, which increases the expense and the complexity of setup. Additionally, it is advisable to provide individuals with feedback, especially while helping them form the hand washing habit. However, these sensors cannot identify different users unless assisted with other sensors, for example, Beacon or motion sensors []. Despite being able to recognise people, cameras cause privacy concerns, especially when installed in the bathroom. In addition, they cannot detect hand washing steps (techniques).

As an alternative method, wearable sensors, including an accelerometer and gyroscope, can be applied. In the study of Mondol et al. [], the Harmony system was developed to detect the hand washing quality for food workers. In this system, a smartwatch was attached to the participant's wrist to collect the movement signals, a Bluetooth transmitter was integrated into the dispenser to detect whether the participant used the soap, and a Beacon sensor was used to send alerts when the participant was in the area that required hand hygiene or when the participant did not properly wash hands. The accelerometer and gyroscope signals were processed to extract the time-domain features, and a decision tree (DT) was utilised to detect classes of hand washing. The classifier was trained in the user-independent mode. In this model, all the participants are split into two groups, the data of one group is used for training, and the other group's data is used for testing. As a result, the DT classifier received an average accuracy of 85%. In the later study [], researchers explored the impact of the sensor position, and they tried to detect hand washing activity among other ADLs using the Feed Forward Neural Network (FFN). With the F1-scores of 0.72 and 0.74 for the left and right wrist, respectively, there was no significant difference between which sides the wearable device was worn. Because each participant only wore the smartwatch on one side, the result may be impacted by the different performances of participants. The WISDM dataset [] was added without including hand washing to test whether other ADLs are misclassified as hand washing. The authors achieved the result by reducing the false positive rate by 77% compared to the baseline method []. However, they did not discuss which ADLs are more likely to be classified as hand washing.

In study [], also one wristwatch was used, including the accelerometer and gyroscope sensors. The data were collected both in a controlled and uncontrolled environment. A hidden Markov model (HMM) classifier was used with an accuracy of 85% with firstly determining the order of hand washing classes in the user-independent mode. Compared to study [], the before (rubbing hands) and after (turning off the faucet) hand washing activities were included. These two classes were detected using an extra binary classifier, with an accuracy ranging from 35% to 54%. Rather than attaching sensors to the wrist, Wang et al. used four armbands which also included Electromyography (EMG) sensors []. Two armbands were placed on the forearms and two on the upper arms. This study investigated two types of hand hygiene, alcohol-based hand rubbing and hand washing with soap and water. During data acquisition, each type was performed 15 times/participant. The after hand washing classes were categorised into three classes: rinsing hands, drying hands with a towel, and turning off the faucet. EXtreme Gradient Boosting (XGBoost) combined with E.Divisive [] was applied. Furthermore, Wang et al. discussed the impact of different sensors and the attached position. The highest accuracy was achieved using the signals from all three sensors (accelerometer, gyroscope, and EMG) from both armbands attached to the forearm for 91% in the user-independent mode. The classifier was also trained in the user-dependent mode: one part of the dataset of a participant used for training and the remaining part of the same participant used for testing. The number of sessions used for training the model was examined in the user-dependent mode, which indicated that after using the data of 16 sessions, the classification accuracy could reach 90%. In study [], an online monitoring system including a smartwatch was developed with the Convolutional (CNN) combined with Long Short Term Memory (LSTM) neural network as the classifier. The accuracy of each step was over 90% in the real-life dataset.

Previous studies have proven the feasibility of using wearable sensors, especially accelerometers and gyroscopes, to monitor the hand washing techniques suggested by the WHO and determine the duration of hand washing. However, they did not detect all the steps covered in the hand washing techniques, for example, getting soap. In addition, few studies [] looked at detecting hand washing among other ADLs. At last, the impact of sensors and the wearing side was examined based on the average classification performance rather than for each hand washing step. Therefore, this study proposes a wireless system with wearable devices to monitor hand washing. The contributions of this study are:

• Classifying hand washing steps in the WHO regulation.
• Classifying hand washing activity among other ADLs
• Differentiating hand washing activities between not following (untrained) and following (trained) the WHO regulation.
• Investigating the impact of the combination of sensors on the classification of each hand washing step.

To accomplish this, two wearable sensors composed of three-axis accelerometers and gyroscopes were attached to each wrist. Features are extracted from both the time and frequency domain, and multiple machine learning algorithms are explored in the leave-one-subject-out (LOSO) mode.

METHODOLOGY

The wearable device

As shown in Figure , two Byteflies sensors [] were used, one on each wrist. The sensor includes a three-axis accelerometer and a three-axis gyroscope, both running at 100 Hz. The placement direction on each wrist was fixed. The sensors were automatically synchronised. Two radar sensors were also applied to detect the hand washing movement during the experiment, but their received signals will not be discussed in this paper.

[IMAGE OMITTED. SEE PDF]

Dataset preparation

Hand washing data collection

This study involved ten individuals (seven males, three females, aged 20–23). Participants washed their hands with the WHO regulation poster placed in front of them and instructed by a researcher. Each participant performed hand washing three times, each time spending 60–80 s. One camera was placed aside for recording as the ground-truth annotation. Eleven activities (classes a–k) were labelled, as listed in Table . The classes were categorised into three groups: before hand washing (steps 0, 1, and 2), hand washing (steps 3–11), after hand washing (steps 0 and 1). Class a covers three hand washing steps: opening the faucet, wetting/ rinsing hands, and turning off the faucet. These activities were performed both before and after hand washing. Therefore, they were categorised into one class. Step 9, drying hands, was not included. Considering real application, people usually first turn off the faucet and then get the towel to dry their hands. Thus, hand drying was not included in this study.

TABLE 1 Annotated hand washing classes

Class	Detail (Related steps proposed by the WHO as shown in Figure )
a	0, 8, 10
opening the faucet, wetting/ rinsing hands
and turning off the faucet
b	1, getting the soap
c	2, rubbing hands palm to palm
d	3, right palm over left dorsum with interlaced fingers
e	3, left palm over right dorsum with interlaced fingers
f	4, palm to palm with interlaced fingers
g	5, backs of fingers to opposing palms with interlaced fingers
h	6, rubbing left thumb
i	6, rubbing right thumb
j	7, clasped fingers of right hand in left palm
k	7, clasped fingers of left hand in right palm

ADLs data collection

To test the system's ability to classify hand washing among other ADLs, we also conducted another experiment that included eight ADLs: sitting, standing, walking, computer typing, going upstairs, and going downstairs, brushing teeth, and untrained hand washing. These activities were included because:

• These activities are basic ADLs and are normally performed every day.
• Brushing teeth is also one type of bathroom activity and it also includes high-frequency movements of the wrists.

Eight participants joined the experiment (three females and five males, aged 23–31). The right wrist was the dominant wrist of all participants. Among the eight participants, two people joined the experiment of hand washing dataset collection as well. Same as the experiment of hand washing, all participants wore two Byteflies sensors, one on each wrist. In this experiment, participants were asked to perform activities in their own way. The performed activities and procedures are as following:

• Sitting (3 min). Participants could talk, use smartphones, or with slight arm movements.
• Computer typing (3 min). Participants were asked to type the given text material.
• Standing (3 min). The requirement is the same as sitting.
• Walking (180 m). Participants were asked to walk at their normal speed.
• Going upstairs (60 stairs $\times 2$ times). Participants were asked to go at their normal speed.
• Going downstairs (60 stairs $\times 2$ times). Participants were asked to go at their normal speed.
• Hand washing (3 times). The time was recorded from participants opening the faucet to rinsing hands and closing the faucet.
• Brushing teeth. The time was recorded from participants squirting the toothpaste on the toothbrush to rinsing the mouth. All the participants used manual toothbrushes.

One researcher was by aside to annotate activities. The experiments were approved by the KU LEUVEN Social and Societal Ethics Committee (SMEC), with the assigned serial number G-2020-2705. All participants signed the informed consent form before participating in the experiment.

Data analysis

Data pre-processing

The collected accelerometer and gyroscope signals were segmented based on the annotation of the video recording. Afterwards, segmented signals were then sliced by a fixed-length window. In the previous studies, the applied windows size ranged from 0.06 s (three data points) [] to 1 s (50 data points) [], with the overlap rate in the range of (50%, 70%). In this study, the testing window size was varied between (0.08 s, 3.5 s), with an overlap rate of 0/50%. A Support Vector Machine (SVM) classifier was used as the classifier, trained in the user-independent mode. The result of optimal window size is shown in Section .

Feature extraction

The features were extracted from each axis and the magnitude of the three axes. In the time domain, the extracted features were the mean, standard deviation, root mean square (RMS), minimum (min), maximum (max), range (= max–min), interquartile range, skewness, kurtosis, mean-crossing times (= the number of the time the signal value passing through the mean value), and energy. In addition, the correlation between every two axes and zero-crossing times (= the number of the time the signal value passing through zero) were calculated from each axis, except for the magnitude signal. In the frequency domain, the frequency power, the frequency at the highest amplitude, 25% frequency (F25d), 75% frequency (F75d), Normalised frequency ($1-F25d/F75d$) and spectral energy were extracted. As a result, 74 features were extracted from each sensor, with 296 features extracted from all sensors.

Classification model

Four types of machine learning (ML) models were investigated as they were applied in the previous works: SVM [], K-Nearest Neighborhood (KNN) [], XGBoost [], and FFN []. The SVM model applied the Gaussian kernel and was trained in the one-versus-one mode. Hence, 55 binary classification models were trained, and the predicted class was assigned to the one with the highest score. The hyper-parameters of the models were tuned using the grid-search method with testing all the combinations []. The searching range and the tuning result are shown in Table . While developing the FFN architecture, one hidden layer was added each time, and the number of neurons in this layer was tuned. We stopped adding the layer until the classification performance dropped on the cross-validation set. Moreover, the searching range of the number of neurons of the new hidden layer did not exceed that of the previous one. All the models were trained in the LOSO mode: in each fold, one participant's data were selected for testing, one participant's data for validation, and the remaining eight participants' data were for training. In this study, the models were trained in ten folds. The F1-score was applied as the evaluation metric. The class weight balancing method [] was used to solve the imbalanced dataset problem by assigning the class weight to be the inverse of the number of windows. The model with the highest performance was used in the following cases:

• Investigating the optimal combination of the sensors. Various combinations of sensors were tested, including only the accelerometer sensor from one side, both accelerometer and gyroscope sensors on one side, to all sensors.
• Detecting the hand washing activity among other ADLs. The hand washing dataset was combined with the ADLs dataset. As aforementioned, the ADLs dataset did not include hand washing activity that followed the WHO regulation. Hence, half of the participants' data were used for training in each dataset and the remaining for testing. In other words, the data of five participants in the hand washing dataset and four participants in the ADLs dataset were selected as the training dataset. The data of the remaining participants were selected as the testing dataset.
• Differentiate trained and untrained hand washing activities. While combining the hand washing dataset and ADLs dataset, the hand washing activities in these two datasets were categorised into two classes.

TABLE 2 Hyper-parameters tuning results of classifiers in the user-independent mode

Classifier	Hyper-parameter	Searching range	Tuned value
SVM	C - penalty parameter	$2^n$, $n\in (-5,15)$ []	$2^5$
	$\gamma$	$2^n,n\in (-15,5)$	$2^{-7}$
KNN	K	$1-15$	3
XGBoost	maximum depth of decision trees	$1-10$	2
	minimum child weight	$1-5$	4
	$\gamma$	$2^n,n\in (-5,5)$	$2^5$
FFN	Number of hidden layers	$1-5$	3
	Number of neurons	$2^n,n\in (1,8)$	128 - layer 1 128 - layer 2 64 - layer 3

RESULT

Classification of hand washing techniques

Optimal window size selection

Figure  displays the F1-score result of multiple sizes of windows and overlap rates. The result implied that with the increase in the window length, the classification performance rises as well. This is because more information can be extracted in a longer window, and it can reduce the impact of abrupt changes in the signal. The 0% overlap has a similar result as the 50% one, and it is even a little better than the 50% when the window length  equals to 2.56 s and 3.00 s. Hence, in the following study, the 2.56 s window with no overlap is selected.

[IMAGE OMITTED. SEE PDF]

Comparison among classifiers

Table  presents the classification result of each classifier for detecting hand washing classes. Among the 11 classes, KNN obtains the highest performance in classes b, h, and i, and SVM obtains the highest result in the remaining eight classes. As a result, SVM achieves the highest average F1-score of $0.8501\pm 0.1655$, followed by FFN with the F1-score of $0.8236\pm 0.2014$. The mean F1-score of class j is only 0.2286 using the KNN classifier, which impacts the average performance of KNN. This will be explained in detail in Section . The standard deviation (std) of the F1-score of all classifiers ranges from 0.1655 to 0.2799, which implies the subjective difference.

TABLE 3 The comparison of the F1-score (mean $\pm$ std) result of hand washing classes detection among classifiers. The highest result of each class is highlighted in bold

	Classes
Classifier	a	b	c	d	e	f	g	h	i	j	k	Mean
SVM	$\mathbf{0}.\mathbf{8332}\pm \mathbf{0}.\mathbf{1657}$	$0.6555\pm 0.2353$	$\mathbf{0}.\mathbf{9168}\pm \mathbf{0}.\mathbf{0714}$	$\mathbf{0}.\mathbf{9657}\pm \mathbf{0}.\mathbf{0698}$	$\mathbf{0}.\mathbf{9500}\pm \mathbf{0}.\mathbf{1500}$	$\mathbf{0}.\mathbf{9856}\pm \mathbf{0}.\mathbf{0300}$	$\mathbf{0}.\mathbf{9676}\pm \mathbf{0}.\mathbf{0408}$	$0.6883\pm 0.3679$	$0.6861\pm 0.3158$	$\mathbf{0}.\mathbf{7564}\pm \mathbf{0}.\mathbf{2898}$	$\mathbf{0}.\mathbf{9457}\pm \mathbf{0}.\mathbf{0842}$	$\mathbf{0}.\mathbf{8501}\pm \mathbf{0}.\mathbf{1655}$
KNN	$0.8349\pm 0.1743$	$\mathbf{0}.\mathbf{6589}\pm \mathbf{0}.\mathbf{2385}$	$0.8427\pm 0.0722$	$0.9314\pm 0.0859$	$0.9400\pm 0.0917$	$0.9404\pm 0.1176$	$0.9428\pm 0.0995$	$\mathbf{0}.\mathbf{7336}\pm \mathbf{0}.\mathbf{2945}$	$\mathbf{0}.\mathbf{6967}\pm \mathbf{0}.\mathbf{1939}$	$0.2286\pm 0.3255$	$0.9057\pm 0.1310$	$0.7869\pm 0.1659$
XGBoost	$0.7507\pm 0.1794$	$0.5657\pm 0.3006$	$0.8129\pm 0.2783$	$0.8350\pm 0.2950$	$0.7536\pm 0.2073$	$0.8365\pm 0.3192$	$0.9105\pm 0.1866$	$0.6163\pm 0.3541$	$0.6557\pm 0.2861$	$0.4919\pm 0.3693$	$0.7833\pm 0.3036$	$0.7284\pm 0.2799$
FFN	$0.7767\pm 0.1339$	$0.6217\pm 0.3581$	$0.9227\pm 0.0964$	$0.9300\pm 0.1522$	$0.8900\pm 0.1578$	$0.9856\pm 0.0300$	$0.9531\pm 0.0719$	$0.6832\pm 0.3974$	$0.6824\pm 0.3759$	$0.7186\pm 0.2833$	$0.8957\pm 0.1555$	$0.8236\pm 0.2014$

Misclassification among hand washing classes

Figure  presents the confusion matrix of the SVM model using features extracted from all sensors. The main misclassifications exist in three groups: classes a & b, classes h & i, and classes j & c. The misclassification between classes a and b could be because different people use different sides of the hand to open and turn off the faucet and get the soap. During the experiment, six of the participants used their right hands to open and turn off the faucet, and five of them kept using their right hand to get the soap. In addition, the major movement of classes a and b is the same, which is moving the hand from one point to another point. Classes h and i belong to the same step in the WHO regulation, but the hand side varies. While performing these two classes, participants were asked to rub their thumb, but different subjects did so differently, either rotating only one hand or both hands simultaneously. At last, the movements of classes c and j were similar, with the right hand rotating on the palm of the left hand. The only difference is that the fingers of the right hand were clasped while performing class j, which our sensors cannot detect. Thus, it is reasonable that these two classes are confused with each other.

[IMAGE OMITTED. SEE PDF]

Combination of sensors

Figure  presents the mean F1-score results of various combinations of sensors:

• only using the accelerometer on the left/right wrist
• only using accelerometer and gyroscope sensors on the left/right wrist
• using all the sensors on both wrists.

• The average results of only using the accelerometer on the left/right side are similar. The accelerometer worn on the left wrist performs better in detecting classes b, d and h. In contrast, the accelerometer on the right side achieves significantly better results in classes c, e, i, and k. The reason could be that the main wrist is different in every step, and the accelerometer is sensitive to the movement amplitude. Hence, although classes d and e are categorised as the same step in the WHO regulation, their classification results are different while using an accelerometer on a different side. This is the same reason for the results of classes h and i. For class k, the requirement was clasping the left finger on the right palm. Since among the participants, nine of them had the right wrists as their dominant wrists, they also rotated their right hands while performing class k. Moreover, the rotation amplitude was larger than the one of class j.
• After including the gyroscope sensors, the performance of the sensors on the right side is improved in most classes, except for class i. The average F1-score of using the accelerometer and gyroscope attached to the left wrist is a little higher than only using the accelerometer. This is because class b is getting the soap. Therefore, the gyroscope sensor can only detect rotations when the participants using their hands to catch the soap drops, but which hand to get the soap was varied in participants.
• Among all the combinations, using all the sensors on both sides achieved the best results, except for class b. This indicates the importance of the sensors on both sides and gyroscope sensors. The reasons for the reduced performance of class b can be that: (1) the number of windows of class b is smaller than other classes; (2) the movements performed in classes a and b were similar to each other. Hence, with the increased types of features, classes a and b were misclassified, which mainly reduced the precision and recall result.

[IMAGE OMITTED. SEE PDF]

Hand washing detection among other ADLs and with hand washing activity without following the WHO regulation

Combination of sensors

In the ADLs dataset, the right wrist was the dominant wrist of all participants. In addition, while performing brushing teeth, all participants held the toothbrush using their right wrists. Hence, in this part, the data of sensors attached to the right wrist were always included. The F1-score of trained hand washing of each combination is shown in Table . Using all the sensors achieved the highest result in detecting trained hand washing. In addition, the result of using both the accelerometer and gyroscope on the right wrist is around 0.2 higher than only using one accelerometer and 0.13 higher than using two accelerometers. This could be because hand washing is only associated with hands movements and most hand washing postures are periodic rotation movements, which is detectable by the gyroscope.

TABLE 4 The F1-score of trained hand washing among other daily activities with various combinations of sensors

Sensors	ACM- right wrist	ACM- both wrists	ACM + GYR - right wrist	All sensors
F1-score	0.7625	0.8256	0.9545	0.9871

Difference between trained and untrained hand washing activities

Figure  shows the confusion matrix of detecting hand washing activity among other ADLs using all types of sensors. The trained hand washing activity is rarely confused with other ADLs activities, implying that using wearable sensors can recognise the hand washing activities among the other seven ADLs. In addition, the trained hand washing class is not confused with the untrained one. The could be due to the types of movement, as the hand washing movement following the WHO regulation is more periodic than the untrained one. One example can be found in Figure .

[IMAGE OMITTED. SEE PDF]

[IMAGE OMITTED. SEE PDF]

DISCUSSION

Compared with previous studies

Table  summarises the accuracy results of every hand washing step of the proposed method and previous studies. As some previous studies [] only presented their results in figures, their results are not listed here. Study [] and study [] included hand washing steps 8–10: rinsing hands, drying hands, and turning off the faucet. However, we categorised these three steps into one class (class a). Hence, to comparison, the results of steps 8 and 9 of study [] are listed together in Table . Study [] and study [] separated class f into two classes, left and right, so there are two values in that column. As a result, the proposed method is comparable to the other studies, especially for classifying classes d, e, f, and k. However, the results of classes h, i, and j for the proposed method are lower than those of other studies. The reason could be that the number of sensors used in the proposed method is less than study [], or the dataset size is smaller.

TABLE 5 The comparison of the accuracy result of the proposed method with previous study

			Activities
			a					f	g					Drying
Study	Sensor	Classifier	Rinsing hands	Closing the faucet	b	c	d	e	Left	Right	Left	Right	h	i	j	k	hands
Proposed method	accelerometer+gyroscope	SVM	84%	71%	93%	97%	93%	99%	95%	76%	74%	77%	94%	-
[]	accelerometer+gyroscope+EMG	XGBoost+E.Divisive	100%	100%	-	83%	83%	99%	87%	86%	84%	84%	94%	94%	100%
[]	accelerometer+gyroscope	HMM	80%	75%		97%	82%	85%	93%	77%	93%	86%	75%	87%	89%	87%
[]	accelerometer+gyroscope	KNN	-	-	-	93%	91%	84%	93%	98%	94%	87%	88%	84%	84%	-

In our study, the optimal window size is 256 data points with no overlap, which is not in the range proposed by other studies that were 3-50 data points with 50–75% overlap. This shows that the optimal window size depends on the types and number of activity classes. For instance, in study [], the average duration of turning off the faucet was around 0.5 s, so it was reasonable to select the 0.2 s window with 75% overlap, and it can increase the detection precision in real-life application. However turning off the faucet is incorporated with turning on the faucet and rinsing hands as in one class in our study. For the overlap rate, although a higher overlap rate can enlarge dataset size, it can also increase the biased result with similar windows tested.

Compared to previous studies, our study has the advantages that:

• We include getting the soap as one class, even though the accuracy is 71%. Since getting soap was detected by the Bluetooth sensors integrated into the dispenser in the previous study [], our result from wearable sensors suggests that the IoT sensors in the dispenser can be removed, which would simplify the installation and reduce the cost. In addition, this study categorises opening the faucet, rinsing hands, and turning off the faucet into one class. This is because the period of each action is short, and they are typically performed sequentially.
• We prove the feasibility of recognising WHO recommended hand washing activity among other ADLs, using two wearable devices. Hence, the next step is to detect occasions in which hand hygiene is necessary, which can help people form good hand washing habits via sending alarm signals when they forget to wash their hands,  such as after going to the toilet.
• We analyse the difference between trained hand washing activity from the untrained one. This analysis result can be used to develop the training system to help people learn how to wash their hands properly.
• In our study, we do not consider the order of the hand washing steps. Thus, we used the SVM model. Moreover, the classification results of six classes are higher than the ones of study [] using HMM model, which implies that the order of hand washing steps is not essential to classification.

Limitations and future work

• Although SVM achieved 0.8501 for the average F1-score in classifying hand washing steps, the results of getting the soap (class b) and rubbing thumbs (classes h and i) are much lower than the average score, less than 0.7. Thus, the future study will focus on increasing the classification performance of these steps. The possible solution can be including other sensors, for instance, the pressure sensor/motion sensors for detecting getting the soap and EMG sensors for detecting the movement of fingers.
• In this study, we only considered one method of performing brushing teeth, which is using a manual toothbrush. The future work can also include samples of using an electric toothbrush.
• In this study, sensors attached to both wrists achieved the highest results for both classification of hand washing techniques and detecting hand washing activity among other ADLs. However, we mainly focused on the participants with the right wrist as the dominant side. The future study will also take participants with the left wrist as the dominant side into consideration.
• Byteflies sensors were used in our study, but they may not be convenient for users. Because the sensors were attached to the body via patches, which was not reusable. In the future, this sensor can be replaced with other similar devices, such as the smartwatch that also includes both the accelerometer and gyroscope sensors.

CONCLUSION

This study demonstrates the feasibility of a system using two Byteflies sensors and an SVM classifier to classify 11 hand washing steps. The result suggests that the optimal combination of the devices is two devices integrated with an accelerometer and a gyroscope, one device on each wrist. The hand washing activity can also be classified among other ADLs and untrained hand washing activity. The system will be improved to serve as a training system for individuals who need to form a good hand washing habit.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

ETHICAL APPROVAL

The experiments were approved by the KU LEUVEN Social and Societal Ethics Committee (SMEC), with the assigned serial number G-2020-2705. All participants signed the informed consent form before participating in the experiment.

DATA AVAILABILITY STATEMENT

The datasets analysed in this study are not shared due to privacy or ethical restrictions.

References

Burton, M., Cobb, E., Donachie, P., Judah, G., Curtis, V., Schmidt, W.‐P.: The effect of handwashing with water or soap on bacterial contamination of hands. Int. J. Environ. Res. Public Health 8(1), 97–104 (2011). [DOI: https://dx.doi.org/10.3390/ijerph8010097]