Introduction
Background
First responders, such as police officers and firefighters, are regularly faced with highly demanding working conditions [1] such as threatening or emotionally demanding situations. These roles frequently involve irregular shifts and peak workloads, sometimes compounded by understaffing [2]. Such conditions can lead to increased stress levels [3], reduced mental well-being [1,4], sleep disturbances [5], and difficulties in maintaining a healthy lifestyle [6], thereby escalating the risks of long-term health issues [7] and absenteeism [8]. Although some of the inherent challenges of these professions are unavoidable, it could be beneficial for employees to gain insights into their own stress levels, recovery patterns, and lifestyle-related factors like sleep and physical activity [9]. Such personal monitoring could help them to improve their stress- and well-being–related awareness and self-efficacy to make behavioral adjustments in order to better manage demanding aspects of their professions and thus optimize their well-being.
The emergence of wearable sensor technology facilitates promising opportunities to monitor a broad set of health-related outcomes [10]. Besides the initial use cases of wearables in estimating daily physical activity levels [11] and sleep duration [12], the introduction of photoplethysmography sensors enabled the measurement of other metrics such as heart rate [13], heart rate variability (HRV) [14], oxygen saturation [15], and potentially blood pressure [16]. Wearable-based measurements such as sleep and resting HRV have already been linked to stress- and well-being–related outcomes in first responders [17,18], but are sometimes also used to estimate other relevant outcomes. Using measures such as HRV as inputs, wearable companies now also estimate mental stress levels [19,20] and physiological recovery [21] in an attempt to provide more easily interpretable feedback. Although conceptually useful, in consumer-available wearables this is almost exclusively done via black box algorithms and the created measures do not always correlate optimally with similar subjective outcomes [22]. Despite these limitations and need for more scientific evaluation of the performance of these metrics, the wearables that utilize them are already broadly commercially available and adopted and are being integrated into wearable-based interventions [23]. In these wearable-based interventions, individuals use real-time data from physiological measurements to improve their health and performance, also known as biofeedback [24]. Given the continuing adoption of these wearables, it is important to understand the impact of these devices on behavior change and stress- and well-being–related outcomes.
The Effectiveness of Wearable-Based Interventions
Wearable-based interventions can be supportive at increasing physical activity [25], reducing body weight [25-27], and improving sleep outcomes [28]. Regarding the application of wearables to reduce stress and support mental well-being, most research efforts are focused on the modeling of stress-related outcomes using wearables [29]. Although studies show that wearable-based stress detection is feasible, this is especially true in controlled (eg, laboratory) settings, whereas challenges remain for accurate measurement in real-life circumstances [30]. A few studies have been performed investigating the effectiveness of wearable-based interventions to reduce stress and improve well-being, and they found some promising results [31-33]. A recent review that surveyed the broader literature on this topic showed that these effect studies tend to assess acute stress responses in laboratory settings, often in young (student) populations [34]. The authors concluded that there is therefore a need for more studies that investigate the effects of longitudinal wearable-based interventions on stress- and well-being–related outcomes in a natural, real-life context.
To improve our understanding of the effects of wearable-based interventions, it is not only helpful to gain insight into their direct impact on outcomes such as stress and mental well-being, but also in the possible precursors to those outcomes. Effects on these precursors could be meaningful even in the absence of short-term effects on the primary outcomes and may substantiate why the found effects occur. With regard to these precursors, the Health Belief Model proposes that health-related awareness (eg, on the susceptibility for and severity of the potential health problem) and perceived self-efficacy to cope with the situation are necessary to come to action and change behavior, which in turn may improve health-related outcomes [35,36]. Metacognitive awareness (MCA) has been shown to be related to learning [37] in general and learning about coping with stress specifically [38]. MCA is defined as the ability to reflect upon, understand, and regulate one’s learning [37]. This concept distinguishes between insight in one’s responses and potential causes and ways to regulate these responses. MCA helps people to develop strategies to change their behavior in accordance with the situation.
Self-efficacy is considered a key mechanism through which behavioral change can be observed [39]. This has also been shown for the way people cope with stress [40]. People’s beliefs about their ability to change behavior is determined, among other factors, by mastery experiences (succeeding at changing behavior) and vicarious learning (seeing other succeed). Daily feedback through monitoring can provide valuable insight into how to change behavior and mastery of these behaviors in attainable steps.
Since most wearables are designed to improve their users’ awareness of their own health-related behavior and provide them with leads on what needs to change and how they are progressing, improvement in health outcomes may be preceded by increased awareness and self-efficacy. Some studies have confirmed that wearable-based interventions can indeed positively influence health-related awareness [41] and self-efficacy in patient populations [42,43], but it is unknown to what extent these findings translate to occupational settings. Health-related behavior change interventions generally require a duration of months to years in order to ingrain the new behaviors [44], but when studying awareness and other precursors of behavior change, a shorter duration (eg, weeks) is sufficient.
Therefore, the first research question that this study addresses is the following:
Research question 1: Does offering a 5-week wearable-based intervention improve awareness, self-efficacy, and stress- and well-being–related outcomes in police officers in comparison to a control period?
The Potential Added Value of Enhanced Feedback and Implementation
For policymakers considering the deployment of wearable-based interventions, it is also helpful to know to what extent it is sufficient to use an “off-the-shelf” solution, where consumer-available wearables are directly deployed to the target group, or whether it is necessary, for instance, to provide (1) more extensive feedback and (2) (peer) support for the intervention to be effective. Based on self-efficacy theory, the latter could be of added value as it provides potential for vicarious learning and supports awareness of ways to attain mastery experience, whereas the former may contribute to the development of increased awareness.
First, it is possible to additionally collect subjective data via brief daily questionnaires, also known as ecological momentary assessment (EMA), and utilize the data to provide feedback on outcomes that cannot be measured using wearables. These self-report data can also be used to provide contextual and subjective information to the objective wearable-based measures [45]. Providing this more enriched feedback could potentially have beneficial effects on awareness as it links personal experience to objective measurement and provides information on the behavioral context that affects stress and well-being. This in turn can inform users on strategies to attain behavior change.
Second, organizing peer support groups can be a way to facilitate collaborative reflection and group learning [46]. Literature on the effectiveness of peer consultation groups in a preventive occupational health context is scarce, but there are indications that these can positively contribute to recovery from mental health conditions [47-49]. Although different from peer consultation through the guidance of more experienced professionals, peer supervision has also been shown to have positive effects on self-awareness and self-efficacy in occupational settings [50,51]. It is therefore possible that utilizing peer support groups may benefit the effectiveness of wearable-based interventions that aim to achieve health-related behavior change and optimization of stress- and well-being–related outcomes.
To contrast the effects of a condition that is enriched by both enhanced feedback and added peer support to a more simple “off-the-shelf” intervention that just utilizes a consumer-available wearable, the second research question is as follows:
Research question 2: Is offering a 5-week wearable-based intervention that is extended by EMA, personalized weekly feedback, and peer support meetings (“Wearable+”) more effective at improving awareness, self-efficacy, and stress- and well-being–related outcomes in police officers than an “off-the-shelf” (“Wearable”) intervention?
Methods
Ethical Considerations
The study protocol was approved (case 2022‐100) by the internal Research Ethics Committee of the Netherlands Organisation for Applied Scientific Research (TC-nWMO). Participation occurred voluntarily based on informed consent, and participants could opt out at any time without further consequences. Data were collected in a custom-built app that allowed for full control over the collected data without using consumer cloud services. During data collection, data were pseudonymously stored by using participant identifiers and anonymized after the data collection was completed by deleting the participant identifiers. Participants did not receive monetary compensation, but in each participating police team a lottery was held among the participants that adhered to the measurement protocol, with one wearable awarded per team.
Design
A mixed research design was applied, combining elements of a within-subject design with those of a between-subject design (Figure 1). In this study, 2 teams from 2 separate units of the Dutch police force were recruited. Of the teams in each unit, one team enrolled into the Wearable condition, whereas the other team enrolled into the Wearable+ condition (the between-subject comparison). Before undergoing the respective 5-week interventions, the participants in each team would first partake in a 5-week baseline control period (the within-subject comparison). This approach ensured that we would have a well-comparable control condition. No a priori stratification methods were applied, but χ2 tests to assess possible between-group differences in gender, age group, and wearable ownership before their participation were performed afterwards.
Enlarge this image.Figure 1. A mixed design was applied in this study, combining elements of within- and between-subject designs.
Participants
The recruitment of participants was initiated in 2 teams from 2 units of the Dutch police. In each unit, police officers of one team were recruited for the Wearable condition, whereas the police officers of the other team were recruited for the Wearable+ condition. In each team, up to 25 participants were recruited (up to 100 participants in total). This clustered recruitment method ensured geographical distribution of participants while preventing contamination of the different interventions within teams. Due to a lack of response in one team (the Wearable condition team in the rural unit), participants were also recruited from an additional team within that same unit.
In each team, recruitment was done via a combination of posters, flyers, a mention of the study during a team meeting, and an email to all police officers. Interested police officers would use an online tool to check if they met the inclusion criteria, which meant that they were (1) aged 18 years or older, (2) executive police officers, and (3) willing to put aside their own wearable during the duration of the study if they owned one. The police officers then received an information brochure about the study and filled in an online informed consent form before they received their baseline questionnaire and started their participation in the 5-week control period. To stimulate participant adherence during the intervention condition, within each team, a lottery was held that included the participants that collected complete daily data for at least 80% of their participation period and filled in all 5-weekly outcome questionnaires, where the winning participant got to keep the wearable that was used during the study.
Intervention
Control Condition
After filling in their baseline questionnaire, all participants first enrolled in a 5-week control period. Participants were instructed to live their lives as usual but to not use their own wearable (eg, smartwatch) if they owned one.
Wearable Condition
After the control period, the participants that enrolled into the Wearable condition received an “off-the-shelf” intervention that was based on just a consumer-available smartwatch. Due to the sensitive profession of the participants in combination with Dutch privacy laws, it was not possible to use a platform that stores data outside the European Union. To accommodate this need, a Garmin Forerunner 255 smartwatch was used in combination with a smartphone app that was approved for use with the Dutch police (built by Sensorium 69 BV), as it directed data directly from the device to self-controlled servers. The app displayed much of the same data as the Garmin Connect app that regular Garmin consumers use (Figure 2). The displayed wearable-based metrics were steps, walking distance, active calories, total sleeping time, resting and maximum heart rate, physiological stress, and a “body battery” metric that estimates the individual’s energy levels. The wearable also measured the daily HRV status and (after 3 weeks) provided feedback on the device itself regarding the extent in which the participant’s resting physiology was in a normal state; if it was not, the device would advise the individual to take it easy that day. Other functions such as sports tracking were also only available on the wearable device itself.
During the 5-week intervention period, Wearable condition participants used the wearable and app at their own discretion. The researchers were available via email and a messaging app to help with technical issues, but no other support was given.
Enlarge this image.Figure 2. Screenshots of the (Dutch-language) Sensorium smartphone app that used the Garmin Health software development kit to directly retrieve data from the Garmin Forerunner 255. From left-to-right, the screenshots display the home screen dashboard (the version of the Wearable+ condition, which includes answers to the ecological momentary assessment questionnaires on the bottom right corner), a monthly view of sleep periods, a daily view of physiological stress, and a weekly view of the “body battery” metric, which estimates the individual’s energy levels.
Wearable+ Condition
The Wearable+ condition represented an enriched situation, where a consumer-available wearable was complemented by additional feedback and support. The composition of the Wearable+ condition was based on a prior (unpublished) study that was done with the Dutch police force, where police officers tested the “off-the-shelf” solution of the current Wearable condition in this study and described how the intervention could be improved to more optimally fit their needs and working context.
Participants enrolled in the 5-week Wearable+ condition used the same Garmin Forerunner 255 that the participants in the regular Wearable condition used. The accompanying custom-built smartphone app was also the same, but it included a feature that allowed the Wearable+ condition participants to also fill in a brief daily EMA questionnaire. The daily EMA questionnaire was available each day during the late afternoon, and participants were instructed to fill it in before they went to bed. The questionnaire contained items on the type of shift that they worked that day, as well as their perceived stress, need for recovery, affective state (valence and arousal), and an optional custom annotation to report incidents or issues that they thought were potentially relevant, where they could write a brief note of up to 35 characters.
The EMA questionnaire data were visible for the participants in their app and were also used for more comprehensive personalized feedback reports that the participants received on a weekly basis. In the clickable weekly feedback report that offered tabs with alternative views and metrics (based on an R Notebook), the data of the subjective EMA outcomes were combined with the objective wearable-based outcomes in order to provide enriched contextual information and compare trends. In addition, textual messages were added to support participants in reflecting on the data. The contextual information could help participants in the interpretation of the wearable-based outcomes, for instance, to better understand why certain scores were abnormally low or high and thus identify potentially actionable associations. When little EMA data was available, the participant was reminded to fill in their daily EMA questionnaire, as it would improve the richness of the feedback report. In the second-to-last week of the intervention period, the report also included a personalized reflection on the data (eg, highlighting relevant trends or associations) and, if possible, advice from the researchers (HJdV, HV, and MvZ) based on the data of the participant. This advice was seen as a proxy for a future algorithm-based function that provides overarching personalized advice based on the broader range of available data sources.
Finally, peer support groups were organized twice per week, where researchers (HV and MvZ) were also available, for instance, to help with the interpretation of the weekly reports. In the peer support groups, participants primarily discussed their personal experiences, for example, with the smartwatch, personal feedback reports, and potential lifestyle changes that they were considering or executing. The personal data of the feedback reports of the participants were not publicly shared by the researchers, but participant questions on the interpretation of certain outcomes were discussed in general terms in order to optimize participants’ understanding of the feedback reports. Additionally, participants could contact the researchers via email, telephone, or an online video call on an individual basis for specific questions about their own device or data.
Data Collection
Overview
The intervention effects were assessed via an online questionnaire that was administered at baseline (T0), after the control period (T1), and after the intervention period (T2). For descriptive purposes, questions on the participants’ gender, age category (under 30 years old, 30-45 years old, older than 45 years), and whether they already owned a wearable before participating in the study were included. Wearable ownership was also used as a covariate during hypothesis testing to correct for potential differences in outcome variables among participants that did or did not own a wearable prior to their participation.
Metacognitive Awareness About Sleep, Stress, and Activity Assessment
To determine whether the intervention influenced the participants’ stress- and well-being–related awareness, we used the Metacognitive Awareness about Sleep, Stress, and Activity Assessment (MASSAA). This scale was developed in a pilot study in preparation for this study for the Dutch police [52]. Items are scored on a 7-point Likert scale ranging from “completely disagree” to “completely agree.” Items fall into 3 categories that are in line with the concept of MCA, namely awareness about responses, causes, and regulation. After performing post hoc factor analysis on the collected data, the responses to the used awareness and cause subscales were found to be very highly interrelated, after which these subscales were combined to minimize the number of performed analyses. Reliability of the subscales was good. Therefore, a total of 6 MASSAA subscales were assessed, namely stress awareness and cause (α=.87), stress regulation (α=.91), sleep awareness and cause (α=.86), sleep regulation (α=.93), physical activity awareness and cause (α=.90), and physical activity regulation (α=.92). The complete MASSAA questionnaire (both the original Dutch items and their English translations) can be found in Multimedia Appendix 1.
Self-Efficacy
Self-efficacy was measured via a series of propositions, which were scored on a 5-point Likert scale that ranges from “completely disagree” to “completely agree,” in order to assess if the intervention also affected the individual’s perceived capacity to execute behaviors necessary to produce specific outcomes [39] and therefore possibly indirectly affected stress-related outcomes [53]. Separate scales were used. First, a scale for behavior change relating to stress, recovery, sleep and activity was used. This scale was developed in a pilot study in preparation for this study. Reliability was adequate (α=.79). Furthermore, self-efficacy scales with respect to stress resilience (α=.82), coping (α=.83) and tasks (α=.86) were included [54]. For completeness, all used self-efficacy scales and items can be found in Multimedia Appendix 2.
Stress and Well-Being
Several stress- and well-being–related outcomes were also monitored. The participants’ perceived need for recovery during and after work were queried using a series of propositions that were scored on a 5-point Likert scale that ranges from “completely disagree” to “completely agree” [55]. Reliability was good, with an α of .79 and .87 for recovery during and after work, respectively. A 3-item subscale for sleep issues regarding waking up too early, sleeping through, and having trouble falling asleep was also included and scored on a 5-point Likert scale, ranging from “never” to “always.” Reliability was adequate (α=.74). The subscales for recovery during and after work, as well as for sleep issues, can be found in Multimedia Appendix 2. For the measurement of the emotional, psychological and social well-being of participants, the 14-item Dutch version of the Mental Health Continuum-Short Form (MHC-SF) was administered [56]. The MHC-SF asks how often the respondent experienced a range of well-being–related feelings during the last month and is scored on a 6-point Likert scale that ranges from “never” to “daily.” Reliability was good (α=.92). Finally, the 10-item version of the Perceived Stress Scale, 10-item version (PSS-10) was taken [57]. The PSS-10 inquires about the frequency of the occurrence of stress-related feelings and thoughts during the last month, and it is evaluated on a 5-point Likert scale that ranges from “never” to “very often.” Reliability was good (α=.78).
Data Analysis
Data management and statistical analyses were performed in RStudio (version 2023.06.2) [58] using R (version 4.2.2; R Foundation for Statistical Computing) [59].
Data Management
The data of participants that used their own wearable during the study or that did not fill in all outcome questionnaires (eg, dropouts or nonresponse) were filtered out. For the description of the participants’ gender, age, and wearable ownership prior to the study, as well as the statistical testing for potential between-group baseline differences (via χ2 tests) and the development of supporting data visualizations, the observed data were used. For hypothesis testing, the data were first standardized at the grand mean [60].
Hypothesis Testing
Hypothesis testing was done using version 1.1‐33 of the lme4 R package, which allows for statistical testing with linear mixed models (LMMs) [61]. For each of the 15 awareness, self-efficacy, and well-being–related outcomes, modeling was performed using a 3-step process.
For analysis 1, a first model was formed based on solely the confounding variable wearable ownership prior to participating in the study as a fixed effect and the participant identifier as a random effect to account for repeated measures within-subject. Second, the model was extended by including the main variables of interest “time” (a binary label for the repeated measures before and after the control and intervention periods) and “period” (labels for the control and intervention periods). The third and final model was then extended by the interaction effect of time and period, which assesses if the changes over time differed between the control and intervention periods; it is therefore the key focus for answering research question 1. For each model, the marginal R2 (mR2) and conditional R2 (cR2) [62], as well as the difference (Δ) in each in comparison to the previous step, were described. For the final model, absolute values of Hedges g statistic were also calculated to estimate the observed effect size, along with its 95% CI. In the interpretation of Hedges g, effects below 0.20 were considered to be “very small,” between 0.20‐0.49 as “small,” between 0.50‐0.79 as “medium,” and above 0.8 as “large” [63]. To accommodate the interpretation of the results, a data visualization with interaction plots for each of the 15 outcomes was created.
For analysis 2, this same approach was used, but also included the main effects of the “group” variable (Wearable or Wearable+) in step 2 and the 3-way effect of time, period, and group in step 3. The latter evaluates if the effectiveness of the intervention in comparison to the control condition differed between the Wearable and Wearable+ groups and therefore contributes to answering research question 2.
Results
Overview
A total of 95 participants were recruited (48 Wearable, 47 Wearable+), of which 16 participants (7 Wearable, 9 Wearable+) were excluded from statistical analysis. Of the excluded participants, 12 (3 Wearable, 9 Wearable+) used their own private wearable during the control period, 2 (Wearable) dropped out, and 2 (Wearable) did not fill in the final (T2) questionnaire. The data of the remaining 79 participants—who were predominantly males (n=57, 72%) in the age category of 30-45 years (n=46, 58%) and did not own a wearable before participating in the study (n=47, 59%)—were analyzed. Descriptive statistics of the analyzed participants of both intervention groups and the total sample are described in Table 1. No statistically significant baseline differences between the Wearable and Wearable+ groups in gender, age category, and wearable ownership prior to participation were identified based on χ2 testing.
Table 1.Descriptive statistics of the analyzed participants in the Wearable group, Wearable+ group, and total sample.
| Wearable, n (%) | Wearable+, n (%) | Total, n (%) | ||
| Participants | 41 (52) | 38 (48) | 79 (100) | |
| Gender | ||||
| Male | 28 (68) | 29 (76) | 57 (72) | |
| Female | 13 (32) | 9 (24) | 22 (28) | |
| Age (years) | ||||
| < 30 | 12 (29) | 4 (11) | 16 (20) | |
| 30‐45 | 21 (51) | 25 (66) | 46 (58) | |
| > 45 | 8 (20) | 9 (24) | 17 (22) | |
| Wearable owner | ||||
| False | 27 (66) | 20 (53) | 47 (59) | |
| True | 14 (34) | 18 (47) | 32 (41) | |
Analysis 1: Effect of the Intervention Conditions Against the Control Condition
LMMs were created for each of the 15 awareness-, self-efficacy–, and well-being–related outcomes. Statistically significant 2-way interaction effects were found in 8 outcomes, namely MCA stress awareness and cause (P=.04, mR2=.02, cR2=.59, Hedges g=0.30, 95% CI 0.01-0.58), MCA stress regulation (P=.002, mR2=.02, cR2=.66, Hedges g=0.41, 95% CI 0.16-0.66), MCA physical activity awareness and cause (P=.03, mR2=.07, cR2=.64, Hedges g=0.30, 95% CI 0.04-0.57), self-efficacy behavior change (P=.002, mR2=.06, cR2=.73, Hedges g=0.37, 95% CI 0.13-0.60), self-efficacy stress resilience (P=.04, mR2=.03, cR2=.71, Hedges g=0.25, 95% CI 0.01-0.50), and self-efficacy task (P=.03, mR2=.02, cR2=.68, Hedges g=0.29, 95% CI 0.04-0.54), as well as the MHC-SF (P < .001, mR2=.02, cR2=.74, Hedges g=0.40, 95% CI 0.17-0.63) and PSS-10 (P < .001, mR2=.02, cR2=.71, Hedges g=0.46, 95% CI 0.22-0.70). The detailed results for the 3-step LMM process for each of the 15 outcomes is available in Multimedia Appendix 3. The interaction plots in Figure 3 summarize these findings by visualizing the average scores before (“0” on x-axis) and after (“1” on x-axis) the control (gray) and intervention (blue) conditions, and labeling the statistically significant interaction effects.
Enlarge this image.Figure 3. Interaction plots to support the interpretation of the results of analysis 1, which investigated the effectiveness of a wearable-based intervention in comparison to a control condition. For each of the 15 awareness-, self-efficacy–, and well-being–related outcomes, the mean scores before (“0” on x-axis) and after (“1” on x-axis) the control (gray) or intervention condition (blue) are visualized. Interaction plots with statistically significant (P < .05) 2-way interaction effects are annotated (*). AC: awareness and cause; AW: after work; BC: behavior change; DW: during work; MCA: metacognitive awareness; MHC-SF: Mental Health Continuum-Short Form; PA: physical activity; PSS-10: Perceived Stress Scale, 10-item version; R: regulation; SE: self-efficacy; SR: stress resilience.
For each of the 8 statistically significant outcomes, the wearable-based intervention had a favorable effect in comparison to the control condition. For 6 of the remaining 7 nonstatistically significant outcomes, the direction of the effect also favored the intervention, with the exception of recovery during work. Therefore, the observed effects consistently favored the wearable-based intervention over the control period, for which the statistically significant effects were estimated to be “small” (Hedges g=0.25-0.46) in size. In these outcomes, the fixed effects (the control or intervention period) explained between 2% and 7% of the variance, whereas the full model including the random effect (the participant identifier) explained between 59% and 74% of the variance in the outcomes.
Analysis 2: Effects in the Wearable+ Group Compared to the Wearable Group
In the LMMs that were created for analysis 2, statistically significant 3-way interaction effects were found in 3 of 15 outcomes, namely self-efficacy behavior change (P=.02, mR2=.10, cR2=.75, Hedges g=0.58, 95% CI 0.11-1.05), recovery after work (P < .05, mR2=.01, cR2=.83, Hedges g=0.41, 95% CI 0.04-0.77), and sleep issues (P=.03, mR2=.02, cR2=.77, Hedges g=0.45, 95% CI 0.02-0.88). The detailed results for the 3-step LMM process are available in Multimedia Appendix 4. The interaction plots in Figure 4 summarize these findings by visualizing the average scores before (“0” on x-axis) and after (“1” on x-axis) the control (gray) and intervention (blue) conditions for both the Wearable (striped line) and Wearable+ (solid line) groups and labeling the statistically significant interaction effects.
Enlarge this image.Figure 4. Interaction plots to support the interpretation of the results of analysis 2, which investigated possible differences in the effectiveness in 2 wearable-based interventions. For each of the 15 awareness-, self-efficacy–, and well-being–related outcomes, the mean scores before (“0” on x-axis) and after (“1” on x-axis) the control (gray) or intervention condition (blue) are visualized, for both the Wearable (striped line) and Wearable+ (solid line) groups. Interaction plots with statistically significant (P < .05) 3-way interaction effects are annotated (*). AC: awareness and cause; AW: after work; BC: behavior change; DW: during work; MCA: metacognitive awareness; MHC-SF: Mental Health Continuum-Short Form; PA: physical activity; PSS-10: Perceived Stress Scale, 10-item version; R: regulation; SE: self-efficacy; SR: stress resilience.
Based on the interaction plots of the outcomes with statistically significant 3-way interaction effects (Figure 4), the enhanced Wearable+ intervention was more effective than the “off-the-shelf” Wearable intervention for improving recovery after work, but less effective for improving self-efficacy in behavior change and perceived sleep issues. However, the plots of these 3 outcomes all showed a noticeable change during the control period, all of which benefited the intervention group, which demonstrated statistically significant improvements in the respective analyses. Therefore, these changes during the control period may have influenced the results for these outcomes. Furthermore, the broad 95% CIs of the estimated effect sizes, compared to those in analysis 1, indicate decreased certainty in the magnitude of the reported effects. The results of analysis 2 should therefore be interpreted more cautiously than the results of analysis 1.
Discussion
Principal Results
The Effectiveness of Wearable-Based Interventions (Analysis 1)
The first aim of this study was to assess whether offering a 5-week wearable-based intervention would improve awareness, self-efficacy, and stress- and well-being–related outcomes in police officers in comparison to a control period. The results of analysis 1 showed that the wearable-based intervention consistently benefited the participants in 8 of the 15 outcomes, namely MCA of stress awareness and cause, MCA of stress regulation, MCA of physical activity awareness and cause, self-efficacy in behavior change, self-efficacy in stress resilience, self-efficacy in task execution, well-being, and stress. This means that the wearable-based intervention had a direct positive influence on the primary stress and well-being outcomes, as well as several aspects of MCA and self-efficacy, which were seen as precursors to relevant behavior change, based on the Health Behavior Model [35,36].
The effect sizes of the 8 statistically significant outcomes were estimated to be “small” (Hedges g=0.25-0.46). The variance that could be explained by the fixed intervention effects only was 2%‐7% (based on the mR2), whereas the variance that could be explained when also including the random effects of participant identifiers ranged from 59%‐74% (based on the cR2). This indicates that there are large interindividual differences among participants that can potentially play a role in how individuals respond to the intervention. Future studies with a larger sample size are needed to understand which individual factors influence the differential effectiveness of the intervention and the extent to which personalized approaches could enhance the effectiveness of the intervention, taking into account the variability among participants.
Based on the results of analysis 1, offering a 5-week wearable-based intervention has a small but positive impact on stress- and well-being–related outcomes in police officers. Further studies with a larger sample size are needed to explore if the intervention is particularly effective for certain subgroups and whether more personalized approaches could better accommodate interindividual differences, thereby potentially increasing its overall effectiveness.
The Potential Added Value of Enhanced Feedback and Implementation (Analysis 2)
This study also aimed to investigate if offering an enriched wearable-based intervention that is extended by additional EMA questionnaires, personalized weekly feedback, and peer support meetings would be more effective than offering a more basic “off-the-shelf” wearable-based intervention. Analysis 2 showed mixed results, where the enriched intervention was more effective than the basic intervention at improving recovery after work, less effective for self-efficacy in behavior change and sleep issues, and similarly effective for the other 12 outcomes.
Similar to the results of analysis 1, the estimated effect sizes of the differences between the Wearable and Wearable+ intervention groups were small (Hedges g=0.41-0.45) to medium (0.58), although the much larger confidence intervals in analysis 2 suggest that these results should be interpreted more cautiously. This, combined with the observation that changes in the control period might have exacerbated the between-group effects in the 3 statistically significant outcomes, warrants the conclusion that confirmation in future studies with a larger sample size is desired before policy decisions are made based on these insights. Due to the again observed large differences between the variance that were explained by the effects of the 2 interventions (1%‐10%) and by the full models (75%‐83%), exploration regarding whether certain individuals may benefit more from either intervention and whether additional personalization may increase their overall effectiveness would be interesting.
In view of these findings, the enriched wearable-based intervention was not more effective than the basic wearable-based intervention in its current form. Further research is needed to identify which individuals would particularly benefit from either intervention variation and explore if further personalization could increase their overall effectiveness.
Strengths and Limitations
This research contributes to the literature by assessing the effectiveness of wearable-based interventions for reducing stress and improving well-being in a real-world context. By utilizing consumer-available wearables and investigating the impact of the interventions over several weeks in a real-life setting, where the participants are dealing with true life demands, the current findings can be generalized toward occupational settings, especially highly demanding work settings. Additionally, this study used a nested design that optimized the comparability between the control and intervention periods (ie, for analysis 1). Due to the clustered recruitment strategy and geographical distribution across teams, the potential for spillover effects between the 2 investigated wearable-based interventions was considered to be low (ie, for analysis 2).
A limitation of the study that could have caused a potential underestimation of the true effect sizes via ceiling effects [64] is the inclusion of a relatively healthy population. To at least partially account for this limitation, an LMM approach that was optimized for repeated measures (and therefore accounted for potential baseline differences between participants) was used in favor of using change scores [65]. Another limitation was that the Wearable+ intervention simulated the application of enriched feedback by providing custom-made feedback reports. It is possible that more integrated and continuously available feedback could be more user-friendly and potentially more effective.
Comparison With Prior Work
The results of analysis 1, which showed that offering a wearable-based intervention improved stress- and well-being–related outcomes, are in line with those of prior studies that investigated the effectiveness of wearables in real-life settings. For instance, Ponzo et al [31] reported positive effects on anxiety, depression, and general well-being following a 4-week intervention that used a wearable device (BioBeam) that measured sleep, physical activity, and heart rate, as well as an app that utilized EMA and relaxation exercises based on a randomized controlled trial in a student population. Another randomized controlled trial, by Smith et al [32], showed favorable effects from a 4-week intervention on stress and anxiety, but used a wearable (Spire Stone) that primarily focuses on optimizing breathing patterns. Other studies on the effectiveness of wearable-based interventions were often performed in laboratory settings, often in young (student) populations [34]. The results of this study therefore expand upon this prior work by exploring the effectiveness of wearable-based interventions in a working population with high job demands and assessing actual changes in lifestyle outcomes and potential precursors of those in a real-life context.
We are not aware of similar prior work comparing the effectiveness of enriched and basic wearable-based interventions as in analysis 2. The difference between our enriched and basic wearable-based intervention can be broken down into the addition of (1) EMA questionnaires, (2) complementary and personalized feedback, and (3) peer support groups. The potential of combining wearable data with EMA questionnaires and complementary feedback is an emerging research topic in this field [66-68], but information about the benefits of EMA and expanded feedback next to wearable data is scarce. Similarly, there are indications that peer support groups can have a positive effect on the recovery of patients with mental health conditions [47-49] and may benefit self-care in patients with medically unexplained symptoms [46], but the effects in a preventive occupational setting are unknown. The lack of prior research on these specific topics highlights the relevance of this study, though based on our results, no clear benefits of these 3 extensions to the basic wearable-based intervention were identified.
Conclusions
Given the evidence obtained from our analyses, offering a 5-week wearable-based intervention to police officers has small but consistent positive effects on their stress, well-being, and related MCA and self-efficacy. The results also showed that there are large interindividual differences in the outcomes that were analyzed, which may be a sign that there is potential for further assessment of possible differences in effectiveness in subgroups, as well as for improvements in the overall effectiveness via more personalized approaches. Complementing the basic “off-the-shelf” wearable-based intervention with additional EMA questionnaires, weekly personalized feedback reports, and peer support groups did not appear to improve the effectiveness of the intervention in its current form. Future research is needed to better understand how the properties of the enriched condition can be improved to increase the intervention’s effectiveness and which participants may or may not benefit from such extended feedback and support.
Acknowledgments
The authors thank the Dutch Police Force for the collaboration, especially the officers who participated or helped us during the recruitment and data collection process. We also thank Sensorium BV for facilitating the data collection via their platform. This study was funded by the Dutch Ministry of Economic Affairs and Climate and the Dutch Ministry of Justice.
Authors' Contributions
RD led the overarching program that this study was part of. WK led this study. All authors contributed to the study design. RD, HV, MvZ, and HJdV assisted in the recruitment of participants. RD, HV, and MvZ facilitated the peer support groups. HJdV created the weekly feedback reports, for which HV and MvZ assisted in the development of personalized advice. RD, WK, and HJdV analyzed the data. HJdV created the data visualizations and wrote the first draft of this article. RD and WK cowrote the final version via feedback and suggested revisions. All coauthors approved the final draft of this article.
Conflicts of Interest
None declared.
------------
Abbreviations
| cR2: conditional R2 |
| EMA: ecological momentary assessment |
| HRV: heart rate variability |
| LMM: linear mixed model |
| MASSAA: Metacognitive Awareness about Sleep, Stress, and Activity Assessment |
| MCA: metacognitive awareness |
| MHC-SF: Mental Health Continuum-Short Form |
| mR2: marginal R2 |
| PSS-10: Perceived Stress Scale, 10-item version |
------------
Edited by Lorraine Buis; submitted 19.05.24; peer-reviewed by Maria Luisa González Ramírez, T Chitra; final revised version received 15.08.24; accepted 14.10.24; published 28.01.25.
Bevan MP, Priest SJ, Plume RC, Wilson EE. Emergency first responders and professional wellbeing: a qualitative systematic review. Int J Environ Res Public Health. Nov 8, 2022;19(22):14649.
Collins PA, Gibbs ACC. Stress in police officers: a study of the origins, prevalence and severity of stress-related symptoms within a county police force. Occup Med. Jun 1, 2003;53(4):256-264.
Cheung YK, Li JCM. Predictors, mediators and moderators of police work-related stress: a scoping review. Int J Environ Res Public Health. Jan 27, 2023;20(3):2253.
Naushad VA, Bierens JJ, Nishan KP, et al. A systematic review of the impact of disaster on the mental health of medical responders. Prehosp Disaster Med. Dec 2019;34(6):632-643.
Näslund-Koch C, Vedel-Krogh S, Bojesen SE, Skov L. Plasma vitamin D is not associated with moderate-to-severe psoriasis: results from Danish general population studies. J Invest Dermatol. Oct 2023;143(10):2068-2071.
Yao L, Xiong Y, Yuan F, Luo Y, Yan L, Li Y. Perceived stress and its impact on the health behavior of Chinese residents during the COVID-19 epidemic: an internet-based cross-sectional survey. Health Sci Rep. Sep 2022;5(5):e778.
Magnavita N, Capitanelli I, Garbarino S, Pira E. Work-related stress as a cardiovascular risk factor in police officers: a systematic review of evidence. Int Arch Occup Environ Health. May 2018;91(4):377-389.
Magnavita N, Garbarino S. Is absence related to work stress? A repeated cross‐sectional study on a special police force. Am J Ind Med. Jul 2013;56(7):765-775.
Grothe J, Tucker S, Blake A, et al. Exploring first responders’ use and perceptions on continuous health and environmental monitoring. Int J Environ Res Public Health. Mar 8, 2023;20(6):4787.
Huhn S, Axt M, Gunga HC, et al. The impact of wearable technologies in health research: scoping review. JMIR mHealth uHealth. Jan 25, 2022;10(1):e34384.
Germini F, Noronha N, Borg Debono V, et al. Accuracy and acceptability of wrist-wearable activity-tracking devices. Syst Rev Lit J Med Internet Res. Jan 21, 2022;24(1):e30791.
Haghayegh S, Khoshnevis S, Smolensky MH, Diller KR, Castriotta RJ. Accuracy of wristband Fitbit models in assessing sleep: systematic review and meta-analysis. J Med Internet Res. Nov 28, 2019;21(11):e16273.
Zhang Y, Weaver RG, Armstrong B, Burkart S, Zhang S, Beets MW. Validity of wrist-worn photoplethysmography devices to measure heart rate: a systematic review and meta-analysis. J Sports Sci. Sep 2020;38(17):2021-2034.
Georgiou K, Larentzakis AV, Khamis NN, Alsuhaibani GI, Alaska YA, Giallafos EJ. Can wearable devices accurately measure heart rate variability? A systematic review. Folia Med (Plovdiv). Mar 1, 2018;60(1):7-20.
Almarshad MA, Islam MS, Al-Ahmadi S, BaHammam AS. Diagnostic features and potential applications of PPG signal in healthcare: a systematic review. Healthcare (Basel). Mar 16, 2022;10(3):547.
Elgendi M, Fletcher R, Liang Y, et al. The use of photoplethysmography for assessing hypertension. NPJ Digit Med. 2019;2:60.
de Vries H, Kamphuis W, van der Schans C, Sanderman R, Oldenhuis H. Trends in daily heart rate variability fluctuations are associated with longitudinal changes in stress and somatisation in police officers. Healthcare (Basel). Jan 12, 2022;10(1):144.
de Vries HJ, Pennings HJM, van der Schans CP, Sanderman R, Oldenhuis HKE, Kamphuis W. Wearable-measured sleep and resting heart rate variability as an outcome of and predictor for subjective stress measures: a multiple n-of-1 observational study. Sensors (Basel). Dec 28, 2022;23(1):332.
Hickey BA, Chalmers T, Newton P, et al. Smart devices and wearable technologies to detect and monitor mental health conditions and stress: a systematic review. Sensors (Basel). May 16, 2021;21(10):3461.
Morales A, Barbosa M, Morás L, et al. Occupational stress monitoring using biomarkers and smartwatches: a systematic review. Sensors (Basel). Sep 2, 2022;22(17):6633.
Greenwalt CE, Angeles E, Vukovich MD, et al. Pre-sleep feeding, sleep quality, and markers of recovery in division I NCAA female soccer players. J Int Soc Sports Nutr. Dec 2023;20(1):2236055.
Lundstrom EA, De Souza MJ, Koltun KJ, Strock NCA, Canil HN, Williams NI. Wearable technology metrics are associated with energy deficiency and psychological stress in elite swimmers. Int J Sports Sci Coach. Aug 2024;19(4):1578-1587.
Jerath R, Syam M, Ahmed S. The future of stress management: integration of smartwatches and HRV technology. Sensors (Basel). Aug 22, 2023;23(17):7314.
Schwartz MS, Andrasik F. Biofeedback: A Practitioner’s Guide. Guilford Publications; 2017. ISBN: 1-4625-3194-6
Ferguson T, Olds T, Curtis R, et al. Effectiveness of wearable activity trackers to increase physical activity and improve health: a systematic review of systematic reviews and meta-analyses. Lancet Digit Health. Aug 2022;4(8):e615-e626.
Cheatham SW, Stull KR, Fantigrassi M, Motel I. The efficacy of wearable activity tracking technology as part of a weight loss program: a systematic review. J Sports Med Phys Fitness. Apr 2018;58(4):534-548.
Goode AP, Hall KS, Batch BC, et al. The impact of interventions that integrate accelerometers on physical activity and weight loss: a systematic review. Ann Behav Med. Feb 2017;51(1):79-93.
Lai MYC, Mong MSA, Cheng LJ, Lau Y. The effect of wearable‐delivered sleep interventions on sleep outcomes among adults: a systematic review and meta‐analysis of randomized controlled trials. Nurs Health Sci. Mar 2023;25(1):44-62.
Namvari M, Lipoth J, Knight S, et al. Photoplethysmography enabled wearable devices and stress detection: a scoping review. J Pers Med. Oct 31, 2022;12(11):1792.
Gedam S, Paul S. A review on mental stress detection using wearable sensors and machine learning techniques. IEEE Access. 2021;9:84045-84066.
Ponzo S, Morelli D, Kawadler JM, Hemmings NR, Bird G, Plans D. Efficacy of the digital therapeutic mobile app BioBase to reduce stress and improve mental well-being among university students: randomized controlled trial. JMIR mHealth uHealth. Apr 6, 2020;8(4):e17767.
Smith EN, Santoro E, Moraveji N, Susi M, Crum AJ. Integrating wearables in stress management interventions: promising evidence from a randomized trial. Int J Stress Manag. 2020;27(2):172-182.
Kocielnik R, Sidorova N. Personalized stress management: enabling stress monitoring with LifelogExplorer. Künstl Intell. Jun 2015;29(2):115-122.
González Ramírez ML, García Vázquez JP, Rodríguez MD, Padilla-López LA, Galindo-Aldana GM, Cuevas-González D. Wearables for stress management: a scoping review. Healthcare (Basel). Aug 22, 2023;11(17):2369.
Janz NK, Becker MH. The Health Belief Model: a decade later. Health Educ Q. 1984;11(1):1-47.
Rosenstock IM, Strecher VJ, Becker MH. Social learning theory and the Health Belief Model. Health Educ Q. 1988;15(2):175-183.
Schraw G, Dennison RS. Assessing metacognitive awareness. Contemp Educ Psychol. Oct 1994;19(4):460-475.
Delahaij R, van Dam K. Coping style development: the role of learning goal orientation and metacognitive awareness. Pers Individ Dif. Apr 2016;92:57-62.
Bandura A. Self-efficacy mechanism in human agency. Am Psychol. 1982;37(2):122-147.
Delahaij R, van Dam K, Gaillard AWK, Soeters J. Predicting performance under acute stress: the role of individual characteristics. Int J Stress Manag. 2011;18(1):49-66.
Edward KL, Garvey L, Aziz Rahman M. Wearable activity trackers and health awareness: nursing implications. Int J Nurs Sci. Apr 10, 2020;7(2):179-183.
Park Y, Kim SR, So HY, et al. Effect of mobile health intervention for self-management on self-efficacy, motor and non-motor symptoms, self-management, and quality of life in people with Parkinson’s disease: randomized controlled trial. Geriatr Nurs (Lond). Jul 2022;46:90-97.
Wulfovich S, Fiordelli M, Rivas H, Concepcion W, Wac K. “I must try harder”: design implications for mobile apps and wearables contributing to self-efficacy of patients with chronic conditions. Front Psychol. 2019;10:2388.
Singh N, Stewart RAH, Benatar JR. Intensity and duration of lifestyle interventions for long-term weight loss and association with mortality: a meta-analysis of randomised trials. BMJ Open. Aug 18, 2019;9(8):e029966.
Gromatsky M, Sullivan SR, Spears AP, et al. Ecological momentary assessment (EMA) of mental health outcomes in veterans and servicemembers: a scoping review. Psychiatry Res. Oct 2020;292:113359.
Nebeker C, Weisberg B, Hekler E, Kurisu M. Using self-study and peer-to-peer support to change “sick” care to “health” care: the patient perspective. Front Digit Health. 2020;2:2.
Smit D, Miguel C, Vrijsen JN, Groeneweg B, Spijker J, Cuijpers P. The effectiveness of peer support for individuals with mental illness: systematic review and meta-analysis. Psychol Med. Aug 2023;53(11):5332-5341.
Lyons N, Cooper C, Lloyd-Evans B. A systematic review and meta-analysis of group peer support interventions for people experiencing mental health conditions. BMC Psychiatry. Jun 23, 2021;21(1):315.
White S, Foster R, Marks J, et al. The effectiveness of one-to-one peer support in mental health services: a systematic review and meta-analysis. BMC Psychiatry. Nov 11, 2020;20(1):534.
Tan SY, Chou CC. Supervision effects on self-efficacy, competency, and job involvement of school counsellors. J Psychol Couns Sch. Jun 2018;28(1):18-32.
Lohani G, Sharma P. Effect of clinical supervision on self-awareness and self-efficacy of psychotherapists and counselors: a systematic review. Psychol Serv. May 2023;20(2):291-299.
Delahaij R, Kamphuis W. Measuring the effects of wearable use on metacognitive awareness and self efficacy for behavioral change [Poster presentation]. Presented at: European Association of Occupational Health Psychology; Jun 5-7, 2024; Granada, Spain.
Eikenhout LMJ, Delahaij R, Van Dam K, Kamphuis W, Hulshof IL, Van Ruysseveldt J. Chronic stressors and burnout in Dutch police officers: two studies into the complex role of coping self-efficacy. Front Psychol. 2022;13:1054053.
Kamphuis W, Delahaij R, Preenen P. Validatie weerbaarheidsmonitor politie–basis politie zorg. Validation of the resilience monitor for the police–basic police teams [Report in Dutch]. Soesterberg, Netherlands: TNO; 2014. TNO 2014 M11035
Demerouti E, Bakker AB, Sonnentag S, Fullagar CJ. Work‐related flow and energy at work and at home: a study on the role of daily recovery. J Organ Behav. Feb 2012;33(2):276-295. URL: https://onlinelibrary.wiley.com/toc/10991379/33/2 [Accessed 2025-01-15]
Luijten CC, Kuppens S, van de Bongardt D, Nieboer AP. Evaluating the psychometric properties of the mental health continuum-short form (MHC-SF) in Dutch adolescents. Health Qual Life Outcomes. Oct 22, 2019;17(1):157.
Lee EH. Review of the psychometric evidence of the Perceived Stress Scale. Asian Nurs Res (Korean Soc Nurs Sci). Dec 2012;6(4):121-127.
RStudio Team. RStudio: integrated development environment for R. RStudio, BPC. 2023. URL: https://www.rstudio.com [Accessed 2025-01-13]
R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing. 2023. URL: https://www.R-project.org/ [Accessed 2025-01-13]
Enders CK, Tofighi D. Centering predictor variables in cross-sectional multilevel models: a new look at an old issue. Psychol Methods. Jun 2007;12(2):121-138.
Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. J Stat Softw. Oct 7, 2015;67(1):1-48.
Nakagawa S, Schielzeth H. A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods Ecol Evol. 2013;4(2):133-142.
Cohen J. Statistical Power Analysis for the Behavioral Sciences. Academic Press; 2013. ISBN: 1-4832-7648-1
Šimkovic M, Träuble B. Robustness of statistical methods when measure is affected by ceiling and/or floor effect. PLoS One. 2019;14(8):e0220889.
Thomas DR, Zumbo BD. Difference scores from the point of view of reliability and repeated-measures ANOVA: in defense of difference scores for data analysis. Educ Psychol Meas. 2012;72(1):37-43.
Colombo D, Fernández-Álvarez J, Patané A, et al. Current state and future directions of technology-based ecological momentary assessment and intervention for major depressive disorder: a systematic review. J Clin Med. Apr 5, 2019;8(4):465.
Kim J, Marcusson-Clavertz D, Yoshiuchi K, Smyth JM. Potential benefits of integrating ecological momentary assessment data into mHealth care systems. Biopsychosoc Med. 2019;13:19.
Seppälä J, De Vita I, Jämsä T, et al. Mobile phone and wearable sensor-based mHealth approaches for psychiatric disorders and symptoms: systematic review. JMIR Ment Health. Feb 20, 2019;6(2):e9819.
© 2025. This work is licensed under https://creativecommons.org/licenses/by/4.0/" target="_blank">https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Roos Delahaij
; Marianne van Zwieten
; Verhoef, Helen
; Kamphuis, Wim