Engagement analysis of a persuasive-design-optimized eHealth intervention through machine learning

Abstract

The challenge of sustaining user engagement in eHealth interventions is a pressing issue with significant implications for the effectiveness of these digital health tools. This study investigates user engagement in a cognitive-behavioral therapy-based eHealth intervention for procrastination, using a dataset from a randomized controlled trial of 233 university students. Various machine learning models, including Decision Tree, Gradient Boosting, Logistic Regression, Random Forest, and Support Vector Machines, were employed to predict patterns of user engagement. The study adopted a two-phase analytical approach. In the first phase, all features of the dataset were included, revealing ‘total_minutes’—the total time participants spent on the intervention and the eHealth platform—as the most significant predictor of engagement. This finding emphasizes the intuitive notion that early time spent on the platform and the intervention is a strong indicator of later user engagement. However, to gain a deeper understanding of engagement beyond this predominant metric, the second phase of the analysis excluded ‘total_minutes’. This approach allowed for the exploration of the roles and interdependencies of other engagement indicators, such as ‘number_intervention_answersheets’—the number of completed lessons, ‘logins_first_4_weeks’—login frequency, and ‘number_diary_answersheets’—the number of completed diaries. The results from this phase highlighted the multifaceted nature of engagement, showing that while ‘total_minutes’ is strongly correlated with engagement, indicating that more engaged participants tend to spend more time on the intervention, the comprehensive engagement profile also depends on additional aspects like lesson completions and frequency of platform interactions.