Constant or inconstant? The time-varying effect of danmaku on user engagement in online video platforms

Abstract

Purpose

As an emerging video comment feature, danmaku is gaining more traction and increasing user interaction, thereby altering user engagement. However, existing research seldom explores how the effectiveness of danmaku on user engagement varies over time. To address this research gap, this study proposes a comprehensive framework drawing on social presence theory and information overload theory. The framework aims to explain how the effectiveness of danmaku in increasing user engagement changes over shorter time intervals.

Design/methodology/approach

A research model was proposed and empirically tested using data collected from 1,019 movies via Bilibili.com, one of China’s most popular danmaku video platforms. A time-varying effect model (TVEM) was used to examine the proposed research model.

Findings

The study finds that the volume of danmaku and its valence exert a time-varying influence on user engagement. Notably, the study shows that danmaku volume plays a more substantial role in determining user engagement than danmaku valence.

Originality/value

This research offers theoretical insights into the dynamic impact of danmaku on user engagement. The innovative conceptualization and measurement of user engagement advance research on pseudo-synchronous communication engagement. Furthermore, this study offers practical guidelines for effectively managing danmaku comments on online video platforms.

Full text

Translate

Turn on search term navigation

1. Introduction

Social media platforms are widely used for sharing product and brand information as well as for engaging in conversations with others through online videos (Munaro et al., 2021). According to eMarketer, US consumers watched an average of 149 min of digital video per day in 2021, reflecting a notable increase from 133 min in 2020 (Benes, 2021). In June 2021, the number of Chinese online video platform users reached an impressive 944 m (CNNIN, 2021). The remarkable growth of online videos has led to the emergence of a new form of social interaction called “danmaku” (Wu et al., 2019). As illustrated in Figure 1, Danmaku is a real-time commentary system that overlays users' remarks directly onto the video, progressing from left to right in real-time. Danmaku represents a form of real-time, dynamic interactive communication (Zhang and Cassany, 2020), in contrast to generic impressions and post-hoc remarks (Zhang et al., 2023). It creates a “pseudo-synchronic” experience, enhancing viewer engagement and strengthening viewership (Fang et al., 2018). It is widely popular on numerous online video platforms. For instance, Chinese online video company Bilibili reported a staggering 2.2bn danmaku messages in 2020 alone. Danmaku represents a novel form of word of mouth (WOM), characterized by user-generated comments (Fang et al., 2018). Unlike traditional WOM, danmaku messages are synchronized with specific moments in the video, creating a sense of “live” interaction for viewers (Zhang and Cassany, 2020). The engagement in online video, influenced by these distinctive interaction dynamics of danmaku, may diverge significantly from that of traditional comments.

User engagement represents a fundamental human response to computer-mediated activities (Bitrián et al., 2021). The long-term development of business activities on online video platforms hinges on user engagement. User engagement in social media encompasses both consumption and active participation (e.g. real-time interactions) (Khan, 2017). User engagement on digital media and platforms goes beyond the transactional aspects of buying and using products. It includes elements such as deriving enjoyment, discussing the object and sharing the product experience (Eigenraam et al., 2018). User engagement on digital media and platforms has been extensively studied, focusing on various activities such as reading, liking/disliking, sharing and commenting. These activities are crucial for understanding user interactions on social media platforms (Khan, 2017). Nevertheless, owing to context-specific factors, user engagement on online video platforms may exhibit variations distinct from those observed on traditional social media (Munaro et al., 2021). For instance, user engagement on online video platforms shares common elements with other contexts, including liking, sharing and comments. Additionally, it incorporates unique measures such as providing gifts, which have not been widely explored in previous studies.

Despite mounting evidence that viewers of online videos are showing an increased inclination to interact through danmaku, whether through active participation or passive observation (Zhang et al., 2020; Zhou et al., 2019), the precise impact of danmaku on user engagement in online video platforms and how its effectiveness may evolve over time remain unclear. However, understanding this new digital technology phenomenon is imperative for fostering business growth. Existing studies on the effects of danmaku on different forms of user engagement in online videos present a varied landscape. Some found no effects, as exemplified by Lin et al. (2021) focusing on viewer tips. Positive effects were highlighted in studies such as Zhou et al. (2019), which emphasized the effect on virtual gift sending, while others, like Lu et al. (2021), delved into negative effects, specifically studying total viewer tips. Additionally, a study by Zhang et al. (2017) revealed a dynamic effect of social interactions on goal attainment. Existing studies have notably overlooked the time-related evolution of interactions, limiting their examination to static considerations. Additionally, there is a notable absence of empirical studies quantifying the dynamic effects of these interactions.

The inconclusive nature of these findings motivates this study to investigate the relationship between danmaku and user engagement in greater depth. Two important approaches to understanding why relationships between existing studies are inconsistent are refining construct conceptualization tailored to the specific research context (Fairchild and MacKinnon, 2009) and extending beyond simplistic static influence between the predictor and the dependent variable by characterizing the temporal trend of the influence relationship (Tan et al., 2012).

To understand why the relationship is inconsistent across studies, it is essential to investigate how the dynamic nature of interaction potentially influences the results (Saboo et al., 2016; Tan et al., 2012). In behavioral science research, incorporating the time factor is crucial for addressing process-oriented questions, such as how a predictor is differentially associated with an outcome over time, and it contributes significantly to theory development (Lanza et al., 2016). Prior literature suggests that WOM has a time-varying influence on behavior changes. Considering danmaku as a new type of WOM scrolling across the screen in real-time (Fang et al., 2018), we incorporate the time factor to account for the evolving nature of the interaction and shed light on the relationship between danmaku and user engagement. This study quantifies the dynamic impact of danmaku on user engagement using WOM descriptors (volume and valence).

The dataset from Bilibili.com (Bilibili), a popular Chinese video website known for its danmaku, is analyzed using the time-varying effect model (TVEM) to achieve the research goal. Data on user engagement and danmaku for 1,019 movies were collected over a three-day period in March, April and May 2019. The results suggest that both danmaku volume and danmaku valence have a significant time-varying effect on user consumption and participation. Danmaku valence increases “liking” and “sharing” but may negatively impact other user engagement behaviors, such as collections, coins and reviews.

Our research advances this field in three ways. First, this study emphasized pseudo-synchronization over asynchronous and synchronous social media engagement (Giertz et al., 2022). We examined the conceptual foundations of engagement on online video platforms, which serve as pseudo-synchronous communication platforms for content consumption and participation. We analyzed user engagement via pseudo-synchronous communication. Second, we show how danmaku affects user engagement over time. We responded to the inconsistent nature of the existing studies by finding that the relationship between danmaku and user engagement constantly changes over time. Our finding of the time-varying effect of danmaku on user engagement contributes to the user engagement literature by confirming its utility. Finally, we investigated the dynamic effect of danmaku on user engagement using the TVEM approach. By modeling coefficients as continuous smooth functions of time, the TVEM approach is more flexible in exploring time-varying effects than traditional methods, which pre-set parameter shapes (linear, quadratic or exponential) (Tan et al., 2012). Thus, the TVEM approach has natural advantages in data-driven decision-making. Existing studies only estimate static effectiveness and do not consider dynamic effectiveness. This study demonstrates that TVEM is more effective at understanding temporal dynamics than traditional models.

The paper continues as follows: Section 2 reviews user engagement literature. Section 3 proposes hypotheses. Section 4 discusses empirical analysis methodology and data. Section 5 presents descriptive and empirical results. Section 6 concludes with this paper’s theoretical and managerial limitations and future directions.

2. Theoretical review

2.1 Previous research on danmaku and user engagement

During live broadcasts, viewers and video content providers exchange danmaku messages on the platform. Danmaku can influence broadcasters, platforms and customers. At the broadcaster level, danmaku can enhance viewer interaction and encourage gift sending (Zhou et al., 2019; Li and Guo, 2021). At the platform level, danmaku can help build communities and share knowledge more effectively than traditional video forum comments (Wu et al., 2019). Meanwhile, at the viewer level, danmaku has been shown to increase perceived social presence (Zhang et al., 2020). Additionally, Danmaku is linked to engagement at all levels (Lin et al., 2021).

Engagement studies, originating in sociology, psychology and organizational behavior, suggest that engagement is a multidimensional concept comprising behavioral, cognitive and emotional components (Hollebeek, 2011). According to Bar-Gill and Reichman (2020), “marketing research typically takes engagement as a given concept. In service systems, engagement is defined as a customer’s motivationally-driven, volitional investment of focal operant and operand resources into brand interactions” (Hollebeek et al., 2019). However, as demonstrated in theoretical discussions across various fields such as marketing (Delbaere et al., 2021; Lin et al., 2021), organizational behavior (Lambert et al., 2021) and education (Razmerita et al., 2020), the definition and operationalization of engagement are complex issues. Engagement has been studied in various academic disciplines (Bitrián et al., 2021). Customer engagement, brand engagement, live stream engagement, stakeholder engagement and others have existed in various contexts. This study examines user engagement in online video platforms, which encompasses more than just simple content viewing and reading (consumption). It also includes behavioral aspects such as click-based interactions (Khan, 2017). Consumption involves passively watching the video, reading the comments and checking the “likes”/“dislikes” but not responding. Compared to consumption, participation is considered an active engagement behavior (Shao, 2009).

Danmaku and various forms of engagement have been studied recently. Table 1 shows that live-streaming platform virtual gifting has garnered the most attention. User engagement is a more comprehensive behavior that goes beyond gifting. This paper examines consumption and participation to better understand the relationship between danmaku and user engagement on online video platforms.

2.2 Social presence theory

Social presence theory explains user engagement as a continuum (Wei et al., 2017). Social media behavior is a popular area of research, with social presence theory being one of the most frequently cited frameworks (Chen and Liao, 2022). Social presence refers to the degree to which a person is perceived as a “real person” in mediated communication (Gunawardena, 1995), even in online environments (Abdullah, 2004). This feeling of interacting with others (Sung and Mayer, 2012) and identifying with the online community (Kreijns et al., 2022) are all key aspects of social presence. In contexts with diverse definitions, social presence typically refers to the extent to which participants feel the “realness” of their interactions with others in a technology-mediated environment (Short et al., 1976).

In this study, social presence is considered a psychological connection. As noted by Hassanein and Head (2007), the sense of human warmth and sociability provided by the medium reflects users' feelings about using the communication medium, implying that social presence is a psychological perception. Building on this, social presence reflects a feeling of human connection, sociability and sensitivity fostered by the medium itself (Yoo and Alavi, 2001). Providing the means of interaction or stimulating the imagination of interactions with others can instill warmth and sociability (Hassanein and Head, 2007), enabling participants to experience social presence. A strong sense of social presence, which captures the feeling of psychological connection, is linked to a more satisfying user experience. Features like instant comments (e.g. danmaku comments) contribute to this by fostering a sense of connection with others.

In earlier research, Heeter (1992) discovered that users who perceive a heightened social presence through “entering another world” tend to derive more enjoyment from the experience. When participants experience social presence, they often become deeply engaged, absorbed and engrossed in the interaction, fostering feelings of affection, trust and warmth (Kreijns et al., 2007). Danmaku comments serve as an effective tool for real-time and dynamic interactive communication, allowing people to connect with each other instantly regardless of their location. According to Zhang and Cassany (2020), this form of communication supports a high level of sociability, warmth and a stronger sense of social presence among the participants. Several studies, including one by Fang et al. (2018), have confirmed that danmaku comments can increase viewers' perception of a video website’s social presence, leading to greater immersion. Mou et al. (2022) showed that danmaku comments changed how individuals viewed the videos, moving from a passive and isolated experience to a more active and social one. Chen and Feng (2023) consider that the danmaku comment can reflect social presence and use it as a proxy variable of social presence.

2.3 Information overload theory

The effect of information load on consumer decision-making is described as an inverted U-shape after several seminal studies (Jacoby et al., 1974). Individuals have limited abilities to assimilate information (Malhotra et al., 1982). Thus, information overload occurs when the information supply at any given time exceeds processing capacity (Eppler and Mengis, 2004). Information overload confuses people by making it harder to set priorities or recall prior information (Schick et al., 1990), resulting in dysfunctional performance (Lee and Lee, 2004).

The rich nature of the information available in the online environment can easily become an issue for information overload. Several studies have investigated how online information overload affects consumer choice. For example, Park and Lee (2008) found that electronic word-of-mouth (e-WOM) overload decreased high-involvement consumers' purchase intention. Hu and Krishen (2019) used information overload theory to understand how decision-support information (i.e. online reviews) affects satisfaction, while Zinko et al. (2020) showed excessive e-WOM information can impact satisfaction, trust and purchase intention.

In the case of danmaku, the limited processing capacity may lead to information overload when there is an extra amount of information. Viewers engage with online shows while simultaneously attending to danmaku content. This phenomenon can cause viewers to lose focus on the video or neglect to absorb and process the danmaku comments, resulting in excessive cognitive load (Mou et al., 2022).

While an adequate amount of information provides valuable knowledge for reviewers and reduces ambiguity in decision-making (Liu and Karahanna, 2017), an excessive amount may yield negative outcomes. Therefore, beyond the quantity of information, it is crucial to consider other factors such as context variables, prominent dimensions, timing and valence (Zhao and Pechmann, 2007).

3. Research framework and hypotheses development

According to the social presence and information overload theories, users’ engagement in online video websites depends on the warmth and sociability provided by danmaku (Wei et al., 2017; Fang et al., 2018) and the information contained therein. Drawing upon the social presence and information overload theories and considering the operational aspects of danmaku, Figure 2 encapsulates our research framework.

3.1 The time-varying effectiveness of danmaku volume on user engagement

This study first explores the relationship between danmaku volume and user consumption (Shao, 2009). This association is further elucidated by the social presence and information overload theories (Short et al., 1976; Malhotra et al., 1982). Danmaku is the result of online comments superimposed on the video content. Thus, viewers can comment and interact with each other (Zhang and Cassany, 2020). Computer-mediated communication studies (Zhou et al., 2019) have shown that danmaku affects video consumption, but little research has examined the time-varying relationship between danmaku volume and user consumption.

Several studies have supported this time-varying effectiveness. Danmaku facilitates video sharing among viewers with similar interests, fostering social interaction and a sense of human connection (Shen, 2012). The information conveyed through danmaku can influence user behavior (e.g. attitude, consumption and viewing frequency) positively or negatively (Li and Guo, 2021), depending on the specific content of the danmaku (Li et al., 2021). The projection of danmaku into the comment feed is enhanced by the video timeline (Liu et al., 2017). Specifically, the augmented display of danmaku on online video websites can lead to temporal variations in its effectiveness on user consumption.

Danmaku, which involves viewers' verbal and emoticon-based conversations about the object, creates a “pseudo-synchronic” co-viewing experience (Fang et al., 2018). It enhances viewer interaction and a sense of belonging (Xi et al., 2021). Viewers are more inclined to watch danmaku if it provides information, entertainment and a sense of community (Chen et al., 2017). While danmaku on online video platforms promotes consumption, such as increasing the number of viewers and viewing frequency, viewer reactions can vary – positively or negatively – based on the information conveyed by danmaku. This may manifest in changes to the number of views and shifts in viewer attitudes (Chen et al., 2017).

It is crucial to recognize that danmaku, whether increasing or decreasing, influences user consumption behavior. However, as the volume of danmaku information expands, its impact on user consumption undergoes changes. The key lies in maintaining a balance between social presence and information overload, particularly as danmaku volume increases. Thus, our hypothesis posits that

H1. The effect of danmaku volume on consumption varies over time.

This study also links danmaku volume and participation. Participation includes likes, dislikes, shares, comments, voting coins and more (Zhang et al., 2020). The literature shows that e-WOM volume drives user behavior. Gu et al. (2012) found that Amazon camera sales are strongly correlated with WOM volume, and Rosario et al. (2016) asserted that the underlying dynamic of e-WOM is the conformity effect. These studies inspire danmaku volume effectiveness research. Zhou et al. (2019) found that the number of words and emojis in danmaku comments is positively related to the number of gifts on the live-streaming platform, while Lin et al. (2021) found that live comments significantly affect viewer likes but not viewer tips. These studies demonstrate that danmaku communication fulfills users' needs for information, entertainment, social connections and self-expression, leading to a positive attitude toward online videos. Chen et al. (2017) discovered that viewers tend to avoid danmaku videos because of information overload. While normative marketing frameworks have connected danmaku volume to individual behaviors, empirical research has produced mixed results.

Danmaku shows viewers' comments that are relevant to the video’s playback (Chen et al., 2017). While the danmaku is interactive, viewers may not comment simultaneously (Wu et al., 2019). Instead, comments are synchronized with the video playback time, providing viewers with information relevant to that specific moment. Thus, viewers can see other viewers' danmaku comments while watching videos (Wu et al., 2019). User participation can fluctuate with danmaku volume as danmaku enhances viewer connection and sociability, thereby amplifying the perception of social presence. However, information overload in danmaku may discourage users from active participation (Li et al., 2021). The volume of danmaku could dynamically influence user engagement. We hypothesize as follows:

H2. The effect of danmaku volume on participation varies over time.

3.2 The time-varying effectiveness of danmaku valence on video consumption

The dynamic impact of danmaku valence on consumption is the third relationship examined in this study. Valence, which represents whether e-WOM opinions are positive or negative, is another important characteristic (Liu, 2006). It is also referred to as the “sentiment,” “polarity,” “rating” or “ favorability” of e-WOM. The content of what people say is more important than the volume of their words (Gopinath et al., 2014). Although e-WOM valence has been studied extensively, the results remain inconclusive. Negative e-WOM can adversely impact product sales (Chevalier and Mayzlin, 2006). However, some studies suggest that negative e-WOM does not always have a consistently detrimental impact on sales (Rosario et al., 2016). In addition, some studies suggest that valence does not affect message credibility (Cheung et al., 2009). People’s reliance on mass media and interpersonal communication changes over time due to shifting preferences (Bruce et al., 2012), which may explain discrepancies in research findings.

e-WOM studies support the time-varying effect of valence, wherein valence, also referred to as rating, influences product quality perception. In the advertising literature, this phenomenon is recognized as the persuasive effect (Duan et al., 2008). Danmaku offers users both new information and entertainment (Lin et al., 2021). In situations with limited information, the persuasive effect of valence is heightened, leading to increased recognition of positive evaluations. As danmaku accumulates beyond human processing capacity, valence loses its positive effect (Chewning and Harrell, 1990) and may even become negative. Thus, we hypothesize the following:

H3. The effect of danmaku valence on consumption varies over time.

This study investigates the time-varying impact of danmaku valence on participation. By facilitating social interaction within video content, danmaku comments influence viewers' behaviors, including trust and loyalty (Ding et al., 2021), particularly consumer-to-consumer interaction, which is crucial in network communities (Ding et al., 2021). In addition to social interaction, danmaku aids in video analysis (Wang et al., 2020). The emotional content expressed through danmaku serves as transcendental knowledge, contributing to the analysis of video content popularity (Wang et al., 2020).

Clearly, there is a need for additional research to investigate the dynamic relationship between danmaku valence and participation. Social presence theory and information overload theory support this time-varying effect, demonstrating that user interactions can enhance connections and perceptions of social presence (Fang et al., 2018). Overloading information reduces this positive effect (Chewning and Harrell, 1990) and may even decrease participation. Thus, we hypothesize as follows:

H4. The effect of danmaku valence on participation varies over time.

4. Data and methodology

4.1 Data

We collected granular data from Bilibili, one of China’s most popular danmaku video websites. Unlike some other online video platforms that allow almost all users to send danmaku, Bilibili has stringent rules and measures in place. In certain cases, users may need to pass tests to ensure the relatively high quality of danmaku on Bilibili. Consequently, viewers appreciate the danmaku on the platform. Our data collection focused on movies and users' viewing traces. User traces are aggregated at the movie level, such as the cumulative views of movie i at time t. After removing invalid samples with invalid data or missing values, the dataset consists of 1,019 movies collected every three days from March 6 to May 18, 2019.

For each movie, we recorded three primary information cues: the movie’s category and introduction, details of the danmaku and specific engagement metrics (see Figure 1) such as the number of viewers, likes, collections, shares, coins and comments. Finally, we obtained 25,475 basic information items for those 1,019 movies and 61,770,044 danmaku comments.

4.2 Variables

Danmaku volume – This was computed as the number of danmaku comments sent to a movie by viewers.

Danmaku valence – Danmaku comments were collected for each movie and subsequently analyzed using segmentation and sentiment analysis techniques. Initially, all danmaku received by the movies during the data collection phase were consolidated. The first column in the dataset represents the movie’s unique identifier associated with each danmaku, while the second column denotes the timestamp of the danmaku. Second, to ensure that each danmaku comment is valid, we performed data cleaning by removing meaningless items. Subsequently, we utilized the Rwordseg package in R software to merge and tokenize danmaku comments from 1,019 movies observed during our study period into individual words. Thirdly, we applied a stop word list to exclude non-informative words, retaining only meaningful words for further analysis. Fourthly, we integrated the Chinese National Knowledge Internet (CNKI) sentiment lexicon developed by Tsinghua University and Taiwan University to construct a comprehensive sentiment lexicon, enabling us to determine whether a word carries positive or negative connotations based on its presence. Finally, we annotated all words within each observation period accordingly using the sentiment lexicon approach. Subsequently, we calculated the absolute difference between positive and negative words for each movie.

User engagement – “Video engagement” (Khan, 2017), “customer engagement” (Kumar and Pansari, 2016), “customer engagement behavior” (van Doorn et al., 2010), “brand engagement” (Hollebeek, 2011) and “live streaming engagement” are all types of marketing engagement (Lin et al., 2021; Guo et al., 2021). Major marketing-based engagement research is summarized in Table 2. User engagement was measured following (Khan, 2017) and is conceptualized under two main categories: participation and consumption. We used the number of times that the video had been viewed to measure consumption. We counted video likes, collections, shares, coins and reviews as proxy variables for user participation. Table 3 depicts the measurement of these variables, and Figure 1 shows how these variables appear in the video.

4.3 Model specification

TVEM is a statistical modeling approach that explicitly addresses changes in the association between particular explanatory variables and relevant behavioral covariates over time in a flexible manner (Tan et al., 2012; Saboo et al., 2016). Unlike multilevel linear models and panel data regression models, which make strong linear parametric assumptions about changes in the relationship between two variables (Li et al., 2015), TVEM estimates parameters without pre-setting the shape and models coefficient functions nonparametrically (Saboo et al., 2016). Thus, it is well-suited for studying behavioral outcomes and covariates over time (Tan et al., 2012). Specifically, unlike models with constant parameter estimates, such as a simple market response model shown in Equation (1), TVEM allows the coefficients to vary smoothly as a function of other variables (Saboo et al., 2016), using Equation (2) to express the TVEM.(1) $y_{i j} = β_{0} + β_{1} x_{i j 1} + β_{2} x_{i j 2} + \dots + β_{p} x_{i j p} + ε_{i j}$ (2) $y_{i j} = β_{0} (t_{i j}) + β_{1} (t_{i j}) x_{i j 1} + β_{2} (t_{i j}) x_{i j 2} + \dots + β_{p} (t_{i j}) x_{i j p} + ε_{i j}$ where i is the number of subjects (i.e. movies), j is the number of repeated measures for subject i. y_ij and x_ijp represent the outcome variable and the pth input variable of subject i at time t_ij, respectively. ε_ij is the error term, assumed to be normally and independently distributed. The intercept function β₀(t_ij) represents the mean trajectory of y at time t_ij, while the slope function, β_p(t_ij), describes the time-varying relationship between x_ijp and y_ij.

The TVEM model has proven successful in marketing and behavioral research (Saboo et al., 2016; Lanza et al., 2016). Additionally, it is well-suited for examining user engagement and danmaku dynamics over time. To reduce endogeneity (Rutz and Watson, 2019), we utilized a longitudinal panel dataset containing numerous objects. This dataset offers detailed records of key variables every three days for each video, providing abundant and rich information at shorter time intervals.

Building on prior research that investigated danmaku and user engagement (Zhang et al., 2020), we adopted a log-log formulation. We applied logarithmic transformations to both the outcome variable and the input variables to account for the substantial variability observed in these variables. Expanding on Equation (2), we incorporated the covariates, enabling us to delineate the final model as follows:(3) $\ln ({c o n s u m p t i o n}_{i j}) = {β_{0} (t_{i j}) + β}_{1} (t_{i j}) \ln ({D a n m a k u V o l u m e}_{i j}) + β_{2} (t_{i j}) \ln ({D a n m a k u V a l e n c e}_{i j}) + \sum_{l = 1}^{n} β_{l + 2} (t_{i j}) \ln ({p a r t i c i p a t i o n}_{l i j}) + ε_{i j}$

(4) $\ln ({p a r t i c i p a t i o n}_{l i j}) = {β_{0} (t_{i j}) + β}_{1} (t_{i j}) {\ln (D a n m a k u V o l u m e}_{i j}) + β_{2} (t_{i j}) {\ln (D a n m a k u V a l e n c e}_{i j}) + β_{3} (t_{i j}) \ln ({c o n s u m p t i o n}_{i j}) + \sum_{l = 1}^{n - 1} β_{l + 3} (t_{i j}) \ln ({p a r t i c i p a t i o n}_{- l i j}) + ε_{i j}$ where l represents the lth proxy variable of participation, n represents the number of proxy variables and participation_-lij is a vector of other proxy variables.

5. Results

5.1 Descriptive analysis

Table 4 presents the summary statistics of the key variables for a balanced panel, covering 25 observation periods across 1,019 movies. Among these statistics, the average danmaku volume is found to be 6,660.470. This result indicates that danmaku is a common phenomenon in online videos. Additionally, the engagement behavior presented in Table 4 highlights differences in consumption and participation.

5.2 Model results

To understand the inconsistency in results among existing studies, we first constructed two benchmark models: the panel vector autoregressive model (PVAR) and the multilevel linear model (HLM). We then conducted a comparative analysis before proceeding to analyze the time-varying parameter estimates of our models.

5.2.1 Benchmark 1: baseline model without time-varying effect

In order to identify the dynamic relationship between danmaku, consumption and participation, this study utilized the PVAR model, initially developed by (Love and Zicchino, 2006).(5) $Y_{i, t} = Θ (L) Y_{i, t} + μ_{i} + χ_{t} + ε_{i, t}$ where Y_i,t is an eight-variable vector: {ln_Views, ln_Danmaku volume, ln_Danmaku valence, ln_Liking, ln_Collections, ln_Sharing, ln_Coins, ln_Reviews}. $Θ (L)$ is a matrix polynomial in the lag operator, $μ_{i}$ is assumed to be an individual effect, $χ_{t}$ denotes the time effect and $ε_{i, t}$ represents random errors that are assumed to be normally and independently distributed.

We used the Hahn Kuersteiner Estimator to estimate the model. The maximum likelihood estimator is inconsistent under the “T fixed n large” asymptotic approximation because of the well-known incidental parameter problem. The Hahn Kuersteiner Estimator is well-suited for estimating a dynamic panel AR(1) model with fixed effects when both n and T are large. The models were run in R using the panelvar [1] package, and the results are shown in Table 5. Danmaku volume (β = −0.001) and danmaku valence (β = −0.004) have no significant impact on user consumption. For user participation, danmaku volume significantly affects the amount of collection, sharing and coins. However, danmaku valence has a substantial and negative effect on coins. No evidence supported the significant impact of danmaku volume and danmaku valence on the number of “liking” and reviews.

5.2.2 Benchmark 2: monotonic time-varying parameter model

Hierarchical Linear Modeling (HLM) is an extension of linear regression, which describes the time-varying parameters by incorporating the time factor into the prediction factors. It can be written as follows:(6) $Y_{i j} = β_{00} + β_{01} t_{i j} + (β_{10} + β_{11} t_{i j}) X_{i j} + ε_{i j}$ where Y_ij is the independent variable for movie i measured at time t_ij, X_ij is the input variable for movie i measured at time t_ij, and t_ij is the measurement time of the jth observations for movie i.

The models were also run in R using the lme4 [2] package, and the results are shown in Figure 3. The danmaku volume and valence significantly influence user consumption and participation. Specifically, danmaku volume and danmaku valence have significant effects on views (β_volume = 0.4998, p = 0.0000; β_valence = −0.2361, p = 0.0000), “liking” (β_volume = −0.8463, p = 0.0000; β_valence = 0.5528, p = 0.0000), collections (β_volume = 0.4226, p = 0.0000; β_valence = −0.1387, p = 0.0000), sharing (β_volume = 0.0760, p = 0.0000; β_valence = 0.0700, p = 0.0000), coins (β_volume = 0.1306, p = 0.0000; β_valence = −0.1367, p = 0.0000) and reviews (β_volume = 0.3782, p = 0.0000; β_valence = −0.1538, p = 0.0000). However, except for the effect of danmaku valence on sharing (β_sharing = −0.0021, p = 0.0045) and coins (β_coin = 0.0014, p = 0.0965) varying with time, the others do not change with time. More concretely, there is no evidence to indicate any time-varying effects of danmaku volume and danmaku valence on views (β_volume = −0.0003, p = 0.7485; β_valence = 0.0001, p = 0.8786), “liking” (β_volume = −0.0018, p = 0.9199; β_valence = −0.0009, p = 0.6267), collections (β_volume = −0.0010, p = 0.6329; β_valence = 0.0028, p = 0.1942), sharing (β_volume = 0.0006, p = 0.4147), coins (β_volume = −0.0001, p = 0.8931) and reviews (β_volume = −0.0001, p = 0.1053; β_valence = 0.0002, p = 0.2344).

Moreover, the monotonic time-varying parameter model imposes a strong linear relationship assumption. To provide a more nuanced exploration of the temporal changes in danmaku, we constructed a TVEM for in-depth analysis.

5.2.3 Time-varying effect model

To estimate the time-varying effect, all models were run using TVEM. A P-spline was used for model estimation, as a P-spline is more flexible and has better computational efficiency (Tan et al., 2012). A cubic spline is often chosen for the order of a piecewise polynomial function due to its desirable properties. It ensures that the function has continuous first and second derivatives and allows for smooth transitions around the knot points. Therefore, we used a cubic spline to estimate the time-varying coefficient functions β₀(t_ij) and β_p(t_ij) from Equation (2).(7) $β (t) = α_{0} + α_{1} t + α_{2} t^{2} + α_{3} t^{3} + \sum_{k = 1}^{K} α_{3 + k} {(t - τ_{k})}_{+}^{3}$ where $τ_{k}$ represents the knots over the range t. As there is no standard guideline for choosing the number of knots (K), we followed previous studies and set K to 10 (Tan et al., 2012). The parameters were estimated using the SAS Macro developed by Li et al. (2015). The results are presented in Figure 4.

H1 proposes that the effectiveness of danmaku volume on user consumption varies over time. As depicted in Figure 4a, we observed a nonlinear pattern in the effect of danmaku volume on user consumption. Specifically, the effect increases at an accelerating rate until reaching a peak, after which it begins to decrease. Compared to the baseline model without the time-varying effect, the TVEM analysis indicates that the parameter estimates of danmaku on user consumption increase to a maximum of 0.507 (95% CI = [0.407, 0.608]) and then decrease to β = 0.491 (95% CI = [0.390, 0.592]) toward the end of the time horizon. Therefore, H1 is supported.

We observed that the effect of danmaku volume on the “liking” is negative and varies with time in a U-shape (Figure 4b). This finding is inconsistent with prior research that has studied the positive effect of e-WOM on consumer behavior (Liu, 2006). According to Chen et al. (2017), too much information in danmaku comments makes users feel crowded under cognitive constraints. In this study, it was found that danmaku exceeding 1,000 was prevalent in 50.5% of the samples analyzed. Importantly, this indicates a significant abundance of danmaku across the sampled data. Thus, this abundance of danmaku comments can create information overload, making it difficult for viewers to process the content and potentially reducing their enjoyment (liking). We divided the sample into two subgroups using a median of 1,017 to confirm our inference. Figure 5a shows the time-varying effect of danmaku volume on “liking” using subgroups. Danmaku volume decreases the negative effect on “liking.” Figure 4c shows that danmaku volume positively affects collections over time. The effect of danmaku volume on sharing decreases at first and then increases in a nonlinear pattern (Figure 4d). Danmaku volume and coins vary in effectiveness over time (Figure 4e). Finally, Figure 4f shows the estimated nonlinear association between danmaku and review volume over time. Therefore, H2 is supported.

Figure 4a shows that danmaku valence affects consumption over time. H3 is supported. Viewing behavior suffers. This contradicts previous research that found e-WOM valence boosts sales. Too much danmaku comment information likely contributes to this negative effect. Figure 5b shows the inference verified using subsample data. Danmaku valence reduces consumption due to information overload.

Danmaku valence affects user participation differently over time, supporting H4. Like the baseline time-invariant model, danmaku valence increases liking and sharing (Figures 4b and 4d). The negative impact of danmaku valence on collections, coins and reviews is evident in Figures 4c, 4e, 4 and 4f. Particularly noteworthy is its detrimental effect on review volume. Instant emotional communication reduces post-consumption review motivation. In addition, comparing the main y-axis and the secondary y-axis in Figure 4, it is not difficult to find that the impact of danmaku volume on user engagement is much greater than that of danmaku valence.

To further demonstrate the robustness of the research findings in this paper, we conducted additional data processing for a six-day period instead of a three-day period to investigate the time-varying effect of danmaku on user engagement. The outcomes from these rigorous tests provide supplementary evidence supporting our study’s conclusion. All results are presented in the Appendix.

6. Discussion

Danmaku interaction, whether through participation or observation, is becoming increasingly common among online video viewers (Zhang et al., 2020; Zhou et al., 2019). However, there has been little research examining the time-varying effectiveness of danmaku on user engagement. To build on this stream of research, we examined how danmaku volume and valence affect user engagement. Using the TVEM approach, our findings indicate that danmaku volume and valence influence user engagement (consumption and participation) differently over time.

6.1 Theoretical implications

First, this study contributes by introducing a conceptualization of user engagement in the online video context through danmaku (Munaro et al., 2021). Our work on the conceptualization and measurement of user engagement advances research on pseudo-synchronous communication engagement. Second, our findings provide theoretical insights into the dynamic effect of danmaku on user engagement. While the majority of research has focused on examining the impact of danmaku on specific aspects of user engagement from a static perspective, limited attention has been given to comprehensively understanding its dynamic effectiveness in enhancing user engagement. Building on this research stream, our study innovatively linked user engagement to changes in danmaku, revealing that danmaku volume and valence affect user consumption and participation over time. These findings help explain the inconsistencies in previous studies regarding the relationship between danmaku and user engagement. Third, since danmaku is a new form of e-WOM, our findings also expand e-WOM research. We discovered that danmaku volume has a larger impact on user engagement than valence. Fourth, the dynamic optimization of resource configuration in response to changes in the external environment is of paramount importance (Saboo et al., 2016). We utilize TVEMs to examine and analyze the temporal effects of danmaku on user engagement. TVEM facilitates data analytics by offering valuable insights for data-driven decision-making without assuming predefined variable forms or parameters. This study demonstrates that TVEM serves as an effective methodological approach for understanding temporal variations in the relationships among outcome variables, enabling managers to make real-time strategy adjustments.

6.2 Managerial implications

Danmaku clearly influences platform engagement, such as in online video and live-streaming. The co-viewing experience is valued by viewers; without the ability to communicate “in time,” they may feel isolated and disengaged. Online video providers may benefit from incorporating the danmaku function more extensively. Additionally, our findings suggest that danmaku volume has a greater impact on user engagement than valence. Therefore, it would be beneficial to highlight the number of danmaku messages exchanged on video information display pages.

Furthermore, danmaku’s dynamic nature can both engage and overwhelm users. Our study highlights that its effect on user engagement is not always positive. Therefore, online video platform developers may consider implementing danmaku filtering or display mechanisms to effectively manage information overload while ensuring the timely fulfillment of social needs.

Finally, our results can assist managers in designing danmaku exposure strategies. Danmaku valence dynamically and positively affects viewer sharing and liking, while danmaku volume negatively affects liking during periods of information overload. Notably, in the early stages of a video launch, the impact of danmaku valence on liking and sharing gradually increases, but this effect progressively decreases over time. Therefore, it is imperative for engineers of online video platforms to initially allocate substantial resources towards the management of danmaku text, gradually reducing them in subsequent stages. This approach ensures optimal resource allocation while maximizing user experience enhancement and promoting sharing behavior. For example, managers could highlight positive words in danmaku soon after the video is launched to increase viewer sharing and liking while preventing information overload. Our research, in line with the work of Saboo et al. (2016), aids firms in exploring customer-focused resource allocation activities to optimize the efficiency of resource utilization.

6.3 Future research and limitations

This study examines the impact of danmaku volume and valence on user engagement over time within an online video platform while also highlighting the potential efficacy of danmaku in diverse contexts such as live streaming (Lu et al., 2021; Li et al., 2021). On live-streaming platforms, managers need to enhance engagement to build relationships (Hu and Chaudhry, 2020). Danmaku enables interaction between broadcasters and viewers (Zhou et al., 2019). Therefore, the generalizability of the model developed in this study could be tested in other contexts.

In this study, we evaluated danmaku’s efficacy irrespective of movie genre. Further examination could focus on genre-specific movies, such as romance, horror or others. Additionally, the generalization and boundary conditions of our results require further investigation.

Our findings are based on analyzing secondary data from Bilibili. While it is unlikely that viewers disable danmaku while watching videos on this platform, we cannot completely rule out this possibility. Employing alternative data collection methods, such as experimental research, may help mitigate this issue. Therefore, it would be worthwhile to investigate the effectiveness of danmaku in user engagement using samples that do not involve the behavior of closing danmaku. This approach would help overcome the limitations associated with secondary data in future studies.

Notes

1.The PVAR model can be estimated using the panelvar package in R. The panelvar package is available on CRAN: https://cran.r-project.org/web/packages/panelvar/.

2.The HLM model can be estimated in with me4 package in R. The me4 package is available on CRAN: https://cran.r-project.org/web/packages/lme4/.

Figure 2

The research model

[Figure omitted. See PDF]

Figure 3

Parameter estimates of monotonic time-varying parameter model

[Figure omitted. See PDF]

Figure 4

Parameter estimates of time-varying effect model

[Figure omitted. See PDF]

Figure 5

Time-varying effect in subsamples

[Figure omitted. See PDF]

Figure A1

Parameter estimates of time-varying effect model (a six-day period)

[Figure omitted. See PDF]

Table 1

Relevant studies investigating the relationship between different forms of engagement and danmaku

Reference	Independent variable	Dependent variables	Research design	Key finding
Fang et al. (2018)	Danmaku	Social presence	Survey	The presence of danmaku increases the level of perceived social presence
Wu et al. (2019)	Danmaku	User participation, social	Quantitative and qualitative methods	Danmaku can promote social presence and explicit knowledge sharing
Zhou et al. (2019)	The number of words in danmaku, the level of debate, the similarity of danmaku, the number of excitement-related words in danmaku, and the number of emojis in danmaku	The number of gifts given by viewers	Longitudinal design with data crawled from DOUYU	All independent variables have a positive influence on the number of gifts
Zhang et al. (2020)	MTMS (stands for moment-to-moment synchronicity, which means the synchronicity between temporal changes in the consumption content and the immediate consumer response to those changes)	Rating, coins	Longitudinal design	MTMS has a positive effect on rating and coins
Dou and Ge (2021)	Real-time comments, city-of-origin	Gifting income	Longitudinal design with data crawled from KUAI SHOU	The real-time comments moderate the effect of city-of-origin on gifting income
Li et al. (2021)	Class identity, relational identity	The number of paid and free gifts	Longitudinal design with data crawled from DOUYU	The social density measured by the average number of danmaku moderates the effect of class identity on paid and free gift sending
Li and Guo (2021)	Attention paid to danmaku content	Virtual gifting, sending danmaku	Survey	Danmaku viewing has a significant positive impact on the frequency and the amount of virtual gifting
Lin et al. (2021)	Viewer emotion in danmaku, length of danmaku, viewers emotion, broadcaster emotion, the number of viewers, viewer tips, number of likes	Viewer emotion in danmaku, length of danmaku, viewers emotion, broadcaster emotion, the number of viewers, viewer tips, number of likes	Longitudinal design with data crawled from a popular live-streaming platform in China	Viewer emotion and the length of danmaku significantly positively affect the number of likes but not the number of tips
Lu et al. (2021)	The average number of chats, the number of synthetic viewers	Total tips, the average number of tips, the average number of likes	Field experiment	Chat rate is negatively related to tip rate and positively associated with the average number of likes
Li et al. (2023)	Danmaku	View, Like	Longitudinal design with data crawled from Bilibili.com	The positive effect of danmaku comments was stronger on the number of views than on the number of “likes.”