Content area
Effective oral communication is a crucial skill in academic and professional contexts. However, practising and refining these skills is challenging without structured guidance and feedback. This paper presents a user evaluation of WEBPOSE, a web-based Oral Presentation Automated Feedback (OPAF) system that provides immediate feedback on posture to improve non-verbal communication skills. In this study, WEBPOSE was tested with sixteen researchers specialising in educational technology. Using a mixed-methods approach, which included a Technology Acceptance Model (TAM) questionnaire, open-ended questions, and observational data, we investigated the perceived user experience, usability, and usefulness of the system. The results indicate that WEBPOSE was generally perceived as user-friendly and beneficial for fostering self-awareness around body language and presentation timing. Moreover, user feedback also highlighted nonfunctional and functional points of improvement for WEBPOSE, such as improving the visualisations of the system status, and the display of the immediate feedback. This paper concludes with design implications for improving user guidance, feedback mechanisms, and the integration of structured rehearsal stages. These insights aim to inform the future development of scalable, user-centred OPAF systems that can effectively support the development of presentation skills.
Abstract: Effective oral communication is a crucial skill in academic and professional contexts. However, practising and refining these skills is challenging without structured guidance and feedback. This paper presents a user evaluation of WEBPOSE, a web-based Oral Presentation Automated Feedback (OPAF) system that provides immediate feedback on posture to improve non-verbal communication skills. In this study, WEBPOSE was tested with sixteen researchers specialising in educational technology. Using a mixed-methods approach, which included a Technology Acceptance Model (TAM) questionnaire, open-ended questions, and observational data, we investigated the perceived user experience, usability, and usefulness of the system. The results indicate that WEBPOSE was generally perceived as user-friendly and beneficial for fostering self-awareness around body language and presentation timing. Moreover, user feedback also highlighted nonfunctional and functional points of improvement for WEBPOSE, such as improving the visualisations of the system status, and the display of the immediate feedback. This paper concludes with design implications for improving user guidance, feedback mechanisms, and the integration of structured rehearsal stages. These insights aim to inform the future development of scalable, user-centred OPAF systems that can effectively support the development of presentation skills.
Keywords: Oral Presentation Automated Feedback System, public speaking, Technology Acceptance Model, automated feedback, WEBPOSE, Educational Technologies, user acceptance.
1. Introduction
Effective communication is a cornerstone of success in both personal and professional life. Among its many facets, public speaking stands out as a particularly vital skill in the 21st century, playing a crucial role in situations such as job interviews (Patil and Katre, 2024), negotiations (Svendsen, 2022), and collaborative work (Chan, 2011). However, mastering public speaking is no easy task. It requires deliberate practice to master communication aspects such as body language, tone modulation, and speech clarity (Kerby and Romine, 2009).
Several methods are available to support the development of these skills. Practicing in front of a mirror is a simple but limited strategy due to its lack of feedback (Levasseur, Dean and Pfaff, 2004). Video recordings offer more value by allowing for self-reflection and targeted improvements (Zimmerman and Schunk, 1989); however, watching the recorded videos is time-consuming, and novice presenters have trouble identifying points for improvement (Korthagen et al., 2001). Additionally, one-on-one coaching with feedback from instructors remains highly effective due to its personalized guidance (Heinicke et al., 2022).
Recent technological advances have led to the development of Oral Presentation Automated Feedback Systems (OPAFs) (Ochoa, 2022). These software tools utilise sensors such as cameras and microphones to automatically evaluate presentation performance and support the development of communication skills (Kurihara et al., 2007). While promising, OPAFs have not yet achieved widespread adoption (Ochoa and Zhao, 2024). Many remain at the prototype stage and are typically evaluated only in laboratory settings due to the use of specialized hardware like depth cameras, which are not accessible to the general public. In a prior study (Hummel et al., 2025), we found that 13 out of 14 systems were tested exclusively under such conditions, leaving open questions about their real-world applicability.
To address this gap, we developed WEBPOSE (Web-based Evaluation of Body Posture for Oral Skill Enhancement), a system that focuses on improving non-verbal communication through immediate posture feedback. The system was informed by the expertise of public speaking instructors and grounded in relevant literature.
In this study, we conducted user tests to evaluate WEBPOSE based on perceived user experience, ease of use, and usefulness, with the aim of having a formative evaluation that will allow us to understand the current maturity status of WEBPOSE and how to improve it in future iterations.
To guide this evaluation, we address the following research questions:
RQ-MAIN: What is the perceived general user acceptance of WEBPOSE?
RQ1: What is the perceived usability of WEBPOSE?
RQ2: What is the perceived usefulness of WEBPOSE?
RQ3: How can we improve WEBPOSE based on the test results?
2. WEBPOSE: A Web-based OPAF System
WEBPOSE is a web-based application designed to support the development of non-verbal communication skills for presentations. The purpose of making it a web-based application is that it can run on regular PCs, laptops, tablets, and smartphones without the use of any specialized hardware and thus minimizing its accessibility barriers.
Experts' input inspired WEBPOSE by following a human-centered design approach to provide immediate feedback on improper postures, such as crossed arms (as shown in Figure 1). The goal of WEBPOSE is to enhance non-verbal communication skills and become a more effective presenter. WEBPOSE offers the ability to run either globally (hosted on a server) or locally (on a client machine). WEBPOSE was designed to minimize accessibility barriers by utilizing standard sensors over specialised equipment, which are now built-in even in low-budget devices like budget notebooks or Smartphones, combined with a simple User Interface (see Figure 1).
2.1 Pedagogical Approach and Technical Implementation
WEBPOSE was developed based on insights from a preliminary study involving interviews with 13 university teachers specializing in presentation skills (Blinded for review. What is most important when giving a presentation? Authenticity! - Insights on how presentation teachers imagine a software to help with presentation preparation and presentation skills training).
Our interviews with university teachers revealed that improving non-verbal communication skills is crucial for enhancing overall presentation performance [Blinded for review. What is most important when giving a presentation? Authenticity! - Insights on how presentation teachers imagine a software to help with presentation preparation and presentation skills training].
As mentioned by Brooks and Platz (1968), rehearsing presentations is a widely recognized method for enhancing public speaking performance. It involves repeatedly practicing a talk in a realistic setting to become more familiar with the content, structure, timing, and delivery. Through rehearsal, speakers can refine their verbal communication, improve clarity and confidence, reduce filler words, and become more aware of their body language and vocal tone. Practicing also allows presenters to identify weak points, adapt their message to the audience, and reduce performance anxiety. WEBPOSE supports this process by enabling users to record their pitch using a webcam and microphone while receiving feedback about their posture (see Figure 1). By reviewing these recordings, users can self-assess and observe how their delivery comes across, gaining insights into aspects such as eye contact, pacing, and gesture use. This reflective loop encourages iterative improvement and builds stronger presentation habits over time. The goal is to help users become more confident, engaging, and effective communicators by offering a practical and feedback-driven way to rehearse and improve their presentation skills.
In addition to non-verbal skills, it is important to manage time effectively. For many types of presentations, such as thesis defences or conference talks, it is essential to convey your message within the specified time frame. Significantly exceeding the allotted time is usually penalised in these contexts and should therefore be avoided. To support this, our software features a stopwatch to provide users with real-time feedback on elapsed time (see Figure 1 - 1 minute and 2 seconds have already passed). This stopwatch is triggered when the user presses the "Start Recording" button, which also signals the start of the presentation. The goal is to help presenters optimise their presentations to fit within the given time limits.
2.2 Technical Features of WEBPOSE
To provide immediate feedback on specific body postures, WEBPOSE utilizes a webcam to capture the presenter and extract visual data, which is then fed into a pre-trained machine learning model (specifically, the pose estimation model from the Google MediaPipe framework is used by WEBPOSE). Google decided to call this model "BlazePose", which is capable of detecting a human pose in a video stream and outputting the x, y, z coordinates of 33 joints related to the human body (Bazarevsky et al., 2020). These coordinates can then be used to identify certain patterns.
Designed with flexibility in mind, WEBPOSE supports both server-based and fully client-side operation, accommodating different educational contexts and privacy requirements. For institutions or instructors who prefer centralised access and management, the system can be hosted on a server, following a model similar to that of well-established platforms such as Moodle. Alternatively, for scenarios where data privacy, offline use or unreliable internet connectivity are important considerations, WEBPOSE can run entirely on the client side. In this configuration, all necessary files (HTML, CSS and JavaScript) are executed locally on the user's device, eliminating the need for an internet connection and ensuring that no personal data leaves the user's computer. We used this client-side mode during our study. The interface is intentionally minimalistic and features four core functions (see Figure 1): starting and stopping a video record + starting and resetting a stopwatch (both events are triggered simultaneously), downloading cached pose data (X, Y and Z coordinates in JSON format) and toggling the live feedback display on and off.
The underlying data pipeline of the software (see Figure 2), comprising sensing, analysis, and feedback, was designed to be straightforward to ensure intuitive operation and avoid overwhelming users. It provides feedback on posture-related patterns that according to literature (Blinded for review. What is most important when giving a presentation? Authenticity! - Insights on how presentation teachers imagine a software to help with presentation preparation and presentation skills training) (Schneider et al., 2017), negatively influence the performance of a presentation, such as crossing arms, crossing legs, looking away from the camera, covering face, hands inside pockets, and hands behind the back. To implement these postures, WEBPOSE uses hard-coded thresholds and heuristic conditions based on joint positions provided by the BlazePose pose estimation model. For example, to detect crossed arms, the system compares the relative x- and y-coordinates of wrists, shoulders, and hips to determine whether the arms are positioned across the torso. If the conditions are met, the system immediately displays visual feedback prompting the user to adjust their posture.
3. Method
The purpose of this study is to evaluate how users use and perceive WEBPOSE.
3.1 Participants
The study involved 16 participants who worked as researchers in educational technologies or similar fields. The participants ranged in age from 26 to 54 years, with self-reported gender distribution of 3 male and 13 female. All participants signed an informed consent to participate in the study.
3.2 Procedure
The study took place within the context of a 90-minute workshop on presentation competencies at a summer school for researchers in educational technologies. The procedure started with a general introduction (15 minutes) about the concept of the project in which the study was situated, and the study procedure. After the introduction participants filled in a pre-test questionnaire (see section 3.3). Participants were then asked to prepare a 3 to 5-minute-long elevator pitch following some guided instructions. Once they finished the preparation of their pitch, participants had the chance to practice their pitch using WEBPOSE. After practicing the pitch, participants had the opportunity to review their practice sessions and answer self-reflection questions. Finally, participants answered a post-test questionnaire (see section 3.3).
3.3 Apparatus and Materials
Set-up: The study took place in the main hall of the event venue. Each of the four corners of the room was equipped with a table, a laptop running WEBPOSE, and a wireless microphone paired to the corresponding Laptop.
To collect data, we used a pre-test questionnaire, a post-test questionnaire, logged data, and notes taken by the experimenter. The pre-test questionnaire included demographic questions such as age and gender. The posttest questionnaire included a TAM survey and six open-ended questions concerning the usability and user experience of WEBPOSE (see Table 1).
4. Results
To assess the participants' acceptance of WBPOSE, we asked them to answer the Technology Acceptance Model (TAM) questionnaire. Based on these answers we used descriptive statistics to extract the Perceived Usefulness (PU) and Perceived Ease of Use (PEOU) of WEBPOSE.
The mean score for Perceived Usefulness was 4.99 (SD = 1.25), and the mean score for Perceived Ease of Use was 5.03 (SD = 1.14), both on a 7-point Likert scale. Internal consistency was evaluated using Cronbach's Alpha. The PU scale demonstrated excellent reliability with an alpha of 0.959, and the PEOU scale also showed strong internal consistency with an alpha of 0.906.
Figure 4 illustrates the distribution of Perceived Usefulness ratings, which ranged from 2 to 7 and tended to cluster in the upper half of the scale. This indicates that participants generally perceived WEBPOSE as beneficial, though responses varied. Figure 5 shows the distribution of Perceived Ease of Use ratings, also ranging from 2 to 7, with a similar tendency towards higher values. This suggests that participants considered the system easy to navigate and user-friendly.
To examine how participants used WEBPOSE, we looked at the log data obtained from their interactions with the software. On average, participants took 140 seconds (SD = 66 seconds, median = 136 seconds) to complete their pitch. This corresponds to an average overrun of 20 seconds beyond the intended 2-minute timeframe. The shortest pitch lasted 45 seconds, while the longest extended to 283 seconds.
We asked the following open ended questions on the post-test questionnaire to gain qualitative insights about WEBPOSE:
* Q1: What are three things you would improve about WEBPOSE?
* Q2: What are the three things you liked most about WEBPOSE?
* Q3: What do you think about the system's immediate feedback? How would you make it more beneficial in terms of content and visualization?
* Q4: What do you think about the self-reflection tasks? How would you make it more beneficial in terms of content and visualization
* Q5: What, if anything, did you learn from using this software?
* Q6: Any extra comments you would like to add?
By examining the answers to the first two open-ended questions concerning future improvements and what participants liked about WEBPOSE, we identified three categories of answers: guidance, usability, and look & feel.
Guidance: WEBPOSE seems to be a system whose navigation is easy and self-explanatory. One participant commented that the buttons required more explanation. Another one commented that WEBPOSE would benefit from a general explanation of its purpose and the tasks that the user should follow. The remainder of the participants shared positive impressions about the navigation and guidance of WEBPOSE.
Usability: Participants generally mentioned that WEBPOSE was easy to use. Four participants explicitly mentioned to like the low barrier that the software can be used immediately with no additional hardware/sensors. One usability complaint that we got was related to the fact that participants needed to start the recording, then step back a few meters to start rehearsing the pitch, and then approach the laptop again to finish the recording. Three users pointed out the need for a more visible indication to know when the recording started.
Look and feel: Four participants mentioned that they liked the colors and our parrot avatar presented by WEBPOSE. Moreover, they appreciated the simple design and the fact that everything ran flawlessly.
Feedback (Q3): Regarding the software's feedback aspect, two participants explicitly mentioned that they liked the aspect of getting real-time feedback on their posture. Two other participants explicitly mentioned their positive experience with the timer feature. Five participants stated that if they got feedback, they did not notice it. Suggestions on how to improve the feedback included: increasing the size of the feedback messages (3 participants) and adding an animation to the feedback messages (4 participants).
Concerning the self-reflection tasks (Q4): Four participants appreciated the ability to re-watch the recorded videos and evaluate themselves using the provided self-reflection questions. Three participants mentioned that they find our chosen self-reflection questions beneficial. In terms of improvements to the self-reflection of the system, three participants mentioned that they would want to combine the self-reflection questions with other feedback aspects, such as expert feedback and automated software feedback. Two participants mentioned they prefer a Likert scale option instead of open self-reflection questions.
In terms of perceived learning (Q5), participants reported that using the software helped them become more aware of their body language, such as closed posture, nervous appearance, insufficient eye contact, or gestures. Three participants noted the value of seeing their recordings to identify areas for improvement and acknowledged the importance of rehearsing their pitches.
As additional comments (Q6), four participants encouraged the team to continue developing WEBPOSE further. When examining the notes taken during the study, we identified that three participants vocally expressed discomfort and a lack of confidence in presenting to a laptop. Three participants expressed their dislike of watching their recording. Finally, six participants required help from the experimenter in order to fulfil their tasks.
5. Discussion
This section discusses the key findings in relation to the research questions, incorporating participant feedback, observed challenges, and contextual limitations.
Our Main RQ deals with the general user acceptance of WEBPOSE, and to answer it, we broke it down into the general usability (RQ1) and the perceived usefulness (RQ2) of WEBPOSE. Based on the collected quantitative data (TAM), WEBPOSE is relatively easy to use. Our qualitative results showed that in terms of usability, the navigation and general visual design are aspects that positively influence the user experience of WEBPOSE. Moreover, the low entry barrier, meaning that WEBPOSE can be used without the need for specific hardware, is a positive feature affecting how the general public can experience the use of WEBPOSE.
Concerning the perceived usefulness of WEBPOSE, the collected quantitative data (TAM) show that WEBPOSE is generally perceived as a useful tool. Particularly, due to its real-time feedback and self-reflection features, which can support the improvement of non-verbal communication habits when giving a pitch or presentation. So overall, we can state that the user experience of WEBPOSE is acceptable (Main RQ).
The study also showed us how WEBPOSE can be improved (RQ3). In terms of usability, the recording process can be improved, first, by highlighting when WEBPOSE is recording and assessing the rehearsal. Second, by adding a different interaction mechanism that allows users to start and stop recordings once they are at the correct distance to perform their rehearsals. Examples of these mechanisms can be countdown timers or the use of predefined voice commands, and/or gestures. In terms of usefulness, the biggest point of improvement relies on a careful implementation of the immediate feedback, so that it becomes highlighted enough to be perceived without becoming overly distracting, as pointed out by (Schneider et al., 2015)
While the findings offer valuable insight into the user experience of WEBPOSE, several limitations should be noted. Notably, most of the participants were fellow researchers from the field of educational technology. Their familiarity with educational technology prototypes may have influenced their responses, as reflected in some encouraging or congratulatory remarks, which may have stemmed from a sense of collegial support. Their background may also have impacted their ability to engage with the system critically, compared to lay users or students.
Another limitation was the constrained timeframe of the workshop. Participants were only able to go through one rehearsal cycle, which restricted the ability to measure iterative improvements. Although many users expressed interest in rehearsing multiple times, time constraints prevented a second round. Nonetheless, based on our results we argue that rehearsing a speech with the use of WEBPOSE is perceived to be valuable. To sum up, this study presented a first formative evaluation of WEBPOSE, displaying its potential as an easily accessible OPAF that can support the development of public speaking skills, which are vital 21st-century skills and a cornerstone of success in both personal and professional life.
6. Conclusion
This paper presented a formative evaluation of WEBPOSE, a web-based system designed to provide posturerelated feedback during oral presentation rehearsals. Based on a study conducted with sixteen researchers in educational technology, the findings demonstrate that WEBPOSE is generally well-received, offering a userfriendly and accessible way to support the development of non-verbal communication skills.
As one of the first OPAF systems tested in a real-world setting without specialized hardware, WEBPOSE contributes to closing the gap between lab-based prototypes and practical, scalable tools. Its low setup requirements, ease of use, and integration of self-reflection capabilities make it a promising candidate for broader application in educational and professional training contexts.
At the same time, the evaluation revealed several opportunities for improvement. Enhancing the visibility and clarity of feedback, refining the recording interaction, and offering clearer user guidance emerged as key areas to address in future versions. Furthermore, combining real-time feedback with expert-driven insights or structured reflection stages may further increase its educational impact.
Future research should explore the longitudinal use of WEBPOSE across multiple rehearsal cycles and with more diverse user groups, including novice presenters and students. Investigating the impact of such systems on actual learning gains and presentation performance over time remains an important next step.
In summary, this study highlights the potential of accessible, web-based feedback systems such as WEBPOSE in supporting the development of oral presentation skills, which are becoming increasingly important in modern education and professional life.
Ethics Declaration
This study was conducted in accordance with the ethical standards of research involving human participants. All participants were informed about the purpose and procedure of the study and provided written informed consent prior to participation.
AI Declaration
Portions of this paper were prepared with the assistance of the AI language model ChatGPT (OpenAI, GPT-5). The tool was used to improve the clarity and conciseness of the language, to refine sentence structure, and to suggest alternative phrasings. All AI-generated suggestions were critically reviewed and edited by the authors, and the final content reflects the authors' own analysis and conclusions.
References
Bazarevsky, V., Grishchenko, I., Raveendran, K., Zhu, T., Zhang, F. and Grundmann, M., 2020. BlazePose: On-device Realtime Body Pose tracking. https://doi.org/10.48550/arXiv.2006.10204.
Brooks, W.D. and Platz, S.M., 1968. The effects of speech training upon self-concept as a communicator. The Speech Teacher, 17(1), pp.44-49. https://doi.org/10.1080/03634526809377643.
Chan, V., 2011. Teaching oral communication in undergraduate science: Are we doing enough and doing it right? Journal of Learning Design, 4(3), pp.71-79. https://doi.org/10.5204/jld.v4i3.82.
Heinicke, M.R., Juanico, J.F., Valentino, A.L. and Sellers, T.P., 2022. Improving Behavior Analysts' Public Speaking: Recommendations From Expert Interviews. Behavior Analysis in Practice, 15(1), pp.203-218. https://doi.org/10.1007/s40617-020-00538-4.
Hummel, S., Schneider, J., Mouhammad, N., Klemke, R. and Di Mitri, D., 2025. Enhancing presentation skills: key technical features of automated feedback systems - a systematic feature analysis. International Journal of Technology Enhanced Learning, 17(6), pp. 1-25. doi:10.1504/IJTEL.2025.10073166.
Kerby, D. and Romine, J., 2009. Develop Oral Presentation Skills Through Accounting Curriculum Design and Course- Embedded Assessment. Journal of Education for Business, 85(3), pp.172-179. https://doi.org/10.1080/08832320903252389.
Korthagen, F.A.J., Kessels, J., Koster, B., Lagerwerf, B. and Wubbels, T., 2001. Linking Practice and Theory: The Pedagogy of Realistic Teacher Education. 0 edn. [online] Routledge. https://doi.org/10.4324/9781410600523.
Kurihara, K., Goto, M., Ogata, J., Matsusaka, Y. and Igarashi, T., 2007. Presentation sensei: a presentation training system using speech and image processing. In: Proceedings of the 9th international conference on Multimodal interfaces. [online] ICMI07: International Conference on Multimodal Interface. Nagoya Aichi Japan: ACM. pp.358-365. https://doi.org/10.1145/1322192.1322256.
Levasseur, D.G., Dean, K.W. and Pfaff, J., 2004. Speech pedagogy beyond the basics: a study of instructional methods in the advanced public speaking course. Communication Education, 53(3), pp.234-252. https://doi.org/10.1080/0363452042000265170.
Ochoa, X., 2022. Multimodal Systems for Automated Oral Presentation Feedback: A Comparative Analysis. In: M. Giannakos, D. Spikol, D. Di Mitri, K. Sharma and R. Hammad, eds. The Multimodal Learning Analytics Handbook. [online] Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-08076-0.
Ochoa, X. and Zhao, H., 2024. OpenOPAF: An Open Source Multimodal System for Automated Feedback for Oral Presentations. Journal of Learning Analytics, pp.1-25. https://doi.org/10.18608/jla.2024.8411.
Patil, M., Patil, V. and Katre, U., 2024. Unspoken science: exploring the significance of body language in science and academia. European Heart Journal, 45(4), pp.250-252. https://doi.org/10.1093/eurheartj/ehad598.
Schneider, J., Börner, D., Van Rosmalen, P., & Specht, M. (2015, November). Presentation trainer, your public speaking multimodal coach. In Proceedings of the 2015 ACM on international conference on multimodal interaction (pp. 539- 546).
Schneider, J., Börner, D., van Rosmalen, P. and Specht, M., 2017. Presentation Trainer: what experts and computers can tell about your nonverbal communication: Presentation Trainer. Journal of Computer Assisted Learning, 33(2), pp.164- 177. https://doi.org/10.1111/jcal.12175.
Svendsen, Ø., 2022. Theorizing Public Performances for International Negotiations. International Studies Quarterly, 66(3), p.sqac031. https://doi.org/10.1093/isq/sqac031.
Zimmerman, B.J. and Schunk, D.H. eds, 1989. Self-Regulated Learning and Academic Achievement. Springer Series in Cognitive Development. [online] New York, NY: Springer New York. https://doi.org/10.1007/978-1-4612-3618-4.
[under review] What is most important when giving a presentation? Authenticity! - Insights on how presentation teachers imagine a software to help with presentation preparation and presentation skills training
Copyright Academic Conferences International Limited 2025