Content area
Programming education is burgeoning, but it encounters hurdles in implementing thinking-based intelligence instruction. The emergence of generative artificial intelligence, through the utilization of prompt engineering, not only provides meticulous feedback but also significantly elevates the quality and efficiency of human-computer interaction (HCI), thereby nurturing computational thinking (CT). This study aimed to reveal the developmental characteristics of CT and the “black box” of the HCI process through learning analytics methods (i.e., microgenetic analysis, lag sequential analysis, cluster analysis, paired t-tests). 44 college students participated in progressive prompt-assisted programming learning. The results indicated that generative progressive prompts significantly improved students’ CT and its sub-dimensions (i.e., creativity, problem-solving, algorithmic thinking, critical thinking, and cooperativity). Moreover, algorithmic thinking was identified as the core skill in CT development. Additionally, regarding students’ HCI patterns, students with low-level CT focused on more superficial interaction patterns, such as guided and exploratory behaviors, while students with high-level CT concentrated on more in-depth interaction patterns, including debating and summarizing behaviors. Based on our findings, educators in programming should incorporate generative prompts and tailor strategies to accommodate diverse HCI patterns among students with varying CT levels.
Introduction
In the era of intelligence, programming education has emerged as a pivotal point for educational reform and in-depth development. It extends beyond the mere acquisition of programming knowledge and skills, emphasizing the cultivation of students’ problem-solving abilities through the ideological and methodological principles of computer science. Consequently, programming education has been entrusted with the significant mission of cultivating students’ computational thinking (CT) (Hsu et al., 2018). Programming entails the process of directing a computer to perform calculations in a specific manner to solve a particular problem and yield the desired output. However, this process is often challenging, as students often struggle with independent algorithm modeling, coding errors, and debugging difficulties. Thus, timely, precise, and personalized support and guidance are required to assist students’ programming learning (Sun & Hsu, 2019).
With rapid technological advancements, the educational value of human-computer interaction (HCI) has gained increasing attention. Integrating generative artificial intelligence (GAI) into education holds the promise of enhancing learning processes across various domains (Chen et al., 2020). GAI’s efficient code generation capabilities assist students in effectively completing programming tasks, from the initial creation of code to its refinement (Huang et al., 2023; Humble et al., 2024). Additionally, it has been leveraged to guide students in mastering the logic and methodologies of code generation (Sun et al., 2024), thereby improving their ability to solve programming problems and enhance their CT skills (Yilmaz & Yilmaz, 2023). CT, as an advanced cognitive process for problem-solving through computer or human-computer collaboration, embodies various thinking skills (e.g., creativity, problem-solving, algorithmic thinking, critical thinking, and cooperativity (Korkmaz et al., 2017; Kaleli̇Oğlu et al., 2016).
However, Hartley et al. (2024) and Peng et al. (2023) indicated that the effectiveness of GAI feedback varied according to learners’ proficiency levels. By optimizing the interaction between humans and GAI, prompt engineering (PE) enhances the learning experience and model response efficiency, providing learners with a flexible and effective method that enables the model to handle a diverse range of task requirements (Meskó, 2023; White et al., 2023). PE has emerged as a strategy to reconcile technological advancements with academic integrity. It involves understanding the capabilities and limitations of large language models, guiding them to generate more appropriate and accurate responses through well-structured prompts (Ali et al., 2023). Specifically, PE consists of general prompts and advanced prompts. First, general prompts enhance the model’s generalization ability by inputting prompting text without significantly changing its structure and parameters (Giray, 2023). Second, advanced prompts, such as the Chain-of-Thought (CoT), Structured CoTs, and Aut CoTs (Zhang et al., 2022; Li et al., 2024) can help students understand the programming problem by breaking down the thought process into well-organized steps.
Despite the implementation of PE to improve HCI patterns using general prompt frameworks and the use of advanced prompts to develop studnets’ thinking abilities, there is a lack of comprehensive analysis regarding the effectiveness of these two types of prompts in programming education. From the analytical perspective, learning analytics (LA) highlights mining and analyzing educational data to understand the specific relational patterns within the data (Giannakos & Cukurova, 2023). LA enables the provision of customized learning resources and learning paths based on students’ learning characteristics and needs, making instruction more precise and efficient. Therefore, compared to previous studies, this study innovatively adopts a generative progressive prompting strategy to facilitate programming learning. Furthermore, advanced LA methods (i.e., microgenetic and lag sequential analysis) are used to deeply explore the enhancement of college students’ CT skills through this strategy, the micro-development characteristics of CT, and the HCI pattern characteristics of students with different CT levels. The unique contribution of this study lies in its revelation of the remarkable advantages of generative progressive prompting in promoting programming learning outcomes. Based on the findings, we provide educators with empirically-based insights and guidance aimed at optimizing programming instruction practices and thereby substantially elevating students’ CT skills.
Literature review
Generative artificial intelligence assisted programming learning
Programming provides a structured method for expressing ideas and solving problems, with its primary objective being the development of problem-solving abilities. However, learning programming can be challenging without sufficient supports, characterized by delayed feedback, limited feedback content, and debugging difficulties (Sun & Hsu, 2019). GAI, exemplified by ChatGPT, serves as a dialogue agent, engaging in iterative interactions with users (Wu & Yu, 2023), and bridges the divide between students and the programming realm through conversational language functions. To address the problems of programming learning, GAI-driven environments can strengthen students’ programming problem-solving skills by interacting with them and providing them with immediate, specific, and multidimensional feedback (Shang & Geng, 2024), which can effectively improve their CT (Yilmaz & Yilmaz, 2023).
With the emergence of artificial intelligence (AI) in education, GAI bring many advantages to programming education, such as generating program and code explanations (MacNeil et al., 2022), identifying and solving coding errors (Ray, 2023), improving algorithms to optimize code performance (Liao et al., 2024), enhancing students engagement and motivation (Yilmaz & Yilmaz, 2022), developing programming skills (Hasrod et al., 2024) and CT. Grounded upon Papert’s constructionism, CT is viewed as a process of understanding and solving problems through hands-on practice and creative activities (Lodi & Martini, 2021). Papert argues that learners develop deep understanding and thinking skills by interacting with computers through construction and programming. GAI can further enhance this process by providing personalized feedback and guidance, thereby enriching students’ learning experiences and improving their CT abilities. Previous research has confirmed that GAI can promote the development of CT. For example, Liao et al. (2024) adopted a CT framework (decomposition, abstraction, algorithms, debugging, iterations) and leveraged GAI to design interactive modules that support various aspects of students’ CT. Yilmaz and Yilmaz (2023) used Chat GPT as a scaffolding tool to assist students in developing five sub-competencies of CT (creativity, problem-solving, algorithmic thinking, critical thinking, and cooperativity).
Although the benefits of GAI have been revealed, its application in programming education for fostering CT is still in the early stages of exploration. Prior researchers also asserted that GAI had limitations and potential threats (Becker et al., 2023; Boguslawski et al., 2024). For example, over-reliance on GAI may weaken students’ problem-solving abilities, hinder the development of meta-cognitive skills essential for good programming (Denny et al., 2024a, b), and reduce the satisfaction derived from solving problems independently (Kazemitabaar et al., 2023). Therefore, further interventions in the human-computer interaction process are needed to enhance the effectiveness of GAI feedback.
Prompt engineering in programming learning
Effective PE provides better guidance for GAI to meet specific educational needs. PE is defined as designing, crafting, and optimizing inputs to elicit specific responses from a GAI model, aiming to optimize human-computer interaction through thoughtfully constructed prompts (Bozkurt & Sharma, 2023). It encompasses two distinct types of prompts: (a) general prompts, which are designed to optimize and enrich the interactive experience between learners and GAIs, and (b) advanced prompts, which are specifically crafted to nurture and elevate learners’ thinking development. Specifically, first, for general prompts, Giray (2023) proposed the ICIO framework: instruction, context, input data, and output indicator. Bozkurt and Sharma (2023) proposed another framework including purpose, tone, role, and context. Learners utilize these frameworks to guide GAI, improving response quality through clear instructions, defined contexts, task decomposition, role setting, prompt refinement, and iteration (Cain, 2024). Second, for advanced prompts, it encompass both foundational prompts and progressive prompts. On the one hand, foundational prompts, such as Zero-Shot and Few-Shot prompting (Kojima et al., 2022), provide the GAI with few or no examples, respectively. On the other hand, progressive prompts, like Chain-of-thought (CoT) (Li et al., 2024), direct the GAI to analyze and solve problems through logical and reasoned prompts. The application of CT skills in solving programming problems reflects the CoT process, and splitting the programming problem-solving process into progressive stages based on CoT principles can provide targeted support for CT development. Denny et al. (2024a, b) demonstrated and enhanced their CT by designing the “Prompt Problems” exercise to develop students’ ability to write prompts for an effective AI code generator. Zhang et al. (2022) introduced Auto-CoT to generate reasoning chains by prompting large language models to “think step-by-step.” Li et al. (2024) introduced Structured CoTs (SCoTs) into programming learning to showcase structured programming thinking through structured reasoning steps.
Overall, general prompts have proven effective in enhancing human-computer interaction efficiency by providing a blend of structure and adaptability, while advanced prompts have successfully fostered cognitive development through a progressive learning approach. However, despite the potential advantages of these prompts, research on the effective integration of these prompts in GAI-assisted programming learning is limited. More importantly, there is a notable absence of research analyzing CT during HCI and its influence on interaction patterns. This gap underscores the necessity to combine both progressive and general prompts and highlights a potential area for LA to explore and address.
Learning analysis of thinking and interaction patterns
LA uses the “digital traces” from learners’ interactions with technology and the learning environment to identify supporting learning factors (Giannakos & Cukurova, 2023). LA methods, such as content analysis and structural analysis (e.g., clustering, social network analysis), represent the multi-level and high-dimensional characteristics of interactive learning by extracting fine-grained data (Ouyang et al., 2023b; Stanja et al., 2023). Hence, compared to traditional analysis of outcome data, LA can identify different types of learners and reflect their process-oriented behavioral patterns more accurately.
Recently, LA has been used to assess learners’ thinking patterns (Sein Minn, 2022) and interaction patterns (Andreja, 2019) during the learning processes. Specifically, first, for thinking patterns, multiple assessment methods include learning behavior analysis (Allsop, 2019), learning trajectories (Kong & Wang, 2023), and language. Sullivan and Keith (2019) integrated natural language processing (NLP) with microgenetics to support multimodal analysis of students’ collaborative learning with educational robots, thereby promoting the microgenetic development of conceptual understanding. Wu et al. (2019) employed quantitative ethnography and discovered that the CT of high-performance groups is systematic, while low-performance groups are characterized by tinkering or guess-and-check approaches. Second, for interaction patterns, Xu et al. (2023) categorized the pair programming styles of 19 pairs of university students into four peer interaction patterns—consensus mode, debate-driven mode, individual-oriented mode, and trial-and-error mode—by combining multimodal process and product data. Barke et al. (2023) found that interactions between programmers and GAI primarily fall into two HCI patterns: the “Exploration” pattern (learning new concepts) and the “Acceleration” pattern (using AI for quicker goal achievement). This suggests that learners exhibit different thinking patterns and interaction patterns.
The present study
Although the benefits of programming learning in promoting students’ CT have been revealed in the previous works, in-depth research on how PE-assisted GAI leads students to construct and enhance their CT is lacking. Meanwhile, no existing research examined the characteristics of interactions between GAI and students with varying levels of CT. Therefore, this study designed a generative progressive prompt sheet integrating general and advanced prompts, and further employed LA to explore the characteristics of student CT development and HCI patterns during programming learning. Specifically, the present study included the following three research questions:
RQ1
What are the characteristics of students’ CT development in generative progressive prompt-assisted programming learning?
RQ2
What are the characteristics of students’ HCI in generative progressive prompt-assisted programming learning?
RQ3
What are the characteristics of HCI patterns for students with different levels of CT in generative progressive prompt-assisted programming learning?
Methods
Participants and learning context
We employed a convenient sampling method to recruit 44 undergraduate students aged 18–19 (M = 18.75, SD = 0.65) majoring in Educational Technology from a research university in China, with a 50% male representation. Prior informed consent was obtained from all participants, who had no prior exposure to CT-related courses. To protect the privacy of the participants.all information about the participants in this study was anonymized and used only forresearch purposes.
In this study, we utilized an online Python editor (https://www.sciclass.cn/python) and innovatively integrated it with the Kimi Intelligent Assistant (https://kimi.moonshot.cn/), an online GAI tool, to enhance students’ programming education (see Fig. 1). This integration effectively harnessed the strengths of both Python’s robust programming environment and Kimi’s advanced natural language processing capabilities, resulting in a significant improvement in human-computer interaction. Moreover, since the potential constraints in Kimi’s comprehension stemming from students’ varying question-asking abilities, we introduced the generative progressive prompt sheet as a strategic solution. This sheet served as a guiding scaffold, enabling students to independently design and implement a text-based adventure game titled “Lost Treasure.” In the process, students met the core programming requirements, which included incorporating loop structures, input processing mechanisms, and state management techniques. The ultimate goal of this approach was to foster and refine students’ CT skills through multiple rounds of dialogue and interaction.
[See PDF for image]
Fig. 1
Learning environment
Generative progressive prompt sheet design
A structured HCI prompt strategy was used to construct a generative progressive prompt sheet (see Table 1), which contained general and advanced prompts designed to guide students to explore and learn programming knowledge step by step through active questioning and in-depth dialogues.
General prompt engineering design
The design of general prompt engineering was as intuitive as object-oriented programming, allowing students to invoke it directly before they began their programming tasks. Drawing inspiration from Giray (2023), we have introduced the “CRO” framework, where “C” represented contexts, aiding the large language model in understanding the background information pertinent to the task, encompassing the topic area and target audience; “R” denoted roles, outlining the essential skills required by the large language model, along with the rules and regulations it needed to adhere to; and “O” stood for outputs, detailing the format of the output anticipated to be produced by the large language model (e.g., short answers, paragraphs, etc.).
Advanced prompt engineering design
Taking microgenesis from sociocultural theory as a methodology (Wertsch & Hickmann, 1987), based on fundamental observation principles such as period and frequency, the progressive process of programming scenarios was divided into four stages (i.e., situation exploration, program generation, model optimization, and knowledge internalization). The first two stages primarily focused on the absorption and processing of external information by the learners, while the latter two stages emphasized the output and evaluation of the cognitive processing results within the learners. To enable students to consciously utilize CT-related sub-skills during the process of completing programming tasks, we have designed tailored cue words corresponding to the primary objectives of each stage. For example, during the situation exploration stage, learners may refer to cue words to inquire of GAI: “Please provide creative ideas for [task].” This cue is intended to stimulate students’ creativity.
Table 1. Generative progressive prompt sheet
General prompts | Advanced prompts | ||
|---|---|---|---|
Programming problem-solving progressive stage | Progressive prompt | Function | |
Contexts: I am a college student just starting to learn the basics of Python, and I want to use a simple command-line program, “Login Interface,” to grasp Python programming knowledge. You need to guide me step by step through this task in my learning style and help me understand the basic principles. Role: You will act as my Python programming assistant. Please provide specific explanations for any programming terminology and the essential Python syntax: sequential statements (input, print), conditional statements (if, else, elif), loops (for, while, break, continue), and lists—strings and other fundamental data types, with detailed explanations and examples. Outputs: Do not use third-party libraries or customize functions. Keep each response under 100 characters. If more content is needed, we can engage in a multi-turn dialogue to guide my questions. | Situation exploration stage | Please provide me with creative ideas regarding the [Task]. | Students use the GAI to deepen their understanding of the task context, develop innovative design proposals, and stimulate their creative thinking. |
Program generation stage | Please help me solve [Programming problems] and explain each step involved in the solution. | Students adopt appropriate questioning strategies to interact with GAI efficiently, carefully plan steps to solve problems, and translate solutions into concrete actions, demonstrating their problem-solving abilities. | |
Model optimization stage | Please debug this code, which should execute [Purpose]. | Students accurately identify problems, reflect on and optimize solutions, and apply algorithmic thinking to improve performance. | |
Knowledge internalization stage | You still have problems in the ** aspect, which can be modified with [Method]. | Students analyze and assess their learning outcomes and the GAI feedback, identify the strengths and weaknesses, and adjust their learning strategies accordingly to transform from understanding knowledge to internalizing knowledge, promoting critical thinking development. | |
Research procedure
The entire activity lasted approximately 130 min. Before implementation, the teaching assistant spent 30 min to introduce GAI, including how to use Kimi and its limitations. Participants were given 20 min to complete a self-assessment of their CT skills. Subsequently, participants followed the four stages (i.e., situation exploration, program generation, model optimization, and knowledge internalization) of the generative progressive prompt sheet to assist in completing the text-based adventure game programming task in 60 min, with each stage lasting approximately 15 min. Finally, after the generative progressive prompt-assisted programming learning, participants completed a 20-minute self-assessment of the CT skills.
Data collection
CT measurement data
In this study, students’ CT was measured through two ways, including outcome scale measurement and process behavior measurement.
Outcome scale measurement: To assess students’ CT skills from a self-perception perspective, we used the 29-item, five-point Computational Thinking Scale (CTS) developed by Korkmaz et al. (2017), which contains five factors: (1) creativity, (2) problem-solving, (3) algorithmic thinking, (4) critical thinking, and (5) cooperativity. Scale responses were based on a five-point Likert scale ranging from “1 = never, 2 = rarely, 3 = sometimes, 4 = usually to 5 = always”. The Cronbach’s alpha reliability coefficient for the pre-test full scale was 0.90, and the alpha reliability coefficients for the five factors were 0.80, 0.65, 0.95, 0.88, and 0.92. The Cronbach’s alpha reliability coefficients for the post-test full scale were 0.90, and the alpha reliability coefficients for the five factors were 0.74, 0.67, 0.89, 0.85, and 0.89.
Process behavior measurement: To explore the characteristics and stages of CT micro-development from a behavioral perspective, the study recorded the students’ entire programming process with an online computer screen recording program (silent) (https://www.mnggiflab.com/screen-recorder) with video data totaling 2,160 min (M = 53.68, SD = 4.63).
HCI dialogue data
To understand students’ HCI patterns, their dialogues with Kimi were recorded through the platform’s logs. All the data were organized into Excel according to the sequence of “student question content - GAI feedback content - student response to GAI feedback content”.
Data analysis
Analysis of CT
First, paired-sample t-tests were used to analyze the CT outcome scale scores to explore the effect of generative progressive prompts on students’ CT. Based on Cohen’s (1988) classification scheme of effect size, we further used an effect size calculator for the test statistics of paired sample t-test (https://www.psychometrica.de/effect_size.html). Second, based on previous researchers’ definitions of CT Yağcı, 2019; Ennis, 1962; Korkmaz et al., 2017), we proposed a coding framework to establish connections between behavioral responses and CT sub-dimensions (see Table 2). Specifically, two coders converted silent programming videos into textual data, meticulously noting the dialogue and actions such as copying or pasting feedback from the GAI, modifying programs, and running or debugging them. These actions were recorded in chronological order, adhering to the four programming stages: situation exploration, program generation, model optimization, and knowledge internalization. From this transcribed data, the two coders analyzed students’ CT behaviors. For instance, students sought creative ideas from GAI by using prompts like, “Please provide me with creative ideas for this programming game development.” They then incorporated GAI’s responses into their designs, such as by adding forest and castle elements suggested by GAI. This series of actions was coded as “Creativity (TCR),” indicating that students were effectively utilizing GAI’s feedback to create innovative solutions. A total of 1726 behavioral codes were obtained. The inter-rater reliability between the two coders was 0.80, ensuring accurate categorization and consistency in the coding process.
To explore the microgenetic process of students’ CT, we used microgenetic analysis, a high-density observational research technique that captures the trajectory of individual CT skills and analyzes interactions between students and GAI over a short period, while minimizing extraneous variables (e.g., individual development) (Sullivan et al., 2015). Microgenetic analysis must have three elements (Siegler, 2006): (a) appropriate observation periods, spanning several hours, days, or weeks from the implementation to the conclusion of the intervention strategy (Siegler & Crowley, 1991), (b) high observation frequency, capturing every instance of CT behavioral manifestation during the intervention, and (c) fine-grained data analysis, involving high-precision measurement and analysis. We examined different aspects of change from two dimensions: the path and the rate of change.
Table 2. CT behavioral analysis framework
Dimension | Description | Operational definition of observable behavior | Example |
|---|---|---|---|
Creativity (TCR) (Dobres & Mithen, 2000) | Make new combinations from old ideas and find new products. | Recombine GAI feedback to create novel solutions. | Students incorporate forest and castle elements provided by GAI to create new game levels. |
Problem Solving (TPS) (Mayer, 1998) | Acquire knowledge of the basic syntax (cognitive structures) needed for programming tasks, and complete program design (the process needed to solve problems). | Define the scope of the problem and develop detailed steps to solve it. | Students analyze the logs, identify the issues triggering the storyline, correct conditional judgments, and implement solution steps one by one. |
Algoritmic Thinking (TAT) (Yağcı, 2019) | Arrange procedures (think about, understand, apply, evaluate, and create algorithms) to solve problems in order. | Break down complex problems into a series of simple algorithmic steps. | Students decompose complex character movement logic into three algorithmic steps input processing, conditional judgment, and output feedback, and then write code. |
Critical Thinking (TCT) (Ennis, 1962) | Assess the statements correctly. | Evaluate the advantages and disadvantages of the different technology options offered by GAI to make an informed choice. | Students evaluate the two options provided by GAI and choose the most feasible game character design. |
Cooperativity (TCO) (Korkmaz et al., 2017) | Learn collaboratively for a common goal. | Communicate effectively with GAI to coordinate problem-solving. | Students communicate with GAI about game design and work together to solve development problems. |
Analysis of HCI pattern
To analyze the characteristics of students’ HCI, the HCI content analysis framework was used in this study, including cognitive levels of posing questions to GAI, types of GAI feedback content, and feedback from students on GAI content feedback (see Table 3).
First, regarding students’ questions to GAI, we referred to the openness of conversational questions (Šeďová et al., 2020; Nassaji & Wells, 2000) and cognitive demands levels (Gayle et al., 2007), and put forward four categories of questions: closed questions for low cognitive demand (QCL), open questions for low cognitive demand (QOL), closed questions for high cognitive needs (QCH) and open questions for high cognitive demands (QOH).
Second, regarding GAI’s feedbacks, five categories of programming feedback were proposed: knowledge about task constraints (KTC), knowledge about programming concepts (KPC), knowledge about programming mistakes (KPM), knowledge about how to proceed (KHP) and knowledge about meta-cognition (KMC) (Keuning et al., 2018; Kiesler et al., 2024).
Third, regarding students’ responses to GAI, four categories were proposed (Ouyang et al., 2023a): building consensus (FBC), requesting explanations (FRE), raising objections (FRO), and criticizing and summarizing (FCS). In the coding process, each question raised by the student and each answer provided by the GAI are considered separate coding units for a comprehensive analysis of the interaction between the student and the GAI. Two researchers manually coded the data according to the coding scheme and performed consistency comparisons, achieving a reliability score of 0.83 across 2,068 coded units.
Table 3. HCI content analysis framework
Dimension | Category | Description | Example |
|---|---|---|---|
Question from student to GAI | Closed questions for low cognitive demand (QCL) | Focus on foundational and memorized knowledge and seek clear answers to reinforce memory or understanding of basic concepts. | What are the parameters of this function? |
Open questions for low cognitive demand (QOL) | Focus on foundational and principled knowledge and explore the principles or mechanisms behind the problem to deepen understanding. | What is wrong with this program? | |
Closed questions for high cognitive demand (QCH) | Focus on complexity and applied knowledge, explore precise explanations or comprehensive understandings of complex concepts. | What is the time complexity of this algorithm? | |
Open questions for high cognitive demand (QOH) | Address complex issues in practical applications and seek diverse insights, solutions, and innovative ideas. | How do you properly use conditional statements (e.g., if, elif, or else) to control the game’s flow? | |
GAI’s feedback | Knowledge about task constraints (KTC) | Hints on task requirements and task-processing rules. | You will need to explain the specific requirements and limitations of the program to me. |
Knowledge about programming concepts (KPC) | Explanations of subject matter or concepts | Variables are containers for storing data, and Python automatically determines their type when assigning a value. | |
Knowledge about programming mistakes (KPM) | Test failures and solution errors | Your code has syntax errors that prevent the program from working because of poor writing. | |
Knowledge about how to proceed (KHP) | Task-processing steps (TPS). | You can break down the task to set the steps and then optimize the code through redundancy elimination during the gradual implementation. | |
Knowledge about meta-cognition (KMC) | Knowledge of individual self-assessment, task comprehension, and strategy selection. | You need to learn to reflect on your programming learning process, identify your strengths and weaknesses, and set appropriate learning goals and strategies to improve your programming skills. | |
Response from student to GAI | Building consensus (FBC) (No relevant new issues raised) | Students accept GAI’s point of view. | You are right; I need to understand this concept more deeply. |
Requesting explanations (FRE) | The student asks the GAI to explain the feedback he or she has provided. | Can you explain more about how this algorithm works? I do not quite understand it. | |
Raising objections (FRO) | Students present an opposing view to the GAI feedback and add reasons and evidence. | I do not think the function parameters are set correctly, and a ‘player choice’ variable should be added to control the storyline direction. | |
Criticizing and summarizing (FCS) | Students critique or conduct GAI feedback. | Your game design ideas inspire me, but for character interaction logic, I think adding more actual scenario simulations would be better. |
To further reveal HCI dialogue behaviors as well as HCI behavioral transition patterns of students at different CT levels, three LA methods were used in this study: (a) frequency distribution statistics (FDS) (Fisher & Marshall, 2009), (b) lag sequential analysis (LSA) (Faraone & Dorfman, 1987), and (c) K-means clustering (Kodinariya & Makwana, 2013). First, we conducted FDS on HCI dialogues based on the above coding framework. Second, to conduct LSA, all HCI coding results were inputted into GSEQ software (Bakeman & Quera, 1995), generating 2027 behavioral transition frequency sequences. Significant behavioral sequences were indicated by standardized Z-scores > 1.96 (Berk et al., 2012). Third, students were classified into high-level (top 50% scores) and low-level groups (bottom 50% scores). Subsequently, the K-means clustering method was applied to cluster the significant behavioral sequences of students with different CT levels. The choice of K-means was driven by its unsupervised learning approach, which does not necessitate prior knowledge of the data structure, allowing for an objective identification of natural groupings (Kodinariya & Makwana, 2013). Cluster analysis aims to divide data into groups, maximizing intra-group similarity and inter-group dissimilarity. Based on the clustering results, diagrams depicting the behavioral transition sequences for various cluster groups were generated.
Results
What are the characteristics of students’ CT development in generative progressive prompt-assisted programming learning?
Analysis of the CT outcome scale
We conducted a paired sample t-test on the CT of college students (see Table 4). The results indicated that the generative progressive prompt sheet significantly enhanced CTS (t = 11.81, p < 0.001, d = 0.81) and creativity (t = 9.47, p < 0.001, d = 1.40), with both exhibiting large effect sizes. Additionally, problem-solving (t = 6.24, p < 0.001, d = 0.72), algorithmic thinking (t = 6.50, p < 0.001, d = 0.71), critical thinking (t = 6.24, p < 0.001, d = 0.71), and cooperativity (t = 4.69, p < 0.001, d = 0.64) showed medium effect sizes.
Table 4. Descriptive statistics of students’ outcome scale of CT
CTS dimension | Stage | M | SD | t | p | Cohen’s d |
|---|---|---|---|---|---|---|
1. Creativity | Pre-test | 3.61 | 0.46 | 9.47 | 0.000 | 1.40 |
Post-test | 4.18 | 0.35 | ||||
2. Problem-solving | Pre-test | 3.32 | 0.44 | 6.24 | 0.000 | 0.72 |
Post-test | 3.73 | 0.54 | ||||
3. Algorithmic thinking | Pre-test | 2.89 | 0.88 | 6.50 | 0.000 | 0.71 |
Post-test | 3.52 | 0.74 | ||||
4. Critical thinking | Pre-test | 3.45 | 0.67 | 6.24 | 0.000 | 0.71 |
Post-test | 3.93 | 0.56 | ||||
5. Cooperativity | Pre-test | 3.75 | 0.76 | 4.69 | 0.000 | 0.64 |
Post-test | 4.11 | 0.67 | ||||
Overall | Pre-test | 3.40 | 0.43 | 11.81 | 0.000 | 0.81 |
Post-test | 3.90 | 0.36 |
[See PDF for image]
Fig. 2
Changes in the average frequency of CT process behavior in programming problem-solving progressive stages
Analysis of the CT process behaviors
The path of change refers to the sequence in which students master CT skills (Siegler & Stern, 1998). The programming problem-solving progressive stages (4 different stages, denoted as T1, T2, T3, T4) influenced the students’ CT process behaviors performance. The average frequency of students’ process behaviors related to CT exhibited a consistent and monotonically increasing trend, indicating a gradual improvement in CT throughout the entire stage. Specifically, the frequency obtained in later stages was invariably higher than those recorded in earlier stages (see Fig. 2). T1: Students excelled in creativity (M = 3.23). T2: Students excelled in problem-solving (M = 3.30) and cooperativity (M = 3.11). T3: Students excelled in algorithmic thinking (M = 5.05). T4: Students excelled in critical thinking (M = 3.89).
The rate of change refers to the speed at which a change occurs (Siegler & Stern, 1998). When observing the development and changes in CT, we used the average frequency of CT behaviors as the dependent variable and displayed its changes at different stages in the form of a line graph (see Fig. 2). If there is a significant difference in CT between adjacent stages, it can be considered that the magnitude of change is large, indicating a fast development rate; conversely, the development rate is slow. Repeated measures ANOVA and post hoc multiple comparisons (Scheffe) revealed that CT improved significantly from T1 to T4 in adjacent stages, with the most significant difference between T2 and T3 (MT3-MT2 = 3.23, p < 0.01), which was mainly due to rapid improvement in algorithmic thinking. The period from T3 to T4 was stable with no significant change (MT4-MT3 = 0.39, p > 0.05).
What are the characteristics of students’ HCI in generative progressive prompt-assisted programming learning?
According to the statistical results, the distribution of learners’ HCI behaviors was revealed. First, regarding the cognitive levels of questions to ask GAI, closed questions for low cognitive demand (QCL: M = 6.09, SD = 4.49) had the highest frequency, followed by open questions for low cognitive demand (QOL: M = 6.32, SD = 3.23), open questions for high cognitive demands (QOH: M = 1.14, SD = 1.68) and closed questions for high cognitive needs (QCH: M = 0.75, SD = 1.35). This result indicated that learners tended to pose questions to the GAI that catered to their lower cognitive needs. Second, regarding the types of GAI feedback content, from highest to lowest frequency: knowledge about programming concepts (KPC: M = 9.80, SD = 2.45), knowledge about task constraints (KTC: M = 1.91, SD = 3.48), knowledge about programming mistakes (KPM: M = 2.34, SD = 2.74), knowledge about how to proceed (KHP: M = 1.39, SD = 2.19), knowledge about meta-cognition (KMC: M = 0.84, SD = 1.49). This showed that learners were mainly concerned with understanding programming concepts while interacting with the GAI, while relatively little attention was paid to task constraints, programming errors, and meta-cognitive aspects. Third, regarding the feedback from students on GAI content feedback, from highest to lowest frequency: building consensus (FBC: M = 13.77, SD = 3.65), raising objections (FRO: M = 0.98, SD = 1.45), criticizing and summarizing (FCS: M = 0.84, SD = 1.49) and requesting explanations (FRE: M = 0.75, SD = 1.54). This result implied that learners perceived the GAI mainly as a tool for solving programming problems, especially for helping them to understand the program error. Consequently, they often accepted its feedback without thorough criticism or summarization.
What are the characteristics of HCI patterns for students with different levels of CT in generative progressive prompt-assisted programming learning?
Cluster analysis of HCI significant dialogue behaviors
We used lag sequential analysis to characterize the internal interaction structure of learners’ HCI dialogue behaviors. K-means clustering was applied to cluster the significant behavior sequences of 44 students based on 23 significant behavioral sequences such as “QCL→KTC,” “QCL→KPC,” and “QOL→KPC.” When k = 4, the Silhouette Coefficient attains its highest value of 0.67, indicating that the k-means clustering at this value of k yields the best results. Subsequently, significant dialogue behavioral transition sequences were identified through residual tables and transition maps (Table 5).
Table 5. Clustering results of HCI for students with different levels of CT
Significant behavioral sequences | Low CT level | High CT level | Total | ||
|---|---|---|---|---|---|
Cluster 1 (N = 13) | Cluster 2 (N = 9) | Cluster 3 (N = 10) | Cluster 4 (N = 12) | ||
QCL→KTC | 84 | 0 | 0 | 0 | 84 |
QCL→KPC | 68 | 36 | 0 | 0 | 104 |
QOL→KPC | 55 | 0 | 113 | 0 | 168 |
KTC→FBC | 84 | 0 | 0 | 0 | 84 |
QOL→KPM | 0 | 27 | 0 | 74 | 101 |
KPC→FBC | 123 | 68 | 124 | 108 | 423 |
FBC→QCL | 140 | 31 | 9 | 66 | 246 |
FBC→QOL | 55 | 29 | 112 | 38 | 234 |
KPM→FRE | 0 | 27 | 0 | 0 | 27 |
FRE→KPC | 0 | 33 | 0 | 0 | 33 |
QCL→KPC | 0 | 0 | 9 | 70 | 79 |
QOH→KHP | 0 | 0 | 19 | 32 | 51 |
KHP→FBC | 0 | 0 | 22 | 32 | 54 |
KHP→FRO | 0 | 0 | 6 | 0 | 6 |
FBC→QOH | 0 | 0 | 19 | 28 | 47 |
FRO→KHP | 0 | 0 | 6 | 0 | 6 |
QCH→KMC | 0 | 0 | 0 | 37 | 37 |
KPM→FBC | 0 | 0 | 0 | 37 | 37 |
KPM→FRO | 0 | 0 | 0 | 38 | 38 |
KMC→FCS | 0 | 0 | 0 | 37 | 37 |
FBC→QCH | 0 | 0 | 0 | 32 | 32 |
FRO→KPC | 0 | 0 | 0 | 38 | 38 |
FCS→QOL | 0 | 0 | 0 | 36 | 36 |
Total | 609 | 251 | 439 | 703 | 2,002 |
Analysis of HCI patterns for students with different levels of CT
The significant dialogue behavioral sequence transitions for the four identified patterns were presented, illuminating the distinctive characteristics of HCI within each pattern (see Fig. 3).
[See PDF for image]
Fig. 3
Sequence diagram of students’ HCI patterns
Cluster 1: Guided HCI pattern
The high-frequency behavioral sequence transitions in cluster 1 were mainly concentrated in “QCL→KTC” (frequency = 84, Z-score = 16.31), “KTC→FBC” (frequency = 84, Z-score = 9.44), “QOL→KPC” (frequency = 55, Z-score = 15.92), “KPC→FBC” (frequency = 123, Z-score = 123), and “KPC→FBC” (frequency = 123, Z-score = 15.92). “QOL→KPC” (frequency = 55, Z-score = 15.92), “KPC→FBC” (frequency = 123, Z-score = 10.60) and other types. It illustrated that students sought guidance and explanations from the GAI by asking closed questions for low cognitive demand (QCL). Questions such as “How should I write this syntax?” or “What are the parameters of this function?” The GAI directly taught the code content to the student (KTC), and the student did not ask new questions to build consensus (FBC), demonstrating an interaction pattern reliant on guidance from the GAI.
Cluster 2: Exploratory HCI pattern
The high-frequency behavioral sequence transitions in cluster 2 mainly included “QOL→KPM” (frequency = 27, Z-score = 14.68), “KPM→FRE” (frequency = 27, Z-score = 14.54), “FRE→KPC” (frequency = 33, Z-score = 7.53), “KPC→FBC” (frequency = 68, Z-score = 9.56), and so on. The students could learn more about the program by independently asking open questions for low cognitive demand (QOL), such as “What is wrong with this program?” The GAI provided knowledge about programming mistakes (KPM), such as “how to fix the error or improve the result, and knowledge about responding correctly to the feedback, including the complete, improved code.” Students continued to ask relevant questions (FRE) as they interacted with the GAI, showcasing an exploratory interaction pattern under the GAI’s guidance.
Cluster 3: Debating HCI pattern
The primary behavioral sequence transitions observed in cluster 3 included “QOL→KPC” (frequency = 113, Z-score = 19.43), “KPC→FBC” (frequency = 124, Z-score = 18.67), “QOH→KHP” (frequency = 19, Z-score = 17.19), “KHP→FRO” (frequency = 6, Z-score = 17.19) and “QOH→KHP” (frequency = 19, Z-score = 17.19), and “KHP→FRO” (frequency = 6, Z-score = 9.52). Students initiated with concept learning and recall (QOL), transitioning to critical thinking and questioning (QOH) directed at the GAI. The GAI provided feedback on how to proceed (KHP), including task processing steps and optimizing program ideas. The student received detailed feedback from the GAI again by raising objections (FRO), and this iterative feedback demonstrated a debated interaction pattern.
Cluster 4: Summarizing HCI pattern
The behavioral transition sequences within Cluster 4 exhibited a variety of types, including: “QOL→KPM” (frequency = 74, Z-score = 25.60), “QOH→KHP” (frequency = 32, Z-score = 27.36), the “QCH→KMC” (frequency = 37, Z-score = 26.67), “KMC→FCS” (frequency = 37, Z-score = 27.57), and many other sequences. The discourse behaviors were diverse, with students posing closed questions for high demand (QCH), such as “What is the time complexity of this algorithm?” Rather than directly providing answers, the GAI educated students on meta-cognition (KMC), including personal programming habits, preferences, and strategies. Throughout their interactions with the GAI, students exhibited a proactive stance in criticizing and synthesizing (FCS) the responses provided, demonstrating a summarized pattern of interaction that focused more on achieving a profound understanding.
Discussions
Generative progressive prompt assists in the development of CT
The learners’ scores on the CT outcomes scale measurement increased significantly, suggesting that the generative progressive prompt can enhance the five key dimensions of CT, as Korkmaz et al. (2017) identified from a cognitive perspective: creativity, problem-solving, algorithmic thinking, critical thinking, and cooperativity. This aligns with the viewpoint of Zhang et al. (2023), who believe that acquiring cognitive skills necessitates a supportive learning environment. Utilizing generative prompts to direct GAI in mimicking the structured reasoning process of humans for output, while visualizing GAI’s reasoning process through specific CT skills, enables learners to build reasoning scaffolds and foster particular modes of thinking. This result concurs with the studies by Denny et al. (2024a, b; Li et al. (2024), which leveraged progressive dialogues to enhance personalized question-and-answer and aid individuals in cultivating a CoT for iterative reasoning.
CT process behaviors’ average frequency increased in the programming problem-solving progressive stage, supported by reliable empirical data from a microgenesis analysis. First, students’ creativity has the highest average frequency in the situation exploration stage. Generative progressive prompts can stimulate students’ divergent thinking and creativity by optimizing problem-solving ideas and helping them understand the concepts involved in the task during this stage. Second, students’ problem-solving and cooperativity skills have the highest average frequency during the program generation stage. Problem-solving gradually establishes a pathway through students’ intrinsic cognitive changes, leading to the formation of systematic solutions. In addition, generative progressive prompts can significantly improve students’ collaborative learning ability, consistent with existing research (Guo et al., 2023). Third, students improved their algorithmic thinking most significantly during the model optimization stage, thus significantly affecting the rate of change in CT. This observation is consistent with the study of Liao et al. (2024), which proposed that generative progressive prompts can aid students in pinpointing and optimizing code issues, thereby fostering a deeper understanding of algorithms. Furthermore, this finding also resonates with the viewpoints of Shute et al. (2017) and Tikva and Tambouris (2021), who all emphasize algorithmic thinking as a fundamental skill in CT, and assert that CT progressively defines or refines operational processes through algorithms. Finally, students showed the highest average frequency in critical thinking during the knowledge internalization stage. This may be because Kimi did not provide completely accurate responses, prompting the student to critically judge the answer provided to complete the task. And, providing the generative progressive prompt framework helped students to generate and maintain critical thinking tendencies. Overall, the key to the generative progressive prompt sheet is to attach the development of the five core sub-competencies of CT to the dynamic generation process of the programming problem-solving chain, where the human-computer fully interact with each other at each stage of the node, leading to the full development of each dimension of students’ CT. This aligns with the Büscher (2025) point of view that CT activities do not occur independently but are intertwined and can influence each other.
Dialogues characteristics of HCI patterns
We provided an in-depth analysis of the HCI dialogue characteristics and found that students mainly tended to have superficial conversations when interacting with the GAI. They tended to ask closed questions for low cognitive demand (QCL) and open questions for low cognitive demand (QOL) to obtain answers quickly, thus simplifying the thinking process to some extent. This suggested that the generative progressive prompt sheet might have had some limitations in guiding students to think deeply or that students had failed to fully understand and apply the guiding strategies.
The HCI in GAI was centered on the learner’s immediate needs, focusing on the timely and phased transmission of knowledge rather than the delivery of irrelevant knowledge or complete information (e.g., KTC, KPM, KHP, KMC). This reflected that learners focused more on the foundational programming concepts when utilizing the GAI and were less likely to explore knowledge in depth. This finding is consistent with Chen et al. (2023) and thinks that learners view the GAI as a helpful programming learning resource and rely on it to guide their learning process.
Further observations revealed that when faced with AI feedback, students most frequently engaged in behaviors that built consensus (FBC), tending to accept and affirm the information provided, often lacking critical analysis and the ability for in-depth exploration. Behaviors such as raising objections (FRO), criticizing and summarizing (FCS), and requesting explanations (FRE) were relatively infrequent. This implied that students may have still been deficient in an active questioning spirit and deep inquiry while utilizing the GAI for their studies. To address this issue, the generative progressive prompt sheet could help overcome the GAI’s ‘illusion’ problem by iteratively refining the model. Additionally, it hinted at the potential for enhancing students’ critical thinking skills.
In summary, while the progressive prompt sheet can promote deeper interaction between students and GAI, there are still deficiencies in questioning behavior, the diversity of GAI feedback content, and student feedback behavior. To maximize the effectiveness of the progressive prompt sheet, we can further provide prompt templates for follow-up questions, such as “Can you help me compare which of these viewpoints better meets the task requirements?” “Follow-up questions” are believed to stimulate cognitive conflict, improve the quality of answers, and are used to optimize question design (Wang et al., 2023). At the same time, ethical issues such as algorithmic bias and data security must be continuously addressed throughout the entire process. Algorithmic bias caused by individual differences can lead to unfairness or inaccuracy in data processing and decision-making (Porayska-Pomsta et al., 2022), exacerbating the differences between high-level and low-level CT students. Therefore, by introducing fairness correction measures (Lee & Kizilcec, 2020), data with unfair features such as horizontal discrimination can be removed to prevent measurement bias in algorithm bias from being inherited and amplified during model training. For data security issues, policy coordination and supervision can be implemented to minimize data privacy and compliance risks in GAI applications (Knight et al., 2023).
HCI patterns for students of different CT levels
Students with different CT levels have significant differences in the level of dialogue content, and promoting deep interaction is the key to improving CT (Lei et al., 2024). Students with lower CT levels were likelier to engage in surface-level dialogue interactions, such as guided HCI and exploratory HCI Patterns. The guided HCI pattern was where students relied to some extent on the GAI to guide their programming learning process and viewed it as a helpful resource. The exploratory HCI pattern involved students asking open questions for low cognitive demand and collaborating more effectively with the GAI by posing relevant new questions to identify and solve errors in programming. Specifically, students possessing lower CT levels have leveraged GAI to streamline the information acquisition process. However, this dialogue may foster a dependency on “instant gratification,” triggering an innate sense of security associated with the reptilian brain’s responses, as discussed in MacLean’s (1990) work. Consequently, it may diminish their engagement in reflective thinking and exploratory processes when confronted with learning tasks, potentially hindering their ability to solve problems independently (Kasneci et al., 2023). Therefore, for students with lower CT proficiency, the focus of CT training should be on helping them reduce their reliance on “instant gratification” by designing prompting phrases for follow-up questions.
Students with high CT levels focused on more in-depth dialogue patterns such as debating HCI and summarizing HCI patterns. The debating HCI pattern involved students questioning GAI’s answers by posing open questions and receiving iterative feedback throughout the process. The most frequent interactions were summarizing the HCI pattern, where students began by asking questions that required a deeper understanding. They incorporated their programming habits and preferences into their interactions with the GAI through metacognition and finally demonstrated mastery of progressive programming features through critique, summary, and implementation. Furthermore, students with higher CT levels may be more motivated to learn, leading them to actively engage in HCI. Specifically, students with high CT levels often sought GAI feedback through independent questioning, internalizing their knowledge, and providing feedback to GAI, ultimately forming a circular dialogue loop of “Question formation - student questioning - GAI answering - student understanding - student feedback,” which reflects the advanced prompting strategies proposed by White et al. (2023). The HCI pattern is not only influenced by the learner’s CT level, questioning style, follow-up intensity, and emotional connection but also by the verticality, relevance, and novelty of GAI responses. Therefore, mitigating the aforementioned “human factors” and “machine factors” can help better explore the optimization direction of generative progressive prompt sheets.
Implications
From an instructional perspective, it is crucial to understand how students use GAI tools in programming learning, where GAI can serve as an intelligent tutor or virtual assistant. We have designed generative progressive prompt sheets, employing prompts to standardize the response quality and accuracy of Kim-like GAI, guiding students to invoke CT sub-skills in various stages. Furthermore, we have distinguished HCI patterns among students with different CT levels and proposed differentiated improvement strategies for each group. For students with lower CT levels, we suggest designing prompt sheets with follow-up questions to stimulate in-depth discussions with GAI. In contrast, for those with higher CT levels, we recommend focusing on cultivating their self-regulatory abilities for reflection, analysis, and summarization, with an emphasis on exploring the logic behind problems. From the research perspective, we used microanalytics to reveal fine-grained data, uncovering previously hidden relationships and elucidating how students invoke different dimensions of CT skills for problem-solving. Theoretically, the design of learning content and tasks remains a primary concern (Farrokhnia, 2022). HCI can make students’ thinking explicit. Our research proposes a basic structure for HCI: student inquiry – GAI response – student feedback or follow-up, which aligns with the basic structure of dialogue proposed by Sinclair and Coulthard (1975).
Conclusion, limitations, and future directions
The current research introduced a generative progressive prompt sheet, aimed at designing a programming problem-solving progressive stage and facilitating the establishment of connections with various CT sub-dimensions (i.e., creativity, problem-solving, algorithmic thinking, critical thinking, cooperativity). Findings from the CT outcome scale indicated that the generative progressive prompt significantly enhanced the CT proficiency of college students. Further microanalytics revealed detailed CT process behaviors, with algorithmic thinking demonstrating the highest rate of development, underscoring its pivotal role within CT. Additionally, cluster analysis and lag sequential analysis revealed that students with lower CT levels tended to engage in more superficial dialogues, such as the guided HCI and exploratory HCI pattern, whereas students with higher CT levels focused on more in-depth dialogues, including the debating HCI and summarizing HCI pattern.
There are some limitations in the current study. First, the sample size and diversity were constrained, focusing predominantly on a single discipline. To validate and enhance our current study findings, future studies will broaden the sample scope and incorporate a more diversified student population. Second, the short study duration may not fully reveal long-term effects. Extended use could lead to deeper understanding, improved problem-solving and critical thinking skills, and sustained enhancements in student engagement and motivation over time. Third, due to rapid technological advancements, different GAIs (e.g., Copilot, Gemini, Deepseek) may produce varied results. In student-GAI interaction scenarios, HCI is complex, as system output depends heavily on user input (Lin, 2024), leading to diverse HCI patterns among individuals. Future research should explore progressive prompt sheet design, while minimizing “human” and “machine” factors.
Acknowledgements
The authors express gratitude towards all the participants for their enthusiastic involvement and cooperation in this study.
Author contributions
Xin Gong contributed to the overall research operations, including conceptualizing the study, designing the experiment, writing the original draft, and conducting data analysis. Weiqi Xu was responsible for data collec-tion and revising the manuscript. Ailing Qiao contributed resources and led the project goals.
Funding
This research was supported by the National Education Scientific Planning Projects of China (DCA220449).
Data availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author upon reasonable request.
Declarations
Competing interests
No potential conflict of interest was reported by the author(s).
Ethics approval and consent to participate
All procedures performed in the study involving human participants followed the World Medical Association Declaration of Helsinki. The research participants and their parents agreed to participate in the study, and their anonymity was ensured.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
Ali, F; Choy, D; Divaharan, S; Tay, HY; Chen, W. Supporting self-directed learning and self-assessment using teachergaia, a generative AI chatbot application: Learning approaches and prompt engineering. Learning: Research and Practice; 2023; 9,
Allsop, Y. Assessing computational thinking process using a multiple evaluation approach. International Journal of Child-Computer Interaction; 2019; 19, pp. 30-55. [DOI: https://dx.doi.org/10.1016/j.ijcci.2018.10.004]
Andreja, IS. Human learning and learning analytics in the age of artificial intelligence. British Journal of Educational Technology; 2019; 50,
Bakeman, R., & Quera, V. (1995). Analyzing interaction: Sequential analysis with SDIS and GSEQ. Cambridge University Press.
Barke, S., James, M. B., & Polikarpova, N. (2023). Grounded Copilot: How Programmers Interact with Code-Generating Models. Proceedings of the ACM on Programming Languages, 7(OOPSLA1), 85–111. https://doi.org/10.1145/3586030
Becker, B. A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., & Santos, E. A. (2023). Programming Is Hard - Or at Least It Used to Be: Educational Opportunities and Challenges of AI Code Generation. Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, 500–506. https://doi.org/10.1145/3545945.3569759
Berk, RH; Bakeman, R; Gottman, J. Observing interaction: An introduction to sequential analysis. Technometrics; 2012; 34,
Boguslawski, S; Deer, R; Dawson, MG. Programming education and learner motivation in the age of generative AI: Student and educator perspectives. Information and Learning Sciences; 2024; [DOI: https://dx.doi.org/10.1108/ILS-10-2023-0163]
Bozkurt, A; Sharma, RC. Generative AI and prompt engineering: The Art of whispering to let the genie out of the algorithmic world. Asian Journal of Distance Education; 2023; 18,
Büscher, C. (2025). Differences in students’ computational thinking activities when designing an algorithm for drawing plane figures. International Journal of Science and Mathematics Education, 23(2), 365–386. https://doi.org/10.1007/s10763-024-10465-3
Cain, W. Prompting change: Exploring prompt engineering in large Language model AI and its potential to transform education. TechTrends; 2024; 68, pp. 47-57. [DOI: https://dx.doi.org/10.1007/s11528-023-00896-0]
Chen, L; Chen, P; Lin, Z. Artificial intelligence in education: A review. Ieee Access: Practical Innovations, Open Solutions; 2020; 8, pp. 75264-75278. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.2988510]
Chen, E., Huang, R., Chen, H. S., Tseng, Y. H., & Li, L. Y. (2023). GPTutor: A ChatGPT-Powered Programming Tool for Code Explanation. 1831, 321–327. https://doi.org/10.1007/978-3-031-36336-8_50
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Academic.
Denny, P., Leinonen, J., Prather, J., Luxton-Reilly, A., Amarouche, T., Becker, B. A., & Reeves, B. N. (2024a). Prompt Problems: A New Programming Exercise for the Generative AI Era. Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1, 296–302. https://doi.org/10.1145/3626252.3630909
Denny, P; Prather, J; Becker, BA; Finnie-Ansley, J; Hellas, A; Leinonen, J; Luxton-Reilly, A; Reeves, BN; Santos, EA; Sarsa, S. Computing education in the era of generative AI. Communications of the ACM; 2024; 67,
Dobres, MA; Mithen, S. Creativity in human evolution and prehistory. American Antiquity; 2000; 64,
Ennis, RH. A concept of critical thinking. Harvard Educational Review; 1962; 32,
Faraone, SV; Dorfman, DD. Lag sequential analysis: Robust statistical methods. Psychological Bulletin; 1987; 101,
Farrokhnia, M. Bridging the fields of entrepreneurship and education: The role of philosophical perspectives in fostering opportunity identification. The International Journal of Management Education; 2022; 20,
Fisher, MJ; Marshall, AP. Understanding descriptive statistics. Australian Critical Care; 2009; 22,
Gayle, B. M., Preiss, R. W., & Allen, M. (2007). How effective are teacher-initiated classroom questions in enhancing student learning. In B. M. Gayle, R. W. Preiss, N. Burrell, & M. Allen (Eds.), Classroom communication and instructional processes: Advances through meta-analysis (pp. 279–293). Lawrence Erlbaum Associates.
Giannakos, M; Cukurova, M. The role of learning theory in multimodal learning analytics. British Journal of Educational Technology; 2023; 54,
Giray, L. Prompt engineering with chatgpt: A guide for academic writers. Annals of Biomedical Engineering; 2023; 51, pp. 2629-2633. [DOI: https://dx.doi.org/10.1007/s10439-023-03272-4]
Guo, K; Zhong, Y; Li, D; Chu, SKW. Effects of chatbot-assisted in-class debates on students’ argumentation skills and task motivation. Computers & Education; 2023; 203, 104862. [DOI: https://dx.doi.org/10.1016/j.compedu.2023.104862]
Hartley, K; Hayak, M; Ko, UH. Artificial intelligence supporting independent student learning: An evaluative case study of ChatGPT and learning to code. Education Sciences; 2024; 14,
Hasrod, T; Nuapia, YB; Tutu, H. ChatGPT helped me build a chemistry app, and here’s how you can make one also. Journal of Chemical Education; 2024; 101,
Hsu, TC; Chang, SC; Hung, YT. How to learn and how to teach computational thinking: Suggestions based on a review of the literature. Computers & Education; 2018; 126, pp. 296-310. [DOI: https://dx.doi.org/10.1016/j.compedu.2018.07.004]
Huang, AYQ; Lu, OHT; Yang, SJH. Effects of artificial Intelligence–Enabled personalized recommendations on learners’ learning engagement, motivation, and outcomes in a flipped classroom. Computers & Education; 2023; 194, 104684. [DOI: https://dx.doi.org/10.1016/j.compedu.2022.104684]
Humble, N; Boustedt, J; Holmgren, H; Milutinovic, G; Seipel, S; Östberg, AS. Cheaters or AI-Enhanced learners: Consequences of ChatGPT for programming education. Electronic Journal of E-Learning; 2024; 22,
Kaleli̇Oğlu, FK; Gülbahar, Y; Kukul, V. A framework for computational thinking based on a systematic research review. Baltic Journal of Modern Computing; 2016; 4,
Kasneci, E; Sessler, K; Küchemann, S; Bannert, M; Dementieva, D; Fischer, F; Gasser, U; Groh, G; Günnemann, S; Hüllermeier, E; Krusche, S; Kutyniok, G; Michaeli, T; Nerdel, C; Pfeffer, J; Poquet, O; Sailer, M; Schmidt, A; Seidel, T; Kasneci, G. ChatGPT for good? On opportunities and challenges of large Language models for education. Learning and Individual Differences; 2023; 103, 102274. [DOI: https://dx.doi.org/10.1016/j.lindif.2023.102274]
Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023). Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–23. https://doi.org/10.1145/3544548.3580919
Keuning, H; Jeuring, J; Heeren, B. A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education; 2018; 19,
Kiesler, N., Lohr, D., & Keuning, H. (2024). Exploring the Potential of Large Language Models to Generate Formative Programming Feedback. 18–21. https://doi.org/10.1109/FIE58773.2023.10343457
Knight, S; Dickson-Deane, C; Heggart, K; Kitto, K; Çetindamar Kozanoğlu, D; Maher, D; Narayan, B; Zarrabi, F. Generative AI in the Australian education system: An open data set of stakeholder recommendations and emerging analysis from a public inquiry. Australasian Journal of Educational Technology; 2023; 39,
Kodinariya, TM; Makwana, PR. Review on determining number of cluster in K-Means clustering. International Journal of Advance Research in Computer Science and Management Studies; 2013; 1,
Kojima, T; Gu, SS; Reid, M; Matsuo, Y; Iwasawa, Y. Large Language models are Zero-Shot reasoners. Advances in Neural Information Processing Systems; 2022; 35, pp. 22199-22213.
Kong, SC; Wang, YQ. Monitoring cognitive development through the assessment of computational thinking practices: A longitudinal intervention on primary school students. Computers in Human Behavior; 2023; 145, 107749. [DOI: https://dx.doi.org/10.1016/j.chb.2023.107749]
Korkmaz, Ö; Çakir, R; Özden, MY. A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior; 2017; 72, pp. 558-569. [DOI: https://dx.doi.org/10.1016/j.chb.2017.01.005]
Lee, H., & Kizilcec, R. F. (2020). Evaluation of Fairness Trade-offs in Predicting Student Success (No. arXiv:2007.00088). arXiv. https://doi.org/10.48550/arXiv.2007.00088
Lei, Y., Fu, X., Zhao, J., & Yi, B. (2024). The effect of ability grouping on students’ computational thinking in shared regulation-supported collaborative programming. Education and Information Technologies, 1–27. https://doi.org/10.1007/s10639-024-12838-1
Li, J; Li, G; Li, Y; Jin, Z. Structured Chain-of-Thought prompting for code generation. ACM Transactions on Software Engineering and Methodology; 2024; [DOI: https://dx.doi.org/10.1145/3690635]
Liao, J; Zhong, L; Zhe, L; Xu, H; Liu, M; Xie, T. Scaffolding computational thinking with ChatGPT. IEEE Transactions on Learning Technologies; 2024; 17, pp. 1668-1682. [DOI: https://dx.doi.org/10.1109/TLT.2024.3392896]
Lin, Z. How to write effective prompts for large Language models. Nature Human Behaviour; 2024; 8, pp. 611-615. [DOI: https://dx.doi.org/10.1038/s41562-024-01847-2]
Lodi, M; Martini, S. Computational thinking, between papert and wing. Science & Education; 2021; 30, pp. 883-908. [DOI: https://dx.doi.org/10.1007/s11191-021-00202-5]
MacLean, P. D. (1990). The triune brain in evolution: Role in paleocerebral functions. Springer Science & Business Media.
MacNeil, S., Tran, A., Mogil, D., Bernstein, S., Ross, E., & Huang, Z. (2022). Generating Diverse Code Explanations using the GPT-3 Large Language Model. Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2, 37–39. https://doi.org/10.1145/3501709.3544280
Mayer, R. Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science; 1998; 26,
Meskó, B. Prompt engineering as an important emerging skill for medical professionals: Tutorial. Journal of Medical Internet Research; 2023; 25, e50638. [DOI: https://dx.doi.org/10.2196/50638]
Nassaji, H; Wells, G. What’s the use of’triadic dialogue’? An investigation of teacher-student interaction. Applied Linguistics; 2000; 21,
Ouyang, F., Wu, M., Zhang, L., Xu, Zheng, L., & Cukurova, M. (2023a). A step closer, yet Miles away: Insights into the complexity of regulation of learning in collaborative knowledge construction. Computers in Human Behavior, 142. https://doi.org/10.1016/j.chb.2023.107650
Ouyang, F; Wu, M; Zhang, L; Xu, W; Zheng, L; Cukurova, M. Making strides towards AI-supported regulation of learning in collaborative knowledge construction. Computers in Human Behavior; 2023; 142, 107650. [DOI: https://dx.doi.org/10.1016/j.chb.2023.107650]
Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot (No. arXiv:2302.06590). arXiv. https://doi.org/10.48550/arXiv.2302.06590
Porayska-Pomsta, K; Holstein, K; Sutherland, E; Baker, T; Shum, SB; Santos, OC; Rodrigo, MT; Cukurova, M; Bittencourt, II; Koedinger, KR. Ethics of AI in education: Towards a Community-Wide framework. International Journal of Artificial Intelligence in Education; 2022; 32, pp. 504-526. [DOI: https://dx.doi.org/10.1007/s40593-021-00239-1]
Ray, PP. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems; 2023; 3, pp. 121-154. [DOI: https://dx.doi.org/10.1016/j.iotcps.2023.04.003]
Šeďová, K., Šalamounová, Z., Švaříček, R., & Sedláček, M. (2020). Getting dialogic teaching into classrooms. Springer Singapore.
Sein Minn. AI-assisted knowledge assessment techniques for adaptive learning environments. Computers and Education: Artificial Intelligence; 2022; 3, 100050. [DOI: https://dx.doi.org/10.1016/j.caeai.2022.100050]
Shang, S; Geng, S. Empowering learners with AI-generated content for programming learning and computational thinking: The lens of extended effective use theory. Journal of Computer Assisted Learning; 2024; 40,
Shute, VJ; Sun, C; Asbell-Clarke, J. Demystifying computational thinking. Educational Research Review; 2017; 22, pp. 142-158. [DOI: https://dx.doi.org/10.1016/j.edurev.2017.09.003]
Siegler, R. S. (2007). Microgenetic analyses of learning. In D. Kuhn, R. S. Siegler, W. Damon, & R. M. Lerner (Eds.), Handbook of child psychology: Cognition, perception, and language (6th ed., pp. 464–510). Wiley. https://doi.org/10.1002/9780470147658.chpsy0211
Siegler, RS; Crowley, K. The microgenetic method: A direct means for studying cognitive development. American Psychologist; 1991; 46,
Siegler, RS; Stern, E. Conscious and unconscious strategy discoveries: A microgenetic analysis. Journal of Experimental Psychology: General; 1998; 127,
Sinclair, J. M., & Coulthard, M. (1975). Towards an analysis of discourse: The english used by pupils and teachers. Oxford University Press.
Stanja, J; Gritz, W; Krugel, J; Hoppe, A; Dannemann, S. Formative assessment strategies for students’ conceptions—The potential of learning analytics. British Journal of Educational Technology; 2023; 54,
Sullivan, FR; Keith, PK. Exploring the potential of natural Language processing to support microgenetic analysis of collaborative learning discussions. British Journal of Educational Technology; 2019; 50,
Sullivan, F. R. S. R., Adrion, R., & Keith, P. K. (2015). Microgenetic Learning Analytics: A Computational Approach to Research on Student Learning. 16–20.
Sun, D., Boudouaia, A., Zhu, C., & Li, Y. (2024). Would ChatGPT-facilitated programming mode impact college students’ programming behaviors, performances, and perceptions? An empirical study. International Journal of Educational Technology in Higher Education, 21(1), 14. https://doi.org/10.1186/s41239-024-00446-5
Sun, JCY; Hsu, KYC. A smart eye-tracking feedback scaffolding approach to improving students’ learning self-efficacy and performance in a C programming course. Computers in Human Behavior; 2019; 95, pp. 66-72. [DOI: https://dx.doi.org/10.1016/j.chb.2019.01.036]
Tikva, C; Tambouris, E. Mapping computational thinking through programming in K-12 education: A conceptual model based on a systematic literature review. Computers & Education; 2021; 162, 104083. [DOI: https://dx.doi.org/10.1016/j.compedu.2020.104083]
Wang, X; Liu, Q; Pang, H; Tan, SC; Lei, J; Wallace, MP; Li, L. What matters in AI-supported learning: A study of human-AI interactions in Language learning using cluster analysis and epistemic network analysis. Computers & Education; 2023; 194, 104703. [DOI: https://dx.doi.org/10.1016/j.compedu.2022.104703]
Wertsch, J., & Hickmann, M. (1987). Problem solving in social interaction: A microgenetic analysis. In M. Hickmann (Ed.), Social and functional approaches to language and thought (pp. 251–266). Academic.
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023). A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. https://doi.org/10.48550/arXiv.2302.11382
Wu, R; Yu, Z. Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. British Journal of Educational Technology; 2023; 55,
Wu, B; Hu, Y; Ruis, AR; Wang, M. Analysing computational thinking in collaborative programming: A quantitative ethnography approach. Journal of Computer Assisted Learning; 2019; 35,
Xu, W; Wu, Y; Ouyang, F. Multimodal learning analytics of collaborative patterns during pair programming in higher education. International Journal of Educational Technology in Higher Education; 2023; 20,
Yağcı, M. A valid and reliable tool for examining computational thinking skills. Education and Information Technologies; 2019; 24, pp. 929-951. [DOI: https://dx.doi.org/10.1007/s10639-018-9801-8]
Yilmaz, F. G. K., & Yilmaz, R. (2022). Examining student satisfaction with the use of smart mooc.
Yilmaz, R; Yilmaz, FGK. The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation. Computers and Education: Artificial Intelligence; 2023; 4, 100147. [DOI: https://dx.doi.org/10.1016/j.caeai.2023.100147]
Zhang, Z., Zhang, A., Li, M., & Smola, A. (2022). Automatic Chain of Thought Prompting in Large Language Models (No. arXiv:2210.03493). arXiv. https://doi.org/10.48550/arXiv.2210.03493
Zhang, JH; Meng, B; Zou, LC; Zhu, Y; Hwang, GJ. Progressive flowchart development scaffolding to improve university students’ computational thinking and programming self-efficacy. Interactive Learning Environments; 2023; 31,
© The Author(s) 2025. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.