1. Introduction
Education currently faces many demands. Science, Technology, Engineering, and Mathematics (STEM) topics have been very desirable for decades (Martín-Páez et al., 2019), with skills relevant to distributed computing and the Internet of Things especially in demand (Abichandani et al., 2022). At the same time, STEM education is now expected to reach a broader audience and attract student demographics that historically have been less interested in, or discouraged from, participating in it. Including arts and creativity, to create a larger multidisciplinary domain, Science, Technology, Engineering, Arts, and Mathematics (STEAM), broadens its appeal and can help overcome the stereotype of STEM as being a boring, overly technical field.
However, to meet the demand for education with a wider audience, tools must be novice-friendly and accessible. They must also provide adequate room for student growth. An approach of “lowering the floor, widening the walls, and raising the ceiling” (Papert, 1980; Resnick, 2017) should be taken. A low floor refers to reducing the barriers to entry, widening the walls refers to expanding the domain, and raising the ceiling means allowing (but not requiring) additional complexity for advanced students. Taking these concepts into account provides a learning opportunity inclusive of as many students as possible without limiting their potential. A novice-friendly design approach provides a lower floor, essential for attracting students to further studies. This work expands upon previous research into block-based programming and educational robotics simulation, widening the scope to include more diverse connected devices and student-driven scenario creation. The scope of these scenarios will also expand beyond robotics programming tasks.
Robots have a significant history as tools for teaching not just programming or computer science (Blank, 2006) but also other STEM topics (Catlin & Blamires, 2010). Educational robotics is well established as capturing student attention. However, classroom use for these robots presents many barriers to entry. The financial costs for purchasing robots and the logistical costs of storage are just the beginning. Maintenance often relies on instructors having electrical engineering knowledge or using a commercial-off-the-shelf platform with reduced flexibility. Moving to a digital simulation of educational robots can eliminate the financial and logistical burdens. However, simulations must be compatible with school-provided hardware and networks. Additionally, the need for instructors with electrical engineering knowledge may be replaced with a need for instructors with systems administration skills. Web-based simulations can further reduce costs, though they require a stable internet connection (Tellez, 2017). Additionally, distance education adoption significantly increased during the COVID-19 pandemic, helping to accelerate the ongoing “digital transformation” in education (García-Morales et al., 2021). Despite demand for distance education coming down from its peak over the immediate post-pandemic years, the need for more accessible online education remains clear (Siegel et al., 2021; Z. Zhang & Gillespie, 2023). Educational robotics must adapt in order to prepare for new crises that may take students out of classrooms, while also accommodating a wider range of students (McCarthy et al., 2023). Eliminating the physical interaction with educational robots creates new opportunities to provide accessibility for students with disabilities or in other situations preventing their use.
The use of robots in computer science education, and in education in general, comes from multiple factors. While it is important that popular interest in robots has continued to increase along with their accessibility, robots can provide many educational opportunities (Blank, 2006; Eguchi, 2017). Robots inherently provide possibilities for engineering and programming lessons, but they fundamentally enhance how students learn from them, by their nature creating “hands-on mind-on learning” in a fun and engaging learning environment where collaboration is possible. The existence of robots in a physical space helps students obtain feedback during learning (Mikropoulos & Bellou, 2013), and contributes to their use as learning objects (Zuckerman, 2010). Students are motivated to learn skills across multiple domains to work with robots that a less exciting modality would not encourage (Eguchi, 2014). Students participating in educational robotics activities in general show improvement in computational thinking skills (Hong, 2024) and attitudes towards STEM topics across all age groups and genders (Y. Zhang et al., 2021). Educational robotics also provides for intuitive extensions such as competitions, and can be made more accessible both by simplified development tools and simulation environments.
While educational robots have great promise in the classroom, they are not always possible or feasible to use due to the logistics and limitations of physical robots (Tselegkaridis & Sapounidis, 2021). The most apparent of these costs is the up-front cost of purchasing not just one robot but sufficient robots for an entire class’ activities. This initial barrier to entry can range from hundreds to tens of thousands of dollars per student or group (Kalaitzidou & Pachidis, 2023). Unfortunately, this is not the sole potential financial cost regarding educational robotics use. Robots will naturally experience wear and tear, especially when used with younger students, and maintenance will be required. In the case where some form of warranty was available, repairs are straightforward but may remove a robot from use for a significant length of time. If no repair service is available, the work falls on the students, instructor, or school staff, who would need to have the necessary experience to conduct maintenance on the robot. If maintenance is no longer possible, or an online platform the robot relies on is no longer supported, physical robots may become e-waste, forcing teachers to begin the purchasing process anew. Providing activities for students comes with difficulties as well; these must be set up and cleaned up after. If a large course or other area is created for robots to move within, students will demand a certain level of consistency between uses for the sake of fairness. Some robots may also require software that requires approval and coordination with school information technology departments to establish on school computers, if it is compatible at all. Students with disabilities may be unable to fully participate in these activities, regardless of the accessibility of the programming interface, simply due to the physical interaction required to assemble and relocate the robots. Storage must be provided, both for the robots and the materials for activities. Furthermore, some activities are inherently difficult to perform in an educational setting. Drones may be unsafe or lack sufficient space when working indoors, and otherwise engaging scenarios such as hazardous materials cleanup can only be provided as an abstraction to avoid creating a risk to students and equipment. Moving to simulated robots can reduce or eliminate all these barriers to educational robot use.
Virtual robots have a dramatically reduced cost compared to their physical counterparts, with even high end computers for simulation use being less expensive than the robots under simulation. Simpler robots may be able to be simulated on low-cost computers, creating no additional cost for introducing a simulated robot to the classroom. A simulation may include the ability for robots to become damaged or otherwise inoperable, but a robot in this status is simply a reset button press away from being restored to working condition, without specialized knowledge being necessary. Activities for simulated robots are set up, reset, and cleaned up with no significant effort after their initial design. Materials for these activities do not need physical storage, and the world the virtual robots inhabit may have a scale or other environmental factors totally unlike the space students have available to them physically. Students only need to be able to use a computer to access a simulated robot, making them more accessible while also enabling new use cases such as students being able to “take a robot home” with the same environment present in the classroom. While physical robots still have some advantages when the barriers to their use are overcome, these traits of simulated robots make them very attractive for educational use. During the COVID-19 pandemic, the advantages of simulated robots became especially clear. For example, the RoboCupJunior competition was able to maintain the same level of participation by moving to a simulation for its 2021 event (Martins et al., 2022).
However, there are also disadvantages to using simulations (Tellez, 2017). Students learn less about hardware as only very specialized simulations include granularity beyond modular sensors and actuators, often eliminating opportunities for mechanical and electrical engineering lessons which would otherwise arise naturally with physical robots. Smoothing over many of the difficulties found in real hardware provides an easier entry into working with robots, but it also deprives students of the opportunity to gain some knowledge only found through real-world experience. Simulations are inherently imperfect models of the physical world, potentially causing code to not run identically between both settings. Simulations also do not necessarily eliminate the need for instructors to be familiar with systems administration topics beyond programming, as fully setting up non-web-based simulators may have additional work required. A notable realistic robotics simulation used in education is Gazebo, arising from the Player Project, a free, open-source robotics simulation platform (Gerkey et al., 2003) providing high-fidelity physics simulation along with simulations of diverse sensors and actuators. However, this level of detail may overwhelm both novices and low-end hardware. Additional educational robotics simulation platforms include Robot Virtual Worlds (Robomatter Inc., 2022), VEX Virtual Robotics (VEX Robotics, 2022), and Robotify (Robotify, 2022). RoboScape Online provides a free, open-source, browser-based simulation tied to a novice-friendly programming interface, combining the strengths of these existing platforms.
Alsoliman (2022) explored the experiences of 8th-grade students and their teachers using a virtual robotics platform across five K–12 schools. The research highlighted that virtual robotics facilitated teaching by overcoming obstacles associated with physical robots, such as high costs and maintenance issues. Both students and teachers reported that virtual robotics enhanced engagement and learning outcomes, suggesting that virtual platforms can effectively substitute for physical robots in certain educational contexts.
Berland and Wilensky (2015) investigated two versions of a robotics learning environment called VBOT: one utilizing physical robots and the other employing virtual simulations. Conducted in four 8th-grade classrooms in Chicago, the research aimed to assess students’ understanding of complex systems and computational thinking. Findings indicated that both physical and virtual robotics environments were effective in promoting learning. However, students interacting with virtual robots developed a more top-down (“aggregate”) perspective, while those using physical robots adopted a more bottom-up (“agent”) perspective.
Additionally, this work includes as a contribution the creation of tools for novice students to create interactive virtual worlds. This concept has gained some popularity in the gaming space with many games focusing on this creative aspect being viewed as potential educational tools, for example in LittleBigPlanet (Ross et al., 2012), Roblox (Jawad & Tout, 2021), and Minecraft (Kuhn, 2018). Our approach differs from these platforms in that it exists within an educational platform evaluated for classroom use, with the new additions being coherent with the normal use of the platform. Code that interacts with the environment does not exist in a different mode or syntax than code commanding robots. RoboScape Online builds upon a set of existing, established curricula for distributed computing, cybersecurity, and robotics.
The block-based language used in this work is NetsBlox (Broll et al., 2017). NetsBlox was created as a fork of Snap! (Snap! team, 2011), but where Snap! aimed to create a block-based language that enabled novices to work with functional programming concepts, NetsBlox in turn adds distributed computing concepts. These new features are similarly novice-friendly through block-based abstractions. NetsBlox programs have access to two blocks for running remote procedure calls (RPCs) from a range of provided services. These services give students access to a wide range of web application programming interfaces (APIs) and datasets which would otherwise require significantly more experience to add to a project. For example, a common first project given to students integrates Google Maps and weather data. Students do not need to know how either API works “under the hood”; they can use a “GoogleMaps/getMap” RPC to obtain a usable image for the coordinates they provide to it, and the “Weather” service makes data for current conditions easily available, as seen in Figure 1. NetsBlox services may also add additional features to the underlying APIs to simplify development, with the Google Maps service providing students with RPCs to find latitude and longitude coordinates from pixel coordinates in the returned image, and vice versa. Students are able to continue to add additional services into this mashup, with a “Geolocation” service providing the names of places and local search data from coordinates. With most languages, creating a mashup of this many web services would require finding and installing libraries for each service, understanding documentation, obtaining API keys, and converting data for compatibility, but through NetsBlox services, students are freed from many of these difficulties. The image data returned are immediately displayable, and care is taken in service design to create consistency between different services, even if the underlying APIs use different formats, further reducing the friction for novices tinkering with new services.
NetsBlox’s other primary distributed computing feature is message passing. While Snap! includes a “broadcast” feature for limited message passing between objects in the same project, NetsBlox goes further by allowing messages with defined message types, specifying a list of fields to transmit over the network between objects possibly in entirely separate projects. Messages can be passed via an addressing system, or to a “role”, a sub-project allowing, for example, both the client and server functionality to be packaged together in one project. A NetsBlox “room” is then a distributed execution environment, a collection of users and their roles brought together, restricting message passing to only this set of users. Many distributed computing architectures can be demonstrated through the message passing feature, enabling novices to explore both client–server and peer-to-peer structures. Additionally, cloud storage of projects with sharing functionality is available with a free account, and real-time collaboration tools are provided to enable teamwork between users. These features are very attractive for classroom use, enabling students to work from any location and share their work with others in the classroom.
The most relevant aspect of NetsBlox to this work is the RoboScape platform created to add robotics support (Lédeczi et al., 2019). A custom firmware for an ActivityBot 360 robot and a NetsBlox service were created. As shown in Figure 2, the ActivityBot is a two-wheeled mobile robot, with a breadboard on its top face to enable adding sensors and other components, along with an optional wireless connectivity daughterboard used to add Wi-Fi functionality to access the NetsBlox servers. Two “whisker” touch sensors are attached to the front of the robot, and an ultrasonic rangefinder is installed. A piezoelectric buzzer and a button have also been added for the RoboScape design of the robot circuitry. RoboScape includes many features significantly differentiating it from other educational robotics platforms. Building off of NetsBlox’s primary focus on distributed computing concepts, RoboScape is presented to students as a web service, with an interface similar to how they would interact with Google Maps. While it is more common for educational robotics platforms to provide a development interface where students write code to run on the robot itself, RoboScape robots are networked devices receiving commands passed over the network from student code. This allows for some additional features to be included in the platform, largely revolving around facilitating cybersecurity lessons after the initial distributed computing topics. The architecture for the platform is shown in Figure 3. Robots are given commands through an RPC, and both received commands and live sensor updates are sent to student programs as events. Students are able to eavesdrop and spoof messages sent to and from the robots using these tools to disrupt other robots during cybersecurity lessons. At the same time, the communications can be set to use basic encryption methods, rate limits, and sequence numbers to protect against attackers. Camps and classes have been run introducing students to NetsBlox, the robots, and a series of cybersecurity attacks and defenses taught through a sequence of competitions. As an additional benefit of the architecture used, the firmware running on the robots can be very simple, with new features added on the NetsBlox server instead, eliminating the need for teachers to update the robots after the initial installation. This allowed the addition of a new encryption mode supporting the Speck cipher without requiring any actions from teachers if they wished to use it with their existing set of robots.
An additional NetsBlox service core to this work’s contribution is IoTScape (Tan et al., 2022). This “metaservice” allows devices to announce their capabilities to the NetsBlox server and immediately make available a new service specific to that device for NetsBlox users. Previously, a new NetsBlox service would require code contributions to the main NetsBlox codebase and a full code review process. With IoTScape, services which would simply redirect requests to a device can be implemented on that device itself, assuming the device is Internet-connected and supports User Datagram Protocol (UDP). In RoboScape Online, IoTScape is used to provide services for new sensors and actuators in addition to the services used to provide an interface for directly interacting with the simulation environment.
Earlier iterations of RoboScape Online have taken various forms, such as a program created in the Unity game engine (Stein & Lédeczi, 2022), and a second implementation running directly in the browser (Stein et al., 2023). Student responses to both versions were positive, with the more recent implementation demonstrating improvements in STEM-related attitudes and computational thinking ability. This project continues to follow a design-based research (Cobb et al., 2003) approach, with the work presented here as the newest iteration of the intervention. Informing this new design are many of the limitations discovered while deploying the previous iterations to classrooms. These include both that restrictive IT policies prevented the Unity-based implementation from being installed or communicating with its servers, and the usability difficulties observed when the simulation is not presented as an integrated part of the development environment. Furthermore, by making the code powering the scenarios visible and tinkerable, the iteration presented in this work enables students to become active participants in the design of the platform and demonstrates the “real-world” usefulness of the code they use in class.
This article is based on Gordon Stein’s PhD thesis written under the supervision of Ákos Lédeczi (Stein, 2024).
2. Materials and Methods
2.1. Simulation Platform
The current iteration of RoboScape Online is a complete rewrite of the previous version (Stein et al., 2023) in the Rust (Matsakis & Klock, 2014) programming language (source code available online:
NetsBlox VM (Jean et al., 2024) is a Rust library reimplementing the program interpretation code from the browser-based interface, with increased performance and the ability to run NetsBlox code on a wider variety of platforms, regardless of whether they support a web browser at all. NetsBlox projects are converted to a bytecode executed in a new virtual machine written in Rust. NetsBlox VM is specifically designed to support microcontrollers and mobile devices, presenting new possibilities for how student programs can interact with the world. While typical NetsBlox code interacting with devices, including for use with RoboScape robots, runs in the browser and acts as a “remote control” directing them as part of a larger distributed system, the VM is intended to subvert this assumption by enabling code to be uploaded and run directly on devices. Work is ongoing to evaluate whether student opinions of this change in paradigm differ from the previous one.
With this new capability, a change was made to RoboScape’s design. Students’ robot control programs would continue running in their web browsers as in previous iterations. However, the code used to describe scenarios would be replaced with NetsBlox code running in the NetsBlox VM on the simulation server. By creating scenarios as NetsBlox projects, the barrier to entry for contributions to the project would be reduced to NetsBlox knowledge, lowering the floor. This could lead to a future paradigm shift in how curricula using the platform are developed, allowing for a form of Participatory Design or co-design where students, educators, and researchers all collaborate on building the activities used in the classroom (Robertson & Simonsen, 2012). The inclusion of student and educator input as part of the design process has been noted to not only create a greater sense of empowerment among stakeholders, but also serves as a mechanism to create culturally-relevant artifacts more effectively (Coenraad et al., 2022). Previously, the codebase for RoboScape Online was open to contributions, but the requirement to not only know how to obtain and run the codebase, create new C# code describing a scenario, and finally push it to a GitHub repository was too significant a barrier. At the same time, the NetsBlox VM retains access to NetsBlox’s cloud features, giving scenarios access to message passing and web services, widening the walls of what is possible in RoboScape Online.
An example of a scenario created using the new tools is shown in Figure 4, with the code used to create it shown in Figure 5. The code displayed runs automatically when the simulator is instantiated with the scenario, with a series of calls to the RoboScapeWorld NetsBlox service (see Section 2.1.3) to create first a floor, then a platform with a LIDAR-equipped robot on it, and finally a set of movable boxes for the robot to interact with.
A move to Rust also allows the platform’s codebase to be created entirely in a single programming language again. During the development process, a library was created to enable the creation of WebAssembly-powered NetsBlox extensions programmed entirely in Rust. Allowing the client to be written in Rust allows the common code to server and client to be more easily shared, and allows both to take advantage of Rust’s memory and type safety features, with significant importance on the server where thread safety becomes very meaningful. Additionally, Rust is noted for its focus on both safety and performance (Bugden & Alahmar, 2022), especially in the embedded domain (Uzlu & Şaykol, 2017). This should improve software quality throughout the platform. Although Rust may be considered significantly more difficult for novice students, enabling students to contribute to the scenarios available using only NetsBlox code is expected to more than make up for it.
Many components which were implemented through C# code in the earlier iteration of the simulation environment no longer require any implementation at all. For example, timers are possible to implement entirely in NetsBlox. Both the simple “timer” block and an additional block providing access to date and time values (including a UNIX timestamp) are available for use in creating time-based interactions, along with a “wait for” block to delay further actions for a specified amount of time. When the environment is started or reset, the project’s code can react to this event by running the “reset timer” block or by storing the current timestamp in a variable.
More advanced functionality is also possible to be implemented using purely NetsBlox. NetsBlox variables can be used to implement state machines for various behaviors, without too much complexity. The NetsBlox interface allows for creating multiple scripts which execute concurrently in an intuitive way, along with communications between modules of the program through “broadcast” messages and message handlers. It is common to see student code where multiple scripts are triggered at the program’s start, handling separate tasks within the program. The simplicity of creating multiple NetsBlox “threads” presents new challenges given students’ limited understanding of synchronization, but these double as educational opportunities. NetsBlox is currently used to teach artificial intelligence and machine learning in the CS Frontiers curriculum, demonstrating the high ceiling of NetsBlox projects. Additionally, commands sent to robots are relayed to NetsBlox projects after using the “listen” RPC in the RoboScape service, allowing scenarios to use the same mechanism to intercept and react to robot commands as in projects for interacting with the robots. Students in RoboScape camps and classes have often requested the ability to write code to run directly on the robots, and although that would still not be available, running RoboScape commands on the server creates a very similar capability as an alternative.
2.1.1. Architecture
The infrastructure of RoboScape Online exists as two distinct parts. The first is a main “API server” used to handle user requests to join or create simulation instances, known as “rooms”. The second part is the set of servers capable of running simulation instances. Each simulation server announces its capacity to the main server at a regular interval, along with updating it on the status of its simulations. Students running the client in their browser will make a request to the API server to initiate their session. The main server will repeat this request to the simulation server with the least load if a new room has been requested. If the server requested to create a new room is unresponsive, it is forgotten, and the request will be repeated with the new known server with the least load. The main server then returns the address of the server and the ID of the room to the client. The client initiates a WebSocket connection to the simulation server. This connection is used for bidirectional communication, including both updates of the simulation state from the server, and requests to reset, claim, or encrypt a robot from the client.
Significantly, the main NetsBlox server is not a component belonging to RoboScape Online, by design. No modifications specific to the operation of RoboScape Online have been made to the NetsBlox codebase. The existing RoboScape server is unaware if a robot is running in a virtual or physical reality. In both cases, the robots communicate identically with the RoboScape service on the NetsBlox server through the same UDP messages. The client communicates with the NetsBlox server using the same RoboScape RPCs used with a physical robot. RoboScape Online additionally makes significant use of the IoTScape NetsBlox service (Tan et al., 2022), which is similarly agnostic toward whether its devices are physical or virtual in nature. These design choices allow for greater compatibility between the existing and new platforms, leaving open the potential for introducing some of the virtual sensors’ and actuators’ real world equivalents into later curricula.
A diagram of communications between the components of RoboScape Online is shown in Figure 6. Communications between the student computers and any other system are performed over Transmission Control Protocol (TCP), with HTTP upgraded to WebSocket for the simulation server connection. Communications between simulation servers and the NetsBlox server utilize UDP instead. The existing RoboScape protocol requires UDP, and IoTScape was designed to use UDP to mirror it. The connection between the API server and simulation server are HTTP over TCP, with the intent that communication with a client and the API be transparent with communicating with any individual server. The API server effectively acts as a load balancer between the clients and simulation servers. Note that the three servers are not inherently on separate virtual or physical machines, although the simulation servers in implementation are distributed across a set of multiple cloud instances to allow improved performance through horizontal scaling. As new simulation servers are created or old servers are removed, the API server will adapt to connect users to active servers.
2.1.2. NetsBlox Extension
The client-side software component of RoboScape Online exists in the form of a NetsBlox extension. This extension must not only facilitate the communications between the client and the server-side components of RoboScape Online, but also display a three-dimensional graphical view of the simulation and provide a usable interface to interact with it from within NetsBlox. At the same time, this interface should exist without unnecessarily obstructing the users’ ability to continue working with their NetsBlox code. Moving the client from an external application to the browser was initially motivated by reducing the need to switch back and forth between two windows or applications, so it would be counterproductive if code and visualization cannot be used simultaneously. However, the graphical view is not only impressive visually but important to the operation of robots within RoboScape Online, so it must be possible to resize the viewport to give it full focus when desirable. While it is tempting to replace the “stage” used by NetsBlox sprites with a three-dimensional viewport, this removes some functionality of NetsBlox. Instead, it was decided to create a dialog window within the existing interface with the viewport contained within it.
The newest client is also written in Rust. This allows for code reuse, through a library of common structures shared between the client and server. Shared code for serialization and deserialization prevents many common bugs observed during the migration of NetsBlox itself to a Rust-based server. This will not only prevent deserialization issues, but will also make the platform as a whole easier to maintain. The client-side Rust code is compiled into WebAssembly to be loaded by the browser. A library for creating Rust-based NetsBlox extensions was also created for this project, automatically generating a JavaScript-based extension to load and call the compiled WebAssembly binary based on annotations added to the Rust source code. Extension menu items, blocks, and settings are all exposed to be able to interact with the Rust code. Unfortunately, due to the necessity of UDP communications for the robots and IoTScape devices, it is currently not possible to also run the simulation code as WebAssembly in the browser.
Existing as a NetsBlox extension also provides the opportunity to include new blocks. Blocks provided as part of an extension can run the JavaScript code provided by the extension to interact with its features from NetsBlox code. These blocks (shown in Figure 7) appear to the user and behave as if they were built-in blocks, not a fundamentally different concept. RoboScape Online’s extension provides blocks to obtain a list of robots in the current room, and to obtain the ID of the current room. Both of these greatly simplify the ergonomics of working with RoboScape Online. Previously, students would input robots’ IDs manually, often leading to confusion if they were assigned a different robot later. With the extension block, a student can use the “robots in room” block to obtain a list of robots, and combine that block with the built-in list indexing block to obtain a specific robot’s ID from the current environment. When running their code with a different robot, it will automatically obtain the new robot’s ID. The room ID provides a similar simplification when working with custom scenarios, allowing the code to use the correct ID regardless of what room it is running in. It would also have been possible to provide this functionality through additional RPC methods; however, extension blocks have less latency in their output by not requiring an external web request. This reduction in latency creates a more novice-friendly interface, as students are prone to forgetting to cache similar values in their code.
2.1.3. Scenario Creation Tools
Users interact with the simulation server through a set of NetsBlox service RPCs. As all scenarios are NetsBlox projects using the same set of RPCs as student projects, it is necessary that these services encompass the entire set of available features. At the same time, consideration must be made for maintaining a low floor in the design of these services so that they do not exclude students from their use unnecessarily.
The main service used to interact with scenario creation is “RoboScapeWorld”. The RoboScapeWorld service allows for determining users present in a room, listening for events involving them, and spawning new objects in the simulation. An often underappreciated part of designing an interface, especially in the context of expecting novices to be able to reason intuitively about it, is the names given to concepts within it. Names have to be simple enough to easily convey what something is, without being so abstract that they lose the nuance of the specific implementation. The design of the services and methods available to students required deliberation on what terminology would be preferable. We chose the name “World” for the service controlling the overarching environment of one simulation. This terminology is simpler than some alternatives such as “Environment” and avoids confusion with existing phrasing such as NetsBlox’s pre-existing “rooms”.
RoboScapeWorld features RPCs for adding general entities and IoTScape services, with specialized methods for both cuboids and robots, setting and clearing text overlays, listing the entities in a room and creating entities based on this output, listing the textures and meshes available for use, removing entities, and resetting the environment.
An example of NetsBlox code using this service is shown in Figure 8a. In this example, first, a call to “removeAllEntities” clears the environment. This is unnecessary at the start of published projects as the environment will start empty, but in a project under development, this allows a fresh start each time the code is run. Next, a call to “addBlock” is used to create a static floor 100 units in width and depth with its top surface at the position 0 on the y axis. The call to “addRobot” adds a robot to the environment, slightly above the origin. The last object in the environment is added through the “addEntity” method. This is a more general method for adding objects. Both “addBlock” and “addRobot” are simplifications of “addEntity” for the most common types of entity with their most used configurations exposed. With the entities added to the environment, the code then requests a list of entities with “listEntities”. The output of this method is a table with names, types, positions, and other relevant information, to enable a form of serialization. An “instantiateEntities” method is available to take this structured output as its input to recreate an environment saved in this format. Finally, the program uses the “showText” method to display the number of entities created in the viewport. The result of this program’s execution is shown in Figure 8b.
For individual objects in the simulation, the functionality shared by these objects for controlling physics behavior and transforms is accessed through a single service. This prevents a large number of overlapping services for subtypes of objects which may not require anything beyond this functionality. For example, the ability to set the location of both a block and a robot can be in this shared service, instead of added to two separate services. The name for the service had to be general enough to represent many types of objects while hopefully not being misleading about what interactions would be available as a part of the service. The physics engine inspired the term “RigidBody”, but this was considered unfamiliar to novices. While many objects created for an environment are likely to be rectangular prisms, names like “Cube” or “Block” are too specific or misleading. “Object” was decided to be too vague, as was the name “Part”, as used in Roblox, even if that prior usage may be familiar to students. “Entity” was the final choice for the name of this service, as a term vague enough to represent arbitrary objects within the simulation while also having existing use in video game development.
Most robot-related functionality continues to be made available through the existing RoboScape service, with some additional capabilities inherent in the Entity service. While it would be possible to provide a feature-complete internal reimplementation of the RoboScape service running on the main NetsBlox server, this creates a large duplication of existing code without a compelling use case. The cybersecurity features of the RoboScape service only make sense in a context where the messages can interact with the students’ projects, making things such as encryption not meaningful in the context of the current structure for custom environments. Some limited features may be added in future versions, such as the ability to receive a robot’s simulated “hardware” encryption key through code, giving new functionality to environments while remaining compatible with the existing RoboScape service.
An example of NetsBlox code using the RoboScapeEntity service is shown in Figure 9. In this example, the RoboScapeWorld service is used to create two spheres. While in the previous example, all RPCs were using the “run” block where outputs are discarded, the methods of the RoboScapeWorld service often provide meaningful results. In the case of “addEntity” and related methods, the output is the unique ID of the entity within the room. This can then be input to the “addSensor” method to obtain access to the other RoboScape Online services. To access RoboScapeEntity, a sensor of the “entity” type must be added to an entity in the current environment. This service gives direct access to the transforms of the entity, in this case used to animate the blue sphere orbiting the yellow sphere by setting its position using the “setPosition” method.
Multiple forms of sensor are exposed as IoTScape services. In general, these sensors are intended to be used as part of the task associated with the scenario, where the user makes queries to them as part of an autonomous robotics task. However, they also serve potential uses for the projects used to run custom scenarios as well.
The most commonly used sensor is a “GPS” in the PositionSensor service. This service provides access to the sensor’s position and heading rotation. Many scenarios use this device attached to a robot, but it may be attached to any rigid body in the simulation, giving user code the ability to query any object in the space. Additionally, a “WaypointList” service gives the ability for users to query the next waypoint in a list of waypoints. This functionality could be implemented in several ways, such as using NetsBlox’s existing “CloudVariables” service to store the waypoint, or by passing messages to the user’s program. The implementation of waypoints as an IoTScape service was motivated by maintaining a common interface with the waypoint scenario used in the previous iteration of RoboScape Online, where web services and message passing were not available to the scenario code.
Another sensor commonly used attached to a robot is the “LIDAR” service, simulating a laser imaging, detection, and ranging sensor. In the physical world, this sensor would emit laser beams in multiple directions and ascertain distance to the environment by measuring the time taken for a reflection to return to the sensor. In the simulation, a simple raycasting feature is used to emulate this. Specifically, the simulation is of a “2D LIDAR”, which scans on a single plane, although by restricting it to a single beam, a “1D LIDAR” may also be simulated. The LIDAR sensor in the simulation supports multiple configurations, with varying numbers of beams and angular ranges. Figure 10 shows a NetsBlox program where the LIDAR ranges of a robot’s environment are drawn as a polygon. In this scenario, the robot’s LIDAR has been configured to have a 180-degree range with 11 beams. This specific example has not been used in classrooms yet, due to the available students’ lack of experience with trigonometry.
The ProximitySensor service is used to return a distance to a given point to the NetsBlox program. Figure 11 shows the “Treasure Hunt” scenario, where the student uses the ProximitySensor service as a metal detector to find a hidden buried treasure. This sensor reports the distance between the body it is attached to and a target point. Note that instead of reporting only the raw distance, the response is a configurable linear function of the distance, to allow for multiple use cases. For example, the Treasure Hunt scenario is set up so that the output will be a value of 100 when the robot is on top of the target. The location of the treasure is randomized between runs of the scenario.
Typical RoboScape Online scenarios are likely to only use the RPCs provided to interact directly with the platform, to set up the components of the scenario and detect the relevant conditions for completion or failure of the challenge. However, by taking advantage of additional features of NetsBlox, some notable benefits can be obtained. Even though not all NetsBlox features are available in the VM execution environment, the vast majority of features are available for use and contribute significantly to widening the walls and raising the ceiling for custom scenario development.
One such NetsBlox feature is message passing. Scenario projects are able to receive messages like any other NetsBlox project. These messages can be used to introduce a set of customizable commands to the scenario. A simple example project using message passing is shown in Figure 12a. In the scenario created by this code, a ball is held in place by a block. When a “drop” message is passed to the project (see Figure 12b), the block is removed and the ball is able to fall (see Figure 12c,d). The ball will then follow a path through a set of obstacles added to the structure, similar to a pachinko or Plinko game.
One difficulty of message passing is the need to obtain the VM’s “public role ID” to send a message to it. Message passing between NetsBlox projects requires either users being assigned to two “roles” of the same project, or to have each others’ public role IDs to use as addresses. The example project communicates this role ID by assigning a NetsBlox cloud variable to have the ID as a value. The client code can then retrieve the ID from this variable in order to know the address to send its message to. At the same time, this paradigm could cause issues in a setting with multiple users unless the scenario is modified to store a mapping of RoboScape Online room IDs to NetsBlox public role IDs. Additionally, user-specific cloud variables cannot be used; only public cloud variables can be used. This creates a need for unique identifiers for each scenario to prevent a collision between two users’ scenarios.
While this interaction could also have been triggered by making a robot drive to a certain location, that greatly limits the number and variety of potential interactions. This example only reveals a small fraction of the potential full use of NetsBlox’s distributed computing features unlocks. Messages could also be sent between two RoboScape Online scenarios, creating new collaborative challenges where robots interacting with one environment have some influence over a different scenario. RPCs could be used to create robotics scenarios that interact with phone apps or change based on data obtained from the cloud. Furthermore, RoboScape Online, despite the name, is not in any way restricted to only robotics projects. NetsBlox services expose data on climate change, wildlife tracking, hurricanes, and many more domains where a student may be inspired to create a novel three-dimensional visualization through a scenario project.
An example of using existing NetsBlox services for a novel RoboScape scenario is shown in Figure 13. The current weather in Nashville changes the appearance of the environment, with clouds being added when they are present in a real life location. The same “Weather” NetsBlox service is often used as part of an introduction lesson to programming in NetsBlox, so it is natural that a student may want to combine their earlier knowledge with their new scenario-creation ability. The only difficulty a student would be likely to face in creating a project similar to this is that they are used to having “my latitude” and “my longitude” blocks available, which no longer make sense in the context of running on a remote server.
Debugging in RoboScape Online scenarios can be difficult compared to more conventional NetsBlox programs. When working in the NetsBlox browser interface, a user is able to not only add text-based output but they may also directly observe the values of variables and use a “visual stepping” mode to slow program execution and highlight blocks as they are executed. However, the intended final execution environment of a RoboScape Online scenario program is the NetsBlox VM running on a remote server, where these features are not available. Textual output can be added through the “showText” RPC, but using it to provide debugging information naïvely would require modifying the program to add method calls to it and then again to remove them when debugging is finished. A clever solution to this is to use the NetsBlox “CloudVariables” service, either to set a flag to enable additional code or to store logging information. Additional debugging assistance is an area for continued exploration.
The RPCs provide an interface for interacting with all available scenario-creation features; however, due to the large range of possibilities needed to be expressible through these RPCs, they often have many optional inputs, often expecting structured data. One option to address this potential for confusion would be to create a wider range of methods available with various numbers of inputs for different use cases. However, this may continue to confuse novices, as it may move their choice paralysis from the inputs of a single method call to deciding which method to use out of a larger set. A smaller set of methods available with many optional inputs is also possible, but similarly risks creating confusion. The methods provided by the service additionally allow for data structures as inputs to some fields, for example a list of three numbers for rotation, or a dictionary for setting multiple options at once. Even without the use of these complex types, the interface in NetsBlox for RPC blocks does not show names of fields once they have input, resulting in students becoming unsure of which field is which. While some of these issues are due to design decisions in the NetsBlox platform, improvements are still possible within the scope of the RoboScape Online project.
The approach taken to create an interface appropriate for novices was to create a library of custom blocks with simpler interfaces for less advanced students to pick from. NetsBlox includes a feature for sharing sets of custom blocks as libraries, available to other users. This allows for a library to be provided abstracting the RPCs’ interfaces into simpler forms for a set of likely use cases. At the same time, custom blocks can provide some “quality of life” features for students. The most obvious of these is the ability to add labels to each input field, but blocks could also be provided to assist students in creating three-dimensional rotations or hexadecimal colors more intuitively. A selection of blocks from the provided library are shown in Figure 14.
An example of this library’s use is showcased in Figure 15. While the non-library code shown in Figure 16 is not notably complex, the custom blocks still serve to make simpler, more readable code. Additionally, they assist students in changing the visual appearance of the environment. When creating the scenario using the native RPC blocks, the range for the structured inputs for “visualInfo” or “options” are not readily apparent. With the custom block, a student can see the options for the visual appearance of a block, with the drop-down menu indicating them as colors, textures, or meshes.
A series of written tutorials are being created to help assist students with learning how to create their own custom scenarios. The first of these, “Hello, Custom Scenarios”,1 serves as an introduction for users with no experience with the creation tools of the platform. Figure 16 shows the final code and outcome of this tutorial, giving students a simple environment with a robot, a floor, and a cube that they can push with the robot. This tutorial also introduces students to the mechanism for distributing their work through public NetsBlox projects. The end of the lesson has students set up their work for sharing and load it through the project name. Additionally, a second version of this tutorial2 was created using the custom block library to provide a simplified introduction. Another tutorial3 creates a “Four color” scenario, where four robots each start on a platform, with ramps so they can get off of and back onto the platforms. A video version of the introductory tutorial was also created.4 New tutorials will continue to be created as the program continues to be used. Additionally, materials for the non-scenario creation aspects of RoboScape Online, including the cybersecurity features, have been made previously. As the simulated robots were designed to work identically to the physical robots, materials created for the physical RoboScape robots are in general compatible with the virtual robots as well.
A range of scenarios have been created both recreating activities previously performed with physical robots and expanding the domain of scenarios to new possibilities. The following includes a non-exhaustive selection of scenarios created for the new platform.
The “Obstacle Course” scenario is shown in Figure 17. In this environment, the student is expected to manually control a robot to navigate from the start area (marked in red) to the end area (marked in green). Between these two areas are a series of barriers with gaps in them, and a movable block which must be moved in order for the robot to progress. To allow for competition, there is a timer included in this scenario (shown as text at the top of the display), which activates when the robot leaves its starting area and stops when it reaches the end area. This requires the creation of a simple state machine to manage the value displayed to the user as the time.
A scenario used in the classroom that students found particularly interesting is the “LIDAR Road”, shown in Figure 18. Students are tasked with creating a program to autonomously navigate to the end of the path using a LIDAR sensor onboard the robot. The robot’s LIDAR, with the configuration set in the scenario project, will project three “lasers” into its environment. One beam will go 60 degrees to the right, another straight ahead, and a third 60 degrees to the left. An example student solution to this challenge is shown in Figure 19. Although most students completed the task using the expected solution of a series of conditional statements to determine the current state of the robot based on its sensor readings, this student requested an additional level of difficulty. He was given the extra constraint of having to minimize the number of blocks in his program, resulting in his discovery that a program can be made feeding the LIDAR value from each side into the motor speed for the opposite side.
In the physical camps, a game of “tug of war” was played where two teams share a single robot and try to keep it on one side of the game space. Specifically, this game was used for cybersecurity lessons, where students would each attempt to use the cyber-attack and defense techniques they had learned to maintain control of the robot, while attempting to lock the opposing team out of its control. The simulated version of this activity is shown in Figure 20. The ground has two colored areas, and when the robot is in one of the areas, the text on-screen will update to show which team is currently winning. A timer counts down until the end of the competition, when a winner is crowned. By running this activity in virtual form, students can feel that the initial placement of the robot and the scoring are more objective and equitable between the teams.
2.2. Classroom Study
A class using RoboScape Online was implemented at the School for Science and Math at Vanderbilt (SSMV) (Eeds et al., 2014). SSMV is a collaboration between Vanderbilt University and the Metropolitan Nashville Public Schools district, providing high-school-age students with courses focusing on interdisciplinary science topics and opportunities for participation in research. Participating students attend the program on the Vanderbilt campus one day per week, and their normal high school courses on the other days. An earlier study on a previous iteration of RoboScape Online was run as an SSMV module in Spring 2022 (Stein et al., 2023). This earlier course found that RoboScape Online both improved students’ computational thinking ability and multiple STEM and CS attitudes.
2.2.1. Class Structure and Curriculum
The SSMV class met every Wednesday from the end of September to the end of November, with two weeks off for breaks, for a total of eight sessions during the semester. Each session was two hours long. Small assignments were given to students to complete in between class sessions.
The course topics for each session are listed below:
Introduction to NetsBlox— Students were walked through creating basic NetsBlox programs and a weather app project combining data from multiple web services.
Lists and Networking in NetsBlox—Students were led through projects using additional web services, with a focus on services making use of lists as outputs, along with a basic multiplayer game.
Introduction to RoboScape—Students created a manual driving program to control their robot and used their programs to compete in an obstacle course race.
Introduction to Autonomy—Students completed tasks using dead reckoning and a distance sensor.
Sensors—Two tasks were given requiring the use of a simulated LIDAR.
Sensors Continued and Introduction to Custom Environments—Students made a robot navigate using a simulated GPS and created their first custom environment.
Start Final Project—Additional tutorials were given for working with custom environments and the final project was explained to the class.
Final Project Presentation—Students were given time to complete their environments and then present their projects to the class.
This study is part of a design-based research intervention aimed at exploring the impacts of a virtual robotics environment on students’ learning experiences and attitudes. The goal is not necessarily to produce generalizable results, but to gain insights that can inform the design of future iterations of this or similar interventions in future settings. The group of students who interacted with this intervention as part of their coursework represents a “classroom case study” where their experiences with the platform were examined to enable this iterative improvement through a mixed-method methodology combining quantitative and qualitative methods.
2.2.2. Student Demographics
The 26 students in the SSMV class were high school sophomores. A total of 10 students identified as male, 15 identified as female, and 1 did not wish to identify their gender. A total of 10 students identified as non-Hispanic white, 8 as black, 2 as Asian/Pacific Islander, 2 as South Asian/Indian, 1 as Hispanic/Latino, and the remaining 3 students identified as bi-/multi-racial. SSMV students are selected from incoming 9th grade students attending any Metropolitan Nashville Public Schools district high school. All students with prior curiosity or interest regarding STEM topics are encouraged to apply, with support provided to facilitate the students’ participation in both SSMV and their regular high school classes, including assistance with transportation. No explicit selection of the students was made beyond the SSMV application process, which was independent of this study. Section 5 describes potential biases introduced through the population of students who participated in this study.
2.2.3. Evaluation Methods
The evaluation of the course focused on student opinions of the software and computer science attitudes. An anonymous survey was given to students both before and after the course. Survey data were collected and managed using REDCap electronic data capture tools hosted at Vanderbilt University (Harris et al., 2019, 2009). Questions included in this survey were adapted from the Electronic Textiles Student Survey presented by Kafai et al. (2019), with modifications replacing questions specific to the e-textiles module with robotics-related questions. The post-survey includes additional open-ended questions inquiring about the students’ favorite activities, biggest challenges, if they would recommend the course to others, and what they would tell a friend about their experience. Opinions were collected using a six-point Likert scale, ranging from a score of 1 indicating “Strongly Disagree” to a score of 6 indicating “Strongly Agree”. The choice of six options was motivated by the results found by Simms et al. (2019), indicating it as the optimal number of options for a Likert scale in a survey.
Additionally, semi-structured interviews were conducted with four students who volunteered to provide opinions beyond what was directly requested by the survey. This interview methodology provides additional depth that the survey’s quantitative data alone would be unable to provide (Adams, 2015), able to help inform the continued iteration of the intervention regardless of overall generalizability. The use of convenience sampling is applicable, but may limit the generalization beyond the students participating in this class (Robinson, 2014). No inclusion or exclusion criteria were applied outside of including all students who volunteered and were present to be interviewed. These interviews took place directly after the activities on the final day of class. Although the interviews were conducted by individuals with a connection to the project, students were assured that their responses would be anonymous and all feedback was appreciated. No minimum or maximum duration was set for the interviews; the interviewers only aimed to restrict them to under 15 min to allow all students to be interviewed during the allotted class time. Interviews were recorded and transcribed automatically, with the recordings destroyed after the transcriptions were verified, for privacy reasons. The questions used to initiate the interview protocol are listed in Table 1. Students were asked about their favorite activity, how they would extend that activity if given more time, their feelings toward the virtual robotics environment, their specific likes and dislikes regarding the environment, their feelings about creating their own environments, and how they would change the process to make it more intuitive. Additional questions were added by the interviewers to probe deeper into the thoughts and opinions of the students, based on the responses to these questions.
To analyze the qualitative data from student interviews, open coding (Strauss & Corbin, 1990) was performed, employing a combination of Descriptive Coding and In Vivo Coding. These two methods are considered applicable for “virtually all qualitative studies” (Saldaña, 2021), with In Vivo Coding noted as being especially useful for allowing the perspectives of young participants to be authentically preserved in the analysis. Given the limited dataset and the exploratory nature of this study, these straightforward methods allow for a clear understanding of participants’ perspectives. Given the limited number of interviews, the data were not suitable for a theory-generating methodology such as grounded theory. The purpose of the interviews in this work was for understanding individual experiences to evaluate the intervention and inform future iterations, rather than generating new theoretical frameworks. The selected coding strategies are suited to this objective, providing an effective approach to data analysis at the available scale.
Previous case studies (such as Stein et al. (2023)) had included programming assessments presented to students before the start of and after the completion of the course; however, this was deemed unnecessary in this iteration. The results from the 2022 SSMV class case study have already demonstrated the capability for educational robotics, through a curriculum based on RoboScape Online, to improve students’ computational thinking skills. The additional capabilities of the platform made available during the new study focused on more creative tasks which would be difficult to create an objective evaluation for. Including these tests would also require a significant amount of the already limited class time, which researchers could instead use to provide more educational content to the students.
3. Results
Overall, students expressed that they enjoyed the course. While a few technical hiccups occurred during the classroom activities, they largely remained on-task and productive, often choosing to go beyond the tasks provided to them.
3.1. Survey Results
All 26 students in the class responded to the pre-survey, with 21 (81%) students submitting the post-survey. While fewer students responded to the post-survey, the demographics of the post-survey responses did not appear to show a significant bias in gender or ethnicity in which students chose not to complete the post-test.
To account for the missing responses, an imputation method must be used. To avoid a false positive result, the missing data are assumed to be “missing not at random” (Enders, 2023), meaning that the probability that the data are missing is a function of the missing values. This assumption is based on the likelihood that the missing responses belong to a less motivated portion of the class, who may not have agreed with the very positive responses of the students who successfully completed the survey. While it is not possible to know the true responses these students would have given, beginning with this assumption prevents this potential selection bias from causing an undue effect due to being attributed to the intervention. However, the assumption that the response rate is correlated with the response value creates a significant barrier to creating an accurate estimation without a model of the missing data (Cheema, 2014).
As a simplified form of this, we have elected to examine the distribution of effect sizes and significance across potential sets of dropouts across the available data, reporting the statistics for the worst-case among these possibilities. The assumption this analysis operates under is that the students who did not respond to the post-survey did not have a positive change after the intervention and would respond similarly to the matched questions in the pre-survey. Therefore, the imputed values will be the student’s pre-survey values. While this will not compute the exact test statistics, it provides a conservative upper bound for the effect observed in our case study as retaining the pre-survey values would likely support the null hypothesis. Of the 26 students, 5 did not complete the post-survey. Naively, this results in = 65,780 unique possible sets of students who completed the pre-survey but not the post-survey. However, by utilizing the demographic data attached to the pre- and post-survey results, it is possible to reduce the number of possible combinations dramatically. Two of the students were unique among the student demographics and could be directly matched to their pre-survey results. Of the other three, they were spread across two subgroups with four members each, resulting in a final total of = 24 possibilities. The reported p-values and effect sizes were computed using the worst-case scenarios among these potential combinations.
Pre- and post- responses were compared using Welch’s unequal variances t-test (Welch, 1947) to determine the level of significance. This test was chosen due to the unpaired surveys. Responses which were determined to be statistically significant based on a threshold of are listed in Table 2. Cohen’s d was used to evaluate effect sizes for statistically significant questions. Effect sizes greater than are “small”, greater than are “medium”, and greater than are “large” (Sawilowsky, 2009). All but one of the questions in Table 2 was found to have a medium effect size, with the question “I think I am very good at coming up with new ideas when working on projects.” showing a “small” effect size of at least . Both questions regarding creativity and the question regarding understanding robot programming showed the most significant improvements from the pre-test, with effect sizes greater than .
Notably, the pre-test means for every opinion except for one were all slightly positive. The remaining opinion was “I can explain how robots make decisions.”, which was exactly neutral (mean of 3.5). Although not rising to the level of statistical significance, a pair of questions saw slight decreases in attitudes after the intervention: “Autonomous vehicles are interesting to me.” and “I think I am very good at solving problems.” Additional studies will be required to ascertain if these changes are due to chance and, if they were not, if they were side-effects of the intervention. At the same time, the opinions for “Robots are interesting to me.” showed no significant change after the course, remaining at 4.7. The survey questions with non-significant pre–post differences are listed in Table 3.
Additionally, an analysis was performed to ascertain any effect of gender on the students’ survey responses. However, the differences between the gender groups in the class were not significant for almost all survey questions on both the pre- and post-surveys. The only significant gender difference in opinions before or after the module was “I talk about how things work with friends or family”, which the 10 students identifying as male scored as 3.800 before the module and 4.100 after, and the 15 students identifying as female scored as 5.067 before and 5.000 after, with the intra-group changes not being statistically significant.
Students’ responses to the question about their most liked activity are tabulated in Table 4. One instance of double counting is present where a student referred to liking both the custom environments and the LIDAR activities equally. The students who preferred the custom environments enjoyed the increased freedom: “because we could creatively design things and had the reign to do whatever we wanted”, “I loved the creative freedom we were given and the loose restrictions that were enforced”, “It was fun to learn everything we could add and build whatever we want”. The LIDAR activities were appreciated for their fun in completing the task: “I was able to design my own autonomous robot and I was able to feel the wonderful moment when everything clicked into place”, “It gave me a sense of accomplishment”. With the Obstacle Course, students explained their interest as being from the competitive aspects, “because we [were] able to play against each other”, “I thought that it was really fun to watch our classmates compete against each other”.
The responses to the open-ended question about the most significant challenges faced by students are summarized in Table 5. A much greater diversity of responses was seen for this question, with the most common response being that technical issues experienced with the NetsBlox platform were the biggest barrier to their progress.
The final open-ended question on the post-survey, asking both if the students would recommend the course and what the students would tell a friend about it, had a nearly unanimous (95%) positive response. Twelve of the students (54%) responded that they would give an unconditional recommendation for the course. The remaining students almost all recommended the course, but with caveats for whom they would recommend it to, such as “I would recommend this to some of my friends who are interested in computer science” or “I would only recommend it to them if I knew they liked this kind of challenge”. One student responded that “Personally none of my friends would enjoy the class”, but did not elaborate further on why they would not.
3.2. Interviews
Four students volunteered to be interviewed on the last day of class. While the exact number is small, these students demonstrate a significant amount of diversity within themselves. There was an even split in gender, and the skill level of the students ranged from one complete novice to one “power user”, with two average level students in the middle. The codes obtained through Descriptive Coding are listed in Table 6.
3.2.1. Student 1
The first student interviewed was one of the moderate-level users. He stated that the final project was his favorite part of the class, due to the creative freedom allowed for it and the opportunity to interact with other students more. Increased options for collaboration were a feature that he would like to see in future iterations of the intervention.
When asked about his difficulties in using the platform, he commented on NetsBlox bugs, but also that the simulation content “was complicated, but it was ‘easy complicated’”. This phrase was elaborated to mean that the concepts seemed difficult at first, but after getting a bit of help, they were made clear to him. He found the RPCs for custom environments easy to work with because of the initial demonstration and that “there weren’t too many choices to choose out of”.
The structure of the course was appreciated by Student 1: “The difficulty wasn’t hard, but it wasn’t easy either. It was like medium because it taught you how to learn things, but it wasn’t like it didn’t force things on you. Like it wasn’t too hard, but it wasn’t too easy and it allowed you to learn. But you could get help and stuff. So it wasn’t easy, but it wasn’t hard. So it was kind of a good level”.
3.2.2. Student 2
The second interviewee was the most experienced student of the group. For his final project, he elected to create a simplified version of the voxel terrain rendering from the game Minecraft. While he put significant effort into the final project, he stated that the LIDAR block-pushing environment was his favorite activity. This was because “that was something I’d never actually done before so it’s kind of like a new way to think about how self-driving cars work and how easy it is to make autonomous things”.
Student 2 was generally positive about the course’s use of NetsBlox as a programming interface. While some students may feel a block-based language is limiting, he thought of this as a benefit for education: “you have to really think ahead on what you’re going to do and it makes it a fun challenge and it really improves your science skills for sure”. For the final project specifically, the game-esque project he chose led to some new insights into his computer science activities: “I have a new perspective on how some video games work and I think the limitations that NetsBlox provides actually help me to understand how to better make those kinds of things”. He took on a very difficult final project because he had experience with a block-based language before: “I’ve made some crazy things in Scratch”. Student 2 showed an interest in additional content focusing on NetsBlox itself. Even though “the earlier stuff I kind of found a little boring because I’ve done the coding before” and “that first week we had the introduction to like the Google Weather we were able to use that integration and that was something that was really cool, I feel like we didn’t get to explore enough”.
However, in the sensor-based activities, he reported having issues with the limited capabilities of the robot: “One challenge was that there was a kind of lack of precision with the robot driving mechanism that made it difficult to do precise calculations, which was a little annoying, but I think it was fun overall trying to think through those challenges”.
3.2.3. Student 3
The third student to be interviewed had a moderate level of experience. She considered the final project to be her favorite activity, although she had experienced technical issues trying to work from home that tarnished her view of it. If she had more time, she proposed she would have created a game where a group of robots play a “defend the treasure” tug-of-war game in a castle setting.
Of the robotics activities specifically, she stated the LIDAR Road was her favorite, “because it’s my first time actually trying to use sensors”. She had some robotics experience before, but with physical robots. This presented a very different experience from her perspective: “instead of being able to look through it, the bot itself, it’s not anything with the bot, it’s always something with the code”. When asked to elaborate on the comparison between physical and virtual robots further, she explained that there were some quirks of the simulation that she viewed as distinguishing the two. She saw the use of simulated robots as useful for students working with physical robots as a way to explore and evaluate new ideas without the cost of modifying a physical robot. The robotics team she was involved with previously always was required to reuse the previous year’s robot as a base due to costs. With simulation, she observed that “instead of like wasting a bunch of materials to see if it works, then you could make it better and make a robot that you really like every year instead of just making modifications to the last one because then maybe it’ll take less time, less money to like fix everything because you can debug it online and then produce it”.
She found the custom environments easy to work with in code: “I liked that it was easy to put together and it was easy to see where you went wrong because even though I had this big block of code at the beginning, it was easier to make it into something small where I could make really quick changes to it”. She also found them enjoyable: “I don’t consider myself a coder or like into coding, but this was really, really fun”. The process of working with it specifically was what she found fun: “It’s something that I actually got to figure out as a challenge and you actually got to see that like go into fruition with the virtual environment because you could see it happening in real time. So that was really fun”. But she had technical issues with the NetsBlox interface, especially when she tried to work using an iPad to edit the code.
For improvements to the platform, she requested some interface improvements. For the custom blocks library, she wanted more verbosity in labeling inputs: “But when you have the create blocks and there’s three numbers, you don’t know which number is height, width, or size within the create blocks. So maybe if you put height, width, depth in front of those because you’ve done it for some of them but you haven’t done it for others”. She was also uncertain of which directions were X, Y, and Z in the simulation, possibly after rotation, and wanted a way to view them in the 3D viewport.
3.2.4. Student 4
The final interviewee was the newest to computer science. Her perspective was very different from the other three students, providing insight into what a novice would have the most difficulty with and find the most fun.
Her favorite activity in the class was not one of the RoboScape Online challenges. Instead, she preferred the networking projects where they created a weather app and a simple multiplayer game. The reasoning she gave was “Because I use text messages every day and it’s fun to kind of see. And I use other stuff like that, like email, and it’s just kind of fun to see how you can make it work like what goes into making that stuff quick”. For her, the projects relating to technology she uses in everyday life are more interesting than activities focusing on robots.
Regarding difficulty, she wanted additional introductory material and documentation to use. For example, she was unfamiliar with the terminology “mesh” to refer to a three-dimensional model, and did not know how to figure it out without asking for help. Although unexpected for a young novice to programming, she stated a preference for this additional content to be provided as written material rather than an option of video content. She described her main method of debugging or overcoming frustration to be asking for assistance. However, the syntax in NetsBlox itself was beneficial to her experience: “the blocks made it more self-explanatory because you could figure it out by yourself without, you know, knowing code”.
4. Discussion
All but one of the effect sizes noted for statistically significant survey responses can be described as between “medium” () and “large” (), with the remaining one as “small” () (Sawilowsky, 2009). Within the effect sizes calculated is a notable trend where the sizes have clusters of similar magnitudes, seemingly based on the topic of the question. The questions regarding more generic attitudes toward computer science as a discipline or independent of computer science demonstrated smaller effect sizes compared to the questions created to target the goals of this research. Student attitudes toward computing as creative work and robotics changed more dramatically. No significant differences were associated with gender in pre- or post-survey responses outside of one question with a non-significant change, implying that the module’s positive effects on these opinions was applied to students identifying as male and female equally. Further analysis was not attempted on other demographic traits due to the small sample size. Future classes may have sufficient students to justify further analysis on the equitable effects of the curriculum.
Simultaneously, the survey questions which did not show significant change still may reveal insights into the students’ attitudes and potential avenues for future work to explore the impact of the course in greater detail. The question “I love designing things” remained around 4.84 before and after (a shift from 4.845 to 4.840, with ), implying that there was approximately zero change in this attitude. Students continued to love designing before and after the intervention, but this question did not probe whether this infatuation was applied evenly to all domains. Students gained a significant increase in how much they felt able to express themselves in computer science; they may view their computer science experience as orthogonal to “designing”. With the students’ initial interest in STEM topics, including robotics, being high, it is very interesting that the average student was not very confident in their ability to explain how robots make decisions. Future work may need to reintroduce assessments to determine if this attitude change represents the student’s ability increasing significantly or merely their awareness of the relevant concepts. While recency bias cannot be ignored, the final project received a plurality of responses when students were asked their favorite activity, with all robotics activities combined having a majority of responses.
The student interviews showed that the students did not seem to view the new content as notably difficult and that they generally enjoyed the custom environment creation and the open-endedness of the final project. While technical issues with NetsBlox occasionally posed an obstacle to the students’ enjoyment, the robotics platform performed well and students were motivated to create new scenarios well beyond what the platform was originally intended for. Students across various experience levels did not seem to find the RPCs available as too difficult to work with and enjoyed the tools presented to them for custom scenario creation. A theme across interviews was that students enjoyed the challenges RoboScape Online presents to them and wanted more of its content.
At the same time, the students’ feedback contributes a variety of potential improvements to streamline the experience of future classes. Additional documentation and content should be created to provide help both inside and outside the classroom. Course material should be adjusted to help smooth the pacing of the curriculum while not making things boring for advanced students. New features should be added to help debug difficulties in understanding the orientation of both the coordinate axes and the objects within the scene. The custom blocks should be expanded, possibly into multiple sets of blocks for students who desire different levels of abstraction over the underlying RPCs.
5. Limitations
There is a minor selection bias inherent in which students apply and are admitted to the SSMV program. While the school district students could be drawn from includes 17 high schools, geographic proximity appears to have been a significant deciding factor in applications. Most students attended the nearest four high schools, with the resulting demographics resembling these schools rather than the district as a whole. Additionally, although all students in the class participated in the robotics activity, these students may present a biased sample of their own high schools. This is a known problematic factor in quasi-experimental education research (Gopalan et al., 2020). The students in this study, having elected to join SSMV due to their interest in STEM topics, began the intervention with average positive opinions on all survey questions, which may not be fully representative of high school students in general. The effects of this intervention may be different if repeated in a classroom where the students do not start with as positive attitudes.
The students who chose not to respond to the post-survey were accounted for through an imputation method intended to provide a conservative estimate of their responses. This approximation likely understates the effect observed in this case study and future work will need to address the cause of missing data better. Students should be required to complete the survey in person, and the survey must be adjusted to make pairing responses possible while retaining the students’ privacy. A complete set of paired responses would have allowed more conclusions to be drawn from the data.
A similar selection bias to that presumed for the survey data may also apply to the interviews. The reasons students volunteered for and refused interviews should be explored in greater depth. Adding the option to volunteer for the interview as part of the survey and giving the option to describe their reasoning for refusal or acceptance could both inform future interview protocol and allow an exploration of correlations between other attitudes and interview participation. Additionally, the interviews were performed by a group of adult male researchers, some unfamiliar to the students. It is possible the students would be more eager to be interviewed with a different set of interviewers. Without the students’ reasons for refusing the interviews, it is not possible to determine if they simply preferred to have the time for their own activities, were shy of interviewing in general, or uncomfortable with the interview setting or format.
The instrumentation used for this case study is not able to clearly separate the effect of the different modules of the course. While students demonstrated a larger response to opinions regarding the creative aspects of the curriculum, without repeating the study with a control focusing only on the robotics and NetsBlox lessons, only the outcome of the entire module can be presented. Previous case studies without the custom environments content also found positive changes to student attitudes, which may account for some of the increase also noted here. Students could additionally be surveyed multiple times during the course in a future study. However, increasing the frequency of surveys risks inducing “survey fatigue”. Porter et al. (2004) observed this phenomenon reducing response rates over multiple student surveys, with students viewing the time cost of repeated surveys as too great to participate.
6. Conclusions
The benefits of educational robotics, with both physical and virtual robots, have been well established in the literature. The usefulness of simulation in educational robotics is similarly revealed through existing work. This work in turn expands upon educational robotics simulation by providing a networked educational robotics simulation platform, RoboScape Online, integrated into a block-based programming environment focused on distributed computing concepts. This simulation allows not only for reuse of the previous robotics curricula but also the creation of new materials focusing on topics which were otherwise infeasible in the classroom, while creating a more accessible robotics environment for all students. A significant portion of this integration is making new components of these simulated environments available through the same block-based abstractions previously used to communicate with the robots. RoboScape Online also provides tools to empower students to explore new possibilities both through making new sensors and actuators available as block-based abstractions and by putting the ability to create novel scenarios for the simulation into the hands of the same students who control robots through NetsBlox. Students will not only be able to use NetsBlox to control robots but through this integration, they can see the potential for creative work expressed through their own computer programs. Students who experienced the intervention felt more positive about not only thinking like computer scientists or understanding robots but more significantly, about creativity and self-expression through computers. Future interventions can continue to iterate with a focus on this often underappreciated aspect of coding.
The overall aim for RoboScape Online was to “lower the floor, widen the walls, and raise the ceiling” for classroom robotics. RoboScape made educational robotics available through a novice-friendly, block-based interface, but moving to virtual robots lowers the logistical floor as well. The barrier to entry with a transition to RoboScape Online is no longer a $300 robot per student group—any student with a computer can be given as many robots as they need to use and take home without putting expensive equipment at risk. RoboScape Online has even been developed with use on mobile devices and Chromebooks in mind, to allow it to be utilized by a wider range of students. Furthermore, the experience of a student working at home is equivalent to in the classroom, and by being a networked environment, two students with a great distance between themselves may interact with the same robots in the same space. Educators do not need to maintain or store virtual robots, and no longer need to coordinate with IT departments to determine if attaching robots to a school Wi-Fi network is feasible. At the same time, the unique cybersecurity features originally motivating RoboScape curricula are fully compatible with the virtual robots, allowing the existing content to continue to be usable with the new platform. A vast range of new capabilities for these robots widened the walls and raised the ceiling. The physical robots had an ultrasonic distance sensor, quadrature encoders on its wheels, and two “whisker” touch sensors on its front. These sensors were implemented into the virtual robots, but they serve only as a baseline functionality for compatibility with previous curricular material. The virtual robots feature LIDARs, position sensors, and more, with additional capabilities being added to the platform with no cost for existing users. With the new sensors integrated into the existing platform, limited training is required for students or educators to get started with these new capabilities, allowing them to quickly dive in to exploring autonomous robotics.
However, this goal soon expanded beyond robotics, to allow students to create three-dimensional, interactive worlds through the same NetsBlox syntax they use to control their robots, obtain data from a web service, or access a scientific dataset. Access to Google Maps through a “getMap” RPC and to modify a virtual environment with a RoboScapeWorld “addEntity” RPC are performed through the same blocks, with other RoboScape Online features also utilizing existing NetsBlox abstractions such as message passing. As with RoboScape itself, students are not forced to learn entirely unfamiliar abstractions for this domain; the same programming concepts they were exposed to previously remain relevant for new topics. This allows students to focus on the newly presented information when transitioning from a different domain to RoboScape Online. Additionally, this provides an interface made up of a cohesive whole, with the three-dimensional viewport and their code to populate it, and the code to control robots within it, all sharing a common interface. The existing NetsBlox features such as cloud collaboration and project storage are taken advantage of when working in RoboScape Online, where two students can connect to the same virtual space and the same project code, enabling remote pair programming. All RoboScape Online scenarios are now created through NetsBlox code, placing the floor at a level obtained during normal NetsBlox lessons. A student-created project could be shared with the community and even added to the default list of available scenarios, without any mandatory skills outside the domain of NetsBlox. Compared to NetsBlox itself, these scenario-creation tools greatly widen the walls and raise the ceiling. Students are no longer tied to a two-dimensional “stage”, and can now more easily create challenges and activities for each other to interact with.
RoboScape Online’s design not only addresses the logistical difficulties in using educational robots, helping to broaden access to this exciting modality, but also explores how the design of an educational platform may help inspire students to think about computer science in new ways. Students in the classroom case study most significantly improved their opinions at the intersection of computing and creativity, and the ability to create their own environments was the top activity when students were surveyed after the intervention. These results suggest that the open-endedness of the platform’s creative features can help students interact with programming in ways the previous iteration was too inflexible to provide. RoboScape Online may serve as a foundation for future interventions investigating the learning impacts of student-driven creative tasks in curricula previously lacking such opportunities. Educators may utilize this platform for novel interdisciplinary activities, for example integrating art or environmental science rather than robotics. By continuing to embrace student creativity, future iterations of RoboScape Online may be used to facilitate many domains serving as on-ramps to computer science.
Conceptualization, G.S., D.J., and Á.L.; methodology, G.S.; software, G.S. and D.J.; validation, G.S., D.J., S.K., and Á.L.; formal analysis, G.S.; investigation, G.S., D.J., S.K., and Á.L.; resources, G.S., M.D., and Á.L.; data curation, G.S.; writing—original draft preparation, G.S.; writing—review and editing, D.J., S.K., M.D., and Á.L.; visualization, G.S.; supervision, M.D. and Á.L.; project administration, G.S., M.D., and Á.L.; funding acquisition, Á.L. All authors have read and agreed to the published version of the manuscript.
The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of Vanderbilt University (approval #070624, date of approval 22 October 2024).
Informed consent was obtained from all subjects involved in the study.
The data presented in this study are available on request from the corresponding author due to privacy reasons.
The authors would also like to thank the School for Science and Math at Vanderbilt, a collaborative endeavor of Vanderbilt University and Metropolitan Nashville Public Schools (MNPS).
The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
The following abbreviations are used in this manuscript:
| STEM | Science, Technology, Engineering, and Mathematics |
| CS | Computer Science |
| CT | Computational Thinking |
| UDP | User Datagram Protocol |
| TCP | Transmission Control Protocol |
| HTTP | Hypertext Transfer Protocol |
Footnotes
1. Available online:
2. Available online:
3. Available online:
4. Available online:
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Enlarge this image.Figure 1. Example script made in the NetsBlox programming environment. A script on the “Stage” will send the “Map Clicked” message, triggering a series of interactions with web services to obtain geolocation and weather information to present to the user.
Enlarge this image.Figure 8. Sample NetsBlox program using various methods of the RoboScapeWorld service.
Enlarge this image.Figure 9. Example NetsBlox program using the RoboScapeEntity service. When run, the blue sphere orbits the yellow sphere.
Enlarge this image.Figure 10. NetsBlox program drawing LIDAR output graphically. The output shown would be displayed in the NetsBlox stage for the robot in the simulation shown.
Enlarge this image.Figure 11. Treasure Hunt scenario. (a) Output and viewport during search task. (b) After “dig” command is sent in target area.
Enlarge this image.Figure 12. Example of using message passing in RoboScape Online scenario. Note that subfigure (a) uses custom blocks to set up the walls and obstacles to conserve space. Both custom blocks consist of additional RPC run blocks. (a) NetsBlox code for scenario. (b) NetsBlox code to trigger “drop” from client. (c) Before “drop” message. (d) A short time after “drop” message.
Enlarge this image.Figure 13. Example of using a non-IoTScape NetsBlox web service in the RoboScape Online scenario. The simulated environment is modified based on weather conditions in the real world requested from a web service. (a) NetsBlox code for scenario. (b) When weather is clear. (c) When weather is cloudy.
Enlarge this image.Figure 15. “Hello, Custom Scenarios” scenario code variation using blocks library.
Enlarge this image.Figure 17. Obstacle Course scenario. Robot completes the course by moving from the initial red area to the final green area.
Enlarge this image.Figure 18. “LIDAR Road” scenario seen from two different perspectives. Robot begins in red area and must reach the exit marked in green. (a) Free camera view. (b) Robot first-person view.
Enlarge this image.Figure 19. Example of simple student program for controlling robot in LIDAR Road scenario.
Enlarge this image.Figure 20. “Tug of War” challenge scenario. “Blue” and “Red” teams compete over which half of the area the robot is in.
Interview guide.
| Question |
|---|
| What was your favorite activity? |
| If you had more time, what would you add to your project from that activity? |
| Did you like the virtual robotics environment? |
| What did you like about it? |
| What didn’t you like about it? |
| How did you feel about creating your own environments? |
| How could you make [creating environments] more intuitive? |
Statistically significant differences between pre- and post-surveys (1 = Strongly Disagree, 6 = Strongly Agree).
| Question | Pre-Survey Average | Post-Survey Average | Effect Size (Cohen’s d) | p-Value |
|---|---|---|---|---|
| I think I am very good at coming up with new ideas when working on projects. | 4.640 | 5.077 | > | < |
| Thinking like a computer scientist will help me do well in my classes. | 4.000 | 4.538 | > | < |
| I can be creative in computer science. | 4.308 | 5.040 | > | < |
| I can express myself in computer science. | 3.769 | 4.600 | > | < |
| I can explain how robots make decisions. | 3.500 | 4.385 | > | < |
Non-significant differences between pre- and post-surveys.
| Question | Pre-Survey Average | Post-Survey Average | p-Value |
|---|---|---|---|
| I think I am very good at figuring out how to fix things that don’t work. | 4.653 | 4.801 | < |
| I think I am very good at explaining my solutions to technical problems. | 4.077 | 4.583 | < |
| I think I am very good at solving problems. | 5.000 | 4.885 | < |
| I think I am very good at coming up with new ways to solve technical problems. | 3.962 | 4.154 | < |
| I love designing things! | 4.846 | 4.840 | < |
| I want to learn as much as possible about computer science. | 3.731 | 4.240 | < |
| Designing new things makes me feel excited. | 4.690 | 4.800 | < |
| I talk about how things work with friends or family. | 4.615 | 4.680 | < |
| Knowing computer science is important for contributing to my community. | 3.962 | 4.385 | < |
| Knowing computer science is important for me in the future. | 4.385 | 4.462 | < |
| I can have fun working with computer science. | 4.269 | 4.423 | <0.30 |
| I can make things that are interesting to me in computer science. | 4.308 | 4.720 | <0.16 |
| Robots are interesting to me. | 4.692 | 4.654 | < |
| Autonomous vehicles are interesting to me. | 4.731 | 4.615 | < |
| Robots will be commonplace in the future. | 4.885 | 5.000 | < |
Students’ favorite activities.
| Activity | Count |
|---|---|
| Custom Environments/Final Project | 9 |
| LIDAR Box Pushing/Road | 6 |
| Obstacle Course | 5 |
| Weather Map | 2 |
| Autonomous Driving (in general) | 1 |
| NetsBlox (in general) | 1 |
Students’ greatest challenges.
| Reason | Count |
|---|---|
| Glitches in NetsBlox | 4 |
| Course Pacing (too fast) | 3 |
| (Im-)Precision of Robot | 2 |
| Starting a Challenge | 1 |
| Autonomous Driving | 1 |
| Critical Thinking | 1 |
| Homework Solution Not Explained | 1 |
| Optimizing Code | 1 |
| Working with LIDAR | 1 |
| Robots (in general) | 1 |
| Map Project | 1 |
| Understanding blocks | 1 |
| Obstacle Course | 1 |
| Programming without Assistance | 1 |
Descriptive codes in student interviews.
| Code | Interview(s) |
|---|---|
| Benefits/Drawbacks of Simulation | 3 |
| Blocks Simplify Programming | 2, 4 |
| Collaboration | 1 |
| Confused by Terminology | 4 |
| Creative Freedom | 1 |
| Desire for More Content | 2, 3, 4 |
| Enjoyed Challenge | 1, 2, 3 |
| Enjoyed LIDAR | 2, 3 |
| Fun with Coding | 1, 2, 3, 4 |
| Interested in Distributed Computing | 2, 4 |
| Previous Experience | 2 |
| Potential for Advanced Projects | 2, 3 |
| Relevance to Everyday Technology | 4 |
| Sensors/Autonomy | 2, 3 |
| Technical Issues | 1, 3 |
| Wants More Documentation | 3, 4 |
References
Abichandani, P.; Sivakumar, V.; Lobo, D.; Iaboni, C.; Shekhar, P. Internet-of-Things Curriculum, Pedagogy, and Assessment for STEM Education: A Review of Literature. IEEE Access; 2022; 10, pp. 38351-38369. [DOI: https://dx.doi.org/10.1109/ACCESS.2022.3164709]
Adams, W. C. Conducting semi-structured interviews; Wiley Online Library: 2015.
Alsoliman, B. S. H. Virtual robotics in education: The experience of eighth grade students in STEM. Frontiers in Education Frontiers in Education; 2022; 7, 950766. [DOI: https://dx.doi.org/10.3389/feduc.2022.950766]
Berland, M.; Wilensky, U. Comparing virtual and physical robotics environments for supporting complex systems and computational thinking. Journal of Science Education and Technology; 2015; 24, pp. 628-647. [DOI: https://dx.doi.org/10.1007/s10956-015-9552-x]
Blank, D. Robots make computer science personal. Communications of the ACM; 2006; 49,
Broll, B.; Lédeczi, A.; Volgyesi, P.; Sallai, J.; Maroti, M.; Carrillo, A.; Weeden-Wright, S. L.; Vanags, C.; Swartz, J. D.; Lu, M. A visual programming environment for learning distributed programming. 2017 ACM SIGCSE Technical Symposium on Computer Science Education; Seattle, WA, USA, March 8–11; 2017; pp. 81-86. [DOI: https://dx.doi.org/10.1145/3017680.3017741]
Bugden, W.; Alahmar, A. Rust: The programming language for safety and performance. 2nd International Graduate Studies Congress; İstanbul, Turkey, June 8–11; 2022.
Catlin, D.; Blamires, M. The Principles of Educational Robotic Applications (ERA): A framework for understanding and developing educational robots and their activities. The 12th EuroLogo Conference; Paris, France, August 16–20; 2010.
Cheema, J. R. A Review of Missing Data Handling Methods in Education Research. Review of Educational Research; 2014; 84,
Cobb, P.; Confrey, J.; diSessa, A. A.; Lehrer, R.; Schauble, L. Design experiments in educational research. Educational Researcher; 2003; 32,
Coenraad, M.; Palmer, J.; Eatinger, D.; Weintrop, D.; Franklin, D. Using participatory design to integrate stakeholder voices in the creation of a culturally relevant computing curriculum. International Journal of Child-Computer Interaction; 2022; 31, 100353. [DOI: https://dx.doi.org/10.1016/j.ijcci.2021.100353]
Eeds, A.; Vanags, C.; Creamer, J.; Loveless, M.; Dixon, A.; Sperling, H.; McCombs, G.; Robinson, D.; Shepherd, V. L. The School for Science and Math at Vanderbilt: An Innovative Research-Based Program for High School Students. CBE—Life Sciences Education; 2014; 13,
Eguchi, A. Robotics as a learning tool for educational transformation. 4th International Workshop Teaching Robotics, Teaching with Robotics and 5th International Conference Robotics in Education; Padova, Italy, July 18; 2014; pp. 7-34.
Eguchi, A. Khine, M. S. Bringing Robotics in Classrooms. Robotics in STEM education: Redesigning the learning experience; Springer International Publishing: 2017; pp. 3-31. [DOI: https://dx.doi.org/10.1007/978-3-319-57786-9_1]
Enders, C. K. Missing data: An update on the state of the art. Psychological Methods; 2023; [DOI: https://dx.doi.org/10.1037/met0000563]
García-Morales, V. J.; Garrido-Moreno, A.; Martín-Rojas, R. The transformation of higher education after the COVID disruption: Emerging challenges in an online learning scenario. Frontiers in Psychology; 2021; 12, 616059. [DOI: https://dx.doi.org/10.3389/fpsyg.2021.616059]
Gerkey, B. P.; Vaughan, R. T.; Howard, A. The Player/Stage project: Tools for multi-robot and distributed sensor systems. 11th International Conference on Advanced Robotics; Coimbra, Portugal, June 30–July 3; 2003; Vol. 1, pp. 317-323.
Gopalan, M.; Rosinger, K.; Ahn, J. B. Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Review of Research in Education; 2020; 44,
Harris, P. A.; Taylor, R.; Minor, B. L.; Elliott, V.; Fernandez, M.; O’Neal, L.; McLeod, L.; Delacqua, G.; Delacqua, F.; Kirby, J.; Duda, S. N. The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics; 2019; 95, 103208. [DOI: https://dx.doi.org/10.1016/j.jbi.2019.103208] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31078660]
Harris, P. A.; Taylor, R.; Thielke, R.; Payne, J.; Gonzalez, N.; Conde, J. G. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics; 2009; 42,
Hong, L. The impact of educational robots on students’ computational thinking: A meta-analysis of K-12. Education and Information Technologies; 2024; 29,
Jawad, H. M.; Tout, S. Gamifying computer science education for Z generation. Information; 2021; 12,
Jean, D.; Stein, G.; Broll, B.; Ledeczi, A. Unleashing creativity with wireless embedded programming for next-generation makers. 16th International Conference on Education and New Learning Technologies; Palma, Spain, July 1–3; 2024.
Kafai, Y. B.; Fields, D. A.; Lui, D. A.; Walker, J. T.; Shaw, M. S.; Jayathirtha, G.; Nakajima, T. M.; Goode, J.; Giang, M. T. Stitching the loop with electronic textiles: Promoting equity in high school students’ competencies and perceptions of computer science. 50th ACM Technical Symposium on Computer Science Education; Minneapolis, MN, USA, February 27–March 2; 2019; pp. 1176-1182. [DOI: https://dx.doi.org/10.1145/3287324.3287426]
Kalaitzidou, M.; Pachidis, T. P. Recent Robots in STEAM Education. Education Sciences; 2023; 13,
Kuhn, J. Minecraft: Education Edition. CALICO Journal; 2018; 35,
Lédeczi, Á.; Maróti, M.; Zare, H.; Yett, B.; Hutchins, N.; Broll, B.; Völgyesi, P.; Smith, M. B.; Darrah, T.; Metelko, M.; Koutsoukos, X.; Biswas, G. Teaching cybersecurity with networked robots. 50th ACM Technical Symposium on Computer Science Education; Minneapolis, MN, USA, February 27–March 2; 2019; pp. 885-891. [DOI: https://dx.doi.org/10.1145/3287324.3287450]
Martins, F. N.; Matejov, A.; Šuppa, M. Moving robotics competitions virtual: The case study of RoboCupJunior Soccer Simulation (SoccerSim). Frontiers in Robotics and AI; 2022; 9, 915322. [DOI: https://dx.doi.org/10.3389/frobt.2022.915322]
Martín-Páez, T.; Aguilera, D.; Perales-Palacios, F. J.; Vílchez-González, J. M. What are we talking about when we talk about STEM education? A review of literature. Science Education; 2019; 103,
Matsakis, N. D.; Klock, F. S. The Rust language. ACM SIGAda Ada Letters; 2014; 34,
McCarthy, A. M.; Maor, D.; McConney, A.; Cavanaugh, C. Digital transformation in education: Critical components for leaders of system change. Social Sciences & Humanities Open; 2023; 8,
Mikropoulos, T. A.; Bellou, I. Educational robotics as mindtools. Themes in Science and Technology Education; 2013; 6,
Papert, S. Mindstorms: Children, computers, and powerful ideas; Basic Books: 1980; Available online: https://dl.acm.org/doi/book/10.5555/1095592 (accessed on 5 February 2025).
Porter, S. R.; Whitcomb, M. E.; Weitzer, W. H. Multiple surveys of students and survey fatigue. New Directions for Institutional Research; 2004; 2004,
Resnick, M. Lifelong kindergarten: Cultivating creativity through projects, passion, peers, and play; MIT Press: 2017.
Robertson, T.; Simonsen, J. Participatory design. Routledge international handbook of participatory design; Informa UK Limited: 2012.
Robinson, O. C. Sampling in interview-based qualitative research: A theoretical and practical guide; Taylor & Francis: 2014; Vol. 11, No. 1
Robomatter Inc. Robot virtual eorlds website; 2022; Available online: https://www.robotvirtualworlds.com/ (accessed on 5 February 2025).
Robotify. Robotify website; 2022; Available online: https://www.robotify.com/ (accessed on 5 February 2025).
Ross, J.; Holmes, O.; Tomlinson, B. Playing with Genre: User-Generated Game Design in LittleBigPlanet 2. LUCI Report; 2012; 3.
Saldaña, J. The coding manual for qualitative researchers; Sage Publications Ltd.: 2021.
Sawilowsky, S. S. New effect size rules of thumb. Journal of Modern Applied Statistical Methods; 2009; 8,
Siegel, A. A.; Zarb, M.; Alshaigy, B.; Blanchard, J.; Crick, T.; Glassey, R.; Hott, J. R.; Latulipe, C.; Riedesel, C.; Senapathi, M.; Simon,; Williams, D. Teaching through a global pandemic: Educational landscapes before, during and after COVID-19. 2021 Working Group Reports on Innovation and Technology in Computer Science Education; Virtual, June 26–July 1; 26 June 2021; pp. 1-25. [DOI: https://dx.doi.org/10.1145/3502870.3506565]
Simms, L. J.; Zelazny, K.; Williams, T. F.; Bernstein, L. Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychological Assessment; 2019; 31,
Snap! team. Snap! website; 2011; Available online: https://snap.berkeley.edu/ (accessed on 5 February 2025).
Stein, G. A novice-friendly networked educational robotics simulation platform. Ph.D. thesis; Vanderbilt University: 2024.
Stein, G.; Lédeczi, A. Shared virtual worlds for accessible classroom robotics. 53rd ACM Technical Symposium on Computer Science Education V. 2; Providence, RI, USA, March 3–5; 2022; 1177. [DOI: https://dx.doi.org/10.1145/3478432.3499259]
Stein, G.; Jean, D.; Brady, C.; Lédeczi, A. Browser-based simulation for novice-friendly classroom robotics. Frontiers in Computer Science; 2023; 4, 1031572. [DOI: https://dx.doi.org/10.3389/fcomp.2022.1031572]
Strauss, A.; Corbin, J. Basics of qualitative research; Sage: 1990.
Tan, Y.; Rizk, M.; Stein, G.; Lédeczi, A. User-Extensible Block-Based Interfaces for Internet of Things Devices as New Educational Tools. Southeastcon 2022; IEEE: 2022; pp. 711-717. [DOI: https://dx.doi.org/10.1109/SoutheastCon48659.2022.9763937]
Tellez, R. Merdan, M.; Lepuschitz, W.; Koppensteiner, G.; Balogh, R. A Thousand Robots for Each Student: Using Cloud Robot Simulations to Teach Robotics. Robotics in education; Springer International Publishing: 2017; pp. 143-155.
Tselegkaridis, S.; Sapounidis, T. Simulators in Educational Robotics: A Review. Education Sciences; 2021; 11,
Uzlu, T.; Şaykol, E. On utilizing rust programming language for Internet of Things. 2017 9th International Conference on Computational Intelligence and Communication Networks (CICN); Girne, Northern Cyprus, September 16–17; 2017; pp. 93-96.
VEX Robotics. VEXcode virtual robotics (VR); 2022; Available online: https://www.vexrobotics.com/vexcode/vr (accessed on 5 February 2025).
Welch, B. L. The generalization of ‘student’s’ problem when several different population varlances are involved. Biometrika; 1947; 34,
Zhang, Y.; Luo, R.; Zhu, Y.; Yin, Y. Educational robots improve K-12 students’ computational thinking and STEM attitudes: Systematic review. Journal of Educational Computing Research; 2021; 59,
Zhang, Z.; Gillespie, C. The impact of teaching and learning changes during the COVID-19 pandemic on the post-pandemic era. 2023 8th International Conference on Distance Education and Learning; Beijing, China, June 9–12; 2023; pp. 256-262. [DOI: https://dx.doi.org/10.1145/3606094.3606116]
Zuckerman, O. Designing digital objects for learning: Lessons from Froebel and Montessori. International Journal of Arts and Technology; 2010; 3,
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Devin, Jean 1
; Kittani, Saman 1 ; Deweese, Menton 2
; Lédeczi, Ákos 1
1 Department of Computer Science, Vanderbilt University, Nashville, TN 37212, USA
2 Department of Teaching and Learning, Vanderbilt University, Nashville, TN 37212, USA;