Lessons Learned from Creating Course Content using Generative AI

Abstract

The release of generative AI tools such as OpenAI's ChatGPT has sparked interest in their implications for education. While early discourse emphasized concerns about plagiarism and academic integrity, recent studies have begun to explore the potential of these tools to support teaching and learning. This paper presents a case study on the use of ChatGPT in the redesign of a first-year systems development project course for informatics students. The course required the integration of various course materials, making it a suitable context for evaluating generative AI's role in course material development. The aim of the study is to present lessons learned from using ChatGPT in the development of course content. Drawing on our practical experience as course designers and instructors, we outline lessons learned from using ChatGPT in the creation of key course elements, including case descriptions, SQL scripts, and requirements specifications. We found that ChatGPT was effective for generating coherent initial drafts of content, but its outputs often required refinement to ensure pedagogical alignment. Challenges included the generation of misleading or irrelevant non-functional requirements and logically flawed code, despite syntactic correctness. Our findings highlight the importance of prompt engineering, critical review, and maintaining a human-in-the-loop approach. We conclude that while ChatGPT can significantly reduce development time for some tasks, it should be used as a complementary tool. This study contributes practical insights to the growing field of AI- assisted education.