Matsuda, N., & Shimmei, M.* (2019). Application of Reinforcement Learning For Automated Contents Validation Towards Self-Improving Online Courseware. In B. Goldberg (Ed.), Proceedings of the Annaul GIFT User Symposium (pp. 57-65). Orlando, FL: U.S. Army Combat Capabilities Development Command Soldier Center.

Introduction: Online education has been growing rapidly for the last decade with exponential growth of diverse students population (Shapiro et al., 2017) and adaptive technology enhancements (Lerís, Sein-Echaluce, Hernández, & Bueno, 2017). However, building practical online courseware is extremely costly—it requires extensive knowledge and expertise in theories of learning and teaching (Clark & Mayer, 2003; Slavich & Zimbardo, 2012). Most of the time, instructional designers and instructors design an initial courseware from their honest intuition, and then the courseware will be iteratively modified to meet better learning outcome. Though, iterative software engineering is a norm for almost any sort of practical software applications (Fishman, Marx, Blumenfeld, Krajcik, & Soloway, 2004), it requires significant knowledge to identify issues to be fixed for improvement. It is therefore critical to develop a transformative theory of practical learning-engineering methods for iterative online courseware creation. Without such methods, it is not likely to have sustainable system of online education. What if the courseware improves itself over time? The larger goal of our current project is to develop a self-improving online courseware that automatically detects and fixes ineffective parts of the existing courseware relative to students’ learning achievement. As a step towards achieving this pivotal goal, we propose to develop an integrated development environment (IDE) where human and AI collaboratively build online courseware through iterative design engineering—a machine detects issues and a human fixes them. As a step towards the proposed human- AI collaboration, this paper describes an innovative application of a reinforcement learning technique called RAFINE (Reinforcement learning Application For INcremental courseware Engineering). The RAFINE aims to identify ineffective instructional elements on existing online courseware given a record of individual students’ learning activity logs. In the rest of the paper, we first discuss related works followed by a detailed description of RAFINE. We then describe details about a simulation study and results as a proof of concept.

PDF: download