PASTEL

Goal

The goal of the PASTEL (Pragmatic methods to develop Adaptive and Scalable Technologies for next generation E-Learning) project is to develop evience-based methods for efficient and practical learning engineering. In particular, we are interested in developing advanced technologies to build adaptive online courseware. The current PASTEL contains following methods:

SMART (Skill Model mining with Automated detection of Resemblance among Texts) - An application of text mining for automatic discovery of latent skill model (a set of skills and concepets to be learned) from written instructional materials including text instructions, formative assessments, and hint meessages.
RAFINE (Reinforcement learning Application For INcremental courseware Engineering) - An application of Reinforcement Leanring to automatically validate the online courseware contents based on students' performance and learning data.
QUADL (QUiz generation with Application of Deep Learning) - An application of deep learning to automatically generate formative assessments from written instructions.
WATSON (Web-based Authoring Technique for adaptive tutoring System on Online courseware) - An application of interactive machin learning to create intelligent tutoring systems (aka cognitive tutors) on a standard web-browser (hence easily embedded into the online courseware).
RADARS (RApid Detection And Recovery from wheel Spinning) - An application of educational data-mining for early predication of unproductive failure (aka wheel-spinning) on particular online courseware.
PROSE (Prompt, Rubric, and Ontology generation to enable Semantic feedback for Essays) - An application of topic mining and ontology engineering for automatic generation of constructive essay feedback.

Publications

(*s indicate student authors)

Journal Papers

Matsuda, N., Wood, J.*, Shrivastava, R.*, Shimmei, M.*, & Bier, N. (2022). Latent Skill Mining and Labeling from Courseware Content. Journal of Educational Data Mining, 14(2), 1-31.

Peer-reviewed Conference Papers

Shimmei, M.*, & Matsuda, N. (2023). Machine-Generated Questions Attract Instructors when Acquainted with Learning Objectives. In N. Wang, G. Rebolledo-Mendez, O. C. Santos, V. Dimitrova & N. Matsuda (Eds.), Proceedings of the International Conference on Artificial Intelligence in Education (pp. 3-15): Springer. [0.21 acceptance rate out of 251 submissions]

Shimmei, M.*, & Matsuda, N. (2023). Can’t Inflate Data? Let the Models Unite and Vote: Data-agnostic Method to Avoid Overfit with Small Data. In R. Agrawal, Y. Narahari, M. Pechenizkiy, M. Feng, T. Käser & P. Talukdar (Eds.), Proceedings of the International Conference on Educational Data Mining (pp. 286-295): Educational Data Mining Society. [Recipient of the Honorable Mention Award]

Shimmei, M.*, & Matsuda, N. (2022). Finding Key Concepts to Automatically Generate Pedagogically Valuable Questions for Learning Objectives. Paper presented at the Annual Meeting of the American Educational Research Association (pp. 1-7). [Nominee for the Best Paper award and the Best Student Paper award]

Shimmei, M.*, & Matsuda, N. (2021). Learning Association between Learning Objectives and Key Concepts to Generate Pedagogically Valuable Questions. In I. Roll & D. McNamara (Eds.), Proceedings of the International Conference on Artificial Intelligence in Education (pp. 320-324, short paper).

Shimmei, M.*, & Matsuda, N. (2020). Learning a Policy Primes Quality Control: Towards Evidence-Based Automation of Learning Engineering. In A. Rafferty & J. Whitehill (Eds.), Proceedings of the International Conference on Educational Data Mining (pp. 224-232): EDM.

Shimmei, M.*, & Matsuda, N. (2019). Evidence-Based Recommendation for Content Improvement UsingReinforcementLearning. In S. Isotani, A. Ogan, B. McLaren, E. Millán, P. Hastings & R. Luckin (Eds.), Proceedings of the International Conference on Artificial Intelligence in Education (pp. 369-373). Cham, Switzerland: Springer.

Matsuda, N., Van Velsen, M., Barbalios, N., Lin, L., Vasa, H., Hosseini, R.*, . . . Bier, N. (2016). Cognitive Tutors Produce Adaptive Online Course: Inaugural Field Trial. In A. Micarelli, J. Stamper & K. Panourgia (Eds.), Proceedings of the International Conference on Intelligent Tutoring Systems (pp. 327-333). Switzerland: Springer. [0.27 acceptance rate out of 119 submissions]

Matsuda, N., Chandrasekaran, S.*, & Stamper, J. (2016). How quickly can wheel spinning be detected? In T. Barnes, M. Chi & M. Feng (Eds.), Proceedings of the International Conference on Educational Data Mining (pp. 607-608).

Matsuda, N., Furukawa, T., Bier, N., & Faloutsos, C. (2015). Machine beats experts: Automatic discovery of skill models for data-driven online course refinement. In J. G. Boticario, O. C. Santos, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros, J. M. Luna, C. Michaescu, P. Moreno, A. Hershkovitz, S. Ventura & M. C. Desmarais (Eds.), Proceedings of the International Conference on Educational Data Mining (pp. 101-108). Madrid, Spain.

Workshop, Symposium, and other Conference Papers

Matsuda, N., & Shimmei, M.* (2019). Application of Reinforcement Learning For Automated Contents Validation Towards Self-Improving Online Courseware. In B. Goldberg (Ed.), Proceedings of the Annaul GIFT User Symposium (pp. 57-65). Orlando, FL: U.S. Army Combat Capabilities Development Command Soldier Center.

Park, S.*, & Matsuda, N. (2018). A Scalable Method for Rapid and Efficient Skill Discovery for Next-Generation Adaptive Online Courseware Paper presented at the Annual Meeting of the American Educational Research Association. New York, NY.

Book Chapters

Matsuda, N., Shimmei, M.*, Chaudhuri, P.*, Makam, D.*, Shrivastava, R.*, Wood, J.*, & Taneja, P*. (2023). PASTEL: Evidence-based learning engineering methods to facilitate creation of adaptive online courseware. In F. Ouyang, P. Jiao, B. M. McLaren & A. H. Alavi (Eds.), Artificial Intelligence in STEM Education: The Paradigmatic Shifts in Research, Education, and Technology (pp.93-108). New York, NY: CSC Press.

Shimmei, M.*, & Matsuda, N. (2021). Interactive Online Course Engineering Using Reinforcement Learning with Students’ Performance Profile. In H. Jiao & R. Lissitz (Eds.), Enhancing Effective Instruction and Learning Using Assessment Data (pp. 47-59). Charlotte, NC: Information Age Publishing.

Shen, S.*, Shimmei, M.*, Chi, M., & Matsuda, N. (2019). Applications of Reinforcement Learning to Self-Improving Educational Systems. In A. M. Sinatra, A. C. Graesser, X. Hu, K. Brawner & V. Rus (Eds.), Design Recommendations for Intelligent Tutoring Systems (Vol. 7: Self-Improving Systems, pp. 77-96). Orlando, FL: US Army Research Lab.

Press

Pop Quiz: AI Matches Human Performance at Developing Good Test Questions (2023). NC State University News. Also found at the CSC News website.

Major Grants

National Science Foundation, Cyberlearning and Future Learning Technologies. Collaborative Research: Cyberinfrastructure for Robust Learning of Interconnected Knowledge. Principal Investigator (with Norman Bier and David Yaron). July 1, 2020 to June 30, 2023. Award No. 2016966. $386,884

National Science Foundation, Cyberlearning and Future Learning Technologies. Exploratory study on the Adaptive Online Course and its implication on synergetic competency. August 1, 2016 to July 31, 2018. Award No. 1623702. $550,000

National Science Foundation, Research on Education and Learning (DIR). Data-Driven Methods to Improve Student Learning from Online Courses. August 1, 2014 to July 31, 2017. Award No. 1418244. $504,740.