Shayan Doroudi

Integrating Human and Machine Intelligence for Enhanced Curriculum Design Degree Type: Ph.D. in Computer Science
Advisor(s): Emma Brunskill
Graduated: May 2019

Abstract:

From the mechanical teaching machines of the early twentieth century to the wave of massive open online courses in recent years, many have been motivated by the dream of delivering a personalized adaptive curriculum to each learner. To achieve this dream, many researchers have focused on rule-based systems that rely on extensive domain expertise and psychological theories. While this approach has led to the development of successful intelligent tutoring systems with high quality content, (1) developing such systems can be very costly and (2) these systems typically employ a very limited form of adaptive instructional sequencing. In contrast, some researchers are now starting to apply black box machine learning algorithms to do adaptive instructional sequencing. However, these approaches have had relatively limited impact to date. Instead, I propose several techniques for impactful, cost-effective semi-automated curriculum design that combine machine learning, human computation, and principles from the learning sciences. My thesis will focus on two pieces of the curriculum design process: (1) content creation and curation and (2) instructional sequencing. First, I study the prospects of using learner-generated work for low-cost content creation. I explore this in the context of crowdsourcing tasks, where new kinds of work may require on-demand training. For two different kinds of crowdsourcing tasks, I show that learner-generated content can potentially be a useful way of teaching future learners, provided that the best content is automatically curated. Second, I show that due to model misspecification, relying on simple models of student learning can lead to making misinformed judgments about how to sequence content for students, including inequitable outcomes for low performing students. To mitigate this problem, I suggest two ways in which we can effectively use models to perform instructional sequencing: (1) using multiple models of learning to develop instructional policies that are robust to how students actually learn, and (2) combining models of learning based on psychological theory with data-driven approaches. The broader theme of my thesis is that by integrating human and machine intelligence, we can improve upon efforts to better teach students in semi-automated ways.

Thesis Committee:
Emma Brunskill (Chair)
Vincent Aleven
Ken Koedinger
Chinmay Kulkarni
Eric Horvitz (Microsoft Research)

Srinivasan Seshan, Head, Computer Science Department
Tom M. Mitchell, Interim Dean, School of Computer Science

Keywords:
Curriculum design, learner sourcing, content creation, reinforcement learning, instructional sequencing, student modeling, model misspecification, bias-variance tradeoff