A Discourse Centric Framework For Facilitating Instructor Intervention In Mooc Discussion Forums
COM1 Level 3
MR1, COM1-03-19
Abstract:
We propose a discourse framework --- predictive models that use discourse level information in student posts --- to guide Massive Open Online Course (MOOC) instructors to selectively post on student discussions on forums, hereafter referred to as interventions, which otherwise is infeasible to scale. Prior works on intervention guidance, from pre-MOOC era, and on MOOCs do not address diversity and scale. Our models and evaluation explicitly cater to diversity and scale, both inherent to MOOCs. One way to scale interventions is to build prediction models that learn instructor intervention patterns and predict future interventions including on previously unseen courses. However, simplistic vocabulary based prediction models fail to adapt sufficiently due to the diversity in MOOCs in terms of subject areas, volume of discussions and pedagogical styles emanating from the instructor's culture and geography. Discourse relations are open-class words that are not only domain agnostic but also signal intervention due to the context of their occurrence in student posts. We show that PDTB discourse sense (e.g., expansion, contingency) based models scale prediction performance with training data. This further leads us to investigate inter-post discourse structures. We propose a pedagogically grounded discourse taxonomy and build an annotated corpus of student posts in instructor intervened threads. We address the key issue of position bias that affects instructor's decision to intervene since they create biased training samples. We propose a debiasing classifier to unlearn the bias and predict interventions. Finally, we investigate the context, the contiguous (sub)set of posts, that trigger intervention. Unsurprisingly, context significantly affects prediction over predicting intervention on individual posts. We show that neural dense vector representations of threads and a model of thread as a sequence of posts significantly improve the state-of-the-art towards production-ready models. Our models were evaluated on a diverse corpus of 14 MOOCs from various subject areas.
We have proposed predictive models to help instructors intervene effectively on MOOC forums. The models can be integrated to an instructor dashboard that can flag a discussion thread up to the precision of a post. A corpus of interventions annotated according to a discourse taxonomy will serve to build classification models of intervention to prompt instructors and peers to intervene on their respective types (extension vs clarification).