Integrating Human and Automated Tutoring (IHATS) Closed

The Challenge

Learner interaction with both human and automated instruction has become a norm in classrooms worldwide. Blended learning is available in K-12 and universities, as well as professional and other training scenarios where students are interacting virtually or in person, synchronously or asynchronously. However, researchers generally do not have access to rich, large data sets that would improve how blended learning is provided. Such data could help optimize the “handoffs” between human and automated learning modalities in the future.

The Solution

Tools such as predictive models to determine whether and why learners seek human versus automated assistance when both are available, and research on how this information can improve the quality of tutoring in blended contexts.

About the Project

The IHATS project set out to examine whether a student using a computer-based training session would seek human tutor support when given the option, and what kind of support should be delivered to optimize learning. This research analyzed both human tutorial dialogue and records of student interactions using an intelligent tutoring system. Insight derived from this project can be used to develop systems that effectively guide students (i.e., make instructional handoffs) between automated and human sources of assistance.

The IHATS study examined a unique dataset of recorded interactions from students in two online developmental mathematics courses offered through an online university. The courses required students to complete work using a mastery-based intelligent tutoring system, while also offering unlimited access to an online chat-based tutoring service. Machine learning models were used to make inferences about the intelligent tutoring system that may play important roles in students’ decisions to seek out human tutoring. Models were also used to predict whether and when learners were likely to turn to the tutoring support provided by the chat service, and whether human tutor chat sessions were likely to be effective. Data were sampled during a six-month window when students took the course using both the required and optional tutoring support. Transcripts from thousands of chat sessions between these students and human tutors were collected. From the chat sessions, a coding taxonomy for tutoring sessions was developed to uncover interaction patterns.

A 2016 report titled Toward Intelligent Instructional Handoffs Between Humans and Machines outlines this study in detail and provides analysis of the findings. One key observation was that students who were poorly prepared for the course, as illustrated by the amount of time that they took to complete initial work and the amount of assistance that they required throughout the course, were more likely to use on demand, one-to-one tutoring service via a web-based chat system. Although unlimited free access to human tutors was made available, most students did not take advantage of this resource. However, a small group of “Super Users” turned to the human tutoring frequently, taking up a majority of human tutors’ time. This Super User sub-population could be further explored to understand particular student scenarios for seeking human tutoring assistance and identify potential cost-savings if this type of tutor support can be provided by adaptive tutoring software.

Publication