Intelligent Coaching Systems in Higher-Order Applications: Lessons from Automated Content Creation Bottlenecks
2016; IAC; Greuel, C.; Yarnall, L.; Ziker, C.; Kernbaum, A.; Murray, J.
Procedural skills are an increasingly pervasive requirement in today's world, in areas ranging from IT system administration to complex data analyses, from automotive equipment repair to intricate medical diagnosis. The acquisition of procedural skills requires learning by doing learners gain knowledge by trying to solve challenge problems, exploring the usage and limitations of tools and techniques, getting feedback on oversights and mistakes, and requesting assistance in the face of impasses and confusion. Intelligent virtual environments (VEs) hold promise for improving learner directed instruction in these contexts. Such systems trace the progress of learners as they perform training tasks, and can insert immediate coaching or provide performance evaluation to focus learner attention, link knowledge to activity, and accelerate the shifts between abstract and concrete learning. VE technology is widely used to improve self directed learning of hands on manual procedures, but it also shows appreciable promise for the use of modeling tools in a diverse range of higher order applied fields, such as design engineering, policy analytics, and econometrics. To realize this vision, research must address the formidable bottlenecks around content creation and explore the types of reusable content libraries relevant to the subject domains. In this paper, we describe two interactive training projects that developed prototypes for automated content creation. A third project illustrates a suite of learning object libraries to support engineering instruction. The first project, Semantically enabled Automated Assessment in Virtual Environments (SAVE), uses a 3D browser based simulation environment for hands-on training in equipment maintenance, supplemented by automated generation of instructional exercise solutions. SAVE allows a subject matter expert (SME) to use interactive simulations for modeling the correct steps applied to given procedural tasks and provide a rapid way to extract their knowledge. The system logs an SME's activity, which becomes the reference model against which learner activity is compared in automated assessment. The second project, ARMentor, delivers augmented reality (AR) overlays in headmounted displays worn by student technicians while they learn vehicle maintenance. An automated speech system interacts with the learners as they perform equipment adjustments and troubleshoot electrical faults. To deliver audible step-by-step guidance, a prototype text-to-speech translator was developed to convert steps as written in the technical manual into the voice of a virtual coach. The third project, Simulation for Manufacturing and Prototyping with a Learning Environment (SiMPLE), developed tools to allow learners to construct electromechanical simulations, providing an intelligent coaching system to enable them to iteratively refine their design specifications. These tools include object libraries with embedded engineering computations and suites of scripts for design coaching, design testing, and physical prototyping once a working simulation is achieved.