Intelligent Coaching Systems in Higher-Order Applications: Lessons from Automated Content Creation Bottlenecks2016; IAC; Greuel, C.; Yarnall, L.; Ziker, C.; Kernbaum, A.; Murray, J.
Procedural skills are an increasingly pervasive requirement in today’s world, in areas ranging from IT system administration to complex data analyses, from automotive equipment repair to intricate medical diagnosis. The acquisition of procedural skills requires learning by doing learners gain knowledge by trying to solve challenge problems, exploring the usage and limitations of tools and techniques, getting feedback on oversights and mistakes, and requesting assistance in the face of impasses and confusion. Intelligent virtual environments (VEs) hold promise for improving learnerdirected instruction in these contexts. Such systems trace the progress of learners as they perform training tasks, and can insert immediate coaching or provide performance evaluation to focus learner attention, link knowledge to activity, and accelerate the shifts between abstract and concrete learning. VE technology is widely used to improve selfdirected learning of handson manual procedures, but it also shows appreciable promise for the use of modeling tools in a diverse range of higherorder applied fields, such as design engineering, policy analytics, and econometrics. To realize this vision, research must address the formidable bottlenecks around content creation and explore the types of reusable content libraries relevant to the subject domains. In this paper, we describe two interactive training projects that developed prototypes for automated content creation. A third project illustrates a suite of learning object libraries to support engineering instruction. The first project, Semanticallyenabled Automated Assessment in Virtual Environments (SAVE), uses a 3D browserbased simulation environment for hands-on training in equipment maintenance, supplemented by automated generation of instructional exercise solutions. SAVE allows a subject matter expert (SME) to use interactive simulations for modeling the correct steps applied to given procedural tasks andprovidea rapid way to extract their knowledge. The system logs anSME’s activity, which becomes the reference model against which learner activity is compared in automated assessment. The second project, ARMentor, delivers augmented reality (AR) overlays in headmounted displays worn by student technicians while they learn vehicle maintenance. An automated speech system interacts with the learners as they perform equipment adjustments and troubleshoot electrical faults. To deliver audible stepbystep guidance, a prototype texttospeech translator was developed to convert steps as written in the technical manual into the voice of a virtual coach. The third project, Simulation for Manufacturing and Prototyping with a Learning Environment (SiMPLE), developed tools to allow learners to construct electromechanical simulations, providing an intelligent coaching system to enable them to iteratively refine their design specifications. These tools include object libraries with embedded engineering computations and suites of scripts for design coaching, design testing, and physical prototyping once a working simulation is achieved.