As the US Army continues to accumulate learner performance and assessment data across a spectrum of courses, uncovering meaningful insights into student performance becomes increasingly challenging. The US Army Research Lab (ARL) has prototyped a framework that demonstrates an automated data generation and analysis capability for the ARL’s Generalized Intelligent Framework for Tutoring (GIFT) that addresses this challenge. These capabilities allow instructors to create simulated learner performance data, assess training effectiveness, and evaluate alternative instructional strategies. The prototype serves two primary functions: a) instructors can utilize the authoring tool to simulated class populations using distributional properties that can be adjusted and visualized by the author; and b) instructors can utilize the analytics framework to make content and curricula adjustments after instructional delivery. The nature of GIFT dictates that this framework be highly scalable and extensible in order to accommodate different types of instructions and assessments. In contrast to other efforts that focus on the technical components training effectiveness and data analytics, this prototype focuses utilizing User-centered design (UCD) principles to anticipate how users are likely to use the product and test the validity of their assumptions with regard to user behavior. We also describe the associated analytics outputs, which reads and analyzes both simulated and real data, and generates user-centered ‘human readable’ results for users without a strong statistical background.
Michael Smith has more than 13 years of experience in analytics, strategic planning, risk assessment, innovation, and project management. Mr. Smith currently advises several Department of Defense clients on how to adapt emerging data science, big data, and analytics practices to improve their performance. Mr. Smith currently leads a research project for Army Research Laboratory, Human Research and Engineering Directorate and manages a team of computational scientists and learning researchers developing novel methods for automated course effectiveness analytics using simulation, pattern recognition, and factor analysis. Mr. Smith regularly publishes and presents on the topics of big data, data science, and analytics. Mr. Smith holds a Master of Public Policy from Georgetown University and a Bachelor of Arts in International Economics from Longwood University.