Advanced Distributed Learning Initiative Logo

The Advanced Distributed Learning Initiative

Kara L. Orvis, PhD iFest Presentation

Innovative and Unobtrusive Assessment of Learning in Distributed Multi-Team Systems

Kara L. Orvis, PhD

In distributed multi-team systems (MTSs), trainers and leaders face the on-going challenge of efficiently and accurately assess the learning that is occurring within the organization. Within these contexts, the staff must coordinate across warfighting function teams to share knowledge, learn, and adapt to environmental demands. Traditionally, during formal Army training exercises, trainers observe the staff and take notes as to the strengths and areas of improvement of the unit. However, observer ratings are limited, as they are time- intensive, subjective, biased, and often fail to detect nuances in learning. In the current work, we will present research using an innovative method for collecting continuous, real- time, objective information about the functioning of an Army Brigade staff organization. Leveraging the unit’s interaction data, collected via multiple data sources (e.g., email, worn sociometric sensors), we are able to develop and implement valid measures that can assess and provide near-real- time feedback as to the state of a team (e.g., shared understanding, cohesion), and how it changes over time, via a tailored user interface. This approach provides trainers and leaders with unique insights into how the unit is learning and changing over time. For this presentation, we will describe ongoing research that demonstrates how this data and feedback system can be used to assess the development of a unit’s shared understanding over time as a result of cross-team coordination efforts. We will also discuss the shift toward integrating tailored training recommendations within the UI that promote further learning based on the current state of the unit.


Dr. Kara Orvis is a Principal Scientist and Vice President of the Research and Development Group at Aptima Inc. She has 20 years of experience in government research and development with the Army, Air Force and Marine Corps. Her expertise is in the areas of training, leadership, teams, culture, distributed work, and unobtrusive measurement for which she has over 80 publications/presentations. At Aptima, she leads projects related to military assessment, formation, training, and development. Dr. Orvis has worked across all echelons of the military from squad to Division staffs. The quality of her work has been formally recognized throughout the years (e.g. three best paper awards and two patent awards). Dr. Orvis holds an M. A. and a Ph.D. in Industrial-Organizational Psychology from George Mason University and a B.A. in Psychology from Ohio Wesleyan University. She is a member of the American Psychological Association and the Society for Industrial and Organizational Psychology.


iFest 2017 Schedule:

iFest 2017 Agenda Download: