Publications
The ADL Initiative is an active participant in research and academia. Here you will find a full library of conference papers, conference abstracts, conference PPTs, books, journal articles, and technical reports composed by partners, collaborators, and ADL Initiative team members. Check back often for new entries.
Publication Results
TLA Standards Digital Learning Acquisition Techniques Report
2023;
This quarterly report provides techniques on how to integrate DoDI 1322.26 learning technology standards, which enable personalized, data-driven, and technology-enabled lifelong learning across the DoD.
xAPI Statement Template
2023;
This one page document provides an example of an xAPI statement in its native JSON formatting, followed by a simplified view of an xAPI statement with color coding to show the related portions.
The NATO Advanced Digital Learning Handbook
2023;
This handbook provides practical guidance to organizations for the creation and/or the procurement, implementation, and evaluation of Advanced Digital Learning (ADL). This handbook is not meant to be all-encompassing. It is an overarching look at ADL concepts and some of the tools used to develop ADL. Nations are intended to use this handbook...
DADLAC Annual Report 2022
2023;
The Defense ADL Advisory Committee (DADLAC) 2022 Annual Report highlights the distributed learning community’s activities in 2022
Enterpise Learning Record Repository (ELRR) Initial Operational Capability (IOC) Report
2022;
This document presents the requirements needed to advance the ELRR toward its Initial Operational Capability (IOC)
Enterprise Course Catalog (ECC) Minimum Viable Product (MVP) Test & Evaluation Report
2022;
The Enterprise Course Catalog (ECC) Minimum Viable Product (MVP) Test & Evaluation Report provides findings from the ECC MVP test and evaluation (T&E) conducted with designated DoD stakeholders at the end of FY21.
ADL Enterprise Course Catalog Systems Integration Plan
2022;
Provides a summary overview of the ECC Initial Operational Capability's implementation, guidance on the required interfaces between the ECC and the ADL TLA reference implementation
Design Recommendations for Intelligent Tutoring Systems; vol. 9
2022;
This publication focuses on the topic of competency-based scenario design as it relates to Intelligent Tutoring Systems (ITSs). The chapters specifically relate the work presented to applications for the Generalized Intelligent Framework for Tutoring (GIFT).
TLA Standards Digital Learning Acquisition Guidance
2022;
This document explores existing successful acquisitions and supplies that language as well as successful ways to implement best practices of standards and to identify mature standards (as a part of the ADL research portfolio, the Total Learning Architecture (TLA)).
2021 TLA Functional Requirements Document
2022;
Covers the objective end-state for Department of Defense (DoD) learning organizations migrating to Total Learning Architecture (TLA) compliant data and microservices.
DADLAC Annual Report 2021
2022;
The Defense ADL Advisory Committee (DADLAC) 2021 Annual Report highlights the distributed learning community’s activities in 2021
ADL Enterprise Learner Record Repository Prototype Final Report
2021;
This report details the project design and development activities associated with building the prototype Enterprise Learner Record Repository (ELRR) capability.
ADL Enterprise Learner Record Repository User Flow Diagrams
2021;
The (ELRR) Prototype User Flow Diagrams documents the specific tasks defined via ELRR Prototype requirements.
ADL Enterprise Learner Record Repository Systems Integration Plan
2021;
The scope of the ELRR Prototype Systems Integration Plan (SIP) is to provide technical guidance for integrating ELRR Prototype with ADL TLA sandbox components, such as the Authoritative Learner Record Store (LRS), Competency and Skills System (CaSS), and Enterprise Course Catalog (ECC), along with a recommended resource mixture to conduct the integration activities.
ADL Enterprise Learner Record Repository Database Design Description
2021;
The purpose of this document is to define the detailed Database Design Description for the ADL Initiative Enterprise Learner Record Repository (ELRR).
ADL Enterprise Learner Record Repository System Architecture Report
2021;
The (ELRR) System Architecture Report (SAR) outlines solution architecture by describing the specification and deployment of hardware and software interfaces.
cmi5 Best Practices Guide: From Conception to Conformance
2021;
This guide is intended to help you understand where, when, and how to incorporate cmi5 into your Distributed Learning (DL) strategy.
ADL xAPI RMF Accreditation Project (xRAP) Final Project Report / Test Results
2021;
A description of the changes and associated impacts from xAPI Version 1.0.3 to the draft IEEE Standard 9274.1.1.
An Application of the Learning Science Evaluation Checklist: Use Case Report - Science of Learning and Readiness (SoLaR)
2021;
Arizona State University;
Two DoD training and educational organizations were successfully evaluated with the previously proposed Learning Science Evaluation Checklist. This report provides a how-to guide for the checklist’s implementation and the results of the use case evaluations.
DATASIM Stakeholder Analysis Report
2021;
The purpose of this document is to present a report and summary documentation regarding the Stakeholder Analysis for DATASIM.
DATASIM Needs Analysis Report
2021;
This document presents the definitions, needs, features, and considerations necessary to examine the Option Year 1 iteration of the work to continue development of the Data and Training Analytics Simulated Input Modeler (DATASIM).
DATASIM Testing and Evaluation Plan
2021;
The purpose of this document is to explore the methodology used to test the TRL5 Beta version of the Data and Training Analytics Simulated Input Modeler (DATASIM).
DATASIM Requirements Report
2021;
This document enumerates the technical and design requirements for the Beta development phase of DATASIM.
DATASIM Option Year 1 Final Report
2021;
The purpose of this document is to present a report and related documentation regarding the Technology Readiness Level 5 (TRL5) prototype of the Data and Training Analytics Simulated Input Modeler (DATASIM).
ADL Enterprise Learner Record Repository Systems Integration Plan
2021;
The aim of this document is to provide a summary overview of the ELRR Prototype’s initial implementation, guidance on the required interfaces between the ELRR Prototype and the ADL Total Learning Architecture (TLA) reference implementation in an Amazon Web Services (AWS) sandbox environment, a corresponding integration approach, and a stakeholder engagement strategy to support a reference implementation of the ELRR Prototype.
ADL DAU Sandbox Final Report
2021;
The purpose of this project was to recreate key components of the Reference Implementation with commercially-available, open source, and customized solutions to demonstrate the value of organizations choosing to adopt the architecture to improve their ability to track competency-based learning.
DADLAC Annual Report 2020
2021;
The 2020 Annual Report showcases the efforts of the DoD distributed learning community to innovate and modernize education and training.
Total Learning Architecture (TLA) Data Pillars and their Applicability to Adaptive Instructional Systems
2021;
The TLA data strategy includes linkages across different standards and collectively provide a data foundation for adaptive systems to build upon. This paper and discussion will walk viewers through the different data models that are being used to drive development of these standards.
MADLx: Setting Foundations to Measure ROI - Learning Analytics Survey Report: DI-MISC-80711A
2021;
The Maturing ADL in Multinational Exercises (MADLx) project aims to design and develop a Return on Investment (ROI) analytics dashboard for use in multinational and coalition exercises. This report presents the analysis of detailed feedback from exercise stakeholders concerning their requirements for learning analytics and associated visualizations.
CaSS Authoring Tools: Final Report
2021;
This report outlines the functionality that CaSS Authoring Tools (CAT) supports, the challenges faced in developing CAT, and recommendations for future work.
MADLx: Setting Foundations to Measure ROI - Protocol/Recommendations Report: DI-MISC-80711A Bold Quest 20.2
2021;
This report provides the results and findings of online training focused on the Android Tactical Assault Kit (ATAK) and includes impacts on exercise performance due to COVID-19 pandemic circumstances. It documents the ADL contributions to the exercise, the planning process and implementation protocol ("how to"), lessons learned, and recommendations.
MADLx: Setting Foundations to Measure ROI - UX Exercise Memo: DI-MGMT-80227 Bold Quest 20.2
2021;
This report provides the results and findings of online training focused on the Android Tactical Assault Kit and includes impacts on exercise performance due to COVID-19 pandemic circumstances. It assesses the usability and user experience (UX) of the ADL offerings in the exercise and identifies specific, actionable recommendations for enhancing ease-of-access, usability, UX, and associated learner motivation of ADL participants.
Why do we adopt e-internships in eLearning curriculum development? A Model of Career-oriented Learning Experiences, Motivation, and Self-Regulated Learning
2020;
Arizona State University;
A conceptual paper proposed an integrated eLearning curriculum development model that emphasized the valuable role of career-oriented learning experiences in improving students' motivation and self-regulated learning.
Immersive Learning Environments at Scale: Constraints and Opportunities
2020;
Arizona State University;
This theoretical paper will briefly define immersive learning environments and the potential benefits of incorporating immersive learning environments into scalable educational systems.
Educational Data Mining and Learning Analytics for Improving Online Learning Environments
2020;
Arizona State University;
This paper aims to provide a brief introduction to the fields of EDM and LA, discuss data visualization and dashboards that are used to convey results to learners, and finally to identify where human factors can aid other fields.
Enterprise Course Catalog: Pre-Alpha Prototype Final Report
2020;
The 21st century is marked by significant technological progress in every field. For learning and development, these advancements have helped us realize the promise of “anytime, anywhere” learning as well as learning personalized to individual needs. More than that, emerging capabilities have thrown open the door to transformative possibilities, facilitating learning at scale, optimizing learning in response to large and varied data sets, and developing fully integrated talent management systems for managing and enhancing the future workforce.
The Advanced Digital Learning's Total Learning Architecture: IDA Report 2020
2020;
The Institute for Defense Analyses (IDA) has been active in assessing the need and value of the ADL Initiative from the beginning. It has been asked by the ADL office to support a variety of projects over the years. In 2017 and 2018, IDA evaluated the first trials of the Total Learning Architecture at Fort Bragg. Additionally, through its connections to the Institute for Electrical and Electronic Engineers, IDA continued to support the development of accredited international software standards required to realize the economic benefits of the TLA in both government and civilian implementation.
Advanced Digital Learning: Capability Maturity Model - Technical Report
2020;
The purpose of this report is to describe the development of ADL's Distributed Learning Capability Maturity Model (DL-CMM), illustrate its major components.
A Heuristic Evaluative Framework for Self-Regulated Learning Design
2020;
Arizona State University;
This paper then discusses how usability inspection methods, such as heuristic evaluations and cognitive walkthroughs.
Science of Learning and Readiness (SoLaR) Recommendation Report: Science of Learning Practices for Distributed Online Environments
2020;
Arizona State University;
This report provides recommendations on best practices for supporting and creating modern learning environments. The recommendations are based on findings from our two previous reports. The first report, the State of the Art Report, was a review of the state of the art for distributed learning environments that searched for best practices with evidence of supporting learning and improving learning organizations. The second report, The Exemplar Report, was a review of exemplar learning organizations that have successfully grown their learning enterprise.
Science of Learning and Readiness (SoLaR) Exemplar Report: A Path Toward Learning at Scale
2020;
Arizona State University;
This report seeks to provide an understanding of a State-of-the-Art Learning Organization. The features of an exemplar learning organization at scale were identified based on a broad review of the literature.
Science of Learning and Readiness (SoLaR) State-of-the-Art Report
2020;
Arizona State University;
This report consists of a review of the current state of the art for distributed learning environments. The report is structured for use by multiple types of end users. The main report provides a 22-page, high-level overview of findings of the current state of the art. This section can serve as a quick reference. The report’s Appendixes provide a detailed summary of the empirical literature on state-of-the-art distributed learning environments. These materials enable “deeper dives” into learning organization, technology, and human pedagogy topics.
2019 Total Learning Architecture Report
2020;
Data is a critical asset that enables effective decision making. The TLA data strategy provides a common set of goals and objectives across DoD’s education and training community to ensure data are used effectively. This overarching strategy will ensure that all data resources are positioned in a way that they can be used, shared, and moved efficiently across the organization. This report describes the four pillars of the TLA data strategy...
Technical Report: DATASIM
2020;
The purpose of this document is to present a report and related documentation regarding the Alpha TRL4 prototype of the Data and Training Analytics Simulated Input Modeler (DATASIM). The report explores the intended use cases of the DATASIM platform, the progress that has been made on the prototype reference software, and the aspects of the application which would benefit from further development in the next iteration of the platform.
Usability TRL4 Report: DATASIM
2020;
The current state of the Data and Training Analytics Simulated Input Modeler (DATASIM) project from the perspective of a DoD Technology Readiness Assessment
Competency Framework Development Process Report
2020;
This document is the final report for the Competency and Skills System (CaSS), contract number, W911QY-16-C-0002.
Technical Report: DAVE
2020;
The purpose of this document is to present a report and related summary documentation regarding the beta prototype of the Data and Visualization Environment (DAVE) for xAPI and the Total Learning Architecture.
PS4TLA: Privacy Support for the Total Learning Architecture Volume 5 - Policy Requirements
2019;
Clemson University;
The purpose of this document is to outline how privacy regulations and recommendations outlined in Federal laws and memorandums, DoD Directives and Instructions, and standardized guidelines are applicable to Total Learning Architecture (TLA)-based systems. The set of recommendations put forth in this document will allow ADL and other TLA performers to build the TLA specifications and TLA-based systems with compliance to these regulations and recommendations in mind.
PERvasive Learning System (PERLS): Verification, Validation, and Experimental Testing - Learning Strategy Analysis Report
2019;
This report provides a summary of a review of the learning strategy of the PERvasive Learning System (PERLS). PERLS is a mobile, personalized system for delivering content and recommendations to learners in the workplace or informal settings. The system was designed as a platform for SRL–a means for employees in office settings or persons serving in the military to advance their education in a self-determined manner. In addition, PERLS was created to support self-regulation using functions, interfaces, and prompts to guide attention and interest.
Data Analytics and Visualization Environment for xAPI and the Total Learning Architecture: DAVE Learning Analytics Algorithms
2019;
This report introduces a language for defining the functionality of learning analytics algorithms in terms of Operations, Primitives and Algorithms which will be used to define Algorithms corresponding to an initial set of learning analytics questions.
Military Instructors - Understanding Motivation and Job Satisfaction Across Two Nations
2019;
Canada and the United Kingdom provide research on instructor satisfaction/motivation issues facing militaries; and the theories, methods, and results.
DAU xAPI Content Module Analysis
2019;
The scope of this report is to provide configuration & launch instructions and recommendations for improving the quality of the Defense Acquisition University’s (DAU’s) xAPI implementation based on the results of analyzing the xAPI statements for the FPD420 Reviewer Training Course. The report will also provide some general considerations for converting SCORM content to xAPI for future DAU research efforts.
Competency-Based Learning
2019;
Military education and training encompass many different schools, universities, and training programs designed to foster technical, professional, and leadership skills in military service members. Historically, there has been a separation between the education and training communities across the services. Education occurs incrementally and...
DATASIM Conceptual UX/UI Design Report
2019;
Yet Analytics;
DATASIM is an open source application that will provide a valid means of producing the datasets necessary to benchmark and stress test the Total Learning Architecture (TLA) and distributed learning acquisitions. Additionally, DATASIM can help learning scientists, ISDs, IT and technology acquisition, and decision-making stakeholders to determine the effectiveness of xAPI data design and implementation across the TLA.
Talent Development Toolkit Requirements and Architecture Study
2019;
This report defines detailed technical requirements, an initial architecture, a phased implementation strategy, and evaluation metrics for the Talent Development Toolkit (TDT), i.e., the operational learning environment for the Intelligence Community (IC). The target audience for this report includes technical managers and technology...
2018 Total Learning Architecture Report
2019;
The Total Learning Architecture (TLA) program sponsored by the ADL Initiative seeks to develop a set of policy and standards defining the process for developing a learning ecology, where multiple services and learning opportunities (of various modalities and points of delivery) can be managed in an integrated environment.
Modernizing Learning: Building the Future Learning environment
2019;
Modernizing Learning: Building the Future Learning environment is an implementation blueprint for connecting learning experiences across time and space.
ADL in Exercises
2018;
The success of a training event depends upon its return on investment, that is, how well it prepares the participants, given the time and available resources.
ADL AS A TOOL OF SECURITY SECTOR REFORM: Observations and the Case of the Balkan RADLI project
2018;
In a modern democratically controlled military, everyone must continually improve throughout her career, not only during periods of regular education. On the other hand, a small military needs all its personnel at work. Courses offered requiring months away from military posts are no longer acceptable. Downward budget pressure on partners also means...
Online Access, Distribution and Further Collaborative Development of the NATO ADL Handbook
2018;
Several options were reviewed for the online access, interactive distribution and further collaborative development of the NATO ADL handbook. Mapping the immediate, adjacent and theoretical stakeholders, firm recommendations can be made for LRS options, server set up requirements, and a list of recommended verbs to be monitored in the collaborative environment.
The Truth About Blockchains and How They Apply to Training
2018;
IITSEC;
This presentation covers how blockchains work, real world implementations, blockchains and training, and resources and Q&A's.
Learning Analytics with xAPI in a Multinational Military Exercise
2018;
IITSEC;
As the truism goes, "You can't manage what you don't measure." However, assessing performance in training exercises has classically presented a measurement challenge, made more complex by the paucity of timely, relevant, comparable data on the training audience's performance. Even as the field of learning analytics becomes increasingly...
At the Tipping Point: Learning Science and Technology as Key Strategic Enablers for the Future of Defense and Security
2018;
IITSEC;
According to former U.S. Secretary of Defense, Ash Carter, today's national security environment is "dramatically different - and more divergent and complex in the scope of its challenges - than the one we've been engaged with for the last 25 years, and it requires new ways of thinking and new ways of acting" (2016, emphasis is ours). These new ways cannot be...
Integrating Advanced Digital Learning into Multinational Exercises
2018;
IITSEC;
Armed forces around the world face similar challenges regarding exercises and live training. The context is increasingly multinational, and operations and supporting systems are more complex, all requiring more extensive training. To thrive in these volatile, complex and evolving security environments, military personnel also require an...
Total Learning Architecture: Moving Into the Future
2018;
IITSEC;
Increasingly, the defense community requires a continuous, adaptive learning enterprise that delivers the right training, education, and just-in-time support, in the right ways and at the right time. The Total Learning Architecture (TLA), now in its second iteration of development, is intended to help meet that vision. The TLA is a set of internet and...
Team Orlando: Community of Progress
2018;
IITSEC;
Team Orlando has become an international icon for the modeling, simulation, and training community. Countless examples of cooperation demonstrate the exceptional advantage of having co-located representatives from every military branch, academia, and industry. The benefit of this community exceeds the sum contributions of the players but...
Leveraging Science and Technology to Launch Innovation in Learning
2018;
IITSEC;
In remarks about the National Defense Strategy, the U.S. Secretary of Defense, Jim Mattis, observed that in a time of rapid technological change and an over-stretched military, "Success does not go to the country that develops a new technology first, but rather, to the one that better integrates it and more swiftly adapts its way of fighting" (2018,...
Advanced Digital Learning Task 6 Acquisition for New Distributed Learning Report
2018;
This report documents our research on successful acquisition related processes associated to distributed learning modernization and acquisition templates/tools related to standards and enablers. It identifies and provides common language, practices, and procedures that enable successful acquisition of DL technology for the DoD. This...
Advanced Digital Learning Task 5(b) Cybersecurity Barriers to Distributed Learning Modernization Report
2018;
The Advanced Distributed Advanced Digital Learning (ADL) Initiative asked the Potomac Institute for Policy Studies (PIPS) to conduct a study to identify cybersecurity distributed learning (DL) modernization barriers to understand the potential implications of promulgating the DoDI 1322.cm on costs, implementation, consequences, and...
Financial Readiness for Learning (FR-PAL) Final Report
2018;
Sen$e is a mobile application designed to support service members and their families with financial readiness information using micro-learning, gamification, and other motivational techniques to cultivate life-long learning. Sen$e supports service member financial literacy and preparedness through engaging "bite-sized," personally...
US DoD xAPI Profile Server Recommendations
2018;
In order for a federated network of xAPI Profile servers to serve their purpose, there needs to be a way for an organization or group to create an xAPI Profile and share the information they authored without risk of losing data or outside groups changing the given xAPI Profile. This requires a governance model where a central authority informs profile...
PS4TLA: Privacy Support for the Total Learning Architecture: Summit Report
2018;
How can we reconcile the need for extensive customizability with users' apparent lack of skills and motivation to manage their own privacy settings? In this report we investigate User-Tailored Privacy as means to support users' privacy decision-making. With User-Tailored Privacy (UTP), a system would first measure users' privacy-related...
PS4TLA: Privacy Support for the Total Learning Architecture: Modeling Factors
2018;
The purpose of this document is to make recommendations for implementing User-Tailored Privacy (UTP) into Total Learning Architecture (TLA)-based systems and to inform ADL and other TLA performers about the Modeling Factors that need to be considered in the context of this implementation. The set of recommendations put forth in this document will...
Recommendation across Many Learning Systems to Optimize Teaching and Training
2018;
AHFE, Applied Human Factors and Ergonomics;
To help learners navigate the multitude of learning resources soon to become available in the Total Learning Architecture (TLA) environment, a Recommender algorithm will give learners learning resource recommendations. Recommendations will support immediate training needs and provide guidance throughout one's career. This paper describes...
Financial Readiness for Learning (FR-PAL) Summative Usability Test Results Report
2018;
The Financial Readiness-Personal Assistant for Learning (FR-PAL), Sen$e, is a native mobile application designed to support service members and their families with financial readiness information using micro-learning, gamification, and other motivational techniques to cultivate life-long learning. The usability and effectiveness of Sen$e...
Learning Record Provider Professional Certification Recommendations
2018;
Professionals who work with the Experience API (xAPI) need a formal body of knowledge to generate high quality data with xAPI. Some of this knowledge can be sourced directly from xAPI's specifications and the multitude articles, documents, reports and books devoted to applied research and development with xAPI. Practice leaders, even in these early...
Sero!: A Learning Assessment Platform for Adult Learning Environments
2018;
Training and development programs within corporate environments have seen a significant expansion in recent years. Yet there is little research regarding workplace learning assessments. Learning assessments could be used as a tool to gauge employee knowledge in a specific domain area, which can be useful in identifying areas of strengths and weaknesses and indicate where improvement is needed...
Mobile Learning Decision Path
2018;
Government agencies interested in implementing mobile learning (mLearning) solutions are doing so through pilots and trial and error. There are many important decisions to make prior to and during the design, development, and implementation of an mLearning solution. Best practices for reusing and redesigning existing learning content for the mobile platform do not currently exist. The many variables...
Using Competencies to Map Performance across Multiple Activities
2017;
IITSEC;
When a single training system accumulates data on learner performance, the data are stored in a way determined by the system's designers. This enables the system to access these data and to apply them to its interactions with learners.In environments such as live-virtual-constructive federations, each component may store performance data in its own...
Total Learning Architecture Development: A Design-Based Research Approach
2017;
IITSEC;
Organizations that use learning technology to educate and train are facing a new set of interoperability problems. Many new products - including adaptive systems, intelligent digital tutors, real-time data analytics, and interactive e-books - offer dramatic learning benefits. However, these products primarily "stand alone" and work outside of...
A Mobile Strategy for Self-Directed Learning in the Workplace
2017;
IITSEC;
Traditional approaches to workplace training often treat learners as equally prepared, drive them through too much content in too short a time, and conclude before ensuring retention. These departures from ideal instructional practice have a common cause –the need to fit learning activities into constrained episodes such as classroom...
Teaching and Learning Differently: Personalized E-Books for Learning (PEBL)
2017;
IITSEC;
Although current eBooks are little more than print shown in an e-reader, as a mobile, connected platform, eBooks have the unrealized potential to support more pedagogical approaches than traditional books, including experiential, problem-based, dynamic, and social learning. To realize this potential, the Personalized eBooks for Learning (PEBL)...
Developing a Financial Readiness Mobile Personal Assistant for Learning
2017;
IITSEC;
While Service members and their families receive unique financial benefits as a result of their service, few are well educated in the best ways to manage their finances. Consequently, they may fall prey to predatory lending practices, high pressure sales pitches, and other financial pitfalls. Adding to the complexity of this problem is the fact that...
Performance Measurement in LVC Distributed Simulations: Lessons from OBW
2017;
IITSEC;
Operation Blended Warrior (OBW) 2016 marked the second year of a three-year effort to document lessons learned and understand barriers to implementing Live, Virtual, Constructive (LVC) distributed training. In the first year of the event, LVC focus areas included connectivity, interoperability, data standards, after-action review, and cyber...
Approximate Graph Matching for Mistake-tolerant Skill Assessment
2017;
ACS, Advances in Cognitive Systems;
This paper presents an approach to automated assessment for online training based on approximate graph matching. The algorithm lies at the core of two prototype training systems that we have built in accord with U.S. Army training materials: one for the use of a collaborative visualization and planning tool, the other for rifle maintenance. The...
Maturation, Evaluation, Alignment and Testing of Sero!
2017;
ADL Publication;
This Technical Report Study/Services Ad Hoc Reports and Works is intended to deliver 3.2.4 a Base POP Final Report by summarizing the work performed during the Base POP, assess progress, identify solved and unsolved technical problems, and set forth specific work details for the Optional POP. An itemized assessment of progress is included in Appendix...
Exploring Assessment Mechanisms in the Total Learning Architecture (TLA)
2017;
Chapter in Book - GIFT;
The focus of this chapter is on the challenges and potential solutions to conducting realtime and long-term assessments of performance, learning, and domain competency in the Total Learning Architecture (TLA). TLA, a distributed learning environment, is being developed by the US Office of the Secretary of Defense to support capabilities for...
Humans as the Strong Link in Securing the Total Learning Architecture
2017;
Applied Human Factors and Ergonomics, AHFE;
This paper describes a proposed approach, centered on human factors, for securing the Total Learning Architecture (TLA). The TLA, which is being developed for the United States Department of Defense, will rely on large stores of personal data that could be targeted by sophisticated adversaries. We describe the TLA and its envisioned users at a fairly...
Intelligent Instructional Hand Offs
2017;
EDM;
Learners in various contemporary settings (e.g., K-12 classrooms, online courses, professional/vocational training) find themselves in situations in which they have access to multiple technology-based learning platforms and often one or more non-technological resources (e.g., human instructors or on-demand human tutors). Instructors,...
Demonstration-based Solution Authoring for Skill Assessment
2017;
ACS, Advances in Cognitive Systems;
Enable automated skill assessment, traditional approaches have required significant time investment by highly trained individuals.
More Than the Sum of Their Parts: Case Study and General Approach for Integrating Learning Applications
2017;
MODSIM;
Learning system interoperability standards let applications share or connect infrastructure, reducing development costs, administrative burden, and user friction. We propose a more comprehensive form of interoperability that supports shared userdata, user interfaces, and session management services. This will let applications leverage...
Distributed Learning Gap Report
2017;
ADL Publication;
The DoD faces a growing challenge to meet the breadth, depth, and tempo of its expanding education and training needs. While budgets are shrinking, the complexity of missions is increasing and demands on personnel are growing. Technologies - such as distributed learning capabilities - can help address these challenges, but the training, education,...
The Changing Face of Military Learning
2017;
Army Learning Journal;
Globalization, social media, ever-increasing computing power, and the proliferation of low cost advanced technologies have created a level of worldwide complexity and rapid change never before seen. To remain competitive in this environment, the Department of Defense and our coalition allies must identify new ways to empower our forces. In this...
Death To The Privacy Calculus?
2017;
SSRN Journal, Social Science Research Network;
The "privacy calculus" has been used extensively to describe how people make privacy-related decisions. At the same time, many researchers have found that such decisions are often anything but calculated. More recently, the privacy calculus has been used in service of machine learning approaches to privacy. This position paper discusses the...
User-Tailored Privacy by Design
2017;
USEC, Usable Security Conference;
The "privacy by design" philosophy addresses privacy aspects early in the design and development of an information system. While privacy by design solutions often provide considerable advantages over "post hoc" privacy solutions, they are usually not customized to the needs of individual users. Further, research shows that users...
PS4TLA: Privacy Support for the Total Learning Architecture Operational Characteristics
2017;
The Total Learning Architecture (TLA) is a set of specifications to enable the development of next-generation learning systems. As the TLA specifications are being developed, there exists an opportunity to implement Privacy by Design(PbD), where privacy is treated as a fundamental part of the system, and taken into account throughout the entire...
Advancements in Distributed Learning (ADL) Environment in Support of Transformation
2017;
RTG HFM-212 successfully addressed its stated objectives as listed in the TAD/TAP. This success was due to a substantial NATO/PDP organizational and policy base as well as it developed a strong working relationship with the NATO Allied Command Transformation (ACT) in Norfolk, VA, USA. With the funding support provided by the CWP (Coalition Warfare...
The Experience API (xAPI) Certification Program Recommendations for Learning Record Stores
2017;
The document describes a program for LRS Certification, including: governance roles and responsibilities for ADL, opportunities for organizations beyond ADL to develop product and technology certification programs that employ xAPI’s LRS functionality, testing procedures and supporting business processes.
Joint Inference for Mode Identification in Tutorial Dialogues
2016;
COLING, Conference on Computational Linguistics;
Identifying dialogue acts and dialogue modes during tutorial interactions is an extremely crucial sub-step in understanding patterns of effective tutor tutee interactions. In this work, we devel opa novel joint inference method that labels each utterance in a tutoring dialogue session with a dialogue act and a specific mode from a set of pre-defined...
Toward Intelligent Instructional Handoffs Between Humans and Machines
2016;
NIPS, Neural Information Processing Systems;
The IHATS Project, leveraging a unique dataset of both an automated tutoring system and transcripts of chat sessions between students and human tutors.
Black Swans and the Limits of Hierarchy
2016;
IITSEC;
Black Swans are unknowable events, by their nature unpredictable, asymmetric and easier to recognize in retrospect:nothing is clear at the outset, there is only an emergent sense of understanding, and traditional diagnose/act/review cycles may trap us into mismatched known responses. The unknowable cannot be mitigated or fully understood through...
Strategic Compression and the Military's Pursuit of Cognitive Readiness
2016;
IITSEC;
The Joint Force of today is facing an inflection point with respect to the way it believes it should learn, train, and educate. The generation that fought America's longest war is more savvy and intuitive but also sober on what really "works." The authors have observed, for instance, that "Jointness" is being redefined, and correspondingly preparation...
Total Learning Architecture (TLA) Enables Next-generation Learning via Meta-adaptation
2016;
IITSEC;
Technology is becoming ever more central to teaching and training. In classrooms, students use intelligent tutors and adaptive tests instead of textbooks and worksheets. In daily life, mobile devices enable blended, on-demand and ubiquitous life-long learning applications. Connected, pervasive media enables compelling transmedia learning...
Education and Training Technology Requirements for DoD Distributed Learning
2016;
IITSEC;
The Department of Defense faces a growing challenge to meet the breadth, depth, and tempo of its expanding education and training needs. While budgets are shrinking, the complexity of missions is increasing and demands on personnel are growing. Learning technologies can help address this, but the research and acquisition communities must make...
Advanced Digital Learning: A Global Perspective
2016;
IITSEC;
Coalition military forces have become the norm, and to be ready for multinational operations, militaries must prepare using relevant training, education, and exercises. Many nations and security coalitions use Advanced Digital Learning technologies to, at least partially, meet this demand. Towards that end, this paper examines the...
Choosing a Learning Management System (LMS)
2016;
The purpose of this paper is to help those involved in the process of choosing a learning management system (LMS) to make an informed decision.
Review and Assessment of Personnel Competencies and Job Description Models and Methods
2016;
Institute for Defense Analysis, IDA;
Organizations are always looking for new ways to acquire, manage, and retain the talent needed to reach their goals. To achieve this goal, some government and private organizations emphasize the need to better understand personnel and occupational information and data for improved competency management. Competencies and detailed job descriptions...
Intelligent Coaching Systems in Higher-Order Applications: Lessons from Automated Content Creation Bottlenecks
2016;
IAC;
Procedural skills are an increasingly pervasive requirement in today's world, in areas ranging from IT system administration to complex data analyses, from automotive equipment repair to intricate medical diagnosis. The acquisition of procedural skills requires learning by doing learners gain knowledge by trying to solve challenge problems,...
Tensor Factorization for Student Modeling and Performance Prediction in Unstructured Domain
2016;
EDM;
We propose a novel tensor factorization approach, Feedback-Driven Tensor Factorization (FDTF), for modeling student learning process and predicting student performance. This approach decomposes a tensor that is built upon students' attempt sequence, while considering the quizzes students select to work with as its feedback. FDTF does not...
Preliminary Results on Dialogue Act Classification in Chat-based Online Tutorial Dialogues
2016;
EDM;
We present in this paper preliminary results with dialogue act classification in human-to-human tutorial dialogues. Dialogue acts are ways to characterize the intentions and actions of the speakers in dialogues based on the language-as-action theory. This work serves our larger goal of identifying patterns of tutors' actions, in the form of...
A Comparative Study of Visual Cues for Annotation-Based Navigation Support in Adaptive Educational Hypermedia
2016;
UMAP;
Adaptive link annotation is one of the most well-known adaptive navigation support technologies that aims to guide hypermedia users to the most relevant information by personalizing the appearance of hyperlinks. Past work assumed no difference between different interface implementations of personalization approaches that are conceptually the...
Choosing Authoring Tools
2016;
The purpose of this paper is to help those involved in the process of choosing authoring tools to make an informed decision. The paper presents a range of considerations for choosing tools, whether as an enterprise-wide acquisition or a single user purchase, and includes a sampling of current tools categorized according to the kind of product they are intended to produce.
Choosing a Learning Record Store (LRS)
2016;
The purpose of this paper is to help those involved in the process of choosing a learning record store (LRS) to make an informed decision.
Tailored Cybersecurity Training in LVC Environments (Slides)
2016;
MODSIM;
This presentation in tailored cybersecurity training addresses the cybersecurity workforce need, describes stakeholders, reviews what actions are being done today and what is new.
Tailored Cybersecurity Training in LVC Environments
2016;
MODSIM;
Cyber vulnerabilities are continually emerging as a threat to our national and economic security and stability. Reports indicate a tremendous gap in skilled personnel capable of filling our growing need for a Cyber Security workforce to operate, analyze, protect, and defend our critical infrastructure systems. In response, the Department of...
An Intelligent Interface for Learning Content: Combining an Open Learner Model and Social Comparison to Support Self-Regulated Learning and Engagement
2016;
IUI, Intelligent User Interfaces;
We present the Mastery Grids system, an intelligent interface for online learning content that combines open learner modeling (OLM) and social comparison features. We grounded the design of Mastery Grids in self-regulated learning and learning motivation theories, as well as in our past work in social comparison, OLM, and adaptive navigation...
Animated Examples as Practice Content in a Java Programming Course
2016;
Special Interest Group on Computer Science Education, SIGCSE;
Code examples are commonly used learning resources that help students grasp various programming structures and concepts. However, example code usually requires explanations about what each line or part of the code does. Otherwise, students may find it difficult to follow an example. In this paper, we compare two types of code examples that use...
Open Social Student Modeling for Personalized Learning
2015;
IEEE TRANSACTIONS;
Open student modeling (OSM) is an approach to technology-based learning, which makes student models available to the learners for exploration. OSM is known for its ability to increase student engagement, motivation, and knowledge reflection. A recent extension of OSM known as open social student modeling (OSSM) complements cognitive aspects of OSM...
Impact of Open Social Student Modeling on Self-Assessment of Performance
2015;
E-Learn, AACE;
This study examines the impact of Open Student Modeling (OSM) and its extension known as Open Social Student Modeling (OSSM) on students' self-assessment of their SQL programming performance. It also explores the relationship between self-assessment and normalized gain. The study was performed with graduate students of University of Pittsburgh...
The Value of Social: Comparing Open Student Modeling and Open Social Student Modeling (Slides)
2015;
UMAP;
This presentation describes Open Social Student Modeling and its evaluation. The presentation goes into why they studied the comparison and what they have done since submitting their paper.
Value of Social: Comparing Open Student Modeling and Open Social Student Modeling
2015;
UMAP;
Open Student Modeling (OSM) is a popular technology that makes traditionally hidden student models available to the learners for exploration. OSM is known for its ability to increase student engagement, motivation, and knowledge reflection. A recent extension of OSM known as Open Social Student Modeling (OSSM) attempts to enhance cognitive aspects...
A Framework for Multifaceted Evaluation of Student Models (Slides)
2015;
EDM;
These slides are related to the paper called A Framework for Multifaceted Evaluation of Student Models that can be found at http://www.educationaldatamining.org/EDM2015/uploads/papers/paper_164.pdf.
A Framework for Multifaceted Evaluation of Student Models
2015;
EDM;
Latent variable models, such as the popular Knowledge Tracing method, are often used to enable adaptive tutoring systems to personalize education. However, finding optimal model parameters is usually a difficult non-convex optimization problem when considering latent variable models. Prior work has reported that latent variable models obtained...
Institutionalizing Blended Learning into Joint Training: A Case Study and Ten Recommendations
2014;
Teachers and trainers have relied upon blended learning for at least two decades, and scholars have authored (and continue to write!) thousands of papers about it. Yet, despite the wealth of individual experience and reams of published theory available, institutionalizing high-quality blended learning processes in large, dispersed organizations...
Nextgen Navy eLearning Tracking
2014;
The Navy's eLearning (NeL) computer-based learning system relies on a Learning Management System (LMS) for content delivery and tracking learning information. One major obstacle with NeL's current LMS implementation is that tracking of learning can only be done while a user is on a desktop computer using an Internet browser to connect to the LMS...
Journal of Advanced Digital Learning Technology
2014;
Journal of Advanced Digital Learning Technology, JADLeT;
The year 2014 marks the fifteenth anniversary of the Advanced Digital Learning (ADL) Initiative that was established in the United States (U.S.) under executive order in 1999 by then President William Clinton. As a result of ExecutiveOrder 13111, the U.S. Department of Defense (DoD) was directed to lead federal government participation in...
Goal Orientation, Self-Efficacy, and "Online Measures" in Intelligent Tutoring Systems
2014;
Cognitive Science Society, COGSCI;
While goal orientation and related factors like learner self-efficacy are of great interest to learning science researchers, some voice concerns regarding the measurement of such factors using self-report questionnaires. To address these concerns, recent work has explored the use of behavioral indicators like hint-seeking and glossary use in...
Where in the Data Stream Are We?: Analyzing the Flow of Text in Dialogue-Based Systems for Learning (ADL Chapter 19 pg 237-245) 1
2014;
(Army Research Lab Book);
This chapter continues a discussion we began in the first volume in this series (Hu, Morrison&Cai, 2013) concerning the use of "learner micromodels" in dialogue-based ITSs. As we originally defined it, a learner micromodel in an ITS is an estimate of a learner's cognitive and/or affective state at a given time in an ITS session, based entirely on the...
Reinforcing Math Knowledge by Immersing Students in a Simulated Learning-By-Teaching Experience
2014;
International Artificial Intelligence in Education Society, IAIIES;
We often understand something only after we've had to teach or explain it to someone else. Learning by teaching (LBT) systems exploit this phenomenon by playing the role of tutee. BELLA, our sixth-grade mathematics LBT systems, departs from other LBT systems in several ways: (1) It was built not from scratch but by very slightly extending the ontology and...
Cognitive Task Analysis: Analyzing the Cognition of Gameplay & Game Design
2013;
IITSEC;
A prior study performed by ADL, which measured the effects of certain video game design features thought to increase cognitive adaptability, brought to light how little is understood of the cognitive elements of video games, even by those who design them, let alone those who wish to study them or utilize them for learning or improving cognitive...
Advanced Digital Learning: Enabling Enhanced Learning Experiences (Slides)
2013;
e-Learning Software for Education, eLSE;
Abstract not available.
Videogame Design for Cognitive Enhancement through Micro-Puzzle Cognitive Profiling
2013;
IITSEC;
The Advanced Digital Learning (ADL) Initiative's Next Generation Learner researchers previously investigated whether five video game design features hypothesized to be contained within Portal 2 might increase cognitive adaptability (CA). Their results highlighted a lack of understanding of the cognitive elements of video games within the...
Drill Evaluation for Training Procedural Skills
2013;
Artificial Intelligence in Education;
The acquisition of procedural skills requires learning by doing. Ideally, a student would receive real-time assessment and feedback as he attempts practice problems designed to exercise the targeted skills. This paper describes an automated assessment and feedback capability that has been applied to training for a complex software system in...
Toward “Hyper-Personalized” Cognitive Tutors Non-Cognitive Personalization in the Generalized Intelligent Framework for Tutoring
2013;
We are starting to integrate Carnegie Learning’s Cognitive Tutor (CT) into the Army Research Laboratory’s Generalized Intelligent Framework for Tutoring (GIFT), with the aim of extending the tutoring systems to understand the impact of integrating non-cognitive factors into our tutoring. As part of this integration, we focus on ways in which non-cognitive factors can be assessed, measured, and/or “detected.” This research provides the groundwork for an Office of the Secretary of Defense (OSD) Advanced Digital Learning (ADL)-funded project on developing a “Hyper-Personalized” Intelligent Tutor (HPIT). We discuss the integration of the HPIT project with GIFT, highlighting several important questions that such integration raises for the GIFT architecture and explore several possible resolutions.
The Next Generation of SCORM: Innovation for the Global Force
2012;
IITSEC;
This presentation covers the background and requirements of SCORM.
Mobile Learning: Not Just Another Delivery Method
2012;
IITSEC;
This presentation explores the effectiveness of mobile course delivery. It covers the background, mobile learning definitions, ISD and ID Models, research goals, literature review learning theories and instructional strategies. The presentation also introduces the proposed ADL Initiative framework, along with lessons learned and future research work.
Sharing Learning Content: Beyond the Technology
2012;
IITSEC;
This presentation covers goals, DITIS, CORDRA and the ADL registry, content repositories, human perspective, and growing the sharing community.
Continuum of eLearning: 2012 Project Summary Report
2012;
This report outlines a variety of challenges associated with joint military training and education. It begins by detailing gaps in collective (live) training and discussing limitations associated with online learning. These limitations were uncovered through reviews of the literature, as well as interviews with military stakeholders and...
Research Foundations for the Advanced Digital Learning Initiative
2010;
Institute for Defense Analysis, IDA;
The Advanced Digital Learning (ADL) Initiative was established by the Office of the Undersecretary of Defense for Personnel and Readiness (OUSD(P&R)) in 1997. Its purpose was to assist the military Services in making "learning" (education, training, and performance/decision aiding) available on-demand, anytime and anywhere. "Learning" in...
Implementation Fest: The Last Decade
2010;
ADL Vision: Provide access to the highest quality education and training, tailored to individual needs, delivered cost effectively, anywhere and anytime. Web-based Learning * Content Repositories * Distributed Simulation * Job Aiding * Immersive Learning Technologies.
Cost-Benefit Study of a Project to Lower Cost and Improve Fleet Readiness through Integrating the Management of Technical Information
2010;
This paper describes a cost-benefit analysis by the Institute for Defense Analyses of the Bridge Project that ADL (Advanced Digital Learning) is conducting for the Office of Secretary of Defense for Acquisition, Technology and Logistics (OSD(AT&L)) to improve the management of Integrated Logistics Support (ILS). The Project is part of the OSD...
Immersive Environments in ADL
2009;
This ADL Initiative presentation covers the potential of immersive environments by describing current beliefs, issues in the market and targeting a wide array of audiences.
ADL State of the Union. Where We Are and Where We Are Going
2009;
ADL VISION: Provide access to the highest quality education and training, tailored to individual needs, delivered cost effectively, anywhere and anytime: Web-based Learning - Content Repositories - Distributed Simulation - Job Aiding - Immersive Learning Technologies. NEW DIRECTION: Priorities * New Strategic Plan (Sep 09) - R&D (BAA) - Technical...
The Prospects for Increasing the Reuse of Digital Training Content
2009;
Distributed learning (DL) offers the promise of self-paced learning and training at any time and in any place, as well as new technologies for developing and delivering content and tracking student performance. Although demand for DL is increasing, DL still represents a small percentage of all learning and training, in part because of the high cost of...
The Advanced Digital Learning (ADL) Vision and Getting From Here To There
2005;
The Advanced Digital Learning (ADL) initiative was undertaken by the Department of Defense (DoD) at the request of the White House Office of Science and Technology Policy (OSTP) and in cooperation with the other Federal Agencies. Its goal is to make education, training, and performance aiding accessible anytime and anywhere. This goal is being...
Utility and Applicability of the Sharable Content Object Reference Model (SCORM) Within Navy Higher Education
2004;
Analyzes the Sharable Content Object Reference Model (SCORM) within higher education and examines SCORM's limitations within a realistic application environment
Military Transformation: Progress and Challenges for DOD's Advanced Digital Learning Programs
2003;
The Department of Defense (DOD) spends more than $17 billion annually for military schools that offer nearly 30,000 military training courses to almost 3 million military personnel and DOD civilians, much of it to maintain readiness. To better meet the divergent defense challenges of the future, DOD is transforming its forces, including its training,...
Gateway Newsletter. Volume 14. Number 1, 2003
2003;
This newsletter contains the following articles: Transitioning Perspectives to Optimize Advanced Training Designs Future Combat Systems Manpower, Personnel, and Training (MPT): The Challenges and Paths Forward Calendar Training in the 21st Century: A Human Systems Integration Perspective and The Advanced Digital Learning (ADL)...
What do Sharable Instructional Objects Have to do With Intelligent Tutoring Systems, and Vice Versa?
2001;
By reviewing data on classroom and tutorial instruction, this document presents a perspective on the value of technology-based instruction in general and intelligent tutoring systems (ITSs) in particular. It finds the mixed-initiative dialogue and on-demand, real-time generative capabilities to be defining functionalities of ITSs. These...
Theoretical Foundation for Advanced Digital Learning Research
2001;
Advanced Digital Learning (ADL) has been embraced by the Department of Defense, other Government agencies, private industry, and academia as a tool that has the potential to improve the efficiency and effectiveness of modem instructional systems. As with any other instructional approach, ADL cannot achieve its full potential if the individuals...
The Impact of the DoD Mobile Code Policy on Advanced Digital Learning, Web-Based Distance Learning and Other Educational Missions
2001;
Mobile code is software that downloads via the Internet and runs on users' workstations without the users' knowledge. Mobile code can be both potentially beneficial and harmful to systems and networks in the Department of Defense (DoD). The DoD Memorandum, dated 7 November 2000, Policy Guidance for Use of Mobile Code Technologies in Department of...
Sharable Courseware Object Reference Model (SCORM), Version 1.0
2000;
The Department of Defense (DoD), in cooperation with the Office of Science and Technology, established the Advanced Digital Learning (ADL) Initiative to develop a strategy and process for using learning and information technologies together to modernize education and training in the DoD and across federal agencies. The ADL initiative has...
Department of Defense Strategic Plan for Advanced Digital Learning
1999;
Section 378 of Public Law 105-261, the Strom Thurmond National Defense Authorization Act for Fiscal Year 1999 (hereafter, The Act), requires the Secretary of Defense to develop a strategic plan for guiding and expanding distance learning initiatives within the Department of Defense, to include a provision for the expansion of such initiatives over...