Building the Infrastructure for DoD Digital Learning Modernization

The ADL Initiative has released a report detailing its progress in developing the Total Learning Architecture (TLA). The TLA is a collection of data standards, interoperability specifications, IT architectural schemes, and policies that are collectively designed to enable federated digital learning ecosystems. This approach promises to modernize education and training for the DoD enterprise and beyond.

The 2019 TLA Report provides context for multiple efforts, each focused on developing foundational components of the “future learning ecosystem.” This ecosystem concept links together DoD’s disparate education and training digital learning systems into an interoperable system of systems that is fueled by the collection, sharing, and analysis of terabytes of learner data.

The TLA project began in earnest in 2016 with the vision of establishing a common data strategy across the DoD education and training enterprise. Over the past four years, various working groups and research projects have focused on components of this vision. As described in the latest report, the 2019 focus was on moving the research into practice, in particular, refining and hardening the architecture with an updated “Reference Implementation” (i.e., example software prototype buildout) to support the demonstration of TLA components in real-world DoD use cases.

Based on these efforts, the report’s appendices provide information for IT professionals, software system designers, and education/training system administrators. The technical guidance is consistent with the DoD Architectural Framework (DoDAF), and it describes how to integrate and implement TLA-based capabilities in detail.

Data Standards

The TLA data strategy uses a standards-based approach for learner data. Research in 2018 assessed the applicability of various commercially available standards to TLA requirements. By 2019, four specifications were identified as candidate standards and an effort was launched to have them formally adopted through the Institute of Electrical and Electronics Engineers (IEEE). Through 2019 and into 2020, the ADL Initiative has coordinated with DoD and other stakeholders on working groups focused on shepherding these candidate standards through the IEEE process.

The newly proposed standards include:

• IEEE P92741.1 – This is an update to the xAPI standard, which focuses on learning activity tracking using activity streams. This standard also includes xAPI Profiles, such as cmi5 and the TLA’s Master Object Model. The standard will be formalized as xAPI 2.0, and it is expected to be approved in 2020.
• IEEE 1484.12.1 – The Learning Object Metadata 2.0 standard is used to describe learning activities (e.g., courses, training exercises) and their associated content (e.g., videos, scenarios). It builds on a modified version of the Learning Resource Metadata Initiative standard. A draft standard is being submitted for finalization in early 2020.
• IEEE 1484.20.1 – The Reusable Competency Definitions standard defines a data model for describing, referencing, and sharing competency definitions, their relationships to other competencies, and the alignment of evidence to help measure proficiency of the competencies. An update to this standard is expected to be approved in 2020.
• IEEE 1484.2 – The Integrated Learner Record standard provides guidance for learner records related to: 1) Identity and trust, 2) Open ontology references, 3) Verifiable assertions, and 4) Integrated learner record payloads supporting the spectrum of formal to informal learning and associated learning records. This initiative intends to align existing learner record standards, commonly used across different academic sectors (e.g., K-12, post-secondary, continuing education), to help guide their integration into a record of lifelong learning.

Beyond these four, the ADL Initiative is supporting other IEEE standards updates that will contribute to TLA functionality, including IEEE 1484.1, the Conceptual Model for Learning Technology Systems. A study group is working to replace the deprecated Learning Technology Systems Architecture standard with this updated version. It aligns the IEEE standards that inform the TLA data strategy with other relevant standards backed by the Common Educational Standards Council and IMS Global. The objective is to guide the design, development, and delivery of learning technologies for primary, secondary, post-secondary, workforce, and informal education providers.

Federated Data Sharing

In parallel with the development of data standards, the ADL Initiative team prototyped new and more efficient approaches for data sharing.

The TLA uses a data federation approach to enable global discoverability and semantic interoperability for data stored by different DoD Components within their associated systems. This approach decouples the functions typically performed by traditional standalone learning applications, such as those found in Learning Management Systems and Student Information Systems, into composable services. This approach promises to support a broader array of learning experiences and enable learning at-scale across the DoD enterprise.

Research in 2019-2020 also addressed a key challenge to the federated architecture approach: compliance with Federal Identity, Credential, and Access Management (FICAM) security requirements. The ADL Initiative is integrating Federal Government and DoD guidance on FICAM within the TLA, and the TLA’s 2019-2020 Reference Implementation was configured in accordance with current draft FICAM policies. Currently, ADL Initiative engineers are working with personnel from the Center for Development of Security Excellence to validate those TLA FICAM specifications.

Finally, given the ever-growing proliferation of data systems and learning services, an enterprise learning ecosystem is envisioned to have a high throughput, with millions of daily data interactions. Hence the TLA needs to accommodate a high volume of data. But the point-to-point data communications used in the ADL Initiative’s 2018 TLA reference implementation were not sufficient for scaled-up TLA implementations, so in 2019, the team reconfigured the architecture to a publish/subscribe data messaging approach using Kafka, an open-source stream-processing software. This streaming-data architectural design proved to be a more scalable and effective approach.

TLA Reference Implementation Components

The 2019-2020 TLA Reference Implementation, leveraging lessons learned in 2018, included seven core components; these are:

• Competency Management System - Services that manage evidence of individual or team competencies, verifying chains of evidence, handling the associated trust functions, generating authoritative assertions of capability (credentialing or de-credentialing), and making estimations of competence based on granular or inferential data.
• Learning Event Management System - Services concerned with scheduling learning activities and capturing data output from those activities. This output is relayed to individual learner profiles. Scheduling learning activities may involve a scheduling service, potentially aided by a recommender or content curation service.
• Activity and Resource Management System - Services for registering and maintaining references to the location of learning resources as well as the verification of the resources required to access them. Key features include the ability to generate and manage metadata, the alignment of content with competencies, paradata, and semantic search services.
• Learner Profile - Learner records, with user data and locally generated assertions of competence housed in local enclaves. Competency or credential evidence earned locally is stored initially in a local learner record and can be forwarded to other systems. Federated competency/credential data from non-local systems can be ingested into the local learner profiles.
• Competency Registry - Data describing competencies, their granular components, and relationships to prerequisites and co-requisites are encapsulated into Competency Frameworks. These frameworks relate competency data with measures of performance and effectiveness, based on demonstrated levels of mastery for specific jobs or duties. These are stored in the registry for use by other connected systems.
• Learning Record Store - The LRS is the server-side component of the xAPI specification. A noisy LRS collects all xAPI statements that are generated by a learning activity. A transactional LRS collects “normalized” xAPI statements that are generated using the TLA’s Master Object Model. And an authoritative LRS collects and stores the validated assertions of learner competency.
• Experience Index - An Experience Index stores metadata about available learning activities, providers, and their available learning content. An Experience Index also stores information on the relationships among content, competencies, activity and content metadata, and evaluated paradata.

Relatedly, in 2019 the ADL Initiative began two projects to support this work, including the development of a DoD-wide Enterprise Course Catalog and an Enterprise Learner Record Repository. Also, the program continues to mature its Competency and Skills System (CaSS) to include cybersecurity hardening and the addition of more usable competency authoring tools.

Applied Testing of TLA-based Architectures with DoD Stakeholders

The TLA project is helping to forge a path toward an enterprise-wide DoD digital learning ecosystem. However, even after the TLA specifications are complete, much work will be needed to implement that architecture within DoD’s legacy systems. To examine implementation best practices and support adoption, the ADL Initiative is working with DoD stakeholders to test the integration of TLA components and standards into their systems. These collaborations provide the opportunity to demonstrate TLA concepts in real-world DoD environments and to potentially identify technology and policy challenges.

• In 2018 and 2019, Working with the Office of the Undersecretary of Defense for Intelligence (OUSD(I)), the ADL Initiative examined deployment of TLA standards and systems within intelligence community (IC) training and education organizations. The resulting Talent Development Toolkit (TDT) will enable an enterprise approach to data and content management across the IC.
• Starting in 2019, the ADL Initiative began supporting the Army’s Combat Capabilities Development Command with a different challenge: the design and development of a Squad Performance Model and a prototype capability to track and measure individual and team competencies. The project will enable the Army to track and measure individual and team competencies across the service’s Synthetic Training Environment, including training aids, devices, simulators and simulations. This work also leverages the CaSS and DATASIM projects previously developed through the ADL Initiative.
• The Defense Health Agency (DHA), with support from the Naval Air Warfare Center Training Systems Division (NAWCTSD), is modernizing the Military Health System (MHS) learning infrastructure. DHA is responsible for the management of the MHS, including all military hospitals and clinics in the United States and dozens of disparate healthcare education and training programs across the Army, Navy, and Air Force. NAWCTSD is supporting the DHA by defining requirements and a technology roadmap for a DHA-wide TLA that integrates all these education and training programs into an interoperable system of systems. The ADL Initiative will continue this collaboration through 2020 and potentially beyond.
• In a collaboration with USALearning, the ADL Initiative is supporting the testing and evaluation of TLA components for government stakeholders including Air Education and Training Command, the Defense Acquisition University, and others. This work includes a digital learning sandbox that supports testing and evaluation of different approaches for integrating TLA tools, technologies, and standards into existing learning technology stacks. This effort also uses the DATASIM project to stream simulated learner data (i.e., xAPI statements) to emulate the scope, scale, and breadth of learners, learning activities, and competencies.

• 

  Finally, building on the TDT work with OUSD(I), the ADL Initiative has been working with the Center for Development of Security Excellence, a component of the Defense Counterintelligence and Security Agency, to develop xAPI policy guidance that aligns with FICAM policies. This work will benefit all of DoD by enabling a Defense-wide approach for accessing, sharing, and protecting learner data across organizational boundaries. It also involves the development of a preferred approach for protecting personally identifiable information encoded in xAPI statements.

The 2019 TLA report is available from the ADL Initiative. DoD organizations are encouraged to contact the ADL Initiative for more information, including demonstrations and implementation assistance.