# Introduction

The Total Learning Architecture (TLA) is a research and development project sponsored by the ADL Initiative (opens new window) and conducted in collaboration with stakeholders from across the defense community, professional standards organizations, industry, and academia. It includes a set of technical specifications, standards, and policy guidance that define a uniform approach for integrating current and emerging learning technologies into a learning services ecosystem. Within this ecosystem, multiple services and learning opportunities (of various modalities and points of delivery) can be managed in an integrated, interoperable 'plug and play' environment.

The TLA Sandbox is a testing environment built around the TLA Reference Implementation that includes core systems for managing and transforming learner data into meaningful information. The sandbox includes the minimum functionality needed to accurately test software systems, learning activities, instructional strategies, and other related information systems for compatibility with the TLA's data interoperability standards. Additional information on the technical underpinnings of the TLA Reference Implementation including various DoDAF views, the System/SubSystem Design Documentation, and benchmarks used in its development are included in the 2019 TLA Report (opens new window).

# Document Scope and Intended Audience

The intended audience for this document is developers or engineers who are familiar with TLA standards. Readers are not required to know all the details about each standard. However, some knowledge of xAPI basic concepts is advantageous for reading this document. Readers should also have a fundamental understanding of how data is exchanged across the internet including how to represent and exchange data using JavaScript Object Notation (JSON), and how to access RESTful API's on the web. Server-side Javascript experience using web frameworks such as Node.js and NoSQL databases (e.g., Mongo DB) are also recommended.

The purpose of the TLA Reference Implementation is to enable DoD stakeholders to test, evaluate, or stand up their own TLA sandbox implementations in a way that exposes TLA data so that it may be shared across components and organizations. This document covers the specific implementation details of the ADL Initiative's TLA Reference Implementation and its various components, connections, data consumers, and data producers. It demonstrates how ADL engineers have implemented the various TLA tools, components, and technologies to demonstrate the data transformations required by the TLA. However, this is only one possible way of implementing these capabilities.

# Understanding the Total Learning Architecture (TLA)

The TLA project started in 2016 with the strategic vision of establishing a common data strategy across the education and training industry that enables lifelong learning. This goal required a multi-faceted understanding of the tools, technologies, modalities, and learning science used to support education and training within different communities of practice.

Learning takes place both inside and outside the classroom. The digital world around us creates a web of learning resources that are accessible in the workplace, from home, through social circles, and on a growing variety of media and devices. The exponential growth of data generated by these systems has the potential to enable better insights while reducing costs through continuous process improvement across all education and training activities. The next generation of learning activities cannot be defined within the context of a single LMS. In today’s world, an individual’s lifelong learning continuum is distributed across numerous technology platforms that use different instructional methodologies and learning activities. To fully realize the vision of lifelong learning, the future learning ecosystem will be defined by personalized and competency-based learning environments that rely on the availability and accessibility of learner data across organizational and institutional boundaries.

Learner data is a critical asset that enables effective decision making for both trainees to identify gaps in competencies, and for organizations to track employee capabilities across emerging needs. The key to managing lifelong learning data within the TLA is the interoperability afforded through the technical standards, specifications, and practices that underpin an integrated data strategy. The TLA Data Strategy is necessary to provide the semantic interoperability required for enterprise-level analysis and decision support. Data-driven decisions are enabled through enterprise-level analyses of learning data, supporting the continual refinement of occupational skills and the creation, selection, and maintenance of learning activities necessary to achieve proficiency.

The TLA Data Strategy provides a common set of goals and objectives across DoD’s education and training community to ensure data are used effectively. This overarching strategy will ensure that all data resources are positioned in a way that they can be used, shared, and moved efficiently across the organization. The ADL Initiative is working with the Institute of Electrical and Electronics Engineers (IEEE), an internationally recognized standards-development organization, to formally establish the data standards required for successful TLA implementation. While these standards will continue to evolve, DoD education and training communities are urged to adopt and employ them now. These commercial standards describe the data within the four pillars of the TLA Data Strategy:

  • IEEE P9274.1 Experience API (xAPI) 2.0 – Learner performance tracking within different learning activities use the Experience API (xAPI) to capture learning activity streams. This standard defines how learner performance is captured, communicated, and shared via a Learner Record Store (LRS), the server-side implementation of xAPI. The xAPI standard also includes xAPI Profiles (opens new window) such as cmi5 and the TLA’s Master Object Model. xAPI 2.0 is targeted for approval by IEEE in 2021. Additional information about xAPI can be found at the IEEE's Technical Advisory Group (opens new window).

  • IEEE P2881 Learning Activity Metadata – Descriptions of learning activities and their associated content are stored in the TLA’s Experience Index (XI). This draft standard builds upon IEEE 1484.12.1 Learning Object Metadata (LOM) to increase the granularity of how learning resources are defined. It was developed by harmonizing with other educational data standards such as the Council for Educational Data Standards (CEDS), the Postsecondary Educational Standards Council (PESC), Credential Engine's Learning Opportunity Type, the Learning Resource Metadata Initiative (LRMI), and Schema.org. This draft standard also includes numerous data types and properties that were derived from MILHDBK 29612, TRADOC FM 350-70, and the USAF 36-2235 Instructional Systems Design guidebook.

  • IEEE 1484.20.1 Reusable Competency Definitions (RCD) - The RCD standard enables a common approach for describing competencies, aligning competencies to related competencies in the context of a framework, and defining the assessment and evaluation criteria for the evidence a learner must demonstrate to help measure proficiency. This standard is being designed to facilitate a common language for describing the knowledge, skills, abilities, and other behaviors (KSAOs) required for performing different jobs, duties, and tasks associated with an occupational specialty. Competencies provide a common approach for aligning education and training activities to the desired operational performance expected from learners to perform with proficiency. Additional information about RCDs can be found in the Competency Data Standards Work Group (opens new window).

  • IEEE Enterprise Learner Records - This draft standard is built around a data model created by the ADL Initiative to meet stakeholder requirements on the Enterprise Learner Record Repository (opens new window). this model was developed using the T3 Innovation Network's Learning and Employment Records (LER) Resource Hub (opens new window) and the LER Mapping Tool located under T3 Network Tools. The data model builds upon the work performed by the T3 Innovation network to meet the evidentiary requirements (e.g., storage of raw learner data for a period of time) that many DoD organizations adhere to. It also supports future artificial intelligence / machine learning solutions that enable instructor support tools, intelligent tutoring, and additional insight into each learner that can be used to optimize and tailor their continuum of learning.

Ongoing TLA research is centered around achieving the vision of learning with “any device, anytime, anywhere.” This requires understanding how devices are to be discovered, connected, secured, identified, and instrumented to generate understandable learning data, while maintaining the loosely coupled (opens new window) nature required of a true ecosystem. This work requires adherence to the DoD's Identity, Credential, and Access Management Services (opens new window) and the work required to support NIST Zero Trust Architecture (opens new window) that was finalized in August 2020. The ADL is currently working through the development and testing paths to ensure proper DoD authentication, authorization, and accountability is in place to establish and provide attribution of unique identifiers for all DoD personnel.

# TLA Maturity Levels

Figure 1 Figure 1 - TLA Capability Maturity Model

The ADL Initiative's Capability Maturity Model (opens new window) (CMM) shown in Figure 1 provides a thorough description of maturity, based on the context of policies, instructional design processes, technology infrastructure, and governance of it's adherance to the TLA's data interoperability standards. More importantly, the model can be used by an organization to quantify their current maturity and to highlight areas where improvement is needed to help guide future investments.

  • Level 1 - The CMM allows for the gradual migration of legacy systems to a microservice-based infrastructure of core services that federate data across other technology components. Organizations migrating towards TLA interoperability start at Level 1 by using learning activities that have been instrumented with the xAPI standard. This level of maturity requires an organization to begin moving away from SCORM®-managed learning environments in favor of solutions where learner performance is stored within a centralized Learner Record Store (opens new window) (LRS). The LRS enables analytics that go beyond the statistics typically available within an LMS. ICAM is managed locally by connected systems and there is minimal ability to track learners across the different systems they interact with.

  • Level 2 - Level 2 supports the aggregation and analysis of learning data at the enterprise level. It may include numerous LRS solutions that are connected to any number of learning activities. This requires enterprise level services and the adoption of governance procedures to standardize learner identity, data labeling, and data reporting to enable enterprise analytics. Level 2 introduces an enterprise approach to ICAM, which enables performance tracking for learners across numerous disparate systems. This level of maturity also introduces the concept of a course catalog for describing courses, activities, and other learning resources within the organization. Learner performance is tied to catalog descriptions using the TLA Master Object Model (MOM) and each activity's catalog identifier. A single organization may have multiple different course catalogs that are used (e.g., to support different types of learning). For example, an organization may have a course catalog that resides inside its LMS, a website with a listing of serious games, or a simulator system that includes a scenario library. Each catalog system at this level is disconnected from the other catalogs.

  • Level 3 - Level 3 maturity introduces a common standard for describing all education and training resources within the organization. Adherence to the IEEE's draft P2881 Learning Activity Metadata standard promotes data interoperability between disconnected catalog systems. An API is used to federate disconnected course catalogs into a single organizational course catalog. Metadata aggregation services are used to automatically derive metadata attributes from other connected systems. This information is stored within an Experience Index (XI), the server-side component of the draft P2881 data model. The XI is designed to work with other TLA systems that require additional knowledge about the different learning activities students interacted with. Another key differentiator this level of maturity includes is the alignment of learning resources to competencies and credentials. Each organization may have numerous competency registries that store definitions of competencies. Registries may or may not be connected to a Competency Management System (CMS).

  • Level 4 - Level 4 formalizes the way competency-based learning takes place in the organization and establishes a learner profile that pulls learner data from different learning activities (e.g., a Learning Management System) via the TLA MOM and a federated LRS structure. This results in the preservation of learner performance data (i.e., the evidentiary chain) where raw performance is stored at the local level (e.g., with a learning activity) but is normalized and rolled up into extrapolated learner data stored in a transactional LRS. Learner performance is aligned to workplace (e.g., operational) competencies via a CMS using the IEEE 1484.20.1 standard for Reusable Competency Definitions. This level is characterized by the enterprise strategy for managing competencies that enables global competencies to be tailored to meet local contexts. Organizational credentials are aligned to the competencies they represent, and these are linked to the learner profile. Authoring and alignment between local definitions is required to ensure local tasks, conditions, and standards are encapsulated in the definitions of local competencies.

  • Level 5 - Level 5 is the highest level of maturity. This level of TLA adoption is characterized by its lifecycle approach to the Human Capital Supply Chain. Enterprise learner records are tied to Human Resource, Talent Management, or other Manpower & Personnel systems to drive career management, workforce planning, and mission readiness. Competency management is tied to operational performance goals and key performance indicators in the workplace. Interconnected systems and interoperable data enable adaptive instructional systems and automation to improve the efficiency of how people learn. These levels represent the objective state of policy, technical specifications, and standards that will enable the future learning ecosystem. The standards that comprise the TLA technical specification act as a “distributed ledger” for learner data that is globally discoverable and usable across the DoD enterprise.

# How this Document is organized

The TLA Standup Guide follows the TLA Maturity Model and walks the reader through the different systems, settings, and tools used within each level of maturity. This allows organizations to test their own learning activities, tools, systems, and workflows to evaluate their current TLA maturity, and their ability to connect to other TLA-enabled systems.

The current TLA Reference Implementation is working toward CMM Level 3 conformance. CMM Levels 4 and 5 are not yet addressed in this quick-start guide.