Total Learning Architecture (TLA)

What’s the TLA?

The TLA is a research and development project sponsored by the ADL Initiative and conducted in collaboration with stakeholders from across the defense community, professional standards organizations, industry, and academia. It includes a set of technical specifications, standards, and policy guidance that define a uniform approach for integrating current and emerging learning technologies into a learning services ecosystem. Within this ecosystem, multiple services and learning opportunities (of various modalities and points of delivery) can be managed in an integrated, interoperable “plug and play” environment.

Once matured and implemented, the TLA will enable personalized, data-driven, and technology-enabled lifelong learning across the DoD, other Federal Government agencies, and beyond.

This video – “Out-Learn, Out-Think, Win: Future Learning and Development” – shows the possibilities in future learning ecosystems, such as the TLA, as well as potential for efficiencies in training and mission successes when the management of our talent and learning are data-driven.

Rationale

In today’s DoD, personnel must develop an ever-expanding set of sophisticated, agile knowledge and skills—without significantly increasing time or costs for education and training. Advances in learning science and technology, ubiquitous access to the Internet on a growing variety of devices, and new capabilities for data analytics are enabling new approaches for DoD education and training.

Military Services and other DoD organizations have generally adopted a shared vision for a future learning ecosystem comprised of interconnected learning opportunities, supported by technology, driven by data, and integrated with other talent management capabilities.

A servicemember or DoD worker progressing through their career will have access to hundreds of formal and informal learning opportunities, tailored to their goals and based on their existing competencies. Teams can be assembled with a specific skills mix, and managers will know exactly what additional training they may require to support a mission. Distributed learning courses, device-driven micro-learning activities, and traditional learning management systems will be interoperable, using common data standards. Terabytes of learner-related data will be distributed throughout this interconnected ecosystem, available for leaders to make data-driven decisions on investments and force structures to optimize readiness while minimizing costs.

This envisioned “continuum of learning” includes features such as the following:

• Continuous: Career-long, continuous learning replaces the status quo’s stovepipe, episodic learning
• Blended: Formal education and training, just-in-time support, and informal learning are integrated
• Enterprise Focused: Education, training, and talent management are considered in concert, holistically
• Diverse: Disparate learning technologies and methods are interoperable within a cohesive ecosystem
• Learner Centric: Learning adapts to individual and team needs, contexts, and characteristics
• Data Driven: Learner data from across many sources are aggregated and analyzed to drive decisions
• Competency Based: Competency frameworks support assessment and guide developmental trajectories
• On Demand: Modular education and training can be delivered at the point of need
• Cloud Based: Software services and network-based repositories support flexibility and discoverability

TLA Focus Areas

Concerted development of TLA components largely began in 2016 and culminated with empirical testing in 2017. Throughout the development process, the ADL Initiative adopted a multiyear “spiral” design-based research approach. In 2019 the focus was on development and refinement of the formal requirements, specifications, and architectural design. The TLA design is now aligned with modern education and training architectures favored by its stakeholders, with a streaming data architecture emphasizing the articulation of content metadata, persistent learner profiles, and competency data definitions. The approach builds a foundation for future capabilities that will leverage adaptive artificial intelligence and machine-learning algorithms.

Multiple TLA components are now being tested and validated in collaboration with various DoD organizations:

Data Strategy Standards Development

The TLA data strategy provides a common set of goals and objectives across DoD’s education and training community to ensure data are used effectively. Common data standards, business rules, governance rules, and policies are the TLA ingredients required for the interoperability of DoD education and training systems. This interoperability will ensure that all data resources can be used, shared, and moved efficiently across multiple DoD organizations.

Data standards must be broadly accepted throughout the user and developer communities. The ADL Initiative is working with the Institute of Electrical and Electronics Engineers (IEEE), an internationally recognized standards-development organization, to formally establish the data standards required for successful TLA implementation:

• IEEE P9274.1 Experience API (xAPI) 2.0 – Learning activity tracking uses the xAPI to capture learning activity streams. The xAPI standard also includes xAPI Profiles such as cmi5 and the TLA’s Master Object Model (MOM). xAPI 2.0 is targeted for approval in 2020.
• IEEE 1484.12.1 Learner Object Metadata 2.0 – Descriptions of learning activities and their associated content are stored in the TLA’s Experience Index and use a modified version of the Learning Resource Metadata Initiative standard. A draft standard is being submitted for finalization in early 2020.
• IEEE 1484.20.3 Sharable Competency Definitions – The definition of a competency, the relationship to other competencies, and the alignment of evidence to help measure proficiency of the competency, are included in this standard. This standard is expected for approval in 2022.
• IEEE 2997 Enterprise Learning Record – Learner profile standards do not currently meet all TLA requirements. These new standards are actively being developed and modified based on input from numerous industry groups and associations.

TLA DevSecOps Pipeline

Bridging the research-to-practice divide has been a challenge for distributed learning technologies. Transitioning R&D efforts into operational systems is hampered by resource constraints, security concerns, and inconsistent approaches to test and evaluation. The ADL Initiative’s DevSecOps pipeline takes an idea or concept, matures it into a capability, transitions it through Information Assurance protocols, and houses it using containers so that it’s easy to install and configure.

At the end of this development pipeline is the Learning Technology Warehouse portal. This is a secure, cloud-based repository where DoD and Federal Government education and training stakeholders can acquire distributed learning tools and technologies for testing and evaluation. This portal provides front-end services, secure authentication, enhanced workflow to aid transitioning of tools, and support resources for testing activities.

TLA Reference Implementation (Prototype)

The TLA Reference Implementation is a process for investigating specifications and standards to determine their applicability and suitability for the future learning ecosystem. While existing specifications and standards are considered a starting point for the development of the TLA, the ADL Initiative is investigating additional specifications and standards for continued interoperability as new technologies are developed. Within the TLA Reference Implementation, candidate standards and specifications are analyzed to determine whether they are partially or entirely appropriate for meeting the TLA’s various data integration requirements. If changes are needed to extend the specification/standard, this work is documented to inform future specification/standard development bodies.

The 2020 TLA Reference Implementation work is centered around achieving the vision of learning with “any device, anytime, anywhere.” This requires understanding how devices are to be discovered, connected, secured, identified, and instrumented to generate understandable learning data, while maintaining the loosely coupled nature required of a true ecosystem. This effort includes completion of software for the testbed and the incorporation of cybersecurity, and network management for devices, generally prescribed in the National Institute for Standards and Technology (NIST) 800 series national standards for computer security. “Any device anywhere” specifically suggests an approach called the “zero trust network,” which ensures user identity and system security assuming no physical control over the origin or location of the device.

TLA Sandbox

The TLA Sandbox enables the testing, evaluation, and demonstration of next generation learning tools, technologies, and capabilities. Established in collaboration with the Office of Personnel Management’s (OPM) USALearning, the TLA Sandbox provides a shared infrastructure for other Federal and DoD organizations to test and evaluate their modernization strategies and individual components. It lowers the barrier to entry by enabling a shared DoD resources that emulates the operational systems in use today. The TLA Sandbox is available to support various modernization efforts across the DoD and other agencies with a range of complexity.

Conformance Testing

In addition to the functional testing of tools within the TLA Sandbox, the data products they produce must conform to a variety of specifications and standards. This requires various methods of conformance testing, including for the structure of learner records, xAPI Profiles, and other data. The ADL Initiative is developing a suite of conformance test products for this.

One example is conformance testing to ensure online learning content designed to Shareable Content Object Reference Model (SCORM®) standards remains functional in a more interoperable environment than it was designed to support. The ADL Initiative developed the cmi5 specification to ensure SCORM content can be used in broader TLA-compliant environments, and a corresponding cmi5 Conformance Test Suite to ensure its compatibility.

Testing and Evaluation of TLA through Implementation

Development of the TLA includes a rigorous, independent efficiency and effectiveness assessment. A comprehensive assessment is comprised of the draft specifications and associated reference implementation model, scoping test and demonstration events, developing and executing the research assessment plans, and reporting findings (as appropriate) in conferences, technical reports, and targeted academic journals. This project includes participation in ADL Initiative-sponsored committees and working groups, as well as other specifications and standards working groups, to illustrate current and emerging capabilities and eventual contributions to DoD operations and the ADL Initiative mission.

As new learning approaches move from concept to practice, the TLA has become the foundation for education and training modernization efforts by multiple DoD organizations.

• The ADL Initiative is placing an Air Force Learning Services Ecosystem (AFLSE) sandbox within the TLA sandbox to test and evaluate the interoperability of TLA components with ongoing Air Education and Training Command (AETC) efforts.
• The Defense Health Agency (DHA) is modernizing their learning infrastructure into a TLA-compliant ecosystem. The Sandbox in 2020 is expected to provide DHA with the computing infrastructure to test and evaluate their own TLA implementation. Beyond the shared computational resources, the lessons learned and operational experience from working together is expected to expedite the maturity of key TLA systems.
• The Army Futures Command, Combat Capabilities Development Center (CCDC) is leveraging the TLA Sandbox to support a combined effort that uses the TLA to inform the Army’s Synthetic Training Environment (STE) and Army Training Information Systems (ATIS) programs.
• The Office of the Under Secretary of Defense for Intelligence (OUSD(I)) is working with the ADL Initiative on a Talent Development Toolkit (TDT) for the DoD intelligence community (IC). This addresses the challenges for exchanging learner-related data among 17 different IC organizations.
• The Center for Development of Security Excellence, a component of the Defense Counterintelligence and Security Agency (DCSA), is using the TLA Sandbox to expedite testing and evaluation of different approaches for enabling Federated Identity, Credentials, and Access Management (FICAM).

Projects

Competency Based Learning / Competency and Skills System (CaSS)

The CaSS project focuses on the research, design, and development of services that enable competency frameworks to be leveraged within the TLA. The CaSS enables collection, processing, and incorporation of credentials and data (“assertions”) about an individual’s competencies into accessible, sharable learner profiles. CaSS will create an infrastructure enabling competencies, competency frameworks, and competency-based learner models to be managed and accessed independently of a learning management system, course, training program, or credential.

Data Analytics and Visualizations Efficiently (DAVE)

The DAVE project explores the use of a novel assessment tool in the context of the TLA, which helps inform the maturation of the “learning ecosystem” concept by generating unique learner data. DAVE extends xAPI within the TLA by developing models, prototypes, and specifications for analyzing, interpreting, and visualizing data.

DATASIM

DATASIM is an open source application that will provide a valid means of producing the datasets necessary to benchmark and stress test the TLA and distributed learning acquisitions. Additionally, DATASIM can help learning scientists, Instructional System Designers, IT and technology acquisition, and decision-making stakeholders to determine the effectiveness of xAPI data design and implementation across the TLA.

Privacy Support for the Total Learning Architecture (PS4TLA)

The PS4TLA project investigates user-tailored privacy to support TLA users’ privacy decisions. The user-tailored privacy methodology accounts for the high variability and context-dependency of people’s privacy decisions by creating a personalized, context-dependent model of users’ privacy preferences, and then tailors users’ privacy settings to these modeled preferences.

Fast Learning from Unlabeled Episodes for Next-generation Tailoring (FLUENT)

FLUENT is an adaptive recommendation system that evaluates xAPI Statements within a Learning Record Store (LRS) to collect details about learning interactions across different sequences of learning events. These “Learning Episodes” include specific information about learning events, their effectiveness for different types of users, and the context of each learning event (e.g., the string of learning events that preceded it). FLUENT provides a platform to build on, and mature, the TLA by identifying, implementing, and evaluating candidate standards and specifications required to interoperate with other TLA components.

Talent Development Toolkit (TDT)

The TDT project was conducted within the Office of Deputy Assistant Secretary of Defense for Force Education and Training (DASD(FE&T)), in collaboration with OUSD(I)’s Human Capital Management Office. The project leverages the TLA to establish the policy, standards, and specifications for IC learning ecosystems. Engineering requirements and architectural designs were developed for the TDT’s data specifications (e.g., learner profiles) and core services (e.g., content management). It uses the general TLA framework to improve the quality of personnel readiness through optimized talent development, the quality of workforce planning through data analytics, and the long-term sustainability of learning systems by reducing dependency on single-vendor solutions.

Publications

News