Total Learning Architecture (TLA) Sandbox

Background

In 2019, the ADL Initiative established a TLA Sandbox project in collaboration with the Office of Personnel Management’s (OPM) USALearning to accommodate technical experimentation with the 2019 TLA Reference Implementation. This Sandbox included Amazon Web Service (AWS) assets outside of the OPM accreditation boundary, servers to host the Navigator for Integrated Learning Experience (NILE) project, several different Learner Record Providers (e.g., a Learning Management System (LMS)), and additional internal services (e.g., competency and activity management).

Purpose

As shown in Figure 1, the TLA Sandbox was configured to support numerous experiments, as well as host the TLA microservices and systems used in the 2019 architecture, including the Competency and Skills System (CaSS), Moodle, Kafka, and a Learning Record Store (LRS). CaSS provided the competency framework and generated signals in place of a full competency management service (pending completion of two interdependent projects). Apache Kafka provided a publish/subscribe streaming architecture instead of a point-to-point polling-messaging topology as used in 2018. The migration to a data streaming architecture stemmed from lessons learned in 2018 and a need to re-architect the TLA Reference Implementation to better support the scope and scale of education and training across the Department of Defense (DoD).

Figure 1. Physical Architecture Driven by the TLA Sandbox. The 2019 TLA Sandbox consisted of 22 servers: 5 were used to host the 2019 TLA Reference Implementation’s core services, 10 were used to support the NILE project, and 7 were used to host various Learning Record Providers (LRPs).

The 2019 Reference Implementation used a production pipeline that took advantage of Docker-based containerization to streamline the deployment of software changes. This approach shortens the time required to commit software changes to any TLA component and to integrate the updated component with other TLA components. The server instances within the TLA Sandbox communicate between themselves using either HTTP/S over TCP/IP or by producing and consuming messages to the centralized Kafka cluster, internal to the TLA Reference Implementation.

The TLA Sandbox supports future experimentation with ADL Initiative stakeholders by providing client-side access points for integration with different LRPs (i.e., different learning-delivery systems such as LMSs or ebooks) or separate TLA enclaves. Stakeholder systems and other LRPs may be located outside the AWS campus and connect via Representational State Transfer (REST) protocols. Integration of a web portal front-end is planned in 2020. This will improve the usability of the TLA Sandbox for testing and evaluation, including experimentation for ADL Initiative projects (e.g., federated data strategies, federated identity management) and TLA-compliant modernization initiatives by other stakeholders.

The ADL Initiative currently hosts several LRPs within the TLA Sandbox, including the Perceptual Adaptive Learning Modules (PALMs), PERvasive Learning Systems, Personalized eBook for Learning (PeBL), and NILE. Table 1 shows the current server allocation within the TLA Sandbox. As additional systems are integrated and its maturity progresses to operational testing, the Sandbox will be expanded to include both a test and production environment. The PERLS and PeBL systems already use multiple servers for this purpose.

Table 1. TLA Sandbox Server Utilization

Results and Lessons-Learned

Message Topology: The TLA Sandbox supported numerous experiments related to the re-architected TLA Reference Implementation. Experimentation around the Kafka system resulted in a census of data topics (streams) used by TLA services to publish and subscribe data. The research also showed that TLA components could communicate across different TLA enclaves using this approach.

Cybersecurity: The Moodle LMS with its current Experience Application Program Interface (xAPI) plugin was installed and configured to run within the TLA Reference Implementation enclave. To mirror DoD FedRAMP requirements, Moodle’s front-end web portal had to be separated from the databases used to store data.

Federated Data: An Experience Index is a shared metadata repository used to describe learning experiences (e.g., courses or other learning resources) within an organization. TLA Sandbox servers were used to test how the TLA’s Experience Index would handle a federation of experience indices across domains. An instance of the 2019 TLA Experience Index was hosted in the TLA Sandbox while another was hosted on a separate TLA instance. Both were queried at the same time and the response times and dropped messages were logged; the results demonstrated that the data model and database design will federate cleanly. This work supports the Enterprise Course Catalog line of effort outlined by the DoD Chief Management Office’s reform initiative¹.

Federated Identity: The TLA Sandbox was also used to test a scalable method for using graph databases and third-party identity management providers to construct and query a network of an individual’s multiple personas (name aliases used for a single individual to log-in to different network domains). A testing harness was configured to load thousands of users into the TLA Reference Implementation and send xAPI statements from each of those users (aliases) to the Kafka-integrated LRS.

The federated identity service was evaluated by how quickly it could recognize a given user’s alias and return that user’s unique master account. After indexing the alias network with the ID provider and alias pairing, the system operated in near-constant time for up to 160,000 aliases, the highest count tested. It could construct a full alias mapping in ≈14ms. This process was so efficient that the computational overhead of the HTTP requests started to introduce noise into the analysis. The experiment demonstrated that a graph database can be used to federate multiple identity management systems using minimal resources.

Core/Edge Decoupling of Systems: A 2019 TLA research objective was to segregate between core systems and data stores (e.g., competency management, learner profile), and edge systems (e.g., learning activities, intelligent tutoring systems). The TLA core systems provide the services and data necessary to manage and deliver education and training across an organization. The TLA defines edge systems as independent systems that sit on the outside periphery of the TLA core.

Research also focused on the communication protocols between core/edge systems. The TLA Master Object Model (MOM) was developed to describe the topology of xAPI data streams. The MOM is used to normalize data originating from different edge systems across the learner’s lifecycle. For example, the 2019 Reference Implementation moved the LMS to the edge so that it was perceived the same way as any other Learning Record Provider within an organization. The LMS tracks learners and communicates with its learning content using the Sharable Content Object Reference Model (SCORM) but uses the TLA MOM to communicate learner performance to the TLA core systems, in this case the transactional LRS that feeds the CaSS competency management system.

xAPI Integration: A SCORM course for Faculty Professional Development (FPD 420) was integrated into the 2019 TLA Reference Implementation in collaboration with the Defense Acquisition University (DAU). First, the course was updated to incorporate xAPI functionality using a TLA-compliant xAPI wrapper, and then it was implemented using the TLA Sandbox’s Moodle LMS.

The TLA Sandbox was used to analyze DAU’s xAPI Content Model, which provided a comparison of xAPI with the SCORM data model and allowed detailed insights on best practices to share moving forward. This research showed that many xAPI users view the xAPI specification only as the Actor-Verb-Object model. These users generally failed to grasp other potential xAPI benefits such as the Context property and how it can add value to all generated data. This work sets the stage for maturing the DAU course and documenting the process, lessons learned, and best practices for maximizing value when using the xAPI specification with traditional SCORM content.

Next Steps

The TLA Sandbox was established to allow testing, evaluation, and demonstrations of next generation learning tools, technologies, and capabilities. The development, test, and production environments enable a DevOps approach to mature the tools, technologies, and technical underpinnings (e.g., standards, implementation guidance, business rules) that are being invested in to demonstrate the capabilities enabled by this unified approach to data interoperability.

The 2019 TLA Reference Implementation provides a shared infrastructure for other Federal and DoD organizations to test and evaluate their modernization strategies. It lowers the barrier to entry by enabling a shared DoD resources that emulates the operational systems in use today. In 2020, we expect additional research and experimentation in partnership with other government organizations.

• The Army Futures Command, Combat Capabilities Development Center will leverage the TLA Sandbox to support a combined effort that uses the TLA to inform the U.S. Army’s Synthetic Training Environment (STE) and Army Training Information Systems (ATIS) programs. This work will also build upon the ADL Initiative’s research into Competency-Based Learning by looking at team competencies in support of the Squad Performance Model initiative out of the U.S. Army’s PEO Soldier.
• The Center for Development of Security Excellence, a component of the Defense Counterintelligence and Security Agency (DCSA), will continue using the TLA Sandbox in partnership with the ADL Initiative to expedite testing and evaluation of different approaches for enabling Federated Identity, Credentials, and Access Management. While all approaches will follow DoD Chief Information Officer (CIO) and Defense Information Security Administration (DISA) guidance, this work focuses on approaches for ensuring privacy and security of xAPI data by aligning the actor field in the statement with an anonymized token that can be reconciled by an authoritative DoD system.
• The ADL Initiative is placing an Air Force Learning Services Ecosystem (AFLSE) sandbox within the TLA sandbox to test and evaluate the interoperability of TLA components with ongoing Air Education and Training Command (AETC) efforts such as the Airmen Learner Record, Competency-Based Learning efforts using CaSS, and the integration of other TLA components such as the Experience Index and an LRS. In 2020 this work will culminate in a 6-month data collection event that can be analyzed to inform the transition of CaSS into an operational system.
• The Defense Health Agency (DHA) is modernizing their learning infrastructure into a TLA-compliant ecosystem. The Sandbox in 2020 is expected to provide DHA with the computing infrastructure to test and evaluate their own TLA implementation. Beyond the shared computational resources, the lessons learned and operational experience from working together is expected to expedite the maturity of key TLA systems.

The ADL Initiative continues to seek new government partners to work with and learn from. The TLA Sandbox is available to support a range of modernization efforts across the DoD and other agencies, and with a range of complexity from evaluating the xAPI specification and using an LRS to the migration toward Competency-Based Learning and Adaptive Instructional Systems. Organizations interested in working with the ADL Initiative on this or other digital learning modernization projects can reach out here to start the conversation.