Toward the Harmonization of Learning Activity Metadata

Across DoD and the broader education and training community, organizations use many different and often incompatible methods to publish and describe their courses, learning activities, and instructional resources. This results in hundreds of proprietary and unconnected catalog descriptions, causing a lack of awareness about available learning opportunities, and duplications of effort when organizations develop learning activities that are similar to others already available.

Part of the solution for DoD is to build an Enterprise Course Catalog that federates local course catalogs into a single portal. To do this effectively, a new standard is required to harmonize the metadata associated with learning activities, allowing interoperability across the catalog systems that host them. This is addressed by the P2881 Learning Metadata Standard, which is now being designed by a working group of the Institute of Electrical and Electronics Engineers (IEEE), that includes the ADL Initiative and other DoD stakeholders.

In its development of the ECC prototype, the ADL Initiative has established a metadata model that defines requirements that could be used for the new P2881 standard. The model, which seeks to capture the attributes and relationships across a wide array of education and training assets, borrows from an existing standard, the IEEE 1484.12.1 Learning Object Metadata (LOM) standard. However, the LOM standard did not fully meet the ECC requirement.

Why do we need a new metadata standard for learning content?

Existing standards like IEEE 1484.12.1 were implemented at a time when distributed learning was largely limited to traditional online courses that could be described by a small number of attributes. In contrast, today’s digital learning environments feature learning activities on an ever-growing number of platforms and formats. This diversity, combined with demands for greater interoperability, and advances in artificial intelligence-driven data analytics, creates the need for more, and more detailed, learning content metadata.

The P2881 standard will accommodate metadata from a larger variety of learning paradigms, practices, and modalities, including new types of learning technologies (e.g., biometrics, sensors). The P2881 standard is being developed with enterprise automated systems in mind, establishing a richer vocabulary that adds granularity and context to each learning activity. With P2881, organizations and educators will be able to share the abundance of non-traditional learning materials and content now available, allowing the Enterprise Course Catalog to serve as a common DoD portal for accessing them.

“Anchoring on a data standard that provides a common language for organizations to describe their training and education data is critical to scaling and improving our offerings to learners,” said Phil Abi-Najm, IT Center Director for Digital Experience at Defense Acquisition University. “Much of the impediment this industry is experiencing today can be attributed to complex translations of data between systems. Adoption of a new standard will help the proliferation of interoperable data across disparate systems and environments and enable training organizations to analyze and act on data in a much more seamless and rapid manner, ultimately to improve outcomes for our learners.”

Building upon established standards

While the initial Enterprise Course Catalog metadata model built upon the IEEE LOM standard, other standards were also evaluated and applied during a comprehensive requirements definition process supported by the ADL Initiative in collaboration with industry, academic, and government partners. It’s anticipated that the P2881 Working Group will take similar measures to align these same metadata standards for the P2881 design.

Early Enterprise Course Catalog work built on the “Course” type from the WC3’s Schema.org, which incorporates work done by the Learning Resource Metadata Initiative (LRMI). Other standards included the Credential Engine’s Learning Opportunity Profile and the Department of Education’s Common Education Data Standards (CEDS). Additionally, data elements from the HR Open Standards Consortium, the Postsecondary Electronic Standards Council, IMS Global, and the National Information Exchange Model (NIEM) were examined.

Once a consolidated list of desired metadata attributes was compiled, the ADL Initiative worked with stakeholders to refine the requirements. The research team explored each stakeholder’s instructional design (or learning engineering) processes, including the Army’s TRADOC Regulation FM 350-70, Navy’s Systems Approach to Training (MILHDBK 29612), and Air Force’s HDBK 36-2235.

Early work on the metadata model borrowed from the LOM standard’s 10 classification categories, adding new data elements as they were discovered. For example, the Army’s Synthetic Training Environment Enhanced Learning for Readiness (STEEL-R) project informed requirements to include data elements from sensors and biometric devices, which also necessitated the pointers to the logs for those devices so authorized systems could use those data.

P2881 is now being harmonized with other standards that are key to enabling the Total Learning Architecture (TLA), the foundation for DoD’s new digital learning ecosystem. P2881 is one part of the TLA’s four-pillar data strategy that enables organizations to capture and leverage their education and training data assets. Each pillar is built around a set of international data standards that combine to increase the granularity and fidelity of learner data.

Authoritative sources

A key consideration in the development of P2881 is that information about learning activities is stored in numerous authoritative systems across an organization. Some information might be encapsulated within a course catalog while other information is stored within registration systems, survey systems, or instructional design artifacts that are created as part of any courseware development activity. The draft metadata model created in support of the Enterprise Course Catalog includes automated metadata aggregation services (e.g., intelligent agents) that will connect to these sources and convert those data into the appropriate P2881 data elements. This information is stored within the server-side implementation of the standard called the Experience Index.

For example, one important concept within the P2881 effort is its potential alignment to the various educational frameworks being used today across the education and training community. In competency-based learning, these are referred to as competency frameworks where each node of the framework is a defined competency with its own unique identifier. These definitions may be hosted in numerous locations, but DoD standardized definitions of competency must be stored as an authoritative data source that provides configuration management, versioning, and alignment between local and global definitions of competency.

For other organizational systems to make semantic and syntactic sense of the P2881 metadata, a Linked Data and Schema Server is being put in place to enable namespace governance of shared vocabularies, schemas, and ontologies. Shared vocabularies define agreed-upon terms, definitions, relationships, and formats for data being exchanged. In other words, while one DoD Component may use “successful completion” and another uses “course passed,” these different terms would have the same meaning in a digital vocabulary. This work supports the translation of learning data across different systems, even those used in different functional areas.

Next steps

The ADL Initiative considers the Enterprise Course Catalog metadata model to be capable, in whole or in part, of becoming the first draft of the P2881 standard. The DoD’s vast range of digital education and training resources has many highly specified metadata requirements that are addressed in the ECC model. The next steps include a community-wide review to further define, refine, and harmonize the data elements and properties used to describe learning resources. The model is being reviewed by multiple DoD entities and will be formally presented to the IEEE P2881 Working Group for further review, discussion, and refinement.

By increasing the granularity of how organizational learning activities are described, these activities can be aligned with key performance indicators in the operational environment. Also, standardizing and automating the approach used to describe courses can help reduce the time spent on labor-intensive manual processes, increase awareness of available learning activity assets, improve the efficiency of learning activity assessments, and eliminate the duplication of content. As a result, this effort can ultimately save millions of dollars in manpower, courseware acquisition, and maintenance costs.