# TLA CMM Level 3 - Enabling Competency-Based Learning
The TLA Capability Maturity Model (CMM) Level 3 uses a standardized data model for organizing one or more local course catalogs into a network of federated course catalogs. Courses are described using the IEEE P2881 Learning Activity Metadata which is stored in an Experience Index (XI). The XI is the server-side implementation of the P2881 standard and allows learning activity metadata to be shared and used by other systems that require or use these data. For example, the Competency Management System (CMS) used in the TLA Reference Implementation can read the TLA MOM statements stored within the Transactional LRS. Each learning activity generates a series of TLA MOM statements that link learner performance data with the course catalog's
In this CMM level, each TLA MOM statement includes the following information:
- Actor: This field is populated by each learner's universally unique identifier (UUID). Within the TLA Sandbox, this is generated by KeyCloak and is used by the Competency Management System to read in the learner profile for each learner.
- Verb: Any connected learning activity will generate these statements in accordance with the TLA MOM's Learner State Models.
- Object: This field is populated using the Unique Activity ID associated with each connected Learning Activity within the TLA enclave (e.g., sandbox). The Activity ID is formatted so that it is compatible with the xAPI Object ID. The Competency Management System uses this data to read in information about each Learning Activity that generates a TLA MOM statement.
# Activity and Resource Management
An Activity and Resource Management (ARM) service is associated with capturing, connecting, and sharing data about learning resources available within the TLA enclave. Key features include the ability to generate and manage metadata about each learning activity. Learning Activity Metadata is stored in a structured database called an Experience Index (opens new window). The ARM Service allows connected systems to perform create, read, update, and delete (CRUD) operations on the metadata. The existing API for this service is rudimentary, but can be used by any system. Within the TLA Reference Implementation, the following systems currently use the ARM service to read Learning Activity Metadata:
- Competency and Skills System (CaSS): CaSS is the Competency management system being used within the TLA Reference Implementation.
- Enterprise Course Catalog (ECC): The ADL Initiative's ECC Program (opens new window) is currently working to mature the Experience Index and the underlying P2881 Learning Activity Metadata standard.
- Learner Profile System: (Future) The TLA's Learner Profile currently reads from the
authority-xapikafka topic. In the future, it will use the TLA MOM's Activity ID and the ARM services to populate different sections of each profile.
Currently, no systems can write to the XI; however, the ECC program is creating a suite of Metadata Aggregation Services that may be used to connect the XI to any system that includes information about a learning resource. For example, many organizations use course surveys to collect feedback courses they deliver. This feedback can be used to update each learning activity's metadata. For example, if students are asked to provide a 1-5-star rating for a course, the 'AggregateRating' field of the metadata can be continuously updated.
In the TLA Sandbox, the XI and ARM services are not using the P2881 Learning Activity Metadata format. The XI is divided into four tables that use some data elements from the P2881 data model:
Table 3: Experience Index Metadata Types
| Metadata type | Description |
|---|---|
| Learning content | A logical aggregation of resources that can be assembled and delivered via one or more activities (e.g., video, PDF, simulation scenario) |
| Content Set | A compilation of related content. Each chapter in an eBook may have its own metadata file, but they are connected and ordered within a Content Set. |
| Learning Activity | Any event, software, or set of interactions used to deliver Learning Content to a learner (e.g., classroom instruction, web-based course, simulated scenario). |
| Learning Experience | A Learning Experience is a result of a Learning Activity delivering Learning Content to a learner. |
| Course (future) | Any sequence of Learning Activities delivering Learning Content in pursuit of a goal (e.g., TLO, Competency). |
The XI will be continuously updated as the ECC work continues. This work will be refined and harmonized with the IEEE's P2881 Study Group (opens new window) to develop a new Learning Activity Metadata standard. A key component of the metadata is the EducationalAlignment of a learning resource to the competencies it supports.
# Querying the Experience Index
To query the XI, a GET command must be made to the appropriate table (via the ARM API) to read the metadata stored within each table. Arguments are allowed on any of the data elements (i.e., variables) within each queried table. It is recommended to include the Accept: Application/JSON header as well.
The XI root is not currently public and access to the ARM services is limited. To gain access or to log metadata for your own content, please contact TLA Technical Support.
The primary endpoint(s) for retrieving and filtering the XI metadata files are:
Endpoints:
<ExperienceIndexURL>/api/v1/activities<ExperienceIndexURL>/api/v1/content<ExperienceIndexURL>/api/v1/contentset<ExperienceIndexURL>/api/v1/learningexperience
Each endpoint accepts a few arguments as shown in Table 4.
Table 4: ARM API Arguments
| Argument | Default Value | Description |
|---|---|---|
| Competency | None | A Competency IRI to filter metadata entries in the Experience Index. When present, will only return entries that define this competency within their EducationalAlignment array. |
| ID | None | The Unique Activity ID for each learning resource. When presented, will only return a single entry whose ID matches the path parameter. |
| Secret | None | API password for querying entries outside of a Single Sign On (SSO) context; configurable with the ENV file. |
| Limit | 1000 | The maximum number of entries to return. |
| Offset | 0 | The offset / pagination index for the entries to return. |
| URL | None | The URL for a piece of content assigned to an entry. When present, will only return entries with a matching URL. |
Here is an example of a full query:
GET <ExperienceIndexURL>/api/v1/Activities/Description/Short%20Description?limit=5&offset=0
Accept: Application/json
The XI will respond to this query with an array of five or less JSON objects with a description of “Short Description.”
# Competency Management
A CMS manages evidence of an individual’s knowledge, skills, abilities, attributes, experiences, personality traits, and motivators to predict their value toward effective performance. Competencies might include technical skills, business skills, leadership skills, people skills, ethics, professionalism, or any number of topics related to a job. Within the TLA Reference Implementation, the TLA MOM statements stored in the Transactional LRS provide the evidence upon which competency assertions are made.
The TLA Reference Implementation uses the Competency and Skills System (CaSS) (opens new window) to manage Competencies. CaSS is an open-source competency management resource that provides an infrastructure, enabling competencies, competency frameworks, and competency-based learner models to be managed and accessed independent of any given learning management system, course, training program, or credential. CaSS is used to serve three primary purposes within the TLA Reference Implementation:
- Competency Management: CaSS interprets TLA MOM statements and makes assertions of competency.
- Competency Framework Registry: The CaSS Competency Framework Library uses Javascript to provide access to competency frameworks (and definitions) by connected systems.
- CaSS Authoring Tools: CaSS Authoring Tools are used to ingest a wide range of formats for how competencies are defined (e.g., ASN, CASE, O*Net).
The CaSS source code is available on the CaSS GitHub (opens new window) page. Please see the High-Level CaSS Documentation pages (opens new window) or the CaSS Developer Documentation (opens new window) for additional documentation on how to use CaSS. The ADL team can be contacted at TLA Technical Support for more information on the services and ports included on the CaSS server instance.
# Maintaining a Chain of Evidence for Competency-Based Learning
Figure 8 - Competency Management Services in the TLA Reference Implementation
The Competency Management service within the TLA Reference Implementation relies on the open source Competency and Skills System (CaSS) (opens new window). Figure 8 shows the services used to support Competency Management within the TLA Reference Implementation. Learning activities store raw learner records (i.e., xAPI statements) in the noisy LRS. The raw xAPI statements are rolled up to the Transactional LRS using the TLA MOM statements. TLA MOM statements are implemented via the TLA MOM xAPI Profile stored in the xAPI Profile Server (opens new window).
To generate assertions of competence, CaSS uses a simple listener service to consume xAPI statements from the resolve-xapi Kafka stream. CaSS reads the TLA MOM statements that are stored in the Transactional LRS and parses each TLA MOM statement. CaSS correlates the xAPI actor with the learner profile using the learner's UUID. CaSS uses the unique Activity ID stored in the xAPI Object field to pull the metadata for the learning activity that generates the MOM statement. Each learning activity's metadata file includes its EducationalAlignment which informs CaSS which Competency Definitions need to be pulled from the Competency Registry.
Using the learner profile, learning activity metadata, and relevant competency definitions, CaSS is able to estimate proficiency levels for each competency. Competency Assertions are generated by CaSS and pushed to the authority-xapi Kafka stream.
# Competency Registry
The TLA Reference Implementation's Competency Registry is currently an implementation of the CaSS Competency Framework Library. A framework created in CaSS can be viewed by anyone but can only be edited or deleted by the original creator. CaSS consists of open-source code that allows users to define, store, manage, and access Competency Definitions (e.g., Competency Objects). Competencies are organized into structured collections called Competency Frameworks. Competency Frameworks define the relationship between competencies in the context of a specific domain (e.g., job, duty, occupation). The CaSS Framework Library contains persistent, unique URLs that any connected system can use to access and reference the different Competency Frameworks and their respective Competencies.
Any system that connects to the TLA Reference Implementation can interact with CaSS through CaSS JavaScript libraries. CaSS Web Service APIs enable competencies and Competency Frameworks to be created, retrieved, updated, and deleted (CRUD operations). To query the CaSS API, make a GET request to https://cass.castle.adlnet.gov/#/api/data and add a query by adding ?q=@\<query\>. Users can add an additional query argument by adding &q=\<query\>.
Here is an example query:
https://cass.castle.adlnet.gov/#/api/data?q=@type:competency
There are no required headers to make these queries.
Additional information on how to query frameworks can be found in the CaSS Documentation (opens new window).
# Making a New Competency Framework
Adding and editing Competency Frameworks in CaSS is simple. To make a new Competency Framework in the TLA Sandbox’s CaSS instance, users can navigate to https://cass.castle.adlnet.gov/#/ (opens new window) in their preferred internet browser. If users do not already have an account, they can create an account, log-in, then click on the CaSS Competency Editor, as shown in Figure 9. First-time users will be given a simple tutorial on creating, editing, and deleting a Competency Framework.
Figure 9 - The CaSS Competency Framework Editor