MathCraft Closed

The Challenge

The use of computer games for learning has gained attention in the education community. A key concern is that game features intended to entertain and engage learners may in fact distract from acquiring new knowledge and retaining information presented in the game.

The Solution

A science-fiction adventure video game designed specifically for 6th grade math that incorporates learning-by-teaching, followed by a study aimed to determine the impact of gameplay on learning.

About the Project

The purpose of the MathCraft project was to create a game that uses artificial intelligence to engage students and amplify their prior understandings of skills and concepts. The game was designed to help students develop a deeper understanding of math through the process of peer tutoring. MathCraft’s learning-by-teaching approach is based on Cyc, a knowledge-based reasoning system that includes a knowledge base (containing approximately 10 million facts and rules encoding both commonsense knowledge and specialized domain knowledge), an inference engine, graphical and textual interfaces for human users, and API interfaces to enable other programs to call on it. The role of Cyc is to watch and learn about the player by his or her action or non-action in the game storyline in order to personalize the game for the player’s needs. MathCraft is available online for educator use.

The game developed for this project encouraged 6th grade students to tutor an avatar named Elle who is also in 6th grade. Math problems were incorporated as a natural component of the game world for Elle to solve. These problems were introduced to be most pedagogically useful for the students to optimize learning based on MathCraft’s model of the students’ math skills. For example, given the students’ level of understanding, Elle intentionally made certain mistakes which prompted the students to help her solve the problem. MathCraft changed Elle’s mental models based on the actions and skill levels of the student tutors. As a result, students were able to focus on teaching Elle in areas where they have the most difficulty.

During the first year of this project, a prototype of the MathCraft game was designed and implemented with a text-based interface. The storyline of Elle was further developed as a fully animated 3D world, which students could explore as in an online role-playing game. This interface improvement was designed to encourage students to become emotionally invested in helping Elle overcome obstacles and accomplish goals. Sound and ambient effects, in addition to background music, were added to align with the expected emotional effects of the intended plot points. During the second year, usability testing of the MathCraft system was conducted twice with groups of 6th grade students. Student feedback was used to improve basic aspects of the gameplay.

An interface was later developed that allowed teachers to monitor the progress of all their students from different classes using a “class view” and a “single-student view”. Both views were adjustable to provide progress on general topic areas, and very detailed breakdowns of students’ skills and concepts. A class summary page and a student summary page were also made available as “rollup” views to complement the detailed drill-down views of overall class and individual student performance.

A MathCraft study was conducted to test the effects of the game within a middle school math class. Two groups of students participated in the study—a MathCraft playing group, which completed a total of approximately 10 hours of gameplay over four days, and a control group, which received conventional instruction, consisting of guided and independent math practice exercises, without playing the game.

The game incorporated one primary entertainment feature and one primary instructional feature. The entertainment feature involved a narrative theme in which students navigated a spacecraft, interacted with a computer-based imaginary student, and completed various missions. The instructional feature engaged the students periodically in learning-by-teaching activities by interacting with a computer-based student. Students received prompts to either teach the next correct step in a problem by selecting it from a list of options or determine whether a step chosen by the computer-based student was correct or incorrect.

Results indicated no significant differences in learning outcomes or motivation between the two groups. Observational data indicated that students in the game group spent much of their time during gameplay engaging in activities unrelated to the educational content of the game and only about 20% of their time engaged in learning-by-teaching activities. These results highlight the importance of designing educational games that effectively balance features intended to entertain learners and features intended to promote learning. Implications for implementing educational games into classroom instruction are discussed in a journal article about this study.

Publication