An analytical review of the literature of gaming and learning

Katherine Halas Moulton

Gaming and learning INTRODUCTION The purpose of this paper is to present an analysis of current literature on the theme of gaming and learning in the classroom. Panoutsopoulos and Sampson (2012) define game based learning as being the multi-faceted integration of the educational content of computer games. According to them and their sources this allows motivated learners to construct and apply knowledge, and solve real-life problems. Furthermore, this method enhances teaching, assessment, and evaluation leading to higher achievement than traditional approaches. It is important to study gaming and learning because, as Abrams (2009) states, gaming provides a meaningful context for educational information and requires active engaged learners who advance through experimentation, critical thinking and practice. Earlier studies have shown that gaming enhances visual literacy, visual attention, reaction time and problem solving (Delwiche, 2006). Moreover, Batson and Feinberg (2005) enumerate the goals of transferring knowledge, motivating and enticing users. All of the studies analyzed consider some aspect of the implementation of gaming software in the classroom which has been called the learning environment of the future (Delwiche, 2006). Organization of the paper This paper begins with a methods section that describes how the analysis and the studies were conducted. The findings section states the results of the various studies. The discussion analyzes and compares the overall quality of the studies. The final section sums up the conclusions, limitations and implications of these studies. METHODS The 10 sources were selected from seven peer-reviewed educational technology journals. All sources were chosen from some of the top peer-reviewed academic

journals in the field of educational technology. This analysis only included sources whos titles contained clear references to gaming and learning. To figure as part of the analysis, the journal sources had to include research participants. This means that metaanalysis, book reviews etc were excluded. The studies selected ranged from 2005 to 2013. The purpose of the analysis was to identify similarities, differences, patterns and to account for what has been published (Taylor, 2012). According to the Santa Cruz University Library (2012), the purpose is to offer an overview of significant literature published on a topic. In this case the goal is to find scientific research on the use of gaming in the classroom and to identify reoccurring themes and patterns in the literature. According to Taylor, one must organize, synthesize, identify controversy in the literature and formulated questions that need further research. The studies were conducted in several countries including Singapore, Turkey, Australia and the United States. Four of these studies were quantitative

(Panoutsopoulos & Sampson, 2012, Ventura, Shute & Zhao, 2013, Batson & Ferby, 2005, and Ke, 2008), five were quantitative (Abrams, 2009; Delwich, 2006, Hakan, 2007, Lee, Eustace, Fellows, Bytheway, & Irving, 2005; Lim & Nonis, 2006) and one used mixed methods (Chen, Looi, Lin, Shao & Chan, 2012). Within these studies each one uses a different piece of software. They also have different focuses: one focuses on struggling students (Abrams, 2009), several on motivating learners and improving their learning though in different subjects (Batson & Feinburg, 2005; Chen et al. 2012, Delwiche, 2006; Lee et al. 2005; Panoutsopoulos, H., & Sampson, D. G. (2012).), effective implementation (Hakan, 2007; Ke, 2008; Lim & Nonis, 2006) and one specifically focused on persistence which is an important predictor of academic performance (Ventua et al, 2013). Though each study had a different specific focus essentially they all considered some aspect of how gaming can contribute to learning in

the classroom. Some of the studies used off-the-shelf software while others designed their own for use in the study. The table below compares the variety of software, purposes, participants, methods and curriculums in each of the studies. Table 1 Descriptive comparison of reviewed gaming and learning studies Authors Software Purpose Participants Methodology Abrams, S. (2009) un-specified To discuss how gaming environments can provide context for academic information To apply learning theory to an e-learning game that promoted educational knowledge and motivation To understand whether collaborative or individualistic play would improve learning of math concepts with a video game To generate data about using MMO's effectively in the classroom 3 struggling male high school students Classroom observations, individual interviews, and a focus-group interview

Curriculum English and History

Baston & Feinburg (2005)

Credit Safe

99 high school Survey students

Mathematics

Chen et al. (2012)

Cross number puzzle game

52 grade 4 students

Pre- and posttests, questionnaires, interviews and video-taped interactions

Mathematics

Delwiche (2006)

Everquest (massivelymultiplayeronline game)

41 undergraduate students

Recording anecdotal comments, evaluated comments of web logs and assessed quality of work

Ethnographic research skills, principles of social science research and experimenting with game designing

Hakan (2007)

Ke (2008)

Made-by grad students software about continents for grade 4, first aid for grade 9 students, and basic computer programming for undergraduat es* Astra Eagle Web-based games

To generate suggestions for effective implementation of video games in the classroom

24 grade 4/5 students; 22 grade 9 students and 31 undergraduat e students in Turkey

Observations, field notes, interviews, reflections, and student artifacts

Social Studies, First Aid and Basic Computer Programming

Lee et al. (2005)

Everquest

Lim & Nonis (2006)

Quest Atlantis

To compare whether computer games are more motivational and effective for learning and metacognitive development than paper and pencil. Also whether cooperative, individualistic or competitive classroom structure is more effective Prove whether gaming increases collaboration, social and emotional skills in the classroom. Research the issues and core challenges of using video games in the classroom

358 grade 5 students

Pre- and posttested, observed, surveyed, and recorded in think-alouds

Mathematics

48 high school Calculated students in hours spent on Australian the program and analyzed script

English and Science

Eight 10-11 year olds in Singapore

Observations, interviews, preand posttesting of learning and engagement

Science

Panoutso poulos & Sampson (2012)

Sims Open for Business

Ventura et al. (2013)

Self reported Not curriculum questionnaire specificabout gender, studying GPA, SAT persistence scores, only persistence and video game usage, calculating time spent on each task Note: The * refers to software which was designed specifically for the research study FINDINGS The researchers in these studies have tested various measures of gaming and learning. Achievement is usually defined in terms of academic success, but other factors such as motivation and persistence can create an environment where learning can occur (Ke, 2008; Panoutsopoulos & Sampson, 2012; Ventura et al 2013). The main findings in the studies were about collaboration, motivation and achievement through gaming as well as the lack of appropriate software for the classroom. Collaborative gaming and learning The three studies that considered specifically whether collaborative learning games contributed to academic success proved their hypotheses. The Chen et al. (2012) study found that the collaborative class did significantly better than the other

Anagram Riddle Task*

Research whether off-theshelf commercial games can meet math objectives, general learning objectives and improve attitudes towards learning in the classroom Test hypotheses that video game use is positively correlated to persistence

Thirty 13-14 year old middle school students

Pre- and posttesting of math objectives, general education objectives and attitudes towards learning

Mathematics

102 undergraduat e students

groups who were working independently or competitively with the low-achievers making the greatest gains. Significantly, 87.5% of participants said it was easier to work collaboratively and all students responded that they benefited from discussions with classmates. Ke (2008) also found that the collaborative group was the most successful and the most motivated. During the Lee et al. study (2005) they felt that collaborative learning was facilitated by the role playing and problem solving as was shown by an average of 6-7 players playing collaboratively at one time. These studies show that collaborative games specifically improve achievement over other forms of gaming. Motivation to learn through gaming In the studies that looked specifically at engagement and motivation most found high levels of these with two exceptions (Ke, 2008; Hakan, 2007; Delwiche, 2006; Batson & Feinburg, 2006). Of the studies that found improvements, the Abrams' study, showed two of the students interviewed who were normally disengaged from learning were excited about learning about the history of warfare after playing Medal of Honor, and Battlefield 1942, video games about World War II (2009). Delwiche (2006) found that his undergraduate students were highly motivated, learning, and engaged with the material in an atmosphere of playful intellectual freedom. This is comparable to Hakan's (2007) positive, messy classroom culture where the students were collaboratively, enthusiastically and actively involved in problem solving. One of the studies that considered motivation that found lower than expected levels of engagement was the Lim and Nonis study (2006). They believe that time limitations and lack of experience with this game frustrated students as they were not able to learn the navigation or accomplish the quests in the time allotted. This lack of ability to play the game lead to frustration. They conclude that cognitive demands of open learning may be too difficult for some learners. Furthermore, another study showed that though students are engaged in the game they may not always be engaged

in the task (Lim & Nonis, 2006). Panoutsopoulos, and Sampson (2012) were also surprised that attitudes towards school, learning and math did not change significantly. The authors speculate that there was not really enough time in the project to apply the abstract math concepts to the game though they did improve in general learning skills. Educational achievement through gaming Most of the studies found that the students not only enjoyed themselves but they were learning too (Abrams, 2009; Baston & Feinburg, 2005; Chen et al. 2012; Delwiche, 2006; Hakan, 2007; Ke, 2008; Lee et al. 2005; Lim & Nonis, 2006; Panoutsopoulos & Sampson, 2012). The Batson and Feinburg (2005) study not only found that students were motivated but 77% of them also felt that learning had occurred. Only three students were motivated and did not feel that they learned; therefore, authors speculate that perhaps this type of learning does not work for those students. Delwiche (2006) found good quality of students research and in-depth analysis of classroom material. Not all researchers found an academic improvement. Ke (2008) did not show a statistically significant difference between the computer and paper groups in the math testing though the computer group had much higher motivation and showed use of metacognitive strategies during the think-aloud. The group who used the computers cooperatively was the most successful and motivated. Further, Lim and Nonis (2006) also noted that some students choose to spend time exploring the virtual world and did not necessarily work on the quests which would help them to reach the learning objectives. Several students even found ways to avoid the educational tasks in the game. Lack of appropriate software Even though it was not the expressed purpose of the studies many of them expressed the same finding: there is a dearth of appropriate quality gaming software available for the classroom. The challenge of putting the right game in teachers' hands

is multifaceted: first of all there is limited funding in schools for games and it is a challenge to find games that would meet the needs of all teachers in all subjects and teaching styles (Delwiche, 2006). For Hakan (2007), who was working in a Turkish context, the main challenge lay in finding or creating a suitable game for teaching the classroom content. Other challenges for him included lack of access to computers, computer hardware limitations, time restraints of the school periods, amount of time spent orientating themselves to the game, and technological glitches. Whether it is a school in Turkey or in Singapore there is evidence of a lack of available resources in their languages (Lim & Nonis, 2006). But even for those working in English there is a challenge to find exactly the right activity at the right level (Panoutsopoulos & Sampson, 2012). Games also need to be easy to use and focused on the educational objective. The finding that students were initial frustration with the game Everquest shows that much attention must be given to the selection of games that are accessible, and fit the curriculum goals. Moreover, if a lot of time needs to be devoted to learning the navigation of a game it becomes a challenge to fit it into a school schedule (Lim & Nonis, 2006). The time spent creating games to use in the classroom is what Hakan (2007) calls excessive; and therefore, not practical for most teaching situations. Several of the studies had to create their own software to meet the needs of what they were trying to teach (Hakan, 2007; Ke, 2008; Chen et al. 2012). A further challenge is for those creating their own educational software that is hard to compete with students expectations for games which is based on the multi-billion dollar industry standards (Hakan, 2007).

DISCUSSION

It is a challenge to generalize about the quality of these studies some of the studies were innovative, well-thought out and clearly reported. On the other hand, a few of the studies are quite irrelevant in their scope, lack rigor and even clear research goals. The others fall somewhere in the middle of those extremes where they are reasonably well executed and written. Study structure The study with the clearest purpose, design and execution was the Ventura et al. (2013) correlational study. They designed a piece of software with anagrams and riddles to be solved. The software gave very specific results about how long each participant spent on a question and was able to give such definitive results. These numbers were then correlated positively to self-reported perceptions of persistence and video game use. Chen et al. (2012) also executed a very well managed experiment with the innovative idea of creating software with a split-screen where students in the experimental group could work collaboratively, if they chose, on problems. The control group was doing identical work without the benefit of the collaborative screen. The use of identical work for all six groups adds to considerably to the validity of these experiments. Similarly, the Panoutsopoulos and Sampson (2012) and the Ke (2008) research was also well controlled where the two groups did the exact same work with one working on paper and the other on the computer. One study in particular lacked a clear purpose and offers very little to the scientific literature about gaming. The Lee et al. (2005) study's sole described purpose was to provide "proof of concept" that games, could be used in the classroom as the overlap between computer supported collaborative learning (CSCL), play theory and games design theory can be applied to K-12 teaching practice. However, in their study the only evidence they provide is number of hours spent playing the game, some anecdotal comments and the number of people playing at any given time. There is no

evidence of whether any learning took place, or any clear link whatsoever between this gaming simulation and the English and Science curriculum the game was meant to be facilitating. During the Baston and Feinberg (2005) study there was a similar problem where they did not actually prove whether learning had occurred and only asserted that in surveys participants felt that they had learned. Some kind of assessment of learning would have supported the claims in both of these studies. A couple of the studies recommended contextualizing learning but did not provide any detail or suggestions about how the teachers in the study actually achieved this structure (Hakan, 2007, and Batson & Feinber, 2005). Delwiche recommended using of worksheets and reflections as effective tools for implementing games in the classroom, but did not provide examples. Even though the reflection tools is seen to be a good indicator of learning, one study even found that students resisted doing the reflections because they did not like them (Lim & Nonis, 2006). Small sample size Some studies used sample sizes that are too small to be generalizable. In certain studies like Abram's three students or Lim and Nonis' eight the use of such small numbers of students makes for an interesting beginning of a conversation, but is not a large enough sample to be considered definitive. However, since several studies found that motivation is increased with gaming despite their relatively small sample size the fact that several researchers had similar finding makes it is more generalizable (Baston & Feinburg, 2005, Chen et al. 2012, Delwich, 2006, and Ke, 2008). Overrated results The treatment of the results is not always even handed. Some researchers play up the results that fit their hypotheses and try to justify results ideas that don't fit. In two cases where the results did not show that learning was improved in the experimental group over the control group the authors of the study just placed the emphasis in their

conclusions on increased motivation or generalized learning outcomes (Lim & Nonis, 2006, and Panoutsopoulos & Sampson, 2012)

Lack of Time Several of the studies concluded that the lack of time available to do the experiment was not favourable to their results. Some of the software used in the studies required time to learn orientation and navigation. Students were frustrated and learning goals were not met in the limited time that they had with the software (Delwiche, 2006; Lee et al. 2005; Lim & Nonis, 2007; Panoutsopoulos & Sampson, 2012). In the Hakan study there was an excessive amount of time spent creating software that met the needs of the study which in the end did not meeting the students' expectations (2007). If the students and researchers had more time to fully explore appropriate software and create the ideal game for the class learning gains may have been more remarkable. CONCLUSIONS, LIMITATIONS AND IMPLICATIONS Limitations There is a limitation on how much these studies can be generalized because each study used different pieces of software, contexts, implementation strategies and research purposes. Furthermore, certain studies have less valid results because of their limited sample size (Abrams, 2009; Lim & Nonis 2006), use of software that is no longer current (Lim & Nonis, 2006; Lee et al. 2005) or limited conclusions (Lee et al. 2005). Despite the fact there was so much variety in type of software used and the methodology there was consensus between most of the studies about some key issues: the need to choose appropriate software in the classroom, the need to contextualize gaming in a classroom lesson; and an increase in motivation and learning if these needs are met.

Conclusions Motivated learning and persistent does occur with gaming Several studies showed that there was an increase in motivation and that students reported enjoying their learning more (Chen et al. 2012; Batson, & Feinberg, S, 2005; Delwiche, A., 2006; Ke, F, 2008). One study went a step further and looked at whether collaborating on games improved achievement learning and motivation which it did (Ke, 2008). The Ventura et al. (2013) study shows that gamers develop the important academic skill of persistence which, they theorize, will transfer to future tasks like academics. Panoutsopoulos, & Sampson (2012) did not find that attitudes towards school, learning and math changed significantly; however they speculate that there was not really enough time in the project to apply the abstract math concepts to the game. Improved achievement The consensus of all these ten studies is that gaming does improve learning in the classroom. There are limitations and challenges but essentially each study concludes that gaming lead to improved achievement, motivation, persistence, or engagement. IMPLICATIONS The implications of these studies is that video games are an effective tool in the classroom; however there needs to be enough time, resources and contextualization to successful implement games as part of a lesson. Moreover, collaborative games can further increase achievement and motivation. Teacher as facilitator The most crucial finding was the importance of context, goal setting, and design of in-class learning exercises for effective teaching with gaming. The role of the teacher needs to become one of the facilitator to ensure that learning occurs. Teachers need to realize when learning is not occurring and react appropriately with scaffolding and

resources, as well as provide assessment, and establishing clear learning objectives (Delwiche, 2006). Abrams' study while small also shows that teachers can help students learn new academic information by contextualizing new learning with their knowledge of video games. Moreover, teachers need to be aware and react to ensure that students are learning and not just spending time free roaming in the game (Lim & Nonis, 2006). They also need to allow time in the schedule to orientate to the game to prevent future frustrations. Also if good collaborative learning is to occur the lesson needs to be structured and contextualized by the teacher (Hakan, 2007). There also needs to be teacher training and resources to help with the implementation of games in the classroom (Abrams, 2009). Teachers should also use a cooperative structure for their classes. Several studies showed a cooperative game structured can improve learning over a competitive or individualistic one (Ke, 2008; Lee et al. 2005) with low-achievers making the greatest gains (Chen et al. 2012). Whereas games take time to learn how to navigate and play where possible the actual school infrastructure could change to allow for longer blocks of time spent on-task (Hakan, 2007). Before any of this can happen time needs to be spent on getting a buyin from parents and the school (Lim & Nonis, 2006). Funding good games Clearly as Abrams (2009) puts it, all games are not created equal and the right game needs to be found for the lesson. When good games are used to simulate real situations like Credit Safe in the Baston and Feinberg study than learning and motivation are really good. The choice needs to be very selective for there are games like the Astra Eagle game used by Ke where the actual learning task can be avoided by a good gamer (2008). One of the implications of this analysis is that funding needs to be available for video game design specifically for education (Hakan, 2007). In order to

enhance learning and teach the curriculum software needs to be well chosen and designed.

REFERENCES Abrams, S. (2009). A gaming frame of mind: digital contexts and academic implications. Educational Media International, 46 (4), 335-347. Batson, L., & Feinberg, S. (2005). Game designs that enhance motivation and learning for teenagers. Electronic Journal for the Integration of Technology in Education. 5. Chen, Y.H., Looi, C.-K., Lin, C.-P., Shao, Y.-J., & Chan, T.W. (2012). Utilizing a collaborative cross number puzzle game to develop the computing ability of addition and subtraction. Educational Technology & Society, 15 (1), 354366. Delwiche, A. (2006). Massively multiplayer online games (MMOs) in the new media classroom. Journal of Educational Society and Technology, 9 (3), 160. Hakan, T. (2007). Blending video games with learning: Issues and challenges with classroom implementations in the Turkish context. British Journal of Educational Technology, 38 (3), 465-477. Ke, F. (2008). Computer games application within alternative classroom goal structures: cognitive, metacognitive and affective evaluation. Educational Technology Research and Development, 56 (5-6) 539-556. Lee, M., Eustace, K., Fellows, G., Bytheway, A., & Irving, L. (2005). Rochester Castle MMORPG: Instructional gaming and collaborative learning at a Western Australian school. Australasian Journal of Educational Technology, 21(4), 446-469. Lim, C. P., & Nonis, P. (2006). Gaming in a 3D multiuser virtual environment: engaging students in Science lessons. British Journal of Educational Technology, 37 (2), 211-231. Panoutsopoulos, H., & Sampson, D. G. (2012). A Study on exploiting commercial digital games into school context. Educational Technology & Society, 15 (1), 1527. Santa Cruz University Library (2012). How to Write a Literature Review. In Santa Cruz University Library. Retrieved October 16, 2012, from

http://guides.library.ucsc.edu/write-a-literature-review.
Taylor, D (2012). The Literature Review: A Few Tips on Conduting It. In Universtiy of Toronto website. Retrieved October 16, 2012, from

http://www.writing.utoronto.ca/advice/specific-types-of-writing/literaturereview
Ventura, M., Shute, V., & Zhao, W. (2013). The Relationship between video game use and a performance-based measure of persistence. Computers & Education, 60, 5258.