Interactive Learning Environments

ISSN: 1049-4820 (Print) 1744-5191 (Online) Journal homepage: http://www.tandfonline.com/loi/nile20

Using educational games and simulation

software in a computer science course: learning
achievements and student flow experiences

Tsung-Yu Liu

To cite this article: Tsung-Yu Liu (2016) Using educational games and simulation software in
a computer science course: learning achievements and student flow experiences, Interactive
Learning Environments, 24:4, 724-744, DOI: 10.1080/10494820.2014.917109

To link to this article: http://dx.doi.org/10.1080/10494820.2014.917109

Published online: 28 May 2014.

Submit your article to this journal

Article views: 252

View related articles

View Crossmark data

Citing articles: 1 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=nile20

Download by: [Universiti Sains Malaysia] Date: 26 March 2017, At: 21:49
Interactive Learning Environments, 2016
Vol. 24, No. 4, 724744, http://dx.doi.org/10.1080/10494820.2014.917109

Using educational games and simulation software in a computer science

course: learning achievements and student ow experiences
Tsung-Yu Liu*

Department of Multimedia and Game Science, Lunghwa University of Science and Technology, No.
300, Sec. 1, Wanshou Rd., Guishan, Taoyuan county 333, Taiwan, ROC
(Received 7 August 2013; nal version received 16 February 2014)

This study investigates how educational games impact on students academic

performance and multimedia ow experiences in a computer science course. A
curriculum consists of ve basic learning units, that is, the stack, queue, sort, tree
traversal, and binary search tree, was conducted for 110 university students during
one semester. Two groups of students participated in learning activities: the
experimental group was instructed using the gaming method; and the control group
was instructed using the non-gaming method. During the study, tests, a survey, and
interviews were conducted with students. The evaluation results for academic
performance and multimedia ow experiences show that compared to the non-gaming
method, incorporating the gaming method into the learning process can enhance
students academic performance and multimedia ow experiences. The results also
indicated that there is a non-signicant and positive relationship between students
academic performance and multimedia ow experiences.
Keywords: computer game; data structure; ow experience; simulation software

1. Introduction
The data structures subject is an essential foundational topic in computer science education
and is provided to IT students as a second year subject. The data structures subject has very
abstract concepts and students typically nd the subject difcult and challenging (Tan &
Seng, 2010). To help university students understand data structures, increasingly more
courses have employed simulation software in recent years to improve learning effective-
ness, reduce instruction costs, and reduce experiment risks (Farrell, 2005; Musselwhite,
2006). Recently, some studies have begun to use games to enhance learning motivation,
engagement, and effectiveness in computer science courses (Ford & Minsker, 2003; Tan
& Seng, 2010).
A simulation represents a real-world system; learners follow its rules and adjust par-
ameters to control simulation results. The real cost of error in a simulation is low (Croo-
kall & Saunders, 1989). By contrast, a game represents a virtual-world system; learners
follow its rules and play the game to control game results. Generally, substantial
virtual costs exist when learners lose a game (Crookall, Oxford, & Saunders, 1987).
Gredler (2004) dened simulation as a dynamic set of relationships among several

*Email: tyliu8@gmail.com

2014 Informa UK Limited, trading as Taylor & Francis Group

 Interactive Learning Environments 725

variables that (1) change over time and (2) reect authentic causal processes (1996,
p. 523). On the other hand, games have a goal of winning, whereas simulations have a
goal of discovering causal relationships. The goal structure is linear in a game because a
game comprises multiple goals; a player achieves one goal and works toward the next
goal. By contrast, the goal structure of a simulation is nonlinear because the goal of simu-
lation is to achieve a desired output result, based on adjusting the input values (Gredler,
2004).
Moreover, ow was coined by Csikszentmihalyi (1975), who dened it as the holistic
sensation that people feel when they act with total involvement (Csikszentmihalyi, 1975,
p. 36). Flow is a state of deep absorption in an activity that is intrinsically enjoyable,
such as when artists or athletes focus on their play or performance (Csikszentmihalyi,
1991). Flow experiences are described as states of intense concentration or absolute absorp-
tion in an activity (Csikszentmihalyi, 1991). Many studies have indicated that ow experi-
ence leads to increased learning (Skadberg & Kimmel, 2004), increased exploratory
behavior (Webster, Trevino, & Ryan, 1993), positive learning performance (Pearce,
Ainley, & Howard, 2005), an acceptance of information technology (Ghani, 1991), and per-
ceived behavioral control. The antecedents of ow include focused attention (Hoffman &
Novak, 1996), a clear set of goals, immediate and appropriate feedback (Chen, Wigand, &
Nilan, 1999), potential control (Finneran & Zhang, 2003), a perception of challenges that
are matched to the persons skills (Chen et al., 1999), playfulness (Webster et al., 1993),
speed and ease of use (Skadberg & Kimmel, 2004).
Therefore, investigating how educational games and simulation software affect stu-
dents academic performance and ow experiences is essential and will help teachers use
effective learning strategies and adopt useful multimedia-based software to assist computer
science education.

2. Related works and our work

Algorithm visualization began with video sorting out sorting presented by Baecker (1998)
to help students understand how algorithms work. Algorithm visualizations help students
increasing interaction and engagement. Several studies argued that learning with algorithm
visualization systems is much more efcient when students are getting involved in active
learning rather than being passive viewers (Saraiya, Shaffer, McCrickard, & North,
2004). For instance, de Jong (1991) argued that the use of simulations must be instruction-
ally guided to engage students in active exploration and learning. Using algorithm visual-
izations to assist teaching, Lawrence found signicant learning benets in the ability to
understand algorithm procedures and to answer conceptual questions about the algorithm
(Lawrence, 1993; Lawrence, Badre, & Stasko, 1994). Hansen, Narayanan, and Hegarty
(2002) also found signicant learning gains using a visualization system to help teach stu-
dents with algorithms. Later, algorithm visualization was gradually replaced by algorithm
simulation in learning. The visualization is only looked at without any interaction but simu-
lation allows learner to interact with learning objects via input parameters. Hundhausen and
Douglas (2004) emphasized that the more actively learners are involved with algorithm
simulation, the better they learn the algorithms. The benets of simulation-assisted learning
include help in conveying complex systems (Tversky, Morrison, & Betrancourt, 2002), help
for engineering education (Ebner & Holzinger, 2003), help for developing scientic under-
standing (Jimoyiannis & Komis, 2001), help for solving real-world problems (Magee,
2006), reducing costs, and savings in training time (Farrell, 2005; Musselwhite, 2006).
726 T.-Y. Liu

Young people play online games regularly because playing online games is a social
activity for them (Olson et al., 2007). Playing computer games is a highly applicable enter-
tainment activity that stimulates the interest of young learners. Therefore, many researchers
have begun to adopt games as learning activities for presenting course instructional
materials (Liu, 2009; Liu, Tan, & Chu, 2009; Prensky, 2001; Virvou, Katsionis, &
Manos, 2005). This is the reason why we compare the game and simulation effect.
Game-based learning (GBL) is designed to combine learning and game playing to
improve learner outcomes, including information retention and real-world application.
Ford and Minsker (2003) developed an educational game called TREEZ to help students
reinforce their knowledge of tree traversal techniques. Tan and Seng (2010) used an edu-
cational game to teach students stack and queue data structures. Their ndings showed
that students are positive toward using educational games for learning about data structures.
Students felt that GBL is engaging and fun. Chafn, Doran, Hicks, and Barnes (2009) used
EleMental: The Recurrence an educational game that that requires students to write pro-
grams and perform interactive visualizations to learn about recursion through a depth-rst
search of a binary tree. Their results showed that students achieved signicant learning
gains when they played the game.
Flow is a state of deep absorption in an activity. Csikszentmihalyi (1991) stated that
ow experiences comprise the following nine elements: (1) challenge-skill balance: a
task that can be completed; (2) concentration: the ability to concentrate on the task; (3)
clear goals: concentration is possible because the task has clear goals; (4) feedback: concen-
tration is possible because the task provides immediate feedback; (5) sense of control: the
ability to control ones actions; (6) action-awareness merging: a deep but effortless involve-
ment that eliminates awareness of the frustrations of everyday life; (7) loss of self-con-
sciousness: concern for self disappears, but the sense of self increases; (8) time
distortion: the sense of the time is altered; and (9) autotelic experience: a feeling of
doing something for its own sake.
Hoffman, Novak, and Yung (1999) reported that ow experiences among Web users
may occur under the following elements: (1) skill: training personal skills; (2) challenge:
evaluating personal knowledge and skills; (3) playfulness: feeling enjoyment and leisure
from the process; (4) interactivity: interacting with the computer smoothly; (5) attention
(time distortion): ignoring the surrounding world and the passage of time; (6) telepresence:
feeling exposed to the virtual world; (7) arousal: feeling excited and surprised in the
process; (8) positive effect: feeling happy and enjoying performing activities; (9) explora-
tory behavior: exploring without a specic goal; (10) control: the ability to control the entire
process without a fear of losing control; and (11) goals: knowing the goals and the most
appropriate things to do.
Kiili (2005) conducted a study by using an educational game, IT-Emperor, to examine
the ow experiences of university students. The results show that IT-Emperor could provide
ow experience. Voiskounsky, Mitina, and Avetisova (2004) conducted a study by using
online gaming environments to examine the ow experiences of players. The results
show that computer games may initiate ow experience. Furthermore, Sweetser and
Wyeth (2005) developed the GameFlow model which includes a set of central criteria
that can be used to design and evaluate games on player enjoyment. The GameFlow
model contains eight elements for players: (1) concentration: attract the players attention;
(2) challenge: provide challenges that match the players skill level; (3) player skills:
support players skill development and mastery; (4) control: enable the player to feel in
control; (5) clear goals: establish clear goals; (6) feedback: provide feedback at appropriate
 Interactive Learning Environments 727

times; (7) immersion: ensure a deep experience but effortless involvement; and (8) social
interaction: support and create opportunities for social interaction.
People appear to manifest different ow characteristics when they engage in different
activities. On the basis of the above-mentioned discourse, we dened multimedia ow,
which includes the following elements: heed (drawing personal attention), clear goal (estab-
lishing clear objectives), feedback (receiving immediate feedback), challenge (evaluating
knowledge and skills), domination (interacting with the computer freely), immersion
(feeling emotionally involved in the environment), playfulness (experiencing fun and
leisure in the process), and self-actualization (feeling satisfaction from performing a task,
playing a role, etc.).
The above-mentioned studies have argued that the ow experience leads to improved
academic performance. However, some researchers in the eld of e-learning did not nd
evidence for a relation between the appearance of a ow experience and the learning
effect (Konradt, Filip, & Hoffmann, 2003; Pavlas, 2010), so that a clear relationship prob-
ably cannot be stated (Karimi, Nosrati, & Hariri, 2012; Schiefele & Rheinberg, 1997).
Although many studies have explored the learning effects of a simulation system for a
data-structure course, studies assessing the comparative effect on students academic per-
formance and ow experiences between simulation and an educational game are scant.
In addition, Liu, Chu, and Tan (2012) show that the use of computer games in learning
can produce better technology acceptance than using simulation software. Therefore, this
study aims to investigate how game-assisted and simulation-assisted learning impacts stu-
dents academic performance and multimedia ow experiences.

3. Methodology
This study adopted a quasi-experimental design. Two classes of students participated in this
study. The independent variable was using different types of learning software to assist
learning. The control group used simulation software and the experiment group used an
educational game. Tests were used to evaluate student academic performance. Quantitative
data were collected from surveys and used to evaluate student multimedia ow. Qualitative
data were collected from interviews and used to understand the students opinions and
explain the statistical results.

3.1. Research questions

Chung, Huang, and Hsiao (2011) conducted an empirical study involving 1147 undergradu-
ate students in Taiwan, who dedicated more than 18 hours a week to playing computer
games. The ndings demonstrated that traditional computer simulation software may not
be able to fully meet young peoples requirements as regards sensory stimulation and
attracting their attention to achieve effective learning. Therefore, this study investigates
the following research questions:

Q1. What is the difference between the academic performance behind the game-assisted learn-
ing and that of simulation-assisted learning? Additionally, what is the reason for the difference?
Q2. What is the difference between the multimedia ow behind the game-assisted learning and
that of simulation-assisted learning? Additionally, what is the reason for the difference?
Q3. What is the correlation between students academic performance and multimedia ow for
game-assisted and simulation-assisted learning?
728 T.-Y. Liu

3.2. Participants
Participants included 110 students (in two classes) with background knowledge of Compu-
ter Game Science, an assistant with background knowledge of computer science, and a
teacher who had taught computer science-related courses for more than 10 years at a uni-
versity level. The experimental group comprises a class of students with 44 males and 11
females aged 2021, and the control group comprises a class of students with 46 males and
9 females aged 2021.

3.3. Learning environment

A web-based learning environment (WBLE) was proposed for conducting online learning
in the data-structure course. Figure 1 illustrates the WBLE architecture. A WBLE has the
following features:

. Permanence. The students learning portfolios can be recorded into the WBLE server
for the teachers to review and grade.
. Accessibility. Teachers input materials and assessments into the WBLE server. Stu-
dents use the browser to play and watch learning materials via the Internet.
. Immediacy. Teachers utilize a personal computer to access the WBLE server via the
Internet. In addition, students can take tests and evaluate their learning progress
immediately.
. Interactivity. Students can use online discussions to interact with their peers and
teacher.
. Individuality. Students can select proper learning materials according to personal
ability, interest, requirement, objective, and schedule.

3.4. Curriculum design

The Introduction to Data Structures curriculum included topics related to stack, queue, sort,
tree traversal, and binary search tree. An eight-week experiment was conducted during class
time, and a 3-hour course was conducted each week. This curriculum was designed to
enhance the learning of students by improving their multimedia ow experiences while

Figure 1. Architecture of WBLE.

 Interactive Learning Environments 729

they played games or engaged in simulations to learn data structures. The students in the
experimental group used GBL approach (using Star Chef game), while the students in
the control group used a non-gaming learning approach (using simulation software). The
two groups used the same course content although the learning software they used
during the classes was different. There were same time-on-task and equivalent tasks for
two groups. The curriculum used same paper-printed tests to evaluate students academic
performance. Table 1 shows that the curriculum and instruction treatments were designed
in ve phases.

3.5. Learning software

The multimedia learning cognitive model (Mayer, 1997) and the experiential gaming model
(Kiili, 2005) were applied to the design of the educational game Star Chef. The multi-
media learning cognitive model argues that a ow condition can trigger high levels of cog-
nitive information processing (selecting, organizing, and integrating). For the presentation
of multimedia that helps students understand a scientic explanation, Mayer (2001) pro-
posed ve major principles: contiguity, multiple representation, split-attention, individual
differences, and coherence. Figure 2 displays a screenshot and principles of the multimedia
learning cognitive model. The educational game possesses the following principles:

. Contiguity principle: Corresponding game rules and screen shots are presented con-
tiguously rather than separately in the multimedia explanation. Gameplay was pre-
sented stepwise in text and pictures. In this case, gameplay is presented in both
text and pictures.
. Multiple representation principle: Each explanation is presented in text and pictures
rather than only in text. In this case, gameplay is presented in both text and pictures.
. Split-attention principle: Both auditory narration and text are presented in the multi-
media explanation. In this case, game rules are presented in both text and auditory
narration.
. Coherence principle: Few words and pictures are used in the multimedia explanation.
In this case, gameplay is presented in short texts and few pictures.

The experiential gaming model (Kiili, 2005) emphasizes the importance of considering
ow theory (Csikszentmihalyi, 1975) in game design to generate optimal learning

Table 1. Curriculum and Instruction treatment.

Phase Control group Experimental group
1. Preparation The instructor explained experiment The instructor explained experiment
(Week 1) objectives and evaluation methods objectives and evaluation methods
2. Instruction The instructor taught pre-requirement The instructor taught pre-requirement
(Weeks 23) knowledge knowledge
3. Evaluation 1 The instructor administered pre-test The instructor administered pre-test
(Week 4)
4. Learning The students used simulation software The students used an educational game
(Weeks 57) to learn and practice to learn and practice
5. Evaluation 2 The instructor administered a post-test, The instructor administered a post-test,
(Week 8) surveys, and interviews surveys, and interviews
730 T.-Y. Liu

Figure 2. The screenshot of bake a pizza game and principles of the multimedia learning cognitive
model.

experiences for players. The model describes learning as a cyclic process that occurs
through direct experience in the game world. Both constructivist and pragmatist views of
learning are adopted. The model stresses that the activity necessary for learning is not
only cognitive but also behavioral. Thus, learning is dened as the construction of cognitive
structures through action in the game world.
Designing an educational game that enhances ow experiences is justiable because
many studies have indicated that ow positively affects academic performance and
learner attitudes (Ghani, 1991; Kiili, 2005; Skadberg & Kimmel, 2004; Webster et al.,
1993). In order to measure students multimedia ow and academic performance, both
learning software (i.e. educational games and simulation software) have the following
common elements:

. Clear goals. The student is given clear learning goals.

. Learning content. The student is provided with complete learning materials.
. Guidance. The student is given gaming rules.
. Challenge. The student encounters a sufcient challenge (e.g. time limit) that
matches his/her skill level.
. Feedback. The student is provided with appropriate feedback, such as animation,
sound, score, status, and awards.
. Evaluation. The student is provided with judgment mechanics for making a pass/fail
decision at each gate.
. Social interaction. The student can build an interactive social relationship with his/
her classmates via discussions/forums.
. Domination. The student feels a sense of control over his/her actions.
 Interactive Learning Environments 731

. Immersion. The student feels emotionally involved in the learning environment.

. Playfulness. The student feels enjoyment and interest when using the tools.

For the experimental group, this study developed an educational game entitled Star
Chef, which included the following ve learning units: (1) Fry meatballs (learning stack
algorithm), where learners must follow the rule of last in rst out to deliver fried meatballs
(data list) to each customer; (2) bake pizza (learning queue algorithm), where learners
must follow the rule of rst in rst out to deliver a pizza (data list) to each customer;
(3) barbecue hot dog skewers (learning bubble sort algorithm), where learners must
compare each adjacent pair and switching the items if they are ordered incorrectly to
organize the barbecue hot dog skewers (data list) before delivering them to customers;
(4) barbecue meat skewers (learning tree traversal algorithm), where learners must
place the root node to the left of the left node (pre-order), between the left and right
node (in-order), or to the right of the right node (post-order) to prepare a meat
skewers (complete the tree visiting); and (5) make sushi (learning binary search tree algor-
ithm), where learners must place smaller nodes to the left of the root node and larger
nodes to the right of the root node to organize the price-marked sushi plates (construct
a binary search tree). The learners execute each ash game on the learning management
system (LMS) and follow the game rules of the data structures, which they study during
the course by controlling the characters and objects to complete the learning goals. Figure
3 shows a screenshot and elements of the experiential gaming model from the educational
game Star Chef.
The learning software used by the control group was simulation software, which was
built by the Java Applets. We selected the following ve algorithms: stack applet, queue
applet, bubble sort applet, tree traversal applet, and binary search tree applet. The students
execute each java applet in a web browser and follow the operation rules of the data struc-
tures, which they study during the course by setting parameters and inputting values to com-
plete the learning goals. Figure 4 shows the screenshot of simulation web page.

Figure 3. Screenshot of bake a pizza game and elements of experiential gaming mode.
732 T.-Y. Liu

Figure 4. Screenshot of simulation web page.

3.6. Theoretical fundamentals

Csikszentmihalyi (1991) stated that ow experiences comprise the nine elements: chal-
lenge-skill balance, concentration, clear goals, feedback, control, action-awareness
merging, loss of self-consciousness, time distortion, and autotelic experience. Hoffman
et al. (1999) reported that ow experiences among Web users may occur under the 11
elements: skill, challenge, playfulness, interactivity, attention (time distortion), telepre-
sence, arousal, positive effect, exploratory behavior, control, and goals. Sweetser and
Wyeth (2005) developed the GameFlow model which contains eight elements for
players: concentration, challenge, player skills, control, goals, feedback, immersion, and
social interaction. Findings of numerous researches revealed that students ow experience
has signicant benets to learning performance (Choi, Kim, & Kim, 2007; Pearce, Ainley,
& Howard, 2005; Prensky, 2001). Because people experience different ow states
when engaging in different activities, this study used multimedia ow, including the
factors heed, clear goal, feedback, challenge, domination, immersion, playfulness, and
self-actualization to examine the students multimedia ow experience. Learning tools
foster playing, which produces a ow state and supports the learning process (Paras & Biz-
zocchi, 2005).

3.7. Research hypotheses

Although many studies have explored the learning effects of simulation software or
educational games for a course, studies assessing the comparative effect on students
academic performance and multimedia ow between simulation and an educational
game are scant. Moreover, several e-learning studies did not reveal a clear relationship
between the appearance of ow experience and academic performance (Konradt et al.,
2003; Schiefele & Rheinberg, 1997). Therefore, the following research hypotheses were
developed:

Hypothesis 1 students who experience different interventions show no signicant difference

in their academic performance.
 Interactive Learning Environments 733

Hypothesis 2 students who experience different interventions show no signicant difference

in their multimedia ow.
Hypothesis 3 there is a signicant correlation between students multimedia ow and aca-
demic performance in learning.

3.8. Measurements
Both qualitative and quantitative methods were used to collect data. The SPSS (Statistical
Package for the Social Science, version 20) and SmartPLS 2.0 (Ringle, Wende, & Will,
2005) were used to score the data and answer the research questions. Quantitative data
included the test scores and the survey of both groups. Scores for pre-test and post-test,
ranging between 0 and 100, were given by teacher for examining academic performance.
A survey was administered to evaluate students multimedia ow experiences. A 20-
question multimedia ow scale was developed, based on the Flow State Scale (Jackson
& Marsh, 1996). Responses to all questions were on a 5-point Likert-scale, from 5 for
strongly agree to 1 for strongly disagree. The signicance level was set at p < .05 for
all statistical analysis, as is standard practice. This study adopted Cronbachs coefcient
in order to evaluate the internal consistency reliability of the tests. Cronbachs coefcient
ranges between 0 and 1, and Nunnaly (1978) suggests that 0.7 is an acceptable minimum
reliability coefcient.
The academic performance and multimedia ow of the statistical results of the question-
naire were obtained using ANCOVA analysis. Descriptive statistics, including means and
standard deviations were used to calculate students responses to the questions in the ques-
tionnaire. ANCOVA was used to measure the signicance of the difference between the
evaluation results of the two groups.
Because of the relatively small number of participants, this study used partial least
squares (PLS) to achieve good prediction and interpretation. The PLS technique was used
to examine the correlation between academic performance and multimedia ow. This
method was designed by Wold (1985) for the analysis of high-dimensional data in a low-
structure environment. PLS path models are dened by two sets of linear equations: the
inner model and the outer model. The inner model species the relationships between
latent variables, whereas the outer model species the relationships between a latent variable
and its observed variables. PLS model analysis has two stages: (1) examining reliability and
validity and (2) examining the signicance and prediction ability of the path coefcient in the
inner model. Figure 5 illustrates the potential model structure for learning in this study:
In the rst stage, the loading of each latent variable should be higher than 0.5. Cron-
bachs measures the internal consistency reliability of the latent variables, which must
be higher than 0.6 (Henseler, Ringle, & Sinkovics, 2009). Accordingly, the absolute stan-
dardized outer loadings should be higher than 0.7 (Henseler et al., 2009). In assessing val-
idity, two subtypes are usually examined: convergent validity and discriminant validity.
Convergent validity signies that a set of indicators represents the same underlying con-
struct, which can be demonstrated through their unidimensionality. Fornell and Larcker
(1981) suggest using the average variance extracted (AVE) as a criterion of convergent val-
idity. The AVE value should be higher than 0.5 (Gtz, Liehr-Gobbers, & Krafft, 2010). Two
measures of discriminant validity have been described: the FornellLarcker criterion and
the cross-loadings. The FornellLarcker criterion (Fornell & Larcker, 1981) postulates
that the AVE of each latent variable should be greater than the latent variables highest
squared correlation with any other latent variable. The loading of each indicator should
be greater than its cross-loadings (Chin, 1998; Gtz et al., 2010).
734 T.-Y. Liu

Figure 5. A potential model structure.

In the inner model analysis, the R2 values of .67, .33, or .19 for endogenous latent vari-
ables in the inner path model are described by Chin (1998) as substantial, moderate, or weak,
respectively. The estimated values for path relationships in the structural model should be
evaluated in terms of sign, magnitude, and signicance (Fornell & Larcker, 1981; Pavlou
& Fygenson, 2006). This study used SmartPLS, which was developed by Ringle et al.
(2005) to explore the relationship between multimedia ow and academic performance.
To understand student perceptions, the qualitative data obtained from the in-depth inter-
views were examined. The interviews were conducted to explore students points of view,
feelings, and perspectives on multimedia-assisted learning. The open-ended questionnaire
and interviews contained the following questions:

(1) Did the learning software draw your attention? Why or why not?
(2) Was the goal of the learning software clear? Why or why not?
(3) Was the task in the learning software challenging? Why or why not?
(4) Did the learning software provide feedback? Why or why not?
(5) Is the learning software controllable? Why or why not?
(6) Do you think the learning software is useful? Why or why not?
(7) Do you think playing the learning software is enjoyable? Why or why not?
(8) Do you think the learning software is funny? Why or why not?
(9) Do you think the learning software is playful? Why or why not?
(10) Are you satised with the learning software? Why or why not?

3.9. Procedures
The teacher introduced experiment goals, phases, and course objectives to students before
they began. During each phase, the teacher also observed students learning conditions, atti-
tude, and facial expressions from a side view and recorded their observations in a report
form. Figure 6 is a diagram outlining experiment procedures.
During the preparation phase, the teacher administered a survey to the two groups in
order to understand the prerequisite conditions of the two group students. The teacher
observed student learning conditions, attitudes, and facial expressions from a side view
and recorded these observations in a report.
During the instructional phase, the teacher adopted traditional instruction approaches
with the two student groups. The teacher taught basic knowledge and operational skills
 Interactive Learning Environments 735

Figure 6. Diagram outlining the procedure of the case study.

related to topics stack, queue, sort, tree traversal, and binary search tree, which would be
employed in the learning phase. During the evaluation rst phase, the teacher administered
pre-test to both student groups in order to understand the prerequisite conditions of the stu-
dents. The pre-test had 10 single-choice questions and 10 essay questions related to topics
stack, queue, sort, tree traversal, and binary search tree.
During the learning phase, each student used a desktop PC to log onto a LMS. The LMS
performs numerous learning functions, including providing the course list, course materials,
and a bulletin board and facilitating online discussions, homework uploads, and online
chats. Students in the control group used simulation software to practice the units until
they fully understood every learning unit. Conversely, students in the experimental group
employed the educational game, Star Chef, to play each unit game until they were compe-
tent at every unit game.
During the evaluation second phase, the teacher administered a post-test and a survey to
the two groups to evaluate students academic performance and multimedia ow. The post-
test had 10 single-choice questions and 10 essay questions related to topics stack, queue,
sort, tree traversal, and binary search tree. To understand students academic performance
and multimedia ow experiences deeply, interviews were conducted with both groups.

4. Results and discussions

Through quantitative analysis of the data sources, we examined the academic performance,
multimedia ow, and the relationship between these factors. The student responses, inter-
view records, and observation records are used to explain the analysis results and are
described in the discussion paragraphs.

4.1. Academic performance

The Cronbachs for tests is 0.778 which conrmed the internal consistency of the tests.
The result (p > .05) of Levenes test for the equality of variances indicates that the assump-
tion of the homogeneity of variances in the groups is satised. In addition, the test results (p
> .05) of between-subjects effects, which indicate the assumption of homogeneity of
regression coefcients for the two groups, is satisfactory for the remainder of the tests.
With this in hand, an ANCOVA analysis was performed using the pre-test as a covariate.
Table 2 presents the mean grades and standard deviation of evaluations for each test.
The effect size was adopted to measure the signicance of the difference between the
evaluation results of the two groups. Cohens d (Cohen, 1992) is an appropriate measure-
ment of the effect size, for use in the ANCOVA. The values 0.2, 0.5, and 0.8 represent a
small, medium, and large effect size, respectively. In Phase 5, the ANCOVA and
Cohens d result of post-test (F = 87.450, p < .05, d = 1.0384 > 0.8) indicated that the
average grade of the experimental group signicantly exceeded that of the control group
by about 20 points, indicating that the effectiveness of the Star Chef game improves learn-
ing. According to interviews, the reason for low scores on the pre-test is that the subjects of
736 T.-Y. Liu

Table 2. Mean grades and SD of evaluations for each test.

Control group Experimental group
Item Mean SD SE Mean SD SE F
Pre-test 45.76 24.628 3.321 48.78 27.664 3.730
Post-test 59.11 21.936 2.958 78.64 22.910 3.089 87.450*
Note: n = 110.
*p < .05.

the data structures involve abstract concepts and are difcult to understand. The improve-
ment of the post-test scores occurred because the proposed educational game provided more
interesting learning methods than non-gaming simulation programs. Students in the exper-
imental group felt that although they encountered many setbacks at the start of the game,
they were able to beat the stages and achieve learning goals after a few attempts. Students
in the experimental group generally agreed that traditional teaching methods make it dif-
cult to understand basic concepts. By contrast, games are more entertaining and can stimu-
late interest in learning. During game play, they could learn basic data-structure operating
skills. Some students in the experimental group believed that the GBL method made teach-
ing difcult-to-understand concepts more exciting. The method was particularly benecial
to students who loved playing games, allowing them to easily learn basic data-structure
concepts. They all agreed that gaming was helpful when learning abstract concepts.
Because the p-value is less than .05, we reject the null hypothesis. There is a signicant
difference in the academic performance of the two groups, which can explain research ques-
tion Q1 and conrm Hypothesis 1.

4.2. Students multimedia ow

A total of 110 valid questionnaires were submitted, with a response rate of 100%. The Cron-
bachs for group heed, goal, feedback, challenge, domination, immersion, playfulness,
and self-actualization are 0.804, 0.795, 0.801, 0.845, 0.874, 0.852, 0.845, and 0.909.
These alpha coefcients exceeded 0.75, which conrmed the internal consistency of the
survey. The results (p > .05) of Levenes test for equality of variances indicate that the
assumption of the homogeneity of variances in each item is satisfactory. Accordingly, an
ANOVA was then performed. The ANOVA evaluation results of learning multimedia
ow experiences are listed in Table 3.
Responses to items FH1 and FH2 indicated that the experimental group had signi-
cantly higher concentration than the control group had: p < .05. The results indicate that
the mean for students in the experiment group was higher than that of the control group
students. Students in the experiment group show that the games attracted their attention
because of the lively, playful screen display images, their desire to beat different stages
and compete with classmates, the games creativity, and their interest in educational games.
The responses to items FG1 and FG2 indicated that the experimental group had signi-
cantly higher understanding than the control group had about the learning objectives of each
unit: p < .05. The results show that the game allows learners to more clearly understand
learning objectives than simulation did.
Items FF1 and FF2 indicated that the experimental group had a signicantly enhanced
assessment of feedback than the control group had (p < .05). The results indicate that the
game enabled students to receive more learning feedback. Most students in the experiment
 Interactive Learning Environments 737

Table 3. The results of ANOVA multimedia ow test.

Control Experimental
group group
Item Mean SD Mean SD F
FH1: The multimedia drew my attention to learning 3.31 0.858 3.82 0.925 8.958*
FH2: I was able to concentrate on the learning software 3.24 0.860 3.84 0.788 14.556*
during the course
FG1: I know the goal of each learning unit 3.47 0.813 3.78 0.712 4.497*
FG2: Before the start of each section, a graphic 3.27 0.827 3.82 0.772 12.789*
description helped me understand the learning
objectives
FF1: I received immediate feedback whenever I acted 3.22 0.854 3.78 0.786 12.967*
during the course
FF2: I am able to know my current learning status, such 3.20 0.869 3.87 0.747 18.955*
as score earned
FC1: This learning software is not easy, but it is not too 3.07 1.086 3.76 0.881 13.421*
hard either
FC2: It is a challenging task to accomplish each mission 3.36 0.890 3.69 0.900 3.679*
in time
FD1: I always know what to do next to complete the 3.09 0.888 3.53 0.790 7.416*
learning unit
FD2: I can control my progress in each unit 3.15 0.891 3.82 0.796 17.452*
FI1: During the learning, I did not notice what was 3.35 0.844 3.82 0.819 8.895*
happening around me
FI2: Time ies fast when I use learning software for 3.05 0.911 3.85 0.848 22.717*
learning
FP1: I was happy during the learning activity 3.05 0.911 4.13 0.771 44.077*
FP2: I felt enjoyment when I played the learning software 3.31 0.858 4.33 0.747 44.417*
FS1: I wanted to proceed to the next unit after completing 3.29 0.896 3.78 0. 896 8.253*
each learning unit
FS2: I felt great self-actualization when I achieved the 3.29 0.896 3.89 0. 737 14.709*
goal of each learning unit
Note: Heed (FH), goal (FG), feedback (FF), challenge (FC), domination (FD), immersion (FI), playfulness (FP),
and self-actualization (FS).
*p < .05.

group agreed that the sound and animation effects immediately showed them what stage
they were at and whether they had succeeded or failed at beating the stage.
Responses to item FC1 indicated that the experimental group had signicantly higher
awareness than the control group did about the appropriate challenge of learning software:
F = 13.421, p = .000 < .05. The results suggest that the game difculty was relatively mod-
erate. Most students in the experiment group were regular computer-game players.
The challenges in the game such as speed, memory, and skill tests were not too difcult
or too easy for them. Responses to item FC2 indicated that there was not signicant
difference between the two groups in completing each task in the allotted time: F =
3.679, p = .058 > .05. The results indicate that the difference between the experimental
group students and the control group students in completing challenges was not signicant.
For students in the control group, although the simulation software was not easy to under-
stand and operate, they were condent enough to complete the tasks. Students in the exper-
iment group suggested that because they were regular computer-game players, they were
condent enough to complete the tasks.
738 T.-Y. Liu

Item FD1 indicated that there was a signicant difference between the two groups in
knowing what to do next: F = 7.416, p = .008 < .05. Students in the experiment group
were better at understanding the learning software control methods. In the interviews,
most students in the experimental group agreed that the game was relevant to life.
Because they often played games, it was easy to understand the game control methods.
Item FD2 indicated that there was not signicant difference between the two groups in con-
trolling task progress: F = 17.452, p = .000 < .05. In interviews, students in the experiment
group reported that games had specic character representations, making it easy for them to
control objects to complete tasks.
Item FI1 indicated that there was a signicant difference between the two groups in
ignoring surrounding events: F = 8.895, p = .004 < .05. Students in the experimental
group reported that they wanted to pass the levels within the time limit; thus, they concen-
trated effectively without noticing their surroundings. Item FI2 indicated that there was a
signicant difference between the two groups in feeling time ew when using the software:
F = 22.717, p =.000 < .05. Most students in the experiment group thought that the game was
interesting and did not notice time progressing.
The responses to item FP1 indicate a signicant difference between the two groups in
the students happiness during the learning activity: F = 44.077, p < .05. The students of the
experimental group (M = 4.13, SD = 0.848) were happier than those of the control group
(M = 3.05, SD = 0.911) in the activity. The responses to item FP2 indicate a signicant
difference between the two groups in the students playfulness when they used the learning
software: F = 44.417, p < .05. The experimental group (M = 4.33, SD = 0.747) had more fun
in playing the educational game than the control group did with the simulation (M = 3.31,
SD = 0.858). Generally students in the experimental group considered the game novel and
had not seen similar educational games before. Using games as a teaching tool in class-
rooms can make students feel funny and happy in class. They enjoyed using games to
learn and hoped that other chapter exercises have similar games. As a result, they had
lower anxiety of learning and could learn effectively and see obvious results. Most students
in the experimental group mentioned that they grew up playing educational games and
would have the urge to play them as long as they are enjoyable, funny, playful, or
interesting.
The responses to item FS1 indicate a signicant difference between the two groups in
self-actualization: F = 8.253, p < .05. The experimental group felt more satisfaction (M =
3.78, SD = 0.896) than did the control group (M = 3.29, SD = 0.896). The students in the
experimental group were satised with their current achievement and had enough con-
dence to nish next unit. The responses to item FS2 indicate a signicant difference
between the two groups in achieving the goal of each learning unit: F = 14.709, p < .05.
The students of the experimental group had a higher sense of fulllment (M = 3.89, SD
= 0.737) than those of the control group (M = 3.29, SD = 0.896). Students in the experiment
group felt that although they encountered many setbacks at the start of the game, they were
able to beat the stages and achieve learning goals. Students in the experiment group gener-
ally stated that games are more entertaining and can stimulate interest in learning. During
game play, they could learn basic data-structure operating skills. Some students in the
experiment group believed that the GBL method made teaching difcult-to-understand con-
cepts more exciting. Some students in the experimental group also mentioned that this type
of gaming world design is life-like, making the satisfaction gained from beating the stages
an interesting learning experience. They all agreed that gaming was helpful when learning
abstract concepts. They all had deeper impressions of the learning content and were better at
memorizing abstract rules in the data structure. Finally, most students suggested teachers
 Interactive Learning Environments 739

apply games to all the subjects so as to make learning more efcient and effective. Because
the p-value is less than .05, we reject the null hypothesis. There is a signicant difference in
the multimedia ow of the two groups that can explain research question Q2 and conrm
Hypothesis 2.

4.3. Correlation between academic performance and multimedia ow

The correlation coefcient ranges from 1 to 1 and a correlation of 1 means that the two
variables are perfectly correlated. Table 4 presents the correlation matrix for the variables
of multimedia ow. There are middle and high correlation values among eight subscales
of multimedia ow, ranging from 0.4 to 0.8 at the .01 level, indicating that one who has
a specic multimedia ow might have other kinds of multimedia ow.
Regarding the assessment of reliability, the outer loading of each observed variable of
multimedia ow and academic performance is higher than 0.5; the values for heed, goal,
feedback, challenge, domination, immersion, playfulness, self-actualization, and perform-
ance are 0.887, 0.706, 0.751, 0.760, 0.737, 0.848, 0.726, 0.731 and 1.000, respectively.
Cronbachs is 0.92 (higher than 0.6), which means that the internal consistency reliability
has been satised. The absolute standardized outer loading of each observed variable for
heed, goal, feedback, challenge, domination, immersion, playfulness, self-actualization,
and performance has values of 0.887, 0.706, 0.751, 0.760, 0.737, 0.848, 0.726, 0.781,
and 1.0, respectively. In addition, composite reliability has a value of 0.924 and is higher
than 0.7. The estimation result shows there is a high reliability.
In the assessment of validity, the AVE has a value of 0.603. Since it is larger than 0.5, it
means that a latent variable is able to explain more than half of the variance of its indicators
and showing a good convergent validity ( Gtz et al., 2010). In the discriminate validity
estimation, Table 5 shows that the loading of each indicator is greater than its cross-load-
ings. Moreover, the latent variable correlation has a value of 0.214, which is smaller
than 0.7. The results show that the model has good discriminant validity.
The result of the PLS path model containing path coefcient and outer loading is shown
in Figure 7. The R 2 value of 0.146 for the endogenous latent variables in the inner path
model is described as weak (Chin, 1998). The path coefcient value of 0.214 (T-value =
1.263 < 1.96, p > .05), which implies that the correlation between multimedia ow and aca-
demic performance is positive but non-signicant; therefore, Hypothesis 3 could not be
supported.

Table 4. Correlation matrix of multimedia ow.

Item FH FG FF FC FD FI FP PS
FH 1
FG .572** 1
FF .648** .580** 1
FC .622** .664** .569** 1
FD .576** .704** .647** .638** 1
FI .721** .664** .594** .679** .690** 1
FP .531** .429** .399** .519** .419** .576** 1
FS .672** .688** .623** .652** .737** .795** .478** 1
Note: Heed (FH), goal (FG), feedback (FF), challenge (FC), domination (FD), immersion (FI), playfulness (FP),
and self-actualization (FS).
**Correlation is signicant at the .01 level (two-tailed).
740 T.-Y. Liu

Table 5. PLS report: cross-loadings.

FH FG FF FC FD FI FP FS Test Score
Multimedia ow .887 .706 .751 .760 .737 .848 .726 .781 0.214
Performance .280 .041 .096 .090 .006 .131 .174 .023 1.000
Note: Heed (FH), goal (FG), feedback (FF), challenge (FC), domination (FD), immersion (FI), playfulness (FP),
and self-actualization (FS).

Figure 7. Result of the PLS path model for correlation between multimedia ow and academic
performance.

4.4. Research limitations

We conducted the experiment exclusively for university students aged 2021. Each student
possessed a varying amount of background knowledge in computer science, and their
experience in computer-assisted learning inuenced the experimental results. Moreover,
the experimental period was not adequately long to inuence academic performance in
the specied course topic.

5. Conclusions and future work

The data structures subject has abstract concepts and students generally nd the subject dif-
cult. This study investigated how an educational game and simulation affects the students
academic performance and multimedia ow experiences in a course, Introduction to Data
Structures. This work proposed an educational game called Star Chef which helps univer-
sity students to engage in learning based on the multimedia learning cognitive model and
experiential gaming model. A case study was performed with the participation of a teacher,
an assistant, and 110 university students during two semesters. Evaluation, surveys, and
interviews were conducted to explore research questions.
 Interactive Learning Environments 741

The statistical results show that using an educational game to learn data structure could
achieve better academic performance and multimedia ow experiences than using simulation
software. The enhanced learning result due to the use of educational game matches the studies
of Ford and Minsker (2003), and Chafn, Doran, Hicks, and Barnes (2009). The educational
game being able to provide more ow experience veried the resultant studies of Kiili (2005)
and Voiskounsky et al. (2004). The ndings also demonstrate a non-signicant positive
relationship between students multimedia ow and academic performance that also consists
with the resultant studies of Pavlas (2010) and Karimi et al. (2012). The experiment data show
that the two groups had a higher mean difference in playfulness and immersion. This implies
that educational games made the students better able to experience more playfulness and
immersion, thereby enhancing their self-actualization than simulation did. The mean is
when we want to design an educational game to assist learning, the game design should
strengthen the two factors: playfulness and immersion to reach better efcient and effective-
ness. According to the open-ended questionnaire, many students in the experimental group
found the educational game, Star Chef, attractive, playful, and useful in helping them
learn. Thus, they were satised with game-assisted learning. During interviews, many stu-
dents said that they were grateful to the teacher who spent time designing the game. More-
over, students pointed out that using the educational game, Star Chef, to conduct game-
assisted learning can engage in enjoyable experiences for learning data structures. Students
were excited and gained a feeling for the playful games when playing with classmates.
They also mentioned that their classes would be more interesting and their learning would
be more efcient and effective if similar games were used in other subjects.
Because the statistical results indicate a non-signicant positive correlation between
students multimedia ow and academic performance, we intend to continue working
with computer science teachers in conducting additional research to investigate new
design models and nd new multimedia ow characteristics that actuate signicant positive
correlations with academic performance. The multimedia ow characteristics will facilitate
the designing of efcient educational games that would help students learn abstract compu-
ter science concepts.

Notes on contributor
Tsung-Yu Liu received the M.S. degree in engineering science from National Cheng Kung University
in 1990. He received the Ph.D. degree in Electrical Engineering at National Taipei University of Tech-
nology, Taipei, Taiwan, R. O. C. His research interests include Internet, mobile computing, e-learning,
and computer games. He is currently an assistant professor at Lunghwa University of Science and
Technology, Taoyuan, Taiwan.

References
Baecker, R. (1998). Sorting out sorting: A case study of software visualization for teaching computer
science. In M. Brown, J. Domingue, B. Price, & J. Stasko (Eds.), Software visualization:
Programming as a multimedia experience (pp. 369382). Cambridge, MA: The MIT Press.
Chafn, A., Doran, K., Hicks, D., & Barnes, T. (2009). Experimental evaluation of teaching recursion
in a video game. In Stephen N. Spencer (Ed.), Proceedings of the 2009 ACM SIGGRAPH
Symposium on Video Games (pp. 7986). New York, NY: ACM.
Chen, H., Wigand, R., & Nilan, M. S. (1999). Optimal experience of web activities. Computers in
Human Behavior, 15, 585608.
Chin, W. W. (1998). The partial least squares approach to structural equation modelling. In G. A.
Marcoulides (Ed.), Modern methods for business research (pp. 295336). Mahwah, NJ:
Lawrence Erlbaum Associates.
742 T.-Y. Liu

Choi, D., Kim, J., & Kim, S. (2007). ERP training with a web-based electronic learning system: The
ow theory perspective. International Journal of HumanComputer Studies, 65(3), 223243.
Chung, C.-W., Huang, H.-P., & Hsiao, Y.-L. (2011). A Leisure benet research of undergraduate stu-
dents playing video games: Using a college as case study. Journal of Sport and Recreation
Management, 8(2), 3854.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155159.
Crookall, D., Oxford, R. L., & Saunders, D. (1987). Towards a reconceptualization of simulation:
From representation to reality. Simulation/Games for Learning, 17, 147171.
Crookall, D., & Saunders, D. (1989). Towards an integration of communication and simulation. In D.
Crookall & D. Saunders (Eds.), Communication and simulation: From two elds to one theme
(pp. 329). Clevedon: Multilingual Matters.
Csikszentmihalyi, M. (1975). Beyond boredom and anxiety. San Francisco, CA: Jossey-Bass.
Csikszentmihalyi, M. (1991). Flow: The psychology of optimal experience. New York, NY: Harper and
Row.
Ebner, M., & Holzinger, A. (2003). Instructional use of engineering visualization: Interaction-design
in e-learning for civil engineering. In J. Jacko & C. Stephanidis (Eds.), Humancomputer inter-
action, theory and practice (pp. 926930). Mahwah, NJ: Lawrence Erlbaum.
Farrell, C. (2005). Perceived effectiveness of simulations in international business pedagogy: An
exploratory analysis. Journal of Teaching in International Business, 16(3), 7188.
Finneran, C. M., & Zhang, P. (2003). A person-artefact-task (PAT) model of ow antecedents in com-
puter-mediated environments. International Journal of HumanComputer Studies, 59, 475496.
Ford, C. W., & Minsker, S. (2003). TREEZ an educational data structures game. Journal of
Computing Sciences in Colleges, 18(6), 180185.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equations models with unobservable vari-
ables and measurement error. Journal of Marketing Research, 18(1), 3950.
Ghani, J. A. (1991). Flow in humancomputer interactions: Test of a model. In J. Carey (Ed.), Human
factors in management information systems: Emerging theoretical bases (pp. 291312). Ablex,
NJ: Publishing Corp.
Gtz, O., Liehr-Gobbers, K., & Krafft, M. (2010). Evaluation of structural equation models using the
partial least squares (PLS) approach. In V. Esposito Vinzi, W. Chin, J. Henseler, & H. Wang
(Eds.), Handbook of partial least squares concepts, methods and applications (pp. 691711).
Berlin Heidelberg: Springer-Verlag.
Gredler, M. E. (2004). Games and simulations and their relationships to learning. In D. H. Jonassen
(Ed.), Handbook of research on educational communications and technology (pp. 571581).
Mahwah, NJ: Lawrence Erlbaum.
Hansen, S. R., Narayanan, N. H., & Hegarty, M. (2002). Designing educationally effective algorithm
visualizations: Embedding analogies and animations in hypermedia. Journal of Visual Languages
and Computing, 13(3), 291317.
Henseler, J., Ringle, C. M., & Sinkovics, R. R. (2009). The use of partial least squares path modeling
in international marketing. Advances in International Marketing, 20(1), 277320.
Hoffman, D. L., & Novak, T. P. (1996). Marketing in hypermedia computer-mediated environments:
Conceptual foundations. Journal of Marketing, 60, 5068.
Hoffman, L. D., & Novak, P. T., & Yung, Y. F. (1999). Measuring the ow construct in online environ-
ment: A structural modeling approach. Retrieved April 30, 2010, from http://www.yaffecenter.
org/downloads/Complete_List_of_Working_Papers/novak20.pdf
Hundhausen, D., & Douglas, S. (2004). Low-Fidelity algorithm visualization. Journal of Visual
Languages and Computing, 13(5), 449470.
Jackson, S., & Marsh, H. (1996). Development and validation of a scale to measure optimal experi-
ence: The ow state scale. Journal of Sport & Exercise Psychology, 18, 1735.
Jimoyiannis, A., & Komis, V. (2001). Computer simulations in physics teaching and learning:
A case study on students understanding of trajectory motion. Computers & Education, 36(2),
183204.
de Jong, T. (1991). Learning and instruction with computer simulations. Education & Computing, 6,
217230.
Karimi, R., Nosrati, M., & Hariri, M. (2012). Digital game-based learning plan with psychological
features. World Applied Programming, 2(9), 421424.
Kiili, K. (2005). Digital game-based learning: Towards an experiential gaming model. The Internet
and Higher Education, 8(1), 1324.
 Interactive Learning Environments 743

Konradt, U., Filip, R., & Hoffmann, S. (2003). Flow experience and positive affect during hypermedia
learning. British Journal of Educational Technology, 34, 309327.
Lawrence, A. (1993). Empirical studies of the value of algorithm animation in algorithm understand-
ing (Unpublished doctoral dissertation). Georgia Institute of Technology, Atlanta, GA.
Lawrence, A. W., Badre, A. N., & Stasko, J. T. (1994). Empirically evaluating the use of animations to
teach algorithms. In A. Ambler & T. Kimura (Eds.), Proceedings of the 1994 IEEE symposium on
visual languages (pp. 4854). Los Alamitos, CA: IEEE Computer Society Press.
Liu, T. Y. (2009). A context-aware ubiquitous learning environment for language listening and speak-
ing. Journal of Computer Assisted Learning, 25(6), 515527.
Liu, T.-Y., Chu, Y.-L., & Tan, T.-H. (2012). Using computer games in a computer course to improve
learning. In M. J. W. Lee & H. V. Leong (Eds.), Proceedings of the IEEE international conference
on teaching, assessment, and learning for engineering (TALE) (pp. W2C-1619). San Francisco,
CA: IEEE Education Society.
Liu, T. Y., Tan, T. H., & Chu, Y. L. (2009). Outdoor natural science learning with an RFID-sup-
ported immersive ubiquitous learning environment. Educational Technology & Society, 12
(4), 161175.
Magee, M. (2006). State of the eld review simulation in education. Calgary, AB: Canadian Council
on Learning.
Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist,
32, 119.
Mayer, R. E. (2001). Multimedia learning. New York, NY: Cambridge University Press.
Musselwhite, C. (2006). University executive education gets real. Training and Development
Magazine, 6, 5759.
Nunnaly, J. (1978). Psychometric theory. New York, NY: McGraw-Hill.
Olson, C. K., Kutner, L. A., Warner, D. E., Almerigi, J. B., Baer, L., Nicholi, A. M. Jr, & Beresin, E. V.
(2007). Factors correlated with violent video games use by adolescent boys and girls. Journal of
Adolescence Health, 41(1), 7783.
Paras, B., & Bizzocchi, J. (2005). Game, motivation, and effective learning: An integrated model for
educational game design. Paper presented at the Digital Games Research Association (DiGRA):
Changing Views Worlds in Play, Vancouver, BC.
Pavlas, D. (2010). A model of ow and play in game-based learning: The impact of game characteristics,
player traits and player states (Doctoral Dissertation). University of Central Florida, Retrieved
September 11, 2013, from http://www.navistudios.net/Pavlas_Davin_A_201012_PhD.pdf
Pavlou, P. A., & Fygenson, M. (2006). Understanding and predicting electronic commerce adoption:
An extension of the theory of planned behavior. MIS Quarterly, 30(1), 115143.
Pearce, J. M., Ainley, M., & Howard, S. (2005). The Ebb and ow of online learning. Computers in
Human Behavior, 21(5), 745771.
Prensky, M. (2001). Digital game-based learning. New York, NY: McGraw-Hill.
Ringle, C. M., Wende, S., & Will, A. (2005). SmartPLS-Version 2.0. Germany: University at
Hamburg. Retrieved January 12, 2013, from http://www.smartpls.de
Saraiya, P., Shaffer, C., McCrickard, D. S., & North, C. (2004). Effective features of algorithm visu-
alizations. ACM SIGCSE Bulletin, 36(1), 382386.
Schiefele, U., & Rheinberg, F. (1997). Motivation and knowledge acquisition: Searching for mediat-
ing processes. In M. L. Maehr & P. P. Pintrich (Eds.), Advances in motivation and achievement
(Vol. 10, pp. 251301). Greenwich, CT: JAI Press.
Skadberg, Y. X., & Kimmel, J. R. (2004). Visitors ow experience while browsing a web site:
Its measurement, contributing factors, and consequences. Computers in Human Behavior, 20,
403422.
Sweetser, P., & Wyeth, P. (2005). Game ow: A model for evaluating player enjoyment in games.
Computers in Entertainment, 3(3), 124.
Tan, B., & Seng, J. L. K. (2010). Game-based learning for data structures: A case study. Paper
presented at the 2nd international conference on in computer engineering and technology
(ICCET 2010), Chengdu, China.
Tversky, B., Morrison, J. B., & Betrancourt, M. (2002). Animation: Can it facilitate? International
Journal of HumanComputer Studies, 57(4), 247262.
Virvou, M., Katsionis, G., & Manos, K. (2005). Combining software games with education:
Evaluation of its educational effectiveness. Educational Technology & Society, 8(2), 5465.
744 T.-Y. Liu

Voiskounsky, A. E., Mitina, O. V., & Avetisova, A. A. (2004). Playing online games: Flow experience.
PsychoNology Journal, 2(3), 259281.
Webster, J., Trevino, L. K., & Ryan, L. (1993). The dimensionality and correlates of ow in human
computer interaction. Computers in Human Behavior, 9, 411426.
Wold, H. O. (1985). Partial least squares. In S. Kotz & N. L. Johnson (Eds.), Encyclopedia of statisti-
cal sciences (Vol. 6, pp. 581591). New York, NY: Wiley.