Knowledge-Based Systems 22 (2009) 275278

Contents lists available at ScienceDirect

Knowledge-Based Systems
journal homepage: www.elsevier.com/locate/knosys

Augmented fuzzy cognitive maps for modelling LMS critical success factors
Jose L. Salmeron
School of Engineering, University Pablo de Olavide (Spain), Ctra. de Utrera, km. 1, 41013 Seville, Spain

a r t i c l e i n f o a b s t r a c t

Article history: This paper proposes to build an Augmented Fuzzy Cognitive Map-based for modelling Critical Success
Available online 7 February 2009 Factors in Learning Management Systems. The study of Critical Success Factors helps decision makers
to extract from the multidimensional learning process the core activities that are essential for success.
Keywords: Using Fuzzy Cognitive Maps for modelling Critical Success Factors provides major assistance to the
e-Learning e-learning community, by permitting prediction comparisons to be made between numerous tools
Fuzzy Cognitive Maps measured by multiple factors and its relations.
Critical Success Factors
2009 Elsevier B.V. All rights reserved.

1. Introduction student; (3) information technology; and (4) university support.

Learning Management System (LMS) are software packages,
usually web-based, that enables the management and delivery of
learning content and resources to students. The main goal is to pro-
pose a tool for evaluate LMS in a social computing point of view.
In addition, a Fuzzy Cognitive Map (FCM) is proposed for model-
ling Critical Success Factors (CSFs). FCM is a knowledge-based meth-
odology suitable to model complex systems and handle information
from an abstract point of view [1]. Soft computing techniques, as
FCMs, have been successfully applied to model complex systems
involving concepts, factors, states, events and trends [2].
2. Theoretical background
This work is focused on the IT one.
A LMS is a software bundle that enables the management and
delivery of learning contents and collaborative resources to online
students. While most LMS are commercially delivery (e.g.: WebCT,
Scholar360, Desire2Learn, CentraOne, ANGEL Learning, and Edu-
mate), open source software models do exist (e.g.: Moodle, .LRN,
Atutor, and LON-CAPA).
Numerous scientic publications address the issue of CSFs in
the e-learning eld [8]. However, little efforts have been done for
modelling LMS Critical Success Factors. In addition, none of them
used a formal methodology. Therefore, a formal method to this
to model CSF of LMSs is a useful endeavour.

2.1. Critical Success Factors 3. Research model

The study of CSF was developed by Rockart [3] as a method to
enable Chief Executive Ofcers to recognize their own information
needs so that information systems could be built to meet those
needs. Rockart dened CSFs as the needed elements for achieving a
goal. This concept has wide acceptance among scholars and practi-
tioners [4].
Some authors [5] analysed some aspects of CSF just by the use
of personal interviews. However, a few of them used a formal
methodology [6,7].
2.2. Learning Management Systems
e-Learning CSFs within a higher education institution can be
grouped [8] into the following categories: (1) instructor; (2)
E-mail address: salmeron@upo.es
3.1. Fuzzy Cognitive Maps
FCMs [911] constitute neuro-fuzzy systems, which are able to
incorporate experts knowledge [12]. FCM describes a cognitive
map model with two characteristics. Firstly, causal relationships
between nodes have different intensities, represented with a num-
ber from 0 to 1. As we analyze the cognitive maps, the causal value
that they establish is the sign plus or minus. However, a FCM sub-
stitutes these signs by a fuzzy value between 1 and +1 where the
zero value indicates the absence of causality. Secondly, it involves
feedback, where the effect of change in a concept node may affect
other concept nodes.
From an Articial Intelligence perspective, FCMs are supervised
learning neural systems, whereas more and more data is available
to model the problem, the system becomes better at adapting itself
and reaching a solution [6].

0950-7051/$ - see front matter 2009 Elsevier B.V. All rights reserved.
doi:10.1016/j.knosys.2009.01.002
276 J.L. Salmeron / Knowledge-Based Systems 22 (2009) 275278

With the purpose of model the CSFs, advice was taken from a adjacency matrix of FCMA is AA wAij ; and the adjacency matrix
panel of experts. The expert panel number is quite difcult to of FCMB is AB wBij . The augmented adjacency matrix is
establish and no study has been conclusive with respect to it !
[13]. We select six experts for this study. Literature [14] suggests wAij 0
a range of 518 to be an ideal number. Multiple choices were con- A
0 wBij
templated. In this sense, the main selection criterion considered
was recognized knowledge in research topic, absence of conicts If there are common nodes, then the element wAug
ij in the aug-
of interest and geographic diversity. All conditions were respected. mented matrix is
Pn k
3.2. Augmented Fuzzy Cognitive Maps k1 wij
wAug
ij ;
n
It is possible to represent the different relationships among fac-
tors by means of a matrix (A), called the adjacency matrix: n being the number of FCMs added, one by expert, k the identier of
0 1 each expert, and i and j the identier of the relationships (Fig. 2).
::: ::: ::: Let two FCMs with common nodes. Starting from each adja-
B C
A @ ::: eij ::: A; eij 2 1; 1j8i; j cency matrix
::: ::: ::: A B C
0 1
eij indicates the relationship between the i and j concepts, enabling A 0 0:5 0:2
us to obtain values between [1, 1]. Three types of relationships can AExpert 1
B C
B @0 0 0:1 A
be seen: (1) eij > 0, indicating a positive relationship, (2) eij < 0, indi-
cating a negative one, and (3) eij = 0, where no relationship exists. C 0 0 0
Therefore, when an expert assigns a value eij, three issues must be
A C D
considered. Firstly, the eij intensity to indicate how strong the i con- 0 1
cept is in j. Secondly, the sign (+/) of eij must be decided to indicate A 0 0:4 0
AExpert 2
B C
if the relationship between the i and j concepts is direct or inverse. C @ 0 0 0A
Lastly, the causality relationship needs to be indicated to establish if D 0:7 0:1 0
the i concept is a cause of j or vice-versa. This process is presented
in Fig. 1. The augmented adjacency matrix will be built as follows.
Various methodologies could be used in order to reach a con-
A B C D
sensus among the experts [15]. Finally, the Augmented FCM ap- 0 1
A 0 0:25 0:1 0
proach has been adopted, because it doesnt need that experts
AAug B B 0 0 0:05 0 C
change slightly their judgement for consensus as Delphi methodol- B C
B C
ogy [16]. The augmented adjacency matrix is built adding the adja- C @ 0 0 0 0A
cency matrix of each expert [1]. D 0:35 0 0:05 0
Let two FCMs with no common nodes. FCMA with cAi as nodes
FCMA fcAi g, and FCMB with cBj as nodes FCMB fcBj g. The The resulting Augmented FCM graph is shown in the Fig. 3.

Fig. 1. Basic stages for building a Fuzzy Cognitive Map.

Fig. 2. Building an Augmented Fuzzy Cognitive Map.

Fig. 3. Adding Fuzzy Cognitive Maps.

 J.L. Salmeron / Knowledge-Based Systems 22 (2009) 275278
Table 1
LMS Critical Success Factors.
Label Critical Success Factor
A Asynchronous communication tools
B Synchronous communication tools
C Usability
D Content structure
E Standards compliancy
F LMS cost
G Easy maintenance
H Students attitude
I Assignments
J Multimedia
4. Modelling LMSs Critical Success Factors
Ten CSFs was proposed by experts (Table 1). Communication
tools are extremely important in an e-learning environment. Asyn-
chronous ones could be used for providing students work in teams,
rather than trying to respond to each individual posting [17]. On
the other hand, synchronous communication tools could be used
for meeting with smaller groups of students online.
The need for usability has been recognized in web design and
development literature as critical when determining user satisfac-
tion in such systems. Therefore LMSs usability can signicantly af-
fect learning [18].
Content structure is focused on the structure of the learning
materials, rather than classical system usability. Regarding stan-
dards, the unshared learning resource will reduce its use and use-
fulness. In this sense, standards as Sharable Content Object
Reference Model (SCORM) resolve that issue. LMS costs and main-
tenance are obviously an important factor for managers, rather
than students, but it is critical for assessing the efforts associated
with the LMS use in the long term.
One central point is the students attitude to IT. If they are com-
fortably with the LMS their performance will be higher. Online
assignments could motivate to students for going on. Finally, mul-
timedia has been included in LMSs in the last years.
After the CSFs detection, the relationships and its intensities be-
tween them were modelled. After this, the Augmented Fuzzy Cog-
nitive Map was built as explained above. The resulting model is
shown in the Fig. 4.
Fig. 4. Augmented Fuzzy Cognitive Map.
277
5. Conclusions
The LMS selection is a complicated process of developing an
integrated information technology system. This paper, according
with an expert panel, specied ten LMS Critical Success Factors cat-
egories that can assist decision makers to efciently and effectively
select e-learning technologies. In addition, CSFs have been mod-
elled as an Augmented FCM. Furthermore, the relationships be-
tween the CSFs have been shown in the model.
The ten CSFs are asynchronous and synchronous communica-
tion tools, usability, content structure, standards compliancy, cost,
easy maintenance, students attitude, assignments, and multime-
dia. In addition, a new AI-based tool (Augmented Fuzzy Cognitive
Maps) is incorporated to e-learning research.
References
[1] B. Kosko, Fuzzy Engineering, Prentice Hall, 1996.
[2] V. Georgopoulos, C. Stylios, Augmented fuzzy cognitive maps supplemented
with case based reasoning for advanced medical decision support, in: Soft
Computing for Information Processing and Analysis, Springer, 2005.
[3] J.F. Rockart, Chief executives dene their own data needs, Harvard Business
Review 57 (2) (1979) 8193.
[4] IHEP, Quality on the Line: Benchmarks for Success in Internet-based Distance
Education, 2000. Available from: http://www.ihep.com/Pubs/PDF/Quality.pdf
(accessed 02.02.08).
[5] P. Poon, C. Wagner, Critical success factors revisited: success and failure cases
of information systems for senior executives, Decision Support Systems 30 (4)
(2001) 393418.
[6] L. Rodriguez-Repiso, R. Setchi, J.L. Salmeron, Modelling IT projects success with
fuzzy cognitive maps, Expert Systems with Applications 32 (2) (2007) 543
559.
[7] J.L. Salmeron, I. Herrero, An AHP-based methodology to rank critical success
factors of executive information systems, Computer Standards and Interfaces
28 (1) (2005) 112.
[8] H.M. Selim, Critical success factors for e-learning acceptance. Conrmatory
factor models, Computers and Education 49 (2) (2007) 396413.
[9] B. Kosko, Fuzzy cognitive maps, International Journal of ManMachine Studies
24 (1) (1986) 6575.
[10] K.C. Lee, J.S. Kim, H.N. Chung, S.J. Kwon, Fuzzy cognitive map approach to web-
mining inference amplication, Expert Systems with Applications 22 (2002)
197211.
[11] S. Lee, B.G. Kim, K. Lee, Fuzzy cognitive map-based approach to evaluate EDI
performance: a test of causal model, Expert Systems with Applications 27 (2)
(2004) 287299.
[12] C.D. Stylos, P.P. Groumpos, Fuzzy cognitive maps in modeling supervisory
control systems, Journal of Intelligent and Fuzzy Systems 8 (2000) 8398.
[13] H.K. Cho, M. Turoff, S.R. Hiltz, The impacts of Delphi communication structure
on small and medium sized asynchronous groups: preliminary results, in: 36th
Annual Hawaii International Conference on System Sciences, 2002, pp. 1726.
278 J.L. Salmeron / Knowledge-Based Systems 22 (2009) 275278

[14] C. Okoli, S. Pawlowski, The Delphi method as a research tool: an example,
design considerations and applications, Information and Management 42 (1)
(2004) 1529.
[15] N. Bryson, A. Mobolurin, A. Joseph, Generating consensus fuzzy cognitive
maps, Intelligent Information Systems 8 (10) (1997) 231235.
[16] S. Bueno, J.L. Salmeron, Fuzzy modeling enterprise resource planning tool
selection, Computer Standards and Interfaces 30 (3) (2008) 137147.
[17] R.F. Branon, C. Essex, Synchronous and asynchronous communication tools in
distance education, TechTrends 45 (1) (2001) 4142.
[18] M. Debevc, J.L. Bele, Usability testing of e-learning content as used in two
learning management systems, European Journal of Open and Distance
Learning 14 (1) (2008). Available from: http://www.eurodl.org/materials/
contrib/2008/Debevc_Bele.htm.