Visualisation of Knowledge Mapping for Information Systems Evaluation:

A Manufacturing Context
Amir Sharif
Brunel Business School
Brunel University
amir.sharif@brunel.ac.uk
Muhammad Mustafa Kamal
Brunel Business School
Brunel University
muhammad.kamal@brunel.ac.uk
Zahir Irani
Brunel Business School
Brunel University
zahir.irani@brunel.ac.uk
Abstract
Information Systems (IS) are primary enablers in
instigating organisational change, increasing
organisational responsiveness and decreasing supply
chain overheads. This paper aims to contribute
through exploring and visualising Knowledge
Mapping (KM) from the perspective of Information
Systems Investment Evaluation (ISIE). Complexity of
IS evaluation process increases with the increase in
intricacy of IS. A number of interrelating factors (e.g.
costs, benefits and risks) contribute towards the
intricacy of IS evaluation. There seems to be a
growing need to evaluate the IS investment decision-
making processes, to better understand the often far
reaching repercussions related with IS
implementation and interconnected Knowledge
Components (KC). In seeking to edify the often vague
IS evaluation process, this paper attempts to,
emphasise the increase of knowledge and learning
through the application of a Fuzzy Cognitive
Mapping (FCM) technique. The resulting FCM
determines the intricate and developing behaviour of
causal relationships within the KM area.
1. Introduction
In todays global economic environment, the
world-wide business organisational ambience has
significantly transformed which contributed to
todays competition in many sectors of industry,
commerce and government [1]. In such a hyper-
competitive state, organisations need to be aware that
their operations would abruptly cease to function
should the technology that underpins their activities
and help to automate organisational workflows, ever
come to a halt. Thus, business organisations need to
constantly explore state-of-the-art ways to re-
orchestrate their products and delivery of services. In
recent years, enterprise IS have played a significant
role in supporting organisational agility, dealing with
improbability in decision-making, supplementing
their competitiveness, and coordinating information
in the supply chain [22, 46]. According to Stockdale
and Standing [44] a substantial growth in enterprise
IS investments has enforced a number of business
organisations to focus on the usefulness and
assessment of their business processes and
approaches. The latter argument is supported by
Sharif et al., [38] who state that in essence, enterprise
IS evaluation is a decision-making process, which
enables an organisation in defining costs, benefits,
risks and more importantly, in highlighting the
repercussions of investing in IS. According to
Hedman and Borell [10] the evaluation of enterprise
IS is essentially based upon knowledge of the
organisation and strategic, tactical and operational
prerequisites. Such enterprise IS support
organisations in capturing and storing knowledge of
human experts and then imitating human reasoning
and decision-making in the design, production and
delivery of manufactured goods [22, 44].
Regardless of using any approach in an
organisation, the evaluation process aims to ascertain
a link between the anticipated value of an investment
and analysis [often quantitative] of the costs, benefits
and risks [13]. To address the requisite for an
organised evaluation instrument for supporting the
top management in better understanding the human,
organisational and technical repercussions of their
investment decisions, academics and practitioners
have approached investment decision-making from a
variety of perspectives. For example, Expert Systems
(ES) perform a number of tasks that are carried out
by humans with certain knowledge and experience.
Assessing performance necessitates an understanding
of human expert performance and in the way it can
be assessed. The knowledge and investigational
learning that is a prerequisite within a decision-
making process, is thus essential to the ending result.
Kim et al., [20] points here that sharing and
managing knowledge in all its forms requires being
stable and controlled to make the most of its effect.
The motivation for this paper is to attempt to map
out and visualise the range and aspects of knowledge
that are relevant to the ISIE process in the
manufacturing context, based upon the extant
literature and managerial, operational, organisational,
technological and strategic aspects of an
organisations strategy. As such, the motivation for
this research rests on understanding what aspects of
the relevant expert knowledge ultimately drive this
knowledge-intensive evaluation task, thereby
highlighting some of the dynamic inter-relationships
inherent within the field and in a practical context.
This paper therefore seeks to investigate and map
factors influencing the decision-making process for
ISIE and their pertinent KC. The mapping of ISIE
factors and related KCs is achieved by using a FCM
technique, resulting in exploring the inter-
relationships and intricacies of decision-making
2014 47th Hawaii International Conference on System Science
978-1-4799-2504-9/14 $31.00 2014 IEEE
DOI 10.1109/HICSS.2014.433
3478
factors in a manufacturing context. FCM was
selected to explore the cognitive perspective of ISIE
factors the focus is to examine the link based
context, not the agent based context. Moreover, the
authors are not limiting or neglecting the usefulness
of other techniques, instead we are seeking to apply
FCM as it has limited application to the ISIE related
context as compared to agent based modelling.
2. Research Design
A robust research design (based on four steps)
was constructed and proposed, which acted as a
blueprint for the research process. Each step in this
process acts as a basis for the next step:
x Step 1 is about investigating factors that define
ISIE in the manufacturing sector. This was
achieved by studying the extant general IS and
manufacturing literature in-depth. This research
exercise facilitated in understanding of ISIE
practices in manufacturing sector and as a
result, supported the identification and defining
of 15 influential factors. These factors are
classified according to the proposed MOOTS
dimensions Managerial, Organisational,
Operational, Technological, and Strategic.
x Step 2 is about identifying and correlating KC
with the relevant ISIE factor. These KCs are
identified using the proposed five-step Pairwise
IS Theory Equivalence (PIE) framework (Figure
1). The proposed PIE process is further divided
into five sub-steps. For example, for each ISIE
factor a supposition is developed, then, two
relevant IS theories are identified for each ISIE
factor this allowed more flexibility in
extracting a relevant KC. Then a rationale is
developed that supports the identification of the
dependent and independent constructs relevant
to each IS theory. From these constructs only
those are selected that clearly associate the ISIE
factor with the two chosen IS theories. After
identifying the constructs, a relevance check is
conducted this sub-step is to ensure the whole
process is moving rightly, resulting in
identifying the gap. This void is then translated
into a single KC for each ISIE related factor.
x Step 3 details the process by which the MOOTS
and the PIE classification approach is combined
with expert knowledge to construct a matrix (of
ISIE factors. Through pairwise comparison
the so-called Field Anomaly Relaxation (FAR)
as stated by Rhynne [37] these factors then
determine the scope of the knowledge to be
mapped. Each of these factors is then assigned
fuzzy weightings using a range of positive to
negative values. A directed graph can be
constructed of these pairwise fuzzy values
which ultimately becomes the FCM.
x Step 4 involves the algorithmic process of the
FCM simulation. This needs several simulation
scenarios to be identified. These scenarios are
effectively vectors which denote the initial
states of the ISIE factors from Step 3. These
vectors are details of expert knowledge encoded
into numerical fuzzy values per factor. These
vectors are, in turn, fed into the simulation
algorithm where the successive nodal states of
each factor in the directed graph are updated
from the preceding nodal state until equilibrium
is achieved. The output values for each node,
hence the ISIE factor, are plotted against
iterative steps. Then updated FCM is created
through calculating the inverse of the fuzzy
weight matrix and the final ISIE nodal values.
3. Information Systems Investment
Evaluation
Information systems constitute a substantial
financial investment for organisations [12], thus, they
should be justified, evaluated and managed with
caution [4]. To invest in new IS and or enhance the
efficiency of existing IS and technological
infrastructure, managements are required to consider
investment risks and payoffs and acquire knowledge
of existing IS inefficiencies, respectively [21]. In this
context, evaluating IS investments becomes a
requisite for management, for the purpose that
enterprise IS implementation influence the way
organisations operate and impact their strategies,
tactics and operational decisions. Evaluation is vital
to rationalise higher IS investment costs,
improbability of returns from IS investments and act
as a control and management mechanism [4].
However, Stockdale and Standing [44] argue here
that in evaluating IS, a crucial task is to develop
frameworks that are effectively standardised and can
be applied to a broad range of applications but at the
same time thoroughly detailed to offer effective
support. In this context, systematic but equally
coherent methodologies are a prerequisite to
determine IS justification concerns emerging from
the complexity of topical technologies [8]. Though,
successfully pursuing ISIE can lead an organisation
to sustain its corporate viability and success.
3.1 ISIE in Manufacturing Context
To improve business operations and productivity
of the supply chain, manufacturing organisations
have developed themselves as responsive industrial
entities by implementing pertinent IS. The latter
argument is supported by Mondragon et al., [26] who
report that the operations of agile organisations
require the existence of efficient IS within their
supply chains. Wang et al., [47] report that
3479
information systems are vital for (a) instigating
transformational changes within and between
manufacturing organisations and (b) the efficient
functioning of their operations i.e. design,
production, and delivery of manufactured goods.
According to Coronado et al., [5] IS implementation
enhances the agility of manufacturing organisations
and enables the development of resilient and
collaborative relations. Despite the significance of IS,
there are a number of cases where manufacturing
organisations have been unsuccessful in completing
their IS projects. This has resulted in manufacturing
organisations failing to gratify their key stakeholders.
For example Irani et al., [16] in their study on SME
manufacturing enterprises focused on gaining
insights SME IS implementation failures. The
reasoning concluded from their IS failure was failing
to realise the human and organisational factors
influencing the evaluation/implementation process.
Perera and Costa [33] also alleged that most of the IS
investments by manufacturing organisations have not
reaped the desired returns. This is because selecting
an appropriate IS is primarily complicated and vital
decision for manufacturing organisations [2].
Such commentaries on IS performance show that
while manufacturing organisations have benefited
from IS, many of them have been less than satisfied.
One of the apparent reasons for this discontentment is
reported by Small [41] who states that inapt IS
investment justification practices can result in
organisations not able to differentiate the benefits that
manufacturing IS may be capable of conferring,
whilst, there is a lack of evidence on returns on IS
investment as managements have failed to prove the
tangible returns on the resources deployed to plan,
develop, implement and operate IS. Advocates claim
that ISIE when managed and pursued effectively can
have a positive impact on organisational performance
and productivity [9]. Thus, a formal justification
proposal must be prepared and accepted by decision-
makers, prior to IS investments [14].
3.2 MOOTs Classification of ISIE Factors
In Table 1, the authors present 15 factors (based
on the MOOTS dimension) that define ISIE in the
manufacturing context. This list of factors may not
be comprehensive, nevertheless, these 15 factors are
identified based on the literature specifically
focusing on IS, ISIE, manufacturing organisations
and supply chain management. These factors also
directly related to the context of this research.
4. From Knowledge to Knowledge
Management
Knowledge has long been deliberated as a vital
organisational asset supporting the top management
Dimension ISIE Factors Ref.
Managerial
Management Commitment (MC) [29]
Management Style (MS) [29]
Managerial Capability (MC*) [6]
Organisational
Organisational Cultural (OC) [19]
Organisational Performance (OP) [45]
Organisational Size (OS) [25]
Operational
Employee Commitment (EC) [27]
Training and Education (TE) [3]
Information Systems & Manufacturing Agility
(ISMA)
[31]
Technological
Enterprise Integration in Manufacturing (EIM) [31]
Information Systems & Organisational Fit (ISOF) [15]
Information Systems Quality Output &
Performance (ISQOP)
[36]
Strategic
Strategic Information Systems Impact (SISI) [11]
Business Strategy & IS Alignment (BSISA) [36]
Strategic IS Business Partnership (SISBP) [48]
Table 1: MOOTS Classification of Factors Defining ISIE
in their decision-making process and augmenting
organisational competitiveness [34]. Advocates
highlight that its effective management is essential
for organisational success [7]. Knowledge is a blend
of experience, values, circumstantial information and
professional understanding that support the
evaluation and incorporation of new experiences and
information. Researchers have thus studied
Knowledge Management (KM) so as to define its
involvement in managing and leveraging
organisational knowledge [28]. Organisations that
focus on KM tools and techniques recognise the
significance of exchanging knowledge, as this
sharing of exchange increases efficiency and
sustained competitiveness. Such use of knowledge
tools and techniques to generate economic benefits is
one facet of knowledge economy [42]. According to
Powell and Snellman [35], the key component of
knowledge economy is a higher dependence on
intellectual competencies of human resources than
contribution from other physical or natural resources.
From the economic perspective, Shin [42] claims that
employing KM tools and techniques for knowledge
sharing should not be assumed on a simple conjecture
that organisational competitiveness is positively
interrelated with knowledge sharing. Shin [42]
further argues that knowledge sharing has a negative
link with competitiveness and a positive one.
Considering both the knowledge-sharing aspects, the
economic viewpoint may offer a way to explore how
to curtail barriers and promote enablers so as to
acquire knowledge sharing benefits in organisations.
Thus, three important aspects can be drawn from
the above discussion i.e., People (organisational and
cultural aspects of the use of knowledge); Process
(methods and techniques for managing the flow of
knowledge); and Technology (tools and infrastructure
that provide access to knowledge). From these three
key aspects, a rational theme has been to relate
explicit and tacit forms of knowledge together [17].
3480
In the context of organisational knowledge, Nonaka
[30] reported that this level of knowledge is created
through a series of socialisation, externalisation,
combination and internalisation that transform
knowledge between the tacit and explicit modes. In
line with this dynamic process of knowledge
creation, the process of transmitting tacit knowledge
to explicit (and vice versa), is a collective act, where
knowledge is relocated and shared with other
individuals in the organisation. Two key points can
be extracted from the knowledge conceptualisation:
x First, knowledge is in a reformed mode; though,
to make an individuals knowledge useful for
others, it must be conveyed effectively and that
it is understandable and open to others. This can
be attained by using expert systems the
fundamental idea is that knowledge from the
human mind is stored in the computer and users
call upon the computer for specific advice as
and when necessary. The computer can make
extrapolations and reach to a decision. Then like
a human expert, the computer offers suggestions
and explains, if need be, the rationality behind
the suggestion. Expert systems offer an effective
and flexible means of exploring solutions to
problems that often cannot be dealt with by
other, more orthodox approaches.
x Second, hoard of information is of limited value
to organisations it is only when this
information is dynamically processed in an
individuals mind through a process of
discussion and learning that it can be effective.
At this point in time, KM plays a vital role by
performing as a meticulous process for managing
intellectual and knowledge assets to meet
organisational aim and objectives. KM aims to make
knowledge approachable and re-utilisable for and by
the organisation. Therefore, knowledge not only
exists in documents and repositories, but it becomes
rooted in individuals minds over time and is
revealed through their actions and behaviours.
Likewise, in an organisational context, decision
makers and their decision-making processes are
influenced by the knowledge that is generated as a
result of evaluating organisational IS investments
[18]. Kulkarni et al., [24] highlight that knowledge is
entrenched and streams across various units within an
organisation. For example, experts with specific
domain capability, categorical best practice measures,
or lessons learned from related experiences,
documents, daily operational practices, and IS. It is,
therefore, important to understand the different types
of KCs so as to uncover its likely contribution to the
performance of an organisation [32].
4.1 Process of Identifying ISIE related KC
In an attempt to determine the KC resulting from
ISIE, a five-step Pairwise IS Theory Equivalence
(PIE) process is followed (Figure 1). Each ISIE factor
defined under MOOTS classification will follow the
same process in order to identify related knowledge
components. As highlighted in Figure 1, the steps
include (with an example of how a KC for
management commitment factor is identified from
Step 1 to Step 5 in Table 2) the following:
x Step 1 is about identifying the
assumptions/starting point For each ISIE
factor an assumption is developed. The
assumption is divided into the Focus and
Dependence of the ISIE factor. For example,
Focus signifies the central theme i.e. the
management is committed to ISIE this
indicates the Focus of management; whereas,
Dependence shows the state of being
determined, influenced or controlled by
something else i.e. MC is dependent on the
availability and utilisation of resources. From
this assumption (i.e., dependence), the keywords
extracted are availability, utilisation, resources,
and evaluation.
x Step 2 is about identifying the relevant IS
theories, models or frameworks In this step,
the authors identified two relevant IS theories
for each ISIE factor. The decision to select two
appropriate IS theories, models, or framework
was made on the understanding that it would
allow flexibility in extracting a relevant KC. For
example, for the management commitment
factor, the resource-based view and contingency
theory were considered relevant based on their
dependent and independent constructs.
ISIE Factors
STEP 4
Relevance Check
Identifying the Link
between Step 1 and
Step 3
ISIE related Knowledge
Components
STEP 5
Identifying the
Gap as a result of
Step 4
STEP 1
Identifying the
assumptions /
starting point
for ISIE Factor
A
ssu
m
p
tio
n
s IS
T
h
e
o
r
ie
s
STEP 2
Identifying relevant IS
theories, models or
frameworks for each
ISIE Factor
Dependent /
Independent Constructs
STEP 3
Rationale Identifying
the Key Dependent/
Independent Constructs
that link ISIE Factor
with a relevant Theory
Gap Analysis
R
e
le
v
a
n
c
e
K
C
Figure 1: Three-Step PIE Process for Identifying ISIE
related KC
3481
x Step 3 is about developing the rationale
Identifying the main dependent and independent
constructs relevant to an IS theory, model or a
framework. However, from these available
constructs, only those constructs were selected
that associate an individual ISIE factor with the
two relevant IS theories, models or frameworks.
For example, the constructs that were deemed
relevant from the two IS theories, models or
frameworks are assets, resources, efficiency,
capabilities and organisational performance.
x Step 4 is about conducting a relevance check
Identifying the link between Step 1 and Step 3.
This relevance check enabled the authors to
relate the keywords extracted from the
assumption to the dependent and independent
constructs of each IS theory, model or
framework to identify a gap.
Table 2: Example of Managerial Dimension ISIE Factor
(Management Commitment) and related KC
x Step 5 is about identifying a gap as a result of
Step 4. Based on the relevance check of
keywords and dependent/independent constructs
a KC is extracted. The authors assert that a KC
is based upon relevant IS theories, models or
frameworks that, in turn, support that KC. For
example, for the management commitment
factor, the knowledge component identified is
Effective Use of Resources.
A similar process for extracting the relevant KCs
is followed for each of the other 14 ISIE factors. The
remaining ISIE factors and their related KCs are
presented in Table 3.
Table 3: All ISIE Factors and their related KCs
5. Visualising the Knowledge Map
The 15 ISIE knowledge components presented in
Tables 2 and 3 were grouped into six key thematic
areas as shown as follows, i.e., constructed to be
within a morphological field of factors [37].
x ICT Actual Use of IS, Use of System.
x MGMT Past Management Experience, Better
Management of Resources.
x PERF Improved Performance and
Management of Resources, Effective
Organisational Benchmarks and Performance,
Performance Management Metrics, Consistent
Information Output, Radical Transformation, &
Quality Production, Performance Measurement.
x RES Effective Use of Resources,
Development and Effectiveness of Relationship.
MANAGERIAL DIMENSION
S
T
E
P

1

A
s
s
u
m
p
t
i
o
n
s
(a) FOCUS: Management is committed to
evaluating their IS investments.
(b) DEPENDENCE: MC is dependent on
the availability and effective utilisation of
financial and other organisational resources
e.g. if there is enough investment to
implement and evaluate IS, management
will be committed towards promoting /
pursuing the evaluation.
S
T
E
P

2

I
S
T
h
e
o
r
i
e
s
Resource
Based
View
Contingency
Theory
S
T
E
P

3

M
a
i
n

D
e
p
e
n
d
e
n
t

&

I
n
d
e
p
e
n
d
e
n
t

C
o
n
s
t
r
u
c
t
s

f
o
r

S
e
l
e
c
t
i
n
g

T
h
e
o
r
i
e
s

I
n
d
e
p
e
n
d
e
n
t
x Assets
x Capabilities
x Resources
x Strategy
x Technology
x Task
x Organisational Size
x Structure & Culture
D
e
p
e
n
d
e
n
t
x Competitive
Advantage
x Organisational
Performance
x Efficiency
x Organisational
Performance
S
T
E
P

4

R
e
l
e
v
a
n
c
e

C
h
e
c
k
R
e
l
e
v
a
n
t

C
o
n
s
t
r
u
c
t
s

f
r
o
m

S
t
e
p

3
x Assets and Resources
x Efficiency
x Capabilities
x Organisational Performance
R
e
l
e
v
a
n
t

K
e
y
w
o
r
d
s

f
r
o
m

S
t
e
p

1
x Availability
x Utilisation
x Resources
x Evaluation
S
T
E
P

5

K
C
r
e
l
a
t
e
d

t
o

M
C
x Effective Use of Resources
ISIE Factors Knowledge Components
M
a
n
a
g
e
r
i
a
l


D
i
m
e
n
s
i
o
n
MC Effective Use of Resources
MS Past Management Experience
MC*
Improved Performance &
Management of Resources
O
r
g
a
n
i
s
a
t
i
o
n
a
l


D
i
m
e
n
s
i
o
n
OC
Effective Organisational
Benchmarks & Performance
OP
Performance Management
Metrics
OS
Better Management of
Resources
O
p
e
r
a
t
i
o
n
a
l


D
i
m
e
n
s
i
o
n
EC Actual Use of IS
TE
Skills Identification & System
Training
ISMA
Consistent Information
Output
T
e
c
h
n
o
l
o
g
i
c
a
l

D
i
m
e
n
s
i
o
n
EIM Radical Transformation
ISOF Use of System
ISQOP
Quality Production &
Performance Measurement
S
t
r
a
t
e
g
i
c


D
i
m
e
n
s
i
o
n
SISI
Enhancing Organisational
Competitiveness
BSISA
Strategic Alignment/Fit
Procedures
SISBP
Development & Effectiveness
of Relationship
3482
x SKILLS Skills Identification & Systems
Training.
x STRAT Enhancing Organisational
Competitiveness, Strategic Alignment/Fit
Procedures.
By doing so, the authors wished to carry out a
pairwise analysis to determine and remove any
redundant / duplicated factors. This approach has
been successfully used before [39]. In comparing any
and every two sets of factors, a reduced
morphological field was generated leading to an 83%
reduction of ISIE constructs (i.e. from 90 to 15). The
method for doing so was based on identifying those
ISIE pairwise combinations where four or more
similar dependent / independent sub-constructs
existed. The resulting reduced morphological field of
ISIE factors is therefore shown in Table 4.
ICT MGMT PERF
Actual Use
of IS
Past
Management
Experience
Effective
Organisational
Benchmarks &
Performance
Better
Management of
Resources
Radical
Transformation
Table 4: ISIE Factors Following Pairwise Analysis
This set of ISIE factors was then used as the basis for
constructing the fuzzy cognitive map. Causal
relationships were developed by the authors based
upon the PIE construct, to yield the fuzzy weight
matrix in Table 5. Subsequently the FCM in Figure 2
was constructed as a directed graph, where the
strength of causality between each node (henceforth
USEIT TRANSF in the weight matrix) was
determined by the thickness of the line connecting
each factor. A thicker line / thinner line denotes
stronger / weaker causal relationships, respectively,
and a value of 0 or no line indicates no relationship.
Figure 2: FCM of ISIE Factors
USEIT PAST MGMT RES PERF TRANSF
Actual Use
of IS
Past Management
Experience
Better
Management
of Resources
Effective Organisational
Benchmarks &
Performance
Radical
Transformation
USEIT Actual Use of IS 0 0.333 0.667 0 1
PAST
MGMT
Past Management
Experience
1 0 -0.333 -0.333 0
RES
Better Management of
Resources
0.333 0 0 0.333 0.667
PERF
Effective Organisational
Benchmarks & Performance
0.333 0 1 0 1
TRANSF Radical Transformation -0.667 0.333 0 0.667 0
Table 5: Fuzzy Weight Matrix for the ISIE Factors
5.1 Knowledge Mapping / Simulation of FCM
The mapping and simulation of the FCM follows
the technique defined by Kosko [23] and as denoted
by Sharif and Irani [39]. The authors as a result used
the TAPE framework [40] to identify two scenarios
in manufacturing ISIE (with users and managers
perspective of evaluating an IS). These were used as
seed factors in the FCM wherein each of the FCM
nodes were classified into explicit or tacit KCs.
5.2 Analysis
In both scenarios (Figures 3 & 7) the overall
results show that the dynamics are founded on high to
low causal responses ranging from TRANSF, PERF,
RES, PAST MGMT through to USEIT. First
scenario, there is a major shift in terms of the
negative causal response related to USEIT and RES;
with opposite causal responses from other factors.
Second scenario, there is a constant large negative
reduction in the USEIT variable (from 0.93 to 0.09
showing little or no causal effect); and a
corresponding fall in causal effect for RES and
3483
TRANSF. But, there are causal increases for PAST
MGMT and PERF. This is more prominent when the
starting and ending nodal values are plotted for each
scenario (Figures 5, 6, 9 and 10), respectively.
Figure 3: FCM Results for Scenario 1
Figure 4: Resulting FCM for Scenario 1
Figure 5: Starting Nodal Values for Scenario 1
Figure 6: End Nodal Values for Scenario 1
Figure 7: FCM Results for Scenario 2
Figure 8: Resulting FCM for Scenario 2
Figure 9: Starting Nodal Values for Scenario 2
Figure 10: End Nodal Values for Scenario 2
Plotting the separate responses on a polar and/or
Cartesian scale, magnifies the respective scenario
results further as shown in Figures 11 and 12. Here,
across both scenarios there appear to be two key
dynamic factors which dominate the results namely
the interactions between nodes PERF and RES which
show an out-of-phase relationship with one another
(highlighting that as the causality of organisational
benchmarks and performance increases, the causality
of better management of resources decreases (and
vice versa). Underlying all of these factors, the
dynamic of the past management experience node
(PAST MGMT) appears to have a stabilising effect
on all of the other nodes i.e., when this nodal
response stabilises, all the other nodes stabilise soon
afterwards as well. Hence, in mapping the knowledge
involved in ISIE, the PAST MGMT factor appears to
have a tacit controlling impact over other factors.
But, analysing the FCM results which at last encode
the knowledge within the ISIE process in this vein
reveals that this is only part of the overall picture.
USEIT
PAST
MGMT
RES
PERF
TRANSF
(++++)
(++++)
(++++)
(----)
(--)
(--)
(---)
(--) (----)
(----)
(++++)
(----) (----)
(----)
(----)
(++++)
(---)
(--)
(----)
(----)
(----)
3484
Figure 11: Scenario 1 Phase Plot
Figure 12: Scenario 2 Phase Plot
Figures 4 and 8 show the respective reconstructed
FCM diagrams as following each of the simulation
runs. The redrawn FCM in both cases were
constructed through matrix manipulation of the
computed results, scenario vectors and the original
fuzzy weight matrix. This yielded the computed
fuzzy eight matrices in Tables 6 and 7 for scenarios 1
and 2, respectively. The FCMs were once again
drawn and constructed based upon the pairwise
relationship between each node and the strength of
each of the causal links determined the thickness of
the lines connecting nodal points. Ultimately for both
scenarios and FCMs, these diagrams show that each
scenario involves an inherent range of inter-
relationships belying the initial FCM.
USEIT
PAST
MGMT
RES PERF TRANSF
USEIT 0.000 0.956 0.915 0.977 0.842
PAST
MGMT
-0.948 0.000 -0.445 -0.445 -0.671
RES -0.742 -0.552 0.000 -0.742 -0.859
PERF -0.897 -0.809 -0.972 0.000 -0.972
TRANSF -0.915 -0.988 -0.977 -0.994 0.000
Table 6: Final Fuzzy Weight Matrix for Scenario 1
USEIT
PAST
MGMT
RES PERF TRANSF
USEIT 0.000 -0.883 -0.938 -0.785 -0.968
PAST
MGMT
-0.546 0.000 0.617 0.617 0.369
RES -0.561 -0.293 0.000 -0.561 -0.748
PERF 0.729 0.851 0.253 0.000 0.253
TRANSF 0.938 0.620 0.785 0.372 0.000
Table 7: Final fuzzy weight matrix for Scenario 2
First, analysing scenario 1 through the redrawn
FCM (Figure 4) shows that strong causal
relationships continue to exist between PERF RES
and USEIT-TRANSF, a range of others have either
strengthened, weakened significantly or new
relationships have emerged (as shown in Figure 13).
The resulting FCM shows a unique set of internal
knowledge loops arising from the USEIT node
supplemented by inputs into the PERF component
also. The strength of these causal inter-relationships
appear to suggest that factors of USEIT, PERF and
RES are given more prominence by users of IT in
any IS evaluation, based upon how they may perceive
the utility and benefit of IS in the work that they do.
In this FCM it is also interesting to note that there are
considerably weak causal relationships in the outer
loop of relationships (between PERF-PAST MGMT
and PERF-RES especially) that reveal the challenges
that management faces in overcoming the benefit of
hindsight effect concerning past experiences of IT
benefits and risks. This is consistent with literature
where tactical/operational considerations within the
IS evaluation process shared alike relationship [16].
Decrease of
Causal
Knowledge
Relationships
Decrease of
Causal
Knowledge
Relationships
Increase of
Causal
Knowledge
Relationships
N/A
USEIT-TRANSF
PAST MGMT-TRANSF
RES-USEIT
RES-TRANSF
PERF-USEIT
PERF-RES
PERF-TRANSF
TRANSF-USEIT
TRANSF-PAST MGMT
TRANSF-PERF
USEIT-PAST MGMT
USEIT-RES
USEIT-TRANSF
PAST MGMT-USEIT
RES-USEIT
PERF-RES
PERF-TRANSF
TRANSF-PERF
TRANSF-PERF
USEIT-PAST MGMT
USEIT-RES
PAST MGMT-PERF
PAST MGMT-TRANSF
PERF-USEIT
TRANSF-USEIT
TRANSF-PAST MGMT
New Causal
Relationships
USEIT-PERF
RES-PASTMGMT
RES-PERF
PERF-PASTMGT
TRANSF-RES
USEIT-PERF
PAST MGMT-RES
PERF-PAST MGMT
TRANSF-RES
Total
Negative
Causal
Relationships
17 9
Total Positive
Causal
Relationships
5 10
N/A
15
3
12
FCM Scenario 1 Scenario 2 Aspect
Figure 13: Summary of Causal Relationships
Second, analysing scenario 2 through the redrawn
FCM in Figure 8 shows that there are a range of
strong causal inter-relationships principally
emanating from the PERF and TRANSF nodes.
These knowledge loops subsequently jointly feed into
3485
the USEIT and PAST MGMT nodes (with the latter
having a strong reinforcing loop back to PERF).
Here, it is evident that while there may be implicit
and weak causal links between the impact of
resources applied to IS and the impact that IS has on
organisational performance, it is interesting to note
that the majority of the negative causal loops in the
resulting FCM are related to those factors which
implicitly involve the interactions between users and
how resources may be used to derive benefit from IT.
As identified within Figure 13, even though the initial
FCM started off with a greater proportion of
positive/strong to negative/weak causal links (i.e. 12
vs. 3) between the scenarios explored, there are 88%
more weak causal links that emerged overall as a
result of each of each of the FCM simulations. There
are almost three times as many weak relationships as
strong relationships in scenario 1 (i.e. 17 vs. 5),
although there is a better balance between these in
scenario 2 (i.e. 10 vs. 9). This shows that ultimately a
mapping of the knowledge inherent in these IS
evaluation scenarios gives an indication of the
dynamics of just how difficult and complex it is to
overcome organisational culture and technology
adoption factors (encapsulated by the explicit
knowledge PERF component) as well as integrated
and effective efficiency strategies (encapsulated by
the tacit knowledge TRANSF component).
6. Conclusions
This paper sought to extend the view and
understanding of knowledge by seeking to apply a
cognitive technique to explore knowledge-based
decisions involved in evaluating IS investments. This
has also been with a view to researching into how
strategic IS is used in manufacturing where a paucity
of research into the breadth and impact of how
knowledge is used in such a context has previously
been identified [7]; and, how knowledge may be
represented in knowledge-intensive environments
[41]. In light of the examination of the literature, this
paper presented a MOOTS classification of factors
defining ISIE in the manufacturing organisations.
Therefore, the authors identified the key KCs related
to each ISIE factor, using the five-step PIE
framework. Subsequently, the MOOTS and the PIE
classification approach is pooled with expert
knowledge to construct a matrix of ISIE factors this
step led towards the development of the FCM of ISIE
factors. The latter process allowed the authors to
identify the inter-relationships and intricacies of
decision-making constructs in this research case.
The FCM approach used sought to identify those
knowledge constructs which are relevant to the ISIE.
This paper has been able to present this technique as a
means of exploring and therefore mapping the
knowledge inherent in IS evaluation from both user
and managerial perspectives. The resulting
knowledge mapping through the application of an
FCM has shown the intricate and developing
behaviour of causal relationships within the
knowledge area. The main relationships and
knowledge within ISIE have been shown to be driven
by a mixture of managerial and user perspectives.
These are eventually balanced by strong (and weak)
driving elements centring around: the actual
(intended) usage of IS and the instant operational /
tactical benefits this can provide from a user
perspective (FCM scenario 1); and a clear set of
relationships based upon how organisational culture,
technology adoption and the integration of IS for the
benefit of the organisation from a managers
perspective (FCM scenario 2).
7. References
[1] Caldeira, M. M. and Ward, J. M. Understanding the
successful adoption and use of IS/IT in SMEs: an explanation
from Portuguese manufacturing industries Information
Systems Journal, 2002, 12(2), pp. 121-152.
[2] Cebeci, U. Fuzzy AHP-based decision support system for
selecting ERP systems in textile industry by using balanced
scorecard, Expert Systems with Applications, 2009, 36, pp.
8900-8909.
[3] Choi, D., Kim, J. and Kim, S. ERP training with a web-
based electronic learning system: the flow theory perspective,
International Journal of Human Computer Studies, 2007,
65(3), pp. 223-243.
[4] Chou, T-Y., Chou, S-C. T. and Tzeng, G-H. Evaluating
IT/IS investments: A Fuzzy Multi-criteria decision model
approach, European Journal of Operational Research, 2006,
173(3), pp. 1026-1046.
[5] Coronado, A. E., Lyons, A. C. and Kehoe, D. F. Assessing
the value of information systems in supporting agility in high-
tech manufacturing enterprises, International Journal of
Operations and Production Management, 2004, 24(12), pp.
1219-1246.
[6] Fink, L. and Neumann, S. Exploring the perceived
business value of the flexibility enabled by information
technology infrastructure, Information & Management, 2009,
46, pp. 90-99.
[7] Gunasekaran, A. and Ngai, E. W. T. Knowledge
management in 21st century manufacturing, International
Journal of Production Research, 2007, 45(11), 2391-2418.
[8] Gunasekaran, A., Ngai, E. W. T. and McGaughey, R. E.
Information Technology and Systems Justification: A Review
for Research and Applications, European Journal of
Operational Research, 2006, 173(3), pp. 957-983.
[9] Gunasekaran, A., Love, P. E. D., Rahimi, F. and Miele, R.
A model for investment justification in information
technology projects, International Journal of Information
Management, 2001, 21, pp. 349-364.
[10] Hedman, J. and Borell, A. Narratives in ERP systems
evaluation, Journal of Enterprise Information Management,
2004, 17(4), pp. 283-290.
[11] Hendricks, K. B., Singhal, V. R. and Stratman, J. K. The
impact of enterprise systems on corporate performance: A
study of ERP, SCM, and CRM system implementations,
Journal of Operations Management, 2007, 25, pp. 65-82.
3486
[12] Irani, Z. Investment evaluation within project
management: an information systems perspective, Journal of
Operational Research Society, 2010, 61, pp. 917-928.
[13] Irani, Z. and Love, P. E. D. The propagation of
technology management taxonomies for evaluating
information systems, Journal of Management Information
Systems, 2001, 17(3), pp. 161-177.
[14] Irani, Z., Sharif, A. M., Love, P. E. D. and Kahraman, C.
Applying concepts of fuzzy cognitive mapping to model the
IT/IS Investment evaluation process, International Journal of
Production Economics, 2002, 75(1), pp. 199-211.
[15] Irani, Z. Information Systems Evaluation: Navigating
through the Problem Domain, Information & Management,
2002, 40(1), pp. 11-24.
[16] Irani, Z., Sharif, A. M. and Love, P. E. D. Transforming
failure into success through organisational learning: An
analysis of a Manufacturing Information System, European
Journal of Information Systems, 2001, 10(1), pp. 55-66.
[17] Irani, Z., Sharif, A. M. and Love, P. E. D. Linking
Knowledge Transformation to Information Systems
Evaluation, European Journal of Information Systems, 2005,
14(3), pp. 213-228.
[18] Irani, Z., Sharif, A. M. and Love, P. E. D. Knowledge
Mapping and Information Systems Evaluation in
Manufacturing, International Journal of Production
Research, 2007, 45(11), pp. 2435-2457.
[19] Ke, W. and Wei, K. K. Organizational culture and
leadership in ERP implementation, Decision Support Systems,
2008, 45(2), pp. 208-218.
[20] Kim, S., Hong, J. and Suh, E. A Diagnosis Framework
for Identifying the Current Knowledge Sharing Activity Status
in a Community of Practice, Expert Systems with
Applications, 2012, 39(18), pp. 13093-13107.
[21] Kim, Y. J. and Sanders, G. L. Strategic actions in
information technology investment based on real option
theory, Decision Support Systems, 2002, 33, pp. 1-11.
[22] Koduru, H., Xiao, F., Amirkhanian, S. and Juang, C.
Using Fuzzy Logic and Expert System Approaches in
Evaluating Flexible Pavement Distress: Case Study, Journal
of Transportation Engineering, 2010, 136(2), pp. 149-157.
[23] Kosko, B. Neural Networks and Fuzzy Systems, Saddle
River, NJ: Prentice-Hall, 1991.
[24] Kulkarni, U. R., Ravindran, S. and Freeze, R. A
Knowledge Management Success Model: Theoretical
Development and Empirical Validation, Journal of
Management Information Systems, 2006, 23(3), 309-347.
[25] Love, P. E. D. and Irani, Z. An exploratory study of
information technology evaluation and benefits management
practices of SMEs in the construction industry, Information &
Management, 2004, 42(1), pp. 227-242.
[26] Mondragon, A. E. C., Lyons, A. C. and Kehoe, D. F.
Assessing the value of information systems in supporting
agility in high-tech manufacturing enterprises, International
Journal of Operations & Production Management, 2004,
24(12), pp. 1219-1246.
[27] Moynihan, D. P. and Pandey, S. K. Finding Workable
Levers over Work Motivation Comparing Job Satisfaction, Job
Involvement, and Organizational Commitment,
Administration & Society, 2007, 39(7), pp. 803-832.
[28] Nevo, D. and Chan, Y. E. A Delphi study of Knowledge
Management Systems: Scope and Requirements, Information
& Management, 2007, 44, pp. 583-597.
[29] Ngai, E. W. T., Law, C. C. H. and Wat, F. K. T.
Examining the critical success factors in the adoption of
enterprise resource planning, Computers in Industry, 2008,
59(6), pp. 548-564.
[30] Nonaka, I. A Dynamic Theory of Organizational
Knowledge Creation, Organization Science, 1994, 5(1), pp.
14-37.
[31] Panetto, H. and Molina, A. Enterprise integration and
interoperability in manufacturing systems: trends and issues,
Computers in Industry, 2008, 59(7), pp. 641-646.
[32] Pemberton, J. and Stonehouse, G. Organizational
learning and knowledge assets an essential partnership, The
Learning Organization, 2008, 7(4), pp. 184-193.
[33] Perera, H. S. and Costa, W. K. Analytic Hierarchy
Process for Selection of ERP Software for Manufacturing
Companies, The Journal of Business Perspective, 2008,
12(4), pp. 1-11.
[34] Polanyi, M. The Tacit Dimension, Garden City, NY:
Doubleday, 1967.
[35] Powell, W. W. and Snellman, K. The Knowledge
Economy, Annual Review of Sociology, 2004, 30,199-220.
[36] Raymond. L., Croteau, A. -M. and Bergeron, F. The
Strategic Role of IT: An Empirical Study of its Impact on IT
Performance in Manufacturing SMEs, Proceedings on the
Sixth International Conference on Internet and Web
Applications and Services, 2011, pp. 89-97.
[37] Rhynne, R. Field Anomaly Relaxation The Arts of
Usage, Futures, 1995, 27(6), pp. 657-674.
[38] Sharif, A. M., Irani, Z. and Weerakkody, V. Evaluating
and modelling constructs for e-Government decision making,
Journal of the Operational Research Society, 2010, 61 (6), pp.
928-952.
[39] Sharif, A. M. and Irani, Z. Applying a Fuzzy-
Morphological approach to Complexity within management
decision-making, Management Decision, 2006, 44(7), pp.
930-961.
[40] Sharif, A. M. Knowledge representation within
Information Systems in Manufacturing Environments. PhD
Thesis, Brunel University, UK, 2004.
[41] Shaw, D. and Edwards, J. S. Manufacturing knowledge
management strategy, International Journal of Production
Research, 2006, 44(10), pp. 1907-1925.
[42] Shin, M. A Framework for evaluating economics of
knowledge management systems, Information &
Management, 2004, 42(1), pp. 179-196.
[43] Small, M. H. Planning, justifying and installing advanced
manufacturing technology: a managerial framework, Journal
of Manufacturing Technology Management, 2007, 18(5), pp.
513-537
[44] Stockdale, R. and Standing, C. An interpretive approach
to evaluating information systems: A content, context, process
framework, European Journal of Operational Research,
2006, 173(3), pp. 1090-1102.
[45] Velcu, O. Exploring the effects of ERP systems on
organizational performance: Evidence from Finnish
companies, Industrial Management & Data Systems, 2007,
107(9), pp. 1316-1334.
[46] Vundavilli, P. R., Parappagoudar, M. B., Kodali, S. P. and
Benguluri, S. Fuzzy logic-based expert system for prediction
of depth of cut in abrasive water jet machining process,
Knowledge-Based Systems, 2012, 27, pp.456-464.
[47] Wang, E., Klein, G. and Jiang, J. J. IT support in
manufacturing firms for a knowledge management dynamic
capability link to performance, International Journal of
Production Research, 2007, 45(11), pp. 2419-2434.
[48] Zhang, M. Sarker, S. and Sarker, S. Unpacking the effect
of IT capability on the performance of export-focused SMEs: a
report from China, Information Systems Journal, 2008, 18(4),
pp. 37-380.
3487