Modeling and Analysis for Educational Software Quality Hierarchy Triangle

Haiguang Fang
Department of Educational Technology, Capital Normal University, No.105 North Road of West Third Ring,
Beijing, 100037, China
fanghaiguang@hotmail.com


AbstractThe area of educational software evaluation has
been increasingly muddled because of a lack of consensus
among software evaluators. This article presents the
classification and evaluation granularity of educational
software, proposes an educational software quality
hierarchy triangle model based on some major software
quality models, analyzes the signification and relativity of
the factors. At last, it indicates the primary analysis for the
model formalization.
Keywords-educational software; evaluation; quality;
model; formalization
I. INTRODUCTION
Primitive computer assisted educational software is
researched as projects towards purposes and then
performed appraisement. The software is appraised
individually without any comparability among them.
Nowadays, computer assisted educational software is
regarded as electron teaching material and the
standards of testing and evaluation educational
software is established based on education, technology
and document. However, the evaluation process is
established based on exterior attribute rather than
software quality specifically. The evaluation on
educational software is draggled than educational
software development for without consensus
evaluation and testing technique.
II. EDUCATIONAL SOFTWARE CLASSIFICATIONS
AND QUALITY GRANULARITY LEVELS
A. Educational Software Classifications
The definition of educational software is
important to evaluate the software. There are many
educational software examples such as: SuperKids,
which is a website for parents and teachers who want
the best in education for their children; Redwood
Games, which offers a word game, math game, and
more; SuperSketchpad, which is a synthetic teaching
software and can implement geometrical drawing,
mathematical calculations, and making courseware;
Online Examination System, Hand-held/PDA
educational software; School OA; School MIS and etc.
Therefore, educational software is the software
designed to facilitate teaching and learning knowledge.
Educational software has three properties: software,
educational tool and medium of thought and
knowledge. Based on present products, educational
software can be classified as following:
1) Educational Resource Database
2) Courseware
3) Educational Administration software
4) Examination Database software
5) Platform software
6) Personal Learning software
These six categories educational software can be
abstracted as a same software product model to be
evaluated to the quality.
B. Educational Software and Knowledge
Management
Educational software and knowledge management
continuum is a model for presenting the relationship of
the educational software and knowledge management.
The relationship is illustrated in Figure 1.

Intelligent
Educational Software
Topic Database
Software
Assistant Teaching
Software
Knowledge
Data
Information
Universe
Educational Software
Knowledge Management
Continuum

Figure 1. Educational Software and Knowledge Management
Continuum
Data is defined as number or word without
relationships. Information is data that has been given
meaning by way of relational connection. Knowledge
is the appropriate collection of information, such that
its intent is to be useful.
2008 Seventh International Conference on Web-based Learning
978-0-7695-3518-0/08 $25.00 2008 IEEE
DOI 10.1109/ICWL.2008.19
14
There are two significant parts in the figure the
educational software and knowledge management
continuum. In the knowledge management continuum,
data can be thought of as the expanse of facts or
universe measures available to and about human being.
Data become information when they are placed in
context through interpretation. Knowledge is the
understanding that develops as people react to and use
the information that is available to them. On the other
hand, in the educational software, according to the
knowledge management continuum, the educational
software can be divided into three components: topic
database software, assistant teaching software and
intelligent educational software. There are also
important linkages to supporting sectors like
components intelligent educational software and
knowledge, related sectors like assistant teaching
software and information. And the management
content to intelligent educational software is
knowledge.
C. Educational Software Quality
The Quality is defined in ISO-8402 as "the totality
of features and characteristics of a product or service
that bear on its ability to satisfy stated or implied needs
", which explains that quality depends on a series of
stated or implied needs. The software product
evaluation is defined in ISO-8402 as "quality
characteristics and guidelines for their uses ".
Accordingly, quality evaluation depends on the quality
characteristics needs and the target software products.
Educational software quality can be divided into
different granularity levels which describe abstract
level views to evaluation such as single code or system
architecture. Educational software quality evaluation
can be located on different system views, and single
component is associated by implement relationship.
Educational software has four granularity levels(top-
down): Product, Architecture, Design and Coding.
Each granularity level indicates the quality level to be
evaluated. According to different granularity level,
there are different quality models which are expatiated
in Parnas and Myers paper.
By Weyuker's Anti-decomposition Axiom, testing
all components of software may be adequate, but not
necessarily adequate for the software. That is, testing
low granularity level quality may be adequate, but not
necessarily adequate for high granularity level quality.
Therefore, testing is an approach to evaluation and
educational software quality evaluation granularity is
the high level ---- product quality, which is adequate.

III. ESHTRI MODEL
In present software quality models, there is a kind
of model which is comprised by three main factors:
structure, adaptability and validity. According to the
educational software features, based on McCall model
1977 Boehm model 1978 and ISO9126
factor criteria model 1993 , the Educational
Software Quality Hierarchy Triangle model can be
implemented, which is named ESHTri model and
illustrated in Figure 2. There are three roles in the
model: producer, deliverer, software product. Producer
of the software (such as some software development
companies) and the deliverer (such as some schools)
specify the software product (such as Super Sketchpad
software product) by their contract, which prescribes
software quality factors such as reliability. Producer
defines the software quality factors dynamically such
as portability by projects and processes. Deliverer
contrives software quality factors such as flexibility by
users and services.

Sof t war e
Qual i t y
Pr oducer
Pr oduct
Del i ver er
Cont r act
Pr ocess
Pr oj ect
Ser vi ce
User
Maintainability
Flexibility
Usability
Portability
Reusability
Interoperability
Changeability
Economy
Reliability
Efficiency
Integrity
Testability
Self-descriptiveness

Figure 2. ESHTri model
A. ESHTri model signification
ESHTri model divides educational software
quality into three processes and thirteen factors.
Defining process is comprised of reliability, efficiency,
integrity, testability and self-descriptiveness.
Producing process is comprised of portability,
reusability, interoperability, changeability and
economy. Operating process is comprised of usability,
maintainability and flexibility.
B. ESHTri model consideration
The thirteen factors are significant factors to
influence the final quality of educational software.
Associated with each factor is some direct metrics,
15
which serve as a quantitative representation of a
quality factor. Metric is easy to measure, to
understand, and is comparable, such as the number of
lines of code(LOC) , is a measure derived from the
product correctness(maintainability).
TABLE I. ESHTRI MODEL FACTORS COMPARISON
Quality factors McCall 1977 Boehm 1978 ISO9126 1993 ESHTri
Model
Correctness Y Y Maintainability Maintainability
Reliability Y Y Y Y
Integrity Y Y Y
Usability Y Y Y Y
Execution efficiency Y Y Y Y
Maintainability Y Y Y Y
Testability Y Y
Interoperability Y Y
Flexibility Y Y Y
Reusability Y Y Y
Portability Y Y Y Y
Clarity Y
Changeability Y Maintainability Y
Self-descriptiveness Y Y
Ease of Modify Y
Understandability Y
Efficiency Y Maintainability Maintainability
Functionality Y
Generality Y
Economy Y Y


Table1 is the comparison between ESHTri
model and other quality models. The first column is
quality factors, and the second column is the McCall
model, in which X represents selected factor of the
model. The third column is the Boehm model, and the
fourth is ISO9126 model. The last column is ESHTri
model. The factors in ESHTri model are defined as
following which focus on a software system or
component:
1. Maintainability: be modified to correct faults,
improve performance, or other attributes, or adapt
to a changed environment.
2. Reliability: performs its required functions under
stated conditions for a specified period of time.
3. Integrity: prevents unauthorized access to, or
modification of, computer programs or data.
4. Usability: user can learn to operate, prepare inputs
for, and interpret outputs.
5. Execution efficiency: performs its designated
functions with minimum consumption of
resources.
6. Testability: facilitates the establishment of test
criteria and the performance of tests to determine
whether those criteria have been met.
7. Interoperability: exchanges information and to use
the information that has been exchanged.
8. Flexibility: be modified for use in applications or
environments other than those for which it was
specifically designed.
9. Reusability: a software module or other work
product can be used in more than one computing
program or software system.
10. Portability: be transferred from one hardware or
software environment to another.
11. Changeability: be needed for modification, fault
removal or for environmental change.
12. Self-descriptiveness: contains enough information
to explain its objectives and properties.
13. Economy: be efficient, sparing, or conservative
used.
More information and definitions about these
quality factors can be referred to ISO9126 standards.
16
C. ESHTri model factors relationship
If one factor has a high quality, then the other
factor will have a high quality, which is named as
positive relevancy, and marked it plus. If one factor has
a high quality, then the other factor will have a low
quality, which is named as negative relevancy, and
marked it minus. If there is no relationship between
two factors, which is named as no-relevancy, and mark
it zero. According to Perrys analysis in his paper, the
ESHTri model factors' relationship is illuminated in
Table2, which is a further complementarily to the
model.

TABLE II. ESHTRI MODEL FACTORS RELATIONSHIP

Correctness
Reliability
Integrity 0 0
Usability 0 0 0
Execution efficiency 0
Maintainability 0
Testability 0
Interoperability
Flexibility 0 0 0 0
Reusability 0
Portability 0 0 0 0 0 0 0
Self-descriptiveness 0 0 0 0 0
Changeability 0 0 0 0
Economy 0 0 0 0
IV. ESHTRI MODEL FORMALIZATION
In the ESHTri model, educational software
implement process can be regarded as a process of
software quality standards implement. the purpose of
formalization is to make software implement
adequately to the quality standards. Furthermore,
estimating the software implement adequateness by
judging whether all the quality standard factors are
implemented.
Let T is an implement data set of software S,
S
G
is quality standard factors definition set of software S,
G
N is a concrete operation quality standard factors set
of
S
G ,
T
L is quality standard factors set of operation
which implements
S
G of data set T. Therefore,
implement rat can be defined formally based on
software implement quality standard factors data.
Only if
T
L implements all the quality standard
factors definition of
S
G , software implement data set T
can be called software quality implement adequate. Let
NODE(
T
L ) is quality standard factors data which set
T
L implement quality factors of
S
G is
( )
100%
T
G
NODE L
N

It can be named as quality factor Defining Implement
Rate of
T
L .
According to the ESHTri model definition,
Producing Implement Rate and Operating Implement
Rate can be defined as above. Therefore, educational
software quality can be described formally from three
aspects: Defining Implement Rate, Producing
Implement Rate and Operating Implement Rate.
Although software quality standard implement
rates are the basic requirement, they cannot be
achieved all the time. They are not limited, because
particular software quality probably contains some
constrained factors which are unfeasible. Whether a
factor is feasible in particular software quality is an
undetermined problem. Therefore, redefine quality
factor required the feasible metric, the specific quality
is feasible. The formula above then expressed as
'
( )
100%
T
G
NODE L
N


'
G
N is feasible quality factors set. Finally, based
on ESHTri model, educational software quality
ES
Q
can be expressed as
1 2 3
' ' '
1 2 3
( ) ( ) ( )
, ,
T T T
ES
G G G
NODE L NODE L NODE L
Q
N N N


=



The three parameters in the formula above
represent quality implement rates in three processes:
software defining process, software producing process
and software operating process.
17
V. CONCLUSIONS AND FUTURE WORK
We believe that there are benefits of applying
ESHTri model to testing and evaluating educational
software. Although our model does not account for
several features of the process, the model is propitious
to capture the essential process of educational software
defining, producing and operating. Based on the
ESHTri model, we will continue to provide an
Educational Software Value Unique Model (ESVUM)
and a framework of testing and evaluation to
educational software, and then to develop Educational
Software Engineering, Educational Software
Engineering Economics and Educational Software
Usability Engineering.
ACKNOWLEDGMENTS
This work was supported by the National Grand
Fundamental Research 973 Program of China
(NKBRSF-2004CB318003).
REFERENCES
[1] Chow, T. S. Tutorial: Software Quality Assurance ---- A
Practical Approach, IEEE Computer Society Press,
1985.
[2] Dunn, R. H., Software Quality ---- Concepts and Plans,
Pretice Hall, 1990.
[3] Gillies, A. C., Software Quality ---- Theory and
Management, Chapman & Hall, 1992.
[4] ISO, Quality Vocabulary, ISO 8402, 1986(E).
[5] Boehm, B.W., J. R. Brown, J. R. Kaspar, M. Lipow,
G.MacCleod., Characteristics of Software Quality.
Amsterdam: North Holland.1978.
[6] F.Buschmann, R. Meunier, H. Rohnert, P. Sommerlad,
M. Stal, Pattern-Oriented Software Architecture. John
Wiley & Son Ltd.: A System of Patterns, Vol. 1, 1996.
[7] D. L. Parnas, On the Criteria to be used in Decomposing
Systems into Modules, Communications of the ACM,
1972.
[8] G. J. Myers, Software Reliability: Principles and
Practices, John Wiley Publications, 1976.
[9] Cosmann, R. (1996). The Evolution of Educational
Computer Software. Education, 116, pp. 619-623.
[10] Hardin, L., Patrick, T. B., Content Review of Medical
Educational Software Assessments. Medical Teacher,
20, pp. 207-212,1998.
[11] Maslowski, R., & Visshcher, R., Formative Evaluation
in Educational Computing Research and Development.
Journal of Research on Computing in Education, 32, pp.
239-256, 1999.
[12] WD&S., How to Evaluate Educational Software.
Curriculum Review, 39, 15,2000.
[13] Weyuker, E. J., Axiomatizing Software Test Data
Adequacy, IEEE Transactions on Software
Engineering, Vol. SE-12, No.12, pp. 1128-1138, 1986.

18