You are on page 1of 24

Benchmarking: An International Journal

Setting benchmarks and evaluating balanced scorecards with data envelopment analysis
Robert C. Rickards
Article information:
To cite this document:
Robert C. Rickards, (2003),"Setting benchmarks and evaluating balanced scorecards with data
envelopment analysis", Benchmarking: An International Journal, Vol. 10 Iss 3 pp. 226 - 245
Permanent link to this document:
http://dx.doi.org/10.1108/14635770310477762
Downloaded on: 27 June 2016, At: 16:03 (PT)
References: this document contains references to 30 other documents.
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

To copy this document: permissions@emeraldinsight.com


The fulltext of this document has been downloaded 2158 times since 2006*
Users who downloaded this article also downloaded:
(2008),"Balanced score for the balanced scorecard: a benchmarking tool", Benchmarking: An International
Journal, Vol. 15 Iss 4 pp. 420-443 http://dx.doi.org/10.1108/14635770810887230
(2008),"Firm operation performance analysis using data envelopment analysis and balanced scorecard:
A case study of a credit cooperative bank", International Journal of Productivity and Performance
Management, Vol. 57 Iss 7 pp. 523-539 http://dx.doi.org/10.1108/17410400810904010
(2008),"A data envelopment analysis-based balanced scorecard for measuring the comparative efficiency
of Korean luxury hotels", International Journal of Quality & Reliability Management, Vol. 25 Iss 4 pp.
349-365 http://dx.doi.org/10.1108/02656710810865249

Access to this document was granted through an Emerald subscription provided by emerald-srm:472283 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald for
Authors service information about how to choose which publication to write for and submission guidelines
are available for all. Please visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company
manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as well as
providing an extensive range of online products and additional customer resources and services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the Committee
on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive
preservation.

*Related content and download information correct at time of download.


The Emerald Research Register for this journal is available at The current issue and full text archive of this journal is available at
http://www.emeraldinsight.com/researchregister http://www.emeraldinsight.com/1463-5771.htm

BIJ
10,3 Setting benchmarks and
evaluating balanced
226
scorecards with data
envelopment analysis
Robert C. Rickards
The Leipzig Graduate School of Management, Handelshochschule
Leipzig, Leipzig, Germany and Hochschule Harz, Wernigerode, Germany
Keywords Benchmarking, Control, Data envelopment analysis, Performance measurement,
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

Strategic management
Abstract Balanced scorecards are having a major impact on executives strategic decision
making by encouraging their use of future-oriented, non-monetary success indicators. This article
explains how to create a balanced scorecard with a reasonable number of indicators, set
appropriate benchmarks for them, and evaluate overall management performance against those
benchmarks. In doing so, it relies heavily on a controlling tool that is new to the business sector:
data envelopment analysis. After discussing the theoretical foundations of this nonparametric
optimization procedure and defining several key terms, the article presents a practical example.
The example assesses managements performance from several perspectives, paying particular
attention to implications of the analytic results for a firms strategic and operational controlling.
The article also offers an overview of various models for data envelopment analysis, describes
limits to its use, and concludes with a summary of key findings.

Introduction
The increasing use of balanced scorecards (BSCs) is changing the way top
managers run their companies. When envisioning a firms future development,
they no longer focus chiefly on monetary success indicators in the financial
area. Instead, they now also devote roughly equal attention to non-monetary
variables in three other areas. As shown in Table I, those areas are: customer
relations; business processes; and both the commitment and the capacity to
innovate of their employees. This expanded focus is having major
consequences for controlling and management accounting staffs.
To help top management realize its visions, these staffs are assisting in
developing appropriate strategies, setting new goals, establishing standards or
benchmarks, measuring progress, and reporting results pertaining to both
monetary and non-monetary variables. Here is the rub. The BSCs goals are
interrelated. Efforts undertaken to attain, say, monetary goals therefore also
have effects on non-monetary results and vice versa.
Benchmarking: An International
Journal Hence, the BSC clearly confronts managers with an extraordinarily complex
Vol. 10 No. 3, 2003
pp. 226-245
optimization problem. As explained below, the complexity stems partly from
q MCB UP Limited
1463-5771
the large number of variables often included on a BSC. Frequently, there are 40
DOI 10.1108/14635770310477762 or 50 of them. Some larger organizations even try to track hundreds of success
factors. Additional complexity arises from the lack of a common scale of Data
measurement. While money is the common denominator for many financial envelopment
variables, business processes often involve time, and both customer relations analysis
and employee commitment/innovative capacity make heavy use of qualitative
measures. Moreover, dimensionless ratios and index numbers may appear in
any one or all four of the variable groups. Fortunately, data envelopment
analysis (DEA) can be a helpful tool in dealing with this complexity. 227
Although widely employed to evaluate efficiency in both non-profit and
governmental organizations, DEA is less well known within the business
sector. That probably will change as more decision makers recognize how
useful DEA can be in setting benchmarks and evaluating BSC results.
Therefore, after briefly discussing key aspects of the BSC, this article examines
problems associated with traditional efficiency measures. Next, it sketches the
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

theoretical foundations of DEA. In order to make the presentation accessible to


as many readers as possible, it deliberately omits a formal treatment of the
mathematics involved[1]. Having defined terms and clarified concepts, the
article next turns to a simplified application based on data from a real-world
firm. The results provide an objective basis for assessing managements overall
performance as well as guidance for the firms strategic and operational
controlling. Following overviews of alternative DEA models and limits to
DEAs use, the article closes with a summary of its key findings.

Hierarchical management and implementation of the BSC


From the outset, it is important to be mindful of the different roles various
levels in the management hierarchy have to play in developing and using BSCs.
Based on its vision, executive management sets overall, strategic goals for a
company in each of the BSCs four areas (Kaplan and Norton, 1996a). Together
with the controlling and management accounting staffs, top management
subsequently identifies variables crucial to the firms success in each area and
establishes standards or benchmarks for them. In this fashion, an enterprises
executives construct a common, numerical, strategic indicator system for use
throughout the company.

Customer
Finances relations Processes Employees/innovation
Shareholder/investor trust Customer Supplier Employee motivation
commitment commitment
Table I.
Cash flow/cash-flow return on Market Internal service Capacity for learning The four areas of a
investment/economic or resistance quality BSC with
shareholder value added representative
Investments securing the Public Time Manager evaluation strategic success
firms future image/trust indicators
BIJ Thereafter, decision makers in the firms various operating units adopt
10,3 measures to attain the strategic goals top management has set for them. Again
working with the controlling and management accounting staffs, these
midlevel managers adapt the overall strategy to the individual circumstances
of their respective units. They also flesh out the strategic indicator system with
additional, unit-specific, operational goals, variables, and standards or
228 benchmarks. Once implemented, periodic reviews then assess scorecard
results with regard to both top managements strategic vision and unit
operations (Olve et al., 2000).

The BSCs utility for controlling


A BSC enhances the quality of a firms controlling system in numerous ways.
First, through choice of appropriate variables, the BSC can incorporate many
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

familiar management principles in a single instrument. Among others, these


principles may include elements of: customer-oriented organization; employee
empowerment; just-in-time production and logistics; lean management; a
learning organization; reengineering; risk management; stakeholder
management; time for innovation; time management; total quality
management; and value-based activity management.
Second, with a BSC, the controlling systems focus expands beyond the
analysis of historical financial data. By encompassing a broader view of the
companys goals, the controlling system at once becomes more balanced and
more future-oriented. That is because customer relations and the quality of
business processes as well as employee commitment and capacity for
innovation will influence the enterprises success in coming years. This more
even-handed consideration of different organizational goals and increased
future-orientation are precisely what many independent auditors long have
recommended (Institut der Wirtschaftsprufer, 1999). In fact, some national
legislation (e.g. Germanys KonTraG) now requires firms to put into place
controlling systems embodying these recommendations (Naumann and Zeis,
2000, pp. 103-8). By giving early warning, such systems help managers to
detect potential business threats in time to initiate effective countermeasures.
Third, by providing for disaggregation via the organizations hierarchy, a
BSC ensures that top managements strategic goals pervade the entire
enterprise. At the same time, though, subordinate managers can tailor the BSC
to their respective units by adding operational goals and indicators as needed.
That establishes a flexible, formal interface between the firms strategic and
operational controlling. Moreover, with a BSC, managers easily can direct their
attention to those variables affecting the companys success at any given
organizational level.
Fourth, the empirical information reported on BSCs makes progress toward
goal attainment readily apparent. Assuming management and the controlling
staff both disaggregate and quantitatively measure the strategic goals,
numerical indicators, and standards, this information also facilitates Data
performance comparisons among an enterprises various units and across envelopment
firms. analysis
Creation and use of a BSC
Summarizing the discussion thus far, one can say that a BSC is an instrument
with the potential to link a companys strategy closely to its business 229
operations. Its quantitative measures can give management a quick, yet
comprehensive picture of a firms goals as well as its financial and operational
performance. Thus, both the challenges confronting a company and its
progress toward meeting them become transparent. This transparency, in turn,
assists managers in steering an enterprise along the course planned for its
development.
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

But creating and using a BSC involves challenges too. First, it is necessary to
decide how many units within a given organization really require one.
Observation suggests firms typically allow every strategic business unit to
have its own BSC. The resulting instruments have proved especially useful for
new units produced by mergers or acquisitions and for carve outs spun off to
become independent firms (Scharioth and Huber, 2002, p. 52).
The second challenge has to do with the number of variables reported in
each area of a BSC. Jack Welch, arguably the twentieth centurys most
successful executive, believes top management has to deal with too many
numbers. In his view, just three numerical indicators are enough to direct any
firm on earth: cash flow; customer commitment; and employee motivation
(Welch and Byrne, 2001). Together, these three variables comprise the worlds
shortest BSC! All the same, they cover three of the four areas or perspectives
mentioned by BSC originators Kaplan and Norton (1992, 1993, 1996b).
This anecdote conveys the important message that, for controlling purposes,
less often is more. Hence, as illustrated in Table I, the optimal number of
strategic indicators for many companies probably lies somewhere between
eight and 12. Aside from the simple variables cash flow, investments, and time,
however, all the other indicators in Table I are indices involving at least some
qualitative measurements. The third challenge therefore is how to construct
such indices.

Indices as numerical indicators on the BSC


For illustrative purposes, take the customer commitment index. Its area of the
BSC might include the following goals:
.
strengthening relationships with existing customers to develop the
longest-lasting, most comprehensive ties possible;
.
acquiring new customers, not least by means of existing customers
recommendations; and
.
increasing the profitability of every individual customer relationship.
BIJ The controlling system requires an indicator embodying these three goals, which
10,3 delivers management relevant decision-making information about changes in
customer commitment. A single index number, computed on the basis of
numerically coded answers to standardized questions from the areas listed in
Table II, can do just that. Reporting a single number is essential if customer
commitment is to have roughly the same importance on the BSC as, say, cash
230 flow. For the same reason, only a single number should represent internal service
quality, employee commitment, and each of the other index variables.
Possible values for these indices range from 0 to 100. A value of 100 on the
customer commitment index indicates that the firm and its customers are
inseparably close to one another. It is virtually impossible for a competitor to
break up such tight business relationships. In contrast, an index value of 40
denotes weak ties. There is little bonding between customers and the firm.
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

Their interaction scarcely goes beyond the most basic business activities.
Essentially, the customers are open to offers from every competitor.
Thousands of firms have commissioned surveys yielding empirical results
for this kind of standardized index. For instance, the German benchmark
values for customer commitment, internal service quality, and employee
motivation currently are 68, 56, and 59, respectively (Scharioth and Huber,
2002, p. 55). Apparently, there is ample room for German firms to improve on
all three of these BSC indices. Nonetheless, their performance is relatively
better with regard to customer commitment than it is in relation to internal
service quality and employee motivation.
Having met the challenges of deciding which units require a BSC, limited the
choice of strategic variables for inclusion, and constructed meaningful indices,
a fourth challenge still remains. How can executives evaluate overall unit
performance on the basis of scores for various strategic and operational
indicators?

The problem of efficiency measurement


Since the 1990s, globalization has increased enormously the competitive
pressures on firms worldwide. In response, many of them have tried to boost
simultaneously the efficiency and the quality of their production, while paying
more attention to the customers perspective[2]. The bases for their efforts are
benchmarks and standard costs embodying best practice. Consequently,

Viewpoint Question areas Results

Table II. Rational Overall satisfaction Evaluation of the product


Components of a Emotional Recommendation Customer satisfaction
customer Intentional Repeat purchase intention Strength of the tie
commitment index Objective barriers Advantage in comparison with the competition Customer bond intensity
benchmarks and standard costs serve both as planning objectives and currently Data
attainable goals for performance evaluation (Horngren et al., 1997, p. 235). envelopment
Yet just how does one recognize the best? Here three problems arise. First, in analysis
the age of BSCs, one has to consider numerous efficiency and success indicators
in evaluating performance. Because scorecards often include large numbers of
indicators, it is highly unlikely that a single plant location or a particular firm
would be the leader in every respect. Second, for many plant locations or firms, 231
the necessary data to facilitate external comparisons usually are difficult to
obtain. Third, even if one succeeds in gathering the requisite data, there still
remains the complex problem of evaluating them objectively and meaningfully.
In contrast to pyramid-shaped systems of indicator numbers for controlling
purposes, the BSC lacks a capstone like return on equity, cash-flow return on
investment, economic value added, or shareholder value added. Furthermore,
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

BSC advocates specify neither mathematical-logical relationships among the


individual scorecard criteria nor a unitary, objective weighting scheme for
them. Hence, it is difficult to make comparisons within and across firms on the
basis of BSCs. In addition, the inefficient use of resources may go unrecognized
(Werner and Brokemper, 1996, p. 168).
In order nevertheless to arrive at some judgments about the efficiency of
resource usage, one can turn to parametric methods, for example regression
analysis. As portrayed in Figure 1 by the dotted line RR, regression analysis
generates an optimal production function for a given set of observations. Firm
or plant efficiency appears as the vertical distance between their respective
coordinates and that function.
Evaluating efficiency in this fashion, though, has three important
disadvantages. First, implicit in a regression analysis is the assumption that

Figure 1.
DEA and regression
analysis
BIJ all observed plants or firms combine their input factors in the same way. But, in
10,3 practice, the production technology involved typically varies (see Atkinson and
Stiglitz, 1969; Freeman, 1994; Imai and Yamazaki, 1992; Vromen, 1995). Second,
regression analysis only can determine average values, which probably do not
actually occur in any of the units examined. The results therefore hardly can
serve as benchmarks and standard costs because they neither represent best
232 practice nor do they exist in the real world (the case illustrated in Figure 1)
(Charnes et al., 1994, p. 5). Third, although regression equations indeed can
include several inputs, one only can use them to analyze a single output at a
time. Accordingly, one must repeat the regression analysis as often as the
number of the criteria included on the BSC.
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

Theoretical foundations of DEA


Because it has none of the disadvantages associated with regression
procedures, DEA is a far superior way to assess the efficiency of goal
attainment. It thus answers the question posed earlier of how executives can
evaluate overall unit performance on the basis of several strategic and
operational indicators. Moreover, unlike statistical regression, DEA is a
nonparametric linear programming procedure. The American economists
A. Charnes, W.W. Cooper and E. Rhodes developed it in 1978 (Charnes et al.,
1978). They did so to compute a measure of currently attainable efficiency (i.e. a
benchmark) without first calculating average relationships or determining
factor weightings. Subsequently, DEA also has proved useful for making
qualitative judgments about the relative efficiency of economic organizations,
whose inputs and outputs are susceptible solely to multidimensional
measurement. Furthermore, DEA permits one to analyze multiple input and
output factors simultaneously (Burkle, 1997, Foreword).
All these properties are helpful because real-world management situations
usually involve multiple, multidimensional inputs and outputs (Werner and
Brokemper, 1996, p. 164). The products of a DEA emerge as a series of
traditional productivity indicator numbers. Weighting and aggregating them
appropriately yields an index for factor productivity overall.
In a DEA, one need not assume a priori the existence of a particular
production function for weighting and aggregating inputs or outputs. Instead,
the respective aggregation weights result from solving an optimization problem.
Hence, they are solely dependent on the empirical observations involved. This
fact gives the DEA method a decisive advantage over ordinary optimization
procedures. Even if one does not value the inputs and outputs at market prices,
or indeed in money, one still can execute the necessary aggregations. The
aggregation weights or prices are implicit in the results of the DEA.
DEA places a frontier function around the input and output factors under
investigation. This function appears as the curve FF in Figure 1. It consists of
linear segments, which connect the most efficient units in the study sample or
population with one another. The envelopment provides information not only Data
about the currently most efficient units, but at the same time also permits envelopment
analysis of the inefficient units. analysis
One can examine the latter from two distinct perspectives. First, one can
show by how many percent the inputs of the inefficient units would have to fall
in order for them to lie on the frontier function too. That would identify the
maximum input level permissible in order to attain a given output level. It thus 233
would correspond to the minimal principle of efficiency: achieving a given
output level with the lowest amount of inputs.
Second, one can show just as well by what percentage the outputs would
have to increase in order for an initially inefficient unit to reach the frontier
function. That would correspond to the maximum principle of efficiency: the
highest output level attained for a given amount of inputs.
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

We shall return shortly to a specific investigation of efficiency with the help


of a frontier function. Before doing so, though, it will be helpful to introduce
and define several terms used in connection with DEA.

Terms and definitions: decision-making units


The objects of a DEA are decision-making units (DMUs). In practice, they are
branch stores, business offices, company divisions, diverse sites
manufacturing a particular product, product groups, subsidiary corporations,
work teams, and so forth.
Underlying every DEA is the assumption that a DMUs (in-)efficiency results
from decisions taken within the unit itself. One thus calculates an efficiency
coefficient for each DMU under investigation. This coefficient represents the
sum of the DMUs weighted outputs relative to its weighted inputs. The
efficiency coefficient can take on any value between zero and one. An
optimization model weights the variables so as to yield the highest possible
efficiency coefficient for each DMU (Burkle, 1997, p. 7). The most efficient
DMUs observed are optimal units according to the Pareto-Koopmans criterion.
Accordingly, they serve as standards of comparison or benchmarks for the
other DMUs (Chen, 1997, p. 5).

Terms and definitions: reference DMU and efficiencies


One can investigate various kinds of efficiency on the basis of a DEA.
Although this article discusses allocative efficiency, it focuses mainly on
technical efficiency. It also deals with the utilization efficiency of available
customer potential and scale efficiency.
Because both terms make use of inputs and outputs, technical efficiency is
synonymous with productivity. It shows whether, and if so, to what extent
there is wastage of input factors (i.e. whether a DMU produces too little output
for a given amount of input(s)). A DMU is technically efficient, if no other unit
can produce the attained level of output with a smaller amount of input(s).
Moreover, by taking factor prices into consideration, one at the same time
BIJ obtains the minimum cost tangent (JJ in Figure 2), which represents the least-
10,3 cost factor combination. Any difference between the cost minimizing solution
and the technically efficient one represents allocative (in-)efficiency.
Figure 2 illustrates technical and allocative efficiency from an input
perspective (Welzel, 1996, p. 3). It shows five DMUs, each employing two inputs
234 to produce one output. The output isoquant, KK, represents the boundary of the
theoretical production possibilities. Consequently, every combination of the two
input factors lying on the curve, yields a maximum level of output. In contrast,
MP designates the frontier function of the observed, technically efficient DMUs
2-5. They operate at points M, N, O, and P, respectively. All four of these DMUs
therefore require the smallest input amounts to generate a unit of output.
However, they obviously differ with regard to their input consumption. That
suggests they use different production technologies. The dotted lines mark each
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

of the four technically efficient DMUs respective area of influence. They


indicate commonalities among all the DMUs lying between a pair of these lines.
Now assume DMU 1 operates at point L. Its input decisions are technically
inefficient because point L lies above isoquant KK. In other words, DMU 1
consumes more of both inputs, in production of the output, than any of the
other four DMUs.
Closest to point L on frontier function MP is point N, where DMU 3 produces.
Although both DMUs combine inputs in a similar way, DMU 3 attains a higher
output level, while consuming less of the inputs. Hence, it is the reference
DMU for DMU 1 because, as shown in Figure 2, DMU 3 performs better.
(Indeed, DMU 3 even is technically efficient!) As a result, the distance from

Figure 2.
Reference DMUs,
technical and allocative
efficiency
point L to point N represents the technical inefficiency in DMU 1s combination Data
of the productive factors relative to DMU 3. envelopment
Nevertheless, production at point N is allocatively inefficient because it does
not lie on the minimum cost tangent JJ. To optimize the input amounts with
analysis
respect to their factor prices would require movement along isoquant KK to the
cost efficient point O, where DMU 4 operates. Because point Q, which is the
intersection between the minimal cost tangent and the production line, 235
embodies the least-cost input combination, the distance from point Q to point N
indicates the allocative inefficiency associated with DMU 3.
Thus, depending on ones objective, a DEA can measure either or both these
kinds of efficiency. In any event, though, the values determined always pertain
to a decision units efficiency with regard to the performance of other DMUs.
So, when interpreting analytic results, it is important to remember that one is
dealing with relative, as opposed to absolute, measures of efficiency (Burkle,
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

1997, pp. 4-5).

Terms and definitions: slacks


Comparing the technical efficiency values of the various DMUs thus reveals
both overconsumption of inputs and underproduction of outputs. One terms
these inefficiencies slacks. Accordingly, slacks represent improvement
potential (Burkle, 1997, pp. 10-22).
For example, Figure 3 shows six DMUs, each using two inputs to produce
two outputs. Among them, DMUs 5 and 6 operate at points Y and Z,
respectively. With the former DMU located vertically above the latter one on a
segment of the envelopment, this case illustrates the simplest form of an

Figure 3.
Slack
BIJ analysis from the output perspective. The difference Y 2 Z embodies the
10,3 amount of slack present. In other words, by operating more efficiently, DMU 6
at point Z could increase output 1 until it reached the efficiency of DMU 5 at
point Y, without any reduction in its production of output 2. Consequently, one
also says that DMU 5 dominates DMU 6 with regard to efficiency.
Therefore, as depicted in Figures 2 and 3, slack refers either to output
236 foregone or to input wastage. In practice, where slacks are present over time,
one should check whether it wouldnt be worthwhile to make use of the
available improvement potential. For a DEA, that means the interpretation of
observed inefficiencies always should take slacks relative to dominant points
into consideration.

A practical example using DEA to evaluate a BSC: methodology


Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

The following illustrative application of DEA to evaluate a BSC employs a


large data set made available by a real firm. The firm has a long-term contract
with a leading European market research company to generate data on many
nonfinancial variables of interest to management. In keeping with the contract,
he market research company periodically conducts telephone and/or face-to-
face interviews with samples of the firms various stakeholders. Interview
instruments rely heavily on standardized, closed-ended questions the market
research company also uses for other clients. Reportedly, response rates for
external stakeholders are close to 60 percent, while the ones for the firms
employees run above 80 percent. The market research company communicates
survey results to the firms management in the form of statistics describing
both answers to individual questions and indices aggregating responses across
multiple questions.
Data for the remainder of the nonfinancial as well as all financial variables
come from the firms internal management information system. A well-known
business software manufacturer produced, installed, and supposedly tailored
that system to the firms needs. The information generated by the system is
subject to regular internal and external audits.
Just as Jack Welch points out, though, the firms management found itself
dealing with too many numbers. Furthermore, it had no objective basis on
which to evaluate unit performance overall. Consequently, management sought
advice on how to overcome these problems.
The following example showed the firms top executives how DEA could
solve them. Given its heuristic purpose, the example uses a BSC with just four
input and four output variables, all chosen by management from the firms own
data base, to evaluate the performance of two DMUs in a basic ratio model.
Readily available, off-the-shelf DEA software performed all the requisite
calculations.
However, before turning to the example, one last explanatory comment is in
order. To maintain the firms anonymity, as promised at the studys outset, this
example fictionalizes the enterprises field of business and employs some
proportionally altered data. It thus summarizes important insights gained from Data
the analysis without revealing facts management wants to remain confidential envelopment
(Burkle, 1997, pp. 4-5).
analysis
The initial situation
The company studied has a flat organizational hierarchy with 69 DMUs
operating throughout Europe. Each DMU produces construction materials, 237
which it sells exclusively to local customers. Table III lists values for the eight
variables on the BSCs of two of the DMUs studied. Here, the output variables
are cash flow, customer commitment, internal service quality, and employee
motivation, while machine capacity, number of employees, salesroom floor
space, and advertising expenditure are the input variables. Table III thus
includes both monetary and non-monetary data for both strategic and
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

operational variables. As reported in Table III, DMU 11 consumes absolutely


higher levels of all four inputs and attains absolutely higher levels of the four
outputs than does DMU 27.

Technical efficiency
For the purposes of benchmarking and performance evaluation, one now may
wish to have answers to the following kinds of questions. Is it really fair to
compare the two DMUs with one another? How large is each DMUs technical
efficiency? Where a DMU wastes inputs or outputs, which ones are they? By
how much could that DMU decrease its input consumption or increase its
output production? For given levels of machine capacity, personnel, salesroom
floor space, and advertising expenditure, how high should the levels of cash
flow, customer commitment, internal service quality, and employee motivation
be? Or, the other way around, how high must input levels be so that one can
expect a DMU to attain certain output levels? To demonstrate how to answer
such questions by analyzing BSCs with DEA, the example first examines the
technical input and output efficiencies, then the usage efficiency of each DMUs
customer potential.
Accordingly, the analysis begins by comparing the structures of the DMUs
studied. The results of this comparison appear in Table IV. They reveal that

Variable DMU 11 DMU 27

Machine capacity (units of output) 49,200 3,150


Personnel (full-time equivalents) 93.4 11.7
Salesroom floor space (m2) 1,499 126
Advertising outlay (e000) 630 41
Cash flow (e million) 10.48 0.80
Customer commitment (index) 61 58 Table III.
Internal service quality (index) 50 46 Selected input and
Employee motivation (index) 53 51 output variables
BIJ the reference DMUs for DMUs 11 and 27 are different. That means their input
10,3 and output structures diverge so strongly from one another, that one cannot
use the same measuring stick to compare them[3]. On the contrary, reference
DMU 36 embodies the currently attainable standards for unit 11 because both
have similar structures. Likewise, for unit 27, reference DMU 43 is the
appropriate standard of comparison.
238
Table IV also reports indicator numbers for the technical input efficiency of
units 11 and 27. The value of 0.71 suggests that, compared with a value of 1.0
for the efficient DMU 36, unit 11 has considerable room for improvement. With
its similar structures, the reference DMU could achieve the same output levels
with substantially fewer inputs (2 29 percent). The potential for savings in
advertising expenditure (2 39 percent), machine capacity (236 percent), and
salesroom floor space (2 23 percent) is rather large. Even in the case of
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

personnel, the input waste is not small (211 percent).


In contrast, given its different structures, unit 27 operates much more
efficiently. Compared with the value of 1.0 for its reference DMU 43, it attains
an indicator number of 0.89 for technical efficiency. In order to function as
efficiently as this reference DMU does, unit 27 would have to reduce its
consumption of inputs by 0.11 overall. Here, the largest savings potential is in
the personnel area (2 14 percent). With respect to machine capacity and
advertising expenditure, the savings potential is considerably smaller (27
percent each). As for salesroom space, DMU 27s savings potential is small (just
24 percent).
Table V presents the DEA results of technical efficiency from the output-
oriented perspective. The value of 1.21 in the second column shows that unit 11

Variable DMU 11 DMU 27

Reference DMU 36 43
Input efficiency 0.71 0.89
Machine capacity waste (%) 2 36 27
Table IV. Personnel waste (%) 2 11 214
Technical input Salesroom floor space waste (%) 2 23 24
efficiency Advertising outlay waste (%) 2 39 27

Variable DMU 11 DMU 27

Reference DMU 36 43
Output efficiency 1.21 1.07
Cash flow (%) +60 16
Table V. Customer commitment (%) +7 5
Technical output Internal service quality (%) +8 3
efficiency Employee motivation (%) +9 4
must increase its output by 21 percent in order to reach the level of reference Data
DMU 36. Spread across the BSCs four areas, that would require an increase in envelopment
cash flow of +60 percent, in customer commitment of +7 percent, in internal analysis
service quality of +8, and in employee motivation of +9 percent.
Because unit 27 works much more efficiently, it only would have to increase
its output by 7 percent overall in order to attain the levels of reference DMU 43.
In this case, the increase in cash flow would have to be +16 percent, in customer 239
commitment +5 percent, in internal service quality +3 percent, and in employee
motivation +4 percent.

Usage efficiency
With additional information taken from the firms database, one also can
analyze how efficiently each DMU uses its customer potential[4]. Table VI
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

summarizes relevant data for DMUs 11 and 27. Table VI shows that small
jobbers comprise the majority of each DMUs potential customers. It also is
apparent that DMU 11 has absolutely more customers than DMU 27. However,
large builders and retail outlets are relatively more important for the former,
while middle-sized general contractors and small jobbers are relatively more
important for the latter. That is entirely plausible because, for example,
economic structures in highly urbanized areas tend to differ from the ones
present in more rural regions.
In analyzing these data, the first step again was to form reference groups,
taking into consideration the DMUs different customer bases and business
structures. Table VII presents analytic results from the output-oriented
perspective. For DMUs 11 and 27, the reference DMUs now are units 63 and 14,
respectively. Apparently, the customer/output structures of DMUs 11 and 27

Customer type DMU 11 DMU 27

Small jobbers 496 172


Middle-sized general contractors 103 31 Table VI.
Large builders 54 9 The DMUs
Retail outlets 92 21 customers

Variable DMU 11 DMU 27

Reference DMU 63 14
Efficiency 0.75 0.83
Increase in small jobbers (%) +17 +23
Increase in middle-sized general contractors (%) +28 +7 Table VII.
Increase in large builders (%) +36 +11 Customer potential
Increase in retail outlets (%) +21 +9 usage efficiency
BIJ differ not only from one another, but also from their respective technical
10,3 input/output structures.
Compared with the respective reference DMU, neither unit 11 nor unit 27
exhausts its customer potential (efficiency 0:75 and 0.83, respectively).
Although the gap between the two branch locations in this regard is not as
large as it is in the case of their technical efficiency, unit 27 also performs
240 relatively better in exhausting the available customer potential. Nevertheless, it
is important for both DMUs to identify and eliminate obstacles in the
operations area, so that they can increase their usage efficiency. As described
below, the identification and elimination of these obstacles have further
consequences for the DMUs strategic planning and development.
If we had had different results here, e.g. if unit 11 had a high level of
technical inefficiency, but were perfectly efficient in the usage of its customer
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

potential, it would be easy to recommend appropriate therapeutic measures


(see, for example, Westermann et al., 1996, pp. 79-80). One only would need to
follow an either-or strategy. The DMU then either should save inputs in order
to become technically more efficient, or, alternatively, acquire additional
customer potential in order to utilize the currently employed inputs more fully.
In practice, though, results like those in Table VII are much more common,
especially when a DEA includes numerous DMUs. Consequently, one may have
to try a two-track approach in conceiving planning goals. That, in turn,
demands the controller or management accountant bring his/her experience
and craftsmanship into play. Because both DMUs 11 and 27 are technically
inefficient in their operations and fail to utilize their customer potential fully,
mixed therapies seem to be appropriate. The two DMUs should reduce their
consumption of inputs, while at the same time striving for better utilization of
their available customer potential. Nevertheless, each DMU requires a different
dosage of these recommended remedies.

DEA models
Four types of models are available for DEA applications[5]. Table VIII presents
an overview of them, together with several of their most important
characteristics. In 1978, Charnes, Cooper, and Rhodes (CCR) developed a
basic DEA model for determining either input or output efficiency. As shown in
the example above, with it one calculates an efficiency indicator number, h0, for

Model Year developed Orientation of the weighting Returns to scale

CCR ratio 1978 Input or output Constant


Multiplicative 1982/1983 Input and output Constant or variable
BCC 1984 Input or output Variable
Table VIII. Additive 1985/1987 Input and output Constant or variable
Overview of DEA
models Source: Charnes and Cooper (1994)
a particular DMU (Charnes et al., 1978). The size of this number depends on the Data
relationship between the weighted inputs and the weighted outputs. If the envelopment
analysis is input-oriented, the model sets the input equal to one and maximizes
the sum of the weighted outputs. An output-oriented analysis simply takes the
analysis
reverse tack: the model sets the output equal to one and minimizes the sum of
the weighted inputs. The CCR-ratio model thus generates an objective measure
of a DMUs current efficiency or inefficiency. However, implicit in the form of 241
the CCR-ratio model is the assumption, that an increase of the DMUs inputs
leads to a proportional increase in its outputs.
Multiplicative models drop this assumption about the envelopments linearity.
One then can assume that a logarithmic-linear or a Cobb-Douglas function
describes the production relationship (Burkle, 1997, p. 14). Such models find little
use in practice, though, because they give up one of the great advantages of the
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

nonparametric method. By specifying the form of the production function, they


stipulate a priori a particular relationship among the variables.
The third model, developed by Banker, Charnes and Cooper (BCC),
differentiates between technical and scalar efficiency. It estimates the technical
efficiency for a given operational size and evaluates whether increasing,
constant, or decreasing returns to scale would boost the efficiency observed. In
the case of constant returns to scale, the output changes proportionally to input,
as it also does in the CCR-ratio model. But with variable returns to scale, a
change in the input leads to a disproportional change in the output. In these
cases, the BCC model designates only those DMUs as DEA-efficient, which are
both technically efficient and also produce on the optimal operational scale. In
contrast, for those DMUs whose scale of operations is not optimal, the BCC
model reports just their technical (in-)efficiency.
Additive models (as well as the closely related, so-called expanded additive
models) juxtapose a DEA with the CCR-ratio models analyses of inefficiency.
In this fashion, they combine the results of such analyses with the economic
concept of Pareto optimality as interpreted by Koopmans. They do so by
maximizing the sum of all input and output slacks without making use of the
non-Archimedean quantity. That is possible because, from the outset, they
establish a lower boundary of 1.0 for the input and output weights[6]. The
envelopment resulting from an additive model thus is equivalent to that of the
classic CCR-ratio model, but offers other points of comparison. However, with
an additive model, the assumption of proportional changes among the inputs
and outputs is not valid.
In order to make an appropriate choice among the model types, it is essential
to decide a priori, whether the observed DMUs are subject to scalar effects. In
the example above, one thus would have to answer the following concrete
question: does a growing DMU, which continuously produces more goods and
services, also require proportionally more units of input?
The simplest way to answer this question would be to examine the
correlation between inputs and outputs graphically. Figure 4 shows such a
BIJ
10,3

242
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

Figure 4.
A nonlinear input/output
correlation

relationship between a hypothetical output and an hypothetical input. As


depicted, the relationship is nonlinear, displaying decreasing returns to scale.
Given such evidence, one would assume variable returns to scale and choose
the BCC model for a DEA.

Possibilities for and limits to the use of DEA


Although there are many possibilities for employing DEA in the areas of
controlling and management accounting, this method nevertheless has its
limits. The following four are the most important ones. First, all the inputs and
outputs for study must be present in and measurable for each DMU (i.e. there
can be no missing data).
Second, the larger the number of inputs and outputs, the more differentiated
individual DMUs are from one another. Greater differentiation, in turn,
increases the number of efficient DMUs. In situations where relatively few
DMUs are under examination, a DEA then very well may report nearly all (or
indeed all) DMUs as being efficient. Therefore, the relationship between the
number of input and output variables to the number of DMUs studied should
not exceed a certain upper limit. In practice, this limit generally is a ratio of 1:2.
Third, the potential input savings or output increases identified in a DEA
are not always attainable. Particularly in an operational setting, one simply
may not be able to eliminate a DMUs inefficiencies when they involve
absolutely small amounts of an indivisible input or output unit. For example, a
DEA might show that a DMU has a savings potential of, say, 9 percent in its
consumption of rented salesroom floor space. The rental contract, though, may
not be subject to immediate renegotiation or the recommended smaller floor
space currently may not be available on the market. The same often is true for Data
skilled labor, especially if many employees already work part-time. They may envelopment
be unwilling to reduce their hours further. In such instances, although a
potential for savings indeed exists, it may prove impossible to realize them.
analysis
Fourth, the source of the revealed savings or increased production potential
is not always evident from the analysis. Table II, for instance, shows the
personnel of DMU 11 is more efficient than that of DMU 27. The reason, 243
though, is not apparent. On the one hand, this greater efficiency could stem
from the former unit having more large builders as customers. On the other
hand, it also could be that the personnel in the latter unit is less well-trained,
less experienced, or less motivated. Hence, one always has to interpret the
results of a DEA carefully. They indicate to a controller or management
accountant where there are difficulties. Yet they seldom eliminate altogether
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

the considerable effort necessary to discover what has led to the inefficiencies
and how to reduce or eliminate them.
Still, in comparison with other methods for estimating efficiency, DEAs
possibilities far outweigh these limitations. DEA is a robust, standardized, and
transparent methodology. It allows a controller or management accountant to
analyze simultaneously a relatively large number of inputs and outputs
measured on different scales. This feature permits inclusion of qualitative data,
such as observations on the satisfaction of various stakeholder groups,
essential for setting benchmarks to evaluate BSCs. The objectivity stemming
from weighting variables during the optimization procedure furthermore frees
the analysis from subjective estimates and randomness. That, in turn, increases
the acceptability of its results by affected parties. In addition, the results both
identify appropriate reference DMUs and include easily interpretable efficiency
parameters. They are especially helpful in setting realistic, currently attainable
standards or benchmarks. Finally, a DEA quantifies inefficiencies precisely,
while also indicating starting points to discover their sources and eliminate
them.

Summary conclusions
Inclusion of large numbers of variables and lack of an overall success indicator
often frustrate management efforts to implement strategy and evaluate
performance with a BSC. This article has shown how indices can reduce the
number of variables under consideration. It also has demonstrated the utility of
DEA both in setting benchmarks for the variables/indices employed and in
generating overall performance measures. Along the way, it has pointed out
how to determine objectively:
.
whether one fairly can compare the performance of two DMUs;
.
the degree of a DMUs technical efficiency;
.
where and how much a DMU wastes inputs or outputs; and
.
the extent to which a DMU exploits its customer potential.
BIJ Through a simple example, the article furthermore has described how a
10,3 business-sector controller can use this information to guide strategy
implementation and operational decision making. Lastly, it has drawn
attention to the availability of alternative DEA models and explained limits to
the procedures use in connection with BSCs.
244
Notes
1. The interested reader can find formal mathematical treatments of DEA in Charnes et al.
(1978, 1994), Cooper (1996), Sengupta (1995) and Suhren (1997).
2. See, for example, the chapters by S. Covey, C. K. Prahalad, M. Hammer, E. Goldratt, P. Senge,
W. Bennis, J. Kotter, A. Ries and J. Trout, and P. Kotler in Gibson (1997).
3. Determining whether this divergence is due to the employment of different technologies,
different scales of operations, different national operating environments, or other factors
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

would require additional data and analysis.


4. This kind of analysis assumes the firms information technology is capable of supporting a
customer-based cost accounting system. See Rickards (1999) for an overview of the
possibilities and limits of such accounting.
5. Software containing all the models discussed is available from various vendors, including
1 Consulting Inc. (IDEAS 6 for Windows).
6. One minimizes the sum of the variances from the optimal inputs and outputs

References
Atkinson, A.B. and Stiglitz, J.E. (1969), A new view of technological change, Economic Journal,
Vol. 79, pp. 573-8.
Burkle, B. (1997), Effizienzmessung im Gesundheitswesen: Moglichkeiten und Grenzen der Data
Envelopment Analyse anhand von Anwendungen aus dem Krankenhausbereich,
Forschungsgruppe Medizinokonomie am Lehrstuhl fur Betriebswirtschaftslehre und
Operationsresearch der Universitat Erlangen-Nurnberg, Nuremberg.
Gibson, R. (Ed.) (1994), Data Envelopment Analysis: Theory, Methodology and Applications,
Kluwer Academic Pulishers, Boston, MA/Dordrecht/London.
Charnes, A., Cooper, W.W. and Rhodes, E. (1978), Measuring the efficiency of decision-making
units, European Journal of Operational Research, Vol. 2, p. 429 ff.
Chen, T.Y. (1997), Bank efficiency and ownership in taiwan: an evolution with data envelopment
analysis, Chung-Hua Institute for Economic Research, Tapei.
Cooper, W.W. (1996), Extensions and New Developments in Data Envelopment Analysis, Baltzer
Science Publishers, Amsterdam.
Freeman, C. (1994), The economics of technical change, Cambridge Journal of Economics,
Vol. 18, pp. 463-514.
Gibson, R. (Ed.) (1997), Rethinking the Future, Nicholas Brealey Publishing, London/Sonoma, CA.
Horngren, C.T., Foster, G. and Datar, S. (1997), Cost Accounting: A Managerial Emphasis, 9th ed.,
Prentice-Hall, Upper Saddle River, NJ.
Imai, K. and Yamazaki, A. (1992), Dynamics of the Japanese industrial system from a
Schumpeterian perspective, discussion paper presented at the International J.A.
Schumpeter Society Conference, Kyoto.
Institut der Wirtschaftsprufer (1999), Die Prufung des Risiko-Fruherkennungssystem nach 317 Data
Abs. 4 HGB (IDW PS 340), Wirtschaftsprufung, pp. 658-62.
Kaplan, R.S. and Norton, D.P. (1992), The balanced scorecard measures that drive
envelopment
performance, Harvard Business Review, pp. 71-9. analysis
Kaplan, R.S. and Norton, D.P. (1993), Putting the balanced scorecard to work, Harvard Business
Review, pp. 134-47.
Kaplan, R.S. and Norton, D.P. (1996a), Using the balanced scorecard as a strategic management 245
system, Harvard Business Review, pp. 75-85.
Kaplan, R.S. and Norton, D.P. (1996b), Balanced Scorecard: Translating Strategy into Action,
Harvard Business School Press, Boston, MA.
Naumann, H. and Zeis, J. (2000), Risiko-Controlling unter den Bedingungen des KonTraG, Der
Controlling-Berater, Heft 1, pp. 101-22.
Olve, N.-G., Roy, J. and Wetter, M. (2000), Performance Drivers A Practical Guide to Using the
Balanced Scorecard, J. Wiley, New York, NY/Chichester.
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

Rickards, R.C. (1999), Kundencontrolling und die Grenzen einer Kundenorientierung, in


Bastian, H., Born, K. and Dreyer, A. (Eds), Kundenorientierung im Touristik-management,
Oldenbourg, Munich, pp. 101-14.
Scharioth, J. and Huber, M. (2002), Balanced Scorecard als Werkzeug fur den Controller, Der
Controlling-Berater, Heft 1, pp. 43-62.
Sengupta, J. (1995), Dynamics of Data Envelopment Analysis: Theory of Systems Efficiency,
Kluwer Academic Publishers, Boston, MA.
Suhren, V. (1997), Data Envelopment Analysis: Vorstellung der Methode und Konzeption zur
Integration in ein umfassendes Fuhrungsinformationssystem, Institut fur
landwirtschaftliche Betriebslehre der Rheinischen Friedrich-Wilhelm-Universitat Bonn,
Bonn.
Vromen, J.J. (1995), Economic Evolution: An Enquiry into the Foundations of New Institutional
Economics, Routledge, New York, NY.
Welch, J. and Byrne, J.A. (2001), Jack: Straight from the Gut, Warner, New York, NY.
Welzel, P. (1996), Kosten und Groeneffizienz im Bankgewerbe Data Envelopment Analysis
der bayrischen Genossenschaftsbanken, (s.n.), Munich.
Werner, T. and Brokemper, A. (1996), Leistungsmessung mit System: Data Envelopment
Analyse als Instrument des Controlling, Controlling, Vol. Heft 1, pp. 164-70.
Westermann, G., Proll, R. and Canter, U. (1996), DEA effektives Instrument zur
Effizienzmessung, Betriebswirtschaftliche Blatter, Vol. Heft 2, pp. 77-81.

Further reading
Benz, E. (n.d.), TQM als integriertes Managementkonzept und Controlling, Der Controlling-
Berater, pp. 7/143-196.
Mayer, G. (n.d.) KAIZEN: Management-Philosophie zur Erfolgssteigerung, Der Controlling-
Berater, pp. 7/197-232.
Reichmann, T. (1997), Controlling mit Kennzahlen und Managementberichten: Grundlagen einer
Systemgestutzten Controlling-Konzeption, 5th ed., Vahlen, Munich.
Stermetz, E. (1999), Wertorientiertes Controlling die wichtigsten Kennzahlen im Uberblick,
Der Controlling-Berater, Heft 6, pp. 121-42.
This article has been cited by:

1. Sangeeta Sahney Vinod Gupta School of Management, Indian Institute of Technology, Kharagpur, India
Jitesh Thakkar Department of Industrial and Systems Engineering, Indian Institute of Technology,
Kharagpur, India. . 2016. A comparative assessment of the performance of select higher education
institutes in India. Quality Assurance in Education 24:2, 278-302. [Abstract] [Full Text] [PDF]
2. Panagiotis D. Zervopoulos, Theodora S. Brisimi, Ali Emrouznejad, Gang Cheng. 2016. Performance
measurement with multiple interrelated variables and threshold target levels: Evidence from retail firms
in the US. European Journal of Operational Research 250:1, 262-272. [CrossRef]
3. Peter A Aghimien Department of Accounting, Indiana University South Bend, South Bend, Indiana,
USA Fakarudin Kamarudin Universiti Putra Malaysia, Serdang, Malaysia Mohamad Hamid Universiti
Putra Malaysia, Serdang, Malaysia Bany Noordin Universiti Putra Malaysia, Serdang, Malaysia . 2016.
Efficiency of Gulf Cooperation Council Banks. Review of International Business and Strategy 26:1, 118-136.
[Abstract] [Full Text] [PDF]
4. Elisa Kusrini, Subagyo, Nur Aini Masruroh. 2016. Designing Performance Measurement For Supply
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

Chain's Actors And Regulator Using Scale Balanced Scorecard And Data Envelopment Analysis. IOP
Conference Series: Materials Science and Engineering 105, 012032. [CrossRef]
5. Te-Min Chang, Chao-Hsien Sung, Guo-Hsin Hu, Ming-Fu Hsu, Keng-Pei LinIdentifying Highly
Potential Enterprises with Social Computing on Supply Chain Networks 261-266. [CrossRef]
6. Ali Azadeh, Mansour Zarrin, Mohammad Abdollahi, Saeid Noury, Shabnam Farahmand. 2015. Leanness
assessment and optimization by fuzzy cognitive map and multivariate analysis. Expert Systems with
Applications 42:15-16, 6050-6064. [CrossRef]
7. Morteza Shafiee, Farhad Hosseinzadeh Lotfi, Hilda Saleh. 2014. Supply chain performance evaluation
with data envelopment analysis and balanced scorecard approach. Applied Mathematical Modelling
38:21-22, 5092-5112. [CrossRef]
8. Mohamed A.M. El-Hindawy, Adnan M.S. Alamasi. 2014. Measurement of the Strategic Performance of
Hospitality in the Kingdom of Saudi Arabia: a balanced scorecard Approach (BSC). Arab Economic and
Business Journal 9:1, 12-26. [CrossRef]
9. Farhad Hosseinzadeh Lotfi, Gholam Reza Jahanshahloo, Mohsen Vaez-Ghasemi, Zohreh Moghaddas.
2013. Evaluation progress and regress of balanced scorecards by multi-stage Malmquist Productivity Index.
Journal of Industrial and Production Engineering 30:5, 345-354. [CrossRef]
10. Nikos Kartalis, John Velentzas, Georgia Broni. 2013. Balance Scorecard and Performance Measurement
in a Greek Industry. Procedia Economics and Finance 5, 413-422. [CrossRef]
11. Teresa Garca Valderrama, Vanesa Rodrguez Cornejo, Daniel Revuelta Bordoy. 2013. Balanced Scorecard
and Efficiency: Design and Empirical Validation of a Strategic Map in the University by Means of DEA.
American Journal of Operations Research 03:01, 30-52. [CrossRef]
12. Carla A.F. Amado, Srgio P. Santos, Pedro M. Marques. 2012. Integrating the Data Envelopment Analysis
and the Balanced Scorecard approaches for enhanced performance assessment. Omega 40:3, 390-403.
[CrossRef]
13. Ioannis E. Tsolas. 2011. Modelling profitability and effectiveness of Greek-listed construction firms: an
integrated DEA and ratio analysis. Construction Management and Economics 29:8, 795-807. [CrossRef]
14. Reza Farzipoor SaenDepartment of Industrial Management, Faculty of Management and Accounting,
Islamic Azad UniversityKaraj Branch, Karaj, Iran. 2010. Performance measurement of power plants in
the existence of weight restrictions via slacksbased model. Benchmarking: An International Journal 17:5,
677-691. [Abstract] [Full Text] [PDF]
15. Mohamed M. MostafaCollege of Business, Auburn University, Auburn, Alabama, USA. 2010. A neuro
computational intelligence analysis of the US retailers' efficiency. International Journal of Intelligent
Computing and Cybernetics 3:1, 135-162. [Abstract] [Full Text] [PDF]
16. Mohamed M. MostafaNew York Institute of Technology, Global Program in Bahrain, Adliya, Bahrain.
2010. Does efficiency matter?. International Journal of Productivity and Performance Management 59:3,
255-273. [Abstract] [Full Text] [PDF]
17. Chwan-Yi Chiang, Binshan Lin. 2009. An integration of balanced scorecards and data envelopment
analysis for firm's benchmarking management. Total Quality Management & Business Excellence 20:11,
1153-1172. [CrossRef]
18. Teresa Garca-Valderrama, Eva Mulero-Mendigorri, Daniel Revuelta-Bordoy. 2009. Relating the
perspectives of the balanced scorecard for R&D by means of DEA. European Journal of Operational
Research 196:3, 1177-1189. [CrossRef]
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

19. Mohamed M. Mostafa. 2009. Modeling the competitive market efficiency of Egyptian companies: A
probabilistic neural network analysis. Expert Systems with Applications 36:5, 8839-8848. [CrossRef]
20. Mohamed M. MostafaCollege of Business, Auburn University, Auburn, Alabama, USA. 2009.
Benchmarking the US specialty retailers and food consumer stores using data envelopment analysis.
International Journal of Retail & Distribution Management 37:8, 661-679. [Abstract] [Full Text] [PDF]
21. Mohamed M. Mostafa. 2009. A probabilistic neural network approach for modelling and classifying
efficiency of GCC banks. International Journal of Business Performance Management 11:3, 236. [CrossRef]
22. HeBoong KwonCollege of Business, Central Washington University, Ellensburg, Washington, USA
Philipp A. StoeberlJohn Cook School of Business, Saint Louis University, St Louis, Missouri, USA
SeongJong JooCollege of Business, Central Washington University, Ellensburg, Washington, USA.
2008. Measuring comparative efficiencies and merger impacts of wireless communication companies.
Benchmarking: An International Journal 15:3, 212-224. [Abstract] [Full Text] [PDF]
23. Wai Peng WongDepartment of Industrial and Systems Engineering, National University of Singapore,
Singapore Kuan Yew WongFaculty of Mechanical Engineering, Universiti Teknologi Malaysia, Malaysia.
2008. A review on benchmarking of supply chain performance measures. Benchmarking: An International
Journal 15:1, 25-51. [Abstract] [Full Text] [PDF]
24. Mohamed M. MostafaGulf University for Science and Technology, Hawally, Kuwait. 2007. Evaluating
the competitive market efficiency of top listed companies in Egypt. Journal of Economic Studies 34:5,
430-452. [Abstract] [Full Text] [PDF]
25. Mohamed MostafaGulf University for Science and Technology, Hawally, Kuwait. 2007. Modeling the
efficiency of GCC banks: a data envelopment analysis approach. International Journal of Productivity and
Performance Management 56:7, 623-643. [Abstract] [Full Text] [PDF]
26. Katja RajaniemiABB Corporate Research, BadenDattwil, Switzerland ABB Distribution Automation,
Mustasaari, Finland. 2007. Internetbased scanning of the competitive environment. Benchmarking: An
International Journal 14:4, 465-481. [Abstract] [Full Text] [PDF]
27. Mohamed MostafaGulf University for Science and Technology, Hawally, Kuwait. 2007. Benchmarking
top Arab banks' efficiency through efficient frontier analysis. Industrial Management & Data Systems 107:6,
802-823. [Abstract] [Full Text] [PDF]
28. Robert C. RickardsThe Leipzig Graduate School of Management, Handelshochschule Leipzig, Leipzig,
Germany Hochschule Harz, Wernigerode, Germany. 2007. BSC and benchmark development for an e
commerce SME. Benchmarking: An International Journal 14:2, 222-250. [Abstract] [Full Text] [PDF]
29. Wai Peng WongDepartment of Industrial and Systems Engineering, National University of Singapore,
Singapore Kuan Yew WongFaculty of Mechanical Engineering, Universiti Teknologi Malaysia, Skudai,
Malaysia. 2007. Supply chain performance measurement system using DEA modeling. Industrial
Management & Data Systems 107:3, 361-381. [Abstract] [Full Text] [PDF]
30. Salaman Abbasian-Naghneh, Mahboobeh Samiei, Marziyeh Felahat, Marziyeh MahdaviA New Integrative
Approach Based on Balanced Scorecard, Data Envelopment Analysis, and Management Performance to
Prioritize Research and Development Projects 324-348. [CrossRef]
31. Abdel Latef Anouze, Ibrahim H. OsmanMismanagement or Mismeasurement: 276-322. [CrossRef]
32. Ibrahim H. Osman, Abdel Latef AnouzeA Cognitive Analytics Management Framework (CAM-Part 1):
1-79. [CrossRef]
Downloaded by University of Ghana At 16:03 27 June 2016 (PT)

You might also like