See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/281620345

Educational evaluation in Pakistani higher education context

Article · March 2013

CITATION READS
1 1,344

1 author:

Muhammad Abdul Wahid Usmani

King Saud University
10 PUBLICATIONS 6 CITATIONS

SEE PROFILE

All content following this page was uploaded by Muhammad Abdul Wahid Usmani on 12 April 2017.

The user has requested enhancement of the downloaded file.

 Contemporary Educational Researches Journal
Volume 03, Issue 1, (2013) 12-20
www.awer-center/cerj/

Educational evaluation in Pakistani higher education context

Muhammad Abdul Wahid Usmani *, Quality and Head of Quality Planning Unit, King Saud University,
Kingdom of Saudi Arabia.
Suraiya Khatoon, Quality Enhancement Cell, Dow University of Health Sciences, Karachi, Pakistan.

Suggested Citation:
Usmani, M., A., W. & Khatoon, S. (2013). Educational evaluation in Pakistani higher education context.
Contemporary Educational Researches Journal. 3(1), 12-20.

Received 02 January, 2013; revised 13 February, 2013; accepted 08 March, 2013.

Selection and peer review under responsibility of Assoc. Prof. Dr. Gulsun Atanur Baskan, Hacettepe University, Turkey
©
2013 SPROC LTD. Academic World Education & Research Center. All rights reserved.

Abstract

Educational Evaluation is one of the growing concerns of higher Education institutions in Pakistan. Since it
is relatively new and have been imbedded through higher education commission of Pakistan, there is a
need to focus on various areas of academic evaluation and use the right level of evaluation with a clear
focus on the expected outcomes. Three areas of educational evaluation need to be addressed on priority
basis in higher education institution in Pakistan namely course, faculty and program evaluation. Though
universities in Pakistan have introduced this evaluation in some form at different stages and results are
used for performance improvement, the missing link is triangulation. This paper focuses on these areas of
evaluation discussing how they can be made more effective and suggests how triangulation of results of
these three areas would increase the positive impact on the quality of higher education in Pakistan.

Keywords: academic evaluation, teacher evaluation, course evaluation, program evaluation, connection in
evaluation.

*ADDRESS FOR CORRESPONDENCE: Muhammad Abdul Wahid Usmani, Quality and Head of Quality Planning Unit,
King Saud University, Kingdom of Saudi Arabia, E-mail address: awusmani@yahoo.com
Usmani, M., A., W. & Khatoon, S. (2013). Educational evaluation in Pakistani higher education context. Contemporary Educational
Researches Journal. 3(1), 12-20.

1. What is Educational Evaluation?

Evaluation is one of the cornerstones of any academic program. It is the backbone in the
overall running of the program. Evaluation, in general, is regarded as a systemic and ongoing
process of judging the value, merit, scope or significance of the performance of somebody or
something on set standards or criteria. But specifically, there is no any fixed definition of
evaluation. Experts and evaluators have defined it variously. To quote Rossi, Lipsey & Freeman,
(2004), evaluation is a methodical, thorough, careful and detailed application of scientific
methods to judge the design, its execution, progress and finally the product or the result of a
program”. St Leger and Walsworth-Bell have the similar opinion. They regard evaluation as a
careful and objective judgment of a program or a degree to see how well it has met the
objectives stated earlier (cited in, Reeve & Peerbhoy, 2007). Another group of scholars belonging
to the same school of thought are Stake and Schwandt (2006). They believe that the aim of
evaluation is to form a judgment to estimate the quality of a program or something.
The above cited definitions clearly point to one fact that evaluation is a sort of structured
investigation aiming to find out the extent of performance appraisal of a program or a person or
anything in focus. But as there are various types of evaluation, one can say that the purpose of all
types of evaluation is not same. Alkin and Ellett (1990) have clearly asserted in their studies that
evaluation is a broad domain that is not limited to one area and so it cannot be encapsulated in
one theory to describe all functions that it serves under various heads or categories. In
Educational context, evaluation may vary depending upon the context in which it has to be
implemented. What may be suitable in one context may not be applicable in the other context.
Even, within context, it may vary based on the available resources (Stake, 1999; Usmani, 2010).
Although evaluation has been a regular feature of everyday life, it has become a policy matter
and a tool for quality assurance and development for higher education institutions at national
and international levels. Higher education sector, all over the world, have become much more
concerned about developing evaluation policies where lot of weight age is given in establishing
units, departments or organizations who are skilled and specialized in conducting evaluations. In
this situation, has become 'a multi-purpose activity with many possible approaches.' (Hanne,
2009).

2. Educational Evaluation & Pakistani Context

Evaluation in education always varies in different context. What may seem suitable in one
context may not be appropriate in other context. Event the purposes and procedures of
educational evaluation will vary from instance to instance. What is quite appropriate for one
school may be less appropriate for another. Looking at the current Educational Evaluation in
Pakistani context, it is observed that educational evaluation was practised at institutional level
only and even that was not structured and organized. The Deans or the Administrative Heads
applied various tools for course and faculty evaluation, even that appraisal varied in approach in
quality of tools. Program Evaluation was a rare practice and that was done by very few
prestigious private sector universities.
With the emergence of Higher Education Commission Pakistan, it was realized that a separate
Quality Assurance Agency needs to be established for a structured Academic Evaluation. The
Agency established Quality Enhancement Cells in Public and Private sector universities in a phase
wise manner for conducting program evaluation through self assessment. At the same time,
course evaluation and faculty evaluation were also added to program assessment along with
other feedback tools.

3. Domains of Evaluation
The current paper is based on a recommendation report by a Task Force of Teaching and
Learning in January 2010 released by the Simon Fraser University (SFU) in which they have
13
Usmani, M., A., W. & Khatoon, S. (2013). Educational evaluation in Pakistani higher education context. Contemporary Educational
Researches Journal. 3(1), 12-20.

recommended to “evaluate Program, Courses, and Instructors regularly, systematically, and

effectively for learning effectiveness.” The study suggests evaluation of all three elements with
regular frequency and triangulation of all three for integrated outcomes.

3.1. Course Evaluation

Course Evaluation is the first and foremost areas of educational evaluation. Despite the fact
that many universities in Pakistani context provide a structured course outline to students in the
beginning of the sessions, however, students and faculty are not required to evaluate the course
at any stage. Thus, the course is taught and examination is conducted without any evaluation
which, if done, would provide useful data for future improvement.
In Pakistani context, it is highly desirable to have a structured course evaluation by students
and faculty. A suggested frame work for the course evaluation by students may have the
following major heads.
• Course Content
• Course Delivery
• Course Management
• Course Assessment
• Course Evaluation

Under these heads questions can be framed for student feedback. These heads have been
derived from various sources such as QAA of Higher Education Commission, Pakistan, Course
Evaluation form designed by Brandeis University, Boston, Course Evaluation form designed by
the University of Greenwich, England. (http://www2.gre.ac.uk/).
For faculty course review, a suggested framework is available on QAA website. This form
provides input of the teacher on course and its effectiveness. The results may be correlated with
students’ feedback.
The implementation of course evaluation should also be structured and all the students need
not to be involved because in Pakistani context, there is always a problem of large classes and at
times there are more than hundred students in a class taking five courses. If a feedback is
collected from each student, a sum of two thousands forms will be collected for evaluation in a
four years degree program. If manual feedback is not taken and online is done that reduces
validity of data and the ration of online feedback of the students is not that high as there is no
culture of course evaluation till the students are admitted in Higher Education. Therefore, a
group feedback may be obtained from classes where the size is more than fifty. The classes may
be divided into five groups and each group should evaluate one course separately. Later, Corn
Boch Alpha may be used for validating the consistency in responses.
Course evaluation refers to a systematic examination of ' all aspects of the course' that include
its prerequisites and place in the total curriculum, the selection and ordering of content, the
choice of teaching and assessment methods, and the destination of graduates'. The first and
foremost aim of designing and implementing the course evaluation process is to identify the
strengths and areas for improvement in a course and to suggest possible changes in its content,
organization, teaching or assessment (Allen, 1984). Because a course, despite being interesting
and important can create problems in its proper understanding if the topics or the course
contents are deficient in logical sequencing. In such cases, periodic course evaluation of students
can point out this lacking through their feedback. Denson, Loveday & Dalton (2010) have also
identified the same that the course evaluations are conducted mainly to collect student feedback
to seek areas for improvement and to devise timely corrective and preventive actions to rectify
the gaps identified to ensure provision of quality education in each course. Allen (1984) has
suggested that this is the responsibility of the evaluator or the program
14
Usmani, M., A., W. & Khatoon, S. (2013). Educational evaluation in Pakistani higher education context. Contemporary Educational
Researches Journal. 3(1), 12-20.

administration to see and decide carefully that whether the decisions related to the possible
areas for suggested changes in a course after its are within the management or faculty control or
are beyond their control because it will not be useful to seek students feedback on the areas
where the changes can be made only after the due approval by the concerned professional
regulatory bodies or accreditation councils.

3.2. Faculty Evaluation

Faculty performance appraisal is an ongoing process and is a regular feature of teaching and
learning in higher education. There are several ways to collect feedback on teaching and
learning. However, students’ rating for teachers has some validity as the teachers who are rated
high are those from whom students learn most (Keache-1969). Murray (1995) and Rifkin (1995)
hold similar opinion that “Students Evaluation aims preliminary to facilitate faculty growth,
development and self improvement”. Students' feedback to assess the quality of teaching and
learning in higher education has been considered as the primary source of collecting such useful
data (Weinberg, Hashimoto & Fleisher 2009). Implementation of Faculty evaluation, at the end
of each semester, has become a worldwide common practice of obtaining feedback from
students regarding the quality of teaching (Zabaleta 2007). Arcot, Ramon Durazo-Arvizu, Amy
Hoyt & John (2013) have reported that s students ratings are universally used in higher
education institutes to collect information on the efficacy of the faculty teaching. This
information is then used as the basis for decision making regarding the faculty promotion,
performance appraisal and best teaching awards. Keeping such high importance of these
evaluations, they have become somewhat controversial with regard to their reliability and
validity (Cruse, 1987; Fich, 2003; Shaw, 2013; Wright, 2006).
There are researches conducted in the past which have concluded that student evaluations
are least reliable in determining the level of learning as they are more influenced by the
teachers' personality attributes and appearance, their sense of humor, their attitude and their
easy grading or assigning good marks (Neath, 1996; Wright, 2006).
On the other hand, Lori, Kogan, Regina Schoenfeld-Tacher & Peter (2010) have cited that
there are many researchers who have supported the idea that faculty evaluation through
students has been one of the most effective tools of assessing the quality of teaching. Just to
quote few examples, Marsh (1987) has found students feedback as multidimensional, reliable,
and a useful tool for faculty, students, and administration. Aleamoni (1999) has explored that
students feedback can be used as an effective measuring tool to 'enhance teaching, course
content, and document instructional effectiveness.' Likewise, Baird (1987) has maintained that
teachers evaluation through students is a valid performance indicator that shows what students
have learned from that particular instructor.
But one thing that all agree upon is the fact that student evaluations aim to provide formative
feedback which ultimately help the faculty improve 'their teaching performances' (Elzubeir &
Rizk, 2002; Green, Ellis, Frémont & Batty, 1998; Neath, 1996). With this single goal, one can
expect that the faculty evaluations through students would improve over time. (Arcot J.et al.
2013)
However, the research on faculty evaluation through students’ ratings has been focused for
couple of decades and other elements of teachers’ performance such as teaching portfolios,
peer review and class observation, have not been explored (Ory, 2001). Even researchers who
have given their views in favor of faculty evaluation through students feedback have suggested
that they should not be used alone as the only performance indicator but 'should be used in
conjunction with other forms of teaching performance evaluation' (Marsh 1984).
In Pakistani context, faculty evaluation is least structured and was not practised in many
Public Sector Universities however, Higher Education Commission Pakistan, through Quality
Enhancement Cells made it a mandatory part of teaching and learning. So gradually it is getting
strength in this system. However the problem of logistic and mechanism remains there.
Intuitions where there are large number of classes, it becomes extremely difficult to collect
15
Usmani, M., A., W. & Khatoon, S. (2013). Educational evaluation in Pakistani higher education context. Contemporary Educational
Researches Journal. 3(1), 12-20.

students’ feedback of each teacher at the given time for example if there are fifty students in the
class and six teachers are teaching to that class, students will be required to complete one form
six times. Secondly, again, a huge data will be collected for analysis. Thirdly, the questions in the
feedback form will be simple and easy to understand.
Okpala and Ellis (2005) collected and analyzed data from 218 US college students regarding
their perception of teachers’ quality components. Following table shows their findings:
S. No. Key Components Weightage
1 Caring for students and their learning. 89.6%
2 Teaching skills 83.2%
3 Content Knowledge 76.8%
4 Dedication to teaching 75.3%
5 Verbal Skills 73.9%

Similarly, several other studies have been conducting in which student perception of effective
Instructors but no studies have been conducted in Pakistani context. Therefore, the Performa
seems less fitting. Keeping the Pakistani context in mind, following heads are suggested for
inclusion in a faculty evaluation form:
 Teacher content knowledge
 Organization of the lecture
 Method of delivery of lecture
 Teacher presentation skill
 Teacher communication skill
 Regularity in teaching
 Teacher Interaction with student during and after the class
 Availability for consultation after the class
The last three areas are particular in Pakistani context. Regularity of teaching faculty in state
owned universities is one of the issues. Either classes do not start on time and often end before
time. At times, the classes do not take place as per the schedule. Secondly, teacher interaction or
classroom questioning is not fully developed. Teaching is usually one sided. Finally, faculty
availability for students’ consultation is not the culture in Pakistani context.

3.2. Program Evaluation

Program evaluation is the third significant area of educational evaluation. Program evaluation
is a measuring tool that is often used to check and validate the efficacy of the program to see
how well it has met its stated vision, mission and objectives (Fleischman & Williams, 1996).
Carter McNamara in his book, Basic Guide to Program Evaluation defines Program Evaluation as
a process of gathering useful information regarding the various aspects or a single aspect of a
program to make informed decisions about it. It can involve a variety of approaches, methods,
designs, techniques and tools to measure the overall performance (Carter McNarman).
There are several advantages of Program Evaluation as pointed out by Carter McNarman. First
of all, it helps the institution to see how well or how effectively the program is satisfying the
expectations of their stakeholders such as students, teachers, management, job market and also
society. Then, it helps to identify if the program is yielding desired results as per the set mission
and objectives and producing what it claims to produce or not. By doing structured program
evaluation, one can not only easily identify areas for improvement so that better results can be
produced by focusing the areas or 'vital few' to rectify the causes responsible for
16
Usmani, M., A., W. & Khatoon, S. (2013). Educational evaluation in Pakistani higher education context. Contemporary Educational
Researches Journal. 3(1), 12-20.

that but also draw comparisons with other programs in the same or related disciplines for bench
marking (Carter McNarman).
Program evaluation provides the program developers with a structured guideline to help
them see whether their planning, implementing and delivering to the program stakeholders is
exactly as per the program requirements.
Bray (2008) has also maintained the same that Program Evaluation is a tool of Quality
Assurance that is used in most of higher education institutions with an aim to improve the
quality of the program and although the program evaluators often have to face severe resistance
from the faculty and management while implementing this useful tool in making them realize
that the results of such evaluations are for the benefit of the program if the recommendations
are followed, eventually they come to admit that such practices really prove to be fruitful in
enhancing the program quality because they do not focus on highlighting the weaknesses of
individuals who are running the programs but the overall intended and actual goals and
practices of the program are evaluated and appraised (Bray, 2008).
Posavac, 2007 views Program evaluation as a process of evaluating the instructional programs
that have completed their various academic cycles for the purpose of documenting their quality
and usefulness in the attainment of their preset mission (Posavac 2007; Fitzpatrick, Sanders &
Worthen, 2004).
Another group of researchers Rossi et al. (2004) have regarded the process of program
evaluation as ‘the systematic application of scientific methods to assess the design,
implementation, improvement or outcomes of a program’ (Rossi & Freeman, 1993; Short,
Hennessy & Campbell, 1996).
Likewise, Kirkpatrick defines Program evaluation as a systematic process of utilizing data to
judge a given program (Kirkpatrick, 2006).
Program Evaluation helps determine the extent to which a program meets its preset
objectives as well as the extent to which it falls short of accomplishing the stated goals and
objectives (Cronbach, Ambron, Dornbusch, Hess, Hornik, Phillips, Walker & Weiner, 1980). With
the emergence of Quality Assurance demands in higher education, the higher education
institutions that have been acclaimed to be providing quality education “are called into question
the capacity of the university sector to deliver quality outcomes” (Harker, 1995).
In some educational contexts, program evaluation and program assessment are used
synonymously however, there is a difference between the two as pointed out by Suskie (2004)
Evaluation focuses on the ' appropriateness and quality of a program or curriculum' whereas
assessments focus on the 'achievements' of individual learners (Suskie, 2004).
Different QA consultants have proposed and implemented different approaches and models
to assure quality in higher education. A famous Program Evaluation model presented by
Kirkpatrick (2006) has formed its bases on four basic levels of evaluation areas. These levels
include:
1. Reaction that results as a satisfaction with the quality of a program
2. Learning that results from the desired changes in enhanced knowledge, skills and
attitude coming from the participation in the program
3. Behavior that is characterized by the application of learned knowledge, skills and
attitude to the real life and profession
4. Results that is the outcome of the summative assessments indicating the changes in a
system or organization related to program participation (Kirkpatrick & Kirkpatrick,
2006)

17
Usmani, M., A., W. & Khatoon, S. (2013). Educational evaluation in Pakistani higher education context. Contemporary Educational
Researches Journal. 3(1), 12-20.

Several strategies or practices for program evaluation are there. Cooper has given several
strategies. Self Assessment is one of such strategies which has been adopted by Quality
Assurance Agency of the Higher Education Commission of Pakistan. This Self Assessment of
programs is based on eight criterion and several standards as suggested by Prof. Dr. Abdul
Raoof.45 public sector universities have implemented this approach however, with the exception
of few universities, the program assessment reports have not yet deepened its roots into system.
That may be because the system is new to university administration and faculty. Furthermore,
there is no criterion for continuous improvement is not included in this self assessment
approach. Therefore, after one cycle of program assessment, continuous improvement is not
observed. Secondly, most of the officials engaged in developing program assessment reports
overlook the fact that they are supposed to attain the standards mentioned in the self
assessment manual rather they document the current status under each standard.

4. Triangulation in Evaluation
The most pertinent question is that if these elements namely course, faculty and program
evaluation are already a part of educational evaluation in Pakistani context, why there is the
need and significance of this suggested framework. Generally, universities practice one or two
elements (usually course or faculty evaluation) and program evaluation is usually neglected.
After the establishment of Quality Assurance Departments in universities through QAA, of HEC,
Pakistan, all three elements have been made mandatory. However, the missing element is of
triangulation. The QA department developed self assessment reports of different programs but
there is little portion for course evaluation and no room for faculty evaluation results (See Self
Assessment Manual of HEC, QAA).
QAA has provided separate feedback proforma for course and faculty evaluation but since
there is no reflection in the SARs, the data is not utilized. In some universities, teachers’
evaluation is conducted as a separate entity and results are communicated to faculty members.
In addition, program evaluation frequency that has been suggested by QAA is two years. It would
focus on broad areas such as program objectives, outcomes and curriculum. It does not address
individual courses or faculty members. Thus, these three elements stay apart.
Considering the above, this paper stresses the need for a structured implementation of
course, faculty and program evaluation and once the data is collected, the results should be
drawn after triangulation of these elements. This means findings of course evaluation and
faculty evaluation should be first correlated in each semester and their findings should be
correlated with the findings of program evaluation. This triangulation would lead to continuous
improvement plan for the program which is missing from program evaluation. Taking the
example of a program where course evaluation by students and faculty gives poor results,
immediate measures for improvement will be required for rectification. At the same time, the
results will be kept till program evaluation is complete so that this continuous course evaluation
would either supplement or challenge program evaluation results. Similarly, faculty evaluation by
students would make teaching and learning more interactive and students voice will be
entertained for effective teaching and learning. At the same time, the data and rectification
measures after teachers’ evaluation would be used during program evaluation for a global
scenario of the strengths and weaknesses of the program.

5. Summary and Conclusion

This paper is based on a recommendation by Simon Fraser University task Force which says
that “evaluate programs, courses, and instructors regularly, systematically, and appropriately for
learning effectiveness”. This paper highlights the significance of educational evaluation and
emphasizes that these three elements should not be treated separately as it happens in
Pakistani context. It suggests the triangulation of these elements for continuous improvement.

18
Usmani, M., A., W. & Khatoon, S. (2013). Educational evaluation in Pakistani higher education context. Contemporary Educational
Researches Journal. 3(1), 12-20.

Concluding this discussion, it can be said that course evaluation would provide data for
contents, organization and learning outcomes of the course. Teachers’ evaluation would focus
more on the delivery of that course, its effectiveness and students learning. Finally, program
evaluation would encompass the overall program, effectiveness including all stake holders. Thus,
a triangulation at the time of program evaluation would not only increase the validity of these
three elements but it would also provide continuous improvement plan.

References

Allen H. Miller (1984). The evaluation of university courses, Studies in Higher Education, 9(1), 1-15
Aleamoni, L.M. 1999. Student rating myths versus research facts from 1924 to 1998. Journal of Personnel
Evaluation in Education, 13, 153-66.
Alkin, M. C., & Ellet, F. S. (1990). Development of evaluation models. The international encyclopedia of
educational evaluation, 15-21.Arcot J. C., Ramon Durazo-Arvizu, Amy Hoyt & John A. McNulty
(2013) Do student evaluations influence the teaching skills of clerkship clinical faculty?, Educational
Research and Evaluation: An International Journal on Theory and Practice, 19(7), 628-635
Baird, J.S. (1987). Perceived learning in relation to student evaluation of university instruction. Journal of
Educational Psychology, 79, 90-1.
Cruse, D.B. (1987). Student evaluations and the university professor: Caveat professor. Higher Education,
16, 723–737.
Cronbach, L., J., Ambron, S., R., Dornbusch, S., M., Hess, R., D., Hornik, R., C., Phillips, D., C., Walker, D., F.
& Weiner, S., S. (1980). Toward Reform of Program Evaluation. San Francisco: Jossey-Bass
Publishers.
Denson, N., Loveday, T., & Dalton, H. (2010). Student evaluation of courses: what predicts satisfaction?.
Higher Education Research & Development, 29(4), 339-356.
Elzubeir, M., & Rizk, D. (2002). Evaluating the quality of teaching in medical education: Are we using the
evidence for both formative and summative purposes? Medical Teacher, 24, 313–319.
Encyclopedia Wikipedia (2010). Evaluation, Retrieved August 16, 2013 from:
http://en.wikipedia.org/wiki/Evaluation#cite_ref-3
Fich, F. (2003). Are student evaluations fair? Computing Research News, 15(2), 2–10.
Fleischman, H. L., & Williams, L. (1996). An introduction to program evaluation for classroom
teachers. Retrieved August, 15, 2004.
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative approaches and
practical guidelines. Boston, MA: Person Education.
Green, M. E., Ellis, C. L., Frémont, P., & Batty, H. (1998). Faculty evaluation in departments of family
medicine: Do our universities measure up? Medical Education, 32, 597–606.
Harker, B. (1995). Postmodernism and quality. Quality in higher education, 1(1), 31-39
Kirkpatrick, A. (2006). No experience necessary. Guardian Weekly.
Lori R. Kogan, Regina Schoenfeld-Tacher & Peter W. Hellyer (2010). Student evaluations of teaching:
perceptions of faculty based on gender, position, and rank, Teaching in Higher Education, 15(6),
623-636
Marsh, H.W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues
and directions for future research. International Journal of Educational Research, 11, 253-388.

Marsh, H.W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity,
potential biases and utility. Journal of Educational Psychology, 76, 707_54.
Mc Keachie, W.J. (1969) . Student Ratings of Faculty. AAUP Bulletin, 55, 439- 444.
Murray (1995) “Task Force on Assessing and Improving Teaching and Learning" at Indiana State University,
on line http://www.cedanet.com/indiana.htm accessed in March 2005 .
Neath, I. (1996). How to improve your teaching evaluations without improving your teaching.
Psychological Reports, 78, 1363–1372
Okpala, C. O., & Ellis, R. (2005). The perceptions of college students on teacher quality:A focus on teacher
qualifications. Education, 126, 374–378.

19
 Usmani, M., A., W. & Khatoon, S. (2013). Educational evaluation in Pakistani higher education context. Contemporary Educational
Researches Journal. 3(1), 12-20.

Ory, J. C., and Ryan, K. (2001). “How do Student Ratings Measure up to a New Validity Framework?” In M.
Theall, P. Abrami, and L. Mets (eds.), The Student Ratings Debate: Are they Valid? How can we best
Use Them? New Directions for Institutional Research, no. 109, San Francisco: Jossey-Bass.
Posavac, D. (2007). Uloga kelatora željeza kod bolesti bubrega.
Reeve, J., & Peerbhoy, D. (2007). Evaluating the evaluation: Understanding the utility and limitations of
evaluation as a tool for organizational learning. Health Education Journal, 66(2), 120-131.
Rifkin, T. (1995). The status and scope of faculty evaluation. ERIC Clearinghouse.
Rossi, P. H., & Howard, E. Freeman. (1993). Evaluation: A systematic approach, 5.
Shaw, G. P. (2013). Measuring teaching effectiveness – or not. Journal of the American Podiatric Medical
Association, 103, 94–96.
Stake, R. E. (1999). Summary of evaluation of reader focused writing for the veterans benefits
administration, American Journal of Evaluation, 20(2), 323–343.
Stake, R. E., & Schwandt, T. A. (2006). On discerning quality in evaluation. The Sage handbook of
evaluation, 404-418.
Short, L., Hennessy, M., & Campbell, J. (1996). Tracking the work. Family violence: Building a coordinated
community response: A guide for communities, 59-72.
Suskie, L. (2004). Assessing student learning: A common sense guide. Bolton, MA: Anker Publishing
Usmani, A., W., & Khatoon, S. (2010). Meta Evaluation of Teachers Evaluation Program Using CIPP Model.
Proceeding of second international conference on Quality in Higher Education, 2010. Retrieved
from: http://www.icaqhe2010.org/.../23-Dr%20M%20Abdul%20Wahid%20Usmani.pdf
Weinberg, B.A., M. Hashimoto, & B.M. Fleisher. 2009. Evaluating teaching in higher education. Journal of
Economic Education, 40, 227-54.
Wright, R.E. (2006). Student evaluations of faculty: Concerns raised in the literature, and possible
solutions. College Student Journal, 40, 417–422.
Zabaleta, F. (2007). The use and misuse of student evaluations of teaching. Teaching in Higher Education
12, 55-76.

20

View publication stats