Guidelines for the

Development of Learning
Outcomes






Presentation within the frame of the project
INARM

Veronika Nitsche and Michaela Handke
(WUS Austria)

www.wus-austria.org

Learning Outcomes


... are a statement of what a learner is
expected to know, understand, or able to do
at the end of a learning process.


From the definition of Learning
Outcomes we see

... an emphasis on the learner and an emphasis on
the learners ability to do something.

Focus on teaching
aims and objectives and
use of terms like know,
understand, be familiar
with.
Focus learning -
outcomes: what we
want the student to be
able to do - use of terms
like define, list, name,
recall, analyse,
calculate, design, etc.
Where LO can be found
Learning
Outcomes
Occupational
standards
Assessment
criteria
Qualifications

Qualifications
framework
Curricula
Where LO can be found

. on the example of curriculum development
at the University of Graz/Austria (I)
Structure:
Bachelor (180 ECTS, 240 ECTS) - >specific occupational field
Master (min. 120 ECTS) -> oriented towards occupation and research
Bachelor, Master, PhD should be clearly differentiated -> EUA recommends to use the Dublin
Descriptors

Change of paradigm: from teaching to learning
before: top-down on the basis of the research in
the field
now: bottom-up model on the basis of the
potential occupational fields
-> Definition of a qualification profile (now first
step in curriculum development)
Curriculum development at the
University of Graz/Austria (II)
Qualification profiles:
Define the qualifications a graduate should
have at the end of the studies (professional
and social/generic competences);
This qualification profil is the basis for the
definition of the learning outcomes of the
study programme and its modules.

Curriculum development at the
University of Graz/Austria (III)
Learning outcomes: are defined on the basis of
the qualification profile
> Basis for the definition of the type of courses,
allocation of ECTS, examination, etc.


Qualification Profiles (IV)
Frameworks for the development of
Qualification Profiles:
European Qualifications Framework
National Qualifications Framework
Dublin Descriptors
Standards by Law: in Austria 2002 University
Studies Act (UG 2002)

Writing Learning
Outcomes


Writing Learning Outcomes is a
Process not an Event (Bloom)
Blooms Classification of
Cognitive Skills

Category

Definition

Related Behaviors
Knowledge

recalling or remembering something without
necessarily understanding, using, or changing it
define, describe, identify, label, list, match, memorize,
point to, recall, select, state
Comprehension understanding something that has been
communicated without necessarily
alter, account for, annotate, calculate, change, convert,
group, explain, generalize, give examples, infer, interpret,
paraphrase, predict, review, summarize,
Application

using a general concept to solve problems in a
particular situation; using learned material in new
and concrete situations
apply, adopt, collect, construct, demonstrate, discover,
illustrate, interview, make use of, manipulate, relate,
show, solve, use
Analysis

breaking something down into its parts; may focus
on identification of parts or analysis of relationships
between parts, or recognition of organizational
principles
analyze, compare, contrast, diagram, differentiate, dissect,
distinguish, identify, illustrate, infer, outline, point out,
select, separate, sort, subdivide
Synthesis creating something new by putting parts of different
ideas together to make a whole.
blend, build, change, combine, compile, compose,
conceive, create, design, formulate, generate,
hypothesize, plan, predict, produce, reorder, revise, tell,
write

Evaluation

judging the value of material or methods as they
might be applied in a particular situation; judging
with the use of definite criteria
accept, appraise, assess, arbitrate, award, choose,
conclude, criticize, defend, evaluate, grade, judge,
prioritize, recommend, referee, reject, select, support

Some examples ...
At the end of the course students will be able to:

- Identify and consider ethical implications of scientific
investigations.
- Differentiate between civil and criminal law
- Select and employ sophisticated techniques for analysing
the efficiencies of energy usage in complex industrial
processes.
- Compare and contrast the different electronic business
models
- Propose solutions to complex energy management
problems both verbally and in writing
- Predict the effect of change of temperature on the position
of equilibrium.


Practical advice for writing
learning outcomes (I)

Begin each learning outcome with an action verb.
Use only one verb per learning outcome.
Avoid vague terms like know, understand, learn, be familiar with,
be exposed to, be acquainted with, and be aware of. These terms
are associated with teaching objectives rather than learning
outcomes.
Avoid complicated sentences. If necessary use more one than
one sentence to ensure clarity.
Ensure that the learning outcomes of the module relate to the
overall outcomes of the program.
General recommendation: 5 8 learning outcomes per module.
The learning outcomes must be observable and measurable.
Ensure that the learning outcomes are capable of being
assessed.

Practical advice for writing
learning outcomes (II)

Timescale: ask yourself if it is realistic to achieve the LO within
the time and resources available.
Assessment: How will you know if the student has achieved these
learning outcomes? Very broad LO may be difficult to assess
effectively. With very narrow LO, the list of learning outcomes
may be too long and detailed.
Before finalizing the learning outcomes, ask your colleagues and
possibly (former) students if the LO make sense to them.
When writing LO for students at levels beyond first year, try to
avoid overloading the list with learning outcomes which are drawn
from the bottom of Blooms taxonomy.
It is vital that learning outcomes are clearly written so that they
are understood by students, colleagues and external examiners.


ASSESSMENT

-> STEPS involved in linking Learning
Outcomes, Teaching and Learning Activities
and Assessment
Clearly define the learning outcomes.
Select teaching and learning methods that are
likely to ensure that the learning outcomes are
achieved.
Choose a technique or techniques to assess the
achievement of the learning outcomes.
Assess the learning outcomes and check to see
how well they match with what was intended
Linking Learning Outcomes
and Assessment Criteria
Learning outcomes imply the assessment
criteria!
Assessment criteria may be developed from the
learning outcome or from the assessment task
but in either case they should relate to the
learning outcome
Assessment criteria should test, assess or
relate to the learning that is mentioned in
the learning outcome. e.g if you say students
must write something, you should not test
them orally.


Benefits and potential problems
of Learning Outcomes?
The benefits of Learning
Outcomes
Help to explain more clearly to students what is
expected of them and thus help to guide them in their
studies.
Help teachers to focus more clearly on what exactly
they want students to achieve in terms of knowledge
and skills.
Help teachers to define the assessment criteria more
effectively.
Help to provide guidance to employers about the
knowledge and understanding possessed by graduates
of programmes.
Help to start discussion on teaching and learning in
third level institutions.
Potential problems with
Learning Outcomes
Could limit learning if learning outcomes written
within a very narrow framework lack of
intellectual challenge to learners.
Learning outcomes should not be reductionist but
rather expansive and intended to promote the
higher order thinking skills.
Danger of assessment-driven curriculum if
learning outcomes too confined.
Could give rise to confusion among students and
staff if guidelines not adhered to when drawing
up learning outcomes, etc.
Learning Outcomes
(summary)


Discussions at European and national level
highlight the need for common ground with
respect to LO this does not imply that
their need to be a common approach to
defining and using LO.
Further reading
Jenny Moon, Exeter University: Linking Levels, Learning
Outcomes and Assessment Criteria.
http://www.aic.lv/bolona/Bologna/Bol_semin/Edinburgh/
J_Moon_backgrP.pdf
Kennedy, D. et.al.: Writing and Using Learning
Outcomes: a Practical Guide.
http://www.tcd.ie/vpcao/academicdevelopment/assets
/pdf/Kennedy_Writing_and_Using_Learning_Outcomes
.pdf
John Biggs: Aligning teaching and assessing to course
objectives. Teaching and Learning in Higher
Education: New Trends and Innovations. University of
Aveiro, 2003
Using Learning Outcomes. European Qualifications
Framework Series: Note 4 (2011)


Further reading


http://www.eua.be/eua-work-and-policy-area/building-the
european-higher-education-area/bologna-basics/Bologna-an-
overview-of-the-main-elements.aspx

Shared descriptors for Short Cycle, First Cycle, Second Cycle and
Third Cycle Awards -> Known as the 'Dublin Descriptors' after
the meeting in which they were agreed, in Dublin, March
2004

WP 4
Learning outcomes for MA and BA



4.1 Intended learning outcomes for MA and BA

4.2 Peer-review by EU partners

4.3 Methodology of developing LOs

4.4 Adapted versions of the LOs for MA and BA
Next steps in WP 4?

4.1 Intended learning outcomes for MA and BA: The next step of
the project would be to design and develop intended LOs
drawing on the SQFs for informatics and management.

(1) Develop a general set of competences (related to occupation)
in the field of IT and Management


Timeframe:
- Dev. of set of competences by PCU/WG: submit it until 15/12/2013
(lists of profiles and functions, and related learning outcomes)
- Discussion and decision on the set of competences (1 for Mgmt., 1 IT):
-> at the Meeting in Kosice (Feb. 12-15, 2014)

(2) based on the general set of competences (and the methodology
developed under 4.3?) to develop a set of learning outcomes ->
Meeting in Kiev, Lviv (April 2014)


Next steps in WP 4?


4.2 Peer-review by EU partners (not only of the LO, but also on
the SQF/WP 3)

4.3 Methodology of developing LOs: To ensure wider application of
the methodology, guidelines on the LO development will be
developed. ->

WHO-> WG (tbc): Jolanta Rodzos, Veronika Nitsche, Svetlana Khapaeva,
Ruben Topchan, Tetyana Daragan, Prof. Telizhenko Alexander (UKR)

HOW: presentation and comparison of different approaches and
taxonomies, development of (a) methodologies

WHEN -> at the meeting in Kiev, Lviv (April 2014) tbc.

4.4 Adapted versions of the LOs for MA and BA: PC universities will
adapt the LO to the local needs. The adapted versions will be
circulated among the stakeholders for feedback, the LOs will be
refined and prepared for publications.

Thank you for your attention!