Professional Documents
Culture Documents
4,1998
This study replicated a pile sort task involving 135 emotion terms originally pre-
sented by Shaver, Schwartz, Kirson, and O'Connor (1987) with a slight change in
procedures, from an unconstrained single-sort task to a constrained successive-
sort task, made necessary by methodological problems during their data analysis.
Subjects (86 male and female undergraduates) were asked to assign names to their
piles during the successive stages of sorting. This change produced a hierarchi-
cal cluster analysis solution supporting interpretations other than the prototype
structure found by Shaver et al.. Decision criteria reported by subjects during
sorting were described and revealed dimensions reported by previous investiga-
tors, suggesting that this is a viable method of determining dimensions important
in distinguishing emotion terms.
1Preparation of this article was supported in part by National Institute of Mental Health grant MH18931
to Paul Ekman and Robert Levenson for the NIMH Postdoctoral Training Program in Emotion Re-
search. Portions of this work were completed as part of a doctoral dissertation (see Alvarado, 1993).
I gratefully acknowledge the assistance of Louis Narens, Paul Ekman and William Irwin in the
completion of this work. I also thank the UC Irvine Semiotics Laboratory research assistants for their
help with data collection.
2Nancy Alvarado is now at the Center for Human Information Processing, University of California at
San Diego, 9300 Gilman Dr., MC-0109, La Jolla, California 92093-0109.
329
0146-7239/98/1200-0329$15.00 1998 Plenum Publishing Corporation
330 Alvarado
than data from subjects sorting into more piles. A replication correcting this flaw,
presented here, suggests a different solution and interpretation.
The study reported here performed successive constrained sorts rather than
a single unconstrained sort. It kept most other procedures identical to the Shaver
et al. (1987) study. An additional procedure was added in order to collect additional
information about the basis for subject decisions. Subjects were asked to label their
piles at each stage of the successive sort. If the underlying structure proposed by
Shaver et al. were present in the lexicon, it was expected that the results would
confirm those of Shaver et al.. However, it will be shown that a small procedural
change has a large impact upon the clustering solution, calling into question the
claims for prototypicality of certain emotion terms made by Shaver et al.
METHOD
The pile sort task reported by Shaver et al. (1987) was replicated with confor-
mance in every respect except the location (University of California), number
of subjects (86 vs. 100 male and female undergraduates), and the use of a suc-
cessive, constrained sort procedure rather than a single, unconstrained sort, as
described below. Modifications to the sort procedure for the present study were
made because, according to Weller and Romney (1988), data from individual, sin-
gle unconstrained sorts may not be aggregated, and because a successive sort better
allows a hierarchical structure to emerge during cluster analysis. Addition of the
naming procedure allows evaluation of whether the taxonomic structure imposed
by successive sorting is appropriate to the domain.
Materials
Each of the 135 emotion terms used by Shaver et al. (1987) was printed on a
2-in. x 4-in. card. Cards were shuffled before each session. See Shaver et al. for a
description of the method used to define the domain of emotion terms and verify
the suitability of items as members of that domain.
Procedure
Subjects were asked to sort the 135 cards into two piles based on meaning.
After sorting the terms into two piles, the subject was asked to assign a name
to each of the piles. These two steps were repeated until each of the first two
piles was subdivided into two piles, and each of those was again subdivided into
two piles. This resulted in a three-level sorting hierarchy with two piles at the
first level, four at the second, and eight at the third level. Sorting stopped after
three levels because sorting 135 items into additional levels would have presented
Determining the Structure of the Emotion Lexicon 331
practical difficulties for both subjects and experimenter. All subjects sorted into
the same number of piles. The assignment of names to each sorted pile was made
in order to investigate the rule or criteria subjects were using at each point in the
process.
RESULTS
Cluster Analysis
3
Boster has developed an alternative technique for circumventing this problem that allows subjects
to sort into as many piles as desired, but also provides consistent input data (see Weller & Romney,
1988). However, this technique is only workable with a much smaller set of items due to the burden
it places upon subjects.
332 Alvarado
produced, as would be expected from the change in sort procedure. While there
are obvious similarities between the two solutions, the number of negative piles
is greater in this solution, and the kinds of terms contributing to those piles are
different. A few terms have switched from being negative to positive and vice versa
(i.e., longing, sympathy).
Even without any analysis of prototypicality, it appears obvious that the tran-
sition from general through basic (at 50) to specific in the subcategories described
by Shaver et al. (1987) is not evident in this solution. Without this organizing
principle, it makes no sense to name the nodes using basic emotion terms (e.g.,
lust and melancholy are unlikely candidates for basic terms). Selection of 50 as
a "basic level" by Shaver et al. was arbitrary. Note that in Fig. 2, use of 50 as a
cutoff selects a very different set of categories than in Fig. 3, due to the deeper
hierarchical structure of the solution in Fig. 2.
Determining the Structure of the Emotion Lexicon 333
Fig. 1Continued
Analysis of Centrality
The terms produced by this analysis can be subjected to the same type of pro-
totypicality analysis used by Shaver et al. (1987) through comparison of centrality
scores, but given the composition of the categories, the resulting basic level terms
are likely to be unsatisfactory in the context of prototype theory. Multidimensional
scaling (MDS) also provides a visual display of the analysis of centrality per-
formed by Shaver et al.. The most basic or central items within a category will
also tend to be those nearest the origin in an MDS plot of the data upon which
centrality was calculated (Weller & Romney, 1988). This analysis used MINISSA,
as implemented by Borgatti (1993) in Anthropac, Version 4.02.
To test, using data from this study, Shaver et al.'s (1987) claim that love is a
basic term, different combinations of subgroups of positive terms, all taken from
Fig. 2 and Table I at different levels of division, were scaled. Love was the most
central term only when the categories with passion terms were scaled together
with those representing affection and caring, and those relating to happiness, ex-
citement, and joy were excluded. When the category containing love was scaled
334 Alvarado
Determining the Structure of the Emotion Lexicon 335
Fig. 3. Results of cluster analysis for unconstrained single sort of 135 emotion terms. (From "Emotion
Knowledge: Further Exploration of a Prototype Approach" by P. Shaver, J. Schwartz, D. Kirson, and
C. O'Connor, 1987, Journal of Personality and Social Psychology, 52, pp. 1061-1086. Copyright
1987 by the American Psychological Association. Reprinted by permission.)
alone, affection was the central term. Because love is not central when the other
subcategories (happiness, excitement) are included, it makes no sense to label the
higher-level node love in Fig. 2, as was done in Fig. 3.
Without some justification for selecting one level of splitting or some combi-
nation of clusters in preference to another, there is little way to determine whether
love is a most prototypical term. It cannot be selected in preference to other choices
revealed by analysis solely because the experimenter wishes it to be basic. That
dilemma was also found by Shaver et al. (1987), where affection was found to be
most prototypical within a category but love was asserted in preference to it, on
the basis of theory.
1 pleasure adoration
satisfaction affection
love
fondness
liking
attraction
caring
tenderness
compassion
sentimentality
2 relief arousal
optimism desire
hope lust
eagerness passion
infatuation
3 joy longing
pride
contentment
gaiety
glee
delight
jolliness
joviality
gladness
cheerfulness
happiness
amusement
enjoyment
4 adoration amusement
love bliss
caring cheerfulness
tenderness gaiety
affection glee
fondness jolliness
compassion joviality
liking joy
sentimentality delight
bliss enjoyment
sympathy gladness
rapture happiness
jubilation
elation
satisfaction
ecstasy
euphoria
5 passion enthusiasm
ecstasy zeal
arousal zest
desire excitement
attraction thrill
exhilaration
(Continued)
Determining the Structure of the Emotion Lexicon 337
Table I-Continued
a
Category Number Figure 2 Items Figure 3 Items
6 euphoria contentment
enthrallment pleasure
7 zest pride
zeal triumph
astonishment
8 jubilation eagerness
excitement hope
exhilaration optimism
elation
9 enthusiasm enthrallment
amazement rapture
thrill
triumph
surprise
10 lust relief
infatuation
11 melancholy amazement
longing surprise
astonishment
12 loathing aggravation
spite irritation
hate agitation
hostility annoyance
scorn grouchiness
grumpiness
13 wrath exasperation
anger frustration
rage
fury
outrage
ferocity
vengefulness
contempt
14 insult anger
disgust rage
revulsion outrage
fury
wrath
hostility
ferocity
bitterness
hate
loathing
scorn
spite
vengefulness
dislike
resentment
(Continued)
338 Alvarado
Table IContinued
a
Category Number Figure 2 Items Figure 3 Items
15 dread disgust
fear revulsion
fright contempt
panic
16 misery envy
suffering jealousy
anguish
17 torment torment
terror
horror
agony
hysteria
mortification
18 rejection agony
alienation suffering
dejection hurt
isolation anguish
19 defeat depression
humiliation despair
shame hopelessness
gloom
glumness
sadness
unhappiness
grief
sorrow
woe
misery
melancholy
20 pity dismay
regret disappointment
guilt displeasure
21 embarrassment guilt
tenseness shame
insecurity remorse
uneasiness regret
worry
nervousness
anxiety
22 woe alienation
remorse isolation
grief neglect
distress loneliness
rejection
homesickness
defeat
dejection
insecurity
embarrassment
humiliation
insult
(Continued)
Determining the Structure of the Emotion Lexicon 339
Table I Continued
a
Category Number Figure 2 Items Figure 3 Items
23 despair pity
depression sympathy
hopelessness
24 homesickness alarm
gloom shock
sorrow fear
sadness fright
loneliness horror
unhappiness terror
glumness panic
hysteria
mortification
25 hurt anxiety
displeasure nervousness
dismay tenseness
disappointment uneasiness
neglect apprehension
worry
distress
dread
26 exasperation
alarm
shock
apprehension
27 frustration
grouchiness
grumpiness
28 envy
jealousy
29 bitterness
resentment
30 dislike
aggravation
annoyance
irritation
agitation
a
Corresponds to numbers shown inFigs.2 and 3.
quotation marks). Note that each category names two piles, because at each level
the procedure required subjects to divide a single pile in two. Category 1 (positive/
negative) corresponds to Osgood's (1966) concept of evaluation, and consists of
names like good/bad, happy/sad, optimistic/pessimistic, and secure/insecure. Cat-
egory 2 (intensity) corresponds to Osgood's concepts of activity and potency, and
consists of names like strong/weak and more/less. Category 3 (duration) was coded
when subjects sorted on the basis of the length of experience of an emotion, and
included names like emotions/moods. Category 4 (self vs. others as source or
340 Alvarado
object) was coded whenever a subject made reference to the source or object of
emotion. It generally corresponded to DeRiviera and Grinkis's (1986) distinctions
among emotions and encompassed names like internal/external or active/passive
as well as more explicit designations, e.g., things you feel about other people.
Category 5 (relationship) was coded whenever the subject distinguished between
love and other relationships, between love and sex, or between emotions felt in
relationships versus all other emotions. This was distinguished from Category 4
on the basis of explicit mention of love or sex, or mention of relationship.
Category 6 (categorical) corresponded most closely to the prototype theory
sort criterion because the labels given might be considered to name "basic" emo-
tions or exemplars. Names consisting of any single emotion term within the set of
135 terms (not necessarily basic with respect to Shaver et al., 1987) were recorded
as Category 6, e.g., fear/anger. Names consisting of single terms (not among the
135 terms), e.g., bad feelings or pain, were coded as Category 12. Combining
these two categories might yield a rough estimate of how many subjects sorted
on the basis of resemblance to an exemplar emotion or prototype. Category 7 was
coded when the subject expressed an explicit concern for the specific versus gen-
eral nature of the terms within the set, e.g., common/uncommon or everyday emo-
tions/special occasion emotions. Category 8 was coded when subjects made strong
explicit reference to the judgmental quality (implicit references were coded as Cat-
egory 1) e.g., what a good Christian feels/evil or abnormal/normal. In some sense
all terms are evaluative and the distinction between Category 8 and Category 1 is
moot.
Category 9 (expression) was coded when subjects gave names distinguish-
ing between emotions largely felt versus those readily expressed, e.g., show/feel.
Determining the Structure of the Emotion Lexicon 341
DISCUSSION
Two conclusions can be drawn from this solution: (1) this pile sort method-
ology may be inappropriate for determining prototypicality of emotion terms
and testing prototype theory; (2) dimensionality of the domain of emotion terms
342 Alvarado
subject use of multiple sorting criteria highly visible, whereas it is more difficult
to know what subjects were doing in the task presented by Shaver el al.
This study found a lexical structure that applies a different criterion at each
descending node so that members of categories are distinguished by how they
were separated as each of these various criteria were applied. The analysis of
naming data is important because it verifies that the taxonomic structure produced
by the cluster analysis is appropriate to the domain (successive sorting can force
items into an inappropriate taxonomic structure; Weller & Romney, 1988). In other
words, the shift in criteria was found in the names provided by subjects and hence
exists in their parsing of the domain of terms, rather than as an artifact of the
cluster analysis algorithm. It is possible that the necessity to produce successive
piles produced the criteria. If so, this too is interesting because it demonstrates that
a different methodology can produce information about different dimensions of
the emotion lexicon. Theories of categorization have been assumed to be mutually
exclusive. It may be that subjects are capable of using several cognitive processes
for categorization, that they switch between them as the situation demands, and that
the characteristics of the task and the stimuli may determine which they employ
(Nosofsky, 1986). There may also be personal preference among subjects for one
or another type of categorization process.
Sorting requires subjects to make similarity judgments about items. When
subjects are constrained to two piles, they must decide whether items are the same or
different than previously sorted items. In contrast to a single sort task, a successive
sort better allows subjects to shift dimensions as needed in order to optimize per-
formance (Nosofsky, 1986). Unlike the single sort, subjects making successive
sorts are forced to discover additional decision criteria, which may be trivial in the
overall implicit structure of the lexicon determined by multidimensional scaling of
all 135 terms, or by a single-sort procedure. In most studies, individual subjects who
prefer these additional criteria are seldom noted because scalings produced based
upon group data camouflage such diversity. Usually, the two major criteria noted
in Table II emerge (Categories 1 and 2), with the remaining 10 considered noise.
This procedure permits examination of the relative importance of decision criteria,
as well as providing clues to a broader array of criteria (or dimensions for choice).
The solution produced by this study is less readily interpretable because it
combines data produced by subjects using widely different sorting strategies. Any
coherence likely results from the preferences for Category 1 at Level 1, followed
by Category 2 at Level 2. Subjects must be constrained to the same sort criteria
in order to produce a coherent solution with respect to the remaining dimensions.
Even so, some support exists for other parsings of the emotion lexicon.
In contrast to Shaver et al.'s (1987) solution, this study (Fig. 2 and Table I)
shows the internal/external distinction posited by Clore, Ortony, and Foss (1987)
at the subordinate level (see also Ortony, Clore, and Foss, 1987). Clear subcate-
gories for other-directed versus self-directed or other-inspired versus self-inspired
344 Alvarado
emotions exist (as suggested by DeRiviera and Grinkis, 1986). Clore et al.'s men-
tal versus nonmental division appears in the separation of lust and infatuation
from other forms of happiness, and in the distinction between passion and affec-
tion. Clore et al.'s cognition, behavioral, and affect focal subcategories were all
reflected in the decision criteria of certain subjects (see Table II). Further, several
subjects made either an explicit or an implicit distinction on the basis of word
form (causative vs. noncausative), as discussed by Clore et al. (see especially
pp. 754,756, and 761 of their article). No evidence of the subjective versus objec-
tive distinction in external conditions was found.
Conclusions
REFERENCES
Aldenderfer, M, & Blushfleld, R. (1984). Cluster analysis. Beverly Hills, CA: Sage.
Alvarado, N. (1993). The labeling of emotion. Unpublished doctoral dissertation, University of
California, Irvine. (University Microfilms International No. 9323902)
Borgatti, S. (1993). Anthropac: Version 4.02. Columbia, SC: Analytic Technologies.
Clore G., Ortony, A., & Foss, M. (1987). The psychological foundations of the affective lexicon.
Journal of Personality and Social Psychology, 53,751-766.
DeRivera, J., & Grinkis, C. (1986). Emotions as social relationships. Motivation and Emotion, JO,
351-369.
Nosofsky, R. (1986). Attention, similarity, and the identification-categorization relationship. Journal
of Experimental Psychology: General, 115,39-57.
Ortony, A., Clore, G., & Foss, M. (1987). The referential structure of the affective lexicon. Cognitive
Science, 11,361-384.
Osgood, C. (1966). Dimensionality of the semantic space for communication via facial expressions.
Scandinavian Journal of Psychology, 7,1-30.
Shaver, P., Schwartz, J., Kirson, D., & O'Connor, C. (1987). Emotion knowledge: Further exploration
of a prototype approach. Journal of Personality and Social Psychology, 52,1061-1086.
Smith, C., & Ellsworth, P. (1985). Patterns of cognitive appraisal in emotion. Journal of Personality
and Social Psychology, 48,813-838.
Weller, S., & Romney, A. K. (1988). Systematic data collection. Beverly Hills, CA: Sage.