Professional Documents
Culture Documents
INTRODUCTION..............................................................................................................................................3 1.1 1.2 1.3 1.4 1.5 WHY COGNITIVE ARCHITECTURES NEED EMOTION ......................................................................................3 WHY EMOTION NEEDS COGNITIVE ARCHITECTURES ....................................................................................3 WHAT GOOD ARE EMOTIONS? (HOW EMOTIONS MAY BENEFIT AI) ..............................................................3 WHAT IS EMOTION?.....................................................................................................................................4 OVERVIEW ..................................................................................................................................................5
CLASSIC COGNITIVE ARCHITECTURES ................................................................................................6 2.1 2.2 2.3 OPERATORS .................................................................................................................................................6 MEMORIES ..................................................................................................................................................6 MECHANISMS ..............................................................................................................................................6
THE EMOTION PROCESS: A FRAMEWORK ...........................................................................................8 3.1 3.2 3.3 3.3.1 3.3.2 3.4 APPRAISAL THEORY ....................................................................................................................................8 EMOTION-INDUCED CHANGES ...................................................................................................................11 RESPONSES TO EMOTION ...........................................................................................................................12 Coping .................................................................................................................................................12 Emotion regulation ..............................................................................................................................13 PERSONALITY AND INDIVIDUAL DIFFERENCES...........................................................................................15
UNDERSTANDING THE DESIGN CHOICES ...........................................................................................17 4.1 APPRAISAL THEORY DESIGN CHOICES........................................................................................................17 4.1.1 How are the appraisal values generated? ...........................................................................................17
4.1.1.1 4.1.1.2 4.1.1.3 4.1.1.4 4.1.1.5 4.1.1.6 Comprehension and the decision-making process ..................................................................................... 17 Properties of the comprehension system .................................................................................................... 18 Building blocks of comprehension............................................................................................................. 20 The comprehension process ....................................................................................................................... 21 Properties revisited..................................................................................................................................... 24 Predictions ................................................................................................................................................. 25
4.1.2 What are the proper dimensions? ........................................................................................................26 4.1.3 Are the emotions categories or modal spaces?....................................................................................26 4.1.4 Summary ..............................................................................................................................................27 4.2 POST-APPRAISAL CHANGES DESIGN CHOICES.............................................................................................27 4.2.1 How long does an emotion last and how is its intensity calculated? ...................................................27 4.2.2 What does an agent feel? .....................................................................................................................28 4.2.3 What cognitive changes should be modeled? ......................................................................................29
4.2.3.1 4.2.3.2 4.2.3.3 4.2.3.4 4.2.3.5 4.2.3.6 4.2.3.7 4.2.3.8 4.2.3.9 Mood state dependent retrieval and mood congruent retrieval................................................................... 29 Categorization effects ................................................................................................................................ 30 Broaden and build ...................................................................................................................................... 31 Episodic memory ....................................................................................................................................... 31 Undoing ..................................................................................................................................................... 32 Priming effects ........................................................................................................................................... 32 Judgment effects ........................................................................................................................................ 33 Higher-level phenomena ............................................................................................................................ 33 Summarizing cognitive changes ................................................................................................................ 34
4.2.4 How do action and thought urges fit in?..............................................................................................36 4.2.5 Summary ..............................................................................................................................................37 4.3 RESPONSES TO EMOTION DESIGN CHOICES ................................................................................................37 4.3.1 How does an agent respond to its feelings?.........................................................................................38 4.3.2 What responses should be incorporated? ............................................................................................39 4.3.3 Summary ..............................................................................................................................................40 5 EXPERIMENTAL APPROACH....................................................................................................................41 5.1 5.2 PRIOR EVALUATIONS .................................................................................................................................41 DESIGNING NEW EVALUATIONS .................................................................................................................42
5.2.1 Accelerating reinforcement learning ...................................................................................................42 5.2.2 Improving metacognitive strategies .....................................................................................................43 5.3 CHECKING PREDICTIONS ...........................................................................................................................44 5.4 SUMMARY .................................................................................................................................................44 6 7 PLAN.................................................................................................................................................................46 REFERENCES.................................................................................................................................................48
1 Introduction
1.1 Why cognitive architectures need emotion
Cognitive architectures promise to one day explain how human cognition arises from a set of basic architectural mechanisms. Informally, we all know that emotion plays a major role in our lives, often influencing our behavior and our thoughts. More formally, psychologists have documented several phenomena that show such changes. If cognitive architectures ever hope to explain all of human behavior, then they need to include a theory of how emotion integrates with the architecture. Furthermore, cognitive architectures must support the processes that lead to emotion in the first place.
1.3 What good are emotions? (how emotions may benefit AI)
Old philosophy and modern common sense dictates that emotions are a distraction from logical decision making and, in short, we would be better off without them. Certainly there are times when our judgment seems clouded by our emotions, resulting in counterproductive behavior. However, case studies involving individuals who have lost the ability to experience emotions due to brain damage indicate that emotions have positive roles to play as well. These individuals have no difficulty in most other areas, including speech, motor, and memory. However, they seem to suffer from an inability to plan, learn, and understand the consequences of their actions (Damasio 1994). In other words, they are not capable of leading normal lives. The implication is that emotion plays an important functional role in generating good behavior. Furthermore, from an evolutionary perspective, emotions help decouple sensory input from motor responses by allowing an agent to classify the world relative to its goals and then respond to that more general classification instead of the specific details of the situation (Smith & Lazarus 1990). That is, emotions are a mechanism that is able to identify the aspects of the situation that are most important for survival and learning emotions are evolutions lessons 3
about what is important. Conversely, emotions can also result in useful automatic responses, including motivation, attention shifting, and physiological preparation for action. Finally, emotion may play an important role in metacognition (Flavell 1979, 1987). Simply put, metacognition is thinking about thinking. For my purposes, metacognition is the knowledge, mechanisms and strategies used to regulate cognition (e.g. select appropriate cognitive strategies) in order to generate intelligent behavior. This includes planning, learning, and goal management (e.g. monitoring progress towards goals and even giving up when appropriate). One aspect of metacognition is called metacognitive experience (Flavell 1979, 1987). Metacognitive experience is defined as experiences that are cognitive and affective (Flavell 1987). As we will see, emotions (or more accurately, feelings) fit this definition of metacognitive experience. Emotions effectively summarize the relationship between a situation and ones goals (see section 3.1), which is critical for assessing the progress one is making towards those goals. Furthermore, when someone copes with his emotions, he is changing his cognitive activities in response to this evaluation. Thus, some forms of coping are examples of metacognitive strategies (i.e. strategies used to change ones cognitive processing; see section 3.3). From an AI perspective, if we can identify how emotions help humans, then we may be able to design computational systems which are also able to benefit from similar capabilities. It is even possible that we will discover that some existing computational systems already have features which we would identify as a kind of emotion.
agents emotions. Furthermore, emotion can impact decision making behind the scenes by manipulating how the architecture works.
1.5 Overview
A long-term research program would investigate the integration of physiological, cognitive and sociocultural aspects of emotion and how they impact behavior. Given the need for a narrower focus and my experience with Soar, a cognitive architecture, I will not investigate the physiological or sociocultural aspects of emotion for this thesis. On the cognitive side, as discussed in section 4.2.3, it turns out that most of the knobs necessary to integrate emotional changes at the architectural level do not yet exist (although many are under active development by others). Thus, my research will focus primarily on the cognitive antecedents of emotion (section 4.1) and the cognitive responses to emotion (section 4.3). This will take the form of a domain independent comprehension system that allows an agent to understand the situation with respect to its goals (the basis of the appraisal theory of emotion, section 3.1). In general, evaluation of emotion systems is very difficult (see section 5.1). However, by providing emotional state to the reinforcement learning system (Nason & Laird 2004), I should be able to demonstrate that an emotional agent can learn faster than a non-emotional agent (section 5.2.1). I may also be able to demonstrate improved support for metacognitive strategies such as goal management via the comprehension system that supports the emotion system (section 5.2.2). In conclusion, my research will create a general comprehension system that supports the induction of emotion and then utilizes the resulting feelings to help choose actions. Furthermore, I will answer the question, does emotion (as realized in this system) help improve learning and task performance? This research will also provide a concrete framework for future research involving cognitive mechanism integration, physiological integration, and sociocultural integration. In the remainder of this paper, Section 2 will describe what is meant by cognitive architecture. Section 3 will describe the framework for my emotion research. Section 4 will describe the kinds of choices that must be made in each part of the framework, including the ones I intend to explore. Section 5 will discuss a possible experimental approach. Section 6 will summarize the intended research program.
2.1 Operators
Behavior is generated by repeated selection and application of operators. Operators are the primitive actions that an agent can take in its thinking process. Some operators may merely be internal, mental actions; others may be realized as motor commands that affect the world. An operator is composed of two parts: proposals and applications. A proposal describes the conditions under which the operator may be chosen. An operator may have many proposals, allowing for disjunction. An application describes what to do once an operator has been chosen. An operator may have many applications, each doing different things and applying under different conditions. Operator selection is mediated by knowledge that assigns a numeric preference to each operator. The numeric preference represents how much that operator is worth under those conditions that is, how much future reward the agent expects to get by choosing that operator.
2.2 Memories
There are two broad categories of memories long-term memories and short-term memories. There are at least three kinds of long-term memories: procedural, episodic, and semantic. Procedural memory contains knowledge about what the available operators are, when to perform them, and how to perform them, as well as simple entailments about the situation. It is fast, requires an exact match and allows for parallel retrievals. Episodic memory contains recordings of previous situations that an agent has been in. It is probably slower, allows for partial matches, and only allows for serial retrievals. Semantic memory contains facts that the agent knows, but which are not tied to a particular previous situation. It is also slow, allows for partial matches, and only allows for serial retrievals. During episodic and semantic retrievals, what gets retrieved is determined by the goodness of the match, which may be biased by things like activation (of both cue and memory), noise, and thresholds. All of these long-term memories are sources of knowledge that the agent can use to help choose and apply operators. Short-term memory is the place where knowledge from the various long-term memories and other inputs, such as perception, are brought together so they can be reasoned about together for the purpose of choosing an operator. For my purposes, I will not differentiate between the different kinds of short-term memories.
2.3 Mechanisms
A cognitive architecture contains several mechanisms that allow the various memories to work together and provide a means for choosing and applying operators. In addition to the retrieval mechanisms described in section 2.2, there are learning mechanisms that create new long-term memories. Those will not be important for this research.
At any particular time, there may be multiple operators that can apply in a given situation. In this case, the decision procedure takes the available knowledge into account in order to choose the best one. Sometimes it may choose one of the operators randomly, weighted by its numeric preference. In that case, the agent may get some reward feedback which allows it to adjust the numeric preferences of the operators via reinforcement learning to improve their accuracy. Figure 1 shows how these various parts fit together.
Episodic
al ur d e oc Pr
Long-term Memories
En c Re odi tri ng ev & al
Se ma nt ic
nt me ce o r in g inf rn R e Lea
Body
Body
Shortterm Memory
Proposal & Selection
Decision Procedure
Figure 1: A basic cognitive architecture. The ellipse shows central cognition as separate from but still part of the body.
I will describe each step of the process including some of the alternative approaches that might be taken.
Appraisal Objective
Relevance Detection
Outcome probability Discrepancy from Implication Assessment expectation Goal/need conduciveness Urgency Control Coping Potential Determination Power Adjustment Internal standards Normative Significance External standards Evaluation
Main ideas Suddenness, familiarity, predictability. Pleasantness of object itself, independent of current goals. May be acquired. Does stimulus result in outcomes that impact major goals? Who or what is responsible? What were the intentions? How likely are certain consequences? Degree can be determined by the number of features that fit the original expectation. How conducive (or not) is the event to helping me attain my goals? Are high-priority goals endangered? Will waiting make things worse? To what extent can an event be controlled by natural agents? To what extent can an event be controlled by me (directly or via influence on others)? If I fail to change the event, to what extent can I live with the consequences? To what extent is my behavior in line with my self ideal and moral code? To what extent is my behavior in line with social norms?
Appraisal theory also specifies a mapping from appraisal values to emotional states. For example, a theory might say that if transaction is goal relevant, non-conducive, caused by someone else, and the agent has to power to do something about it, then anger may result. If the agent is powerless to do anything, then fear may result. A more complete example is shown in Table 2 (Scherer 2001). The resulting emotion is an important metacognitive experience that summarizes how things are going for the agent. This information can be used by other processes (e.g. coping; see section 3.3) to help generate intelligent behavior.
Enjoyment/ Happiness Relevance Novelty Suddenness Familiarity Predictability Intrinsic pleasantness Goal/need relevance Implication Cause: agent Cause: motive intentional low medium high medium
Elation/ Joy
Displeasure/ Disgust
Contempt/ Scorn
Sadness/ Dejection
Despair
Anxiety/ Worry
low low
high very low low high other/ nature chance/ negligence very high
low
low other
high
Outcome probability very high very high high medium Discrepancy from consonant dissonant expectation Conduciveness high very high obstruct obstruct obstruct Urgency very low low medium low low high medium Coping potential Control high very low very low Power low very low very low low Adjustment high medium high medium very low medium Normative significance Internal standards very low compatibility External standards very low compatibility Table 2: An example of a partial mapping from appraisals to emotions (from Scherer 2001). Blank entries indicate appraisals that play either no role or an indeterminate role for that emotion.
intentional
10
Appraisal
Em oti on
Agent-Environment Transaction
Emotion-Induced Changes
Responses to Emotion
Figure 3: Appraisal mediates between the agent-environment transaction and the various changes in the emotion-induced changes.
Figure 3 shows how appraisal and emotion fit into the appraisal process.
It may be that some changes are not mediated by emotion and are actually linked to specific appraisals (Smith & Kirby 2001). These kinds of changes provide insight into how cognition and the body are linked, but are beyond the scope of this research.
11
Figure 4 shows the emotion process with the emotion-induced changes I will discuss in this paper.
Appraisal
Em oti on
Agent-Environment Transaction
Responses to Emotion
Figure 4: Some of the changes caused by emotion include thought-action urges, cognitive changes, and subjective feelings.
3.3.1 Coping
Coping is what an agent does in order to improve or maintain his feelings. An agent can not really deal with his emotions directly since all it knows about them is what it can perceive (i.e. his feelings). There are two classes of coping: problem-focused and emotion-focused. Problem-focused coping is taking actions in the world to change the environment part of the agent-environment transaction. For example, if something in the environment is causing fear, the agent might run away so its no longer being exposed to that stimulus. If the agent is angry,
12
it might take actions that fix whatever the problem is. If the agent is happy, it might take actions that protect the source of its happiness. Emotion-focused coping is taking internal actions to change the agent part of the agentenvironment transaction. For example, if an agent determines its goals are unachievable, then it might give up on them, thus relieving the negative emotions caused by its repeated failures. Emotion-focused coping is a metacognitive strategy because the agent is changing the way it thinks in order to help it make progress in the world (even if that progress is now towards new goals). Gratch & Marsella (2004) list several types of coping which I have reproduced in Table 3. Coping in general provides a way to manage ones goals, be it forming new subgoals to get back on track (problem-focused coping) or giving up on some goals (emotion-focused coping). Problemfocused Coping Active coping: taking active steps to try to remove or circumvent the stressor. Planning: thinking about how to cope. Coming up with action strategies. Seeking social support for instrumental reasons: seeking advice, assistance, or information. Suppression of competing activities: put other projects aside or let them slide. Restraint coping: waiting till the appropriate opportunity. Holding back. Seeking social support for emotional reasons: getting moral support, sympathy, or understanding. Positive reinterpretation & growth: look for silver lining; try to grow as a person as a result. Acceptance: accept stressor as real. Learn to live with it. Turning to religion: pray, put trust in god (assume God has a plan). Focus on and vent: can be function to accommodate loss and move forward. Denial: denying the reality of event. Behavioral disengagement: Admit I cannot deal. Reduce effort. Mental disengagement: Use other activities to take mind off problem: daydreaming, sleeping. Alcohol/drug disengagement.
Emotionfocused Coping
13
Response-focused emotion regulation encompasses response modulation, which is actively trying to change your post-appraisal responses (e.g. change ones facial expression, suppress ones anger). Since response-focused emotion regulation seems to require the ability to manipulate physiology (which I am not including) or lower-level cognitive functions (most of which are not available to me; see section 4.2.3), I do not intend to include response-focused emotion regulation in my research. Clearly there is a large overlap between coping and emotion regulation. In fact, one can argue that antecedent-focused emotion regulation and coping are pretty much the same thing. After all, coping can only be successful if there is some (possibly implicit) prediction that the coping effort will somehow improve or maintain the agents emotional state. The antecedent-response difference can be resolved by noting that the emotion process is a cycle, and thus what can be interpreted as a response to one emotion may also be interpreted as a prediction of the next emotion. Thus, the primary difference between coping and antecedent-focused emotion regulation is the set of dimensions used to describe them. It may be possible to transform the antecedent-focused strategies into coping strategies. For example, suppose a woman decides not to go to a particular bar because she is afraid that her ex-boyfriend might be there, which could lead to an angry confrontation. The antecedent-focused view would describe her decision as an attempt to prevent anger. The coping view would describe her decision as a response to her fear. For my research, I will probably focus on responding to emotions and not consider explicit emotion anticipation. That is, I intend to follow the coping model more than the emotion regulation model. Figure 5 shows how responses to emotion can impact the rest of the emotion process, transforming it into a cycle.
14
Appraisal
Em oti on
Co pin g
Responses to Emotion
Figure 5: Responses to emotion include changing the agent-environment transaction and attempting to directly manipulate the emotion-induced changes.
Table 4 summarizes the parts of the emotion process that I will explore. Section 4 contains a more detailed explanation of what I have explored and what I will explore. Emotion process component Agent-environment transaction Appraisal Emotion Post-appraisal changes Responses to emotion Comments This is part of the research in the sense that it needs to be represented in order to be appraised. I will explore what appraisals to use and how to generate them. I will explore which emotions to use and how to represent them. I will explore those elements listed in the emotion process figures. I will not explore physiology. I will explore this from the perspective of coping, not regulation.
Re gu la tio n
Agent-Environment Transaction
15
Personality is a property that can impact each stage of the emotion process. For example, it may impact the appraisals that are generated, and the coping strategies used. There are at least two major sources for personality differences differences in knowledge, and differences in architectural parameters. Differences in knowledge can come from a variety of sources. A culture can influence what different genders and groups consider appropriate behavior, leading to different emotion profiles. For example, women are more likely to express sadness whereas men are more likely to express anger (Citrin et al. 2005). Furthermore, Gross & John (2003) show that minorities in the U.S. are more likely to engage in suppression than European Americans. Differences in architectural parameters may come in the form of subtle differences in working memory capacity or long-term memory retrieval times or noise. They can also take the form of physiological differences for example, there may be differences in heart rate and skin conductance changes. Berenbaum (2002), for example, identified links between personality traits and pleasure reactions. Finally, Feldman Barrett & Gross (2001) describe a concept called emotional intelligence which brings some of these differences together. Emotional intelligence is how good people are at perceiving their current emotions and effectively regulating them. While they stop short of providing a quantitative measure like IQ, it is safe to say that people differ in their emotional intelligence, which in turn impacts their emotional states over time.
16
17
2) 3) 4) 5) 6) 7) 8)
Encode: Interpreting the raw inputs in a domain-independent way. Attend: Attending to an input. Comprehend: Understanding the input in the context of what the agent is doing. Tasking: Performing maintenance on ones goals. Intend: Choosing a course of action. Decode: Translating the action choice into motor commands. Motor: Executing motor commands.
Perception and motor are considered to be outside of central cognition, and encode and decode are on the boundary. This sequence is mostly fixed because of the constraints between the steps (e.g. one cant comprehend without first attending, etc). There are some possible exceptions. For example, an agent may be able to Intend directly from some raw Perception. Tasking may also move around; for example, it may not be necessary to update ones goals after every cycle, or it may make sense to perform goal maintenance before comprehension has taken place. In my implementation, Perceive is controlled by an external system which manages the agents perceptual buffer (i.e. central cognitions access to perception). Encoding is accomplished by a set of parallel elaboration rule firings which annotate input with additional information. Attend is an operator which chooses one of currently encoded inputs to process next. Comprehend is what the agent does with the encoded input; I will go into detail below. Tasking is anything the agent does to maintain its goals. For example, the system has an operator which generates a new declarative goal if the agent doesnt currently have one. Intending is an operator the agent uses to decide what to do in the world. Decoding will simply be the process of sending actual motor commands to the motor processor, which is an external system.
18
2) Limited working memory: The agent should not require an arbitrary amount of working memory. NL-Soars approach to this (and mine as well) is to only represent one interpretation at a time. 3) Incremental: The agent should not have to wait until all inputs necessary for a complete understanding are available. It should be able to build up its comprehension from smaller pieces. 4) Happens over time: Situations unfold over time, so the agent should be able to construct and refine its comprehension over time. 5) Supports immediate comprehension: The agent should be able to create some understanding of the situation from the very first moments of the situation. If the agent waits it is wasting time that it could be using to do some processing, no matter how speculative. Furthermore, the agent may want to respond to the situation immediately. Finally, unlike language, which contains markers that indicate logical stopping points (e.g. periods, commas, pauses in speech, etc.) to try to comprehend the words, situations do not. The world unfolds as an endless stream of events. 6) Supports hierarchical comprehension: Comprehending a series of events requires not only understanding the individual events and the connections between individual events, but also how they fit together at more abstracts levels. Thus, it is not just that an agent is getting into the car, starting the engine, driving down the road, etc. Rather, it is that this agent is driving to work, which can provide context for future comprehension and prediction. 7) Supports prediction: A true understanding of the situation includes a prediction of what is going to happen next. Prediction aids future comprehension for familiar situations because it allows the agent to confirm if an event is consistent with the current prediction instead of attempting to process the event in isolation. Prediction also allows the agent to prepare for future events. The property of immediate comprehension leads the agent to require at least two supporting mechanisms: 8) Immediate ambiguity resolution: Many events will be ambiguous when encountered in isolation. One approach is to defer commitment until a unique interpretation is clear; however, this can result in a combinatorial blowup in processing and memory usage as multiple ambiguous events are encountered. An alternative is for the agent to commit immediately to a particular interpretation (i.e. the best one), which avoids the combinatorial blowup and supports immediate comprehension. 9) Error recovery: Since the correct interpretation is ambiguous, the agent may choose the wrong one. Thus, the agent must have the capability to recover when it discovers it has made an incorrect assumption. As in NL-Soar, I expect that recovery will usually involve a local repair to its interpretation, but can also require a complete reinterpretation in garden path situations. The comprehension process is essentially its own weak method. That is, it provides a universal framework for solving problems. Soar itself provides a lower-level framework for weak methods (Laird & Newell 1983), but comprehension provides additional useful structure.
19
20
To get an idea of how event comprehension works, suppose I have a schema for the crossing the street event: Schema: Crossing the street Event 1: Step down from curb Event 2: Walk across street Event 3: Step up onto curb As you can see, the abstract event crossing the street is defined in terms of more concrete events. These, in turn, may be broken down further. At some point, however, an abstract event will be composed of ground events which are the most basic units. These ground events are the events that the agent recognizes when it encodes raw inputs. They also correspond to actions that the agent can directly execute in the world. For simplicity, I have not shown the explicit actors and actions, which would typically be variablized, in this example. Now suppose the agent observes someone stepping down from a curb. It can now recognize that this is the first step of the crossing the street schema. If it commits to this interpretation, it can make predictions about what will happen next (e.g. that the person will walk across the street and step up onto the other curb). It can also infer something about the other agents goals (e.g. that it wants to cross the street). Perhaps most importantly, the agent can infer what the meaning of this event is with respect to its goals is it ok for me if this person crosses the street? When the next event occurs, this may confirm the predictions or the agent may have to reinterpret using some other event schema. The initial choice of event schema may be impacted by several factors; for example, if the agent wants the person to cross the street, then it may be biased to interpret the initial event that way. As alluded above, the agents goal can also be represented by an event schema. That is, the goal schema is just whatever event the agent wants to occur. Each subevent of the goal schema is really a subgoal, then. That is, the agent can represent a goal hierarchy using event schemas. These declarative goals should not be confused with Soars architectural goal stack. Soars architectural goal stack is used to implement the comprehension process, but the agents task goals (the ones that it actually cares about, in some sense) are represented declaratively as event schemas.
21
different variable bindings), then the event may need to be swapped in for the prediction, and the future predictions updated to reflect the latest information. Finally, if the event is completely different than what was expected, the agent may need to discard the current interpretation and replace it with a reinterpretation that is consistent with its observations. This is the first example of how some appraisals may fall out of the agents processing. The determine discrepancy from expectation operator is an appraisal from Scherers (2001) theory, but similar ones exist in other appraisal theories. One of the design decisions I made with the comprehension system is to require the agent to always have a prediction of what it thinks will happen next. As argued earlier, an agent that does not have some prediction of the future does not really understand the situation. Furthermore, many appraisals rely on having some prediction (e.g. outcome probability, discrepancy from expectation, etc.) Realistically, there may be some situations in which an agent cannot make a reasonable prediction (possibly resulting in confusion), but as a simplification I will not consider that possibility in my research. Thus, when the agent reaches the end of its current schema (e.g. if the person finishes crossing the street), the agent needs to determine what the next schema should be. One way of doing this is to determine what abstract schema the crossing the street event is part of. For example, given the particular person and street, the agent might suppose that the larger event that is taking place (i.e. the goal of the person) is to go to work. The next event in the go to work schema may be to enter the building, which in turn requires walking up to the building and opening the door. Thus the agent might predict that the next event that will take place is walking up to the building. As this demonstrates, predicting the next event requires a series of abstractions and specializations. Figure 6 shows the comprehension aspects of the PEACTIDM process, and Figure 7 shows how event comprehension fits into the emotion process. As depicted in Figure 6, once the agent has comprehended the event, it can generate another appraisal, goal/need conduciveness. This appraisal is different in that it does not fall out of the comprehension process, but rather follows it. Since it isnt required by the agents processing, the agent may only generate it if it has enough resources to do so (e.g. time). Even if there is another event waiting to be processed, the agent could decide to delay or skip that in favor of doing this appraisal. Perhaps reinforcement learning could play a role in helping to determine the best choice to make. In general, appraisals can be divided into at least 3 groups: 1) automatic appraisals (e.g. novelty may be computed automatically by the high-level vision or long-term memory systems), 2) deliberate but required appraisals (i.e. those appraisals which fall out of the comprehension process), and 3) deliberate but optional appraisals (i.e. those which the agent may do if it has time). This is not to imply that the agent can choose not to do optional appraisals. Rather, continued comprehension may take precedence over these appraisals, so they may be skipped if events are occurring rapidly. Table 5 shows some possible appraisals and what types they might have. These particular appraisals are merely exemplars. Rather than take a firm stance on the exact set of appraisals at this point in my research, I view the comprehension process as providing constraints on what the set of appraisals should include. That is, some appraisals may fit more
22
naturally than others into the comprehension process. Of course, the constraint goes both ways in that the comprehension process must allow for many common appraisals. Novelty and intrinsic pleasantness are likely properties that arise during the encoding process, which is mostly automatic. The agent may get causality automatically when it chooses a schema. That is, the schema itself may include the causality information in its structure. That is not to say that determining causality in general is not a complex process that is not automatic; presumably the agent must sometimes go through that process when it is generating a new schema to understand a novel set of events. The distinction between required and optional deliberate appraisals may lie primarily in what is required to understand what is happening in order to immediately act vs. what is required to understand the relationship of what is happening to the agents goals. This distinction is slightly muddied because the agents interpretation is colored by its goals. Thus, I consider goal/need relevance to be required; it may occur as part of the attend process (i.e. the agent may ignore events irrelevant to its goals). Outcome probability is probably also required, although I do not yet know how it fits into the comprehension process. Goal/need conduciveness and coping potential, on the other hand, do not seem to be critical to the immediate understanding of what is happening, but clearly it is in the agents interest to do these if it can.
Comprehend Event Determine Goal/Need Conduciveness
Encode
Attend
Perceive
Update Schema
Reinterpret events
Abstract Schema Figure 6: The comprehension process so far, based on PEACTIDM. Left to right is roughly the operator sequence, and top to bottom shows how some of the steps may break down into architectural subgoals. Double-lined boxes depict operators expected to require architectural subgoals.
23
pr eh en si on
Appraisal
Em oti on
C om
Co pin g
Responses to Emotion
Deliberate-required
Deliberate-optional
Table 5: Possible types for common appraisals.
Re gu la tio n
Agent-Environment Transaction
Appraisal Novelty Intrinsic pleasantness Causality Goal/Need relevance Discrepancy from expectation Outcome probability Goal/need conduciveness Coping potential
24
3) 4)
5) 6) 7)
8) 9)
it destructively abstracts and specializes the current interpretation. This could lead to interesting phenomena in which an agent loses track of what exactly it is doing, but in general I expect the activations of the right things to be high, leading to a low error rate. Incremental: The system can attend to and process one event at a time. Happens over time: Since events occur over time, the agent processes them over time. The delay between events is actually necessary in order for the agent to complete its processing without becoming overwhelmed. The agent may even be able to utilize extra time it has to generate additional appraisals. Immediate comprehension: The agent commits to a current interpretation schema starting with the first event. Supports hierarchical comprehension: The event schemas describe events at multiple levels, directly supporting hierarchical comprehension. Supports prediction: The hierarchical nature of comprehension and event schemas directly support prediction. Once a high-level schema has been recognized, the subsequent events that it contains are predictions. Immediate ambiguity resolution: If multiple interpretations are available, the agent will pick one that best fits its goal, was used most recently, or randomly. Error recovery: There are at least two levels of error recovery: a simple level, in which the schema structures are correct but the variable bindings need to be updated, and a complex level in which the structure itself is incorrect and the agent needs to choose a new schema. The simple level may be implemented via a truth maintenance system that automatically updates the variable bindings used in the next predicted event. The complex level may utilize arbitrary processing to find a schema that fits the whole sequence. For example, the agent can recall the most recent event sequence from episodic memory and try to find a schema that matches that set. I do not intend to explore this complex level if possible. I also have not yet explored to what degree intermediate levels of error recovery are possible.
4.1.1.6 Predictions
From this theory I can derive a number of predictions. First, because the comprehension process results in immediate comprehension, if the agents perception of a stream of events is interrupted, the agent should have an understanding of on the situation up to that point (which will often include predictions as to what might happen next and what other agents are trying to do). Thus, an interrupted agent will behave based on the information received so far, but that behavior may be flawed in predictable ways due to mistakes in the agents interpretation (similar to garden path phenomena in language processing). Furthermore, the comprehension process imposes a partial ordering constraint on appraisal generation. For example, as shown in Figure 6, the Discrepancy from Expectation appraisal occurs before the Goal/Need Conduciveness appraisal. This is consistent with Scherers (2001) theory which also hypothesizes a sequential ordering. However, the reasons for the hypotheses differ. Scherers reasoning is that it would be a waste of processing resources to do some appraisals when the results of others show that they are irrelevant. Thus, for example, Discrepancy from Expectation comes after Goal Relevance, because if an event is not
25
relevant, then the agent should just ignore it. By contrast, our theory imposes an ordering because of the functional requirements of the comprehension process. A corollary of this is that our model allows for the possibility that different emotions will inherently require different amounts of processing. If some emotions only require appraisals that occur earlier in the comprehension process, then those emotions will take less processing than those that require more appraisals. The idea that different emotions may require different amounts of cognitive activity is not actually new (Lazarus 1982). This may lead to timing effects. Given that comprehension takes time, if there are tight time constraints, some appraisals may not get generated, meaning that under certain time pressures, some emotions may not occur, or may be based on appraisals that were generated in earlier situations. Under extreme time constraints, comprehension itself may not be possible, leading to purely reactive behavior. In between, it may be possible that an agent misses some of the events, leading to flawed interpretations. Finally, our model supports appraisals that can happen at different time scales. Some appraisals may be based on comprehension in novel circumstances that require multiple retrievals from long-term memory, or even significant internal problem solving to understand the situation, while others could be based on comprehension in well practiced situations where essentially reactive comprehension is possible. These differences in comprehension processing can lead to very different time scales for generating appraisals. Combining this with the previous two points, I predict that the complete appraisal (and thus the emotional reaction) can change over time as the comprehension of the situation evolves over time.
26
number of variations of anger, corresponding to each point within the space. Some regions in the space may not have any cultural label at all, and the edges between labeled regions may be fuzzy (i.e. how a person decides to label such a region may be inconsistent). I intend to represent emotions as n-dimensional points internally, but the agent may label its feelings and work primarily with those labels.
4.1.4 Summary
The appraisal theory design decisions are summarized in Table 6. Open Questions How are the appraisal values generated? What are the proper appraisal dimensions? Are the emotions categories or modal spaces? Further research? Yes Comments I will refine the situation comprehension process to support an appropriate range of deliberate and automatic appraisals. This will be strongly influenced by what fits well into the comprehension process (and the comprehension process will be strongly influenced by what it needs to fit in). I have decided to use an n-dimensional representation for the emotions.
Yes
No
The first two questions are actually related, so I will address them together.
4.2.1 How long does an emotion last and how is its intensity calculated?
When we think of temporally extended emotions, we are really entering the domain of mood. Most of the emotion literature does not carefully distinguish between emotions and moods because concrete definitions for either have not been settled on (and may not be possible (Smith & Lazarus 1990)). However, Rosenberg (1998) identifies some useful properties; primarily that mood has a longer temporal duration than emotion and that mood and emotion should influence each other.
27
One possible approach is to represent mood as a baseline in the n-dimensional emotion space to which emotion decays once its generating appraisals have disappeared. Emotion, in turn, pushes the baseline in the direction of the emotion. Mood probably also decays towards some neutral point, but much more slowly than emotion. It also makes sense that the intensity of a new emotion would be enhanced or subdued by the current mood depending on whether the mood was congruent or not with the emotion. This approach makes some assumptions and raises some difficult questions. In particular, this assumes that moods and emotions are of the same kind, which implies that there is a corresponding mood for every emotion. There is also the question of how discrete dimensions decay to baseline how does causality decay from, for example, you to me? Finally, there is evidence that the baseline mood for humans is slightly positive (Cacioppo & Gardner 1999) but is it happy, elated, interested, or something else? Previous work by Gratch & Marsella (2004) ignored the notion of mood but still maintained interactions between emotions by creating a matrix that described which emotions offset each other and to what degree. The intensity of a new emotion was combined with the previous emotion using this matrix to determine the final intensity of the new emotion. For example, if the previous emotion was mild anger and the new emotion was strong happiness, the final emotion might be moderate happiness. Given the complexity of exploring the available options, and the lack of guidance that the literature can currently provide, my approach is to keep it simple. Thus, my initial implementation will ignore moods and interactions between emotions in favor of the simplest model: the emotional state is a direct result of the appraisals. If I have time, I may explore one of these other approaches. Even with the simple approach, the question of how the intensity is generated is still unanswered. Gratch & Marsella had continuous appraisal dimensions corresponding to likelihood and desirability, which they simply multiplied together to get intensity. I will have more metric dimensions to work with, so the development of a function that combines the appropriate ones together into a single value will be necessary. Again, I intend to start with a simple model and only explore further as necessary.
28
These rules represent a combination of cultural knowledge and personal experience about how various feelings related to other changes in its past2.
I say represent because I do not intend to explore how the agent learns these rules.
29
represented in a crystallized form which is accessed quickly and automatically. In a cognitive architecture, this strategy maps onto using procedural memory. 2) Motivated processing strategy: Occurs when information processing is guided by a strong, pre-existing objective. Knowledge is found via a goal-directed search. In a cognitive architecture, this strategy maps onto using architectural subgoals. 3) Heuristic processing strategy: Occurs when subjects have neither a crystallized response nor a strong motivational goaland they lack either personal involvement or sufficient processing resources. Knowledge about emotions (affect-as-information) may be brought to bear. In a cognitive architecture, this maps onto having a representation of the current emotional state in working memory (i.e. a feeling) which can then be used by other mechanisms (e.g. procedural memory). 4) Substantive processing strategy: People need to select, learn, interpret and process information about a task, and relate this information to pre-existing knowledge structures using memory processes. This is a default strategy used when none of the others apply. In a cognitive architecture, this maps onto using slower, more error prone mechanisms like episodic and semantic memory. Forgas claims that each of these strategies is successively more effortful, and thus, for example, the substantive processing strategy will only be used if the earlier strategies fail. Forgas key claim is that only the substantive processing strategy is directly impacted by emotion. Thus, the reason some experiments were unable to reproduce the mood state dependent retrieval and mood congruent retrieval phenomena is because the nature of those experiments allowed other nonsubstantive processing strategies to be used. For example, if the processing required was too simplistic, then subjects would not have to use the substantive processing strategy, and thus the effects of emotion would not show through. This is an interesting result, because it means that we dont need to consider how emotion impacts rule firings or impasses; essentially, the traditional mechanisms in Soar are unaffected. For this reason, these and other cognitive phenomena discussed in this section are unlikely to fit into the traditional Soar architecture that is currently available to me and thus I will not be able to explore them. It seems that the cognitive functionality required to explain these retrieval phenomena is metacognitive knowledge that associates emotion (or some region in the n-dimensional emotion space) and knowledge that was acquired in the context of that emotion or is somehow related to that emotion. Given Forgass theory, the cognitive components most likely to be influenced by emotion are semantic and episodic memory.
30
show up in all studies with significance, although this may have been due to a flaw in the experiment design. The authors argue that these phenomena point to a change in processing induced by positive emotion, and possibly also negative emotion. Some possibilities include that emotion provides or induces extra features that can be used when cueing retrieval, which may make categories seem more inclusive (since all the items may seem to have a positive emotion feature, for example). Its unclear, however, how an emotion feature comes to be included with an item in, for example, semantic memory. This seems to require a deeper understanding of how humans do categorization and grouping, which is beyond the scope of my research.
31
were camping and I got lost in the woods. Given these definitions, it seems that specific ABMs map onto what I have called episodic memory, whereas general ABMs seem more like semantic memories about episodic memories. For example, I know that I went camping frequently as a child this is a fact, and not one that I learned at a specific time. Philippot & Schaefer claim that only general ABMs are affected by emotional state. They cite several studies. For example, Philippot & Dozier (1996) had subjects watch an anger-inducing film. The subjects then reported the ABMs that spontaneously occurred to them and the emotional intensity felt during the film. After the film, the subjects rated each ABM for its emotional intensity. Independent judges then classified the ABMs as specific or general. There was a positive correlation between the emotional intensity felt during the film and both the number of reported ABMs and the emotional intensity of the reported general ABMs, but there was no relationship with the specific ABMs. Following my inference that general ABMs are semantic memories and specific ABMs are episodic memory, these results show that episodic memory is not directly impacted by emotion, but semantic memory is. This fits in with some of the ideas about how broaden and build might integrate with semantic memory. This phenomenon, then, helps refine our understanding of how broaden and build might integrate with cognitive architectures in that it may eliminate episodic memorys mechanisms as candidates, leaving just semantic memory.
4.2.3.5 Undoing
Fredrickson & Levenson (1998) describe a phenomenon called undoing. Undoing is the idea that positive emotions undo some of the effects of negative emotions. For example, during fear ones heart rate may increase dramatically. A subsequent positive emotion will cause the heart rate to return to baseline more quickly than if the effect were just allowed to wear off on its own. Importantly, if someone is already at baseline, then inducing a positive emotion will have no effect. In terms of broaden and builds cognitive effects, undoing may reverse the narrowing effects of negative emotions and cause things like the exploration rate, retrieval thresholds and noise to return to their base levels. It seems that there needs to be metacognitive information that tells the architecture that it is ok to return to baseline. The reason the change is not instantaneous is because humans are biological systems that generally do not support instantaneous changes (at least at the physiological level). Since I am not exploring physiology or the broaden and build phenomenon, it does not make sense for me to explore this phenomenon.
32
with self-referent sentences were more likely to express guilt whereas those primed with otherreferent sentences were more likely to express anger. Neumanns goal in this study was to show that attribution of causality caused an emotional response. This is consistent with the appraisal theories I discussed earlier. What I find interesting, however, is the priming effect itself. This study shows that a primable mechanism is involved in generating appraisals. Given the restrictions on the mechanisms that can be involved (i.e. not procedural memory) and the nature of the priming task, it seems likely that semantic memory is used in some way to generate at least the causality appraisal (and likely others as well).
33
positive reaction (probably excitement). However, she then reinterpreted in light of this information she suddenly remembered, and that caused a change in her agent-environment transaction, which resulted in a negative emotion (probably dismay). The comprehension process described earlier doesnt explicitly account for this possibility because it only updates the current interpretation in response to new events. However, we can see that there is a temporal aspect to the interpretation of the information (or the retrieval of relevant information). Possibly there is a delay in the retrieval of the critical information that she will not be home tomorrow, or there may be multiple retrievals involved. In either case, comprehension proceeds with the information it has (as expected given its property of immediate comprehension). Thus, when the critical information finally is available, a reinterpretation is required, resulting in the realization. Unfortunately, Soar does not currently have any retrieval mechanisms that incorporate delayed retrieval. Even if multiple retrievals are used, it is unclear what the separate retrievals are. Given these difficulties, it is unlikely that I will be able to reproduce this phenomenon, but I hope to at least gain a better understanding of the required mechanisms, including how the comprehension process can support this. I do not currently have other high-level phenomena to report, but I hope to add to this list over time.
34
Categorization
Required Functionality/ Understanding Metacognitive knowledge that associates emotion and knowledge that was acquired in the context of that emotion or is somehow related to that emotion. Mechanism that allows emotion to be used as a cue for category information. A deeper understanding of how humans categorize. One-dimensional metacognitive information that affects what knowledge is brought to bear or what decisions are made. N/A Metacognitive information that tells the architecture it can return to baseline. Metacognitive information that describes how recently information has been used. A deeper understanding of how humans judge things. Temporal aspects to information retrieval or interpretation.
Episodic memory is not directly affected by emotion. Same as broaden and build.
Priming
Semantic memory can be primed. Unknown. Semantic or episodic retrieval delays or multiple retrievals.
Figure 8 shows how central cognition with emotion might look. Emotion is represented alongside short-term memory because it is a non-symbolic analogue to short-term memorys symbolic representation of the situation.
35
Episodic
is Ac e, T h t Ap ivat res pr ion hol ais & d, al
al ur d e oc Pr
Long-term Memories
Se ma nt ic
nt me ce o r in g inf rn R e Lea
No
Body
Body
Shortterm Memory
Proposal & Selection
Appraisal
Emotion
Confidence
Decision Procedure
36
linked to the intensity of the feeling that was tested as part of the operators proposal. Following this line of thinking, activation would need to be integrated with operator values. Research versions of Soar are just beginning to incorporate working memory activation; however, given the infancy of its usage and the lack of clear functional gains for my work, I do not intend to explore this more sophisticated notion of thought-action urges. However, Soar by its nature supports thought-action repertoires, and thus the simple sense of thought-action urges will (indeed must) be included.
4.2.5 Summary
The post-appraisal design decisions are summarized in Table 8. Open Questions How long does an emotion last and how is its intensity calculated? What does an agent feel? What cognitive changes should be modeled? Further research? Yes Comments The intensity aspect will probably be based on some of the appraisals, whereas how long the emotion lasts will probably be finessed for now (i.e. it will last until the next one, or some fixed number amount of time). I will give the agent the n-dimensional emotion point. Changes at the cognitive level seem to require mechanisms that Soar does not yet possess and a deeper understanding of various human strategies that are outside the scope of my research. Thus, I cannot model these phenomena. Soar already supports the simple notion of action and thought urges, but does not currently support the mechanisms that would probably be required for a more sophisticated view.
No No
No
37
I do not intend to explore possible individual differences in how feelings are labeled.
38
Figure 9 depicts the feeling comprehension process. Figure 10 shows how feeling comprehension fits into the emotion process.
Encode Attend Comprehend Feeling Emotion-focused Perceive Contextualize Feeling Generate New Goal Reinterpret Events Cope
Figure 9: The feeling comprehension process so far, based on PEACTIDM. Left to right is roughly the operator sequence, and top to bottom shows how some of the steps may break down into architectural subgoals. Double-lined boxes depict operators expected to require architectural subgoals.
pr eh en si on
Appraisal
Em oti on
C om
Re gu la tio n
Agent-Environment Transaction
Figure 10: Responses to emotion result from comprehending one's current feelings.
C om
Co pin g
Responses to Emotion
pr eh en si on
39
On the other hand, Intending is a way of changing the environment part of the agent-environment transaction that is, it is a form of problem-focused coping. Generating new subgoals supports making these changes. The remainder of the response strategies I described in section 3.3 deal primarily with anticipation and physiology. I do not intend to explore these areas. I also do not intend to explore reinterpretation if possible.
4.3.3 Summary
The responses to emotion design decisions are summarized in Table 9. Open Questions Further Comments research? How does an Yes I will develop a feeling comprehension process that allows agent respond to an agent to understand its feelings in the context of its its feelings? interpretation of the situation. This will allow it to effectively cope with the situation by forming new goals and taking actions. What responses Yes Deciding which responses make sense to include will should be depend on the design feeling comprehension process (and incorporated? vice versa). The tasks that we design for evaluation may also suggest an appropriate set of responses.
Table 9: Summary of the responses to emotion design decisions.
40
5 Experimental approach
Evaluating computational emotion models is simultaneously necessary and difficult. An ideal evaluation would compare the comprehension process to human data at the cognitive level; unfortunately, that kind of evaluation is out of reach. Instead, I have a more conservative goal of answering the question, can emotion (as supported by comprehension) improve learning and task performance? In this section I will briefly review how other emotion models have been evaluated in order to set the stage for appropriate evaluation. Then I will describe what kinds of results I hope to achieve.
41
To explore cognitive dynamics, they propose placing the agent in situations described by the Stress and Coping Process Questionnaire (Perrez & Reicherts 1992). This questionnaire asks people how they would respond to various situations, including how they would cope; actual agent behavior can then be compared to human responses. There are a few issues with this approach. One issue is that the questionnaire relies on suitable social behavior, which in turn requires cultural knowledge. I do not intend to address cultural issues in this research, and without a suitable cultural theory, this may degenerate into programming the agent to generate the correct behavior. Furthermore, the time scale of many of the scenarios in the questionnaire is too long (hours, weeks) or unclear. Finally, respondents are merely asked to introspect about these situations, and thus their responses may not accurately reflect what a person actually in the situation would do; rather, they may merely be using their own folk psychology theories to guess at what they might do. This approach may still be worth investigating in the future, but does not seem well suited to my current research. Finally, to explore social impact, they identify a phenomenon called social referencing. Social referencing is when ones appraisal of an ambiguous situation is influenced by the appraisals of others. They propose forcing a human to make a decision in an ambiguous case and varying the decision preferences, expressed non-verbally, of other agents, to see if the human can be influenced. This kind of test actually seems to be a test of facial expression and body language systems more than a test of internal emotion systems. Since I am not exploring physiological or social interactions, this kind of test is not appropriate for my work.
42
reinforcement learning in Soar works as follows. Each operator begins with very general conditions for its proposal. As the agent tries out these different operators and gets reward feedback for them, it may discover that the reward varies widely for the operators. That is, there are multiple situations that result in different values for the same operator, so a single value is inadequate to summarize the expected reward for that operator. To fix this, Soar creates a new operator proposal that is a specialization of the earlier one. That is, it creates a new proposal that has additional conditions. It uses the activation of working memory elements to determine what the best condition is to add. It can now learn a separate value for the operator in the situation described by the new proposal. This process can repeat, with new proposals with new conditions getting created to define the space of possible state-operator values. The hypothesis is that emotion (or more accurately, feeling) summarizes multiple other conditions, so reinforcement learning can learn faster since it only needs to add the emotion as a new condition, instead of iteratively adding each of the features that the emotion summarizes. There is a discrepancy in the amount of processing that must be done between the control agent (which does not have emotion) and the emotional agent. The control agent can essentially test raw input (or close to it), whereas the emotional agent must go through a comprehension process in order to ultimately evoke an emotional response, which it can then use to make a decision. That is, the control agent may be able to propose an action more or less immediately after receiving input, whereas the emotional agent may need to do lots of additional processing. However, it is important to note that the time does not flow more quickly for the control agent, and new inputs will not be available every 50 milliseconds (the Soar decision cycle). Events themselves take time to play out; the exact rate has not been determined yet, but it is likely that the agent will have many cycles to utilize between inputs. That is, the world changes slowly relative to the rate at which the agent processes information. In fact, it makes evolutionary sense that the human brain would have evolved to be just fast enough to process the inputs as it needs to. Thus, I expect the control agent to make decisions quickly but also to then waste the remaining time, whereas the emotional agent will more fully utilize the time available to it in generating these useful high-level features. Thus, not only will the agent learn faster, it will also use its time more efficiently. This evaluation hinges on being able to use a version of Soar that includes the necessary RL and activation features. Supposedly a research version exists that may be suitable.
43
It should be possible to design a task in which an agent needs to engage in some of these metacognitive strategies. For example, an agent might generate new subgoals and need to give up on some goals in order to achieve some overall goal. A control agent will have some domainspecific knowledge that allows it to do these things, but the emotional agent will be able to build on its domain-independent knowledge; therefore, it should require less domain-specific knowledge. Furthermore, the rules required by the emotional agent may be simpler since it can test its feelings, whereas a non-emotional agent may have to test complex aspects of the state directly. It may be that, for a single task, the total knowledge required by the emotional agent will be larger than that required by the control agent, but as the agents are expanded to new tasks, the emotional agent should scale better.
5.4 Summary
Unfortunately, it is difficult to design these tasks in detail without first completing more design work on the system itself (so that I can better understand the constraints). Also, it is somewhat worrisome that the first evaluation relies on reinforcement learning, a system being developed by another graduate student. However, it is the most developed of the new Soar systems, and there are plans to make it available internally to the other graduate students. Thus, I expect it to be ready by the time I am ready to implement this evaluation. Additionally, the metacognitive strategies have not been worked out in detail, so it is possible that I will discover issues that make it unrealistic to pursue that test. Finally, since the implementation of the theory will likely have an impact on the theory itself, I expect the predictions of the theory to evolve over time (more likely grow it seems unlikely to me today that the current predictions will prove invalid). Given the difficulties in determining what is really feasible for evaluation of my research, I may need to follow up with the thesis committee in the future once things become clearer. It may
44
turn out that my research will only allow for one of these evaluations, or that I will come up with a new evaluation to try. In other words, determining appropriate evaluations is research in and of itself.
45
6 Plan
My research plan is essentially to fill in the various parts of the emotion process as depicted in Figure 10. Because of the current limitations of Soar, some parts of the emotion process must be finessed or skipped altogether; most notably, I will pursue the integration of emotion with architectural level components. However, I should be able to make significant headway in the remainder of the process, from situation comprehension to emotion response. Due to the breath of the research, it will necessarily be a thin slice across these various stages. I expect the situation comprehension to be the deepest, whereas the emotion generation and representation itself may be the thinnest. Once an end-to-end system exists (i.e. an agent that can go through the entire cycle repeatedly to achieve goals), I will evaluate it to find whether learning and behavior are actually enhanced. Table 10 summarizes the open questions and whether I intend to do further research to answer those questions. In some cases further research is not intended because it is not possible, and in other cases I have already settled on an answer.
46
Open Questions How are the appraisal values generated? What are the proper appraisal dimensions? Are the emotions categories or modal spaces? How long does an emotion last and how is its intensity calculated? What does an agent feel? What cognitive changes should be modeled? How do action and thought urges fit in? Responses to emotion How does an agent respond to its feelings?
Comments I will refine the situation comprehension process to support an appropriate range of deliberate and automatic appraisals. This will be strongly influenced by what fits well into the comprehension process (and the comprehension process will be strongly influenced by what needs to fit in it). I have decided to use an n-dimensional representation for the emotions. Regions within this space correspond to emotions. The intensity aspect will probably be based on some of the numeric appraisals, whereas how long the emotion lasts will probably be finessed for now (i.e. it will last until the next one, or some fixed number amount of time). I will give the agent the n-dimensional emotion point. Changes at the cognitive level seem to require mechanisms that Soar does not yet possess or more extensive research in other areas, and thus I will not model these phenomena. Soar already supports the simple notion of action and thought urges, but does not currently support the mechanisms that would probably be required for a more sophisticated view. I will develop a feeling comprehension process that allows an agent to understand its feelings in the context of its understanding of the situation. This will allow it to effectively cope with the situation by forming new goals and taking actions. Deciding which responses to include will depend on the design of the feeling comprehension process (and vice versa). The tasks that we design for evaluation may also suggest an appropriate set of responses. I will design at least one task that can be used to evaluate the agent and answer this question. Possible candidates currently include a reinforcement learning task and one in which the agent must use metacognitive strategies to achieve its primary goals.
Appraisal Theory
Yes
No
Yes
Post-appraisal
No No
No
Yes
What responses should be incorporated? Does emotion improve the agents learning and behavior?
Yes
In conclusion, the primary contribution of this thesis is twofold: one, I will establish a framework for future research, and two, I will demonstrate that the framework supports improved learning and behavior.
Evaluation
Yes
47
7 References
Bachorowski, J. (1999). Vocal expression and perception of emotion. Current Directions in Psychological Science, 8(2):53-57. Berenbaum, H. (2002). Varieties of joy-related pleasurable activities and feelings. Cognition and Emotion. 16(4), 473-494. Cacioppo, J. T. et al. (2000). The psychophysiology of emotion. In M. Lewis & J. M. HavilandJones (Eds.) Handbook of Emotions, (2nd ed., pp. 173-191). New York: Guilford. Cacioppo, J. T. & Gardner, W. L. (1999). Emotion. Annual Review of Psychology, 50:191-214. Citrin, L, B., Roberts, T. & Fredrickson, B.L. (2005). Objectification Theory and Emotions: A Feminist Psychological Perspective on Gendered Affect. In L. Z. Teidens & C. W. Leach (Eds.) The social life of emotions. New York: Cambridge University Press. Cosmides, L. & Tooby, J. (2000). Evolutionary psychology and the emotions. In M. Lewis & J. M. Haviland-Jones (Eds.) Handbook of Emotions, (2nd ed., pp. 91-115). New York: Guilford. Damasio, A. (1994). Descartes error: Emotion, reason, and the human brain. New York: Avon Books. Damasio, A. (2003). Looking for spinoza: Joy, sorrow, and the feeling brain. USA: Harcourt. Feldman Barrett, L. (2004). Feelings or words? Understanding the content in self-report ratings of experienced emotion. Journal of personality and social psychology, 87(2):266-281. Feldman Barrett, L., & Gross, J. J. (2001). Emotional intelligence: A process model of emotion representation and regulation. In T. J. Mayne & G. A. Bonanno (Eds.) Emotions: Current Issues and Future Directions (pp. 286-310). New York: Guilford. Flavell, J. H. (1979). Metacognition and cognitive monitoring. American Psychologist, 34(10):906-911. Flavell, J. H. (1987). Speculations about the nature and development of metacognition. In F. E. Weinert and R. N. Kluwel (Eds.) Metacognition, Motivation, and Understanding. (pp. 2129). Hillsdale, NJ: Lawrence Erlbaum Associates. Forgas, J. P. (1999). Network theories and beyond. In T. Dalgleish & M. Power (Eds.) Handbook of Cognition and Emotion. (pp. 591-611). Chichester, England: Wiley & Sons. Fredrickson, B. L. & Branigan, C. (2005). Positive emotions broaden the scope of attention and thought-action repertoires. Cognition and Emotion, 19(3):313-332. Fredrickson, B. L. & Levenson, R. W. (1998). Positive emotions speed recovery from cardiovascular sequelae of negative emotions. Cognition and Emotion, 12(2):191-220. Gratch, J. & Marsella, S. (2004). A Domain-independent framework for modeling emotion, Journal of Cognitive Systems Research, Volume 5, Issue 4, pp. 269-306. Gross, J.J., & John, O.P. (2003). Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. Journal of Personality and Social Psychology, 85, 348-362. Hudlicka, E. & Zacharias, G. (2004). Approaches for modeling individuals within organizational simulations. In R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters (Eds.) Proceedings of the 2004 Winter Simulation Conference. Isen, A. M. & Daubman, K. A. (1984). The influence of affect on categorization. Journal of Personality and Social Psychology, 47(6):1206-1217. Kahneman, D. & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4):237-251.
48
Keltner, D. & Ekman, P. (2000). Facial expression of emotion. In M. Lewis & J. M. HavilandJones (Eds.) Handbook of Emotions, (2nd ed., pp. 236-249). New York: Guilford. Laird, J. & Newell, A. (1983). A universal weak method: Summary of results. In IJCAI 83, (pp. 771773). Larsen, R. J. & Fredrickson, B. L. (1999). Measurement issues in emotion research. In D. Kaheneman, E. Diener, & N. Schwarz (Eds.) Well-being: Foundations of hedonic psychology (pp. 40-60). New York: Russell Sage. Lazarus, R. S. (1982). Thoughts on the relations between emotions and cognition. American Psychologist, 37, 1019-1024. [Reprinted in K. R. Scherer & P. Ekman (Eds). Approaches to Emotion (pp. 247-257). Hillsdale, NJ: Lawrence Erlbaum.] Lazarus, R.S. (1991). Progress on a cognitive-motivational-relational theory of emotion. American Psychologist, 46, 819-834. Lerner, J. S. & Keltner, D. (2001). Fear, anger, and risk. Journal of Personality and Social Psychology, 81, 146-159. Lewis, R. L. (1993). An Architecturally-based theory of Human Sentence Comprehension. PhD thesis. Carnegie Mellon University. Marinier, R. and Laird, J. E. (2004). Towards a Comprehensive Computational Model of Emotions and Feelings. 6th International Conference on Cognitive Modeling. Nason, S. and Laird, J. E. (2004). Soar-RL: Integrating Reinforcement Learning with Soar. 6th International Conference on Cognitive Modeling. Neumann, R. (2001). The causal influences of attributions on emotions: A procedural priming approach. Psychological Science, 11(3):179-182. Newell, A. 1990. Unified Theories of Cognition. Cambridge, MA: Harvard University Press. Perrez, M. & Reicherts, M. (1992). Stress, Coping, and Health: A situation-behavior approach: Theory, methods, applications. Seattle: Hogrefe & Huber Publishers. Philippot, P. & Dozier, S. (1996). Subjective and physiological consequences of associating emotional memories. Proceedings of the 9th Conference of the International Society for Research on Emotion, (pp. 155-159). Philippot, P., & Schaefer, A. (2001). Emotion and memory. In T. J. Mayne & G. A. Bonanno (Eds.) Emotions: Current Issues and Future Directions (pp. 82-122). New York: Guilford. Roseman, I. (2001). A model of appraisal in the emotion system: Integrating theory, research, and applications. In K.R. Scherer, A. Schorr & T. Johnstone (Eds). Appraisal Processes in Emotion (pp. 68-91). Oxford: Oxford University Press. Roseman, I. & Smith, C. A. (2001). Appraisal theory: Overview, Assumptions, Varieties, Controversies. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.) Appraisal processes in emotion: Theory, Methods, Research (pp. 3-19). New York and Oxford: Oxford University Press. Rosenberg, E. L. (1998). Levels of analysis and the organization of affect. Review of General Psychology, 2, 247-270. Rumelhart, D. E. (1980). Schemata: The building blocks of cognition. In R. J. Spiro, B. C. Bruce, & W. F. Brewer (Eds.) Theoretical issues in reading comprehension: Perspectives from cognitive psychology, linguistics, artificial intelligence, and education (pp. 33-58). Hillsdale, NJ: Erlbaum. Scherer, K. R. (2001). Appraisal considered as a process of multi-level sequential checking. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.) Appraisal processes in emotion: Theory, Methods, Research (pp. 92-120). New York and Oxford: Oxford University Press.
49
Smith, C.A. & Kirby, L.A. (2001). Toward delivering on the promise of appraisal theory. In K.R. Scherer, A. Schorr & T. Johnstone (Eds). Appraisal Processes in Emotion (pp. 121138). Oxford: Oxford University Press. Smith, C. A. & Lazarus, R. S. (1990). Emotion and adaptation. In L. A. Pervin (Ed.), Handbook of personality theory and research (pp. 609-637). New York: Guilford. Talmy, L. (1975). Semantics and syntax of motion. In J. P. Kimball, editor, Syntax and semantics Vol. 4 (pp. 181-238). New York: Academic Press. Zacks, J. M. & Tversky, B. (2001). Event Structure in Perception and Conception. Psychological Bulletin, 127(1):3-21.
50