Evaluation of an EEG-workload model in the

Aegis simulation environment

Chris Berka∗1, Daniel J. Levendowski1, Caitlin K. Ramsey1, Gene Davis1, Michelle N. Lumicao1,
Kay Stanney2, Leah Reeves2, Susan Harkness Regli3, Patrice D. Tremoulet3, Kathleen Stibler3
1
Advanced Brain Monitoring, Inc., 2850 Pio Pico Drive, Suite A, Carlsbad, CA USA 92008;
2
Department of Industrial Engineering and Management Systems, University of Central Florida,
4000 Central Florida Blvd., Orlando, FL USA 32816-2450;
3
Lockheed-Martin Advanced Technology Laboratories, 3 Executive Campus, 6th Floor, Cherry Hill,
NJ USA 08002

ABSTRACT

The integration of real-time electroencephalogram (EEG) workload indices into the man-machine interface could greatly
enhance performance of complex tasks, transforming traditionally passive human-system interaction (HSI) into an active
exchange where physiological indicators adjust the interaction to suit a user’s engagement level. The envisioned
outcome is a closed-loop system that utilizes EEG and other physiological indices for dynamic regulation and
optimization of HSI in real-time. As a first step towards a closed-loop system, five individuals performed as
identification supervisors (IDSs) in an Aegis command and control (C2) simulated environment, a combat system with
advanced, automatic detect-and-track, multi-function phased array radar. The Aegis task involved monitoring multiple
data sources (i.e., missile-tracks, alerts, queries, resources), detecting required actions, responding appropriately, and
ensuring system status remains within desired parameters. During task operation, a preliminary workload measure
calculated in real-time for each second of EEG and was used to manipulate the Aegis task demands. In post-hoc
analysis, the use of a five-level workload measure to detect cognitively challenging events was evaluated. Events in
decreasing order of difficulty were track selection-identification, alert-responses, hooking-tracks, and queries.
High/extreme EEG-workload occurred during high cognitive-load tasks with a detection efficiency approaching 100%
for selection-identification and alert-responses, 77% for hooking-tracks and 70% for queries. Over 95% of high/extreme
EEG-workload across participants occurred during high-difficulty events (false positive rate < 5%). The high/extreme
workload occurred between 25-30% of time. These results suggest an intelligent closed-loop system incorporating EEG-
workload measures could be designed to re-allocate tasks and aid in efficiently streamlining a user’s cognitive workload.
Such an approach could ensure the operator remains uninterrupted during high/extreme workload periods, thereby
resulting in increased productivity and reduced errors.

Keywords: Human-computer interface, EEG, workload, task allocation, closed-loop system, cognitive overload,
information delivery, mitigation strategies, command-and-control systems (C2).

1. INTRODUCTION

Since the advent of graphical user interfaces (GUIs), a variety of users have sought to embrace computing technology for
a wide range of applications. GUIs have eased interaction by providing tangible constructs (i.e., windows, icons, menus)
via which users can access functionality primarily via a keyboard or pointing device. Yet, the information age has
spawned vast data sources, which have pushed the limits of what GUIs can effectively convey. For example, C2 systems
inundate commanders with an abundance of real-time data sources providing operational views of the environments (air,
sea, ground) in which their forces are deployed. Through primarily visual GUIs, these systems seek to enable military
commanders to determine the status of forces (i.e., friend versus foe) and which assets to allocate to meet mission
objectives. However, as the amount of data available has increased, the ability of GUIs to transform this data to
decision-supportable information has been exceeded. Today, sources of data, in the military and many other fields, go
unutilized as they are never rendered as information sources or, even if they are conveyed, their representation is such

∗
chris@b-alert.com; phone 1 760 720-0099; fax 1 760 720-0094; www.b-alert.com
that it exceeds limits of the human information processor. HSI must be transformed such that information is presented
when needed, in a modality that is readily perceivable, and in a form that is readily interpretable1.

Current HSI loads information onto users without regard to their current workload. Whether or not a user is loaded at a
given instance, if information is available it will be presented to a user if a computer routine is instructed to present it.
The integration of real-time physiological indices of user workload into the HSI loop could assist in managing
information flow, thereby enhancing human performance of complex tasks, such as those associated with C2
environments. With such a system, physiological indicators of user state would be used to adjust interaction to suit a
user’s engagement level. For instance, when a user was overloaded, the indices would trigger information to be off-
loaded or withheld. When a user was inoperative or had additional capacity, the indices would trigger greater
information dissemination or task re-allocation. The envisioned outcome is a closed-loop system that utilizes
physiological indices for dynamic regulation and optimization of HSI in real-time with a goal of maintaining information
load within the limits of the human information processor2.

Although numerous physiological parameters, including cardiovascular indices, pupil diameter, eye movements and
galvanic skin response, have been employed to detect cognitive state changes, the EEG is the only physiological signal
that has been shown to reflect subtle shifts in alertness, attention and workload that can be identified and quantified on a
second-by-second basis. EEG indices of cognitive state changes are sensitive and reliably correlated with performance3-
9
, and several investigations have reported the ability to predict performance on a second-by-second basis using real-time
EEG analysis10-12. EEG measures of workload and task difficulty have been reported in studies of air traffic
controllers13, airline pilots14,15, drivers16, and participants performing cognitive tasks17,18.

In a preliminary investigation of EEG correlates of workload conducted by the Advanced Brain Monitoring Research
Team, an integrated hardware and software solution for acquisition and real-time analysis of the EEG was implemented
to characterize EEG parameters associated with changes in mental workload17. A previously reported workload model
used a set of EEG parameters to provide real-time identification of alertness and drowsiness and to predict vigilance
decrements19-21. These EEG workload parameters were reliably associated with levels of workload during performance
of simple tasks, including: the Warship Commander Task (WCT), a simulated navy C2 environment that allowed
workload levels to be systematically manipulated22; a cognitive task with three levels of difficulty and consistent sensory
inputs and motor outputs23, and a multi-session image learning and recognition memory test24,25.

The present study was designed to utilize the previously reported EEG workload parameters as inputs to a closed-loop
manipulation of a complex, cognitively challenging task. Specifically, an EEG indicator was used to assess workload
while individuals performed the role of an identification supervisor (IDS) in an Aegis C2 simulated environment. The
Aegis C2 is a combat system with an advanced, automatic detect and track, multi-function phased array radar. The Aegis
task required an individual to monitor the current state of an abundance of entities and data sources (i.e., missile tracks,
alerts, queries, resources), detect required actions, affect the appropriate response, and ensure system status remained
within desired parameters (i.e., keeping air, surface, and sub-surface threats properly engaged). Because the IDS
position entails the second highest workload of 32 operators in a typical Aegis Combat Information Center (CIC), it is a
critical position to evaluate for potential human-system design issues and cognitive overload conditions.

The Aegis C2 simulation environment provided a closer approximation to the cognitive challenges encountered in an
operational military environment, allowing the EEG workload measures to be refined during post-hoc analysis. With the
relationship between the EEG workload and Aegis task demands now established and implemented in real-time, future
efforts will be directed toward the establishment of thresholds for identifying states of cognitive overload or states where
operators can receive additional input. It is anticipated that these thresholds will serve as filters in a closed-loop system
to control the delivery of information within military C2 environments such as Aegis.

2. METHODOLOGY

2.1 Acquisition hardware system

A battery-powered wireless EEG sensor headset was used to acquire six channels in a standard hardware montage that
includes bipolar recordings from Fz-POz and Cz-POz, unipolar recordings from Fz, Cz and POz referenced to linked
mastoids, and a bipolar configuration for horizontal and vertical EOG (see Berka, 2004 for a detailed description of the
sensor headset). Data are sampled at 256 samples/second with a bandpass from 0.5 Hz and 65Hz (at 3dB attenuation)
obtained digitally with Sigma-Delta A/D converters. The EEG sensor headset requires no scalp preparation and provides
a comfortable and secure sensor-scalp interface for 12 to 24 hours of continuous use. The headset was designed with
fixed sensor locations for three sizes (e.g., small, medium and large). Sensor placement was determined using a database
of over 225 participants so that each sensor is no more than one centimeter from the International 10 – 20 system
coordinates. The workload studies described in this paper required only bipolar recordings from Fz-POz and Cz-POz.
The remaining channels are utilized to monitor other cognitive states including attention, learning and memory.
Amplification, digitization and radio frequency (RF) transmission of the signals are accomplished with miniaturized
electronics in a portable unit worn on the head. The combination of amplification and digitization of the EEG close to
the sensors and wireless transmission of the data facilitates the acquisition of high quality signals even in high
electromagnetic interference environments.

2.2 Additional sensors and cognitive state gauges

In addition to the wireless EEG system, three alternative physiological sensor systems were employed to acquire data
during the Aegis C2 task. Additional systems included Functional Near Infrared imaging (fNIR) from Drexel University,
heart rate and galvanic skin response from AnthroTronix, Inc., and eye tracking from EyeTracking, Inc. This paper only
reports on the data collected by the EEG system.

2.3 EEG artifact identification and decontamination, and signal processing

Quantification of the EEG in real-time, referred to as the B-Alert® system, is achieved using signal analysis techniques
to identify and decontaminate eye blinks, and identify and reject data points contaminated with electromyography
(EMG), amplifier saturation, and/or excursions due to movement artifacts (see Berka, 2004 for a detailed description of
the artifact decontamination procedures). Decontaminated EEG is then segmented into overlapping 256 data-point
windows called overlays. An epoch consists of three consecutive overlays. Fast-Fourier transform is applied to each
overlay of the decontaminated EEG signal multiplied by the Kaiser window (α = 6.0) to compute the power spectral
densities (PSD). The PSD values are adjusted to take into account zero values inserted for artifact contaminated data
points. The PSD between 70 and 128 Hz is used to detect EMG artifact. Overlays with excessive EMG artifact
(“EMG”) or with fewer than 128 data points (“missing data”) are rejected. The remaining overlays are then averaged to
derive PSD for each epoch with a 50% overlapping window. Epochs with two or more overlays with EMG or missing
data are classified as invalid. For each channel, PSD values are derived for each one-Hz bin (“bin”) from 3 Hz to 40 Hz
and the total PSD from 3 to 40 Hz (“band”). “Relative power” variables are also computed for each channel and bin
using the formula (“total band power/total bin power”). This analysis creates 76 variables per channel that can be used
to implement the model. The outputs of the PSD are used offline for the development of the model and in the real-time
analysis program to provide the workload and alertness classifications.

2.4 Aegis C2 simulated environment

Within an Aegis environment, an IDS is required to monitor the current state of multiple data sources (i.e., missile
tracks, alerts, queries, resources), detect required actions, affect the appropriate response, and ensure system status
remains within desired parameters (Figures 1 & 2).

Figure 1: Screen shot of Aegis simulated task environment. Figure 2: Participant performing Aegis simulated task.
Within the Aegis simulated environment (Figure 1), two primary and one secondary task were evaluated. The two
primary tasks consisted of 1) selecting the highest threat radar track from 10s to 100s of tracks based on a given ordered
set of rules (Table 1), and 2) correctly identifying (ID) the selected track based on a non-ordered set of rules (Table 2).
A user must select a track to ID by making an explicit action, such as hitting a “Select” button via the mouse after
clicking on the selected track or saying “Select the hooked track” into the microphone. An example of a correctly
selected most critical track would be: Suspect Hostile Air track headed towards Ownship with the shortest Time-to-
Ownship of any similarly critical tracks.

No other tracks could be selected until the user entered an ID for the selected track (based on the ID non-ordered rules
presented in Table 2). Users could ID a track by making an explicit action such as entering the ID designation in a text
box via the keyboard or by speaking into the microphone the designation in a trained phrase format, such as “The ID of
this track is Attack Hostile.” Information could be obtained about a track primarily from a list of designated fields on
the right side of the display, which would appear once the track was selected. Some of the presented field values would
appear missing, requiring the user to request information verbally, through the spoken language interface. Once a track
was identified, it was removed from the display and overall scores were adjusted to reflect a correct ID or an incorrect
ID. “Monitor” tracks were removed as well in order to simulate those tracks being delegated to another console station.

Rules by Options by Priority Criteria Attack Monitor Ignore Friendly

Order of Hostile
Priority
1. Bearing Towards Ownship Platform Armed Armed or Unarmed
(inbound) Type Unarmed
Away from Ownship
2. Pre-ID Suspect Hostile Respond No Yes or No Yes
Unknown to Query
Assumed Friendly
3. Type Air Source of Ownship Remote Ownship radar,
Surface Radar radar or radar or Ownship manual,
Water Ownship Remote Remote radar or
Manual manual Remote manual
4. Time-to- Shorter time Multiple Yes or No No Yes
Ownship/ Longer time Source
Speed Slower speed
Faster speed
Table 1: Ordered rules for selection of Table 2: Non-ordered rules for correct identification of selected track.
highest threat track.

A user’s performance was tracked by response time (i.e., how much time it took them to select a track) and accuracy
(i.e., whether the track was correctly identified as Attack Hostile, Ignore Friendly, or Monitor).

The secondary task was modeled after traditional n-back studies and involved the IDS responding to requests presented
as alerts within the Aegis simulated environment (e.g., “What was the location of the 2nd-to-last track you identified?”).
Users had to respond to alerts when presented—the system prevented users from completing other actions until an alert
was answered. Alerts were categorized as either mostly verbal or mostly spatial information requests in order to allow
flexibility in manipulating the types of cognitive loading induced in a user’s working memory. This allowed another
potential factor to be tracked with EEG signals and subsequently accommodated for when necessary to streamline
cognitive load via interface design changes. Response time and accuracy were also assessed for this task.

The Aegis environment was simulated on an Intel Pentium 4- 2.4 GHz Dell desktop computer with 512 MB memory.
The LCS (spoken language interface) software was run on an Intel Pentium 3- 1.2 GHz Dell laptop computer with 640
MB memory. The operating system of both machines was Red Hat/Fedora Linux 9.0 with 2.4 kernel. The visual
interface was presented on a 17” screen. All user responses were with a standard keyboard, 2 button mouse, lapel
microphone and foot pedal.
2.5 Task scoring
Each of the three types of tasks was worth 10 points. Partial credit was given for selection according to the following
scheme: if the second most critical track is selected instead of the most critical track, user earns 9 points, if the third most
critical track is selected user earns 8 points, … if the 10th most critical track is selected user earns 1 point. No partial
credit was given for identification--you either earn 10 points or earn none. For alerts, there was a penalty: correct
responses earn 10 points and incorrect responses result in a loss of 2 points. The faster users selected critical tracks,
correctly identified them, and correctly responded to alerts, the more points they earned. Thus, both speed and accuracy
contributed to a user’s performance scores to deter random guessing.

2.6 Participants & Procedure

Five participants completed the study (3 males, 2 females). Participants with a history of neurological disorders, head
trauma, use of psychotropic or illicit drugs, or who were currently pregnant were excluded. All participants were college
graduates. Participants read and signed an informed consent and then began studying and learning the rule-based
hierarchies for determining optimal performance in the selection, identification, and alert tasks. The complexity of the
Aegis simulation tasks required participants to complete a minimum of eight hours of supervised training before testing.
Participants were randomly assigned to one of two ordered, counterbalanced task scenario conditions consisting of four 5
minute task scenarios with 30 seconds of rest between each scenario.

Each subject completed a 15-minute baseline EEG session that included three 5-minute baseline conditions (eyes open
(EO), eyes closed (EC), and 3-choice vigilance task (3C-VT)). All participants completed the baseline EEG testing
sessions between the hours of 8am and 2pm. During the Aegis training and testing sessions, EEG data was acquired
with the wireless EEG system. After each Aegis scenario session, users complete the NASA TLX subjective workload
questionnaire and participated in a debriefing session.

2.7 Workload classification model

A linear discriminant function analysis (DFA) model was previously reported17 that used 19 EEG variables empirically
selected to provide EEG indices of alertness. The model classified each one-second EEG epoch, and was then classified
into one of four states: “high vigilance”, “low vigilance”, “relaxed wakefulness”, and “sleep-onset.” These four states
were derived using EEG acquired from participants in sleep deprivation studies. The high and low vigilance states were
modeled by varying the level of task engagement. Relaxed wakefulness is the state induced when participants are
instructed to relax with eyes closed and is generally characterized by predominance of EEG in the alpha frequency band
(8 – 12 Hz.). Data for the sleep-onset class were obtained using EEG samples acquired just subsequent to sleep onset.
Results showed that the percentage of high vigilance classifications increased as a function of mental workload as
measured in the WCT C2 environment and other tests previously mentioned22,23,24,25.

As a measure of workload, the model described above was limited because it did not provide a continuous measure
sensitive to subtle shifts in required effort. In off-line analyses, the mean high vigilance probabilities generated by the
DFA were found to be significantly correlated with workload levels as measured by performance in the WCT and the 3-
Level cognitive and learning and memory tests (referenced previously). The probabilities are generated as an
intermediate result of the discriminant function model and range from 0.0 to 1.0, with the sum of the probabilities across
the four classes equal to 1.0.

The model previously reported17 used linear DFA. The classification model used in this study included a quadratic
discriminant function with the EEG variables in Table 3. The strategy and methods used to develop this study’s model
were similar to the previous models. The probabilities of high vigilance were generated in real-time for each second of
EEG and submitted as inputs to the closed-loop Aegis manipulation during data acquisition.

FzPOz 32Hz 34Hz 36Hz 38Hz 40Hz R6Hz R8Hz

CzPOz 5Hz 10Hz 31Hz 35Hz 37Hz R12Hz R14Hz
Table 3: Selected EEG variables (R connotes relative power variables).

To refine the workload measure for use in the Aegis task, the first goal was to define multiple workload measures using
thresholds that are applicable across participants and within-participants over time and that could be implemented in
real-time. A second goal was to ensure the workload measures accommodated gradual changes in state of the individual,
because users would be participating in the closed-loop application for several hours at a time.

A means to qualify workload for a given one-second epoch was derived beginning with the computation of two-minute
moving means and standard deviations of the high vigilance and relaxed wakefulness probabilities generated by the
DFA. Through the use of data from three baseline conditions to fit the DFA model, the resulting probabilities included
adjustments to accommodate individual differences in EEG pattern. The moving means for each epoch were then
transformed into z-scored values.

In order to apply workload thresholds to the z-score values that could be universally applied across participants, the
standard deviations used for the z-score computations were based on the mean standard deviations across all epochs and
participants. Z-score thresholds were then derived to quantify five levels of workload. Extreme, high, normal and low
workload levels were based on probabilities of high vigilance. The distracted workload category was based on the
probability of the relaxed wakefulness class.

To assess the accuracy (sensitivity and specificity) of the high and extreme (“higher”) workload measure in the Aegis
environment, time windows were established around the designated Aegis activities. The following Aegis events were
selected as those most likely to elicit high workload: 1) Selection-Identification, including analysis of each one-second
epoch between the selection and identification, 2) Alerts, including the period from the initiation of the alert to the alert
response, 3) Hooking tracks, and 4) Querying the system via the LCS.

To accommodate possible timing differences, events extracted from the Aegis Log and the EEG higher workload indices
were aligned according to the following rules. LCS queries and Hooking events were matched with EEG higher
workloads within a one-second window before or after the events. One-second before the Select, the interval between
the Select and ID, and up to one second after the ID events was used to identify EEG workload correlates. For alerts, the
interval between the initiation of the alert until one-second after the user responded was matched.

2.8 Experimental design

A 2 (with or without pacing augmentation) x 2 (high or low workload) within-subjects design was conducted. The
pacing augmentation consisted of delaying the presentation of alert tasks during high workload conditions and until
workload decreased. This paper only reports on data from scenarios without pacing augmentation.

Participants performed 4 scenarios per testing session: 2 with augmentation, one high workload (80 tracks) and one low
workload (40 tracks); 2 without augmentation, one high and one low workload (Table 4). One participant (#9) only
completed one of the two scenarios without augmentation. Response time and accuracy were tracked, and users were
scored according to the aforementioned scoring procedures.

4 Aegis Scenarios
Augmentation No No Yes Yes
Workload Level Low High Low High
Table 4: 4 Aegis Scenarios

3. RESULTS

The relationships between the EEG higher workload levels and the Aegis activities are presented in Table 5 by and
across participants. Each scenario provided approximately 600 seconds of EEG for analysis. The data revealed that
greater than 96% of select-ID and alert events were correlated with EEG higher workload levels. Hooking and LCS
events were associated with higher workload to a lesser extent (77% and 70% respectively). Over 95% of the total EEG
higher workload across all participants was accounted for by Aegis events, resulting in an overall false positive rate of
less than 5%. These results demonstrate the excellent sensitivity and specificity of the EEG high workload indices with
respect to the cognitive demands of the Aegis environment. Figure 3 provides a representative sample of the aligned
patterns between the EEG workload indices and selected Aegis tasks.
 Total Percentage
Task Task Select - Select -
User # Difficulty Hooks IDs Alerts LCS HW Hooks IDs Alerts LCS Workload
8 9 Easy 116 50 12 49 206 77.6% 96.0% 91.7% 77.6% 94.2%
8 12 Medium 166 38 12 42 248 83.7% 97.4% 100.0% 83.3% 97.6%
9 1 Medium 165 26 12 30 165 70.3% 96.2% 91.7% 66.7% 93.9%
10 9 Easy 87 25 12 43 214 73.6% 96.0% 100.0% 72.1% 95.8%
10 1 Medium 111 22 12 27 210 81.1% 100.0% 100.0% 59.3% 95.7%
11 9 Easy 110 26 12 21 197 74.5% 100.0% 100.0% 71.4% 96.4%
11 12 Medium 105 22 12 15 193 73.3% 90.9% 100.0% 60.0% 90.7%
12 9 Easy 118 35 12 30 178 77.1% 100.0% 83.3% 56.7% 98.9%
12 12 Medium 136 27 12 29 193 76.5% 92.6% 100.0% 65.5% 95.9%
Total 1114 271 108 286 1804 76.6% 96.7% 96.3% 69.9% 95.5%
Table 5: Relationship between EEG higher workload levels and Aegis tasks. HW = High Workload.

1 2 3 4a 4b 5 6 7 8 9 10
Correct
Response

Incorrect
Response
Unscored
Activity

Extreme
Workload
High
Workload

Normal
Workload

Low
Workload
High
Distraction 1 2 3 4a 4b 5 6 7 8 9 10
Workload Hook Select ID Alert LCS

1. Subject is presented with an alert, and interprets the request.

2. Subject hooks a track and responds to the alert.
3. Subject realizes his error in hooking this track for the sixth time.
4. a. Subject begins making decision between presumably final two “winning” tracks.
b. Subject makes a correct selection.
5. Subject initiates LCS Query, waits for response, interprets LCS response, and makes ID decision.
6. Subject hooks track for the first time and encodes information.
7. Subject makes decision to select the “winning” track that was not selected in 4.
8. Subject makes non-optimal selection. Score = 0.6.
9. Subject interprets information provided by LCS Query and makes correct ID.
10. Subject hooks track for the first time and encodes information.

Figure 3: Representative sample of the aligned patterns between the EEG workload indices and selected Aegis tasks.
 4. CONCLUSIONS

The primary goal of this study was to assess the feasibility of accurately detecting and quantifying an EEG indicator of
mental workload during performance of a highly complex, cognitively challenging task. The Aegis simulation
environment was selected because of its military relevance and similarities with cognitive challenges encountered in an
operational environment.

This study demonstrated that the wireless EEG system with real-time analysis software, previously shown to provide
valid workload classifications during simple cognitive tasks, was also effective in acquiring high quality data, identifying
and decontaminating artifacts and classifying EEG workload measures in a more complex and cognitively challenging
C2 operational environment. The five-level EEG workload model provided good correlations with operator performance
within the Aegis environment. When participants were engaged in the most cognitively challenging activities, the EEG
workload levels were highest. Importantly, the calculation of workload levels was achieved in near real-time, suggesting
the viability of creating a closed-loop system with EEG-workload as one of the inputs. The implication is that these
indices could be used to intelligently control either the speed or mode of presentation of information to aid in ensuring
an operator’s performance remains uninterrupted during the most cognitively challenging aspects of a task. Further,
when an operator evidences continually high or extreme workload, the system could identify when tasks might need to
be delegated to another operator.

In the second phase of the present study (not reported in this paper), the EEG parameters in combination with functional
near infrared imaging, pupillometry, heart rate and galvanic skin response served as inputs to control various aspects of
the Aegis testbed. Although a full reporting of the results is beyond the scope of this paper, the control of the pacing of
information presentation in the Aegis task showed significant promise as a workload mitigation strategy to improve
performance during challenging work conditions. Other potentially promising mitigation strategies, such as modality-
based task switching and intelligent task sequencing, will be investigated in future studies.

Although a promising approach, the design and implementation of investigations of EEG correlates of cognitive states
during complex, challenging tasks is difficult. For example, the Aegis simulation required a minimum of eight hours of
training for individuals to reach adequate levels of performance. Although this level of training would not be considered
unusual in an operational setting, it is atypical for laboratory experiments investigating EEG and cognition4,5,8,10-12,26-29.
Furthermore, additional investigations are needed to determine whether there is generalization to other complex tasks
and operational environments.

4.1 Workload indices in Aegis

The Advanced Brain Monitoring high and extreme EEG workload indices accurately tracked the most cognitively
challenging events in the Aegis task with a detection efficiency approaching 100% for selection-identification and alert
events. Hooking and LCS queries accounted for the remainder of the periods identified as high workload by the EEG
indices. The high/extreme workload periods occupied between 25-30% of total task time, leaving more than 70% of the
time available for additional allocation of tasks and sub-tasks. Taken together, these results suggest an intelligent
closed-loop system incorporating EEG-workload measures could be designed to re-allocate tasks, control manipulations,
and thus aid in efficiently streamlining a user’s cognitive workload. Such an approach could ensure the operator remains
uninterrupted during high/extreme workload periods, thereby resulting in increased productivity and reduced errors.

Another important finding was that the high and extreme workload EEG not associated with any of the target Aegis
events (false positives) occurred less than 5% of the time across participants and scenarios. This guarantees that the
workload gauge is not overly sensitive. In addition, the higher EEG workload periods occupied between 25-30% of the
scenario total time, presumably leaving more than 70% of the sessions available for additional allocation of tasks and
sub-tasks. Thus, in a closed-loop scenario, the system could re-allocate tasks from periods when the operator was
overloaded or introduce tasks from alternative operators within the system when periods of low workload were
identified.
Although this model proved accurate in the Aegis testbed, there is no guarantee that the model will generalize to other
cognitively complex task environments. Additional investigation is required to determine the applicability of the model
in alternative task domains.

4.3. Workload model development

In order to compare workload levels across participants and to set system thresholds for triggering manipulations, the
EEG outputs in the current model were standardized to account for individual differences. In the current study a two-
minute moving average was selected for the threshold mean and standard deviation because of its capability to adapt to
the individual. The two-minute moving average worked well within the constraints of the current study, but the adaptive
model does not provide information on within-user state changes that may be important to monitor in real-world settings.
Because the entire experimental test session was completed in less than two hours, the second-by-second level of task
engagement was likely higher overall than might be expected in a typical eight or ten-hour shift in an operational
environment. It should be noted that although the computation of the adaptive mean achieved the desired output
thresholds in the current paradigm, the real-time outputs of the model could also include the unadjusted probability
values computed by the DFA for each one-second of EEG data. Changes in this parameter could also be monitored to
identify within-user state changes over the course of the session.

The integration of physiological monitoring into the man-machine interface holds great promise for real-time assessment
of operator status and the possibility of intelligently allocating tasks between machines and humans based on determined
operator cognitive states and performance levels. It is envisioned that once accurate real-time monitoring is achieved,
intelligent feedback or “closed-loop” systems can facilitate active intervention by the operator or through a third party
(man or machine) and thus simultaneously increase safety and productivity. For instance, potential performance
improvement benefits of employing effectively designed augmentations (mitigation strategies) within C2 systems, such
as an Aegis CIC, include: increasing the number of critical tracks correctly identified, increasing the number of alerts
successfully handled, increasing the overall number of tracks identified correctly, improving the overall situational
awareness of CIC operators and, ultimately, leading to reduced manning requirements.

Before the closed-loop system can be implemented in real-world scenarios, additional research must be completed to
ensure that the model will have the ability to account for differences within and across tasks and participants. Although
these preliminary findings suggest that within the Aegis task environment one EEG-workload model may be suitable for
driving closed-loop manipulations, it is not possible to generalize to other complex cognitive tasks and operational
settings until more investigations are conducted. The present study’s findings, in combination with those previously
reported by the investigators, do indicate, however, that this approach to model development will prove useful in a
variety of tasks and settings.

ACKNOWLEDGEMENTS

This work was supported by the DARPA program “Improving Warfighter Information Intake Under Stress”, in which
Advanced Brain Monitoring is a sub-contractor to Lockheed-Martin Advanced Technology Laboratories.

REFERENCES

1. Stanney K, Samman S, Reeves L, et al., "A paradigm shift in interactive computing; Deriving multimodal
design principles from behavioral and neurological foundations", International Journal for Human-Computer
Interaction Special Issue, 17, 229-257, 2004.
2. Schmorrow D, Stanney KM, Wilson GF, et al., "Augmented cognition in human-system interaction", Handbook
of human factors and ergonomics, In press.
3. Akerstedt T, Folkard S, "The three-process model of alertness and its extension to performance, sleep latency,
and sleep length", Chronobiol Int, 14, 115-123, 1997.
4. Gevins A, Smith ME, Le J, et al., "High resolution evoked potential imaging of the cortical dynamics of human
working memory", Electroencephalogr Clin Neurophysiol, 98, 327-348., 1996.
5. Gevins A, Smith ME, Leong H, et al., "Monitoring working memory load during computer-based tasks with
EEG pattern recognition methods", Hum Factors, 40, 79-91., 1998.
6. Gevins A, Smith ME, McEvoy L, et al., "High-resolution EEG mapping of cortical activation related to
working memory: effects of task difficulty, type of processing, and practice", Cereb Cortex, 7, 374-385, 1997.
7. Fabiani M, Gratton G, Coles MG, "Event-related Brain Potentials", Handbook of Psychophysiology, Caciooppo
JT, Tassinary LG, Berntson GG, 53-84, Cambridge University Press, Cambridge, 2000.
8. Wilson GF, Eggemeier FT, Physiological measures of workload in multi-task environments, 329-360, London,
1991.
9. Davidson RJ, Jackson DC, Larson CL, "Human Electroencephalography", Handbook of Psychophysiology,
Caciooppo JT, Tassinary LG, Berntson GG, 27-52, Cambridge University Press, Cambridge, 2000.
10. Makeig S, Inlow M, "Lapses in alertness: coherence of fluctuations in performance and EEG spectrum",
Electroencephalogr Clin Neurophysiol, 86, 23-35, 1993.
11. Makeig S, Jung TP, "Changes in alertness are a principal component of variance in the EEG spectrum",
Neuroreport, 7, 213-216, 1995.
12. Makeig S, Jung TP, "Tonic, phasic, and transient EEG correlates of auditory awareness in drowsiness", Brain
Res Cogn Brain Res, 4, 15-25, 1996.
13. Brookings JB, Wilson GF, Swain CR, "Psychophysiological responses to changes in workload during simulated
air traffic control", Biol Psychol, 42, 361-377., 1996.
14. Sterman MB, Mann CA, "Concepts and applications of EEG analysis in aviation performance evaluation", Biol
Psychol, 40, 115-130. Review., 1995.
15. Sterman MB, Mann CA, Kaiser DA, "Quantitative EEG patterns of differential in-flight workload", Space
Operations, Applications, and Research Proceedings, 3187, 466-473, NASA Conference Publication,
Sepulveda VA Medical Center, 1992.
16. Brookhuis KA, de Waard D, "The use of psychophysiology to assess driver status", Ergonomics, 36, 1099-
1110., 1993.
17. Berka C, Levendowski DJ, Olmstead RE, et al., "Real-time Analysis of EEG Indices of Alertness, Cognition
and Memory with a Wireless EEG Headset", International Journal of Human-Computer Interaction, 17, 151-
170, 2004.
18. Pleydell-Pearce CW, Whitecross SE, Dickson BT, "Multivariate analysis of EEG: Predicting cognition on the
basis of frequency decomposition, inter-electrode correlation, coherence, cross phase, and cross power",
Proceedings of the 36th Annual Hawaii International Conference on System Sciences, 131, Hawaii, 2003.
19. Levendowski DJ, Berka C, Olmstead RE, et al., "Correlations between EEG Indices of Alertness Measures of
Performance and Self-Reported States while Operating a Driving Simulator", 29th Annual Meeting, Society for
Neuroscience, 25, Miami Beach, FL, 1999.
20. Levendowski DJ, Berka C, Olmstead RE, et al., "Electroencephalographic indices predict future vulnerability to
fatigue induced by sleep deprivation." Sleep, 24, A243-A244, 2001.
21. Levendowski DJ, Olmstead RE, Konstantinovic ZR, et al., "Detection of Electroencephalographic Indices of
Drowsiness in Real-time using a Multi-Level Discriminant Function Analysis", Sleep, 23, A243-A244, 2000.
22. St. John M, Kobus DA, Morrison JG, et al., "Overview of the DARPA augmented cognition technical
integration experiment", International Journal for Human-Computer Interaction Special Issue, 17, 131-149,
2004.
23. Mathiak K, Hertrich I, Kincses WE, et al., "Involuntary crossmodal enhancement of preattentive auditory
processing is selective in space", Brain Topogr, 16, 200, 2004.
24. Levendowski DJ, Westbrook P, Berka C, et al., "Event-related potentials during a test of working memory
differentiate sleep apnea patients from healthy subjects", Sleep, 25, A460-A461, 2002.
25. Mitler MM, Westbrook P, Levendowski DJ, et al., "Validation of automated EEG quantification of alertness:
methods for early identification of individuals most susceptible to sleep deprivation", Sleep, 25, A147-A148,
2002.
26. Prinzel LJ, Freeman FG, Scerbo MW, et al., "A closed-loop system for examining psychophysiological
measures for adaptive task allocation", Int J Aviat Psychol, 10, 393-410., 2000.
27. Pope AT, Bogart EH, Bartolome DS, "Biocybernetic system evaluates indices of operator engagement in
automated task", Biol Psychol, 40, 187-195., 1995.
28. Gevins. Spatial measurement of EEG electrodes: Sam Technology, 1997
29. Kramer HC, Trejo LJ, Humphrey DG, Psychophysiological measures of workload: Potential applications to
adaptively automated systems, Lawrence Erlbaum Associates, New Jersey, 1996.