Professional Documents
Culture Documents
Chapter 1:
INTRODUCTION
Animal survival depends on highly developed sensory abilities. Likewise, human cognition depends on highly developed abilities to perceive, integrate and interpret visual, auditory, and touch information. Without a doubt, computers would be much more powerful if they had even a small fraction of the perceptual ability of animals or humans. Adding such perceptual abilities to computers would enable computers and humans to work together more as partners. Blue Eyes uses sensing technology to identify a user's actions and to extract key information. This information is then analyzed to determine the user's physical, emotional, or informational state, which in turn can be used to help make the user more productive by performing expected actions or by providing expected information. The blue eyes technology aims at creating computational machines that have perceptual and sensory ability like those of human beings. It uses non-obtrusive sensing method, employing most modern video cameras and microphones to identify the users actions through the use of imparted sensory abilities. The machine can understand what a user wants, where he is looking at, and even realize his physical or emotional condition. The aim of blue eye technology is 1.To design smarter devices. 2. To create devices with emotional intelligence.3.To create computational devices with perceptual abilities.
It can even understand your emotions at the touch of the mouse .It can verify your identity, feel your presence, and start interacting with you. The machine can understand what a user wants, where he is looking at, and even realize his physical or emotional states. It realizes the urgency of the situation through the mouse. For instance if you ask the computer to dial to your friend at his office, it understands the situation and establishes a connection. It can reconstruct the course of operators work. Blue Eyes system provides technical means for monitoring and recording the operators basic physiological parameters. The most important parameter is saccadic activity, which enables the system to monitor the status of the operators visual attention along with head acceleration, which accompanies large displacement of the visual axis. Complex industrial environment can create a danger of exposing the operator to toxic substances, which can affect his cardiac, circulatory and pulmonary systems. Thus, on the grounds of plethysmographic signal taken from the forehead skin surface, the system computes heart beat rate and blood oxygenation. The Blue Eyes system checks above parameters against abnormal (e.g. a low level of blood oxygenation or a high pulse rate) or undesirable (e.g. a longer period of lowered visual attention) values and triggers user-defined alarms when necessary. Quite often in an emergency situation operators speak to themselves expressing their surprise or stating verbally the problem. Therefore, the operators voice, physiological parameters and an overall view of the operating room are recorded. This helps to reconstruct the course of operators work and provides data for long term analysis. Our system consists of a mobile measuring device and a central analytical system. The mobile device is integrated with Bluetooth module providing wireless interface between sensors worn by the operator and the central unit. ID cards assigned to each of the operators and adequate user profiles on the central unit side provide necessary data personalization so different people can use a single mobile device called DAU Data Acquisition Unit. Some key benefits of blue eyes technology are: Prevention from dangerous incidents,
Page 2
Page 3
Page 4
pulse and skin temperature for each of the six emotions (anger, disgust, fear, happiness, sadness and surprise)] across the five minute baseline and test sessions. GSA data was sampled 80 times per second, GSR and temperature were reported approximately 3 4 times per second and pulse was recorded as a beat was detected approximately once per second. To account for individual variance in physiology, we calculated the difference between the baseline and test scores. Scores DEPT. OF E&C, MITE Page 5
Page 6
Page 7
Page 8
Page 9
Figure 4 the liberal magic pointing technique cursor is placed in the vicinity of a target that the user fixates on
Page 10
Figure 5 Bright (left) and Dark (right) pupil images resulting from on and off axis illumination
Page 11
Odd frames were synchronized with the second light source positioned off the axis. The two light sources are calibrated to provide approximately equivalent whole-scene illumination. Pupil detection was realized by means of subtracting the dark pupil image from the bright pupil image. After thresholding the difference, the largest connected component was identified as the pupil. This technique significantly increased the robustness and reliability of the eye tracking system .
Once the pupil has been detected, the corneal reflection (the glint reflected from the surface of the cornea due to one of the light sources) is determined from the dark pupil mage. The reflection is then used to estimate the users point of gaze in terms of the screen coordinates where the user is looking at.
Page 12
Page 15
Page 16
Bluetooth device
Bluetooth device
Database
Voice interface
Visualisation Module
Page 17
PCM
Bluetooth module
LED indicators
BlueEyes
ID card mechanic interf ace ID card Simple key board Data Acquisition Unit
PCM Codec: PCM codec is used to transmit operators voice and central system sound feedback. The codec that is used here reduces the microcontrollers tasks and lessens the amount of data being sent over the UART. Communication between the Bluetooth module and the microcontroller is carried on using standard UART interface. The speed of the UART is set to 115200 bps in order to assure that the entire sensor data is delivered in time to the central system. Alphanumeric LCD display: The alphanumeric LCD display (two 8-character lines) gives more information of incoming events and helps the operator enter PIN code. The LED indicators show the results of built-in self-test, power level and the state of wireless connection. The simple keyboard is used to react to incoming events (e.g. to silence the alarm sound) and to enter PIN code while performing authorization procedure. ID card interface helps connect the operators DEPT. OF E&C, MITE Page 18
UART
Beeper
4. Visualization Module: The module provides user interface for the supervisors. It enables them
to watch each of the working operators physiological condition along with a preview of selected video source and his related sound stream. All the incoming alarm messages are instantly signaled to the supervisor. Moreover, the visualization module can be set in the off-line mode, where all the data is fetched from the database. The physiological data is presented using a set of custom-built GUI controls: a pie-chart used to present a percentage of time the operator was actively acquiring the visual information A VU-meter showing the present value of a parameter time series displaying a history of selected parameters' value
Page 20
Applications
1. It can be used in the field of security & control, where the contribution of human operator is required round the clock. 2. Engineers at IBM's office: smart tags Research Center in San Jose, CA, report that a number of large retailers have implemented surveillance systems that record and interpret customer movements, using software from Almaden's BlueEyes research project. BlueEyes is developing ways for computers to anticipate users' wants by gathering video data on eye movement and facial expression. Your gaze might rest on a Web site heading, for example, and that would prompt your computer to find similar links and to call them up in a new window. But the first practical use for the research turns out to be snooping on shoppers. BlueEyes software makes sense of what the cameras see to answer key questions for retailers, including, how many shoppers ignored a promotion? How many stopped? How long did they stay? Did their faces register boredom or delight? How many reached for the item and put it in their shopping carts? BlueEyes work by tracking pupil, eyebrow and mouth movement 3. Another application would be in the automobile industry. By simply touching a computer input device such as a mouse, the computer system is designed to be able to determine a person's emotional state. For example, for cars, it could be useful to help with critical decisions like: I know you want to get into the first lane, but Im afraid I cant do that. Your too upset right now and therefore assist in driving safely. 4. Current interfaces between computers and humans can present information vividly, but have no sense of whether that information is ever viewed or understood. In contrast, new real-time computer vision techniques for perceiving people allows us to create "Face-responsive Displays" and "Perceptive Environments", which can sense and respond to users that are viewing them. 5. It can also be implemented in the flight control centers that require the continuous monitoring of a human operator for keeping the track of all flights.
Page 21
Page 22
Conclusion
The Blue eyes technology ensures a convenient way of simplifying the life by providing more delicate and user friendly facilities in computing devices. The gap between the electronic and physical world will be reduced. The computers can be run by using the implicit commands instead of the explicit commands. In future it is possible to create a computer which can interact with us as we do with each other with the use of blue eye technology. It seems to be a fiction, but it will be the life lead by BLUE EYES in the very near future. ordinary household devices -such as televisions, refrigerators, and ovens -- may be able to do their jobs when we look at them and speak to them.
Page 23
Page 24