Professional Documents
Culture Documents
ABSTRACT
People can stay connected to their loving ones ubiquitously that they care about by
sharing awareness information in a passive way. Only very recently the detection
of real-world activities has been attempted by processing multiple sensors data
along with inference logic for real-world activities. This paper proposes to infer
human activity from environmental sound cues and common sense knowledge,
which is an inexpensive alternative to the traditional sensors. Because of their
ubiquity, the mobile phones or hand-held devices (HHD) are ideal channels to
achieve a seamless integration between the physical and virtual worlds. Therefore,
a prototype is proposed here to log daily events by HHD based application to infer
activities from environmental sound cues. To the best of our knowledge, this
system pioneers the use of environmental sound based activity recognition in
mobile computing to reflect one’s real-world activity for ambient communication.
1 INTRODUCTION scene analysis [4] that deals with the first objective.
AED processes acoustic signals and converts those
Although speech is the most informative acoustic into symbolic descriptions corresponding to a
event, other kind of sounds may also carry useful listener's perception of the different sound events that
information regarding the surrounding environment. are present in the signals and their sources. Life-log
In fact, in that environment the human activity is is a chronological list of activities performed by the
reflected in a rich variety of acoustic events, either user with respect to time. Such a list might indicate
produced naturally or by the human body or by the the user’s well-being or abnormality according to the
objects manipulated or interacted by humans. consideration of the person’s self assessment or by
Consequently, detection or classification of acoustic someone else who cares about the person (e.g.,
events may help to detect and describe the human relatives or care-givers). Therefore we apply the
and social activity that takes place in the concept of AED to perform automatic generation of
environment. For example: Jingling sound of life-log. This life-log can be transmitted
cooking utensils (like cooking pan, spoon, knife etc.) autonomously as a simple text message to someone
may lead to infer someone’s cooking activity, else with the notion of ambient communication.
vehicle passing sound may lead to infer that someone In this paper, we describe a listening test made to
is on the road, etc. Additionally, the robustness of facilitate the direct comparison of the system’s
automatic speech recognition systems may be performance to that of human subjects. A forced
increased by a previous detection of the non-speech choice test with identical test samples and reference
sounds lying in the captured signals. classes for the subjects and the system is used. The
Many sources of information for sensing the second main concern in this paper is to evaluate how
environment as well as activity are available acceptable the automatic generation of life-log is.
[1][2][3]. In this paper, we consider two objectives Since we are dealing with a highly varying acoustic
namely, sound-based context awareness, where the material where practically any imaginable sounds
decision is based merely on the available acoustic can occur, we have limited our scope in terms of
information at the surrounding environment of the location and the activities to recognize at a particular
user and automatic life-logging, where the detected location. It is most likely that the acoustic models we
sound context infers an activity to be logged along are using are not able to sufficiently model the
with temporal information. Acoustic Event Detection observation statistics. Therefore, we propose using
(AED) is a recent sub-area of computational auditory
discriminative training instead of conventional containing life log for a particular day as following
maximum likelihood training. might be very relieving to the sons. “Your parents
The paper is organized as follows: Section 2 woke up at 7:30 am today and they had breakfast
explains the motivation of the envisioned application. around 8:15 in the morning. They watched TV
Section 3 reviews the background studies related to several times in the day. Went out of home for two
this research. Our approach, in terms of system times and walked in the roads and parks. They took
architecture and description of the system lunch and dinner at around 2 PM and 8 PM. Your
components is explained in Section 4. Section 5 mother went to toilet for 5 times and father went to
discusses about several development challenges and toilet 6 times in a day. They had communicated with
our response towards those. Section 6 explains the each other or other people by talking. It seems they
experimental setup, the results obtained by the are doing fine”.
detection and classification system as well as user
evaluations. Conclusions are presented in Section 7. 3 BACKGROUND
0
Script Generator
−1
1 1.5 2 2.5 3
Toilet flushing sound
Figure 1: The System’s Architecture 1
Amplitude
0
4.2 Description of System Components
−1
1 1.5 2 2.5 3
In this section the system components are Showering sound
described briefly.
Amplitude
0.2
0
4.2.1 Sound Corpus −0.2
The patterns of sounds arising from activities 1 1.5 2 2.5 3
occurring naturally or due to interaction with the Time (s)
objects are obviously a function of a many Figure 2: Waveforms of different water related
environmental variables like size and layout or the sound types
indoor environment, material of the floors and walls, Spectrum of Face washing sound
type of objects (e.g., electrical or mechanical) and 10
Energy (dB)
0
speech recognition whereby the system is −10
−20
individually trained on each user for speaker −30
dependent recognition because such environmental 0 1 2 3 4 5 6 7 8
Spectrum of showering sound
sounds may vary in different cultures and places.
Energy (dB)
10
Therefore the sample sounds we have collected are 0
from the different places of Tokyo city and Tokyo −10
University. For clear audio-temporal delineation −20
0 1 2 3 4 5 6 7 8
during system training, the sound capture for each Frequency (kHz)
activity of interest was carried out separately. A Figure 3: Spectrum of the sounds of Figure 2.
number of male and female subjects were used to
Table 1: Sample list of sound classes and mapped eight iterative cycles.
objects
4.2.3 Classification
Sound Class Mapped Objects
It was obvious that simple frequency
vehicle_pass
road, bus, car, cycle, wind, people characterization would not be robust enough to
TABLE I. toilet_fl produce good classification results. To find
water, toilet, toilet-flush representative features, previous study [28] carried
ush
TABLE II.
bathroom, shower, towel, water, body-wash out an extensive comparative study on various
n_shower transformation schemes, including the Fourier
TABLE III.
n_open_air_train
train, journey, people, announcement Transform (FT), Homomorphic Cepstral Coefficients
TABLE IV. soba, noodle, people, chop stick, soup, rice, (HCC), Short Time Fourier Transform (STFT), Fast
ood_eating plate, cutleries, food, kitchen, restaurant Wavelet Transform (FWT), Continuous Wavelet
TABLE V. Transform (CWT) and Mel-Frequency Cepstral
TV, game, living room, news
V_watching
TABLE VI.
Coefficient (MFCC). It was concluded that MFCC
people, train, announcement, station might be the best transformation for non-speech
rain_enter_leave
TABLE VII.
restroom, water basin, wash, hand soap
environmental sound recognition. A similar opinion
ater_basin was also articulated in [29][30]. These finding
TABLE VIII.
ater_sink
kitchen, sink, water, dish washer provide the essential motivation for us to use MFCC
in extracting features for environmental sound
The typical waveforms of the water related sounds of classification. For classification, continuous HMM
five different actions are shown in Figure 2. As can recognition is used. The grammar used is as follows:
be seen, the water splashing for face washing (top), (<alarm_clock|ambulance|body_spray|chop_board|cy
toilet flushing (middle) and shower water (bottom), cle_bell|dish_cleaning|egg_fry|face_wash|female_co
the sounds as depicted in the waveforms has distinct ugh|toilet_flush|food_eating|gargle|hair_dry|in_open
patterns with different amplitudes. The Fourier _air_train|insect_semi|in_shower|in_subway_train|in
spectra of those three type sounds are illustrated in _train_station|liquid_stir|lunch_time_cafe|male_coug
Figure 3. According to the location and activities of h|male_male_Speak|male_sniff|meeting_talk|microw
our interest mentioned in Table I, we have collected ave_oven|paper_flip|phone_vibration|plate_clutter|pl
114 types of sounds. Each of the sound types has 15 ate_knife|silence|pan_spoon|tap_water|tooth_brush|tr
samples of varying length from 10 second to 25 ain_enter_leave|tv_watching|urination|vacuum_clean
seconds. er|vehicle_pass|water_basin|water_sink>), which
means that there is no predefined sequence for all the
4.2.2 HMM Training activities and each activity may be repeated at time at
An accurate and robust sound classifier is critical any sequence.
to the overall performance of the system. There are
however many classifier approaches in the literature, 4.2.4 Sound Class
e.g., those based on Hidden Markov Models We have 114 types of sounds that are grouped
(HMMs), Artificial Neural Networks (ANN), into 40 sound classes to infer 17 kinds of activities.
Dynamic Time Warping (DTW), Gaussian Mixture For example, we have sound samples named as the
Models (GMM), etc. From these options, we have following types, “eating soba”, “food munching”,
chosen an approach based on HMM as the model has “belching”, “spoon plate friction”, which are
a proven track record for many sound classification grouped into one sound class called “food_eating”.
applications [28][29][31]. Another advantage is that Similarly, the sound samples named as, “news on
it can be easily implemented using the HMM Tool TV”, “talk-show on TV”, “baseball game on TV”,
Kit (HTK) [32] which is an open source Toolkit “football game on TV” are grouped into
available on the Internet. It is also notifying that “tv_watching” sound class. By such grouping of the
HTK was originally designed for speech recognition, samples into a particular sound class, we mean that
which means we need to adapt the approach when the group of samples is used to train the classifier to
applying in for environmental sounds of our interest. recognize that particular sound class. Thus we have
Each sound file, corresponding to a sample of a 40 sound classes.
sound type, was processed in frames pre-emphasized 4.2.5 Object Mapping
and windowed by a Hamming window (25 ms) with Each of the sound class is labeled with a list of
an overlap of 50%. A feature vector consisting of a objects that are usually conceptually connected with
39-order MFCC characterized each frame. We the produced sound. Table II shows some of the
modeled each sound using a left-to-right forty-state sound classes along with the associated list of objects
continuous-density HMM without state skipping. related to a particular sound class. After the classifier
Each HMM state was composed of two Gaussian detects an input sound sample, the “object mapping”
mixture components. After a model initialization module returns the list of associated object that
stage was done, all the HMM models were trained in belongs to the detected sound class. This list of
object is given to the commonsense knowledge An example of such inference is given in the sub-
module to facilitate activity inference. section “A Schematic Solution”.
4.2.6 Commonsense Knowledgebase 4.2.8 Script Generator
Once we get the list of objects involved in At specific time (e.g., midnight of each day) the
recognized sound class, we must define the object whole day’s activities are summarized and a daily
involvement probabilities with respect to the report is automatically generated and archived in the
activities of our interest. Intuitively, these describe hard disk for further reference. This report typically
the probability of using the object in that activity contains a consolidation of the frequency of
state. For example, the activity “eating” always occurrences of major activities of interest. A series
involves food, plate, people and water. Requiring of pre-prepared words are used and intelligently
humans to specify these probabilities is time strung together to form simple sentences that
consuming and difficult. Instead, the system conform to the basic rules of English grammar. An
automatically determines these probabilities utilizing example of this is given as following:
ConceptNet [33], which is a large corpus of -----------------------------------------------------------------------
commonsense knowledge. If an activity name co- Daily Report of Mr. Rahman’s Activity, Dec 12, 2008
occurs often with some object name in human Mr. Rahman woke up at 8:20 am in the morning. He went
discourse, then the activity will likely involve the to bathroom then. He had his breakfast at 9:38 am. He
spent most of his time in living room. He watched TV. He
object in the physical world. Our approach is in the had lunch at 2:13 PM. He talked with people. He went out
spirit of such manner while we use commonsense and walked by the roads during evening. He didn’t take
knowledgebase corpus to assign a probability value shower today. At night he talked with someone.
to the object pertaining to a sound class as a model of ----------------------------------------------------------------------
relatedness in human activity. Thus, for example, if The generated script is then automatically emailed to
the system detects that the sound cues represent predefined email addresses. The archival of these
frying pan, sink water, and chop board from daily reports may enable a caregiver or a doctor to
consecutive input samples the developed common review records very quickly and in the process, build
sense knowledge can infer a cooking activity. a detailed understanding of the subject’s daily
Moreover we have also considered another kind of behavioral patterns.
knowledge namely, ontology of daily-life that
contains usual routines of peoples of different roles 4.3 A Schematic Solution
in terms of their presence at specific locations at The system continuously listens to the
specific time. In this initial prototyping three types of environment and captures sound regularly at 10
users are considered and based on empirical study seconds of interval for 10 seconds of duration. Each
the ontology of daily life (listed in Table III) of each captured sound clip is sent to a computer wirelessly
user type is considered along with common sense where the clip is processed. Thus for a three-minute
knowledge base to perform legitimate inference. For interval the system gets nine sound clips. These nine
example, if the system detects a sound cue related to sound clips are considered to infer an activity at that
eating activity during BN (before noon) time frame moment. Each sound clip is processed by a HMM
on a weekday and if the user is a service-holder, it classifier that outputs a particular sound class. Each
doesn’t log the event because the user is not sound class actually represents some real-word
happened to be present in a kitchen around that time. objects out of which interaction the sound maybe
We also plan to provide the flexibility to personalize produced. For example, Let’s assume that the system
such ontology of daily life on the basis of each receives the following nine sound clips to process
respective user. and the HMM classifies the nine clips into the
4.2.7 Inference Engine following five unique sound classes: “chop_board”,
The system continuously listens to the “pan_spoon”, “water_sink”, “plate_clutter”, and
environment but it records sounds for ten seconds “plate_knife”.
with an interval of ten seconds pause between two These five classes of sounds are pre-mapped as
recordings. Thus for a minute the system gets three following,
• “chop_board” Æ {knife, chop board, kitchen}
sound clips of equally length (i.e., ten seconds) that
• “pan_spoon” Æ {cooking pan, spoon, stirrer,
serves as the input to the classifier to get three sound
classes in one minute. Then object-mapping module kitchen}
• “water_sink” Æ {kitchen, sink, water, dish
provides a list of objects pertaining to the recognized
sound classes. In this manner the system gathers a washer}
• “plate_clutter” Æ {cup-board, plate, dish washer,
list of objects for every minute. The inference engine
works with the list of objects that are gathered in rack}
• “plate_knife” Æ {plate, knife, spoon, fork}
every three minutes. This list of objects is then
consulted with the involvement probabilities of The list of interacting objects is dealt with common
activities stored in the commonsense knowledge base.
Table 2: Ontology of daily life
5 DEVELOPMENT CHALLENGES
List of Real World Objects Mapped from In order to develop this system we have the
Sound Classes e.g., list of Interacting following challenges and concerns:
Objects: {knife, chop board, cooking pan,
spoon, stirrer, water sink, basin, water,
cup-board, plate, dish-washer, rack} 5.1 Environmental sound corpus and features
For simplicity our collected sound corpus is
limited to sound cues related to certain genres like,
List of Real World
Objects Mapped from cooking, bathroom activity, human physiological
Sound Classes action, outdoor, home, etc. Since the recognition
rate is the only performance measure of such system,
it is hard to judge how suitable the selected feature is.
Activity Common To solve this problem, it is essential to analyze the
Inference Engine sense
feature itself, and measure how good the feature
itself is. It is concluded that MFCC may be the best
transformation for non-speech environmental sound
recognition [29]. This finding provides the essential
User: Shanta, Grad Student motivation for us to use MFCC in extracting features
Time: Monday 9:30 PM
for environmental sound classification.
Activity: In Kitchen, Cooking
5.2 Computing power of iPhone or HHD
Mapping the The computing potentials of hand help devices
activity on a (HHDs) are increasing in a rapid manner with
virtual world respect to memory and incorporation of full-fledged
operating system but they are still inferior to
Figure 4: Different steps of the scheme solution personal computer comparing to the speed of data
processing. Running of the core signal-processing
sense knowledge. The common sense knowledge is task requires high-end computing and therefore the
extracted from ConceptNet [33] and outputs a processing should be done by means of external
devices connected to the HHDs through the Since each sound clip resolves to a set of objects
Bluetooth or wireless data transmission. Therefore in pertaining to the recognized sound class which is
this case a light process to capture environmental then considered to infer activity and location related
sounds after some intervals will be running on the to that sound class, we developed perceptual testing
HHDs to be transmitted wirelessly to a server for methodology to evaluate the system’s performance
activity detection. on continuous sound streams of various sound events
to infer location and activity. 420 test signals were
5.3 Privacy and Social Issues created, each of which contained a mixture of three
To manage their exposure, users should be able sound clips of respective 114 sound types. Since
to disconnect from the virtual worlds at any time or these 420 test signals are the representative sound
be able to interrupt the sound capturing of their clues for the 40 sound classes to infer 17 activities,
activities. The actual representation of users in the we grouped these 420 test signals into 17 groups
virtual world may be disclosed according to pre- according to their expected affinity to a particular
configured user policies or users list. Additional activity and location. Ten human (i.e., five male, five
privacy issues are related to the collection of female) judges were engaged to listen to the test
environmental data by means of people carrying signals and judge an input signal to infer the activity
devices to different places and data collected about from the given list of 17 activities (i.e., forced choice
them. We consider important privacy and security judgment) as well as the possible location of that
challenges related to people-centric sensing [5][6][7]. activity from the list of given nine locations of our
choice. Each judge was given all the 17 groups of
5.4 Scalability of the solution signals to listen and assess. The number of test
Our default approach is to run activity signals in a group varied from 3 to 6 and each test
recognition algorithms on the mobile HHDs to signal was the result of three concatenated sound
decrease communication costs and also to reduce the clips of same sound type. Therefore a judge had to
computational burden on the server. However, we listen each test signal to infer the location and
recognize that signal processing based classification activity that the given signal seemed most likely to
algorithms may be too computationally intensive for be associated with. In the same way the signals were
present HHDs and propose to run the classifier on a given to the system to process. For the system the
back-end server in this case. This may be particularly entire group of signals was given at a time to output
appropriate during the training phase of a one location and activity for each group. Since
classification model. human judges judged each signal individually, in
order to compare the result with the system, a
6 EXPERIMENTAL RESULTS generalization on the human assessment was done.
The generalization was done in the following manner.
The purpose is to test the performance of the A group of signals had at lest more than 3 signals
system in recognizing the major activities of our and each of the signals was assigned a location and
interest. The system was trained and tested to activity label by the judges. Thus a group of signals
recognize the following 17 activities: Listening obtained a list of locations and activities. We counted
Music, Watching TV, Talking, Sitting Idle, Cleaning, the frequencies of location and activity labels for
Sitting idle, Working with PC, Drinking, Eating, each group assigned by each judge and took the
Cooking, Washing, Urinating, Exercising, Waiting maximum of the respective labels to finally assign
for Train, Traveling by Train, Shopping, Traveling the two types of labels (i.e., activity and location) for
on Road. As explained earlier, the sound samples the group of signals. For each type of label, if more
recording for each activity was carried out separately. than one labels obtained equal frequency the random
For example, for Listening Music, each subject choice of the labels are considered. Thus we
played a piece of music of his/her choice, with this considered the judges’ labels and system’s inference
repeated a number of times for the same individual. with respect to the expected labels for the 17 groups
The other subjects followed the same protocol and of signals. Recognition results for activity and
the entire process was repeated for developing the location are presented in Figure 5 and 6 respectively.
sound corpus for each activity being tested. It is also The recognition accuracy for activity and location is
noted that we have various kinds of sounds (i.e., encouraging with most being above than 66% and
different sound-clip types) that are grouped together 64% respectively. From Figure 5 and 6, we notice
to represent one particular activity. The training data that humans are skillful in recognizing the activity
set was formed utilizing a ‘leave-one-out’ strategy. and location from sounds (i.e., for humans’ the
That is, all the samples would be used for their average recognition accuracy of activity and location
corresponding models’ training except those is 96% and 95% respectively). It is also evident that
included in the signal under testing. Hence, each the system receives the highest accuracy (i.e., 85%
time the models were trained respectively to ensure and 81% respectively) to detect “traveling on road”
that the samples in the testing signal were not activity and “road” location respectively, which is a
included in the training data set.
great achievement and pioneer effort in this research the system were performed firstly in a constrained
that no previous research attempted to infer outdoor setting as a proof-of-concept and in future we plan to
activities with sound cues. The correct classification perform actual trials involving peoples in the normal
of sounds related to activity “working with pc” and course of their daily lives to carry the device running
location “work place” were found to be very our application that listens to the environment and
challenging due to the sounds’ shortness in duration automatically logs the daily event based on the
and weakness in strength, hence the increased mentioned approach. Preliminary results are
frequency for them to be wrongly classified as encouraging with the accuracy rate for outdoor and
‘silence’ type sound class. indoor sound categories for activities being above
Human Program
67% and 61% respectively. We sincerely believe that
100 the system contributes towards increased
90 understanding of personal behavioral problems that
80
significantly is a concern to caregivers or loving ones
of elderly people. Besides further improving the
70
Recognition rate (%)
PC
by in g
fo ting
Ta g on Id le
er g
M g
ai Uri ain
W as in
w V
Dr ing
eli Ta er
Sh d
eli tin g
op c
o r hi g
in
Ex kin
n g in
Sh us i
ng Roa
av Sit pin
W tc h in
W ra
ng T
ow
cis
n
ng lk
n i E at
Tr
ok
ith
tin n a
rT
k i ng
ea
in
n g
Cl
a
ki
ste
them.
av
Li
W
Tr
Tr
e
e
in
en
ad
n
ym
let
ac
ac
io
ra
h
Ro
Ro
Pl
Pl
at
To
G
itc
eT
ic
k
K
g
sid
or
bl
in
in
W
Pu
v
a
In
Li
Tr
In this paper, we described a novel acoustic [1] A. H. Kam, J. Zhang, N. Liu, and L. Shue, “Bathroom
indoor and outdoor activities monitoring system that Activity Monitoring Based on Sound Jianfeng Chen1
automatically detects and classifies 17 major Contact Information,” Int. Conf. on PERVASIVE,
activities usually occur at daily life. Carefully LNCS 3468/2005, pp. 47-61, 2005.
[2] M. Philipose, K. P. Fishkin, M. Perkowitz, D. J.
designed HMM parameters using MFCC features are
Patterson, D. Fox, H. Kautz, D. HahnelInferring,
used for accurate and robust sound based activity and “Activities from Interactions with Objects,” IEEE
location classification with the help of commonsense Pervasive Computing, Vol. 3,No. 4, pp. 50-57, 2004.
knowledgebase. Experiments to validate the utility of
[3] A. Temko, C. Nadeu, “Classification of meeting-room Computing Research Committee Grand Challenge
acoustic events with Support Vector Machines and Proposal, 2003.
Confusion-based Clustering,” Proc. of IEEE [21] S. Vemuri and W. Bender, “Next-Generation Personal
ICASSP’05, pp. 505-508, 2005. Memory Aids,” BT Technology Journal, Vol. 22, No.
[4] D. Wang, G. Brown, Computational Auditory Scene 4, pp. 125-138, 2004.
Analysis: Principles, Algorithms and Applications, [22] M. Blum, A. Pentland, G. Troster, et al., “InSense:
Wiley-IEEE Press, 2006. Internet-Based Life Logging”, IEEE Multimedia, Vol.
[5] A. Mihailidis, G. Fernie, and J.C. Barbenel, “The Use 13, Issue 4, pp.40-48, 2006.
of Artificial Intelligence in the Design of an [23] C. Dickie, R. Vertegaal , D. Fono , C. Sohn , D. Chen
Intelligent Cognitive Orthosis for People with , D. Cheng , J. S. Shell , and O. Aoudeh,
Dementia,” Assistive Technology, Vol. 13, No. 1, pp. “Augmenting and sharing memory with eyeBlog,”
23–39, 2001. Proc. of ACM Workshop on Continuous Archival and
[6] D. Wan, “Magic Medicine Cabinet: A Situated Portal Retrieval of Personal Experiences, 2004.
for Consumer Healthcare,” Int. Sym. On Handheld [24] J. Gemmell, L. Williams, K. Wood, R. Lueder, and
and Ubiquitous Computing (HUC’99), LNCS Vol. G. Bell, “Passive Capture and Ensuing Issues for a
1707/1999, pp. 352–355, 1999. Personal Lifetime Store,” Proc. of Continuous
[7] T. Barger et al., “Objective Remote Assessment of Archival and Retrieval of Personal Experiences, 2004.
Activities of Daily Living: Analysis of Meal [25] M. Raento, A. Oulasvirta, R. Petit, H. Toivonen,
Preparation Patterns,” Presentation at Medical “ContextPhone: A Prototyping Platform for Context-
Automation Research Center, Univ. of Virginia aware Mobile Applications”, IEEE Pervasive
Health System, 2002. Computing, Vol. 4, Issue 2, pp. 51–59, 2005.
[8] Q. Tran, K. Truong, and E. Mynatt, “Cook’s Collage: [26] K. Panu, M. Jani, K. Juha, K. Heikki, M. Esko-Juhani,
Recovering from Interruptions,” Proc. of 3rd Int. “ContextPhone: Managing Context Information in
Conf. on Ubiquitous Computing (Ubi-Comp), 2001. Mobile Devices”, IEEE Pervasive Computing, Vol.
[9] A. Glascock and D. Kutzik, “Behavioral 2, Issue 3, pp. 42–51, 2003.
Telemedicine: A New Approach to the Continuous [27] K. Kim, M. Jung, S. Cho, “KeyGraph-based chance
Non-intrusive Monitoring of Activities of Daily discovery for mobile contents management system
Living,” Telemedicine Journal, Vol. 6, No. 1, pp. 33– Source”, Int. Journal of Knowledge-based and
44, 2000. Intelligent Engineering Systems, Vol. 11, Issue 5,
[10] I. Korhonen, P. Paavilainen, and A. Särelä, pp.313-320, 2007.
“Application of Ubiquitous Computing Technologies [28] M. Cowling, “Non-Speech Environmental Sound
for Support of Independent Living of the Elderly in Classification System for Autonomous Surveillance”,
Real Life Settings,” Proc. of 2nd Int. Workshop Ph.D. Thesis, Griffith University, Gold Coast Campus,
Ubiquitous Computing for Pervasive Healthcare School of Information Technology, 2004.
Applications (UbiHealth), 2003. [29] H.G. Okuno, T. Ogata, K. Komatani, and K. Nakadai,
[11] M. Mozer, “The Neural Network House: An “Computational Auditory Scene Analysis and Its
Environment That Adapts to Its Inhabitants,” Proc. of Application to Robot Audition,” Proc. of Int. Conf. on
AAAI Spring Sym. On Intelligent Environments, Informatics Research for Development of Knowledge
Tech. AAAI Press, pp. 110–114, 1998. Society Infrastructure (ICKS), pp. 73–80, 2004.
[12] E. Campo and M. Chan, “Detecting Abnormal [30] A. Eronen, J. Tuomi, A. Klapuri, S. Fagerlund, T.
Behavior by Real-Time Monitoring of Patients,” Proc. Sorsa, G. Lorho, and J. Huopaniemi, “Audio-based
of AAAI Workshop Automation as Caregiver, AAAI Context Awareness-Acoustic Modeling and
Press, pp. 8–12, 2002. Perceptual Evaluation,” Proc. of IEEE ICASSP'03,
[13] V. Guralnik and K. Haigh, “Learning Models of Vol. 5, pp. 529-532, 2003.
Human Behaviour with Sequential Patterns,” Proc. of [31] L. R. Rabiner, and B.H. Juang, Fundamentals of
AAAI Workshop Automation as Caregiver, AAAI Speech Recognition, PTR Prentice-Hall Inc, New
Press, pp. 24–30, 2002. Jersey, 1993
[14] house_n Project: http://architecture.mit.edu/house_n [32] S. Young, The HTK Book, User Manual, Cambridge
[15] V. Bush, “As we may think”, Atlantic Monthly, 1945 University Engineering Department, 1995
[16] B. Rhodes and T. Starner, “Remembrance Agent: A [33] H. Liu, and P. Singh, “ConceptNet: A Practical
Continuously Running Automated Information Commonsense Reasoning Toolkit,” BT Technology
Retrieval System,” Proc. of Int. Conf. on Practical Journal, Vol. 22, Issue 4, pp. 211-226, 2004.
App. of Intelligent Agents and Multi-Agent
Technology, pp. 487-495, 1996.
[17] B. Clarkson and A. Pentland, “Unsupervised
Clustering of Ambulatory Audio and Video,” Proc. of
IEEE ICASSP’99, pp. 3037-3040, 1999.
[18] B. Clarkson, K. Mase, and A. Pentland, “The
Familiar: A Living Diary and Companion,” Proc. of
ACM Conf. on Computer–Human Interaction, ACM
Press, pp. 271-272, 2001.
[19] J. Gemmell, G. Bell, R. Lueder, S. Drucker, and C.
Wong, “MyLifeBits: Fulfilling the Memex Vision,”
Proc. of ACM Multimedia, pp. 235-238, 2002.
[20] A. Fitzgibbon and E. Reiter, “‘Memories for Life’:
Managing Information over a Human Lifetime,” UK