KEYWORDS: Acoustics, Sensors, Classification systems, Signal processing, Surveillance, Glasses, Systems modeling, Detection and tracking algorithms, Environmental sensing, Signal to noise ratio
Understanding of group activity based on analysis of spatiotemporally correlated acoustic sound events has received a
minimum attention in the literature and hence is not well understood. Identification of group sub-activities such as:
Human-Vehicle Interactions (HVI), Human-Object Interactions (HOI), and Human-Human Interactions (HHI) can
significantly improve Situational Awareness (SA) in Persistent Surveillance Systems (PSS). In this paper, salient sound
events associated with group activities are preliminary identified and applied for training a Gaussian Mixture Model
(GMM) whose features are employed as feature vectors for training of algorithms for acoustic sound recognition. In this
paper, discrimination of salient sounds associated with the HVI, HHI, and HOI events is achieved via a Correlation
Based Template Matching (CMTM) classifier. To interlinked salient events representing an ontology-based hypothesis,
a Hidden Markov Model (HMM) is employed to recognize spatiotemporally correlated events. Once such a connection
is established, then, the system generates an annotation of each perceived sound event. This paper discusses the
technical aspects of this approach and presents the experimental results for several outdoor group activities monitored by
an array of acoustic sensors.
KEYWORDS: Sensors, Decision support systems, Information fusion, Data fusion, Intelligence systems, Data modeling, Video, Intelligent sensors, Acoustics, Statistical analysis
This paper presents an ongoing effort towards development of an intelligent Decision-Support System (iDSS)
for fusion of information from multiple sources consisting of data from hard (physical sensors) and soft
(textural sources. Primarily, this paper defines taxonomy of decision support systems for latent semantic data
mining from heterogeneous data sources. A Probabilistic Latent Semantic Analysis (PLSA) approach is
proposed for latent semantic concepts search from heterogeneous data sources. An architectural model for
generating semantic annotation of multi-modality sensors in a modified Transducer Markup Language (TML)
is described. A method for TML messages fusion is discussed for alignment and integration of
spatiotemporally correlated and associated physical sensory observations. Lastly, the experimental results
which exploit fusion of soft/hard sensor sources with support of iDSS are discussed.
Handling, manipulation, and placement of objects, hereon called Human-Object Interaction (HOI), in the environment
generate sounds. Such sounds are readily identifiable by the human hearing. However, in the presence of background
environment noises, recognition of minute HOI sounds is challenging, though vital for improvement of multi-modality
sensor data fusion in Persistent Surveillance Systems (PSS). Identification of HOI sound signatures can be used as
precursors to detection of pertinent threats that otherwise other sensor modalities may miss to detect. In this paper, we
present a robust method for detection and classification of HOI events via clustering of extracted features from training
of HOI acoustic sound waves. In this approach, salient sound events are preliminary identified and segmented from
background via a sound energy tracking method. Upon this segmentation, frequency spectral pattern of each sound
event is modeled and its features are extracted to form a feature vector for training. To reduce dimensionality of training
feature space, a Principal Component Analysis (PCA) technique is employed to expedite fast classification of test feature
vectors, a kd-tree and Random Forest classifiers are trained for rapid classification of training sound waves. Each
classifiers employs different similarity distance matching technique for classification. Performance evaluations of
classifiers are compared for classification of a batch of training HOI acoustic signatures. Furthermore, to facilitate
semantic annotation of acoustic sound events, a scheme based on Transducer Mockup Language (TML) is proposed.
The results demonstrate the proposed approach is both reliable and effective, and can be extended to future PSS
applications.
Application of acoustic sensors in Persistent Surveillance Systems (PSS) has received considerable attention over the last
two decades because they can be rapidly deployed and have low cost. Conventional utilization of acoustic sensors in PSS
spans a wide range of applications including: vehicle classification, target tracking, activity understanding, speech
recognition, shooter detection, etc. This paper presents a current survey of physics-based acoustic signature classification
techniques for outdoor sounds recognition and understanding. Particularly, this paper focuses on taxonomy and ontology
of acoustic signatures resulted from group activities. The taxonomy and supportive ontology considered include: humanvehicle,
human-objects, and human-human interactions. This paper, in particular, exploits applicability of several
spectral analysis techniques as a means to maximize likelihood of correct acoustic source detection, recognition, and
discrimination. Spectral analysis techniques based on Fast Fourier Transform, Discrete Wavelet Transform, and Short
Time Fourier Transform are considered for extraction of features from acoustic sources. In addition, comprehensive
overviews of most current research activities related to scope of this work are presented with their applications.
Furthermore, future potential direction of research in this area is discussed for improvement of acoustic signature
recognition and classification technology suitable for PSS applications.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.