Background: In target detection, the success rates depend strongly on human observer performances. Two prior studies tested the contributions of target detection algorithms and prior training sessions. The aim of this Swiss-German cooperation study was to evaluate the dependency of human observer performance on the quality of supporting image analysis algorithms. Methods: The participants were presented 15 different video sequences. Their task was to detect all targets in the shortest possible time. Each video sequence showed a heavily cluttered simulated public area from a different viewing angle. In each video sequence, the number of avatars in the area was altered to 100, 150 and 200 subjects. The number of targets appearing was kept at 10%. The number of marked targets varied from 0, 5, 10, 20 up to 40 marked subjects while keeping the positive predictive value of the detection algorithm at 20%. During the task, workload level was assessed by applying an acoustic secondary task. Detection rates and detection times for the targets were analyzed using inferential statistics. Results: The study found Target Detection Time to increase and Target Detection Rates to decrease with increasing numbers of avatars. The same is true for the Secondary Task Reaction Time while there was no effect on Secondary Task Hit Rate. Furthermore, we found a trend for a u-shaped correlation between the numbers of markings and RTST indicating increased workload. Conclusion: The trial results may indicate useful criteria for the design of training and support of observers in observational tasks.
The last few years showed that a high risk of asynchronous threats is given in every day life. Especially in large crowds a high probability of asynchronous attacks is evident. High observational abilities to detect threats are desirable. Consequently highly trained security and observation personal is needed. This paper evaluates the effectiveness of a training methodology to enhance performance of observation personnel engaging in a specific target identification task. For this purpose a crowd simulation video is utilized. The study first provides a measurement of the base performance before the training sessions. Furthermore a training procedure will be performed. Base performance will then be compared to the after training performance in order to look for a training effect. A thorough evaluation of both the training sessions as well as the overall performance will be done in this paper. A specific hypotheses based metric is used. Results will be discussed in order to provide guidelines for the design of training for observational tasks.
The statistical methods discussed in this paper are drawn from the area of machine learning or data mining as well as from descriptive statistics. These techniques are discussed with focus on their applicability to the results of observer trials in order to evaluate the effectiveness of signature measures. Signature measures aim at the change of the apparent signature of an object, e.g. a vehicle. So signature measures can be camouflage against infrared sensory, or they can be used for deception reasons. In order to evaluate the effectiveness of signature measures, observer trials provide an efficient method. The department of Signatorics of Fraunhofer IOSB developed a software tool named CARPET (Computer Aided inteRactive Performance Evaluation Tool) for the realization of observer trials. The benefit of this system is the reproducibility and uniformity of trials for every observer. The results from this system consist of marks, that were placed at particular times, as well as computer mouse positions recorded for each human observer. Based on the information gathered from these marks together with the known target object positions the statistical treatment can be done. For the statistics it has to be known to which target object the marks belong. The first problem considered in this paper concentrates on the correct labeling of the marks according to the target objects. The labeling is done using an expectation maximization scheme with the k-means clustering algorithm. The next step involves a second labeling. In this step a linear discriminant is used to decide whether a mark should be considered a hit or miss for every particular target object. After these decisions, a receiver-operating characteristics (ROC) analysis is performed in order to evaluate the detectability of each target object. Furthermore the sample mean and sample covariance formulas are used on the so called hit sets in order to approximate Gaussian distributions for every hit set. These Gaussians facilitate the evaluation of the accuracy and the precision of the hit sets. Accuracy and precision offer information about the quality of the marks set by the observers.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.