As target tracking is arousing more and more interest, the necessity to reliably assess tracking algorithms
in any conditions is becoming essential. The evaluation of such algorithms requires a database of sequences
representative of the whole range of conditions in which the tracking system is likely to operate, together
with its associated ground truth. However, building such a database with real sequences, and collecting the
associated ground truth appears to be hardly possible and very time-consuming.
Therefore, more and more often, synthetic sequences are generated by complex and heavy simulation
platforms to evaluate the performance of tracking algorithms. Some methods have also been proposed using
simple synthetic sequences generated without such complex simulation platforms. These sequences are
generated from a finite number of discriminating parameters, and are statistically representative, as regards
these parameters, of real sequences. They are very simple and not photorealistic, but can be reliably used
for low-level tracking algorithms evaluation in any operating conditions.
The aim of this paper is to assess the reliability of these non-photorealistic synthetic sequences for evaluation
of tracking systems on complex-textured objects, and to show how the number of parameters can be
increased to synthesize more elaborated scenes and deal with more complex dynamics, including occlusions
and three-dimensional deformations.
As more and more research effort is drawn into
object tracking algorithms, the ability to assess the performance of
these algorithms quantitatively has become a fundamental issue in
computer vision. Because tracking systems have to operate in widely
varying conditions (different weather conditions, background and
target characteristics, etc), a large test bed of video sequences is
needed in order to obtain a comprehensive evaluation of a tracker
across the whole range of its operating conditions. However, it is
very unlikely that a dataset of real video sequences representative
of the whole range of operating conditions of a tracker together
with its ground truth could be obtained, and building a realistic
synthetic dataset of such sequences would require costly advanced
simulation platforms.
In the new evaluation method proposed in this paper, the operational
criteria of the tracking system are turned into objective measures
and used to generate a synthetic dataset, non-photorealistic, but
statistically representative of the whole range of operating
conditions. The assessment of an algorithm using our method provides
both a quantitative evaluation of the algorithm and the borders of
its validity domain. The performance measurement of an algorithm on
a synthetic sequence is shown to be consistent with the measurement
on a real sequence with the same criteria. The benefit of this
approach is twofold: it provides the developer with a way to
concentrate on the weaknesses of his algorithm, and helps the system
designer to choose the algorithm that best fits the operating
constraints.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.