Applied research presented in this paper describes an approach to provide meaningful evaluation of the Machine Learning (ML) components in a Full Motion Video (FMV) Machine Learning Enabled System (MLES). The MLES itself is not discussed in the paper. We focus on the experimental activity that has been designed to provide confidence that the MLES, when fielded under dynamic and uncertain conditions, performance will not be undermined by a lack of ML robustness. For example, to real-world changes of the same scene under differing light conditions. The paper details the technical approach and how it is applied to data, across the overall experimental pipeline, consisting of a perturbation engine, test pipeline and metric production. Data is from a small imagery dataset and the results are shown and discussed as part of a proof of concept study.
In order to make sensible decisions during a multi domain battle, autonomous systems, just like humans, need to understand the current military context. They need to ‘know’ important mission context information such as, what is the commander’s intent and where are, and in what state, are friendly and adversary actors. They also need an understanding of the operating environment; the state of the physical systems ‘hosting’ the AI; and just as importantly, the state of the communication networks that allows each AI ‘node’ to receive and share critical information. The problem is: capturing, representing, and reasoning over this contextual information is especially challenging in distributed, dynamic, congested and contested multi domain battlespaces. This is not only due to rapidly changing contexts and noisy, incomplete and potentially erroneous data, but also because, at the tactical edge, we have limited computing, storage and battery resources. The US Army Research Laboratory, Australia’s Defence Science Technology Group and associated University partners are collaborating to develop an autonomous system called SMARTNet that can transform, prioritize and control the flow of information across distributed, intermittent and limited tactical networks. In order to do this however, SMARTNet requires a good understanding of the current military context. This paper describes how we are developing this contextual understanding using new AI and ML approaches. It then describes how we are integrating these approaches into an exemplar tactical network application that improves the distribution of information in complex operating environments. It concludes by summarizing our results to-date and by setting a way forward for future research.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.