As most cameras are currently built to be used alongside machine learning algorithms, image quality requirements still emanate from human perception. To redefine key performance indicators (KPI) for machine vision, optical designs are tested and optimized before their conception using differentiable simulation methods and gradient backpropagation to jointly train an optical design and a neural network. Although this helps to design optical systems for improved machine learning performance, it remains unstable and computationally expensive to model complex compound optics such as wide-angle cameras. We focus on optimizing the distortion profile of ultra wide-angle designs as it constitutes the main KPI during the optical design. Along the way, we highlight the benefits of controlling the distortion profile of such systems, as well as the challenges related to using learning-based methods for optical design.
We present a wide-angle design simulation to predict how its aberrations impact neural networks performances. Our PSF models are optimized for computational efficiency while maintaining accurate predictions which is powerful to support optical design.
This paper delves into the substantial challenges faced in end-to-end computer vision applications where optical systems are jointly optimized alongside downstream data-driven models, and explores available solutions.
Data-driven approaches have proven to be very efficient in many vision tasks, and they are now used for optical parameters’ optimization for application-specific camera designs. Methods such as neural networks are used to estimate camera performance indicators related to the point spread function—such as the root mean square (RMS) spot size—from optical parameters. Such procedures help to understand the connection between optical characteristics and push optical design expertize beyond its limits. We investigate these approaches to model the interaction between the distortion of wide-angle designs and their RMS spot size, which is not explained by aberration theory. Specifically, we test off-the-shelf data-driven methods to determine in which conditions we can establish a model that is able to predict the variations of the RMS spot size along the field of view from the distortion function even in the absence of a mathematical model. Although current methods focus on building accurate models often usable for very specific designs—composed of a few elements only, we present a methodology focusing on more complex and realistic wide-angle designs.
Data-driven methods to assist lens design have recently begun to emerge; in particular, under the form of lens design extrapolation to find starting points (lenses and freeform reflective system). I proposed a trip over the years to better understand why the AI have been applied first to the starting point problems and where we are going in the future. In this talk, we will explore to most recent progress applications of DNN in optical and lens design. We will also show some working example and discuss the future.
Data driven approaches have proven very efficient in many vision tasks and are now used for optical parameters optimization in application-specific camera design. A neural network is trained to estimate images or image quality indicators from the optical characteristics. The complexity and entanglement of such optical parameters raise new challenges we investigate in the case of wide-angle systems. We highlight them by establishing a data-driven prediction model of the RMS spot size from the distortion using mathematical or AI-based methods.
Data-driven methods to assist lens design have recently begun to emerge, in particular under the form of lens design extrapolation: using machine learning, the features of successful lens design forms can be extracted, then recombined to create new designs. Here, we discuss the core aspects and next challenges of the LensNet framework, a deep learning-enabled tool that leverages lens design extrapolation as a more powerful alternative to lens design databases when searching for starting points. We also propose to borrow ideas and tools from the practice of machine learning and deep learning, and integrate them into standard lens design optimization. Namely, we recommend using automatic differentiation to power ray tracing engines, along with considering recent and powerful first-order gradient-based optimizers, and using data-driven glass models that are more suited for optimization than traditional variables.
The new generation of sUAS (small Unmanned Aircraft Systems) aims to extend the range of scenarios in which sense-and-avoid functionality and autonomous operation can be used. Relying on navigation cameras, having a wide field of view can increase the coverage of the drone surroundings, allowing ideal fly path, optimal dynamic route planning and full situational awareness. The first part of this paper will discuss the trade-off space for camera hardware solution to improve vision performance. Severe constraints on size and weight, a situation common to all sUAS components, compete with low-light capabilities and pixel resolution. The second part will explore the benefits and impacts of specific wide-angle lens designs and of wide-angle images rectification (dewarping) on deep-learning methods. We show that distortion can be used to bring more information from the scene and how this extra information can increase the accuracy of learning-based computer vision algorithm. Finally, we present a study that aims at estimating the link between optical design criteria degradation (MTF) and neural network accuracy in the context of wide-angle lens, showing that higher MTF is not always linked to better results, thus helping to set better design targets for navigation lenses.
Data-driven approaches to lens design have only recently begun to emerge. One particular way in which machine learning, and more particularly deep learning, was applied to lens design was by smoothly extrapolating from lens design databases to provide high-quality starting points for lens designers. This mechanism is used by the web application LensNet (which will be publicly available shortly) whose goal is to provide high-quality starting points that are tailored to the desired specifications, namely the effective focal length, f-number and half field of view. Here, we evaluate more thoroughly the designs that are inferred by LensNet and its underlying deep neural network. We provide a global quantitative assessment of the viability of the designs as well as a more targeted comparison among specific design families such as Cooke triplets and Double-Gauss lenses between expert-designed lenses and their automatically inferred counterparts.
Most lens design problems involve the time-consuming task of finding a proper starting point, that is, a lens design that approximately fulfills the desired first-order specifications while decently correcting aberrations. In recent work, a fully-connected (FC) deep neural network was trained to learn this task by extrapolating from known lens design databases. Here, we introduce a new dynamic neural-network architecture for the starting point problem which is based on a recurrent neural network (RNN) architecture. As we show, the dynamic network can learn to infer good starting points on many lens design structures at once whereas the previous model was limited to a given sequence of glass elements and air gaps. We also show that a pretrained RNN model can generalize its knowledge over new lens design structures for which we have no reference lens design and obtain a significantly better optical performance than a RNN trained from scratch.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.