A better understanding of the color constancy mechanism in human color vision [7] can be reached through analyses of
photometric data of all illuminants and patches (Mondrians or other visible objects) involved in visual experiments. In
Part I [3] and in [4, 5 and 6] the integration in the human eye of the geometrical-optical imaging hardware and the
diffractive-optical hardware has been described and illustrated (Fig.1). This combined hardware represents the main
topic of the NAMIROS research project (nano- and micro- 3D gratings for optical sensors) [8] promoted and coordinated
by Corrsys 3D Sensors AG. The hardware relevant to (photopic) human color vision can be described as a diffractive or
interference-optical correlator transforming incident light into diffractive-optical RGB data and relating local RGB onto
global RGB data in the near-field behind the 'inverted' human retina. The relative differences at local/global RGB
interference-optical contrasts are available to photoreceptors (cones and rods) only after this optical pre-processing.
A better understanding of intelligent information processing in human vision can be reached through a closer look at the
macro- and micro-hardware available in the hierarchy of cortical processors along the main visual pathway connecting
the retina, the CGL (corpus geniculatum laterale) and area V1 (cortical visual area 17). The building of the eye is driven
by the brain and the engineering of the main visual pathway back to V1 seems to be driven by the eyes.
The human eye offers to the brain much more intelligent information about the outer visible world than a camera
producing flat 2D images on a CCD. Intelligent processing of visual information in human vision - a strong cooperation
between eyes and brain - relays on axes related symmetry operations relevant for navigation in 4D spectral space-times,
on a hierarchy of dynamically balanced equilibrium states, on diffractive-optical transformation of the Visible into RGB
space, on range mapping based on RGB data (monocular and binocular 3D vision), on illuminant-adaptive optical
correlations of local onto global RGB data (color constancy performances) and on invariant fourier-optical log-polar
processing of image data (generic object classification; identification of objects). These performances are more
compatible with optical processing of modern diffractive-optical sensors and interference-optical correlators than with
cameras. The R+D project NAMIROS (Nano- and Micro-3D gratings for Optical Sensors) [8], coordinated by an
interdisciplinary team of specialists at Corrsys 3D Sensors AG, describes the roadmap towards a technical realization of
outstanding high-tech performances corresponding to human eye-brain co-processing.
KEYWORDS: RGB color model, Eye, Color vision, Data modeling, 3D image processing, Retina, Optical correlators, Colorimetry, Visualization, Imaging systems
A correlator-optical imaging system with three-dimensional nano- and micro-structured diffraction gratings in aperture and in image space allows an adaptive optical correlation of local RGB data in image space with global RGB data (light from overall illumination in the visual field scattered from the aperture of the optical imaging system into image space), diffracted together into reciprocal grating space (photoreceptor space). This correlator-optical hardware seems to be a decisive part of the human eye and leads to new interpretations of color vision and of adaptive color constancy performances in human vision. In Part I, the up to now available data and their corresponding interpretation, together explaining paradoxically colored shadows as well as the data from E.Land's Retinex experiments, will be described. They will serve as premises for the planned experimental setup. In Part II these premises will experimentally be controlled by experiments in a part of an actually starting R+D project and the results will be described in 2007.
KEYWORDS: RGB color model, Diffraction, Cones, Optical correlators, Diffraction gratings, Visible radiation, Eye, Color vision, Colorimetry, Near field optics
3D space and time in optics and in human vision are linked together in spectral diffractive-optical transformations of the visible world. A 4D-RGB correlator hardware - integrated in an optical imaging system like the human eye - processes a hierarchy of relativistic equilibrium states and a sequence of double-cone transformations. The full chain of light-like events ends in von Laue interference maxima in reciprocal space, where 4D-RGB signals are miniaturized down to the level of individual photoreceptors. The diffractive-optical correlator relates local information to global data in the visual field and illustrates the potential of future development of cameras towards more intelligent 4D optical sensors.
KEYWORDS: RGB color model, Eye, Optical correlators, Color vision, Reflectivity, Diffraction, Retina, Data modeling, Matrix multiplication, Near field optics
Edwin Land--based on photometric data--tried to explain through a retinex (retina + cortex) model calculating scaled integrated reflectances, how human color vision determines the perceived hues of colored Mondrian patches by relating illuminants and 'energies at the eye' and including calculation over the whole image. An alternative--purely optical--model, the diffractive-optical correlator hardware in aperture and image space of the human eye, relating 'local' data onto 'global' data in color vision, becomes illustrated. Based on Edwin Land's experimental data it is shown how the perceived hues result from diffractive-optical transformations and cross-correlations between object space and reciprocal space (RGB space), from matrix multiplications and divisions in vector space. Optical pre-processing causes that our eyes do not see what (physically) is real, but what they optically have calculated. This same diffractive-optical mechanism has also lead to an explanation of the phenomenon of paradoxically colored shadows, shortly re-presented in the introduction.
KEYWORDS: RGB color model, Eye, Cones, 3D image processing, Diffraction gratings, 3D vision, Modulators, Human vision and color perception, Optical correlators, Near field optics
The human eye is a good model for the engineering of optical correlators. Three prominent intelligent functionalities in human vision could in the near future become realized by a new diffractive-optical hardware design of optical imaging sensors: (1) Illuminant-adaptive RGB-based color Vision, (2) Monocular 3D Vision based on RGB data processing, (3) Patchwise fourier-optical Object-Classification and Identification. The hardware design of the human eye has specific diffractive-optical elements (DOE's) in aperture and in image space and seems to execute the three jobs at -- or not far behind -- the loci of the images of objects.
Colored shadows are observed when two differently colored lights are combined in twilights. When both lights add to an equi-energy white 'balanced' spectrum, the hues of the shadows show regular opponent colors, being reciprocals of the colors of the lights. When a white and a colored light in a twilight add to an 'unbalanced' spectrum, the hues of the shadows result from the same laws of opponency and reciprocity, but the eyes see "what they have optically calculated" instead of seeing "what really (physically) there is". At adaptation to the colored light, unbalanced states in physics become physiologically re-balanced by diffractive-optical chromatic resonance, guaranteeing color constancy at variations of illuminants. Colored shadows can be interpreted as serial products of diffractive 3D grating-optical von Laue interferences and of optical cross-correlations between local and global information in the human eye. The human eye's hardware, with diffractive-optical multi-layer gratings in aperture and image space, represents an illuminant-adaptive diffractive-optical RGB Color Sensor guaranteeing color space normalization towards RGB equilibrium states (RGB white norms) in reciprocal grating space.
The analysis of the prenatal engineering of the human brain and more specifically that of the human eye may encourage new interpretations and better understanding of cortical processors and lead to better ideas about how to build optical sensors. What human vision at its first processing stages realizes is an adaptive transformation of physical parameters from an outer 4D-spatiotemporal into an inner psychological world or its reciprocal projection and construction of an illusionary (inner or outer) world. The description of some of the most remarkable steps in the development of the human eye before birth, very critical for the optical functionalities in vision, will illustrate the new interpretations.
Intelligent image processing will take a more and more important part of the future opto-electronic sensor business. To filter relevant information out of a large data base and to memorize selectively the most important messages become critical features and the camera-like systems cannot reach the goal. A cortical image processor, similar to how human vision and perception are functioning, is requested. This particularly in a world where global video- communication becomes a ubiquitous standard. The search for new filtering methods and more intelligent visual information processors therefore remains an item of topical interest for R and D activities.
KEYWORDS: Diffraction gratings, Colorimetry, Retina, Diffraction, Human vision and color perception, Visible radiation, Sensors, 3D image processing, 3D imaging standards, Cones
Since 0. Lummer [6] and the industrial development and production of artificial illuminants it became more and more evident that between sunlight and human vision a specific — until today unexplained — "resonance" condition apparently exists. Therefore the recommendation holds to approximate as much as possible the spectral energy distribution of artificial illuminants to the one of sunlight. Especially in human color vision spectral shifts of illuminants always lead to hue shifts (cornbined Brightness-, Hue-, Saturation-Shifts) in the perception of colors. These hue shifts in human vision adaptively become compensated with more or less time delay, leading to a relatively good "color constancy" under variable illurninants. An — always far from perfect — explanation model, the von Kries-model, attributes this adaptive compensation of hue shifts to the photopigments in the cones of the human retina. Other — less perfect — models attributing this adaptation to cortical functions also exist [1, 2]. In parallel the need becomes evident to realize future color sensors "capable to measure colors normalized to the spectral sensitivity curves of human vision" [7]. It might be registered with satisfaction that a growing objectivity comes into this psychophysical field of color constancy in human vision by the publication of more and more precise data on relevant parameters in the physical conditions of the experiments.
KEYWORDS: Diffraction gratings, 3D image processing, Diffraction, RGB color model, Human vision and color perception, Eye, 3D vision, Cones, Visualization, Retina
Diffractive 2D and 3D grating optics as enabling technologies are on a good way to technically realize specific features well known in human vision. How does the human eye do its job in visual information processing?
Innovative technology excels by realizing extraordinary solutions for well-known problems. The grating-optical correlation measurement technology promotes Opto-Electronics into Opto2, and Opto3, and Opton-Electronics enabling precise non-contact, non-slip acquisition of data from length and speed information, to information data about directions of motion, distances, and multiple distances in 1, 2 and 3 directions all the way up to image preprocessing on vehicles, robots, production machines, and aides for blind individuals. This paper describes the technological steps of these developments.
Diffractive 3D phase gratings of spherical scatterers dense in hexagonal packing geometry represent adaptively tunable 4D-spatiotemporal filters with trichromatic resonance in visible spectrum. They are described in the (lambda) - chromatic and the reciprocal (nu) -aspects by reciprocal geometric translations of the lightlike Pythagoras theorem, and by the direction cosine for double cones. The most elementary resonance condition in the lightlike Pythagoras theorem is given by the transformation of the grating constants gx, gy, gz of the hexagonal 3D grating to (lambda) h1h2h3 equals (lambda) 111 with cos (alpha) equals 0.5. Through normalization of the chromaticity in the von Laue-interferences to (lambda) 111, the (nu) (lambda) equals (lambda) h1h2h3/(lambda) 111-factor of phase velocity becomes the crucial resonance factor, the 'regulating device' of the spatiotemporal interaction between 3D grating and light, space and time. In the reciprocal space equal/unequal weights and times in spectral metrics result at positions of interference maxima defined by hyperbolas and circles. A database becomes built up by optical interference for trichromatic image preprocessing, motion detection in vector space, multiple range data analysis, patchwide multiple correlations in the spatial frequency spectrum, etc.
3D multilayer phase gratings (nuclear layers in the human retina) of oscillating cells positioned in the image plane of a monocular optical imaging system transform light double cones by diffraction and pulsation (quantization) into the reciprocal space behind the grating (Fourier-space in the Fresnel-nearfield). This effect we call OPTORETINA not only guarantees the transformation of the physical stimuli parameters (intensity/wavelength) in the visible spectrum into three chromatically tuned adaptive RGB-color channels (diffraction orders with Brightness-Hue-Saturation aspects) and of object distances in 3D-space into spatial frequencies or temporal phase differences in reciprocal space (the MULTIDIST grating optical sensors developed at CORRSYS).
Inheritance or training in human vision? Training allows individual adaptation to environmental conditions, while genetics determine optimized constant i.e. long-term abilities. The biological engineering of the human eye needs both inheritance and training to realize its high performances. Particularly during postnatal training of an eye, adaptive freedom is necessary and available. To test the part of training it would become necessary to experimentally determine the in vivo refractive index differences between cellular nuclei and cytoplasm in retinal nuclear layers before and after birth to see if diffractive optical tuning of trichromatism in a retinal 3D-grating is synchronized with the differentiation of 3 photopigments in photopic vision or if the specialization in photochemistry depends on crystal- optical preprocessing.
We present a number of applications of optical grating technology for use in machine vision and industrial inspection and measurement applications. The gratings are based on an array of prisms that are both very easy and very cheap to produce. Currently optical gratings are used in industry in temporal/spatial correlation systems using non-coherent white- light illumination. Such systems require either the displacement of the object or the displacement of the grating in order to achieve the time signals required for evaluation. New developments in grating manufacturing techniques and electronic signal processing now enable us to electronically simulate and generate the signals required eliminating the requirement for grating or object motion. We also present a range sensor based on a split pupil methodology upon which we are developing a multiple-channel range sensor. We also highlight some of the work we are presently pursuing in multiple-layer gratings. We are investigating the Talbot and Lau self-imaging effects with hopes of using the results to preprocess images. We have also calculated the effect of multiple Bragg planes recorded into holographic materials, which can be used to produce local image transformations. Also summarized is some of the work based on an inverted retina model of the eye which uses multi-layer gratings placed before the photoreceptors to general trichromatic separation of light and explain a number of physiological effects of vision.
We present an optical filter device doing local transformations. This is achieved by recording various sets of Bragg planes in a volume hologram with each set tilted by a different small angle. The incoming light is diffracted by such a bundle of planes resulting in a process which is similar to the refraction of light by a lens. The property of this optical transform is that the transform plane is located directly behind the filter plane. Spatially localized transformations of the input amplitude distribution are achieved. We calculate plane bundles and the field distribution behind the volume hologram using the coupled wave theory. The functionality of the principle is demonstrated for a simple example. This new filtering method results in a local transformation property which is performed in real time; such a filter can be placed directly on a detector array such as a CCD chip achieving a very robust and compact arrangement.
KEYWORDS: Cones, Retina, Near field optics, Integration, Eye, Human vision and color perception, Feature extraction, Optical imaging, Geometrical optics, Near field
The interpretation of the 'inverted' retina of primates as an 'optoretina' (a light cones transforming diffractive cellular 3D-phase grating) integrates the functional, structural, and oscillatory aspects of a cortical layer. It is therefore relevant to consider prenatal developments as a basis of the macro- and micro-geometry of the inner eye. This geometry becomes relevant for the postnatal trichromatic synchrony organization (TSO) as well as the adaptive levels of human vision. It is shown that the functional performances, the trichromatism in photopic vision, the monocular spatiotemporal 3D- and 4D-motion detection, as well as the Fourier optical image transformation with extraction of invariances all become possible. To transform light cones into reciprocal gratings especially the spectral phase conditions in the eikonal of the geometrical optical imaging before the retinal 3D-grating become relevant first, then in the von Laue resp. reciprocal von Laue equation for 3D-grating optics inside the grating and finally in the periodicity of Talbot-2/Fresnel-planes in the near-field behind the grating. It is becoming possible to technically realize -- at least in some specific aspects -- such a cortical optoretina sensor element with its typical hexagonal-concentric structure which leads to these visual functions.
If the nuclear retinal layers of the human eye are interpreted as 3D phase gratings, the aperture effects in human vision, namely the Stiles-Crawford effects I and II and trichromatic vision, can be explained in terms of interference optics. A multilayer grating situated in the image plane of the eye fixes the direction of the diffraction orders through its 3D geometry. The ratio between (lambda) ' or (nu) ' of the light cone incident at an angle and (lambda) and (nu) of the cone incident at 0 degrees can thus be differentiated as a brightness, hue and saturation shift in 3 chromatic RGB diffraction orders in the near field behind the grating, thus providing information on the relative position, distance, 3D shape and movement of objects in the 3D space. The direction cosine of the light cones in the von Laue equation means that lateral distances and movements relative to the visual axis and longitudinal movements relative to the focused distance give rise to the aperture effects and a space-time microrelief of the 3D world. This is regarded as an optical basis for monocular spatial vision and motion detection. Temporal patterns in human vision therefore produce spatial patterns and movement information and vice versa. The intrinsic oscillations of the nuclear layers transform the constant interference-optical object representations become possible. The interference-optical local lateral connections and the retinal feedback NN structure allow the possibility of parallel-optical image correlation in real time. The psychophysical transformations in the retinal 3D grating correspond to an image transformation into a reciprocal grating; the retinal clock is set adaptively by means of (lambda) max of the 111 diffraction order and via the trichromatic white standard.
KEYWORDS: Retina, Diffraction, Diffraction gratings, Human vision and color perception, Visible radiation, Colorimetry, Cones, Color vision, Receptors, 3D vision
The nonlinear relationship between brightness, hue and saturation in human vision becomes clear if, in addition to the pupil as brightness regulator, the inverted retina is interpreted as a cellular multilayer phase grating optical 3D chip, i.e. as a chromaticity and brightness regulator. Both regulators are optical information preprocessors which determine the signal input into the photoreceptors and thus represent the basis for subsequent electrical information processing in retinal neural networks. Data from the interference-optical 3D phase calculation (von Laue equation) are compared with experimental data on phenomena in human vision which are critical to this question, to show the interdependence of brightness, hue and saturation. This gives new insights into the function of the pupil, the Purkinje shift, the Bezold-Brucke phenomenon, the Stiles-Crawford aperture effects I/II and the saturation effects in human vision, all of which can be derived from a single pupil/retina/photopigment equation.
KEYWORDS: Retina, Diffraction, Diffraction gratings, 3D vision, Visible radiation, Color vision, Human vision and color perception, 3D imaging standards, Phase velocity, Eye
In photopic vision, two physical variables (luminance and wavelength) are transformed into three psychological variables (brightness, hue, and saturation). Following on from 3D grating optical explanations of aperture effects (Stiles-Crawford effects SCE I and II), all three variables can be explained via a single 3D chip effect. The 3D grating optical calculations are carried out using the classical von Laue equation and demonstrated using the example of two experimentally confirmed observations in human vision: saturation effects for monochromatic test lights between 485 and 510 nm in the SCE II and the fact that many test lights reverse their hue shift in the SCE II when changing from moderate to high luminances compared with that on changing from low to medium luminances. At the same time, information is obtained on the transition from the trichromatic color system in the retina to the opponent color system.
The question of why the human eye has two axes, a photopic visual axis and an eye axis, is just as justified as the one of why the fovea is not on the eye axis, but instead is on the visual axis. An optical engineer would have omitted the second axis and placed the fovea on the eye axis. The answer to the question of why the design of the real eye differs from the logic of the engineer is found in its prenatal development. The biaxial design was the only possible consequence of the decision to invert the retinal layers. Accordingly, this is of considerable importance. It in turn forms the basis of the interpretation of the retinal nuclear layers as a cellular 3D phase grating, and can provide a diffraction-optical interpretation of adaptive effects (Purkinje shift), aperture phenomena (Stiles-Crawford effects I and II) in photopic vision, and visual acuity data in photopic and scotopic vision.
As the design of contact eye lenses becomes more sophisticated it also becomes desirable to have an accurate measure of the shape of both the lens surfaces. The use of stylus instruments is undesirable because of the risk of leaving permanent marks or damage to the surface and the slow speed of measurement. The interferometrically based techniques are generally unsuitable due to the difficulty of obtaining an independent measure of shape of each surface without the effect of the other or the refractive material of the lens. Also the dimensions and gradients of the lenses are out of range for most commercially available optically based relief measuring instruments. To overcome these problems a projected fringe, phase-profilometry technique' has been extended and adapted for precision measurement of lens shape. A dynamic range of 1500:1 in relief measurement has been achieved by the combination of better quality gratings, the use of phase unwrapping procedures and improved coating of surfaces. The asphericity of the lens surface is evaluated by subtracting a fitted spherical surface from the measured relief. As an example, the results for a contact lens are presented.
A cellular multilayer phase grating with hexagonal closest packing proves to be the ideal focal plane architecture for the human eye, and is thus also the best model for designing stimulus- adaptive robot eyes which achieve the spatial and chromatic performance of the human eye. Crystal-optical calculation of the retinal cellular multilayer chip and the resulting correlations between the physical stimulus parameters and the adaptive shifts in human vision at the retinal level give rise to a time-frequency diagram of the eye and its stimulus-adaptive latitudes, which will become relevant in the design of future chips for robot eyes with performance comparable to that of human vision. The current presentation shows that 3-D grating optical parameters ensure the frequency-related chromatic adaptive shifts (transition from photopic to scotopic vision in the Purkinje shift, Stiles-Crawford effects I/II, Bezold-Bruecke phenomenon, chromatic adaptation to artificial light sources of different spectral composition, etc.) and also indicates what 3-D grating optical parameters are relevant to spatial transfer and adaptation, i.e., the time-related aspects in the time-frequency diagram (adaptation of the spatial modulation transfer function to the image parameters; log term for spatial adaptation to the intensity level; coding of spatial phase relationships between a fundamental spatial frequency and higher frequencies up to the third harmonic, etc.).
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.