Applying optical methods to a simple inspection problem, such as the presence or absence of an assembly component, is likely the least restrictive of any machine vision application. The setup, alignment, and components can be made very simply by using any feature which appears different if the component of interest is present. Gaging to verify proper construction, however, can be a much more complicated task. To take advantage of the capability of machine vision and other optical technologies, some flexibility in part position, orientation, and speed of presentation may be desired. This paper will review the types of methods available from optical methods for industrial dimensional metrology. The emphasis of the discussion will be 2D and 3D gaging applications.

We review some of our most recent works on 3-D shape measurement using the digital fringe projection and phase-shifting method. First, we introduce the measurement principle and phase-shifting algorithms. Then we discuss an effective method for phase error compensation and a novel idea for system calibration. Finally, we describe a 3-D shape measurement system for high-resolution, real-time 3-D shape acquisition, reconstruction and display.

Hitherto, it has been assumed that an industrial Machine Vision systems is constructed as an integrated unit, with the camera, image processing unit and control/display console being located close to one another and to the object/scene being inspected. For several reasons, it may be helpful to separate them, so that only the camera and its associated lighting units are located on the factory floor, while other equipment, such as computers and user terminals, is located elsewhere, out of harm's way. We describe three systems that allow multiple cameras and several separate image processing engines, to be controlled remotely from a single "intelligent" device, or a user working via a standard web browser. The paper describes and compares several different approaches to building such a system. Links to the networked vision systems mentioned here are provided in an accompanying web site.

Characterizing the surface texture of objects or moving webs is sometimes important in identifying and/or determining the quality of products. Fourier analysis is often used for this in laboratory situations, but inspection algorithms using the FFT are too slow for many production situations, even when implemented on fast computers. An alternative technique pioneered in the 1970s by Haralick [1] operates in the time domain and uses grey-level cooccurrence matrices (GLCMs) as a first step toward obtaining useful measures characterizing textures. Although the GLCM approach is much less computationally-intensive than the FFT, it nonetheless requires massive amounts of calculation. Most of this computation time is spent in stepping through the input image and compiling the matrices themselves. Therefore, if the calculation time for these matrices could be reduced, the GLCM technique would become more practical. This paper applies the SKIPSM paradigm to the calculation of GLCMs, and provides execution times for this implementation.

In a typical machine vision algorithm, a few complex operators may account for a significant fraction of the overall execution time. In addition to these, there are usually many simple 3x3 operators which are used over and over. Although individually fairly fast, these operators sometimes dominate the overall execution time because they are used so many times. And because they are fairly fast, little attention is paid to speeding them up. The many previous papers describing the SKIPSM paradigm have concentrated mainly on large-neighborhood operations, because speed improvements are the most dramatic in such cases. In this paper, SKIPSM implementations of some common 3x3 linearly-separable and nonseparable operators are considered. Examples include low-pass (i.e., blurring and smoothing) filters, band-pass filters, high-pass (i.e., edge detector) filters, and gradient operators. Speed comparisons between conventional implementations and SKIPSM implementations are presented.

Plasma nitriding belongs to the group of the thermo chemical surface heat treatments. During this process nitrogen is dissociated into the surface of the material increasing hardness, wear resistance, endurance strength and/or corrosion resistance. This paper presents a new inspection system based on a CCD camera system for monitoring such heat treatment processes (PACVD, plasma assisted chemical vapour deposition). Treatment temperatures commonly used are within the range of 350^oC to 600^oC. A near infrared enhanced CCD camera system equipped with specifically chosen spectral filters is used to measure spectral emittances during the surface modification. In particular the spectral operating range of 950nm to 1150nm of the silicon CCD camera is utilized. The measurement system is based on the principles of ratio pyrometry (dual-band method) known from non-contact temperature measurements, in which two images of the same scene, each taken at slightly different spectral bands, are used to determine the spectral light characteristics. This results in an improved relative sensitivity for spectral changes (i.e. deviations from the gray-body hypothesis) during the surface modification.

Continuous monitoring and control of process temperature(s) is one of the cornerstones in high quality steel making. Given the very high temperatures in the liquid phase of the steel and the slag on top of the steel (approx. 1500^oC...1800^oC) and the particularly harsh environment at the manufacturing plant, only very few temperature sensors are able to cope with the process requirements, in particular a wide variety of thermocouple probes and pyrometers are commonly used. More recently thermography infrared cameras have begun to enter the scenario but are often discarded as an option mainly because of their high cost. In the high temperature range as described above a dual wavelength camera solution working in the visible part of the spectrum offers a viable alternative¹. At a fraction of the cost such a system can deliver images of high spatial resolution while at the same time measuring temperature with an accuracy of better than 5^oC. The thermal camera approach is particularly beneficial in the present case where important process information can be deducted from quantitative observation of the flow patterns of the molten material which could until now only be estimated by a trained operator with all the drawbacks inherent to such an approach. The thermal camera solution thus offers a clear technological advantage for the steel manufacturer.

In this paper, we present a printed circuit board (PCB) inspection system based on using Hausdorff distance for image alignment and defect detection. In addition, we apply support vector machine (SVM) for the defect classification and the metal classification in this system. The three major components in the proposed PCB inspection system consist of image alignment, defect detection, and defect classification. In image alignment, a coarse-to-fine search technique is applied to accelerate the speed of finding the minimal Hausdorff distance between the reference and the inspection images. For defect detection, we calculate the Hausdorff distance of every pixel in the inspection image as the first step and compare the result with a predefined threshold. For the cases where the computed Hausdorff distance is greater than the threshold, the location of that pixel is labeled as a defect suspect. The existence of defect then can be confirmed by merging the nearby suspects into one object. For defect classification, the local image features are extracted and passed to support vector machine for training and identifying defect types. In this work, we focus on distinguishing the type of a defect as one of open, short, pinhole, over-etch, or under-etch types. Support vector machine can be applied to metal classification as well. At the current stage, we supply support vector machine with RGB color information as the feature vector for metal classification. Experimental results show that the Hausdorff distance based method detects defects in a printed circuit board efficiently and accurately, and the support vector machine approach also gives satisfactory results for both defect and metal classifications.

In this paper, several binary mask based Depth From Defocus (DFD) algorithms are proposed to improve autofocusing performance and robustness. A binary mask is defined by thresholding image Laplacian to remove unreliable points with low Signal-to-Noise Ratio (SNR). Three different DFD schemes-- with/without spatial integration and with/without squaring-- are investigated and evaluated, both through simulation and actual experiments. The actual experiments use a large variety of objects including very low contrast Ogata test charts. Experimental results show that autofocusing RMS step error is less than 2.6 lens steps, which corresponds to 1.73%. Although our discussion in this paper is mainly focused on a spatial domain method STM1, this technique should be of general value for different approaches such as STM2 and other spatial domain based algorithms.

Real-time and accurate autofocusing of stationary and moving objects is an important problem in modern digital cameras. Depth From Defocus (DFD) is a technique for autofocusing that needs only two or three images recorded with different camera parameters. In practice, there exist many factors that affect the performance of DFD algorithms, such as nonlinear sensor response, lens vignetting, and magnification variation. In this paper, we present calibration methods and algorithms for these three factors. Their correctness and effects on the performance of DFD have been investigated with experiments.

A confocal microscope can achieve superior contrast when imaging a certain layer in the bulky samples. However, the parallel signal collection feature of the optical system is sacrificed when the sample is scanned pixel by pixel. In this paper, we proposed a novel confocal microscope design that uses a time and spatially multiplexed method, which dramatically increases the time resolution of a confocal microscope. This design has been used to solve a long-standing problem in cardiac research whether or not a small submembranous domain exists with calcium and sodium ion concentrations significantly different from those measured in bulk cytosol. We applied our time and spatially multiplexed confocal microscope to obtain the transient 3-D distribution of calcium ion concentration in rat cardiac myocytes. Our experimental results prove the feasibility of the technique and also demonstrate the huge potential of this design.

Pattern projection using physical gratings or interference effects has successfully been used to perform 3D measurements of parts. However, such systems lack the flexibility to adjust light levels over the area illuminated, leaving some areas too dark or too light to measure, or the ability to mask out parts of the illumination field, often creating spurious reflections and noise from areas not of interest. LCD/DMD digital projection systems have been used to create flexible projection of patterns, but they are limited in their resolution and ability to accurately reconstruct a smooth light pattern such as a sine wave, creating more a binarized approximation. This paper describes a method that combines together a computer-interfaced projector, such as an LCD or DMD based projector, with a high-resolution pattern projection system. The result is a system that has high depth resolution, but with the added flexibility of a programmable light source to control light levels and areas of illumination. This paper will discuss the pros and cons of this method, and suggest ways this approach might be applied to difficult part measurement problems.

In manufacturing, inspection and measurement systems have long been desired to be able to measure as many kinds of parts as possible without treating the surface. Specifically, measuring shiny parts has been a big challenge for optical metrology because of double-bounced light -- a phenomenon that light can be reflected from an area to another on the surface. The unwanted light will result in higher noise and can even make the measured results unacceptable. Traditionally, a polarizer is placed in front of both the light sources and the camera. After properly adjusting the polarizer in front of the camera, the double bounced reflected light can be blocked to some degree while the normally reflected light can go through. By this way, the extra reflections can be reduced but not totally eliminated. This paper presents a new method to totally eliminate double bounced light. Here, color light sources are used to illuminate the part and multiple cameras are used to measure different areas. Each camera views through an appropriate color filter so that only a certain color light is seen. The measured results from all cameras are then merged together to create the complete image. This method is more efficient than the traditional solution that uses polarizers. Both measurement principle and some results are given.

This paper describes a novel phase error compensation method for reducing the measurement error caused by non-sinusoidal waveforms in the phase-shifting method. For 3D shape measurement systems using commercial video projectors, the non-sinusoidal nature of the projected fringe patterns as a result of the nonlinear gamma curve of the projectors causes significant phase measurement error and therefore shape measurement error. The proposed phase error compensation method is based on our finding that the phase error due to the non-sinusoidal waveform of the fringe patterns depends only on the nonlinearity of the projector's gamma curve. Therefore, if the projector's gamma curve is calibrated and the phase error due to the nonlinearity of the gamma curve is calculated, a look-up-table (LUT) that stores the phase error can be constructed for error compensation. Our experimental results demonstrate that by using the proposed method, the measurement error can be reduced by 10 times. In addition to phase error compensation, a similar method is also proposed to correct the nonsinusoidality of the fringe patterns for the purpose of generating a more accurate flat image of the object for texture mapping. While not relevant to applications in metrology, texture mapping is important for applications in computer vision and computer graphics.

We propose a new three-step phase-shifting algorithm, which is much faster than the traditional three-step algorithm. We achieve the speed advantage by using a simple intensity ratio function to replace the arctangent function in the traditional algorithm. The phase error caused by this new algorithm is compensated for by use of a look-up-table (LUT). Our experimental results show that both the new algorithm and the traditional algorithm generate similar results, but the new algorithm is 3.4 times faster. By implementing this new algorithm in a high-resolution, real-time 3D shape measurement system, we were able to achieve a measurement speed of 40 frames per second (fps) at a resolution of 532 × 500 pixels, all with an ordinary personal computer.

With the growing demand for high speed and dynamic 3D profiling for applications such as animation, face recognition and machine vision, we have seen Digital Video Projection (DVP) emerge as the preferred projection technology. In addition to the robust qualities DVP sources offer, many researchers are adopting the technology on the merits of higher resolutions, contrast ratios and sharper images, all of which are characteristic of the highly developing projection technology. However, in regard to the specific application of triangulation based optical profilometers, DVP lacks predominantly in pattern geometric structure and also suffers from colour channel coupling issues. The purpose of this paper is to review the functionality and contrast the two chief DVP technologies, Liquid Crystal Display (LCD) and Digital Light Processing (DLP) from the specific viewpoint of a triangulation based phase measuring profilometer. Experimental verification of the performance of each projection technology is provided and finally some suggestions on projection technologies for single and multichannel phase measuring profilometry applications is offered.

This paper describes a liquid crystal (LC) device for three dimensional profile measurement systems which are based on grating projection method using phase shifting technique. As a key component to these sytems, we propose to apply a liquid crystal (LC) grating instead of a conventional ruled grating because the grating ruled on the glass plate has difficulty in speedy and accurate shifting of the pattern. This LC grating consists of 960 lines of stripe pattern on the substrate of 60x40 mm² in size and has such features as 8 bits of gray levels in dynamic range for mono-chromatic usage. A sinusoidal pattern as well as a binary pattern is realized by combining pulse width modulation control (PWMC) and frame ratio control (FRC) technique. The period of the pattern is arbitrarily controlled and, in addition, shifting of the projected pattern is also electrically realized. We present properties of the LC grating we have developed and demonstrate a few examples obtained by the system which has this LC grating built in.

Recently, various investigations have been carried out using grating projection method based on triangulation. It has advantages such as simple optical arrangement and full-field measurement with high accuracy, and moreover measurement time is shorter compared with other methods like point and/or line scanning. However, it has such problems as occlusion and phase error due to halation. The first problem is inevitable due to principle of triangulation. The second problem is caused mainly by unequal surface reflectivity of the BGA, CSP and solder bumps, etc. Therefore we propose dual projection method to solve these problems. This proposal consists of one camera and dual projection units with specified liquid crystal gratings. These problems can be avoided owing to measurement from two directions. Finally, three-dimensional profiles are obtained by combining these two results. We intend to present a method that can extend the use of fringe projection method based on triangulation. In addition to the principle of this system, BGA sample applications are to be shown up.

This paper describes a surface profile measurement using a varifocus lens by an optical sectioning. The focus method is utilized for a uni-axis optical system of projection and observation. The varifocus lens is mounted on between a liquid crystal grating and a projection lens. A focus length can be continuously varied from a concave to a convex shape by changing the liquid pressure. The contrast of projected pattern onto the sample is approximated the Gauss distribution along the distance and indicates sharpest at the focused plane. It is possible to analyze the contrast distribution by a grating projected method using a liquid crystal grating with 4 steps phase-shifting method. The liquid crystal grating is powerful tool to make arbitrary intensity and frequency distribution. Surface profiles of mechanical parts have been measured to demonstrate for this method.

A laser range imaging system based on the triangulation method was designed and implemented for online high-resolution thickness calculation of poultry fillets. A laser pattern was projected onto the surface of the chicken fillet for calculation of the thickness of the meat. Because chicken fillets are relatively loosely-structured material, a laser light easily penetrates the meat, and scattering occurs both at and under the surface. When laser light is scattered under the surface it is reflected back and further blurs the laser line sharpness. To accurately calculate the thickness of the object, the light transportation has to be considered. In the system, the Bidirectional Reflectance Distribution Function (BSSRDF) was used to model the light transportation and the light pattern reflected into the cameras. BSSRDF gives the reflectance of a target as a function of illumination geometry and viewing geometry. Based on this function, an empirical method has been developed and it has been proven that this method can be used to accurately calculate the thickness of the object from a scattered laser profile. The laser range system is designed as a sub-system that complements the X-ray bone inspection system for non-invasive detection of hazardous materials in boneless poultry meat with irregular thickness.

Wear of railway wheels - especially for engines - has to be checked in regular intervals. For this check, some key dimensions of the cross section of the running surface like flange height, flange gradient or flange thickness have to be measured with accuracies in the range of 0.1 mm. State of the art is either the use of mechanical gauges or stationary optical measurement systems where the carriage has to be driven to a maintenance stand in a servicing centre. We present a new technique for building up a small portable hand held measurement system, which allows to perform data acquisition and dimension evaluation in an easy and convenient way by simply freehand scanning the interesting section of the wheel. The technique is based on the laser light sectioning technique. A special setup with multiple laser lines in combination with appropriate algorithms allows to correct errors due to inaccurate sensor slant as well as fusion of measurement data to generate a cross section of the wheel out from multiple partial measurements.

New needs to determine the crystallography of nanocrystals arise with the advent of science and engineering on the nanometer scale. Direct space high-resolution phase-contrast transmission electron microscopy (HRTEM) and atomic resolution Z-contrast scanning TEM (Z-STEM), when combined with tools for image-based nanocrystallography possess the capacity to meet these needs. This paper introduces such a tool, i.e. fringe fingerprinting in two dimensions (2D), for the identification of unknown nanocrystal phases and compares this method briefly to qualitative standard powder X-ray diffractometry (i.e. spatial frequency fingerprinting). Free-access crystallographic databases are also discussed because the whole fingerprinting concept is only viable if there are comprehensive databases to support the identification of an unknown nanocrystal phase. This discussion provides the rationale for our ongoing development of a dedicated free-access Nano-Crystallography Database (NCD) that contains comprehensive information on both nanocrystal structures and morphologies. The current status of the NCD project and plans for its future developments are briefly outlined. Although feasible in contemporary HRTEMs and Z-STEMs, fringe fingerprinting in 2D (and image-based nanocrystallography in general) will become much more viable with the increased availability of aberration-corrected transmission electron microscopes. When the image acquisition and interpretation are, in addition, automated in such microscopes, fringe fingerprinting in 2D will be able to compete with powder X-ray diffraction for the identification of unknown nanocrystal phases on a routine basis. Since it possesses a range of advantages over powder X-ray diffractometry, e.g., fringe fingerprint plots contain much more information for the identification of an unknown crystal phase, fringe fingerprinting in 2D may then capture a significant part of the nanocrystal metrology market.

Investigation of dynamics of microelectromechanical systems (MEMS) is an important problem from the point view of engineering, technology and metrology. Due to high surface to volume ratio of micro-electromechanical systems (MEMS), more attention must be paid to control their surface characteristics. Time average optical holography - has found many industrial applications and still is a promising method (like others: laser interferometry in small object displacement analysis.) This technique can reveal the shape, direction, and magnitude of the stress induced displacement in the structure under study. In this way, time average holography is a powerful tool for analysis of micro scale vibrations. The threshold of sensitivity of this measurement technique is defined by the magnitude of the wavelength of the illuminating laser beam. Also, this is a full field non-destructive technique capable to register the motion of the whole surface instead of a single point. The time average holography method is proposed to control kinetics of oscillations of the micro scale object, operating at the different amplitudes of periodical excitation. Theoretical calculations as well as experimental verification are described.

Holographic measurement techniques based on optical interferometry theory are widely applied in such areas as mechanics, electronics, computer industry, etc. One of limitation of practical method application is complicated interpretation of holographic interferograms because quantitive analysis is directly related with properties of the analyzed object and with structure of experiment itself. The optical measurement (holographic interferometry) data processing system is presented for identification of characteristics of dynamic of 3D components of mechatronic construction. The system uses optical view and mathematical model of experiment as input data producing the qualitative description of analyzed mechatronic system. Mathematical model of the experiment is built using the geometry of the measurement, elasticity theory and finite element method. The proposed method for identification of the properties of the analysed system has important advantages over other similar digital hologram interpretation schemes as the applied calculation principle is based on the natural eigenmodes of the analysed system. Such approach enables to decrease the volume of calculations without losing accuracy.

Typical nanometrology device standards for scanning probe microscopes that are produced by the top-down approach are described first. This is followed by a discussion of some of the atomic precision standards that nature provides. Since there is an order of magnitude feature size gap between these two classes of nanometrology standards, a novel class of nanometrology device standards that fills this gap is proposed together with its intrinsically inexpensive "bottom-up" fabrication process. This nanofabrication process is heteroepitaxy of semiconductors or ceramics that are morphologically, structurally, and chemically stable in typical laboratory environments. For special crystallographic orientations of the substrates and special material combinations, the heteroepitaxy process leads to self-assembled arrays of nano-islands with a known morphology, dimensions in the range of a few nanometers to a few tens of nanometers, and large height-to-width aspect ratios. Suitable crystallographic directions are then marked macroscopically on the nanometrology device standards in order to be able to direct the scanning probe scans in certain directions for precise calibration procedures.

Machine vision methods are widely used in apple defect detection and quality grading applications. Currently, 2D near-infrared (NIR) imaging of apples is often used to detect apple defects because the image intensity of defects is different from normal apple parts. However, a drawback of this method is that the apple calyx also exhibits similar image intensity to the apple defects. Since an apple calyx often appears in the NIR image, the false alarm rate is high with the 2D NIR imaging method. In this paper, a 2D NIR imaging method is extended to a 3D reconstruction so that the apple calyx can be differentiated from apple defects according to their different 3D depth information. The Lambertian model is used to evaluate the reflectance map of the apple surface, and then Pentland's Shape-From-Shading (SFS) method is applied to reconstruct the 3D surface information of the apple based on Fast Fourier Transform (FFT). Pentland's method is directly derived from human perception properties, making it close to the way human eyes recover 3D information from a 2D scene. In addition, the FFT reduces the computation time significantly. The reconstructed 3D apple surface maps are shown in the results, and different depths of apple calyx and defects are obtained correctly.

A stereovision-based disparity evaluation algorithm was developed for rice crop field recognition. The gray level intensities and the correlation relation were integrated to produce the disparities of stereo-images. The surface of ground and rice were though as two rough planes, but their disparities waved in a narrow range. The cut/uncut edges of rice crops were first detected and track through the images. We used a step model to locate those edge positions. The points besides the edges were matched respectively to get disparity values using area correlation method. The 3D camera coordinates were computed based on those disparities. The vehicle coordinates were obtained by multiplying the 3D camera coordinates with a transform formula. It has been implemented on an agricultural robot and evaluated in rice crop field with straight rows. The results indicated that the developed stereovision navigation system is capable of reconstructing the field image.

The characterization, porosity, permeability and integrity of conductive and non-conductive medical and consumer flexible barrier packaging material are determined utilizing a novel electron beam technology and electronic instrumentation in an open atmosphere for 100% real-time, on-line testing. The electron beam developed in an open atmosphere maintains its prescribed frequency through the use of a nitrogen cover gas, ionizing the gas to create a corona beam. The corona beam discharge, maintained at a high negative voltage, forms from the holes or anomalies in the flexible barrier material. The anomaly is detected and analyzed in order to determine the presence of viral and sub-viral sized voids or holes, as well as other anomalies such as blisters and bubbles. The process can also utilize an established range of acceptability to certify materials that require a well defined level of permeability. This process can be performed by the flexible barrier film manufacturer to certify a specific quality level. It can be performed by the material fabricator to ensure quality standards for preformed materials. It can also be performed by the product packaging manufacturer that uses the packaging material to wrap their products and confirm the integrity of the final sealed package by measuring the atmosphere inside the finished package. There are many other packaging applications that can utilize this technology for film characterization, validation and integrity testing within the pharmaceutical, medical device, and food processing industries, as well as other industrial applications.