In this paper we are examining the issue of overtraining in Fuzzy ARTMAP. Over-training in Fuzzy ARTMAP manifests itself in two different ways: (a) it degrades the generalization performance of Fuzzy ARTMAP as training progresses, and (b) it creates unnecessarily large Fuzzy ARTMAP neural network architectures. In this work we are demonstrating that overtraining happens in Fuzzy ARTMAP and we propose an old remedy for its cure: cross-validation. In our experiments we compare the performance of Fuzzy ARTMAP that is trained (i) until the completion of training, (ii) for one epoch, and (iii) until its performance on a validation set is maximized. The experiments were performed on artificial and real databases. The conclusion derived from these experiments is that cross-validation is a useful procedure in Fuzzy ARTMAP, because it produces smaller Fuzzy ARTMAP architectures with improved generalization performance. The trade-off is that cross-validation introduces additional computational complexity in the training phase of Fuzzy ARTMAP.

This paper establishes a connection between a neurofuzzy network model with the Mixture of Experts Network MEN modeling approach. Based on this connection, a new neurofuzzy MEN construction algorithm is proposed to overcome the curse of dimensionality that is inherent in the majority of associative memory networks and/or other rule based systems. The new construction algorithm is based on a new parallel learning method in which each model rule is trained independently, in which the parameter convergence property of the new learning method is established. By using the expert selective criterion of the MEN model output sensitivity to each expert, each rule can be selected to be trained or inhibited. The construction method is effective in overcoming the curse of dimensionality by reducing the dimensionality of the regression vector with the additional computational advantage of parallel processing. The proposed algorithm is analyzed for effectiveness followed by a numerical example to illustrate the efficacy for some difficult data based modeling problem.

In this paper we introduce new useful, geometric concepts regarding categories in Fuzzy ART and Fuzzy ARTMAP, which shed more light into the process of category competition eligibility upon the presentation of input patterns. First, we reformulate the competition of committed nodes with uncommitted nodes in an F₂ layer as a commitment test very similar to the vigilance test. Next, we introduce a category's match and choice regions, which are the geometric interpretation of the vigilance and commitment test respectively. After examining properties of these regions we reach three results applicable to both Fuzzy ART and Fuzzy ARTMAP. More specifically, we show that only one out of these two tests is required; which test needs to be performed depends on the values of the vigilance parameter (rho) and the choice parameter (alpha) . Also, we show that for a specific relation of (rho) and (alpha) , the vigilance (rho) does not influence the training or performance phase of Fuzzy ART and Fuzzy ARTMAP. Finally, we refine a previously published upper bound on the size of the categories created during training in Fuzzy ART and Fuzzy ARTMAP.

In spite of the featured self-adaptive mutation, implementation of Evolution Strategies ES still suffer from premature convergence. Usually, by readjusting mutation step size or easing up selective pressure, this problem can be alleviated to a certain extent. However, these solutions point to the innate deficiencies in standard ES schemes. These weaknesses include greediness of ranking-based truncation selection and a lack of feedback for mutation step adaptation. Based on previous studies, this paper attempts an alternative modification to standard ES implementation with parental population manipulation. The manipulation scheme consists of dynamic selection pooling and parental population sizing. It not only minimizes adverse interactions between above-mentioned evolution operators but also buttresses algorithm performance. Simulations on several benchmark problems vindicate the virtue of this modification.

Measures of an artificial neural network ANN capability are typically based on the Vapnik-Chernonvekis dimension and its variations. These measures may be underestimating the actual ANN's capabilities and hence overestimating the required number of examples for learning. This is caused by relying on a single invariant description of the problem set, which, in this case is cardinality, and requiring worst case geometric arrangements and colorings. A capability measure of an ANN is usually related to the desired characteristics of the problem sets. The mathematical framework has been established in which to express other desired invariant descriptors of a capability measure e.g., V-C dimension uses cardinality. A new invariant is defined on the problem space that softens the hard shattering constraint and yields a new capability measure of ANN's. The theory is given as well as examples that demonstrate this new measure.

Hans Bremermann was one of the pioneers of evolutionary computation. Many of his early suggestions for designing evolutionary algorithms anticipated future inventions. One such suggestion included the recombination of more than two parents, both in discrete particles (genes) and by blending. We have revisited Bremermann's original experiments from the early 1960s with linear systems of equations and extended them to include multiple trials that compare the use of mutation alone to the use of multi parent discrete recombination. The results indicate that for linear systems of small dimension, mutation alone outperforms multi parent discrete recombination for any number of parents from 2 to 50. In contrast, for linear systems of larger dimension, mutation alone is outperformed by all multi parent discrete recombination operators, for any number of parents from 2 to 50. The results suggest that it may be insufficient to classify a problem to be of a certain type i.e., amenable to a particular operator, in the absence of knowing the number of dimensions.

The ecosystem is used as an evolutionary paradigm of natural laws for the distributed information retrieval via mobile agents to allow the computational load to be added to server nodes of wireless networks, while reducing the traffic on communication links. Based on the Food Web model, a set of computational rules of natural balance form the outer stage to control the evolution of mobile agents providing multimedia services with a wireless Internet protocol WIP. The evolutionary model shows how mobile agents should behave with the WIP, in particular, how mobile agents can cooperate, compete and learn from each other, based on an underlying competition for radio network resources to establish the wireless connections to support the quality of service QoS of user requests. Mobile agents are also allowed to clone themselves, propagate and communicate with other agents. A two-layer model is proposed for agent evolution: the outer layer is based on the law of natural balancing, the inner layer is based on a discrete version of a Kohonen self-organizing feature map SOFM to distribute network resources to meet QoS requirements. The former is embedded in the higher OSI layers of the WIP, while the latter is used in the resource management procedures of Layer 2 and 3 of the protocol. Algorithms for the distributed computation of mobile agent evolutionary behavior are developed by adding a learning state to the agent evolution state diagram. When an agent is in an indeterminate state, it can communicate to other agents. Computing models can be replicated from other agents. Then the agents transitions to the mutating state to wait for a new information-retrieval goal. When a wireless terminal or station lacks a network resource, an agent in the suspending state can change its policy to submit to the environment before it transitions to the searching state. The agents learn the facts of agent state information entered into an external database. In the cloning process, two agents on a host station sharing a common goal can be merged or married to compose a new agent. Application of the two-layer set of algorithms for mobile agent evolution, performed in a distributed processing environment, is made to the QoS management functions of the IP multimedia IM sub-network of the third generation 3G Wideband Code-division Multiple Access W-CDMA wireless network.

This paper summarizes ongoing research. A neural network is used to detect a computer system intrusion basing on data from the system audit trail generated by Solaris Basic Security Module. The data have been provided by Lincoln Labs, MIT. The system alerts the human operator, when it encounters suspicious activity logged in the audit trail. To reduce the false alarm rate and accommodate the temporal indefiniteness of moment of attack a reinforcement learning approach is chosen to train the network.

This work is in the field of automated document processing. This work addresses the problem of representation and recognition of Urdu characters using Fourier representation and a Neural Network architecture. In particular, we show that a two-stage Neural Network scheme is used here to make classification of 36 Urdu characters into seven sub-classes namely subclasses characterized by seven proposed and defined fuzzy features specifically related to Urdu characters. We show that here Fourier Descriptors and Neural Network provide a remarkably simple way to draw definite conclusions from vague, ambiguous, noisy or imprecise information. In particular, we illustrate the concept of interest regions and describe a framing method that provides a way to make the proposed technique for Urdu characters recognition robust and invariant to scaling and translation. We also show that a given character rotation is dealt with by using the Hotelling transform. This transform is based upon the eigenvalue decomposition of the covariance matrix of an image, providing a method of determining the orientation of the major axis of an object within an image. Finally experimental results are presented to show the power and robustness of the proposed two-stage Neural Network based technique for Urdu character recognition, its fault tolerance, and high recognition accuracy.

This paper presents an alternative bang-bang solution for a satellite attitude control problem. The controller used is composed of a fuzzy logic controller and a neural network. In this case, the fuzzy logic controller serves as the main bang-bang controller while the neural network acts as a support unit to the main controller. The previous study has shown that the appropriate setting of each membership function of the fuzzy logic controller depends on the initial attitude of the satellite. This limits the possibility of a real-time implementation since generally the initial attitude is not known in advance. In order to overcome this difficulty, in this paper various values of the initial attitude are used as samples for determining the corresponding fuzzy configurations. The neural network is the taught to generalize the relationship between the fuzzy controller configuration and the value of the initial attitude. After the training, the neural network is used for providing the configuration of the fuzzy controller in real-time operations. The results indicate that the time required to drive the system to the final state when the proposed hybrid controller is used is less than that when the well-known linearized bang-bang control law is implemented.

The present paper proposes a new layout for failure detection and diagnosis in industrial dynamic systems in which, failure vector decoupling is not always possible, due to the failure intrinsic propagation. In this case diagnosis can be determined due to the existing correlation between the failure vector and residual vector time patterns. The greatest benefit of this study is the failure detection method, Luenberger observer based detection filter, through vectorial residual generation combined with the pattern recognition technique based on neural networks theory. The synergy of both methods offer a wider application range to diagnosis problem solutions, in systems under presence of non-decoupled failures.

This work presents the design of an integral environment for the suitable development of neural networks applications. The integrated environment contemplates the following features: A data processing module which encompasses statistical data analysis techniques for variables selection reduction, a variety of learning algorithms, code generator for different computer languages to enable network implementation, a learning sessions planning module and database connectivity facilities via ODBC, RPC, and API.

In this paper a fault detection and isolation scheme using a set of Neo fuzzy neurons will be presented. Such neurons use IF-THEN rules for characterizing the synaptic junctions in order to obtain complex nonlinear input/output maps in a simple structure, allowing an improvement of the learning and representation capabilities. As illustrative example, the fault detection scheme in a three interconnected tank system will be presented.

In this work a Methodology framework for implanting Virtual Sensors using Neural Networks will be presented, including the statistical analysis techniques that can be used for studying and processing the data. The proposed Methodology is based upon Software Engineering, Knowledge-based systems and neural networks methodologies. This methodological framework includes both technical and economical feasibility to build the virtual sensors and considers important aspects as the available computational platform, historical data files, data processing requirements such as filtering, pruning, set of variables that must be selected for the best performance of the virtual sensor, etc. There are also presented the statistical consideration and the corresponding techniques for data analysis and processing. The methodology includes techniques as principal components, cluster analysis, factorial analysis, etc.

In this paper we consider the design of intelligent control policies for water distribution systems. The controller presented in this paper is based upon a hybrid system that utilizes dynamic programming and rules as design constraints, to minimize average costs over a long time horizon under constraints on operation parameters. The method is very general and is reported here as a controller for water distribution system. In the example presented we obtain a 12.5 percent reduction in energy usage over the optimal level-based control design. We present the guiding principles used in the design and the results for a simulated system that is representative of a typical water pumping station. The design is fully adaptable to changing operating conditions and has applicability to a wide range of scheduling problems.

Self-organizing maps SOM can be used as clustering algorithm to discover structure and similarity in data and to capture the descriptive aspect by repeated partitioning and evaluating. It has the ability to represent multidimensional data in topological mapping. If a class label is known, self-organizing map can be also used by a classifier. In this case, each neuron is assigned a class label based on the maximum class frequency and classified by a nearest neighbor strategy. The problem when using this strategy is that each pattern is treated by equal importance in counting class frequency regardless of its typicalness. But, with known class label we can take an advantage of this information by applying fuzzy set theory and assigning the fuzzy class membership into each neuron. In fact, the fuzzy- membership-label neuron gives us insight of the degree of class typicalness and distinguishes itself from a class cluster.

This contribution describes a neural network that self- organizes to recover the original signals from sensor signals. No particular information is required about the statistical properties of the sources and the coefficients of the linear-transformation, except the fact that the source signals are statistically independent and non- stationary. The learning rule for the network's parameters is derived from the steepest descent minimization of a time- dependent cost function that takes the minimum only when the network outputs are correlated with each other.

With the prevalence of the information age, privacy and personalization are forefront in today's society. As such, biometrics are viewed as essential components of current evolving technological systems. Consumers demand unobtrusive and non-invasive approaches. In our previous work, we have demonstrated a speaker verification system that meets these criteria. However, there are additional constraints for fielded systems. The required recognition transactions are often performed in adverse environments and across diverse populations, necessitating robust solutions. There are two significant problem areas in current generation speaker verification systems. The first is the difficulty in acquiring clean audio signals in all environments without encumbering the user with a head- mounted close-talking microphone. Second, unimodal biometric systems do not work with a significant percentage of the population. To combat these issues, multimodal techniques are being investigated to improve system robustness to environmental conditions, as well as improve overall accuracy across the population. We propose a multi modal approach that builds on our current state-of-the-art speaker verification technology. In order to maintain the transparent nature of the speech interface, we focus on optical sensing technology to provide the additional modality-giving us an audio-visual person recognition system. For the audio domain, we use our existing speaker verification system. For the visual domain, we focus on lip motion. This is chosen, rather than static face or iris recognition, because it provides dynamic information about the individual. In addition, the lip dynamics can aid speech recognition to provide liveness testing. The visual processing method makes use of both color and edge information, combined within Markov random field MRF framework, to localize the lips. Geometric features are extracted and input to a polynomial classifier for the person recognition process. A late integration approach, based on a probabilistic model, is employed to combine the two modalities. The system is tested on the XM2VTS database combined with AWGN in the audio domain over a range of signal-to-noise ratios.

Polynomial networks have proven successful in authentication applications such as speaker recognition. A drawback of these methods is that as the degree of the polynomial network is increased, the number of model terms increases rapidly. This rapid increase can result in over fitting and make the network difficult to use in real-world applications because of the large number of model terms. We propose and contrast two solutions to this problem. First, we show how random dimension reduction can be used to effectively control model complexity. We describe a novel method which allows quick reduction of the dimension using an FFT. Applying these methods to a speaker recognition problem shows an approximately linear relation between the log of the number of model parameters and the log of the error rate. Second, we apply several methods of feature selection to reduce both model complexity and computation. We survey several methods and show which method yields the best performance in a speaker recognition application.

To date numerous techniques have been proposed to compress digital images to ease their storage and transmission over communication channels. Recently, a number of image compression algorithms using Neural Networks NNs have been developed. Particularly, several constructive feed-forward neural networks FNNs have been proposed by researchers for image compression, and promising results have been reported. At the previous SPIE AeroSense conference 2000, we proposed to use a constructive One-Hidden-Layer Feedforward Neural Network OHL-FNN for compressing digital images. In this paper, we first investigate the generalization capability of the proposed OHL-FNN in the presence of additive noise for network training and/ or generalization. Extensive experimental results for different scenarios are presented. It is revealed that the constructive OHL-FNN is not as robust to additive noise in input image as expected. Next, the constructive OHL-FNN is applied to moving images, video sequences. The first, or other specified frame in a moving image sequence is used to train the network. The remaining moving images that follow are then generalized/compressed by this trained network. Three types of correlation-like criteria measuring the similarity of any two images are introduced. The relationship between the generalization capability of the constructed net and the similarity of images is investigated in some detail. It is shown that the constructive OHL-FNN is promising even for changing images such as those extracted from a football game.

This paper presents the preliminary results for a system in tree dimension that use a system vision to manipulate plants in a tissue culture process. The system is able to estimate the position of the plant in the work area, first calculate the position and send information to the mechanical system, and recalculate the position again, and if it is necessary, repositioning the mechanical system, using an neural system to improve the location of the plant. The system use only the system vision to sense the position and control loop using a neural system to detect the target and positioning the mechanical system, the results are compared with an open loop system.

Stereopsis consists of the recovery of three-dimensional scenes from two-dimensional images. Major steps in stereopsis are pre-processing, establishing correspondences and recovering depth. Stereo correspondence is the weightiest of concern. A tremendously flexible environment exists compounding the problem. Hopfield neural networks particularly offers promise, as it is fully inter-connected, converges to a solution and is recurrent. Parallel computing becomes an absolute ingredient due to the enormity of computation. This paper utilizes the matching primitive of edge pixel features with optimization as the relaxation method and a focus on parallel implementation. The computational structure consists of layers, epilayer lines and separate orientations. Similarity, smoothness, uniqueness and ordering are the matching constraints. Energy minimization incorporates all the constraints by exciting and inhibiting neurons. Results are obtained from a number of real color images of differing complexity. The computational intelligence is displayed in terms of efficiency and speed- up. Efficiency, by way of a matching function covering color images and the incorporation of all the constraints into the neural network. Speed-up is demonstrated in the parallel implementation. Stereo disparity is thus obtained with only a few mismatches and a sharp step-down from an initial duration of forty hours to a mere nine minutes.

Neural networks, especially in nonlinear system identification and control applications, are typically considered to be black-boxes which are difficult to analyze and understand mathematically. Due to this reason, an in- depth mathematical analysis offering insight into the different neural network transformation layers based on a theoretical transformation scheme is desired, but up to now neither available nor known. In previous works it has been shown how proven engineering methods such as dimensional analysis and the Laplace transform may be used to construct a neural controller topology for time-invariant systems. Using the knowledge of neural correspondences of these two classical methods, the internal nodes of the network could also be successfully interpreted after training. As further extension to these works, the paper describes the latest of a theoretical interpretation framework describing the neural network transformation sequences in nonlinear system identification and control. This can be achieved By incorporation of the method of exact input-output linearization in the above mentioned two transform sequences of dimensional analysis and the Laplace transformation. Based on these three theoretical considerations neural network topologies may be designed in special situations by pure translation in the sense of a structural compilation of the known classical solutions into their correspondent neural topology. Based on known exemplary results, the paper synthesizes the proposed approach into the visionary goals of a structural compiler for neural networks. This structural compiler for neural networks is intended to automatically convert classical control formulations into their equivalent neural network structure based on the principles of equivalence between formula and operator, and operator and structure which are discussed in detail in this work.

One important application of mobile robots is searching a geographical region to locate the origin of a specific sensible phenomenon. A variety of optimization algorithms can be employed to locate the target source which has the maximum intensity of the distribution of illumination function. It is very important to evaluate the performance of those optimization algorithms so that the researchers can adopt the most appropriate optimization approach to save a lot of execution time and cost of both collective robots and human beings. In this paper we provide three different neural network algorithms: steepest ascent algorithm, combined gradient algorithm and stochastic optimization algorithm to solve the collective robotics search problem. Experiments with different pair of number of sources and robots were carried out to investigate the effect of source size and team size on the task performance, as well as the risk of mission failure. The experimental results showed that the performance of steepest ascent method is better than that of combined gradient method, while the stochastic optimization method is better than steepest ascent method.

Robotic manipulators are beginning to be seen doing more tasks in our environment. Classical controls engineers have long known how to control these automated hands. They have failed to address the continued control of these devices after parts of the control infrastructure have failed. A failed motor or actuator in a manipulator decreases its range of motion and changes its control structure. Most failures however do not render the manipulator useless. This paper will discuss the use of a neural network to actively update the controller design as portions of a manipulator fail. Actuators can become stuck and later free themselves. Motors can lose range of motion or stop completely. Connecting arms can become bent or entangled. Results will be presented on the ability to maintain functionality through a variety of failure modes. The neural network is constructed and tested in a Matlab environment. This allows testing of several neural network techniques such as back propagation and temporal processing without the need to continually reconfigure target hardware. In this paper we will demonstrate that a modified ensemble of back propagation experts can be trained to control a robotic manipulator without the need to calculate the inverse kinematics equations. Further individual experts can be retrained online to allow for adaptive control through changing dynamics. This allows for manipulators to remain in service through failures in the manipulator infrastructure without the need for human intervention into control equations.

Movement of a differential drive robot has non-linear dependence on the current position and orientation. A controller must be able to deal with the non-linearity of the plant. The controller must either linearize the plant and deal with special cases, or be non-linear itself. Once the controller is designed, implementation on a real robotic platform presents challenges due to the varying parameters of the plant. Robots of the same model may have different motor frictions. The surface the robot maneuvers on may change e.g. carpet to tile. Batteries will drain, providing less power over time. A feed-forward neural network controller could overcome these challenges. The network could learn the non- linearities of the plant and monitor the error for parameter changes and adapt to them. In this manner, a single controller can be designed for an ideal robot, and then used to populate a multi-robot colony without manually fine tuning the controller for each robot. This paper shall demonstrate such a controller, outlining design in simulation and implementation on Khepera robotic platforms.

This paper presents a Variable Structure Control VSC-based algorithm for adjusting a set of time varying parameters of virtual linear models that resemble linear dynamical neurons, used as on-line representations for a class of uncertain nonlinear processes. These virtual linear models allow the implementation of adaptive controllers in order to achieve predefined specifications for the closed-loop of the uncertain nonlinear process, or to force the tracking of the process output to reference models outputs accurately. A proof of the finite time convergence of the virtual linear model variables to the uncertain nonlinear process variables is included and some examples are contemplated to illustrate the proposed control design approaches.

We consider the problem of clustering a set of objects which are represented by rational data in the form of a dissimilarity matrix which has missing values. Three methods are developed to estimate the missing values, all based on simple triangle inequality-based approximation schemes. With few exceptions, any relational clustering algorithm can then be applied to the completed data matrix to obtain nice clusters. We illustrate our approach by clustering incomplete data built from several data sets. The primary clustering method chosen for our numerical experiments is the non-Euclidean relational fuzzy c-means algorithm. Our examples show that satisfactory clusters can still be obtained even when roughly half of the distance values are missing before completion.

For integrated instability investigation, a sequence of functional matrixes multiplication is transformed by separation of elements describing linear and nonlinear transformations in delay difference map. A quantity of multiplications of diagonal matrixes is minimized by means of dividing initial functional matrix into diagonal, triangle and the matrix of unit cyclic shift. The analytical expressions for product matrix have been obtained and different approaches of reducing the product matrix to rarefied form have been investigated. The analytical scheme for calculating rarefied product matrix has been constructed, its operations are invariant to changing a common length of investigated time series and transformed segments. The developed model can be used for characteristic polynomial determination and for increasing efficiency of computational algorithm of Lyapunov exponents calculation.

We introduce Ellipsoid-ART, EA and Ellipsoid-ARTMAP, EAM as a generalization of Hyper-sphere ART and Hypersphere-ARTMAP respectively. Our novel archetectures are based on ideas rooted on Fuzzy-ART, FA and Fuzzy-ARTMAP, FAM. While FA/FAM summarize input data using hyper-rectangles, EA/EAM utilize hyper-ellipsoids for the same purpose. Due to their learning rules, EA and EAM share virtually all properties and characteristics of their FA/FAM counterparts. Preliminary experimentation implies that EA and EAM are to be viewed as good alternatives to FA and FAM for data clustering and classification tasks. Extensive pseudo-code is provided in the appendices for computationally efficient implementations of EA/EAM training and performance phases.

Neural network based inverse modeling approach is investigated to predict propulsion system rotor unbalance. The frequency response of vibration collected from an engine model is used as inputs to train neural networks, which identify the source of unbalance and determine the amount of rotor unbalance. High-order finite-element structural dynamic models of airplane engines, case, nacelle, and strut are used to produce training/testing data. Performance of several neural networks inverse models, including back- propagation, extended Kalman filter, and support vector machine, are compared. The ability to locate and quantify unbalance source with respect to multiple engine fan and turbine stages is demonstrated.