Hybrid training procedure applied to recurrent neural networks

Xavier Loiseau; Jan Sendler

doi:10.1117/12.235936

22 March 1996 Hybrid training procedure applied to recurrent neural networks

Xavier Loiseau, Jan Sendler

Proceedings Volume 2760, Applications and Science of Artificial Neural Networks II; (1996) https://doi.org/10.1117/12.235936
Event: Aerospace/Defense Sensing and Controls, 1996, Orlando, FL, United States

Abstract

The analysis of stochastic time series has shown to be a problem of major importance in the last several years, especially in the field of speech and handwriting recognition. Two approaches in particular have been proposed to achieve this goal, one based on hidden Markov modeling and the other using recurrent neural networks (RNNs). An internal class representation can easily be found using an hidden Markov model (HMM) trained by an expectation maximation algorithm. This can be regarded as a major advantage of the HMM approach. Since likelihood is being maximized (ML criterion), the classifier shows less discriminant power. In contrast, an RNN can be trained including counter examples. This improves the influence of features which discriminate competing classes. A gradient descent algorithm is applied to minimize the mean square error (MMSE criterion). The initial lack of internal representation of a class leads to serious convergence problems as the sequences lengthen. A new hybrid approach to the training of RNNs is investigated, combining the advantages of the previous methods. Various samples of handwritten letters are used to adapt the RNN, in order to examine the convergence and the discriminating ability of the new algorithm.

Citation Download Citation

Xavier Loiseau and Jan Sendler "Hybrid training procedure applied to recurrent neural networks", Proc. SPIE 2760, Applications and Science of Artificial Neural Networks II, (22 March 1996); https://doi.org/10.1117/12.235936

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available