Abstract:
Addresses the extraction of knowledge from recurrent neural networks trained to behave like deterministic finite-state automata (DFAs). To date, methods used to extract k...Show MoreMetadata
Abstract:
Addresses the extraction of knowledge from recurrent neural networks trained to behave like deterministic finite-state automata (DFAs). To date, methods used to extract knowledge from such networks have relied on the hypothesis that network states tend to cluster and that clusters of network states correspond to DFA states. The computational complexity of such a cluster analysis has led to heuristics which either limit the number of clusters that may form during training or limit the exploration of the output space of hidden recurrent state neurons. These limitations, while necessary, may lead to reduced fidelity, i.e. the extracted knowledge may not model the true behavior of a trained network, perhaps not even for the training set. The method proposed uses a polynomial-time symbolic learning algorithm to infer DFAs solely from the observation of a trained network's input/output behavior. Thus, this method has the potential to increase the fidelity of the extracted knowledge.
Date of Conference: 16-20 November 1999
Date Added to IEEE Xplore: 06 August 2002
Print ISBN:0-7803-5871-6
References is not available for this document.
Select All
1.
Proceedings of the NIPS96 Rule Extraction from Trained Artificial Neural Workshop, 1996-December-6.
2.
R. Andrews, J. Diederich and A. Tickle, "A survey and critique of techniques for extracting rules from trained artificial neural networks", Knowledge-Based Systems, 1995.
3.
M. Casey, "The dynamics of discrete-time computation with application to recurrent neural networks and finite state machine extraction", Neural Computation, vol. 8, no. 6, pp. 1135-1178, 1996.
4.
A. Cleeremans, D. Servan-Schreiber and J. McClelland, "Finite state automata and simple recurrent recurrent networks", Neural Computation, vol. 1, no. 3, pp. 372-381, 1989.
5.
S. Das and M. Mozer, "A unified gradient-descent/clustering architecture for finite state machine induction" in Advances in Neural Information Processing Systems, CA, San Francisco:Morgan Kaufmann, vol. 6, pp. 19-26, 1994.
6.
J. Elman, "Finding structure in time", Cognitive Science, vol. 14, pp. 179-211, 1990.
7.
L. Fu, "Rule generation from neural networks", IEEE Transactions on Systems Man and Cybernetics, vol. 24, no. 8, pp. 1114-1124, 1994.
8.
C. Giles, C. Miller, D. Chen, H. Chen, G. Sun and Y. Lee, "Learning and extracting finite state automata with second-order recurrent neural networks", Neural Computation, vol. 4, no. 3, pp. 380, 1992.
9.
E. Gold, "Complexity of automaton identification from given data", Information and Control, vol. 37, pp. 302-320, 1978.
10.
M. Golea, On the complexity of rule-extraction from neural networks and network-querying, 1996.
11.
J. Hopcroft and 1. Ullman, Introduction to Automata Theory Languages and Computation, MA, Reading:Addison-Wesley Publishing Company, Inc., 1979.
12.
T. Kohonen, Self-Organizing Maps, Germany, Berlin:Springer Verlag, 1995.
13.
K. Lang, "Random dfas can be approximately learned from sparse uniform examples", Proceedings of the Fifth ACM Workshop on Computational Learning Theory, 1992-July.
14.
C. Omlin and C. Giles, "Extraction of rules from discrete-time recurrent neural networks", Neural Networks, vol. 9, no. 1, pp. 41-52, 1996.
15.
J. Pollack, "The induction of dynamical recognizers", Machine Learning, vol. 7, pp. 227-252, 1991.
16.
P. Tino, B. Horne, C. Giles and P. Collingwood, "Finite state machines and recurrent neural networks - automata and dynamical systems approaches", Neural Networks and Pattern Recognition, pp. 171-220, 1998.
17.
P. Tino and J. Sajda, "Learning and extracting initial mealy machines with a modular neural network model", Neural Computation, vol. 7, no. 4, pp. 822-844, 1995.
18.
G. G. Towell and J. W. Shavlik, "The extraction of refined rules from knowledge-based neural networks", Machine Learning, vol. 13, no. 1, pp. 71-101, 1993.
19.
B. Trakhenbrot and Y. Barzdin, Finite automata: Behavior and synthesis, Amsterdam:North-Holland, 1973.
20.
R. Watrous and G. Kuhn, "Induction of finite-state languages using second-order recurrent networks", Neural Computation, vol. 4, no. 3, pp. 406, 1992.
21.
R. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks", Neural Computation, vol. 1, pp. 270-280, 1989.
22.
Z. Zeng, R. Goodman and P. Smyth, "Learning finite state machines with self-clustering recurrent networks", Neural Computation, vol. 5, no. 6, pp. 976-990, 1993.
