Gaussian activation functions using Markov chains | IEEE Journals & Magazine | IEEE Xplore

Gaussian activation functions using Markov chains


Abstract:

We extend, in two major ways, earlier work in which sigmoidal neural nonlinearities were implemented using stochastic counters. 1) We define the signal to noise limitatio...Show More

Abstract:

We extend, in two major ways, earlier work in which sigmoidal neural nonlinearities were implemented using stochastic counters. 1) We define the signal to noise limitations of unipolar and bipolar stochastic arithmetic and signal processing. 2) We generalize the use of stochastic counters to include neural transfer functions employed in Gaussian mixture models. The hardware advantages of (nonlinear) stochastic signal processing (SSP) may be offset by increased processing time; we quantify these issues. The ability to realize accurate Gaussian activation functions for neurons in pulsed digital networks using simple hardware with stochastic signals is also analyzed quantitatively.
Published in: IEEE Transactions on Neural Networks ( Volume: 13, Issue: 6, November 2002)
Page(s): 1465 - 1471
Date of Publication: 30 November 2002

ISSN Information:

PubMed ID: 18244541
Electrical Computer Engineering Department, University of Manitoba, Winnipeg, MAN, Canada
Howard C. Card (S'68–M'72–SM'83–F'94) received the Ph.D. degree from the University of Manchester, Manchester, U.K., in 1971.
He has held academic appointments at Manchester University, the University of Manitoba, Winnipeg, MB, Canada, the University of Waterloo, Waterloo, ON, Canada, and Columbia University, New York, industrial appointments at IBM Watson Research Center and AT&T Bell Labs, and has spent a sabbatical year...Show More
Howard C. Card (S'68–M'72–SM'83–F'94) received the Ph.D. degree from the University of Manchester, Manchester, U.K., in 1971.
He has held academic appointments at Manchester University, the University of Manitoba, Winnipeg, MB, Canada, the University of Waterloo, Waterloo, ON, Canada, and Columbia University, New York, industrial appointments at IBM Watson Research Center and AT&T Bell Labs, and has spent a sabbatical year...View more
Electrical Computer Engineering Department, University of Manitoba, Winnipeg, MAN, Canada
Dean K. McNeill (S'90–M'90) received the Ph.D. degree from the University of Manitoba, Winnipeg, MB, Canada, in 1999.
Since then, he has held an NSERC Postdoctoral Fellowship at the University of Edinburgh, Edinburgh, U.K. He recently joined the Department of Electrical and Computer Engineering, University of Manitoba, as an Assistant Professor of Computer Engineering. His research interests are in the area of neural compu...Show More
Dean K. McNeill (S'90–M'90) received the Ph.D. degree from the University of Manitoba, Winnipeg, MB, Canada, in 1999.
Since then, he has held an NSERC Postdoctoral Fellowship at the University of Edinburgh, Edinburgh, U.K. He recently joined the Department of Electrical and Computer Engineering, University of Manitoba, as an Assistant Professor of Computer Engineering. His research interests are in the area of neural compu...View more

Electrical Computer Engineering Department, University of Manitoba, Winnipeg, MAN, Canada
Howard C. Card (S'68–M'72–SM'83–F'94) received the Ph.D. degree from the University of Manchester, Manchester, U.K., in 1971.
He has held academic appointments at Manchester University, the University of Manitoba, Winnipeg, MB, Canada, the University of Waterloo, Waterloo, ON, Canada, and Columbia University, New York, industrial appointments at IBM Watson Research Center and AT&T Bell Labs, and has spent a sabbatical year at Oxford University, Oxford, U.K. He has worked on device physics and modeling, VLSI systems, parallel processing, and artificial intelligence. His current research is on neural networks and stochastic computation. He is also interested more generally in the relationship between biology, physics, and computation.
Dr. Card has been an Athlone Fellow and a Mullard Fellow. He has received several teaching awards including the Stanton Award for Excellence in Teaching and the UMFA-UTS Award. He also has a number of research awards, including the NSERC E.W.R. Steacie Memorial Fellowship, the Ross Medal of IEEE Canada, the Rh. Institute Award for Multidisciplinary Research, the Sigma Xi Senior Scientist Award, and the ITAC-NSERC Award for Academic Excellence. He is a Fellow of the Canadian Academy of Engineering.
Howard C. Card (S'68–M'72–SM'83–F'94) received the Ph.D. degree from the University of Manchester, Manchester, U.K., in 1971.
He has held academic appointments at Manchester University, the University of Manitoba, Winnipeg, MB, Canada, the University of Waterloo, Waterloo, ON, Canada, and Columbia University, New York, industrial appointments at IBM Watson Research Center and AT&T Bell Labs, and has spent a sabbatical year at Oxford University, Oxford, U.K. He has worked on device physics and modeling, VLSI systems, parallel processing, and artificial intelligence. His current research is on neural networks and stochastic computation. He is also interested more generally in the relationship between biology, physics, and computation.
Dr. Card has been an Athlone Fellow and a Mullard Fellow. He has received several teaching awards including the Stanton Award for Excellence in Teaching and the UMFA-UTS Award. He also has a number of research awards, including the NSERC E.W.R. Steacie Memorial Fellowship, the Ross Medal of IEEE Canada, the Rh. Institute Award for Multidisciplinary Research, the Sigma Xi Senior Scientist Award, and the ITAC-NSERC Award for Academic Excellence. He is a Fellow of the Canadian Academy of Engineering.View more
Electrical Computer Engineering Department, University of Manitoba, Winnipeg, MAN, Canada
Dean K. McNeill (S'90–M'90) received the Ph.D. degree from the University of Manitoba, Winnipeg, MB, Canada, in 1999.
Since then, he has held an NSERC Postdoctoral Fellowship at the University of Edinburgh, Edinburgh, U.K. He recently joined the Department of Electrical and Computer Engineering, University of Manitoba, as an Assistant Professor of Computer Engineering. His research interests are in the area of neural computation and adaptive signal processing for embedded computing systems.
Dean K. McNeill (S'90–M'90) received the Ph.D. degree from the University of Manitoba, Winnipeg, MB, Canada, in 1999.
Since then, he has held an NSERC Postdoctoral Fellowship at the University of Edinburgh, Edinburgh, U.K. He recently joined the Department of Electrical and Computer Engineering, University of Manitoba, as an Assistant Professor of Computer Engineering. His research interests are in the area of neural computation and adaptive signal processing for embedded computing systems.View more

References

References is not available for this document.