Tuesday, March 30, 2004

3:30pm

Redwood Neuroscience Institute

Title: Information-Theoretic Analysis of an Energy-Constrained RNN Model with Feedback

Toby Berger

Electrical and Computer Engineering

Cornell University

Abstract:

Inspired by the fact that the anatomical structure of primate visual cortex can be well modeled as a hierarchy of time-discrete, finite-state channels with feedback, we explore such channels from the perspective of information theory. The channel model we employ respects the biology via incorporation of such phenomena as neural refractoriness, quantal synaptic failures, bottom-up and top-down signaling, and energy consumption. We sketch the proof of a coding theorem and converse which establish that the constrained limit superior of the Marko-Massey directed information rate generates the Shannon capacity-cost function of finite-state channels with feedback. We also show that any input process which drives such a channel at its constrained capacity produces an output process which is such that the joint input/output process is Markovian. Moreover, said output process is marginally Markovian, though the input process itself need not be. The findings explicate how the brain is able to utilize its RNNs in an effectively Shannon optimum manner robustly in the average cost despite the fact that it employs neither encoders nor decoders in the traditional sense of these terms in communication and information theory.

NOTE: Joint work with Chip Levy of UVA Medical School.

Biosketch:

Toby Berger is the Irwin and Joan Jacobs Professor of Engineering at Cornell University. For the past seven years his principal research interest has been the application of information theory to biology with emphasis on neuroscience. The Shannon Lecture he delivered upon being designated the 2002 Shannon Award winner by the IEEE Information Theory Society was entitled “Living Information Theory”.