Redwood Neuroscience
Title "Vector Symbolic Architectures
answer Jackendoff's challenges for cognitive
neuroscience."
Ross
Gayler
Abstract:
Vector Symbolic Architectures (Gayler, 1998; Kanerva,
1997; Plate, 1994) are a little-known class of connectionist models that can
directly implement functions usually taken to form the kernel of symbolic
processing. They are an enhancement of
tensor product variable binding networks (Smolensky,
1990).
Like tensor product networks, VSA's
can create and manipulate recursively-structured representations in a natural
and direct connectionist fashion without requiring lengthy training. However, unlike tensor product networks, VSA's afford a practical basis for implementations because
they require only fixed dimension vector representations. The fact that VSA's
relate directly, without training, to both simple, practical vector implementations
and core symbolic processing functionality suggests that they would provide a
fruitful connectionist basis for the implementation of cognitive functionality.
Ray Jackendoff (2002) posed
four challenges that linguistic combinatoriality and
rules of language present to theories of brain function. These challenges are: the massiveness of the
binding problem, the problem of dealing with multiple instances, the problem of
variables, and the compatibility of representations in working memory and
long-term memory. The essence of these problems is the question of how to neurally instantiate the rapid construction and
transformation of the compositional structures that are
typically taken to be the domain of symbolic processing.
Jackendoff contended that these challenges had not been widely recognised in the cognitive neuroscience community and that
the dialogue between linguistic theory and neural network modelling
would be relatively unproductive until the challenges were answered by some
technical innovation in connectionist models.
Jerome Feldman (2002) broadcast these challenges to the neural network modelling community via the Connectionists Mailing
List. The few responses he received were
unable to convince Feldman that any standard connectionist techniques would
meet Jackendoff's criteria.
I argue that Vector Symbolic Architectures are able to
meet Jackendoff's challenges.
References
Feldman, J. (2002).
Neural binding.
Posted to Connectionists Mailing List,
h.2002-08.gz 0005.txt see also 8, 9, 18, and 21)
Gayler, R. W. (1998).
Multiplicative binding, representation operators, and
analogy. In K. Holyoak, D. Gentner & B. Kokinov (Eds.), Advances in analogy
research: Integration of theory and data from the cognitive,
computational, and neural sciences (p. 405).
Jackendoff, R. (2002).
Foundations of language: Brain, meaning, grammar, evolution.
Kanerva, P. (1997).
Fully distributed representation. In Proceedings Real World
Computing Symposium (RWC'97,
http://www.rni.org/kanerva/pubs.html)
Plate, T. A. (1994). Distributed
representations and nested compositional structure. Ph.D. thesis, Department of
Computer Science,
Smolensky, P. (1990). Tensor product variable binding and
the representation of symbolic structures in connectionist systems.
Artificial Intelligence, 46, 159-216.