Toward an Understanding of Information Integration in the Brain
Our research theme in the RWC Program is called "Transformation between Symbols and Patterns." Since symbol processing and pattern processing are different concepts in an engineering sense, it is not easy to integrate them. However, humans perceive the world with various senses and converse with each other using language. This means that the brain can handle and integrate symbols and patterns at the same time. We believe that knowing the mechanism, architecture, and theory of the brain will help clarify the missing parts of current pattern recognition and knowledge processing.
We have two working hypotheses on the symbol-pattern integration in the brain. The first is the dual coding hypothesis that the brain possesses both multiple pattern representation systems with different modalities and a symbol representation system to link them, and is capable of converting information as needed.
The other one is the sequential attention hypothesis that there is an attention system in the brain which functions sequentially, and that the sequential attention controls behavior of its information representation and processing systems.
We believe that these hypothesis are sufficiently probable in terms of both mathematical definitions of symbols and patterns and of the science of the brain. The current issue is shifting to the engineering aspects, e.g., how they are implemented, how useful they are as engineering systems, and how they affect the theory, architecture, and design theory of intellectual systems [1].
Basic Model and Implementation: Two Flows of Research
Theorization of the Brain Memory Model PATON [2]
As a basic model of symbol-pattern integration, we are analyzing the memory model PATON, which is based on the brain's macroscopic structure, and studying it's applications. PATON is an associative memory model which can control activity level of brain areas with attention. PATON is simple enough to be interpreted theoretically, but it also spontaneously generates context-dependent cognition behavior such as recognition and association.
Currently, we are comparing and analyzing the computation process of PATON and that of conventional intelligent systems to derive a theory of brain intelligent behavior from their difference. As PATON is capable of manipulating conceptual memory depending on context, we are investigating whether the theory can be applied to the understanding of ungrammatical natural language.
Image Understanding through Transformation between Symbols and Patterns [3]
To investigate and extract problems by applying the basic concepts to the actual issues, we are attempting to implement a hybrid system that understands images by combining image processing and symbol processing. The core of this system is a neural net called a"Neocognitron", which performs the transformation between different kinds of information: images and symbols.
The performance of the implemented image understanding is still in its infancy due to the technical difficulties in this transformation system. However, our hypotheses on the information integration of the brain may clarify the advantages of this hybrid system that combine different kinds of processing systems, thus possibly revealing a way of constructing a system which can handle a variety of problems exhisting in the real world. For instance, a hierarchical language to describe the world is indispensable in order to handle the various types of information present in the real world; this was one of the main topics in the Information Integration Workshop held last November.
So far in this article, we have discussed information integration as seen from the brain, and elucidated the purposes of the PATON memory model and of the hybrid image understanding system. Now I would like to give my personal view of the symbol-pattern integration architecture as a result of these studies.
First, the brain can be considered to be a state transition machine driven by the attentions and inputs. The Turing machine can also be simulated. That is, the brain as a computing machine is basically similar to modern computers although the devices and architecture involved are quite different.
Architecturally, the brain consists of elements in which both operation and memory are combined, and the operation is implemented by the convolution process of the associative memory. The program for this is written as a macro level attention sequence, and its computation is implemented in a different way from the Neumann-type architecture. In addition, the brain changes a transition state itself as it learns. It might be comparable to the structural change of a normal CPU. Once these computational principles are understood, a similar computational unit might be fabricated even with normal silicon devices.
Moreover, the computing theory of the hybrid system suggests the possibility of problem-solving programming on more abstract level than the attentions. At present, I suppose this is a constraint satisfaction. The brain memorizes and generalizes actual events for many modalities. The memories work as constraints for the associative memory computation. The attentions select and combine them. The constraints satisfaction is important not only in the brain, but also as a methodology of programming languages, as discussed in the Information Integration Workshop.
The above discussions due to the issues discussed in the"Symbol-Pattern Integration Workshop (SYMPAT)" of the RWCP. Currently, there are many symbol and pattern definitions and methods of integration, and this is only one of them.
[1] Omori T.:"Theory of Symbol and Pattern Integration by Associative Memory with Attention ", MFI96, accepted
[2] Omori T., Mochizuki A., : Computational Theory of Memory and Representation and Attention System, in Ono, McNaughton, Molottchnikoff, Rolls Eds. "Perception, Memory and Emotion : Frontier in Neuroscience ", p549-565, Elsevier Science Ltd., 1996
[3] Mitsumori, Tsuji, Omori T.:"Hybrid Architecture for Image Understanding by Mutual Evaluation of Symbol and Pattern processing", Proc. of ICONIP96- HongKong, 444-449, (1996)