| Ryuichi Oka General ManagerM Theory and Novel Functions Dept. Real World Computing Partnership |
|
| Ryuichi Oka General ManagerM Theory and Novel Functions Dept. Real World Computing Partnership |
I will present the summary of the research results in the Theory and Novel Functions area for the five-year period from 1992. Since this would encompass the results of more than 70 papers regarding Theory and Novel Functions that will be presented in this symposium in a short time, I must limit my talk in general to the perspective from which we view the RWC research results.
Fig. 1
First, one of the objectives of our research (Fig. 1) is to advance the Theory and Novel Functions area itself. The other objective is to construct a core of RWC. To establish this core, it is necessary to make application systems work in the real world, and to construct principles for synthesizing such application systems. To begin, allow me to explain what the real world is by using a simple photograph (Fig. 2).
Fig.2 | 
Fig. 2a |
This picture shows one scene. We may say that this shows a real world, yet it is a simple photograph of nature. When humans are added to the real world equation of this picture, the real world data becomes multimodal data (Fig. 2a).
For instance, humans would recognize the blue sky, white clouds, snow, mountains, and a lake in this picture. This lake may be quiet without a sound, but when a fish jumps a noise is made. There is also the ground in front. This picture of nature is indeed real world data, but when humans step in, the data on nature changes into multimodal information, a change from imagery to symbolism- specifically information accompanied by sound. Now, let's consider more practical scenes of information processing in our daily lives.
This picture (Fig. 3), created five years ago when RWCP started, exemplifies an actual real world scene.
Fig. 3
In this scene, two researchers are discussing whether an unknown animal is a new species by looking at its image. There is a computer between them which listens to their discussion and watches them looking at the animal. The computer monitors their discussion and presents data required from one scene to another. Behind the computer terminal exists a huge database. The computer retrieves the required data from the database and makes it available by understanding the researchers' discussion.
Let's take a look at what subjects may exist when determining a situation in the real world (Fig. 3a).
Fig. 3a
There is an unknown animal here. It is possible to look at this animal using vision. This is one role of vision. Information may be more specifically identified by looking at the facial expressions of these two people and determining which one is talking. This is another challenge of information processing using vision.
Discussions, characterized by their spontaneity, are conducted in a very rough form. The wordÅgroughÅh in this context refers to the difficulty computers have in processing or predicting such speech patterns. The voice and image data existent in such discussion scenes contain huge amounts of information regarding past experiences which is described through symbolism, which requires an interface to comprehend. Of course, this interface itself may exercise a certain kind of autonomous movement by moving around with its own legs. Concurrently, there are other important problems making data occurring in such scenes available with a learning function. Even such a simple scene can pose many problems.
Fig. 4
How are these problems tackled in our project? (Fig. 4) This figure shows what will be demonstrated later on. The T-Fujitsu Lab. and T-GMD Lab. are carrying out research which serves as the basis of the real world computing. You can see demonstrations of multimodal interfaces, such as a voice/image interface, performed by the ETL, F-Sharp Lab., and the F-TRC. We can see the results of face recognition research by the F-ISS Lab. in Singapore. There will be robotics demonstrations by the ETL, F-Fujitsu Lab., F-SNN Lab., and F-SICS Lab. Demonstrations on probabilistic learning have been prepared by the T-NEC Lab. and the F-SNN Lab. Vision related demonstrations will be shown by the N-Toshiba Lab., F-TRC, and F-SNN Lab. Symbolism and the real world will be demonstrated by the F-Hitachi Lab., the F-Mitsubishi Lab., and F-TRC.
Now, I will talk about what should be focused on in these researches (Fig. 5).
Fig. 5
One of the speakers today said that it is very difficult to move from a toy world to the real world. One reason for this difficulty is the need for openness (or adaptability) to unknown data, another is the need for a robust system able to function properly in the midst of noise. Real-time processing is also needed to generate proper output in a limited time. When looking at conventional research along these three axes: openness, robustness, and real-time processing; the toy world is close to the convergence point. The farther from the convergence of these axes some given research may evolve, the more suitable that research is for our Real World Computing. Therefore, I would suggest that each of you evaluate the demonstrations you will see later by checking to see if our RWC research extends from this narrow region when compared with conventional research.
As I said before, there are a number of specific research areas, and it is not always possible when we pursue openness, robustness, and real-time interfacing separately to improve the quality of each these research areas. As Mr. Tachibana said, the real world environment cannot always be divided into separate things, but the integrated whole appeals its existence. Then, what constitutes this integration? Let me explain using the previous picture (Fig. 3b).
Fig. 3b
First, information integration is characterized by the wordÅghorizontal.ÅhÅgHorizontalÅhhere means that integration is sustained among different modalities such as text data, visual data, and spontaneous speech; as many speakers have mentioned in today's symposium. Integration of these modalities is calledÅghorizontal integration.Åh
Second, integration may also be characterized by appearance, for instance, the appearance of an entirely unknown animal. When something is judged from its appearance, the overall impression is used instead of minor details. This is a vertical issue, linking the microscopic with the macroscopic, without dividing the whole into individual parts. As in the case of the two researchers having a discussion, their discussion is spontaneous and particular information is necessary, depending on the topics and time they concern themselves in their discussion. Certain information required at a particular moment may not be useful if provided later, thus real-time responses must be provided. Information relevant to these types of situations require real-time integration. Real-time integration, vertical integration of microscopic and macroscopic factors, and horizontal integration, are the three main issues of information integration.
Many of these issues are included in the research we have conducted during the past five years. Fig.6 shows the specific research topics with the laboratories which conducted the research for your reference in watching the demonstrations later. Horizontal integration will be demonstrated by TRC, Sharp, and ETL. Vertical integration has been studied by SNN and TRC, and real-time integration research has been conducted by University of Erlangen and TRC.
Fig. 6
Research on information integration is expected to make further progress in the future. Although not sufficient, the research results obtained during the past five years may be summarized in two points (Fig. 7).
Fig. 7
Research on Theory and Novel Functions has made considerable progress. However, the other objective, that of constructing a core for RWC research is still under way. This does not mean we lost the course of research; we are actively pursing a variety of paths. One path will be to developÅginformation integrationÅh for the design principle to construct new systems by integrating information.
The second path will be to develop a learning and self-organization capability. As discussed before, a learning capability is essential for a system to be adaptive to an open environment, an environment without humans to teach the system. Regarding self-organization, data should be able to organize itself in a core form which can be used by humans in cases where it is impossible for humans to process huge amounts of data.
So, in looking at the results of our five-year research, we can see that satisfactory results were achieved regarding individual theories and functions for real world computing. However, efforts for establishing a core design principle for RWC research, one which will enable new information processing, are still in the elemental stages. However, I have given only one perspective from which to evaluate our results. Each presenter is anticipating the impressions his/her demonstration today, or poster or oral presentation tomorrow will give. All of us would really appreciate hearing your impressions, suggestions, and discussions as you move throughout the various presentations.