Research on Document Collection Information Processing Technology
Hidekazu Arita
Advanced Technology R&D Center,Mitsubishi Electric Corporation
As computer networks advance, individuals can now freely access huge volumes of document data, but this results in so-called information overload, and they cannot process such massive quantities of information. If, however, a great amount of document information can be used effectively with R&D of new technology, this will provide a new way of supporting individuals' creativity. This technology should be information processing technology for document collections, rather than just for separate documents.
To enable individuals to handle a great amount of document information, R&D of key technologies such as for accessing document information, self-organizing document collections, visualizing information space, and manipulating information space, and R&D on an integrated system of these are essential.
Fig. 1 shows the framework of the research on document collection information processing technology conducted in Novel Functions Mitsubishi Laboratory. We view this technology from two aspects of document information. The first aspect is the utilization of document information. We believe it is essential to carry out R&D that considers the purpose of the use of documents for real world computing. Conventional research focuses on a technology to produce a document which satisfies certain conditions such as information retrieval. However, the next stage requires a technology for information space strolling, where there are not clear goals and where the information space of the document collection is viewed, the contents of any document of intersets are read, and information is retrieved when necessary. In the next stage, a technology for information utilization will be required which helps users think more profoundly by manipulating the information space as a workspace on a trial-and-error basis supported by facts (document information) and thus using document information to be creative.
The second aspect concerns information ownership. It might be better to distinguish first seen documents (those outside a user's scope) and often seen documents (those inside a user's scope). Those documents outside a user's scope are suitable for document collection information processing in a statistical approach, because they should be handled objectively, while those documents inside a user's scope are suitable for an approach which reflects the user's subjectivity.
Novel Functions Mitsubishi Laboratory studies four stages of technologies as document collection information processing technology to reflect these two aspects: a knowledge representation technology for how documents should be represented in information space, a technology for self-organizing information space of document collections represented in some form, a technology for visualizing information space to present self-organized information space to users in an understandable way, and a technology for manipulating visualized information space according to how the user wants.
Novel Functions Mitsubishi Laboratory conducted R&D on four key technologies shown in Fig. 1 from bottom to top: vector representation of documents, self-organization of information space, and visualization of information space.
Fig. 2 shows a sample screen of the current experimental system, in which objects (documents and words) are placed in a three-dimensional information space. Objects are arranged so that semantically close objects are placed close to each other with a self-organizing mechanism. Users can arrange objects subjectively as appropriate. This experimental system has the following features:
Map presentation of information space
Expands the self-organized structure information in the space. This helps information space strolling.
Dynamic reorganization of structure
In the neighborhood of an object arranged subjectively by a user, those objects related to this object are placed according to restrictions. This feature helps information utilization.
Control of computation
Changes priority of computation. Although visualizing the self-organizing process of a large information space requires a very large amount of computation power, turn-around time is improved by giving computation priority to the space which interests the user.
The following R&D activities are planned:
Research on context processing in document collection information processing
Research on semantic processing and context processing in assisting information utilization using user-adapted domain knowledge.
R&D on a large amount of document information
It is planned to deal with document information on a practical scale as an information environment for assisting user's creative activities. The scale will be the typical number of documents one user may encounter in his/her lifetime.
Research on information integration in document collection information processing
Research on assistance in extracting information not gained from separate documents until multiple documents are associated with others by self-organization and visualization of the information space (e.g., overall trend of information, reliability of information, and overall image of a certain event).
(NOTE) The terms "document collection information processing", "information space strolling", and "information utilization" are used to describe the activities in the Novel Functions Mitsubishi Laboratory, and are not necessarily commonly used.