| Junichi Shimada Managing Director, Director of Research Institute Real World Computing Partnership |
|
| Junichi Shimada Managing Director, Director of Research Institute Real World Computing Partnership |
I am going to discuss parallel and distributed systems, but I would like to limit my speech today to the content areas clarified by our study, as explained by Mr. Fujimoto from MITI. Since this area has no definite title yet, I tentatively call it "The Project for Seamless Systems"(Fig. 1).
Fig. 1
To begin with, I would like to explain the backgrounds for developing the Project for Seamless Systems. First, I will talk about the social background (Fig. 2).
Fig. 2
It is well recognized today that the use of computers has penetrated business, engineering workplaces, and homes; they have to become a primary, societal support technology. Because users expect more powerful computers, application software programs increase in size and the need to run computers at higher speeds, with better cost performance, is growing.
Now that the Internet and intranets are getting popular, demands on server systems for WWW, multimedia message handling, and databases and data mining are increasing as are the demands on personal computers. The sizes of the required systems will vary from small to large server, according to how and how often they are used.
Thus, recent computer usage requires not only high-speed computer systems which can meet specified purposes, but also systems that can comply with changes of environment.
Next, I would like to take a look at the technological background (Fig. 3).
To respond to the need for increased computational power, the research and development to establish higher computing performance has been carried out enthusiastically. Technology for high-speed processing is focused on research to optimize existing MPU architecture with current integration density, and is also focused on increasing clock frequency.
Fig. 3
Lately, high-speed microprocessors operating at a clock frequency of 500 MHz have been sold on the market. However, it is predicted that clock frequencies will reach physical limitations in the near future. To further accelerate processing speeds, parallel processing technology is essential, and that is why research and development on parallel processing technology attracts such attention. Even some personal computers incorporate parallel processing technology. In fact, 2-processor desktop PCs and 4-processor PC servers are now emerging in the market place.
The challenge of parallel processing acceleration is in finding the optimal parallel processing unit and to provide data to the processors responsible for such processing without delay (Fig. 4).
Fig. 4
The data exists in memory on board, memory on another board, or memory on another computer via a network. Recently, there are seams between these boards and computers, and the transfer capacity decreases at the seam during data transfer and increase communication delays. This problem is a factor that hinders the improvement of performance.
One way to eliminate a seam is to construct a processor architecture for mounting multi-processors and memory on the chip. This architecture would eliminate the seam between the processors and memory on a chip, however, there still remains a seam between memory and processors outside the chip.
In order to have high-performance processing in parallel systems with a hardware seam by minimizing its impact, it is necessary to develop an operating system that can provide an environment for effectively operating the entire system. This would include: the support of a parallel processing description with programming language and compiler to optimize the execution of individual applications, and the support of optimized compiler.
Conventional parallel computers use a special interconnection network technology and communication architecture, such as message passing and shared memory, to reduce the seams. With the recent emergence of high-performance workstations, PCs, and high-speed networks, research and development on cluster technology for parallel processing has begun to flourish although they are distributed systems. By using 1-Gbps networks in particular, it has become possible to construct workstation clusters comparable in performance to the massively parallel computers of the last generation.
This workstation clusters trend is changing the flow of the development of conventional parallel computers. The development of proprietary parallel computers, using existing MPUs, is shifting to the development of parallel computers using commodity computers and high-speed networks. The advantages of workstation clusters include improved cost performance with the reduction of development cost and time by using the newest commodity hardware and software, and utilization of the software on mature sequential computers. This enables a focus on software development for a parallel programming environment.
By adapting optical interconnection technology to a distributed system environment, a parallel processing comparable to massively parallel computers is expected to be developed, even in a distributed system environment (Fig. 5).
Fig. 5
In other words, various computers connected to one another, via optical interconnection, can eliminate the seam among computers. This type of computer connection can be considered a form of a "seamless parallel and distributed system."h Such a system makes it possible for computational scalability in a distributed environment. Computational demand can be satisfied by simply adding computers to fit the demand. Effective investment in facilities and facilitation of system construction will be accomplished.
A seamless parallel and distributed system can be viewed as a huge multiprocessor system from the software side. Thus far, system software on multiprocessor systems have been mainly for scientific computation, such as large-scale numerical computation, or for server computers (Fig. 6).
Fig. 6
For scientific computation, system software was constructed for performance rather than convenience; and for server computers, system software was mainly for handling high volume jobs on shared-memory computers. These individual jobs were handled by a single processor, but not processed in parallel as seen in scientific computations.
To make general applications including business applications work well on a multiprocessor system, we need to construct system software where applications are processed in parallel conveniently. However, this area still remains immature all over the world.
With the above background information in mind, I would like to explain the technological development target of our project, tentatively called the "Project for a Seamless System"(Fig. 7).
Fig. 7
Briefly put, our target is to realize and refine a seamless computing architecture.
We will develop the next-generation network, called Super Network, with optical high-speed communication technology where distributed computers are connected producing a throughput equivalent to a computer's internal bus, this network can also be reconfigured. We also plan to develop a system, viewed as a virtual supercomputer by users, by connecting various computers on a Super Network and by making computing resources like processors, memory, and disks seamlessly available.
Specific technologies must be developed in order to accomplish this target.
First, improved optical communication technology is important for the advancement of hardware. Therefore, research and development is required to develop an optical communication technology that can achieve transmission speeds of 10 Gbits/sec or higher at a reasonable cost (Fig. 8).
Fig. 8
Another technology required to accomplish our target is system architecture, the first of which is communication architecture (Fig. 9).
Fig. 9
First, the network is important. We need to find an efficient communication architecture that can handle the interconnection of computers on the order of 1,000 units, given a LAN environment.
The main stream of computers may be those with shared-memory multi-processors, and the execution of parallel programs in the LAN environment interconnecting these computers requires global addressing functions such as distributed shared memory. Although there are hardware standards which support distributed shared memory, such as SCI, they are generally not used because of high cost. Taking into account price-effectiveness and the support of a compiler and runtime environment, research and development of global addressing functions with light-weight hardware is required.
In addition, considering the improvement of integration density of a chip in the future, multi-processors on a chip will be feasible. At this time, we need to study what form of communication architecture is possible.
System software will then be important (Fig. 10).
Fig. 10
To provide a software environment on seamless parallel and distributed systems, research and development of programming languages, compilers, operating systems, libraries, applications, and benchmarks are required.
In developing programming languages to support parallel programs, it is important to base the current and future main-stream programming languages. This will make migration of the existing applications smooth, and will accelerate research and development. Applications include not only numerical computations, but also file systems and databases. Such applications mandate object-oriented technologies and require the development of an object-oriented technologies for parallel and distributed systems. By defining the positions of objects in a parallel and distributed environment and the meaning of copy, cache, and parallel execution, programmers will be provided with transparency in a parallel and distributed environment.
Then, automatic parallelization of programs in parallel programming languages and the development of optimizing compilers will be a key to providing scalability. To provide adaptability and reconfigurability, it is necessary to develop compilers which automatically adapt to the environment using execution history. It is difficult to find out the parallelism statically at compilation time in many applications like business applications. To optimize the execution of such applications, it is necessary to establish an optimization technique using execution history.
As for operating systems, it is required to develop system software for unitarily managing computing resources on a parallel and distributed system such as CPU, memory, disks, and I/O, and for improving fault-tolerance. It is also required to develop system software to provide users with a virtual supercomputer environment. Such systems will realize the function to allocate the computing resources required for a certain application to be executed at high speed, and to arbitrate computing resources required for each user or application.
Furthermore, libraries which can be ported to different computing environments while ensuring high performance must be developed. On a system we are assuming, heterogeneous computers are interconnected. Once libraries using a seamless environment are realized, every computer can use them seamlessly from a programmer point of view, rather than using libraries prepared for individual machines. By using object-oriented technology, high performance can be provided while hiding the implementation details from the users.
In addition, we must find out and improve the technological elements which may hinder the improvement of performance during the system construction. To do so, it is necessary to construct an environment to evaluate systems, including the hardware and software environments. Parallel and distributed systems will provide an environment to feed back the evaluation results automatically to users. Through performance evaluation, a benchmark for evaluating parallel object-orientated programs will be developed.
Now, let us consider what needs to be implemented to develop those technologies (Fig. 11).
While the areas of optical technology, computer architecture, system software, and applications will cooperate with one another in this project, it is not our research and development goal to construct one system combining hardware and software. Optical, communication architecture, and software are closely related with one another in a hierarchy of technologies; it is necessary to more clearly define the interfaces between them, but there may be a great delay unless research and development is conducted separately (as much as possible). If delays occur due to too much cooperation between these areas, we cannot lead the current fast progress in technologies.
Optical technology, computer architecture, system software, and application technology are positioned as follows - having technological interfaces and each being significant in separate research (Fig. 12).
Fig. 12
(1) In developing system software and applications, software which only operates on newly developed hardware will not be developed. Rather, a model system will be developed on the Cluster of SMP type hardware that is available now and expected in the future.
(2) The short-term (spanning three years) challenge for computer architecture is the research and development of an external bus for already commercialized hardware, specifically the PCI bus, or the next-generation network cards and network switches connected to an internal bus. Overcoming this challenge will facilitate the port of software developed on existing products by the software research group. In addition, it is important to evaluate the performance of architecture based on realistic software, and is desirable for the aspect of computer architecture. By verifying the superiority of a developed model system compared with commercialized systems in terms of performance, it is possible to build the foundation to turn the next-generation parallel computer systems into commodities.
(3) The short-term challenge for optical technology is to provide the computer architecture group with transmission modules with communication capacity of 10 Gbits/sec equal to or higher than the network performance on the current massively parallel computers.
(4) The short-term challenge for system software and applications is the research on cluster systems connected to SMP via high-speed network. In developing programming languages to support parallel programs, we will develop a programming model with parallel and distributed objects based on object-oriented technology, using Fortran for scientific computation, C++ for other applications in a distributed environment, and Java with the Internet and intranets; these arrangements will form a computing language basis. Research and development of compilers and libraries for those languages will be carried out. We will implement resource management functions for parallel and distributed systems based on operating systems running on SMP. We will carry out research and development of seamless environment software to realize a virtual supercomputer using these resource management functions.
(5) The long-term theme for computer architecture and optical interconnection is to see what system architecture should be when the per-chip integration density is improved in the future. At this time, we will work together to study on what benefits optical technology will bring about in chip-to-chip, board-to-board, and cabinet-to-cabinet connections, and how the computer architecture will change accordingly.
(6) The long-term challenge for system software and applications is to develop system software on the next-generation network implemented by computer architecture, and to verify the effectiveness of the model system.
(7) For our targeted research and development results, we will not simply verify the effectiveness of the model system, but we will standardize communication architecture, network, programming languages, and libraries to promote dissemination.
That is my explanation of the project for developing seamless systems, as discussed to date.