Hidehiko Tanaka
ProfessorDepartment of Electrical
Engineering
Faculty of Engineering
University of Tokyo
|
|
Hidehiko TanakaProfessorDepartment of Electrical Engineering Faculty of Engineering University of Tokyo |
1. Introduction
High-performance computers are now essential tools in scientific and technological research and development. With their high-speed processing capability and large-scale storage capacity, computers are now important tools in simulation experiments and in the field of design. Their importance will continue to grow, since they are vital in high-tech areas such as designing new chemical substances, structural stress and destruction analysis, simulating the global environment, calculating nuclear fusion, seismic focus analysis, analysis of resources, genetic analysis, large-scale economic analysis, and complex system simulation. Future developments in high-speed and large-capacity supercomputers will clearly play a significant role in the research and development of advanced technology for the 21st century. Parallel processing is the key technology to make large-scale processing capability possible.
The performance of single processors has improved remarkably in recent years, due to progress in architecture technology such as reduced instruction set computers (RISC) and in semiconductor technology. By constructing parallel-processing machines that use processors of this type as their processing elements, it will become possible to outstrip the performance of conventional supercomputers that use the pipeline architecture-while at the same time reducing the cost. For some applications such as transaction processing, conventional general-purpose machines are being replaced by parallel-processing machines consisting of a large number of processors.
However, parallel processing is not a technology that is only of interest in specialized fields such as scientific and technical computing and transaction processing. It is a technology that will be fundamental to the implementation of advanced interfaces between humans and computers, and to the development of multimedia in the broader sense-as a basic technology that will support all kinds of information processing in the future.
This notion of "parallel processing for future general-purpose computers" is based on research into data flow computing that was carried out in the 1980s, and research on parallel inference machines under the auspices of the Fifth Generation Computer Systems project. Although the basic technologies needed for parallel processing were more or less established by these research projects, a new scheme of future parallel processing technologies should be established on the basis of current advanced processor technology, LSI technology, and compiler technology. While several high-performance parallel computers are commercially available at present, their uses are limited, and parallel processing is still in its infancy, since it can hardly be claimed yet that its intrinsic capabilities have been fully demonstrated.
In this article, research trends in parallel processing are reviewed in the context of major events, both past and present.
2. Data Flow Machines
Parallel processing is based on the principle of data flow. A process can start when the input data for each operation are available. This principle was clarified during research on ILLIAC IV and in the development phase of the IBM 360/91 in the late 1960s, and it was established as the general data flow principle by the mid-1970s. A great deal of research has been carried out since the late 1970s and during the 1980s; prototype machines were built at the University of Manchester, MIT, and at the Electrotechnical Laboratory. The description of data flow processes is essentially functional, and the data flow description language is therefore a functional language.
The research showed that, although the data flow principle is indeed useful, and can extract a high degree of parallelism from complex processes, there are two problems. First, a hardware implementation of the data flow principle would be quite expensive; and secondly, if the structured data represented by databases are not handled successfully, the copy overhead is liable to be extremely large. Research has therefore moved toward a software implementation of the principle.
3. Parallel Inference Machines
The core research of the Fifth Generation Computer Systems (FGCS) project proposed general-purpose information processing systems using a parallel logic programming language. A prototype of parallel inference machines, PIM, was developed in 1991. Parallel logic programming languages essentially functional programming language using data flows; that is, many process requests (called goals) exist in parallel, waiting for the data to arrive and starting when ready for processing. Parallel logic programming languages are similar to functional languages, but they involve more generalized data flow processes, due to the introduction of generalized data flow, known as "unification."
In the FGCS project, several applications have been developed using these machines. Research has shown that normal problems involve considerable parallelism, and that error-prone programs can be realized by the automatic implementation of synchronization between processes, rather than being produced by programmers, when writing parallel-processing programs, so that the number of bugs are reduced dramatically. This raises significant doubts about the FORTRAN and C languages that are commonly used for practical programming at present. More specifically, this suggests that handling complex processes in such languages may cause major problems. Languages such as FORTRAN and C are really only for problems of regular structures or for simple parallel processing.
The absolute performance of the PIM machines that have been developed is inferior to that of RISC processors, which has improved rapidly in recent years. Nevertheless, these research and development efforts have been of vital importance in studying the principles of the systems concerned. The difference between parallel logic programming languages and C should be analyzed by examining in detail the differences in efficiency that occur when the same processes are executed using each, and the differing extents to which hardware can support in executing the same processes. Preliminary assessments show that parallel logic programming languages have an overhead almost double that of C, but require only one-third of the amount of machine code, while the performance is two or three times faster than that using C.
4. Data Parallel Processing
Data-parallel processing, applying the same processes to different data simultaneously, is familiar to the extent that it involves applying conventional sequential programming to parallel processing without any changes-facilitating a very high degree of parallelism. There are also many applications in this area, such as information retrieval. These factors encouraged computer applications of this type, and the development of suitable computer hardware. One example is CM-2, with applications such as semantic retrieval of original document data such as newspaper articles.
Since data parallel processing does not require any communication between programs that are running simultaneously, the hardware requirements are not so strict, and the hardware is relatively easily constructed-allowing the building of massively parallel machines, each with tens of thousands of processors.
5. Special-Purpose Computers
Another practical example of the use of parallelism is special-purpose computers for limited applications. Some of the recent systems of this type include the FX system for radio astronomy; the Grape system for solving many-body problems in the gravitational field; HAL, a logic simulator for verifying logic design in large-scale integration (LSI); the NWT system for numerical wind tunnels, installed at the Aerospace Institute; and fingerprint-matching machines.
It is characteristic of these systems that applications are narrowed down and analyzed to clarify processing flows; hardware based on the applications is then built, and the necessary software for it is written. That is, the behavior of the processes involved is known in advance, and portions that can be independently processed are implemented by parallel hardware. The degree of parallelism at execution time can therefore be predicted during the design phase, simplifying the operating systems and compilers.
For example, the FX system passes radio waves received by multiple antennas through separate filters, processes the results independently using FFT, and then uses parallel processing to correlate the combined output. With simple processing, the effect is therefore quite large: the equivalent of 500 giga-FLOPs (floating point operations) can be achieved with a system clock of 3 MHz. The Grape system simply computes the gravitation in parallel, and achieves a high performance total of 1.2 tera-FLOPs by combining a large number of pipelines (1,920). The NWT system, not specifically structured for hydraulic calculation, combines a large number of conventional pipeline computers (140) by crossbars, is suitable for local intercommunication-intensive calculations, and can achieve a running performance of over 100 giga-FLOPs in some cases.
6. Projects
There are several massively parallel processing projects that are active, including the "High-Performance Computing and Communication Project (now a part of the NII) " in the USA, "RWC project," the Ministry of Education's "Massively Parallel Priority Area Research Project," and the "Computational Physics-Specific Computer Development Project" here in Japan.
The goal of the RWC project is to achieve flexible information processing, which will have an important role to play in the future. A study of the architecture required for this will lead to the construction of the basic machine on which flexible information processing will take place-the prototype RWC-1 machine, now undergoing research and development. This R & D is being carried out on the basis of an analysis of the requirements of massively parallel machines suited to real-time and fine-grain processing. These requirements are essential for future massively parallel processing, and in this sense, RWC project will develop the key technologies needed for massively parallel computers.
Research in the "Massively Parallel Priority Area Research Project," which started in 1992, with completion planned for March 1996, is being carried out by groups of researchers at various universities. The goal of this project is to develop the fundamental technologies needed for massively parallel computers. The research is centered on the development of a massively parallel computer system called Jump-1. Jump-1 includes 512 processors. The COS operating system for massive parallelism runs on it, and a language called NCX, an extension of C that has been customized for massive parallelism, is implemented. Applications written in NCX run on Jump-1.
Research and development in the "Computational Physics-Specific Computer Development Project" is based at the Computational Physics Research Center at the University of Tsukuba. The goal is to achieve 300 giga-FLOPs using 1024 pseudo-vector processors called CP-PACS. The hardware will be ready by March 1996, and will be in operation from fall 1996.
7. Commercial Parallel Machines
During the 1980s, small-scale commercial parallel-processing machines were built mainly for research purposes. By the late 1980s, the Thinking Machines Corporation was already selling machines with powerful parallel-processing applications. The 1990s have seen sales starting for several large-scale massively parallel machines with high-peak performance, such as the CM-5 from Thinking Machines, T3D from Cray, SP2 from IBM, Paragon XP/S from Intel, KSR-2 from KSR, SPP Exemplar from Convex, nCube3 from nCube, VPP500 from Fujitsu, Cenju-3 from NEC, and SR2201 from Hitachi. These are all pioneering machines for massively parallel processing on the commercial market, mainly for scientific and technological calculations based on FORTRAN and C. The market is still small, and programming tools and language implementations have yet to be completed.
8. Conclusion
I have outlined above the current status in large-scale massively parallel processing machines, mainly in the research trends. The performance of single processors is improving rapidly, and the aggregate performance of massively parallel machines can easily surpass that of conventional vector supercomputers by many orders of magnitude. However, implementations, operating systems, and programming tools capable of taking advantage of the performance of such machines have not yet been completed-a field that will require further long-term study.
Research on massively parallel machines aims to create tools that will be important for advanced scientific and technological developments in the future, and the results of such research are likely to have a significant influence on research in other fields. It can be expected that various types of hardware will emerge for use with various applications. I expect that with the gradual accumulation of application packages and expertise in parallel processing, it will become possible to create a genuinely user-friendly and high-performance information environment. Parallel processing is an absolutely fundamental technology for it.
RWC's goal of flexible processing is therefore a vital element in parallel processing, representing a future-oriented research topic that aims to achieve genuinely user-friendly information processing. As this research proceeds, we can expect its usefulness and practicability will become clear, and a vast stock of parallel processing code will be accumulated, thus emerging a new world of information processing.