TABLE OF CONTENTS At the end of the 1980s, high-end parallelism made the transition from bus-based shared-memory systems, which are today called symmetric multiprocessors (SMPs), to designs in which the system memory was broken up and packaged with individual processors to make a scalable computer. Typical of the early designs were the hypercubes, originated at Caltech and popularized by Intel and other companies. Later designs used different network architectures, while retaining the distributed-memory design. Such machines came to be called distributed-memory multiprocessors.
Because the individual processors in early distributed-memory systems used 32-bit addressing, they could not address every word in the aggregate memory of a large configuration. Therefore, most such machines used some form of "message passing" to communicate data between processors. Whenever one processor needed a block of data resident in the memory of another, the owner would have to explicitly send the data to the processor that needed it, which in turn executed a receive to extract the data block. These send and receive operations were typically programmed via calls to system libraries from Fortran or C. The drawback of such libraries was that each one was specific to a particular machine, making codes difficult to port between systems. This problem was solved by the release of Message-Passing Interface (MPI), a standard interface for message-passing calls from high-level languages. Today, MPI is used on the majority of programs written for scalable parallel machines.
However, MPI and its relatives have a serious drawback they are not easy...
