Generalized Distributed Computing with PVM

In the evolution of the PVM system for cluster and heterogeneous distributed computing, high-performance scientific applications have thus far been the main technical drivers. The computing model, as well as specific software features have been influenced by the requirements of scientific algorithms and their parallel implementations. We believe that by extending this infrastructure along certain important dimensions, the PVM system will be able to cater to a much larger class of application categories. The goal therefore is to enable generalized distributed computing within heterogeneous networked frameworks, i.e. to evolve both a conceptual model and a software infrastructure that integrally support high performance scientific applications as well as other general purpose applications, including, but not limited to, distributed teleconferencing and groupware systems, heterogeneous and multi-databases, high speed on-line transaction processing and geographically distributed information systems. This enhancement to PVM, termed the General Distributed Computing (GDC) layer has recently been undertaken; GDC facilities will consist of infrastructural support for the required operations, and are briefly described below

• Parallel IO facilities: the GDC layer extends PVM functionality by providing support for distributed and parallel input and output to disk files as well as for terminal interaction. The standard Unix file semantics are retained to the extent possible; in addition, facilities for shared but non-conflicting reading and writing, using a variety of different interleaving and consistency semantics are provided. In essence, exclusive, independent, interleaved, and serialized access are supported by the parallel IO subsystem. In addition, support exists for data compression and encryption, as well as for file shadowing - a valuable feature for reliability.

• Synchronization and Locking: the GDC subsystem provides facilities for mutually exclusive access to resources. The model permits these resources to be application dependent - the provided primitives allow for locking an abstract resource identified by a string valued identifier and an integer. Thus applications may establish a convention according to the nature of their requirements and utilize the GDC facilities without any loss of generality or functionality, but with substantial flexibility. For example, to implement record-level file locking, applications may request a lock on the abstraction identified by the filename and record number. In addition to efficient locks, the GDC subsystem also incorporates certain deadlock detection heuristics and, based on option switches, will either attempt recovery or return control to the user after setting locks to a ``safe'' state.

• Client-Server Support: The native PVM facilities are geared towards asynchronous, communicating processes, and do not provide sufficiently high-level access to applications using the client-server paradigm. The GDC subsystem alleviates this deficiency by permitting server components of applications to export services that are identified by symbolic names, and for client components to invoke these services in a location-transparent, heterogeneous, and efficient manner. These features comprise a significant extension of the standard remote procedure call model in that (1) PVM and GDC automatically locate remote services; (2) support for load balancing, using multiple servers, is provided; (3) invocation semantics may be either procedure-argument based or message based; and (4) a certain level of failure resilience is built into the system.

• Transaction Processing: Design and initial implementation and testing efforts are in progress for a distributed transaction facility in the GDC layer. This facility provides the normal transaction processing constructs including beginning and ending transactions, aborting transactions, and nested transactions. These features are consistent with the usual atomicity, consistency, isolation, and durability semantics of traditional database systems. However, since the GDC layer facilities may be used in conjunction with standard PVM message passing features, certain enigmatic situations arise. For example, if a transaction's scope includes sending and receiving of messages, the correct actions in the case of an abort are unclear, as restoring the system to a previously valid state is complex and possibly intractable. We are exploring several alternatives and will proceed incorporating these features into the GDC layer as soon as ``correct'' semantics are decided upon.

Our preliminary experiences with the GDC subsystem indicate that enhancing the PVM features to support generalized distributed computing, with specific focus on commercial, business, and database applications, is very valuable and is being increasingly accepted and adopted. Our performance measurements have also been very encouraging: during testing, overheads of a few to several tens of milliseconds were observed for most of the facilities outlined above, such as locking, synchronization, and parallel input and output with shadowing.