Subject: Re: Subset comments

From: Tony Skjellum <[email-masked]>
Date: Fri, 7 May 93 13:24:37 CDT

IACers,

I think that a subset is essential, because of the number of features
in MPI that are so remote from current practice; furthermore, if one were
to look at the liberal vs. conservative nature of committee (as others
have observed), it is not equal over all features/proposed features.  
Hence, I offer my thoughts.  This is not meant to be a "flame."

For instance, I have argued for the addition of a two-word match against
tags, in order to allow easier layerability.  A tag would be matched as
follows (following Jim Cownie):

		(received_tag xor (not dont_care_bits)) and care_bits

This would allow the user, not only complete freedom in use of tags,
but also the ability to develop further layers on top of MPI that partition
the use of tag.  I will bring this idea up again at the next meeting,
at the second reading of pt-2-pt.  It is necessary to have this, and it
is a small step away from usual practice.  However, it is hard to convince
people to add this, despite its negligible impact on performance (two
logical operations, instead of one, assuming user passes the one's complement
of the dont_care_bits.)  However, its impact on MPI flexibility is immense.
Hence, I view this feature as essential to the subset and full MPI likewise.

Another instance.  Contexts.  We are arguing for/not for contexts that
are independent of groups.  Contexts as an extended, system-registered part
of the tag field help us to build libraries that can co-exist, register
at runtime, and do not interfere with the message-passing of other parts of
the system.  I want an "open system"; hence, I want to see the tag partitioning.
Contexts work very well in Zipcode (my message passing system, developed at
Caltech, LLNL), and are helpful with the libraries we develop on Zipcode.
Because vendor systems do not have contexts, Zipcode, when it layers on
vendor systems must requeue messages.  This is undesirable from a performance
standpoint.  Hence, it is highly desirable for MPI to provide contexts of
the type I describe, as a simple tag registration/partitioning mechanism
that is understandable as an extension of existing practice.  If contexts
are limited, and their is a mechanism to find this out (environment), then
messaging systems like Zipcode could do requeueing of messages as necessary,
or manage contexts themselves at times, and use the precious "fast" contexts
on user-specified communications, leaving others to be requeued and slower.
Hence, I view contexts (whether plentiful or scarce in a given implementation)
as essential to the subset, and the full standard.  As Don Heller of Shell
has noted...
"contexts allow the development of a software industry [for multicomputers]."

Groups.  Yes, we need them too.  They are important for managing who is
communicating with others.  So, they have to stay in the subset as well.
Rik Littlefield, Lyndon Clarke, and I have argued (and will continue to do
so) for attributes based on group/context-scope.  This would allow the
methods implementing communication to be changed in MPI for each group/context
scope, permitting optimizations.  This is not current practice, except
in our Zipcode 1.0 release, which has this useful capability, but it 
justifiably useful.  I think these ideas can/should remain in the standard
and in the subset.  

There are multitudinous types of send/receive that we are currently
proposing, but not  using in practice currently, which have been
proposed and accepted with relative ease by MPI.  Practically, send,
receive, receive unblocked, is enough, provided the kernel is smart
enough to do overlapping of communication and computation.  Actually,
if the semantics of the Reactive Kernel were taken, which allows the
system to handle all memory management, then receive would provide the
pointer to the data, and send would be like free, with an allocate
mechanism like malloc.  These reduce the number of copies of data,
except when extremely regular data structures are in use (less and less
likely).  The RK semantics thought out by Seitz et al are remarkably
simple, but highly optimizable, and can even work very fast in shared
memory.  These semantics do not appear as options in MPI; we only have
multitudinous buffer-oriented operations.  When memory management units
are involved, binding control of the memory and messaging operations
gives even more opportunity for the system to optimize.  Allowing the
user to receive messages without having first to know their size is
elegant, and simplifies error issues.  

As we all know, there are faster implementation strategies than RK
semantics for message-passing that are low level, such as channels, active
messages (unchampioned in this standard), and shared-memory (eg, CRAY
Tera 3D).  These need not be part of this standard, but it would be
helpful if the standard were unhostile to such possibly efficient
implementation mechanisms.  The "buffer descriptor" approach in MPI
is the best match (being the highest level interface) to a runtime
system that exploits channels, and/or active messages, and/or remote
memory writes, etc.  The optimizability of the highest level is complemented
by the fact that the user no longer knows if a buffer is ever formed
on the local or remote end [well it should be written to make that so].
Furthermore, heterogeneity can be encapsulated in transfers at this level.
Hence, I am convinced that "buffer descriptor" stuff should remain in the 
subset.

The committee has shied away from defining the process model, and this
has led not only to a very static model (arguably OK), but a predilection
to the SPMD (handling of groups, definition of subgroups, need for 
dynamic contexts diminished, etc).  All of these factors make the standard
backward-looking if so adopted, and make it really difficult to justify
in the distributed environment.  I am not sure why this has happened, but
it is unfortunate.  It means that MPI codes will be partially portable,
but not totally, as each system will have different process management.
SPMD programs will be reasonably portable, as the process management is
simple, and therefore localized.  The handling of the host/node model
is not well established in MPI, and may not be suitably supported.  That
would be a big problem to my mind.

To summarize, it is my view that the enabling mechanisms: group, context,
tag selection, and buffer descriptors described above are essential aspects
of a standard and subset, and should not be sacrificed.   MPMD programming,
the host/node model, should be supported.

- Tony 

.       .       .       .       .       .       .       .       .      .
"There is no lifeguard at the gene pool." - C. H. Baldwin
"In the end ... there can be only one." - Ramirez (Sean Connery) in <Highlander>

Anthony Skjellum, MSU/ERC, (601)325-8435; FAX: 325-8997; [email-masked]

----- End Included Message -----