Subject: mpi-context: comment and suggestion

From: L J Clarke <[email-masked]>
Date: Tue, 6 Apr 93 20:31:15 BST

Dear MPI context colleagues.

I'd like to say something about contexts and groups and the extant
proposals ...

First off, we have two major concepts floating around, which I need to
define here for purpose of the discussion below. 

Group --- is an ordered collection of distinct processes, or formally of
references to distinct processes.  It provides a naming scheme for
processes in terms of a group name and rank of process within group. 

Context --- is a distinct space of messages, or more formally of message
tags.  It provides management of messages as a message in context A
cannot be received as a message in context B. 

Within these definitions there are exactly two themes in the extant
proposals. 

Marc Snir, in Proposal I, views Group and Context as identical.  This
simplifies the number of concepts in MPI, but does mean that we can have
intragroup communication and no way at all can you have intergroup
communication within the above definition of Group and Context.  

Rik and I amusingly coined the term "grountext" to describe the
group/context entity in this proposal. 

Tony Skjellum, in Proposal III, views Group and Context as independent. 
This means two concepts instead of one, but does mean that we can allow
intragroup communication and some intergroup communication with
restriction on how flexible we can make such communication. 

Proposals VIII and X are identical to III in the manner in which they
treat Context and Group as independent concepts.  Please consider
Proposal VII as not compliant with the above definitions of Group and
Context. 

We need to decide:

1) Are context and group identical or different?

2) Is intergroup communication provided?

Now I want to point out something about intergroup communication which
we have in our system and find most expressive and convenient, but does
not fit in with the above frameworks and the assumption that the message
envelope always contains just (context, process, tag). 

Receive in intergroup communication can wildcard on (sender group and
sender rank) or (sender rank), in addition to message tag. 

We (at EPCC) do, and want to do (in MPI) (written out in longhand
notation)

receive(group, group', rank, tag)

where group is the receiver group, group' is the sender group,
rank is the sender rank in group' and tag is the message tag.
The receiver can never wildcard group.
The receiver can always wildcard tag.
The receiver can always wildcard either (rank) or (group' and rank).

(In fact, group and group' in this expression are more like the
grountext of Marc's proposal or the "context" of historical proposal
VII, but never mind on that point.)

In the framework of Marc we can reasonably do intergroup communication
without wildcard on group'.  To do this we transmit group information in
messages and form a group which is the union of group and group'.  We
cannot add wildcard on group' by saying that to do that one forms a
union of group and all cases of group'.  This requires the sender to
always know too much about the detail of the recieve call with which it
is to match (i.e., that the receiver is or is not doing a wildcard).  If
you disbelive this, then you should probably argue that we do not need
source selection in point-to-point as you can use tag to choose the
source, as it is the same argument (and bogus in my opinion). 

In the framework of Tony we can reasonably do intergroup communication
without wildcard on group'.  To do this we transmit group information in
messages and choose a context for the pair of groups to use for
intergroup communication.  We cannot add wildcard on group' by using a
context agreed for such use between group and all cases of group'.  The
argument is the same as that above after a little substitution. 

If we are serious about intergroup communication then in my opinion we
really should provide the facility to wildcard on sender group.  This
throws up a small number issues, some of which I now address. 

No process addressed: I didn't mention process addressed communication
at all.  Perhaps the demons of speed are bothered by this.  Well, we
could do such as (context,process,tag), and the above does not exclude
it.  We can fit it in, of course. 

Size of point-to-point section: I said above "longhand notation".  Well
that is the most expressive and convenient notation, and if you ask me
then I think that (group,group,rank,tag) or (NULL,group,rank,tag) are
both acceptable for intragroup communication.  On the other hand one can
introduce some grunge syntax for intergroup communication which use the
same framework as intragroup communication and replaces group in
(group,rank,tag) with some glob object which is "shorthand" for (group,
group').  This is not the best syntax in the world but we can live with
it.  We can even fit in the process addressed stuff with this kind of
syntax as I have shown in Proposal X. 

Message envelope: You probably spot that this needs the sender group id
to go into the message envelope.  Perhaps the demons of speed are
bothered by this.  Well, you could have a different enevelope for
groupless communication, intragroup communication and intergroup
communication, and only pay the cost of the bigger envelope when you
need it.  This is going to take two bits for envelope identification. 
Big deal! It will anyway be natural not to match communications of
different kinds (e.g.  intergroup cannot match with intragroup,
groupless cannot match with intergroup) so the extra header bits would
be useful anyway. 

Unknown group: You probably also spot that the receive with wildcard on
group can pick up a group that the receiver knows nothing of.  I would
be happiest if the implementation of MPI at the receiver asked the
implementation of MPI at the sender about the group in this case, so
that the receiver never has to bother about the eventuality.  We (at
EPCC) could accept that the returned group identifier is a NULL
identifier.  This means that groups have to exchange flattened group
descriptions in messages in a reasonable way before they can make a
great deal of sense of intergroup communication.  Not ideal, but we can
live with it. 

Comments please?

Best Wishes
Lyndon

         /--------------------------------------------------------\
    e||) | Lyndon J Clarke    Edinburgh Parallel Computing Centre | e||) 
    c||c | Tel: 031 650 5021  Email: [email-masked]  | c||c 
         \--------------------------------------------------------/