Subject: Communication contexts

From: [email-masked] (Dennis Weeks)
Date: Fri, 15 Jan 93 15:11:17 -0600

Rusty Lusk writes:
> David, perhaps you could create a mailing list on which this discussion can
> take place, parallel to the subcommittee mailing lists, so it doesn't have to
> go to the whole committee.  Thanks.
I believe the default inclusion of everyone is a good idea, although creating
a separate mailing list might be appropriate if some mpi-comm members wish to
"unsubscribe".  In the meantime, I will add one brief opinion:

I felt that the discussions of contexts in the topology subgroup were quite
sufficient, and it would probably be better to have the single subgroup which
covers both groups and contexts, so that both will appear in the standard in
a style which does not make them conflict with each other, and/or to avoid a
proposal where groups and contexts would be totally redundant.

I definitely agree that it is an important topic, and that the discussions
last week in Dallas were far away from getting a proposal for groups and
contexts that would be unanimously acceptable, so much work remains to be done.

---

From: [email-masked] (Tom Henderson)
Date: Fri, 15 Jan 93 14:59:49 MST

> | 
> | On the topic of communication contexts I think (at least) 4 approaches have
> | been proposed.
> | 	1) Implicit communication contexts as in MPI1 controlled by push/pop.
> | 	2) Explicit registration as in Zipcode
> | 	3) Explicit communication contexts should be merged with explicit
> |            group contexts. Since groups cannot communicate with each other
> | 	   a new communication context can be created by creating a new group 
> | 	   with the same members as the current group.
> | 	4) Chuck Simmons of Oracle has suggested that communication contexts
> | 	   are really a particular type of thread, and so can be handled
> |            using existing threads packages.
> | If you have any ideas, or are strongly for or against any of the above
> | approaches let mpi-comm know so we can initiate a discussion of this topic.
> 
> I think that 1) contradicts our genral agreement to avoid state as much as
> possible.  4) is a problem on systems that don't support threads, and we have
> more-or-less agreed to be consistent with thread packages but not depend upon
> them.  3) should be discussed along with the converse: that contexts rather
> than groups should be the primary concept and groups defined in terms of
> contexts.
> 
> Rusty

I'm not sure, but I believe that Zipcode uses push/pop to control contexts 
(Tony, please correct me if I'm wrong).  I'd like to add a 5th approach, 
basically the one proposed by Paul Pierce:  

5)  Contexts are used exclusively to insure that message collisions will not 
    occur if independently developed sub-programs are combined.  Contexts and 
    groups are orthogonal.  Contexts and threads are orthogonal.  Each message 
    has an associated context and tag.  Message context is managed by library 
    routines and is completely out of a user's control.  Message tag is 
    selected by the user.  

    The method of managing message contexts is a separate issue (assuming we 
    want contexts).  Existing proposals are:  

    5a)  Stack-based management (objected to due to hidden states).  
    5b)  Explicit registration with user-defined "names" (probably requires 
         some communication).  
    5c)  Explicit registration by a central authority ("dollar bill" 
         registration mentioned by Jim Cownie.)

There is enough confusion about contexts and groups that we may want to keep 
this discussion open to everyone.  If the concensus is to make a separate 
mailing list, please both of us to it.  

Thanks,

Tom Henderson
[email-masked]

Leslie Hart
[email-masked]

---

From: Tony Skjellum <[email-masked]>
Date: Fri, 15 Jan 93 18:05:36 CST

In Zipcode, contexts have a group-wide scope.  Groups of processes can
involve more than one context.  For instance, if 10 processes are involved
in several stages of a calculation, each stage might have a different notion
of how to do messaging, but each would use all the processes.  Groups
are important, because they allow control over the scope of global operations.
Contexts are important because they provide guarantees to sub-programs
about restricted scope of messages.  Contexts are added typing-like
information, but which is controlled by the system, not ad hoc by each
user program.  When they determine to start messaging, sub-programs
request a context from a "postmaster general" in Zipcode.  Currently, this
is a group-oriented request (loose synchronization) resulting in each
group member getting the context, and other information.  In a lower-level
system, simpler tactics might be possible.  The way we did it was supposed
to avoid possibility of races or other problems like that.

To build large-scale software, without globalization of the local use of
messaging resources, and to provide scalable algorithms with efficient
global operations, both groups and contexts are needed.  The push/pop
state is an implementation detail of little importance.

-Tony

---

From: Charles Simmons <[email-masked]>
Date: Sat, 16 Jan 93 03:08:56 PST

> them.  3) should be discussed along with the converse: that contexts rather

Allow me to clarify my sentiments.  The example of context usage given
in the MPI draft documentation appears to explicitly assume the
existance of threads, and here contexts appear to be being used as
threads.

>     routines and is completely out of a user's control.  Message tag is

I could strongly support almost all of the above.  [The part I
wouldn't support is a "tag" as I will explain below.]  We do our work
on the nCUBE, which, as you may know, has a message passing interface
essentially equivalent to that proposed by MPI.  We ran into the exact
problem quoted above: it was difficult to write libraries that
we're guaranteed not to conflict in their usage of message types.

The usage of the word "contexts" in the above is equivalent to the
usage of the word "ports" amongst operating systems programmers.
Because "port" is slightly less overloaded than "context", I prefer
the word "port".

Note that ports are sufficiently powerful that they subsume the
concepts of "contexts", "pids", and "message types" or "tags" as used
by MPI.

>The only expression stated so far about the difference between tags and
>contexts I had gleaned was that context should now be wildcardable (IE no
>-1 for All contexts) while a receive ALL or some other MASK variant would
>be allowed on tags.

I agree that there is little difference between tags and contexts.
This helps explain why ports are so good at subsuming both contexts
and tags.

>         registration mentioned by Jim Cownie.)

I don't particularly understand the above, probably because I haven't
been listening in on enough of the discussion.  I think I understand
what you mean by stack-based management, and we do use the word
"registration"...

When using ports, stack-based management isn't an issue for the same
reasons you don't manage pids and message types using stack based
management.  For example, when sending a message, instead of using the
MPI

	push_context (contextp);
	send (dest, type, buf, buflen);
	pop_context (contextp);

you use

	send (port, buf, buflen);

In our usage, a "port" is simply a queue to which messages can be
sent.  The receiver can remove a message from the head of the queue.
Wildcards are neither needed nor implemented, making ports more
efficient.  The efficiency arises from the fact that a process can
have multiple ports.  In MPI, you essentially implement one receive
queue for each pid.  If you want to use a wildcard to access messages
out-of-order, you need to waste time scanning the queue.  With
multiple ports, however, you can set up multiple queues so that you
never need to access anything other than the head of a queue.

Sometimes ports do need to be "registered".  In our terminology,
registration is the process of publishing in a well-known location
sufficient information about a "port" created by one process that
another process can send messages to that port.  This is a very high
level action that is almost outside the scope of MPI.  We use a
nameserver for this purpose.  (The nameserver is accessed via a
well-known port, i.e. a port whose bit-pattern is a well-known
constant.)  [Do note that we require that the bit-pattern representing
a port be allowed to be transmitted between to processes without
the operating system needing to know that the bit-pattern represents
a port.  This means that accessing the nameserver is not required in
order to use a port.]

For groups, we have thought about extending the concept of a port to
an array-port.  The simple implementation of this allows each process
in a group to create a port.  These ports are all sent to a central
location which creates an array of ports and broadcasts the array to
all ports in the array.  The complex memory efficient implementation
may require processes of a group to be created in a special fashion.
Basically, it allows the address of a port for a specific process in a
group to be easily computed by another process given the port of the
first process in the group and the index of the target process in the
group.

-- Chuck