Subject: Re: context proposals

From: David W. Walker <[email-masked]>
Date: Wed, 30 Jun 93 09:36:28 -0500

What are contexts for ?
=======================
OK, so I think we agree what contexts are for.

Security
========
The current proposal doesn't enforce security; we don't have a proposal that does enforce
it. When we do then we'll be able to compare them, and see what the pros and cons are.
Of course, being able to shoot oneself (and others) in the foot is a proud part of the 
tradition and culture of parallel programming. I think as a group we should strive to
preserve our cultural heritage. Anyway, when I see a secure proposal I'll think about
it.

I'm still confused about Jim's comments on intergroup communication. I don't see any
difficulties. An important point is that the intergroup communicator is different for
the 2 groups. Suppose groups A and B each have a local context, called context_A and
context_B. Then when they get together to form an intergroup communicator this is what
they end up with:

                    Comm. in group A         Comm. in group B
                        group_A                  group_B             /* local group  */
                        group_B                  group_A             /* remote group */
                        context_A                context_B           /* recv context */
                        context_B                context_A           /* send context */

When group A sends to group B, we use a rank relative to group B and context_B (i.e.,
we use the remote group and the send context).

When group A receives from group B, again the rank is interpreted relative to group B,
and context_A is used (i.e., we use the remote group and the recv context).

The local group is not actually used. However, it needs to be there because somehow MPI
needs to figure out the rank relative to the local group to stick on the envelope.
When group B receives a message, it knows it's from process X of group A. It has no
problem replying.

Directionality
==============
On the topic of directionality, I guess I mean that contexts are unidirectional, not
communicators. Anyway, this is not really important. Since contexts are integers
we can always decide to use the one with the lowest ID number for communicating in
both directions. i.e., in the above example we coould use the smaller of context_A and
context_B as both the send and recv context. That way we end up with a communicator
consisting of just a  local group, a remote group, and a context. Maybe this is
preferable.

Collective communications
=========================
Jim writes:
> So what I want to see is how to write a library routine using the
> context proposal which has the following properties
> 1) It can be called on arbitrary groups. (You should probably read
>    communicator for group, but I'm really interested in the group of
>    processes, not the communicator). 
> 2) It can be called safely independently of the outstanding message
>    traffic and queued non-blocking operations. (i.e. wild card receives
>    can be outstanding, no tag space is available).
> 3) It can be called simultaneously in overlapping groups and correctly
>    resolve the situation.
I have made a distinction between MPI's collective communication
routines and collective librray routines (such as matrix multiply,etc).  The
collective routines in MPI need a group and a context in order to be implemented
in terms of point-to-point routines. The context can either be passed into the routine
in a communicator, or a reserved system context can be used. In general a library
routines will be passed a group and an array of contexts. In many cases only a single
context will actually be need so again the group and context can be passed into the
routine via a communicator. The context(s) that you pass a routine ensure that it can
be called safely regardelss of outstanding message traffic. Similarly, the use of
contexts ensures that a routine can be called correctly simultaneously by overlapping
groups. Note that "simultaneously" is not a correct word to use here within an SPMD
process model in which different groups operate within different conditional
branches. Thus:

	if ( in(group_A) ) then
            call matmul (A, B, C, comm_A)
        end if
        if ( in(group_B) ) then
            call matmul (X, Y, Z, comm_B)
        end if

this is how you would "simultaneously" call matmul in overlapping groups. If the groups
were not overlapping you could use an if-elseif construct.

I don't really understand Jim's concerns about the expense of implementation. Intergroup
and intragroup communication can easily be distinguished by checking if the local and remote
groups are identical. This could be done by setting a hidden bit in the communicator
structure
when its created.

David

---

From: James Cownie <[email-masked]>
Date: Wed, 30 Jun 1993 15:08:15 +0100

David writes
> Anyway, when I see a secure proposal I'll think about it.

I believe that Marc Snir's original proposal which bound groups and
contexts tightly together provided this security. (Though of course it
may have other problems !)

The reason you're having difficulty with my comments about between
group communicators is that you're considering your proposal, I'm
working in the light of the currently tabled context proposal as
presented at the last meeting. In that proposal a communicator
explicitly has only a SINGLE group and SINGLE context. (I actually
prefer your view of the world, but that's not what's been tabled at
present...) 

Most of our other misunderstandings arise from the same problem.

> Directionality
> ==============
> On the topic of directionality, I guess I mean that contexts are
> unidirectional, not communicators.
Exactly, the question is where does the context work as promised, to
filter messages arriving at the the sender, at the receiver, or
both. This is where the current proposal makes between group
communicators uni-directional, as they only have one context they have
to make it meaningful only at the target of a send, and not locally
meaningful for receipt of a message.

>  Anyway, this is not really
> important. Since contexts are integers we can always decide to use the
> one with the lowest ID number for communicating in both
> directions. i.e., in the above example we coould use the smaller of
> context_A and context_B as both the send and recv context. That way we
> end up with a communicator consisting of just a local group, a remote
> group, and a context. Maybe this is preferable.
I don't think this works, because that context number need not be available
at both ends. Of course it is possible to have a liaison when creating
the between group context and find a suitable single context number.
This is fine, and also allows the exchange of the relevant group
information. It is NOT in the current proposal. (Though presumably is
in yours, otherwise how does the communicator get both groups ?)

David writes:
> routines and collective librray routines (such as matrix
> multiply,etc).  
I am questioning exactly this distinction. I thought we wanted to be
able to write routines exactly equivalent to the MPI mandated
collective routines without needing greater power than we expose to
the user.

> The collective routines in MPI need a group and a context in order to
> be implemented in terms of point-to-point routines. The context can
> either be passed into the routine in a communicator, or a reserved
> system context can be used.
I think they need more than one, I think they need one per group...
(which is exactly the point.)

>  In general a library routines will be passed a group and an array of
> contexts. In many cases only a single context will actually be need so
> again the group and context can be passed into the routine via a
> communicator. The context(s) that you pass a routine ensure that it
> can be called safely regardelss of outstanding message traffic.
This seems horrible. It cuts across all of the modularity we wanted to
achieve. The caller now has to know for any library she calls how many
contexts it needs. This must be wrong.

>  Note that "simultaneously" is not a correct word to use here within
> an SPMD process model in which different groups operate within
> different conditional branches. Thus:
> 
> 	if ( in(group_A) ) then
> this is how you would "simultaneously" call matmul in overlapping
> groups. If the groups were not overlapping you could use an if-elseif
> construct.
Why don't you like "simultaneously" ? The events happen at the same time
(which is what causes the problem !), the fact that they come from
different lines of code does not seem relevant to me (as an
implementer !)

> I don't really understand Jim's concerns about the expense of
> implementation. Intergroup and intragroup communication can easily be
> distinguished by checking if the local and remote groups are
> identical. This could be done by setting a hidden bit in the
> communicator structure when its created.
As above, this is a result of working on two different proposals. I
understand how to do it in your world view, not in the Lyndon, Tony,
Mark, ... proposal.

James Cownie 
Meiko Limited			Meiko Inc.
650 Aztec West			Reservoir Place
Bristol BS12 4SD		1601 Trapelo Road
England				Waltham
				MA 02154

Phone : +44 454 616171		+1 617 890 7676
FAX   : +44 454 618188		+1 617 890 5042
E-Mail: [email-masked]   or    [email-masked]