Subject: Re: context proposals

From: [email-masked] (David Walker)
Date: Tue, 29 Jun 1993 13:34:52 -0400

> Date: Tue, 29 Jun 1993 17:15:07 +0100
> From: James Cownie <[email-masked]>
> 
> David Walker has stung me into putting out my ramblings on this
> subject too, so here they are.
> 
> Before I start on the detail, some things I hope are not too
> contentious.
> 
> What are Contexts for ?
> =======================
> The selling point of contexts is that they allow the development of
> modular software by insulating library routines from the state of the
> communications system when they are called.
 
     The insulation is needed in at least 2 cases:
	(1) Asynchronous entry into a library routine
	(2) Synchonous entry intoa library routine when messages are 
	    outstanding

> I think this is the SOLE purpose of contexts.
> 
> Issues with the current proposal
> ================================
> 1) It doesn't supply the security we want.
>    Users are allowed to arbitrarily bind context numbers which they
>    invent to communicators in an entirely anarchic manner. There is
>    therefore no possible security guarantee.
 
    I think MPI should allow secure programs to be written. I don't
    think it should guarantee security.

> 2) Because the user can locally construct communicators and bind
>    contexts to them the receiver of a message where the receive has a
>    wild-carded from field (given widespread current practice which
>    doesn't support selection on from, this is likely to be a large
>    proportion of receives !) has to perform a back translation from an
>    absolute process identifier into a group rank. Unfortunately this
>    is non-trivial to implement, and has to be done even if the message
>    is really an inter-group communication, since there is no easy way
>    (either at sender or receiver) of knowing this fact.
 
    I assume you're performing intergroup communication in which the
    rank is wildcarded on the receiver, i.e., the receiver will receive
    from any process in the other group. I don't see the problem here.
    The actual rank (relative to the send group) of the sender is part of the
    message envelope (according to my interpretation of Section 1.4.2).
    When calling MPI_QUERY to query the return status object, the rank 
    returned is relative to the send group. Maybe I'm misunderstanding
    what you mean.

> 3) The intra group communicators have this funny property of being
>    uni-directional, when inter group ones are not. This is most
>    peculiar. (It also seems unnecessary. If the range of valid
>    contexts really is large, then it should be possible to find one
>    which is free at both ends. This may be expensive, but then the
>    whole publish/subscribe mechanism already is !)

    In the routines that I posted today both intra and inter group
    communicators are unidirectional. Do you have a proposal for
    bidirectional intergroup communicators?

> 4) I have yet to see the example which implements (any of) the
>    collective communications with the properties we desire that
> 	a) The communicator passed in need not be quiescent
> 	b) The operation is insulated from the non-quiescence
> 	c) There is low cost in operating on different communicators
>  
>    All of the library examples shown to date assume that the library
>    can be statically bound to the group of processes on which it is to
>    operate.  This is certainly not the case for the collective comms
>    routines.  (Note that you need a context per group for the
>    collective routines, otherwise overlapping groups will fail. One
>    context for the library is insufficient).

    Well, my assumption has always been that whenever a new group is created
    a reserved system context is also created for use in collective
    communication. So when a collective communication routine is passed
    a communicator it calls MPI_COMM_DUP to create a new communicator
    that is the same as the original, except that it uses the reserved context.
    Can you give a more concrete example of what you perceive to be the
    problem here?

> Where do the problems come from ?
> =================================
> 
> 1) The insecurity arise from the ability of users to invent contexts
>    and use them with no system intervention. This is like being
>    allowed to access physical disk blocks while a file system is
>    running. 
> 
> 2) The need to back translate node ids comes from the lack of
>    negotiation in setting up communicators. If all the members of a
>    communicator understand it the same, then there's no need for a
>    translation. We hope and expect that this will be the case for
>    communicators used for inter-group communication. (I think many
>    many user's will be very confused if this is not the case, just
>    imagine the confusion from the debugging messages you can generate
>    with crossed over groups, e.g.
> 
> 	Process 0 		Process 1
> 	group {0, 1}            group {1, 0}
> 	       bind the same context
> 	send to 1		receive from 1
> 	receive from 1		send to 1	
> 	
>     etc.)  
> 
>    The process names can no longer be guaranteed to be anything more
>    than totally local.
> 	
>    The perverse thing here is that actually we could assert stronger
>    rules about the creation of communicators which ensured that the
>    same context was only ever bound to the same group, which should be
>    sufficient for the common (inter-group) cases, and then relax this
>    somewhat for the intra-group cases, which have to undergo
>    negotiation via publish/subscribe anyway. (Of course we would lose
>    the ability to flatten a group and distribute it through a shared
>    file...) 
> 
    Are you confusing intergroup (i.e., between groups), and intragroup
    (i.e., within a group) communication?

>    Anyhow I really dislike having to pay an extra cost everywhere for
>    something I may never use. 
> 
> Please Please Please let's find a way to avoid the back translation on
>    the common cases.
> 
> 
> -- Jim

David

---

From: James Cownie <[email-masked]>
Date: Wed, 30 Jun 1993 11:15:57 +0100

What are contexts for ?
=======================
OK, so I was a little bit restrictive in my definition of what
contexts are for. I agree with Lyndon that they may be useful for
structuring user code (but then maybe this is just a semantic question
of what a library is...), and I agree with David that there are two
different pieces of insulation needed by the library. However I
haven't seen any great addition to my statement. We all agree they're
to protect you from yourself (and others).

Security
========
On the security side I'd like to understand the gains from throwing
away enforced security. What do we lose by enforcing it ?

So far (and this may just because we haven't seen any examples...) I
have seen no gain from the ability to mix contexts into groups at
whim, which is what loses the security. I am open to arguments that we
need this, but haven't yet understood what we gain from this power
(other than the ability to shoot ourselves in the foot), and am
nervous about what we lose.

>     from any process in the other group. I don't see the problem here.
Yes this (or any case where the rank is wildcarded) is the case.
>     returned is relative to the send group. Maybe I'm misunderstanding
Please can we clarify some terminology, 

1) Send communicator group     == group in communicator used by sender
2) Receive communicator group  == group in communicator used by receiver

These groups are not (in general) the same, and have no relationship
to each other (they may even contain no proper intersection).

If the resulting rank returned to the receiver is NOT relative to the
receive communicator group, then 
1) The group in the receive communicator is irrelevant (except when
   the communicator is sent through).
2) The resulting rank is  useless. I cannot use it to reply to the sender.
   In my mind this was the main reason for getting this result.

I think 2.4.2 (on envelopes) should not be taken as gospel yet. We
need to think what we want to achieve, then worry about how to do it.

Directionality
==============

>     bidirectional intergroup communicators?
I don't actually believe he means this. Surely I don't need TWO
commmunicators just to send a message to someone else and receive a
reply ?

I believe
1) A communicator should always be able to be used both for sending through, and
   receiving from. It is a bi-directional object.
2) The context in a communicator has two functions, 
	when sending it is added to the message, 
	when receiving only messages which match it can be received
        through this communicator. 

This model (which give or take bi-directionality is [I believe] what
is in the proposal) has the implication that a single context number
has meaning at both the sender and the receiver. If I understand
David's proposal, he has two contexts in the communicator, so that the
context used for sending can be different from that used for
receiving. This makes the context numbers more local, and less
co-operation is needed to create them. (The ultimate end of this road
is to have a different context number for each member of the group...)

Collective communications
=========================
>     Can you give a more concrete example of what you perceive to be the

This is certainly a possible implementation. It's just that it isn't
what is in the current proposal. That is most explicit that a
communicator is a group and ONE context.

There has been a strong pressure to be able to let the user write
routines which are similar in their behaviour to the collective
routines we specify. This implies that the context mechanism should
have enough power to let the user do this. If this power exists, then
we should be able to write the collective routines this way, building
solely on the point to point and context proposals, WITHOUT requiring
additional help from the implementation of communicators and groups
(i.e. without a special reserved context, unless we have a way of the
user also being allowed reserved contexts.

So what I want to see is how to write a library routine using the
context proposal which has the following properties
1) It can be called on arbitrary groups. (You should probably read
   communicator for group, but I'm really interested in the group of
   processes, not the communicator). 
2) It can be called safely independently of the outstanding message
   traffic and queued non-blocking operations. (i.e. wild card receives
   can be outstanding, no tag space is available).
3) It can be called simultaneously in overlapping groups and correctly
   resolve the situation.

(Really we just need to see how the routine safely and cheaply
establishes a unique context for itself, I'll buy that once this is
done it can do the necessary comms !)

>     (i.e., within a group) communication?

Yes I am you are correct (David: IOU 1 Beer -- Jim). It's a long time
since I did Latin.  I'll re-phrase once more to make clear...

I am concerned that we are introducing expense into the simple common
practice case which is unnecessary for its implementation, solely to
support a much less common and more intricate case. This cost arises
because we cannot locally distinguish the two cases. 

-- Jim
James Cownie 
Meiko Limited			Meiko Inc.
650 Aztec West			Reservoir Place
Bristol BS12 4SD		1601 Trapelo Road
England				Waltham
				MA 02154

Phone : +44 454 616171		+1 617 890 7676
FAX   : +44 454 618188		+1 617 890 5042
E-Mail: [email-masked]   or    [email-masked]