Subject: Re: mpi-context: context and group (longer)

From: L J Clarke <[email-masked]>
Date: Fri, 9 Apr 93 21:19:33 BST

Paul Pierce writes:

> > For  me the arguments  have  piled up  in favour  of context and group
> > being separate and  independent entities.
> >
> > Lyndon
> 
> I agree, although I also see merit in associating a context with a group.
> 

Hey, some consensus here.  Magic!

BTW, Paul, I wanted to ask what you thought about my suggestions for
secure send/receive being bound to a context kind of thing (now
communicator object as of last mail to mpi-context), as opposed to
having different calls.  I think you are right about the microscopic
effect on the code.  I just tried to give both global default control
and per module instance control over the security question. 

> Consider a SPMD program with these calls. Assume the calls are loosely
> synchronous.
> 
> 		call to LibA (Group1)
> 		call to LibB (Group1)
> 		call to LibB (Group2)
> 
> In a loosely synchronous environment, messages for the next call can come in
> before the previous one has completed. Here we see two forms of overlap.
> 
> Within the call to LibA, we might get messages from processes which have already
> entered LibB. If LibA and LibB are independently written, they might use some
> of the same tags. To avoid messages from LibB matching receives in LibA, we
> must use different contexts. If we have static contexts, allocated when the
> libraries are initialized, each call in the library can quickly provide the
> context to its point-to-point calls. If we only have dynamic contexts,
> especially if contexts are carried inside groups, then a library must be
> prepared to dynamically allocate a new context on any call when it sees a new
> group. I know we discussed ways to do this locally, so the context could be
> created and cached locally on the fly without communication, but I find the
> idea of incorporating such code into every library call horrifying.

Paul, I have a model for libraries like this, which in my mail to
mpi-comm "Subject: mpi-comm: various (long)" I referred to as ClassB
libraries, which maybe you might want to think about.  It's quite
simple.  We write libraries just like this, which are akin to
encapsulated objects. 

We think in terms of library instances.  The library provides is an
instance constructor which accepts a group, creates context(s) for the
instance and constructs the instance, returning an instance id to the
user which is used to refer to the instance for all calls.  That is, all
calls including and up to the instance destructor, which asks an
instance to detruct itself. 

Our experience is that users do not find it difficult to manage this
model for ClassB libraries. 

> 
> So I propose that we need two forms of context, one that is quite static for
> protecting code, and one that is more dynamic for protecting groups.

I cannot see any difference between the latter of these two contexts
"more dynamic for protecting groups" and a global group identifier. 

> The only mechanism I know of that is adequate for protecting code is context
> alloctated via a nameserver. In MIMD programs, one cannot say much about the
> order in which libraries are initialized. Thus, if context is statically
> allocated at initialization time, there must be a way to obtain the global
> context value for a piece of code independently of other processes. A more
> static method, such as a MPI registry or a "dollar bill server" has the
> disadvantage of requiring a much larger value range for context. That uses
> precious bits in the envelope of every message. Once a context is allocated to
> a piece of code, it can be safely stored in a global variable without
> endangering thread safety or shared memory implementations, because no matter
> how many instantiations of the library store into the variable, they will
> always store the same value.

We find that with regard to operations within a process group, and in
particular to library instance construction and desctruction decribed
above, the main user program has a highly SPMD nature.  So we can
exploit sequencing.  This is a most valuable learning experience,
because we had similar thoughts to those you express here, implemented a
name server, and really didn't need it once (for this purpose). 

> 
> The point-to-point calls might be configured to accept (group, rank, context).
> In this configuration, the static context protecting the code is passed in
> explicitly, and the context protecting the group is inside the group object.
> 
> I'm not sure how this interacts with cross-group message passing. Perhaps the
> simplest solution is to use a well-known group context in such cases, which
> effectively disables group protection.

As I point out above, your "group protecting context hidden inside
group" really does just seem to me to be a global group identifier. 
Within the definition of context I see no reason why we necessarily will
cause a problem with intercommunication. 

When you say "use a well know group context in such cases" I take it you
mean a common ancestor like the "group context" of all processes or
something? 

I have promised, I will return quality discussion on intercommunication
next week. 

Did the points in this reply letter help, Paul?

Best Wishes
Lyndon "the temporarily less prolific"

         /--------------------------------------------------------\
    e||) | Lyndon J Clarke    Edinburgh Parallel Computing Centre | e||) 
    c||c | Tel: 031 650 5021  Email: [email-masked]  | c||c 
         \--------------------------------------------------------/

---

From: [email-masked]
Date: Fri, 09 Apr 93 15:43:35 -0700

> From: L J Clarke <[email-masked]>
> 
> Paul Pierce writes:
>
> > entered LibB.
> libraries ...

Yes, that is a matching concept.

> We think in terms of library instances. ...

I found this part hard to understand. However, if you propose to use the
mechanisms in your just previous mail:

> A ClassB library can accept a group as argument and create private
> GroupCommunicator objects.
> For example,
>            void class_b_procedure(int group, ...)
>            {
>              static int communicator = MPI_NULL_COMMUNICATOR;
>
>              if (communicator != MPI_NULL_COMMUNICATOR) 
>              {
>                  communicator = mpi_create_group_communicator(group,
>                                                    MPI_NULL_CONTEXT, 
>                                            MPI_SECURE_COMMUNICATOR);
>              }
>
>              /* do it */
>            }
> This example could  be generalised by adding a group "cache"  facility
> as described by Rik Littlefield.

First of all this code doesn't work - if the library is called with a
different group (see the LibB(Group{1,2}) example above) it will mistakenly
use the communicator for Group1 when called for Group2. This problem can be
fixed using cacheing. But...

This is exactly the sort of too-dynamic, too-intrusive mechanism I find
horrifying. I can't conceive of unleashing on the unsuspecting world a
standard that requires you to put code like that in every library call
(its even more complex with cacheing.) We _must_ come up with a
better mechanism.

The example mechanism I talked about might look like this:

	int my_context;

	void class_b_initialize() /* Called once at the beginning of time */
	{
		my_context = create_and_or_lookup_context("mylib");
	}

	void class_b_procedure(int group, ...)
	{

		/* do it using (group, rank, my_context) */
	}

Note the total absence of context maintenance in the arbitrary library
procedure. For group protection, the group must contain an additional embedded
context.

> "more dynamic for protecting groups" and a global group identifier.

You are right, its the same. My point is that group context is necessary but
_not_ sufficient.

> > alloctated via a nameserver. ...
> 

You learned that you have a SPMD universe. We have a mostly SPMD universe, but
we have customers already with MPMD applications.

One can argue that sequencing is acceptable for a static-process model, but it
is not adequate for dynamic processes. We have talked about defining MPI in
such a way that it is complete for a static process model but without limiting
its extension to a dynamic process model. So we must be careful - if we assume
sequencing now, we must do it in a way that allows for a nameserver later.

> Lyndon "the temporarily less prolific"

Paul