Subject: Re: mpi-context: context and group (longer)

From: L J Clarke <[email-masked]>
Date: Tue, 13 Apr 93 10:32:55 BST

Dear MPI Colleagues

This never got to you, looks like I hit the wrong key in my mailer on
April 10.  Oops. 

Regards
Lyndon

---- Start of forwarded text ----
> From lyndon Sat Apr 10 17:50:29 1993
> Date: Sat, 10 Apr 93 17:50:29
> From: L J Clarke <lyndon>
> Subject: Re: mpi-context: context and group (longer) 
> To: [email-masked]

Paul Pierce writes:

> > We think in terms of library instances. ...
> 
> I found this part hard to understand. However, if you propose to use the
> mechanisms in your just previous mail:
> 
First off no way do I intend to use the simple example mechanism in my
previous email.  Of course it does not work if called with different
groups, it is just a silly little simple example, it was never meant to
work with different groups. 

Why did you find that part hard to understand? It is just an object
oriented approach with a parallel object.  I will try to explain again,
hopefully making myself more clear, at the foot of this letter. I hope
I can make myself clear.

> This is exactly the sort of too-dynamic, too-intrusive mechanism I find
> horrifying. I can't conceive of unleashing on the unsuspecting world a
> standard that requires you to put code like that in every library call
> (its even more complex with cacheing.) We _must_ come up with a
> better mechanism.

Fine, my approach will not do all of this anyway, please see below.

> The example mechanism I talked about might look like this:

Interesting! This example works in exactly one group/context and
therefore is utterly incapable of supporting multiple instances
operating on different data. 

> 
> 	int my_context;
> 
> 	void class_b_initialize() /* Called once at the beginning of time */
> 	{
> 		my_context = create_and_or_lookup_context("mylib");
> 	}
> 
> 	void class_b_procedure(int group, ...)
> 	{
> 
> 		/* do it using (group, rank, my_context) */
> 	}

> > I cannot see any difference between the latter of these two contexts
> > "more dynamic for protecting groups" and a global group identifier.
> 
> You are right, its the same. My point is that group context is necessary but
> _not_ sufficient.

I cannot concur.  I know I mailed a lot of discussion letters recently. 
I hope you can find time to read them because the mechanisms discussed
do mean that you only need the one context. 

> > > The only mechanism I know of that is adequate for protecting code is context
> > > alloctated via a nameserver. ...
> > 
> > We find that with regard to operations within a process group, and in
> > particular to library instance construction and desctruction decribed
> > above, the main user program has a highly SPMD nature.  So we can
> > exploit sequencing.  This is a most valuable learning experience,
> > because we had similar thoughts to those you express here, implemented a
> > name server, and really didn't need it once (for this purpose).
> 

> You learned that you have a SPMD universe. 

Negative! How can you read that from my words?

We have a highly MPMD universe, with lots of groups both overlapping and
distinct, which do lots of things of closed scope within the groups and
lots of things of open scope between groups.  We call this MSPMD or
M(SPMD), meaning multiple (single program multiple data). 

The observation we make, and it is hardly surprising, it that the
computations and communications associated with a process group are
locally SPMD-like in nature.  This is an entirely different observation. 
It is this property of a process group which allows parallel object
constructors to be called in sequence. 

> One can argue that sequencing is acceptable for a static-process model, but it
> is not adequate for dynamic processes.

Eh? I don't understand what you are suggesting sequencing can be used
for in static process model that it cannot in dynamic process model. 
Please do let me/us know. 

I'll second guess the sequencing you are thinking about, but I fail to
understand how it relates to static/dynamic process model.  This is
sequencing within groups of calls of context allocators.  This means
that when the processes of a group allocate contexts as a concerted (and
synchronous?) action they each bind same usage to the N'th context
generated by the group, for all N.  Of course we must realise that we
distinguish the N'th context generated by the process P (member of group
G) from the N'th generated by the group G, since the P will generally
also be a member of other groups (H, ...) and such groups (H, ...) are
not identical in membership to G. 

> We have talked about defining MPI in
> such a way that it is complete for a static process model but without limiting
> its extension to a dynamic process model. So we must be careful - if we assume
> sequencing now, we must do it in a way that allows for a nameserver later.
> 

Well I have no problem with this statement.  It would be trivial to
incorporate a name server into the discussions I have recently posted. 
In fact, when I get on to some helpful discussion of intercommunication
I will probably be asking for some name service anyway. 

The explanation promised above now follows. I really will do it mainly
by an exceptionally simple worked example.

We view ClassB libraries as encapsulated parallel objects.  Each object
class has a constructor, which creates an object, and destructor, which
destroys the object.  Each object also provides a number of public
operators which you can use to do things with the object.  We allow the
object to do things with user data as well as its encapsulate data, so
we have a hybrid between object oriented and the usual ad hoc. 

The user calls constructors in all processes within the group G in
sequence.  Of course, processes which are also in another group say H
interleave calls to consructors across H with those across G.  The
constructor called within G synchronises processes within G.  Its a
complete doddle. 

Let me give a very simple example for class Simple in C, assuming that
object identifiers are expressed as type "void *" and group and
communicator handles are expressed also as type "long".  In practice we
may implement object identifiers as opaque entities, but it's simpler to
show this (void *) stuff here.  I omit all error checks and standard
speed optimisations as well.

#----------------------------------------
# Library fragment

void * Simple_contructor(long G)
{
  Simple_state *state Simple_alloc_state(); /* Class state memory allocator */

  Simple_state->communicator = mpi_create_group_communicator(G,
                                              MPI_NULL_CONTEXT,
                                           MPI_SECURE_CONTEXT);
  return (void *)Simple_state;
}

void Simple_destructor(void *state)
{
  Simple_state *state = (Simple_state *)state; /* Macros can be faster */

  mpi_delete_communicator(Simple_state->communicator);
  Simple_state_dealloc(Simple_state); /* Class state memory deallocator */

  return;
}

void Simple_maketea_double(void *state, double *data, int items)
{
  Simple_state *state = (Simple_state *)state; /* Macros can be faster */
  /* next line is just to show how easy it is to get at the communicator */
  long communicator   = Simple_state->communicator; 

  /* make "items" cups of double precision tea :-)

     with 
     communication 
     using
     communicator */

  return;
}

void Simple_maketea_float(void *state, float *data, int items)
{
  Simple_state *state = (Simple_state *)state; /* Macros can be faster */
  /* next line is just to show how easy it is to get at the communicator */
  long communicator   = Simple_state->communicator; 

  /* make "items" cups of single precision tea :-)

     with 
     communication 
     using
     communicator */

  return;
}

#
#----------------------------------------

#----------------------------------------
# User fragment

/* some decls */

void *simple_ga, *simple_gb, *simple_ha, *simple_hb;
double data[16];
float  fata[16];

/* stuff */

simple_ga = Simple_constructor(G);
if (in_H) {
  simple_ha = Simple_constructor(H);
  simple_maketea_double(simple_hb, data, 16);
  simple_maketea_float(simple_hb, fata, 16);
  simple_hb = Simple_constructor(H);
}
simple_maketea_float(simple_ga, fata, 16);
simple_gb = Simple_constructor(G);
simple_maketea_double(simple_gb, data, 7);

/* more stuff */

#
#----------------------------------------

---- End of forwarded text ----
         /--------------------------------------------------------\
    e||) | Lyndon J Clarke    Edinburgh Parallel Computing Centre | e||) 
    c||c | Tel: 031 650 5021  Email: [email-masked]  | c||c 
         \--------------------------------------------------------/

---

From: Tony Skjellum <[email-masked]>
Date: Fri, 7 May 93 13:27:41 CDT

For your information, the following was sent (by me) to the IAC subcommittee.
- Tony

----- Begin Included Message -----