Subject: red herrings :-)

From: Tony Skjellum <[email-masked]>
Date: Tue, 20 Jul 93 09:51:33 CDT

Lyndon, the issues I see before us is as follows.

I. Thread dilemma for context allocation / safe communication allocation
------------------------------------------------------------------------
First, Ian's assertion that context == thread name is wrong.  Each
concurrent thread can have one or more context.  The single-threaded
limit can have more than one context... Contexts provide thread insulation,
however, so are useful with threads.

We have been asked to adjust the context model in which libraries
have control over their "safety" regardless of the state of message
passing when they are invoked in a loosely synchronous SPMD sense
(Pierce requirements).  I modified proposal to do so (with a reasonable
change).  I do not see how to implement the semantics in a thread-safe
manner, and I am soliciting opinions on whether or not to go back to
previous syntax, which also required the stronger quiescence property
[which Ian has noted is thread safe].

i) mpi_comm_make(group_A,...)  executes in thread-parallelism with
ii) mpi_comm_make(group_B,...)
where group B and group A have overlap.

mpi_comm_make() takes a group (possibly formed only locally, though in
parallel).  MPI would need a mechanism (an implementation-specific
mechanism) to get new context for each, while dealing with the multi-
threadedness problem.  Can one give an example of how to do this?

NB.
The collective communication subcommittee must handle exactly this problem,
for all collective operations, so I wonder what they have decided?
[Note, different contexts provide thread insulation, so that is one answer].

Model A: If MPI were in control of the thread package, in that it could KNOW
when a new concurrent thread were formed, and could create a new 
communication context for it, then we would be far along to a solution.
That is, we would actually have mpi_new_thread() or the like, e.g.,

		mpi_new_thread(group, comm)
		IN group
		OUT comm

A quiescent communicator is the output, providing bootstrap communication.
The call could increase the minimum granularity of concurrent threads
(significantly).

Model A (MPI has direct access to thread package)
|------------------------------------|
|     User code + Library code       |
|------------------------------------|
| MPI [with hooks to thread package] |
|------------------------------------|
|    Stipulated Thread package       |
|------------------------------------|

Model B (MPI is just another thread-safe piece of software,
that has no idea multithreadedness is occuring)
|------------------------------------|
|    User code + Library code        |
|------------------------------------|
|    User-level Thread package       |
|------------------------------------|
| MPI [no hooks to thread package]   |
|------------------------------------|

In Model B: The user has to make sure that communicators don't foul
up because of the use of threads.  This means that a quiescent 
communicator for each concurrent thread (group scope) must be created.
It must be bootstrapped from the MPI_COMM_ALL.  In this model,
one returns to using communicators to make communicators (per last
MPI meeting).  We achieve thread safety, but require the quiescence
property.

By the way, I think model B is more likely to occur in practice.

Default action
--------------
Model B is the default action for modifying the proposal (actually
it is unmodifying the proposal, to previous meeting state).

Desired action
--------------
We find a way to make ibraries have control over
their own communication safety, while permitting multi-threadedness.

II. Inter-communication 
-----------------------

To achieve inter-communication, we are forced to weaken the meaning
of communicator, and permit out-of-band communication.  We have
to add many functions.  Jim, Adam, and others are against this stuff.

I like having inter-group communication, personally, but I am arriving
at the opinion that we cannot get it into MPI1.  Therefore, I personally
favor dropping it, so that the standard itself can go forward (as we have
sufficient problems to describe intra-communication in the fully
multi-threaded model).

Default action
--------------
Restructure chapter to put all inter-communication stuff last,
so we can get intra-communication stuff passed on first reading.

Modify semantics to promote intra-communication safety, putting
greater onus on inter-communication mechanisms, rather than 
weaking basic mechanisms to allow "back-door" inter-communication.

Desired action
--------------
We develop better primitives for intercommunication before the
August 5 deadline (cf, Henderson stuff) and get them in the chapter.
There will still be resistance.  Requirements include that

i) there will never be out-of-band communication 
ii) only one communicator is bound to a context in a process
iii) user servers can be supported

--------------------------------------------------------------------
Please give whatever input you can.

- Tony
.	.	.	.	.	.	.	.	.      .
"There is no lifeguard at the gene pool." - C. H. Baldwin
"In the end ... there can be only one." - Ramirez (Sean Connery) in <Highlander>
"Entia sunt Multiplicanda Praeter necessitatem ad Infinitum"  - MPI Slogan
            -             -                       -
Anthony Skjellum, MSU/ERC, (601)325-8435; FAX: 325-8997; [email-masked]

---

From: Tony Skjellum <[email-masked]>
Date: Tue, 20 Jul 93 13:44:57 CDT

Lyndon, your comments are great... here is stuff from hender.
Please work with him and try to improve inter-comm stuff.  I will do
intra-comm stuff...

One note on your comments...
My favoring of dropping inter-comm is purely because of the political
pressure to do so.  Technically, I favor a superior model of message
passing that includes such capabilities.

I did not realize that the following has not been reflected.  I will
cc the reflector on this mail to be sure.

Thanks,
Tony
------------------------------------------------