Subject: Task identifiers

From: Rusty Lusk <[email-masked]>
Date: Tue, 02 Mar 93 15:16:20 CST

One issue that has been postponed several times is that of what
"process id's" will look like and what they will mean.  We would like
to offer a suggestion.  In this short note we discuss some of the
issues and make a concrete proposal.

Since the word "process" is so overloaded, we use the word "task" here
(possibly only a marginal improvement).

We define a task as a program unit that receives a message.  For most
systems, this is a single process.  In a multi-threaded environment,
this may be a process or a thread.  To allow a process to decide
whether to use threads to handle some operations, we believe that a
threads within a process should have the same task id; different
threads can be identifed by different tags, groups, or contexts.  For
example, an obvious way to provide a nonblocking user-defined
operation is to execute that operation in a separate thread.  This
should not require a separate task id.

The most important issue is whether the number of tasks is fixed or
not.  If the number is fixed, a sequential numbering of the tasks is
possible and simple.  However, the trend seems to be toward allowing
for the dynamic modificaiton of the number of tasks, and this is
impossible if a sequential numbering is used.  To avoid restricting
MPI to a fixed collection of tasks, we advocate using an opaque task
id.

This raises the issue of how the task ids of other processes are
determined.  We suggest that the rank of the task in a specified group
be used as a cannonical representation.  This rank is required to be
in the range 0 to (size of group)-1.  The usual arithmetic operations
can be performed on the rank.  For example, you can define your right
neighbor in a ring as ((myrank + 1) mod p), where p is the number of
tasks in the ring.

Proposal

In general, task ids should be an opaque type.  A routine gets its own
task id with MPI_MyTid.  To get the task ids of other tasks, an
application may use MPI_TidFromRank(group, rank).  A routine gets its
own rank with MPI_MyRank(group).  It is the task id that is used as a
source or destination in the point-to-point message-passing routines.
In Fortran it might as well be an integer.

Thus the new MPI routines being proposed are:

   MPI_MyTid()                          Get caller's task id
   MPI_MyRank(group)                    Get caller's rank in group
   MPI_TidFromRank(group,rank)          Get task id of a task from its
                                          rank in a group
   MPI_RankFromTid(tid,group)           Get rank in a group from a
                                          task id
       
An exception is made for the level 3 routines described in the recent
earlier message; these only have one group and we propose that task id
and rank in the "all" group be the same for them.  This is current
common practice.

Discussion

Note that since a task id is an opaque type, it cannot be sent to
another task and used as a task id on that other task.  Only the rank
is meaningful across tasks.

Making the task id an opaque type allows an implementation to optimize
the task ids.  For example, an implementation may choose to make the
task ids the physical node ids used by the hardware.  In addition, it
encourages users to use library routines to determine neighbors rather
than arithmetic on the task id (note that they can still get a
neighbor by adding one to their rank).  For example, the "best"
neighbor in a virtual ring depends on details of the underlying
hardware and software; calling a routine such as MPI_NbrInRing to get
the neighbor is a more portable way of getting a good neighbor than
MPI_RankToTask(MPI_MyRank(group)+1, group ).

Bill Gropp and Rusty Lusk

---

From: L J Clarke <[email-masked]>
Date: Wed, 3 Mar 93 21:34:17 GMT

Question: How do Rusty and Bill write so many proposals?
Answer: Surely they never sleep:-)
(Oh well, perhaps I won't get featured on the jokes newsgroups after
all.)

But seriously, fellow committee members, I recall that the contexts
subcommittee is about to prepare one or more proposals which will have
considerable bearing on this subject, so perhaps we should again defer
"task identifiers" until this has happened.

One crucial point raised (again), which I do not see evidence of having
really been addressed, is whether MPI will provide a reasonable
interface for a static process model, or for a dynamic process model. 
We have to decide what we want to do here, and I think we ought to
clearly state this as potential future users whom I have spoken to are
asking just this kind of question. 

What are the arguments for and against a dynamic process model?

To get started:

a) The MPP programming world has a whole lot more experience with the
static model, and as such it is better aligned with common practice. 

b) We anticipate people moving toward use of a dynamic process model. 

c) There are a whole load of issues in the dynamic model which
complicate implementations and imply lesser efficiency.

d) In order for programs to be portable in the MPI + dynamic process
model, the interface to dynamic process control has to be uniform as
well as the interface to message passing. Would MPI really consider
addressing the question of a uniform interface to process control at
this point? At least in the static case the process creation is not
part of the program and we can "ignore" it.

         /--------------------------------------------------------\
    e||) | Lyndon J Clarke    Edinburgh Parallel Computing Centre | e||) 
    c||c | Tel: 031 650 5021  Email: [email-masked]  | c||c 
         \--------------------------------------------------------/

---

From: Tony Skjellum <[email-masked]>
Date: Wed, 3 Mar 93 16:01:25 CST

I think that we agreed to defer dynamic issues to a future MPI standard
level, but I do agree that this is never guaranteed to happen, as Rusty
has said.  The lack of control for processes, and the lack of dynamic
group syntax/semantics worries me.  They will almost certainly appear as
non-portable extensions.

Perhaps the original N-month limit was too short to permit the best
possible standard.

- Tony