Subject: MPI Goals and Scope: More Details Please?

From: [email-masked] (Tom Henderson)
Date: Tue, 9 Mar 93 13:12:34 MST

As a potential user of MPI, the discussion of I/O below disturbs me a bit and 
leads to a whole bunch of questions...  

> > We feel that the MPI standard should provide enough information to
> > specify an entire, source-code portable program.  
> > Bill & Rusty

> I agree.
> 
> I'd expect code like this (given an I/O everywhere model) to execute
> fine
> 
> int main(int argc,char ** argv)
> {
>     printf("Hello world\n");
> }
> 
> and print as many "Hello world" messages as the number of nodes on
> which I ran it.
> 
> -- Jim

I also agree with Bill and Rusty's statement about portability.  

Jim's example brings up a whole bunch of interesting problems.  First, should 
the printf() really print one "Hello world" message per node (process)?  I can 
think of several places where I wouldn't expect that behavior (Express 
"single" I/O mode for example).  

The next problem is "ordering".  Suppose Jim's example is modified slightly:

int main (int argc, char **argv)
    {
    int me;

    MPI_INIT(???);              /* Yes, I'm casting my vote for MPI_INIT(). */

    me = MPI_MYRANK();          /* another guess at syntax */

    printf("Hello world from process %d\n", me);
    }

Now what happens?  Do we see first-come first-served?  Do we see the 
"intuitive" ordering?  Do we get an error message?  

At the last meeting a straw vote lead to the adoption of "one processor can do 
standard I/O".  If we do this, then users will use vendor-supplied 
"parallel I/O" routines to get decent performance.  We cannot claim that the 
MPI standard provides enough information to specify an entire, source-code 
portable program unless we include I/O.  I think that there is general 
agreement that "parallel I/O" will not be included in MPI-1.  Maybe we could 
re-state the portability goal:  

  The ultimate goal of the MPI standardization effort is to provide enough 
  information to specify an entire, source-code portable program.  

I certainly won't use MPI unless I believe that this will happen with some 
version of MPI.  

I think we should re-examine the scope of the MPI effort and write down what 
we want to accomplish with MPI-1, MPI-2, etc. in much more detail than is 
currently in the draft introduction.  John Flower has proposed one 
prioritization.  Here's another (from my perspective as a user):  

1)   Static process creation and identification (and associated environmental 
     inquiry routines).  
2)   Contexts.  
3)   Heterogeneous communication semantics (I think we already have this).  
     All communication routines should support heterogeneous communication.  
4)   Blocking send and receive, contiguous data.  
5)   Non-blocking send and receive, contiguous data (status(), wait(), 
     cancel()?).  
6)   Creation/destruction of "static" groups (by "static" I mean processes 
     cannot join/leave an existing group).  Grid/toroidal process topology 
     would go here if we decide to include it.  
7)   Blocking collective communication, contiguous data (broadcast, 
     multiple-broadcast, concatenate, barrier, etc.).  
8)   "Simple" commonly used blocking collective operations (sum, max, min, 
     etc.).  
9)   Blocking send and receive, strided data.  
10)  Non-blocking send and receive, strided data.  
11)  Blocking collective communication, strided data.  
12)  Blocking send and receive with buffer-descriptor.  
13)  Non-blocking send and receive with buffer-descriptor.  
14)  Blocking collective communication with buffer-descriptor.  
15)  "User-defined" blocking collective operations (Express "excombine()"
     style).  
16)  Probe (and associated "precv()" routines, if necessary).  
17)  Non-blocking collective communication, contiguous data, strided data, and 
     buffer-descriptor (and "multi-cancel()"?).  
18)  Definition of parallel I/O.  
19)  Blocking parallel I/O.  
20)  Non-blocking parallel I/O.  
21)  Additional environmental inquiry and "setting" routines for performance 
     optimization (suggestions for system buffering, relative 
     node/interconnect performance, etc.).  
22)  Dynamic process creation and identification.  
23)  Dynamic groups (processes can join/leave an existing group).  
24)  "Safe" send and receive.  
25)  Ready-send and ready-receive.  
26)  General process topologies.  
27)  Even more neat stuff.  

Once a detailed list like this is agreed on, we need to decide where to draw 
the line between MPI-1 and MPI-2 (and maybe MPI-3, etc.).  Various straw polls 
about I/O indicate that 18-20 will probably be in MPI-2.  I imagine an 
absolute minimum might be 1-5 for MPI-1.  Here's three proposals for the list 
above:  

                  MPI-1           MPI-2           MPI-3, etc.

"Ambitious"       1-17            18-26           27

"Less Ambitious"  1-14            15-23           24-27

"Conservative"    1-11            12-19           20-27

I share John Flower's "EXTREME concern" about the complexity of the current 
design.  I believe that the MPIF must agree on SOME prioritization list and 
explicitly decide what is going to be in MPI-1 as soon as possible.  I also 
feel strongly that we should decide what will be in future MPI versions to 
give potential users some confidence that MPI will eventually be useful for 
developing source-code portable programs.   

Tom Henderson  (a potential MPI user)
NOAA Forecast Systems Laboratory
[email-masked]

(Thanks to Leslie Hart for some helpful suggestions.)

---

From: [email-masked] (Tom Henderson)
Date: Wed, 10 Mar 93 10:20:02 MST

Lyndon writes,

> Don't you think it will be some time before the world is ready to even
> think about standardisation of "parallel I/O".  I can't imagine trying
> to do such a thing at the moment. 

I agree.  I'm not proposing we do this now.  I'm proposing that we do it when 
it is as well understood as message passing is now.  I think "eventual 
standardization of parallel I/O" should be a stated goal to give potential MPI 
users confidence that MPI won't be just another message-passing library.  

> [stuff deleted...]
> I reckon you will, actually, because although it won't give you the
> portability asked for above, it will be your best way of getting some
> portability. 

I think Lyndon has a good point here.  I will use MPI if:

    A)  It gives me as much "portability" as existing solutions OR I believe 
        that this will eventually happen.  
    B)  It gives me equal or better "performance".  

I will borrow/build/buy whatever additional stuff I need to make my programs 
more "portable", if necessary (required for my current project).  I am hopeful 
that A will happen with some version of MPI.  Vendor participation gives me 
some confidence that B will happen.  

> > John Flower has proposed one 
> I'm going to walk down this list and give marks out of ten, with ten as
> highest priority and 0 as lowest priority, from my point of view, and
> some justification, from my point of view. 
> 
> [list deleted...]

Thanks Lyndon, this is exactly what I was looking for.  At this point I feel 
it is extremely important that the MPIF come to some agreement about a list 
like this.  It is a lot more important than my actually liking what's on the 
list!  :)

> [stuff deleted...]
> 
> I am again ambivalent.  I think it important to identify what items
> should be suitable for future MPI, but I'm not at all confident that it
> will be practical to commit ourselves to a schedule for trying to
> standardise such items. It will be harder to agree on standards for the
> items which are deferred.
> 
> Best Wishes
> Lyndon
> 

I didn't mean to imply an absolute time-frame for the introduction of future 
MPI features.  (It doesn't make sense for the more "researchy" stuff.)  
Prioritization is more important to me than a strict schedule.  

Tom Henderson

---

From: L J Clarke <[email-masked]>
Date: Wed, 10 Mar 93 17:30:30 GMT

> Thanks Lyndon, this is exactly what I was looking for.  

Oh, good, glad I did that then!

> At this point I feel 
> like this.  

AGREED!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

> Prioritization is more important to me than a strict schedule.  

Sorry, Tom, I misread your email. 

It's over and out for a week from me, colleagues, as I'm visiting my
folks for the weekend and a couple of work meetings - surprise surprise,
one of them to talk about MPI to a potential user, urgle urgle urgle!
May your mail boxes be temporarily lighter ;-)

Best Wishes 
Lyndon

         /--------------------------------------------------------\
    e||) | Lyndon J Clarke    Edinburgh Parallel Computing Centre | e||) 
    c||c | Tel: 031 650 5021  Email: [email-masked]  | c||c 
         \--------------------------------------------------------/