Subject: Cacheing

From: James Cownie <[email-masked]>
Date: Wed, 25 Aug 1993 15:56:06 +0100

At the last meeting the context committee presented a proposal for a
cacheing facility within MPI to allow the user to associate data with
two classes of MPI objects (groups and comunicators).

This proposal was greeted with less than total enthusiasm,
particularly (it appeared) because of concerns about store allocation
issues.

Let me explain why I am in favour of a cacheing proposal, and why I don't
believe that store allocation is a (large) issue.

Recap on the extant proposal
============================

The proposal provides the ability for user code to be given unique keys
which it can use to store a void * on any group or communicator.

If you like there is a symbol table associating the pair 
(MPI_OBJ,key) with the void * value.

(A better implementation might be to have the lookup table stored on
the MPI object, and actually implement it as a flat table indexed by
the key, but this is not necessary for the scheme to work)

In addition it associates with each key value two functions, 

a destructor function called by MPI when an MPI object which has a
value associated with this key is destroyed,

and 

a replicator function called by MPI when an MPI object which has a
value associated with this key is replicated. (e.g. MPI_GROUP_DUP).

Note that these functions are ALWAYS called synchronously to the
execution of the user code, since they are called when the user is
making an MPI call. (MPI_xxx_DUP, or MPI_xxx_FREE), they are NEVER
called by the MPI system "in an interrupt routine".

What does this proposal buy us ?
================================

1) It makes it much easier to support layering of functionality on top of
   MPI. 
e.g. Topology information needs to be associated with MPI
   group objects, and copied or deleted as the groups are changed.

   User collective operations may need to have a separate
   communicator from that passed to them (so that they operate in their
   own context) They can now associate this with the communicator
   they are passed, AND (because of the call back) correctly free it
   when the parent communicator is deleted by the user code.

2) It allows the user to write collective functions which behave
   EXACTLY like the MPI collective functions. Without the callbacks
   this is impossible, as there is no point at which the user code
   gains control to free its data structures when MPI objects are deleted.

The major gain comes from the call back functions. The cacheing is
merely a convenient place to hang the callback functions, and know
which ones should be called.

What does it cost ?
===================

I don't think the cost is large. Indeed it could be implemented using
the function re-naming scam on top of existing MPI without any
problem. As I point out above, all of the store allocation can be done
(should be done ?) in user space on the way into or out of the system.
There is no need for this to impact anything "inside the kernel" at
all. The user functions are only called while we're on the user stack
anyway, as a result of a user call to MPI. They're also only called at
non-time critical points (how often does a good code create or
replicate groups compared with how many times it communicates ?).

Conclusion
==========
I think is buys us a lot (in expansibility) and costs us a little.
I think we shold have it !

-- Jim
James Cownie 
Meiko Limited			Meiko Inc.
650 Aztec West			Reservoir Place
Bristol BS12 4SD		1601 Trapelo Road
England				Waltham
				MA 02154

Phone : +44 454 616171		+1 617 890 7676
FAX   : +44 454 618188		+1 617 890 5042
E-Mail: [email-masked]   or    [email-masked]

----- End Included Message -----