7.1 Collective Data Movement Routines

  There are five basic types of collective data movement routine: broadcast, scatter, and gather, all-gather, and all-to-all. These are illustrated in Figure 6.

  
Figure 6: Basic collective data movement routines for a group of six processes. In each case, each row of boxes represents data locations in one process. Thus, in the broadcast routine, initially just the first process contains the data , but after the broadcast all processes contain it.

The broadcast routine broadcasts data from one process to all other processes in the group. The scatter routine sends distinct data from one process to all processes in the group. This is also known as ``one-to-all personalized communication''. In the gather routine one process receives input from each process and concatenates it in rank order. The all-gather routine broadcasts data from each process to all others, and on completion each has received the same data. Thus, for the all-gather routine each process ends up with the same output data, which is the concatenation of the input data of all processes, in rank order. In the all-to-all routine each process scatters distinct data to all processes in the group, so the processes receive different data from each process. This is also known as ``all-to-all personalized communication''.

In addition, MPI provides vector versions of all these 5 routines, except the one-all broadcast, in which each process can send and/or receive a different number of data items to and/or from each process.