Collective communication allows simultaneous data transmission between multiple processes in a group. The mpi4py library provides collective communications, but only in the blocking version (that is, it blocks the caller method until the buffered data involved can safely be used).

The most commonly used collective communication operations are as follows:

• Barrier synchronization across the group's processes
• Communication functions:
  • Broadcasting data from one process to all processes in the group
  • Gathering data from all processes to one process
  • Scattering data from one process to all processes
• Reduction operations