Section: New Results
Communication Optimization over High Speed Networks
Participants : Alexandre Denis, Raymond Namyst, François Trahay.
The NewMadeleine communication subsystem introduces fundamental changes in communication request handling and optimizations. Traditionally, communication libraries, being synchronous, tightly link the communication requests to the application workflow, and therefore transmit incoming packets immediately to the lower network layer without any accumulation. On the contrary, NewMadeleine keeps accumulating packets in its optimization window while the NICs are busy. As soon as a NIC becomes idle, the optimization window is analyzed so as to generate a new ready-to-send request to be transfered through the card: NICs are exploited at their maximum (they are not overloaded when there is a high demand of transfers and under exploited when there is not) and the communication optimizations are made just-in-time so they closely fit the ongoing communication scheme at any given time.
When at least one of the multiplexing units becomes idle, an optimization function is called to elect the next request to be submitted to each idle unit. In doing so, it may select a packet to be sent from the optimization window, or for instance, synthesize a request out of several packets from that window. A wide panel of arguments may be used as an input to the optimizing function. The optimization function is to be selected among an extensible and programmable set of strategies . Each strategy aims at some particular optimizing goal. A strategy is itself made of one or more tactics that apply some elementary optimizing operations selected from the panel of usual operations.
By design, NewMadeleine is able to be used by multi-threaded applications, and utilizes itself multi-threading. We have measure [39] the impact of various levels of multi-threading support on performance.
Finally, we have integrated NewMadeleine into MPICH [36] and evaluated its performance and the efficiency of our optimization mechanisms when using MPI.
