Section: New Results

High-performance memory migration in Linux

Participants : François Broquedis, Nathalie Furmento, Brice Goglin, Pierre-André Wacrenier.

We diagnosed a dramatic performance problem in the memory migration subsystem of the Linux kernel caused by the quadratic complexity of the model. We reworked the core implementation to restore a linear behavior and thus achieve a constant asymptotic throughput, as well as dramatically reduce its temporary memory consumption. This low-level improvement was integrated in the standard Linux kernel as of 2.6.29 and further optimized in 2.6.31. It enables high-performance static memory migration within multithreaded applications on Linux for the first time [29] .

To further help dynamic management of threads and data within applications, we proposed a dynamic migration model called Next-Touch . It lets data buffers automatically follow their accessing threads when the workload balance changes in the machine. This work has been presented to the Linux kernel community [31] as an improved way to efficiently support irregular and dynamic parallelism within high-performance computing applications.

This low-level work is offered to user-space applications through the memory manager, MaMI , that we added to our thread library Marcel . MaMI allows users to allocate memory by specifying a NUMA node, or based on the First-Touch policy, or the Next-Touch policy. It also supports memory migration between nodes, migration cost prediction, as well as remote read/write access overhead prediction.

MaMI is used by ForestGOMP to place threads and memory in a combined manner so as to take memory affinity into account [26] , [24] .