Project : tropics
Section: New Results
Keywords : static analyses, dead code, reverse mode of AD, checkpointing, snapshots.
Static Analyses for reverse AD
In addition to the activity, and read-write analyses described in section 5.1, we focus on specific analyses that take advantage of adjoint programs' special structure summarized for example on figure 2. Our goal is to improve the speed and memory usage of these adjoint programs. Indeed, classical data-flow analyses performed by compilers could do the job only partly, because those cannot detect not use the mirror structure of adjoint programs i.e. a forward sweep followed by a backward sweep, with matching control decisions. Moreover, specific analyses will run faster because they operate on the original code rather than on the larger and more complex differentiated code.
This year we formalized the notions of variables that are:
needed by the differentials of instructions that are upstream in the original program (TBR),
needed by the adjoint of the instructions that are downstream in the original program (adjoint liveness)
needed to re-execute a checkpointed piece of the original program, and to this end the variables that are overwritten by the adjoint of a piece of the original program (adjoint write analysis),
and expressed them together in a common formalization. We explicited the data-flow equations that specify the activity, adjoint liveness, adjoint write and adjoint read analyses, in the form of simple set equations. This allowed us to study their relationship and find an optimal order which is: study activity first, then study adjoint liveness, then finally compute the adjoint read, TBR, and adjoint write sets. This is part of the PhD work of Mauricio Araya-Polo.
During his student internship in spring, Benjamin Dauvergne implemented these analyses into tapenade and experimented on our set of validation applications, exhibiting speedups ranging from 0 to 30%. In the beginning of his PhD work, Benjamin Dauvergne investigated the application of the adjoint write analysis to build smaller snapshots for checkpointing.
The data-flow equations that specify the activity, adjoint liveness, adjoint write and adjoint read analyses still do not take checkpointing fully into account. We are currently developing a new set of data-flow equations, this time with checkpointing. This way we expect to be able to capture some hand manipulations made by experts on adjoint codes. For example, expert developers often take advantage of a series of successive checkpoints, included into a common parent checkpoint level. It is often possible to share some arrays that appear in many of the snapshots, so as to store them only once. We believe we are able to automate this improvement directly into an AD tool.
This work was presented in the Eccomas 2004 conference in Jyvaskyla, Finland, and at the AD2004 conference in Chicago, Illinois. It will also be published in the book of selected presentations from this conference.