Project : tropics
tapenade is the Automatic Differentiation tool developed by the TROPICS team. tapenade progressively implements the results of our research about models and static analyses for AD. From this standpoint, tapenade is a research tool. Our objective is also to promote the use of AD in the scientific computation world, and therefore in the industry. Therefore the team constantly maintains tapenade to meet the demands of our industrial users. tapenade can be simply used as a web server, available at the URL http://tapenade.inria.fr:8080/tapenade/index.jsp It can also be downloaded and installed from our FTP server ftp://ftp-sop.inria.fr/tropics. A documentation is available on our web page http://www-sop.inria.fr/tropics/ and as an INRIA technical report (RT-0300) http://www.inria.fr/rrrt/rt-0300.html
tapenade differentiates computer programs according to the model described in section 3.1. It supports three modes of differentiation:
the tangent mode that computes a directional derivative ,
the vector tangent mode that computes for many directions Xn simultaneously, and can therefore compute Jacobians, and
the reverse mode that computes the gradient .
A obvious fourth mode could be the vector reverse mode, which is not yet implemented. Many other modes exist in the other AD tools in the world, that compute for example higher degree derivatives or Taylor expansions. For the time being, we restrict ourselves to first-order derivatives and we put our efforts on the reverse mode. But as we said before, we also view tapenade as a platform to build new program transformations, in particular new differentiations. This could be done in cooperation with other teams.
Like any program transformation tool, tapenade needs sophisticated static analyses in order to produce an efficient output. Concerning AD, the following analyses are a must, and tapenade now performs them all:
Activity: The end-user has the opportunity to specify which of the output variables must be differentiated (called the dependent variables), and with respect to which of the input variables (called the independent variables). Activity analysis propagates the dependent, backward through the program, to detect all intermediate variables that possibly influence the dependent. Conversely, activity analysis also propagates the independent, forward through the program, to find all intermediate variables that possibly depend on the independent. Only the intermediate variables that both depend on the independent and influence the dependent are called active, and will receive an associated derivative variable. Activity analysis makes the differentiated program smaller and faster.
Read-Write: Each procedure has a number of arguments, that may be inputs, outputs, or both. Compilers use this to remove useless arguments. tapenade uses Read-Write information more specifically to detect aliasing, which is very harmful in AD, and to reduce the size of snapshots needed by checkpointing in the reverse mode. Read-Write analysis makes the differentiated program safer and less costly in memory space.
Adjoint Liveness and Read-Write: Programs produced by the reverse mode of AD show a very particular structure, due to their mechanism to restore intermediate values of the original program in the reverse order. This has deep consequences on the liveness and Read-Write status of variables, that we can exploit to take away unnecessary instructions and memory usage from the reverse (adjoint) program. This makes the adjoint program smaller and faster by factors that can go up to 40%.
TBR: The reverse mode of AD, with the Store-All strategy, stores all intermediate variables just before they are overwritten. However this is often unnecessary, because derivatives of some expressions (e.g. linear expressions) only use the derivatives of their arguments and not the original arguments themselves. In other words, the local Jacobian matrix of an instruction may not need all the intermediate variables needed by the original instruction. The To Be Restored (TBR) analysis finds which intermediate variables need not be stored during the forward sweep, and therefore makes the differentiated program smaller in memory.
Several other strategies are implemented in tapenade to improve the differentiated code. For example, a data-dependence analysis allows tapenade to move instructions around safely, gathering instructions to reduce cache misses. Also, long expressions are split in a specific way, to minimize duplicate sub-expressions in the derivative expressions.
The input languages of tapenade today are fortran77 and fortran95. Notice however that the internal representation of programs is language-independent, as shown on figure 4, so that extension to other languages should be easier. Development of the prototype of a tapenade for c started in december.
There are two user interfaces for tapenade. One is a simple command that can be called from a shell or from a Makefile. The other is interactive, using javaswing components and html pages. The interactive interface displays the differentiated programs, with html links that implement source-code correspondence, as well as correspondence between error messages and locations in the source. This is shown on figure 3.
tapenade is now available for linux, sun, or windows-xp platforms.
Figure 4 shows the architecture of tapenade. It is implemented mostly in java, apart from the front-ends which are separated and can be written in their own languages.
Notice the clear separation between the general-purpose program analyses, based on a general representation, and the differentiation engine itself. Other tools can be built on top of the Imperative Language Analyzer platform.
The end-user can also specify properties of external or black-box routines. This is essential for real industrial applications that use many libraries. The source of these libraries is generally hidden. However AD needs some information about these black-box routines in order to produce efficient code. tapenade lets the user specify this information in a separate signature file.