Many high performance applications spend considerable time packing data into flat communication buffers. MPI's Derived Datatypes provide an alternative by describing data layout, which empowers MPI to retrieve elements directly from application data structures. However, programmers find complex datatypes hard to use and are reluctant to invest the time and effort needed to rewrite packing code to use datatypes.
Torsten Hoefler and Marc Snir and I developed an algorithm that can automaticaly transform packing code to datatypes.
The algorithm is described in a our technical report A Transformation to Convert Packing Code to Compact Datatypes for Efficient Zero-Copy Data Transfer, and is also outlined in the (slightly outdated) poster Datatype Extraction: Extracting Declarative Descriptions of Data Layout from MPI Packing Statements.
A zipped tarball containing the source code for an Eclipse CDT plugin that implements the transformation as a refactoring can be downloaded from the following link:
Alternatively, the project git repository can be cloned using the following command:
git clone http://people.csail.mit.edu/fred/repos/datatypes.git
The plugin is tested with Eclipse SDK 3.6.1 (M20100909-0800), and currently requires a patched version of Eclipse CDT 7.0.2. The git repository also contains the patched CDT plugin, as well as the folder (cdt) that contains a patch and a README file explaining how to obtain and patch CDT yourself.
The plugin comes with a test suite (edu.illinois.datatype.tests) and has also been applied to all packing and un-packing code in the LU, MG, SP and BT NAS Parallel Benchmarks. The following patches port version 3.2 of the NAS Parallel Benchmark to C and converts the packing code to datatype code. The datatype code was generated by the above tool.
The following slide show screenshots demonstrate the features of the tool. The transformations take less than 0.5 seconds. The PDF can be downloaded here: