Chair for Scientific Computing
University of Kaiserslautern-Landau (RPTU)
Paul-Ehrlich-Straße 36
67663 Kaiserslautern
Germany
Email: johannes.bluehdorn@scicomp.uni-kl.de
Room: 36-415
Phone: +49 (0) 631 205 2802
Research Interests and Activities
- Automatic Differentiation
- High Performance Computing
- Software Engineering
- AD tool development
Publications
2022
M. Aehle, J. Blühdorn, M. Sagebaum, N.R. Gauger
Reverse-Mode Automatic Differentiation of Compiled Programs Miscellaneous
arXiv:2212.13760, 2022.
@misc{nokey,
title = {Reverse-Mode Automatic Differentiation of Compiled Programs},
author = {M. Aehle, J. Blühdorn, M. Sagebaum, N.R. Gauger},
url = {https://arxiv.org/pdf/2212.13760.pdf},
year = {2022},
date = {2022-12-28},
urldate = {2022-12-28},
howpublished = {arXiv:2212.13760},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
J. Blühdorn, M. Sagebaum, N. R. Gauger
Event-Based Automatic Differentiation of OpenMP with OpDiLib Journal Article
In: ACM Transactions on Mathematical Software, 2022, (Just Accepted).
@article{nokey,
title = {Event-Based Automatic Differentiation of OpenMP with OpDiLib},
author = {J. Blühdorn, M. Sagebaum, N. R. Gauger},
doi = {10.1145/3570159},
year = {2022},
date = {2022-11-17},
urldate = {2022-11-17},
journal = {ACM Transactions on Mathematical Software},
abstract = {We present the new software OpDiLib, a universal add-on for classical operator overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on adjoint variables and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.},
note = {Just Accepted},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present the new software OpDiLib, a universal add-on for classical operator overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on adjoint variables and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.
M. Aehle, J. Blühdorn, M. Sagebaum, N. R. Gauger
Forward-Mode Automatic Differentiation of Compiled Programs Miscellaneous
arXiv:2209.01895, 2022.
@misc{nokey,
title = {Forward-Mode Automatic Differentiation of Compiled Programs},
author = {M. Aehle, J. Blühdorn, M. Sagebaum, N. R. Gauger},
url = {https://arxiv.org/pdf/2209.01895},
year = {2022},
date = {2022-09-05},
urldate = {2022-09-05},
abstract = {Algorithmic differentiation (AD) is a set of techniques to obtain accurate derivatives of a computer-implemented function in an automatic fashion. State-of-the-art AD tools rely on the source code of the implementation or internal representations of compilers building it.
We present the new AD tool Derivgrind, which augments the machine code of compiled programs with forward AD logic. Derivgrind leverages the Valgrind instrumentation framework for a structured access to the machine code, and a shadow memory tool to store dot values. Depending on the application scenario, no access to the source code is required at all, or the access is restricted to the parts defining input and output variables.
Derivgrind's versatility comes at the price of scaling the running time by a factor between 60 and 140, measured on a benchmark based on a PDE solver. Results of our extensive test suite indicate that Derivgrind produces correct results on GCC- and Clang-compiled programs, including a Python interpreter, with a small number of exceptions. While we provide a list of scenarios that Derivgrind does not handle correctly, most of them are academic examples or originate from highly optimized math libraries. We will therefore further study the potential of our tool in more complex software projects. },
howpublished = {arXiv:2209.01895},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
Algorithmic differentiation (AD) is a set of techniques to obtain accurate derivatives of a computer-implemented function in an automatic fashion. State-of-the-art AD tools rely on the source code of the implementation or internal representations of compilers building it.
We present the new AD tool Derivgrind, which augments the machine code of compiled programs with forward AD logic. Derivgrind leverages the Valgrind instrumentation framework for a structured access to the machine code, and a shadow memory tool to store dot values. Depending on the application scenario, no access to the source code is required at all, or the access is restricted to the parts defining input and output variables.
Derivgrind’s versatility comes at the price of scaling the running time by a factor between 60 and 140, measured on a benchmark based on a PDE solver. Results of our extensive test suite indicate that Derivgrind produces correct results on GCC- and Clang-compiled programs, including a Python interpreter, with a small number of exceptions. While we provide a list of scenarios that Derivgrind does not handle correctly, most of them are academic examples or originate from highly optimized math libraries. We will therefore further study the potential of our tool in more complex software projects.
We present the new AD tool Derivgrind, which augments the machine code of compiled programs with forward AD logic. Derivgrind leverages the Valgrind instrumentation framework for a structured access to the machine code, and a shadow memory tool to store dot values. Depending on the application scenario, no access to the source code is required at all, or the access is restricted to the parts defining input and output variables.
Derivgrind’s versatility comes at the price of scaling the running time by a factor between 60 and 140, measured on a benchmark based on a PDE solver. Results of our extensive test suite indicate that Derivgrind produces correct results on GCC- and Clang-compiled programs, including a Python interpreter, with a small number of exceptions. While we provide a list of scenarios that Derivgrind does not handle correctly, most of them are academic examples or originate from highly optimized math libraries. We will therefore further study the potential of our tool in more complex software projects.
M. Aehle, J. Alme, G. Barnaföldi, J. Blühdorn, T. Bodova, V. Borshchov, A. van den Brink, M. Chaar, V. Eikeland, G. Feofilov, C. Garth, N.R. Gauger, et al.
Derivatives in Proton CT Miscellaneous
arXiv: 2202.05551, 2022.
@misc{SIVERT2022,
title = {Derivatives in Proton CT},
author = {M. Aehle, J. Alme, G. Barnaföldi, J. Blühdorn, T. Bodova, V. Borshchov, A. van den Brink, M. Chaar, V. Eikeland, G. Feofilov, C. Garth, N.R. Gauger, et al.},
url = {https://arxiv.org/pdf/2202.05551.pdf },
year = {2022},
date = {2022-02-14},
urldate = {2022-02-14},
abstract = {Algorithmic derivatives can be useful to quantify uncertainties and optimize parameters using computer simulations. Whether they actually are, depends on how "well-linearizable" the program is. Proton computed tomography (pCT) is a medical imaging technology with the potential to increase the spatial accuracy of the dose delivered in proton-beam radiotherapy. The Bergen pCT collaboration is developing and constructing a digital tracking calorimeter (DTC) to measure the position, direction and energy of protons after they passed through a patient, and a software pipeline to process these data into a pCT image. We revisit the software pipeline from the perspective of algorithmic differentiation (AD). In the early subprocedures, several obstacles such as discrete variables or frequent discontinuities were identified, and are probably tackled best by using surrogate models. The model-based iterative reconstruction (MBIR) subprocedure in the end seems to be AD-ready, and we propose changes in the AD workflow that can reduce the memory consumption in reverse mode.},
howpublished = {arXiv: 2202.05551},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
Algorithmic derivatives can be useful to quantify uncertainties and optimize parameters using computer simulations. Whether they actually are, depends on how "well-linearizable" the program is. Proton computed tomography (pCT) is a medical imaging technology with the potential to increase the spatial accuracy of the dose delivered in proton-beam radiotherapy. The Bergen pCT collaboration is developing and constructing a digital tracking calorimeter (DTC) to measure the position, direction and energy of protons after they passed through a patient, and a software pipeline to process these data into a pCT image. We revisit the software pipeline from the perspective of algorithmic differentiation (AD). In the early subprocedures, several obstacles such as discrete variables or frequent discontinuities were identified, and are probably tackled best by using surrogate models. The model-based iterative reconstruction (MBIR) subprocedure in the end seems to be AD-ready, and we propose changes in the AD workflow that can reduce the memory consumption in reverse mode.
J. Blühdorn, N. R. Gauger, M. Kabel
AutoMat: automatic differentiation for generalized standard materials on GPUs Journal Article
In: Computational Mechanics, vol. 69, pp. 589–613, 2022.
@article{BluehdornGK2022,
title = {AutoMat: automatic differentiation for generalized standard materials on GPUs},
author = {J. Blühdorn, N. R. Gauger, M. Kabel},
doi = {10.1007/s00466-021-02105-2},
year = {2022},
date = {2022-01-03},
urldate = {2021-11-05},
journal = {Computational Mechanics},
volume = {69},
pages = {589–613},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2021
J. Blühdorn, M. Sagebaum, N. R. Gauger
Event-Based Automatic Differentiation of OpenMP with OpDiLib Miscellaneous
preprint arXiv:2102.11572, 2021.
@misc{BluehdornSG2021,
title = {Event-Based Automatic Differentiation of OpenMP with OpDiLib},
author = {J. Blühdorn, M. Sagebaum, N. R. Gauger},
url = {https://arxiv.org/abs/2102.11572},
year = {2021},
date = {2021-02-23},
howpublished = {preprint arXiv:2102.11572},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
M. Sagebaum, J. Blühdorn, N.R. Gauger
Index handling and assign optimization for Algorithmic Differentiation reuse index managers Miscellaneous
arXiv cs.MS 2006.12992, 2021.
@misc{sagebaum2021index,
title = {Index handling and assign optimization for Algorithmic Differentiation reuse index managers},
author = {M. Sagebaum, J. Blühdorn, N.R. Gauger},
url = {https://arxiv.org/abs/2006.12992},
year = {2021},
date = {2021-01-01},
howpublished = {arXiv cs.MS 2006.12992},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
2020
J. Blühdorn, N. R. Gauger, M. Kabel
AutoMat — Automatic Differentiation for Generalized Standard Materials on GPUs Miscellaneous
preprint arXiv:2006.04391, 2020.
@misc{BluehdornGK2020,
title = {AutoMat -- Automatic Differentiation for Generalized Standard Materials on GPUs},
author = {J. Blühdorn, N. R. Gauger, M. Kabel},
url = {https://arxiv.org/abs/2006.04391},
year = {2020},
date = {2020-10-06},
howpublished = {preprint arXiv:2006.04391},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}