Combinatory Adjoints and Differentiation

Martin Elsman, Fritz Henglein, Robin Kaarsgaard, Mikkel Kragh Mathiesen, Robert Schenck

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

6 Downloads (Pure)

Abstract

We develop a compositional approach for automatic and symbolic differentiation based on categorical constructions in functional analysis where derivatives are linear functions on abstract vectors rather than being limited to scalars, vectors, matrices or tensors represented as multi-dimensional arrays.
We show that both symbolic and automatic differentiation can be performed using a differential calculus for generating linear functions representing Frechet derivatives based on rules for primitive, constant, linear and bilinear functions as well as their sequential and parallel composition. Linear functions are represented in a combinatory domain-specific language.
Finally, we provide a calculus for symbolically computing the adjoint of a derivative without using matrices, which are too inefficient to use on high-dimensional spaces. The resulting symbolic representation of a derivative retains the data-parallel operations from the input program. The combination of combinatory differentiation and computing formal adjoints turns out to be behaviorally equivalent to reverse-mode automatic differentiation. In particular, it provides opportunities for optimizations where matrices are too inefficient to represent linear functions.
Original languageEnglish
Title of host publicationProceedings of the Ninth Workshop on Mathematically Structured Functional Programming (MSFP)
Publication date30. Jun 2022
Pages1-26
DOIs
Publication statusPublished - 30. Jun 2022
Externally publishedYes
EventNinth Workshop on Mathematically Structured Functional Programming -
Duration: 2. Apr 20222. Apr 2022

Workshop

WorkshopNinth Workshop on Mathematically Structured Functional Programming
Period02/04/202202/04/2022
SeriesElectronic Proceedings in Theoretical Computer Science
Volume360
ISSN2075-2180

Fingerprint

Dive into the research topics of 'Combinatory Adjoints and Differentiation'. Together they form a unique fingerprint.

Cite this