Getting to the Point. Index Sets and Parallelism-Preserving Autodiff for Pointful Array Programming
| Autor: | Alexey Radul, Daniel D. Johnson, Dimitrios Vytiniotis, Adam Paszke, Matthew J. Johnson, Jonathan Ragan-Kelley, Dougal Maclaurin, David Duvenaud |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Functional programming Computer Science - Programming Languages Computer science Programming language Automatic differentiation Search engine indexing computer.software_genre Parallelism (grammar) Benchmark (computing) Currying Array programming Safety Risk Reliability and Quality computer Software Associative property Programming Languages (cs.PL) |
| DOI: | 10.48550/arxiv.2104.05372 |
| Popis: | We present a novel programming language design that attempts to combine the clarity and safety of high-level functional languages with the efficiency and parallelism of low-level numerical languages. We treat arrays as eagerly-memoized functions on typed index sets, allowing abstract function manipulations, such as currying, to work on arrays. In contrast to composing primitive bulk-array operations, we argue for an explicit nested indexing style that mirrors application of functions to arguments. We also introduce a fine-grained typed effects system which affords concise and automatically-parallelized in-place updates. Specifically, an associative accumulation effect allows reverse-mode automatic differentiation of in-place updates in a way that preserves parallelism. Empirically, we benchmark against the Futhark array programming language, and demonstrate that aggressive inlining and type-driven compilation allows array programs to be written in an expressive, "pointful" style with little performance penalty. Comment: 31 pages with appendix, 11 figures. A conference submission is still under review |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|