The Elements of
Differentiable Programming
Mathieu Blondel and Vincent Roulet
Google DeepMind
Contents
- Abstract
- How to cite
- Acknowledgements
- Source code
- Notation
- 1 Introduction
- Part I: Fundamentals
- 2 Differentiation
- 3 Probabilistic learning
- Part II: Differentiable programs
- 4 Parameterized programs
- 5 Control flows
- 6 Data structures
- Part III: Differentiating through programs
- 7 Finite differences
- 8 Automatic differentiation
- 9 Second-order automatic differentiation
- 10 Inference in graphical models as differentiation
- 11 Differentiating through optimization
- 12 Differentiating through integration
- Part IV: Smoothing programs
- 13 Smoothing by optimization
- 14 Smoothing by integration
- Part V: Optimizing differentiable programs
- 15 Optimization basics
- 16 First-order optimization
- 17 Second-order optimization
- 18 Duality
- References