What automatic differentiation systems exist
Where do AD systems live? Firstly, they definitely live inside deep learning frameworks such as PyTorch and TensorFlow, and other deep learning frameworks. Without an AD system, these two deep learning frameworks would not work.
Secondly, they also live in independent packages. In Julia, there are two AD packages: one called Zygote.jl
, and the other called AutoGrad.jl
; both of them are actively developed. autograd
, which was the reference Python package that AutoGrad.jl
was written against, is also the precursor to JAX, which I think of as automatic differentiation on steroids.
Notes on differential computing
This is an overview page of my notes on differential computing and JAX.
Contents:
Stuff I've built/done with JAX: