Forward-mode autodiff: 👎
French-mode autodiff: 👍
French-mode autodiff: 👍
December 4, 2024 at 6:45 PM
Forward-mode autodiff: 👎
French-mode autodiff: 👍
French-mode autodiff: 👍
How to make #autodiff user-friendly? What lies beyond the safety of Python-world? Why does it matter for scientific machine learning?
All this, and more, in our latest preprint with @adrhill.bsky.social! Spoiler alert: it describes the most useful software I ever wrote.
arxiv.org/abs/2505.05542
All this, and more, in our latest preprint with @adrhill.bsky.social! Spoiler alert: it describes the most useful software I ever wrote.
arxiv.org/abs/2505.05542
May 12, 2025 at 6:59 PM
How to make #autodiff user-friendly? What lies beyond the safety of Python-world? Why does it matter for scientific machine learning?
All this, and more, in our latest preprint with @adrhill.bsky.social! Spoiler alert: it describes the most useful software I ever wrote.
arxiv.org/abs/2505.05542
All this, and more, in our latest preprint with @adrhill.bsky.social! Spoiler alert: it describes the most useful software I ever wrote.
arxiv.org/abs/2505.05542
New blog post!
Autodiff through function types: Categorical semantics the ultimate backpropagator
https://julesh.com/posts/2026-02-20-categorical-semantics-ultimate-backpropagator.html
Autodiff through function types: Categorical semantics the ultimate backpropagator
https://julesh.com/posts/2026-02-20-categorical-semantics-ultimate-backpropagator.html
February 20, 2026 at 3:57 PM
New blog post!
Autodiff through function types: Categorical semantics the ultimate backpropagator
https://julesh.com/posts/2026-02-20-categorical-semantics-ultimate-backpropagator.html
Autodiff through function types: Categorical semantics the ultimate backpropagator
https://julesh.com/posts/2026-02-20-categorical-semantics-ultimate-backpropagator.html
“Ban the backwards pass”
“Autodiff is off-sides”
“No quarter for non-linearity”
“Autodiff is off-sides”
“No quarter for non-linearity”
December 26, 2025 at 10:00 PM
“Ban the backwards pass”
“Autodiff is off-sides”
“No quarter for non-linearity”
“Autodiff is off-sides”
“No quarter for non-linearity”
soon rust will maybe be able to automatically generate derivatives???
(experimental autodiff feature)
(experimental autodiff feature)
May 11, 2025 at 2:04 PM
soon rust will maybe be able to automatically generate derivatives???
(experimental autodiff feature)
(experimental autodiff feature)
Wanna learn about autodiff and sparsity? Check out our #ICLR2025 blog post with @adrhill.bsky.social and Alexis Montoison. It has everything you need: matrices with lots of zeros, weird compiler tricks, graph coloring techniques, and a bunch of pretty pics!
iclr-blogposts.github.io/2025/blog/sp...
iclr-blogposts.github.io/2025/blog/sp...
April 28, 2025 at 5:07 PM
Wanna learn about autodiff and sparsity? Check out our #ICLR2025 blog post with @adrhill.bsky.social and Alexis Montoison. It has everything you need: matrices with lots of zeros, weird compiler tricks, graph coloring techniques, and a bunch of pretty pics!
iclr-blogposts.github.io/2025/blog/sp...
iclr-blogposts.github.io/2025/blog/sp...
I'd love feedback on this #marginaleffects WIP.
In GLMs with many parameters, the marginaleffects 📦 can get a 5–20x speedup (and better SEs) by calling JAX for automatic differentiation.
Try the instructions at this link and let me know if you run into trouble.
Thanks!
github.com/vincentarelb...
In GLMs with many parameters, the marginaleffects 📦 can get a 5–20x speedup (and better SEs) by calling JAX for automatic differentiation.
Try the instructions at this link and let me know if you run into trouble.
Thanks!
github.com/vincentarelb...
August 31, 2025 at 7:12 PM
I'd love feedback on this #marginaleffects WIP.
In GLMs with many parameters, the marginaleffects 📦 can get a 5–20x speedup (and better SEs) by calling JAX for automatic differentiation.
Try the instructions at this link and let me know if you run into trouble.
Thanks!
github.com/vincentarelb...
In GLMs with many parameters, the marginaleffects 📦 can get a 5–20x speedup (and better SEs) by calling JAX for automatic differentiation.
Try the instructions at this link and let me know if you run into trouble.
Thanks!
github.com/vincentarelb...
Automatic differentiation in Julia, demystified! Whether you're building packages or using them, learn how to navigate Julia’s AD ecosystem and choose the right tools for gradients. youtu.be/ww3ntpyxNtI?... #JuliaLang #AutoDiff #MachineLearning #ScientificComputing
Gradients for everyone: a quick guide to autodiff in Julia | Dalle, Hill | JuliaCon 2024
YouTube video by The Julia Programming Language
youtu.be
March 24, 2025 at 7:40 PM
Automatic differentiation in Julia, demystified! Whether you're building packages or using them, learn how to navigate Julia’s AD ecosystem and choose the right tools for gradients. youtu.be/ww3ntpyxNtI?... #JuliaLang #AutoDiff #MachineLearning #ScientificComputing
Bob Carpenter's comments on autodiff for Stan provide nice insight why Stan has it's own autodiff implementation. Recently Bob has commented that if he would start now, he would probably use one of the autodiff libraries that exist now
November 19, 2024 at 9:05 AM
Bob Carpenter's comments on autodiff for Stan provide nice insight why Stan has it's own autodiff implementation. Recently Bob has commented that if he would start now, he would probably use one of the autodiff libraries that exist now
Look mommy, I’m famous now! Exhibit A: I’m on a podcast 📻
If you’re eager to hear me ramble about open source software, autodiff, French pronunciation, and of course #JuliaLang, tune in below
open.spotify.com/episode/20ml...
If you’re eager to hear me ramble about open source software, autodiff, French pronunciation, and of course #JuliaLang, tune in below
open.spotify.com/episode/20ml...
Automatic differentiation with Guillaume Dalle
Julia Dispatch · Episode
open.spotify.com
December 16, 2024 at 6:00 PM
Look mommy, I’m famous now! Exhibit A: I’m on a podcast 📻
If you’re eager to hear me ramble about open source software, autodiff, French pronunciation, and of course #JuliaLang, tune in below
open.spotify.com/episode/20ml...
If you’re eager to hear me ramble about open source software, autodiff, French pronunciation, and of course #JuliaLang, tune in below
open.spotify.com/episode/20ml...
ported @mattkeeter.com 's autodiff from haskell to rust: github.com/adamchalmers... based on his blog post www.mattkeeter.com/projects/con...
github.com
September 2, 2025 at 2:52 PM
ported @mattkeeter.com 's autodiff from haskell to rust: github.com/adamchalmers... based on his blog post www.mattkeeter.com/projects/con...
Just learned about the {anvil} #rstats 📦, analogous to Jax. Write R code that gets JIT compiled and has easy autodiff. Seems promising for ML and Bayesian methods.
r-xla.github.io/anvil/
r-xla.github.io/anvil/
Framework for R code transformations
Code transformation framework for R.
r-xla.github.io
April 11, 2026 at 1:26 PM
Just learned about the {anvil} #rstats 📦, analogous to Jax. Write R code that gets JIT compiled and has easy autodiff. Seems promising for ML and Bayesian methods.
r-xla.github.io/anvil/
r-xla.github.io/anvil/
Compute.toys finally got the Slang compiler integrated, which means its finally time to ditch WGSL 🎉🎉
I've ported my FFT to it, coding in it is much nicer. compute.toys/view/1922
Should try something with its autodiff next.
I've ported my FFT to it, coding in it is much nicer. compute.toys/view/1922
Should try something with its autodiff next.
May 5, 2025 at 2:24 PM
Compute.toys finally got the Slang compiler integrated, which means its finally time to ditch WGSL 🎉🎉
I've ported my FFT to it, coding in it is much nicer. compute.toys/view/1922
Should try something with its autodiff next.
I've ported my FFT to it, coding in it is much nicer. compute.toys/view/1922
Should try something with its autodiff next.
I done another blog post
Featuring autodiff in Aptwe
https://cybercat.institute/2025/01/03/bidirectional-programming-iv/
Featuring autodiff in Aptwe
https://cybercat.institute/2025/01/03/bidirectional-programming-iv/
January 3, 2025 at 6:33 PM
I done another blog post
Featuring autodiff in Aptwe
https://cybercat.institute/2025/01/03/bidirectional-programming-iv/
Featuring autodiff in Aptwe
https://cybercat.institute/2025/01/03/bidirectional-programming-iv/
Would be fun to extend this to a very unreliable autodiff engine
The is diabolical... a Python object that hallucinates method implementations on demand any time you call them, using my LLM Python library github.com/awwaiid/grem...
July 7, 2025 at 1:32 AM
Would be fun to extend this to a very unreliable autodiff engine
Today is the last day to submit a proposal for our #autodiff minisymposium at JuliaCon25! Hurry up!
Let's face it, conferences always extend deadlines... all the more reason for you to send something to JuliaCon25!
Do you use autodiff in #JuliaLang? Then you're more than welcome in our minisymposium 🤗 If not, submit your work to other sessions!
Call for proposals here: pretalx.com/juliacon-202...
Do you use autodiff in #JuliaLang? Then you're more than welcome in our minisymposium 🤗 If not, submit your work to other sessions!
Call for proposals here: pretalx.com/juliacon-202...
February 14, 2025 at 11:03 AM
Today is the last day to submit a proposal for our #autodiff minisymposium at JuliaCon25! Hurry up!
The recording from this session is now available: youtu.be/gUE-RIOwQKI?...
#Slang #autodiff #shader #graphics #neural #rendering #gpu
#Slang #autodiff #shader #graphics #neural #rendering #gpu
October 23, 2025 at 11:24 PM
The recording from this session is now available: youtu.be/gUE-RIOwQKI?...
#Slang #autodiff #shader #graphics #neural #rendering #gpu
#Slang #autodiff #shader #graphics #neural #rendering #gpu
honestly doomscrolling the standard library is pretty fun, more people should do it. like did you know there’s an unstable autodiff module?
September 4, 2025 at 12:26 PM
honestly doomscrolling the standard library is pretty fun, more people should do it. like did you know there’s an unstable autodiff module?
This year, when students of my optimization class were asking for references related to forward-backward mode autodiff, I didn't suggest books or articles: #JAX documentation was actually the best thing I've found! What's your go-to reference for this?
November 26, 2024 at 3:15 AM
This year, when students of my optimization class were asking for references related to forward-backward mode autodiff, I didn't suggest books or articles: #JAX documentation was actually the best thing I've found! What's your go-to reference for this?
Recent optimizations in SciMLSensitivity.jl are having some pretty good payoffs! Given we just 2.5x'd 2 months ago, this next change of 3.2x is putting us almost an order of magnitude ahead! See the latest autodiff benchmarks docs.sciml.ai/SciMLBenchma...
#sciml #julialang
#sciml #julialang
February 26, 2026 at 11:38 AM
Recent optimizations in SciMLSensitivity.jl are having some pretty good payoffs! Given we just 2.5x'd 2 months ago, this next change of 3.2x is putting us almost an order of magnitude ahead! See the latest autodiff benchmarks docs.sciml.ai/SciMLBenchma...
#sciml #julialang
#sciml #julialang
January 22, 2025 at 10:32 AM
For the theoretical side, there’s no better resource than the 2024 book by Blondel and Roulet:
arxiv.org/abs/2403.14606
For the practical side, my friend @willtebbutt.bsky.social wrote stellar documentation explaining autodiff with mutation:
compintell.github.io/Mooncake.jl/...
arxiv.org/abs/2403.14606
For the practical side, my friend @willtebbutt.bsky.social wrote stellar documentation explaining autodiff with mutation:
compintell.github.io/Mooncake.jl/...
The Elements of Differentiable Programming
Artificial intelligence has recently experienced remarkable advances, fueled by large models, vast datasets, accelerated hardware, and, last but not least, the transformative power of differentiable p...
arxiv.org
November 26, 2024 at 6:05 AM
For the theoretical side, there’s no better resource than the 2024 book by Blondel and Roulet:
arxiv.org/abs/2403.14606
For the practical side, my friend @willtebbutt.bsky.social wrote stellar documentation explaining autodiff with mutation:
compintell.github.io/Mooncake.jl/...
arxiv.org/abs/2403.14606
For the practical side, my friend @willtebbutt.bsky.social wrote stellar documentation explaining autodiff with mutation:
compintell.github.io/Mooncake.jl/...
His "Gradients for everyone" video is a great overview of autodiff in Julia vs. Jax etc.
December 16, 2024 at 6:49 AM
His "Gradients for everyone" video is a great overview of autodiff in Julia vs. Jax etc.