however you can model the pooled BPTT as smooth, and do adjoint backprop (see: neural ODEs) to trade some compute (cheap) for linear memory + reduced mem bandwidth
January 28, 2026 at 7:33 PM
however you can model the pooled BPTT as smooth, and do adjoint backprop (see: neural ODEs) to trade some compute (cheap) for linear memory + reduced mem bandwidth
3/7 We used "data alignment" assumptions ( @saxelab.bsky.social et al., @aproca.bsky.social et al.) to enable analytical calculation of fixed points, stability, and convergence rates for random feedback online learning (RFLO), back-propagation through time (BPTT), and 1-step truncated BPTT (tBPTT).
June 10, 2026 at 12:30 PM
3/7 We used "data alignment" assumptions ( @saxelab.bsky.social et al., @aproca.bsky.social et al.) to enable analytical calculation of fixed points, stability, and convergence rates for random feedback online learning (RFLO), back-propagation through time (BPTT), and 1-step truncated BPTT (tBPTT).
Vanilla RNNs trained with pure BPTT fail on simple memory tasks. Adding Oja's rule to BPTT drastically improves performance.
May 19, 2025 at 3:33 PM
Vanilla RNNs trained with pure BPTT fail on simple memory tasks. Adding Oja's rule to BPTT drastically improves performance.
other than the intrinsic aspects (dynamical complex system, local updates only) of NCAs, they’re also interesting in that they are extremely fast and parameter efficient at inference time. the main problem is that they require N^2 memory in resolution in training with naive BPTT
January 28, 2026 at 7:32 PM
other than the intrinsic aspects (dynamical complex system, local updates only) of NCAs, they’re also interesting in that they are extremely fast and parameter efficient at inference time. the main problem is that they require N^2 memory in resolution in training with naive BPTT
He tried to train a policy network (for simple environments) using BPTT through a world model and he started getting quite good results.
"Wait, Backpropagating Through a World Model Is All You Need?"
f14-bertolotti.github.io///posts/19-1...
"Wait, Backpropagating Through a World Model Is All You Need?"
f14-bertolotti.github.io///posts/19-1...
December 23, 2025 at 1:49 PM
He tried to train a policy network (for simple environments) using BPTT through a world model and he started getting quite good results.
"Wait, Backpropagating Through a World Model Is All You Need?"
f14-bertolotti.github.io///posts/19-1...
"Wait, Backpropagating Through a World Model Is All You Need?"
f14-bertolotti.github.io///posts/19-1...
📢 Presenting our work at #NeurIPS2025 🇺🇸 today (6-9 pm) @wimlworkshop.bsky.social!
🧠 The question: How can the brain learn temporal dependencies without an artificial backward pass? BPTT is notoriously harder than BP and is still uncharted territory (Lillicrap & @adamsantoro.bsky.social , 2019).
🧠 The question: How can the brain learn temporal dependencies without an artificial backward pass? BPTT is notoriously harder than BP and is still uncharted territory (Lillicrap & @adamsantoro.bsky.social , 2019).
December 2, 2025 at 11:37 PM
📢 Presenting our work at #NeurIPS2025 🇺🇸 today (6-9 pm) @wimlworkshop.bsky.social!
🧠 The question: How can the brain learn temporal dependencies without an artificial backward pass? BPTT is notoriously harder than BP and is still uncharted territory (Lillicrap & @adamsantoro.bsky.social , 2019).
🧠 The question: How can the brain learn temporal dependencies without an artificial backward pass? BPTT is notoriously harder than BP and is still uncharted territory (Lillicrap & @adamsantoro.bsky.social , 2019).
DAY 1: Advent of Comp Neuro 🎄🤖🧠🧪
Feedback in RNNs can rotate the neural subspace to separate different source signals!
"Invariant neural subspaces maintained by feedback modulation"
elifesciences.org/articles/76096
Feedback in RNNs can rotate the neural subspace to separate different source signals!
"Invariant neural subspaces maintained by feedback modulation"
elifesciences.org/articles/76096
December 3, 2024 at 3:41 PM
DAY 1: Advent of Comp Neuro 🎄🤖🧠🧪
Feedback in RNNs can rotate the neural subspace to separate different source signals!
"Invariant neural subspaces maintained by feedback modulation"
elifesciences.org/articles/76096
Feedback in RNNs can rotate the neural subspace to separate different source signals!
"Invariant neural subspaces maintained by feedback modulation"
elifesciences.org/articles/76096
Tell this BPTT (which explodes, but unbiased!)
November 29, 2024 at 8:12 PM
Tell this BPTT (which explodes, but unbiased!)
Got a favourite BPTT*?
*Beatles Podcast Theme Tune?
*Beatles Podcast Theme Tune?
March 1, 2026 at 10:05 AM
Got a favourite BPTT*?
*Beatles Podcast Theme Tune?
*Beatles Podcast Theme Tune?
4/7 Under our assumptions, we found that BPTT and tBPTT learn similarly. Interestingly, while RFLO will still converge to an optimal fixed point, there are fewer optima available to it and it takes longer to converge, often requiring long trajectories in parameter space.
June 10, 2026 at 12:31 PM
4/7 Under our assumptions, we found that BPTT and tBPTT learn similarly. Interestingly, while RFLO will still converge to an optimal fixed point, there are fewer optima available to it and it takes longer to converge, often requiring long trajectories in parameter space.
Depends a lot on what you mean. If you count ANNs converted to SNNs there have been some gigantic ones. If you count reservoir systems where only the readout is trained. Or systems trained with local learning rules. Trained with BPTT and surrogate gradient descent? Probably quite small still.
July 15, 2026 at 5:35 PM
Depends a lot on what you mean. If you count ANNs converted to SNNs there have been some gigantic ones. If you count reservoir systems where only the readout is trained. Or systems trained with local learning rules. Trained with BPTT and surrogate gradient descent? Probably quite small still.
O Imperiali$mo extrai a riqueza das regiões colonizadas e deixa um rastro de pobreza pras populações locais.
Trinidad e Tobago, duas ilhas pequenas no Caribe, mas com uma enorme reserva de Petróleo no mar, poderia ter um IDH Nórdico.
O Brasil que ABRA o OLHO !
noticiabrasil.net.br/20250529/gig...
Trinidad e Tobago, duas ilhas pequenas no Caribe, mas com uma enorme reserva de Petróleo no mar, poderia ter um IDH Nórdico.
O Brasil que ABRA o OLHO !
noticiabrasil.net.br/20250529/gig...
Gigante britânica de energia realiza 1ª extração de gás da plataforma de Mento, na costa de Trinidad
A gigante britânica de energia BP anunciou nesta quinta-feira (29) que sua subsidiária BP Trinidad e Tobago (bpTT) extraiu o primeiro lote de gás da plataforma offshore de Mento, perto da costa da ilh...
noticiabrasil.net.br
May 29, 2025 at 10:41 PM
O Imperiali$mo extrai a riqueza das regiões colonizadas e deixa um rastro de pobreza pras populações locais.
Trinidad e Tobago, duas ilhas pequenas no Caribe, mas com uma enorme reserva de Petróleo no mar, poderia ter um IDH Nórdico.
O Brasil que ABRA o OLHO !
noticiabrasil.net.br/20250529/gig...
Trinidad e Tobago, duas ilhas pequenas no Caribe, mas com uma enorme reserva de Petróleo no mar, poderia ter um IDH Nórdico.
O Brasil que ABRA o OLHO !
noticiabrasil.net.br/20250529/gig...
Don't know the numbers off the top of my head but my guess would be the first few in the millions maybe even larger, approx to BPTT like RTRL maybe hundreds of thousands? Full BPTT probably high tens of thousands?
July 15, 2026 at 5:47 PM
Don't know the numbers off the top of my head but my guess would be the first few in the millions maybe even larger, approx to BPTT like RTRL maybe hundreds of thousands? Full BPTT probably high tens of thousands?
Maybe remind them of best practices when working in rural areas and interacting with the public? @bptt.bsky.social has a good thread of tips here (bsky.app/profile/bptt...) which I shared with my team during our weekly meeting. I think heightened awareness is warranted here.
Two Forest Service employees were kidnapped and held hostage while doing fieldwork on the Shasta-T in Northern California. Fortunately after hours of negotiations they were released unharmed. When I worked on the Malheur NF they had us take a red flag training for this reason. R5 doesn’t.
US Forest Service employees zip-tied, held at gunpoint in California
"The hostages are OK," the Siskiyou County Sheriff's Office confirmed.
www.sfgate.com
July 21, 2026 at 5:33 PM
Maybe remind them of best practices when working in rural areas and interacting with the public? @bptt.bsky.social has a good thread of tips here (bsky.app/profile/bptt...) which I shared with my team during our weekly meeting. I think heightened awareness is warranted here.
We train via noisy BPTT. It’s true that other learning rules could produce different internal reps, and likewise w/ network architecture. More knobs!
The benefit of the methodology is that we can isolate the knobs and theoretically understand their effects, rather than relying on random variation.
The benefit of the methodology is that we can isolate the knobs and theoretically understand their effects, rather than relying on random variation.
March 4, 2026 at 9:44 PM
We train via noisy BPTT. It’s true that other learning rules could produce different internal reps, and likewise w/ network architecture. More knobs!
The benefit of the methodology is that we can isolate the knobs and theoretically understand their effects, rather than relying on random variation.
The benefit of the methodology is that we can isolate the knobs and theoretically understand their effects, rather than relying on random variation.
This week's bonus upload is my performance from the @popoffatm.bsky.social Beach Episode Party.
Dabbled in a few fandoms for this one, so enjoy.
SoundCloud: on.soundcloud.com/UJ08fI9Gts1S...
YouTube: www.youtube.com/watch?v=BPtT...
Dabbled in a few fandoms for this one, so enjoy.
SoundCloud: on.soundcloud.com/UJ08fI9Gts1S...
YouTube: www.youtube.com/watch?v=BPtT...
Pop Off! The Beach Episode Performance
Here's my performance for Pop Off! The Beach Episode. Enjoy.
Tracklist:
Trent - OH! NO!
Spott - Orange Smarties
Johnny Hellhound - Vibe Core
Eurobeat Brony - Discord (AWal Remix)
HyeNRG - A Little Li
soundcloud.com
August 29, 2026 at 10:49 PM
This week's bonus upload is my performance from the @popoffatm.bsky.social Beach Episode Party.
Dabbled in a few fandoms for this one, so enjoy.
SoundCloud: on.soundcloud.com/UJ08fI9Gts1S...
YouTube: www.youtube.com/watch?v=BPtT...
Dabbled in a few fandoms for this one, so enjoy.
SoundCloud: on.soundcloud.com/UJ08fI9Gts1S...
YouTube: www.youtube.com/watch?v=BPtT...
4/7 We first showed this by providing a geometric perspective. In scenarios where the dimensionality of the network is higher than that of the output, TF overconstrains the network dynamics during learning, which degrades its benefits. In contrast, EF minimally intervenes in the network dynamics.
January 8, 2026 at 10:10 PM
4/7 We first showed this by providing a geometric perspective. In scenarios where the dimensionality of the network is higher than that of the output, TF overconstrains the network dynamics during learning, which degrades its benefits. In contrast, EF minimally intervenes in the network dynamics.
"BP Plc’s (NYSE:BP) subsidiary BP Trinidad and Tobago (“BPTT”) signed a production-sharing agreement with Trinidad last week for the giant Cypre subsea gas development, and is now targeting first gas in 2025, Zacks reports."
(oilprice.com)
#collapse
(oilprice.com)
#collapse
November 28, 2024 at 8:58 PM
"BP Plc’s (NYSE:BP) subsidiary BP Trinidad and Tobago (“BPTT”) signed a production-sharing agreement with Trinidad last week for the giant Cypre subsea gas development, and is now targeting first gas in 2025, Zacks reports."
(oilprice.com)
#collapse
(oilprice.com)
#collapse
6/7 We tried three different tasks with varying difficulties and showed that using EF together with BPTT improves learning, relative to using TF with BPTT or using BPTT alone. For biological plausibility, we also tried error forcing with RFLO, and found that it can improve RFLO as well.
January 8, 2026 at 10:10 PM
6/7 We tried three different tasks with varying difficulties and showed that using EF together with BPTT improves learning, relative to using TF with BPTT or using BPTT alone. For biological plausibility, we also tried error forcing with RFLO, and found that it can improve RFLO as well.
The Hierarchical Reasoning Model (HRM) isn't just another model. It's a deep synthesis. It marries the iterative soul of an RNN (minus the BPTT nightmare) with the raw power of modern Attention.
I wrote a deep dive on why this is a full-circle moment for me, going back to the RNN finetuning days.
I wrote a deep dive on why this is a full-circle moment for me, going back to the RNN finetuning days.
August 7, 2025 at 8:50 AM
The Hierarchical Reasoning Model (HRM) isn't just another model. It's a deep synthesis. It marries the iterative soul of an RNN (minus the BPTT nightmare) with the raw power of modern Attention.
I wrote a deep dive on why this is a full-circle moment for me, going back to the RNN finetuning days.
I wrote a deep dive on why this is a full-circle moment for me, going back to the RNN finetuning days.
Backprop is the workhorse of AI, but unrealistic in biology, and way too inefficient in recurrent networks (BPTT).
One can also propagate sensitivity forward in time. This can be done efficiently and exactly for a large class of recurrent networks.
💥 Problem solved! 😁
arxiv.org/abs/2411.16695
One can also propagate sensitivity forward in time. This can be done efficiently and exactly for a large class of recurrent networks.
💥 Problem solved! 😁
arxiv.org/abs/2411.16695
November 27, 2024 at 1:33 PM
Backprop is the workhorse of AI, but unrealistic in biology, and way too inefficient in recurrent networks (BPTT).
One can also propagate sensitivity forward in time. This can be done efficiently and exactly for a large class of recurrent networks.
💥 Problem solved! 😁
arxiv.org/abs/2411.16695
One can also propagate sensitivity forward in time. This can be done efficiently and exactly for a large class of recurrent networks.
💥 Problem solved! 😁
arxiv.org/abs/2411.16695
Kris Mitchener & Gonçalo Pina present newly hand-collected daily bond price data from a postwar French experiment to estimate the state-contingent debt premium. They find that state-contingent debt may not be as expensive as previously thought.
ow.ly/CGKY50Z32AR
#EconSky
ow.ly/CGKY50Z32AR
#EconSky
May 22, 2026 at 8:17 AM
Kris Mitchener & Gonçalo Pina present newly hand-collected daily bond price data from a postwar French experiment to estimate the state-contingent debt premium. They find that state-contingent debt may not be as expensive as previously thought.
ow.ly/CGKY50Z32AR
#EconSky
ow.ly/CGKY50Z32AR
#EconSky
New Recurrent Neural Network (RNN) Explorer I built for my students.
Tabs: TDNN's (think FIR), Output feedback (IIR), RNN's with autoregressive feedback of latent states (with randomized reservoir support), BPTT training explorer, then LSTM/GRU explainer.
tpavlic.github.io/asu-bioinspi...
Tabs: TDNN's (think FIR), Output feedback (IIR), RNN's with autoregressive feedback of latent states (with randomized reservoir support), BPTT training explorer, then LSTM/GRU explainer.
tpavlic.github.io/asu-bioinspi...
Recurrent Networks & Temporal Supervision Explorer — interactive explainer
Explore TDNNs, RNNs with output feedback, autoregressive latent-state RNNs, and BPTT training — plus a visual guide to gated architectures (LSTM, GRU) — through interactive diagrams and live experiments.
tpavlic.github.io
April 10, 2026 at 8:12 PM
New Recurrent Neural Network (RNN) Explorer I built for my students.
Tabs: TDNN's (think FIR), Output feedback (IIR), RNN's with autoregressive feedback of latent states (with randomized reservoir support), BPTT training explorer, then LSTM/GRU explainer.
tpavlic.github.io/asu-bioinspi...
Tabs: TDNN's (think FIR), Output feedback (IIR), RNN's with autoregressive feedback of latent states (with randomized reservoir support), BPTT training explorer, then LSTM/GRU explainer.
tpavlic.github.io/asu-bioinspi...
youtu.be/54pUIp-eTcs?...
Sooo Colombia blew tf up and now we gonna drill wow u shud become a hijab distributor at sm jaleel.
Sooo Colombia blew tf up and now we gonna drill wow u shud become a hijab distributor at sm jaleel.
Prime Minister Hails New NGC-bpTT Energy Agreement
YouTube video by TTT Live Online
youtu.be
August 11, 2026 at 10:38 AM
youtu.be/54pUIp-eTcs?...
Sooo Colombia blew tf up and now we gonna drill wow u shud become a hijab distributor at sm jaleel.
Sooo Colombia blew tf up and now we gonna drill wow u shud become a hijab distributor at sm jaleel.