David Hall
@dlwh.bsky.social
Research Engineering Lead at @StanfordCRFM. I do NLP and foundation model things with JAX. Previously Semantic Machines, Microsoft, Berkeley, Breeze
Reposted by David Hall
Marin's 535B-parameter hero run was originally planned as a 360B model.
A new blog from Rafal Wojdyla, MTS, explains how a change in sharding strategy, Expert Parallelism, made room for a ~50% larger model at the same ~23B active parameters, and what it costs.
🔗 openathena.ai/blog/expert-...
A new blog from Rafal Wojdyla, MTS, explains how a change in sharding strategy, Expert Parallelism, made room for a ~50% larger model at the same ~23B active parameters, and what it costs.
🔗 openathena.ai/blog/expert-...
October 1, 2026 at 7:30 PM
Marin's 535B-parameter hero run was originally planned as a 360B model.
A new blog from Rafal Wojdyla, MTS, explains how a change in sharding strategy, Expert Parallelism, made room for a ~50% larger model at the same ~23B active parameters, and what it costs.
🔗 openathena.ai/blog/expert-...
A new blog from Rafal Wojdyla, MTS, explains how a change in sharding strategy, Expert Parallelism, made room for a ~50% larger model at the same ~23B active parameters, and what it costs.
🔗 openathena.ai/blog/expert-...
Reposted by David Hall
A year ago, Marin had one FTE. Recently, the ten-person team launched a 535 billion parameter MoE model. In the Marin 535B-A23B launch note, David Hall @dlwh.bsky.social notes it's going "almost boringly well." Thank you to our colleagues, partners, and advisors for making this possible.
bit.ly/535b
bit.ly/535b
Marin 535B-A23B launch note
A look back at Marin's first year at Open Athena and forward at the 535B-A23B hero run, the largest model the team has ever trained.
openathena.ai
September 3, 2026 at 4:31 PM
A year ago, Marin had one FTE. Recently, the ten-person team launched a 535 billion parameter MoE model. In the Marin 535B-A23B launch note, David Hall @dlwh.bsky.social notes it's going "almost boringly well." Thank you to our colleagues, partners, and advisors for making this possible.
bit.ly/535b
bit.ly/535b
Reposted by David Hall
In a new blog, Russell Power explains how the Marin team nearly doubled its sustained TPU usage by creating a custom global scheduler: Iris. Iris searches every region where Marin has compute, places each job wherever capacity appears, and moves data along as needed.
🔗 openathena.ai/blog/cluster...
🔗 openathena.ai/blog/cluster...
June 25, 2026 at 8:30 PM
In a new blog, Russell Power explains how the Marin team nearly doubled its sustained TPU usage by creating a custom global scheduler: Iris. Iris searches every region where Marin has compute, places each job wherever capacity appears, and moves data along as needed.
🔗 openathena.ai/blog/cluster...
🔗 openathena.ai/blog/cluster...
Reposted by David Hall
To train better open models, we need predictable scaling.
Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error.
Getting there took some work 🧵
Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error.
Getting there took some work 🧵
May 11, 2026 at 7:48 PM
To train better open models, we need predictable scaling.
Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error.
Getting there took some work 🧵
Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error.
Getting there took some work 🧵
Reposted by David Hall
Learn more about the project in Percy's blog post: marin.community/blog/2025/05...
And about the Models we are releasing in @dlwh.bsky.social's training retro: marin.readthedocs.io/en/latest/re...
And about the Models we are releasing in @dlwh.bsky.social's training retro: marin.readthedocs.io/en/latest/re...
Introducing Marin: An Open Lab for Building Foundation Models
Open-source software is a success story: It powers the world’s digital infrastructure. It allows anyone in the world to contribute based on merit. It leads to greater innovation, collaboration, and se...
marin.community
May 19, 2025 at 7:11 PM
Learn more about the project in Percy's blog post: marin.community/blog/2025/05...
And about the Models we are releasing in @dlwh.bsky.social's training retro: marin.readthedocs.io/en/latest/re...
And about the Models we are releasing in @dlwh.bsky.social's training retro: marin.readthedocs.io/en/latest/re...
Super excited Marin is finally out! Come see what we've been building! Code/platform for training fully reproducible models end-to-end, from data to evals. Plus a new high quality 8B base model. Percy did a good job explaining it on the other place. marin.community
x.com/percyliang/s...
x.com/percyliang/s...
Percy Liang on X: "What would truly open-source AI look like? Not just open weights, open code/data, but *open development*, where the entire research and development process is public *and* anyone can contribute. We built Marin, an open lab, to fulfill this vision: https://t.co/racsvmhyA3" / X
What would truly open-source AI look like? Not just open weights, open code/data, but *open development*, where the entire research and development process is public *and* anyone can contribute. We built Marin, an open lab, to fulfill this vision: https://t.co/racsvmhyA3
x.com
May 19, 2025 at 7:35 PM
Super excited Marin is finally out! Come see what we've been building! Code/platform for training fully reproducible models end-to-end, from data to evals. Plus a new high quality 8B base model. Percy did a good job explaining it on the other place. marin.community
x.com/percyliang/s...
x.com/percyliang/s...