Seeing SQLMesh pop up more and more. /ht @ananthdurai.bsky.social
Transitioning from dbt to SQLMesh
Explore the shift from dbt to SQLMesh, highlighting the benefits of improved state management, templating, and efficiency in data transformation.
www.harness.io
November 11, 2024 at 6:48 PM
Seeing SQLMesh pop up more and more. /ht @ananthdurai.bsky.social
After I wired SQLMesh into our existing infra as k8s cron
November 11, 2024 at 10:43 PM
After I wired SQLMesh into our existing infra as k8s cron
Continuing with my series on Tobiko's SQLMesh and building on the work I've done this week so far using dltHub and @duckdb.org. I also get to enjoy Harlequin.sh for the first time as a great way to explore DuckDB data in the terminal.
open.substack.com/pub/davidsj/...
open.substack.com/pub/davidsj/...
sqlmesh init -t dlt --dlt-pipeline bluesky duckdb
dlt metadata -> sqlmesh models
open.substack.com
December 5, 2024 at 5:09 PM
Continuing with my series on Tobiko's SQLMesh and building on the work I've done this week so far using dltHub and @duckdb.org. I also get to enjoy Harlequin.sh for the first time as a great way to explore DuckDB data in the terminal.
open.substack.com/pub/davidsj/...
open.substack.com/pub/davidsj/...
Fivetran acquired SQLMesh.
This is not a drill.
This is not a drill.
September 3, 2025 at 7:56 PM
Fivetran acquired SQLMesh.
This is not a drill.
This is not a drill.
The Linux Foundation welcomes SQLMesh, an open source data transformation framework contributed by @fivetran.bsky.social — to grow under vendor-neutral governance and support the future of data infrastructure.
Learn more about the new project 👇
Learn more about the new project 👇
Linux Foundation Welcomes SQLMesh Project
Linux Foundation Welcomes SQLMesh Project
www.linuxfoundation.org
March 25, 2026 at 1:30 PM
The Linux Foundation welcomes SQLMesh, an open source data transformation framework contributed by @fivetran.bsky.social — to grow under vendor-neutral governance and support the future of data infrastructure.
Learn more about the new project 👇
Learn more about the new project 👇
I finally ran the SQLMesh plan command! I'm continuing with my SQLMesh advent series.
open.substack.com/pub/davidsj/...
open.substack.com/pub/davidsj/...
sqlmesh plan
Hands-on look at state and environment management
open.substack.com
December 10, 2024 at 5:12 PM
I finally ran the SQLMesh plan command! I'm continuing with my SQLMesh advent series.
open.substack.com/pub/davidsj/...
open.substack.com/pub/davidsj/...
Last post of the week in my SQLMesh advent series (this is a working day only advent calendar 💼), covering all the model kinds that exist.
With the exception of the incremental model kinds which need a deeper look alongside how SQLMesh handles environments.
open.substack.com/pub/davidsj/...
With the exception of the incremental model kinds which need a deeper look alongside how SQLMesh handles environments.
open.substack.com/pub/davidsj/...
sqlmesh model kinds - 1
Looking at the types of model and contrasting their function
open.substack.com
December 6, 2024 at 8:54 PM
Last post of the week in my SQLMesh advent series (this is a working day only advent calendar 💼), covering all the model kinds that exist.
With the exception of the incremental model kinds which need a deeper look alongside how SQLMesh handles environments.
open.substack.com/pub/davidsj/...
With the exception of the incremental model kinds which need a deeper look alongside how SQLMesh handles environments.
open.substack.com/pub/davidsj/...
Continuing with my Tobiko SQLMesh advent series, and concluding looking at SQLMesh model kinds by looking at incremental models.
open.substack.com/pub/davidsj/...
open.substack.com/pub/davidsj/...
sqlmesh model kinds - 2
Incremental models and state management
open.substack.com
December 9, 2024 at 6:04 PM
Continuing with my Tobiko SQLMesh advent series, and concluding looking at SQLMesh model kinds by looking at incremental models.
open.substack.com/pub/davidsj/...
open.substack.com/pub/davidsj/...
It turns out AI is very good at using AI.
Yesterday, in my Tobiko SQLMesh advent series, I reached the point where I could generate a JSON representation of the relationships between models in an SQLMesh project.
open.substack.com/pub/davidsj/...
Yesterday, in my Tobiko SQLMesh advent series, I reached the point where I could generate a JSON representation of the relationships between models in an SQLMesh project.
open.substack.com/pub/davidsj/...
sqlmesh cube_generate build part 2
Time for some AI Engineering
open.substack.com
December 18, 2024 at 7:29 PM
It turns out AI is very good at using AI.
Yesterday, in my Tobiko SQLMesh advent series, I reached the point where I could generate a JSON representation of the relationships between models in an SQLMesh project.
open.substack.com/pub/davidsj/...
Yesterday, in my Tobiko SQLMesh advent series, I reached the point where I could generate a JSON representation of the relationships between models in an SQLMesh project.
open.substack.com/pub/davidsj/...
Check out this cool exploration by @jayatillake.bsky.social of dlt+sqlmesh!
davidsj.substack.com/p/sqlmesh-in...
davidsj.substack.com/p/sqlmesh-in...
sqlmesh init -t dlt --dlt-pipeline bluesky duckdb
dlt metadata -> sqlmesh models
davidsj.substack.com
December 6, 2024 at 11:53 AM
Check out this cool exploration by @jayatillake.bsky.social of dlt+sqlmesh!
davidsj.substack.com/p/sqlmesh-in...
davidsj.substack.com/p/sqlmesh-in...
🚀 Thrilled to speak at 5 conferences this Oct & Nov!
🇫🇷 Cloud Nord
🇲🇦 Devoxx Morocco
🇫🇷 @devfesttoulouse.fr
🇫🇷 Devfest Paris
🇫🇷 @devfestlyon.bsky.social
Thanks for the trust 🙏
⚙️ Pulumi Automation
🐳 Docker Bake
🔄 ELT: Dbt vs SQLMesh
@docker.com @pulumi.com #Dbt #GoogleCloud #SQLMesh
🇫🇷 Cloud Nord
🇲🇦 Devoxx Morocco
🇫🇷 @devfesttoulouse.fr
🇫🇷 Devfest Paris
🇫🇷 @devfestlyon.bsky.social
Thanks for the trust 🙏
⚙️ Pulumi Automation
🐳 Docker Bake
🔄 ELT: Dbt vs SQLMesh
@docker.com @pulumi.com #Dbt #GoogleCloud #SQLMesh
August 28, 2025 at 8:16 AM
🚀 Thrilled to speak at 5 conferences this Oct & Nov!
🇫🇷 Cloud Nord
🇲🇦 Devoxx Morocco
🇫🇷 @devfesttoulouse.fr
🇫🇷 Devfest Paris
🇫🇷 @devfestlyon.bsky.social
Thanks for the trust 🙏
⚙️ Pulumi Automation
🐳 Docker Bake
🔄 ELT: Dbt vs SQLMesh
@docker.com @pulumi.com #Dbt #GoogleCloud #SQLMesh
🇫🇷 Cloud Nord
🇲🇦 Devoxx Morocco
🇫🇷 @devfesttoulouse.fr
🇫🇷 Devfest Paris
🇫🇷 @devfestlyon.bsky.social
Thanks for the trust 🙏
⚙️ Pulumi Automation
🐳 Docker Bake
🔄 ELT: Dbt vs SQLMesh
@docker.com @pulumi.com #Dbt #GoogleCloud #SQLMesh
I'm starting a new series on Tobiko Data's SQLMesh. It's going to be a series of many small posts, where I post at the cadence I learn about specific things.
Today, I share about where many teams who are interested will start: with an existing dbt project.
open.substack.com/pub/davidsj/...
Today, I share about where many teams who are interested will start: with an existing dbt project.
open.substack.com/pub/davidsj/...
sqlmesh init -t dbt
Backwards Compatibility
open.substack.com
December 2, 2024 at 4:12 PM
I'm starting a new series on Tobiko Data's SQLMesh. It's going to be a series of many small posts, where I post at the cadence I learn about specific things.
Today, I share about where many teams who are interested will start: with an existing dbt project.
open.substack.com/pub/davidsj/...
Today, I share about where many teams who are interested will start: with an existing dbt project.
open.substack.com/pub/davidsj/...
It’s a circular reference when you use perplexity to research something and it points to your own blog 😂
www.perplexity.ai/search/show-...
www.perplexity.ai/search/show-...
Show me the function in sqlmesh which uses sqlglot to parse a model
SQLMesh utilizes SQLGlot to parse and understand SQL models, but there isn't a single specific function that performs this task. Instead, SQLMesh integrates...
www.perplexity.ai
December 17, 2024 at 12:50 AM
It’s a circular reference when you use perplexity to research something and it points to your own blog 😂
www.perplexity.ai/search/show-...
www.perplexity.ai/search/show-...
SQLMesh is what happens when you take dbt's concept and add semantic understanding of SQL.
SQLMesh parses and understands the SQL statements. It can translate DuckDB to BigQuery-SQL automatically.
Key features: Python macros, compile-time SQL validation, automatic breaking change detection.
SQLMesh parses and understands the SQL statements. It can translate DuckDB to BigQuery-SQL automatically.
Key features: Python macros, compile-time SQL validation, automatic breaking change detection.
March 11, 2026 at 11:54 PM
SQLMesh is what happens when you take dbt's concept and add semantic understanding of SQL.
SQLMesh parses and understands the SQL statements. It can translate DuckDB to BigQuery-SQL automatically.
Key features: Python macros, compile-time SQL validation, automatic breaking change detection.
SQLMesh parses and understands the SQL statements. It can translate DuckDB to BigQuery-SQL automatically.
Key features: Python macros, compile-time SQL validation, automatic breaking change detection.
Moving on to examining tests in Tobiko SQLMesh as part of my advent series.
Tests in SQLMesh are essentially unit tests, while SQLMesh audits are more akin to the data tests that many are familiar with in dbt.
open.substack.com/pub/davidsj/...
Tests in SQLMesh are essentially unit tests, while SQLMesh audits are more akin to the data tests that many are familiar with in dbt.
open.substack.com/pub/davidsj/...
sqlmesh test
unit tests, generate external models and macros
open.substack.com
December 12, 2024 at 6:45 PM
Moving on to examining tests in Tobiko SQLMesh as part of my advent series.
Tests in SQLMesh are essentially unit tests, while SQLMesh audits are more akin to the data tests that many are familiar with in dbt.
open.substack.com/pub/davidsj/...
Tests in SQLMesh are essentially unit tests, while SQLMesh audits are more akin to the data tests that many are familiar with in dbt.
open.substack.com/pub/davidsj/...
This is the last post in my advent series on Tobiko SQLMesh!
I've had a great time this month learning about SQLMesh - its core features and how they relate to dbt ones and using them practically in my Bluesky Social/dltHub/DuckDB project.
open.substack.com/pub/davidsj/...
I've had a great time this month learning about SQLMesh - its core features and how they relate to dbt ones and using them practically in my Bluesky Social/dltHub/DuckDB project.
open.substack.com/pub/davidsj/...
sqlmesh migrate
Final thoughts and Merry Christmas!
open.substack.com
December 20, 2024 at 5:52 PM
This is the last post in my advent series on Tobiko SQLMesh!
I've had a great time this month learning about SQLMesh - its core features and how they relate to dbt ones and using them practically in my Bluesky Social/dltHub/DuckDB project.
open.substack.com/pub/davidsj/...
I've had a great time this month learning about SQLMesh - its core features and how they relate to dbt ones and using them practically in my Bluesky Social/dltHub/DuckDB project.
open.substack.com/pub/davidsj/...
After a positive experience moving most of our data models at atm.com to SQLMesh, I couldn't help but put this together.
github.com/nicosuave/aw...
github.com/nicosuave/aw...
GitHub - nicosuave/awesome-sqlmesh: A curated list of awesome SQLMesh resources
A curated list of awesome SQLMesh resources. Contribute to nicosuave/awesome-sqlmesh development by creating an account on GitHub.
github.com
October 23, 2024 at 5:20 AM
After a positive experience moving most of our data models at atm.com to SQLMesh, I couldn't help but put this together.
github.com/nicosuave/aw...
github.com/nicosuave/aw...
🏭 @tobikodata.com released a great new feature for SQLMesh last week called blueprinting.
This is a feature that the dbt Labs community has been asking for for ages, and I’ve been wanting it too!
open.substack.com/pub/davidsj/...
This is a feature that the dbt Labs community has been asking for for ages, and I’ve been wanting it too!
open.substack.com/pub/davidsj/...
Model Generation
A big feature release by SQLMesh
open.substack.com
February 26, 2025 at 4:48 PM
🏭 @tobikodata.com released a great new feature for SQLMesh last week called blueprinting.
This is a feature that the dbt Labs community has been asking for for ages, and I’ve been wanting it too!
open.substack.com/pub/davidsj/...
This is a feature that the dbt Labs community has been asking for for ages, and I’ve been wanting it too!
open.substack.com/pub/davidsj/...
dlt infers a lot of metadata from your data, starting with the schema.
Check out this article about dlt-SQLMesh metadata handover that enables generation of SQLMesh incremental models from dlt pipelines.
dlthub.com/blog/sqlmesh...
Check out this article about dlt-SQLMesh metadata handover that enables generation of SQLMesh incremental models from dlt pipelines.
dlthub.com/blog/sqlmesh...
dlt-SQLMesh generator: A case of metadata handover
generate incremental sqlmesh models from dlt sources
dlthub.com
November 1, 2024 at 1:56 PM
dlt infers a lot of metadata from your data, starting with the schema.
Check out this article about dlt-SQLMesh metadata handover that enables generation of SQLMesh incremental models from dlt pipelines.
dlthub.com/blog/sqlmesh...
Check out this article about dlt-SQLMesh metadata handover that enables generation of SQLMesh incremental models from dlt pipelines.
dlthub.com/blog/sqlmesh...
now that the team is fully operational and productive with dbt, I will initiate a big migration to sqlmesh after reading a blog I found here. this is what it means to be an innovator and leader. #dataBS
November 8, 2024 at 9:52 PM
now that the team is fully operational and productive with dbt, I will initiate a big migration to sqlmesh after reading a blog I found here. this is what it means to be an innovator and leader. #dataBS
This is the week's final post in my Tobiko SQLMesh advent series. It's also the last post looking directly at SQLMesh features.
I review some of the remaining CLI commands and features that I believe are relevant; it's by no means exhaustive.
open.substack.com/pub/davidsj/...
I review some of the remaining CLI commands and features that I believe are relevant; it's by no means exhaustive.
open.substack.com/pub/davidsj/...
sqlmesh janitor
mopping up some remaining features and other thoughts about sqlmesh
open.substack.com
December 13, 2024 at 6:59 PM
This is the week's final post in my Tobiko SQLMesh advent series. It's also the last post looking directly at SQLMesh features.
I review some of the remaining CLI commands and features that I believe are relevant; it's by no means exhaustive.
open.substack.com/pub/davidsj/...
I review some of the remaining CLI commands and features that I believe are relevant; it's by no means exhaustive.
open.substack.com/pub/davidsj/...
Supabase branches the entire Postgres stack. Tigris forks object storage buckets like code. Turso and Guepard do instant database branching. Snowflake zero-copy clones and BigQuery snapshots are the same pattern in disguise.
A contender I did not expect: SQLMesh with virtual data environments.
A contender I did not expect: SQLMesh with virtual data environments.
September 24, 2026 at 6:45 AM
Supabase branches the entire Postgres stack. Tigris forks object storage buckets like code. Turso and Guepard do instant database branching. Snowflake zero-copy clones and BigQuery snapshots are the same pattern in disguise.
A contender I did not expect: SQLMesh with virtual data environments.
A contender I did not expect: SQLMesh with virtual data environments.
🚀 Exciting times in the data transformation space! Recently, dbt Labs acquired SDF Labs, while @tobikodata.com acquired Quary (YC W24)—both promising Rust-based frameworks.
So is SDF better than SQLMesh?
No.
open.substack.com/pub/davidsj/...
So is SDF better than SQLMesh?
No.
open.substack.com/pub/davidsj/...
SDF
Is it better than SQLMesh?
open.substack.com
February 12, 2025 at 4:22 PM
🚀 Exciting times in the data transformation space! Recently, dbt Labs acquired SDF Labs, while @tobikodata.com acquired Quary (YC W24)—both promising Rust-based frameworks.
So is SDF better than SQLMesh?
No.
open.substack.com/pub/davidsj/...
So is SDF better than SQLMesh?
No.
open.substack.com/pub/davidsj/...
If you're considering SQLMesh, @jayatillake.bsky.social has a great blog series from the end of last year that does in depth comparisons with DBT
sqlmesh init -t dbt
Backwards Compatibility
davidsj.substack.com
January 28, 2025 at 9:21 PM
If you're considering SQLMesh, @jayatillake.bsky.social has a great blog series from the end of last year that does in depth comparisons with DBT