#datafusion
i have been building mizumi: sql over atproto data. it has only @tangled.org lexicons as of now

it reads from hydrant, loads everything into datafusion, and you can query any lexicon with sql. here’s stars on tangled since day one 👇
November 11, 2026 at 3:44 PM
I'll be speaking at the Apache DataFusion meetup tomorrow in SF: luma.com/k04h0om1
San Francisco Apache DataFusion Meetup · Luma
Join us for an evening of talks and community discussion about Apache DataFusion and its growing role in modern data infrastructure. This meetup will spotlight…
luma.com
November 9, 2026 at 4:39 PM
Gabriel Musat Mestre - How we extended Apache DataFusion to execute one query across many machines | Datadog www.datadoghq.com/blog/enginee...
November 10, 2026 at 4:27 PM
We had a great time at the latest Apache DataFusion meetup in San Francisco, with four talks on four very different ways to build on DataFusion. Thanks to everyone who came! The next meetup is in February, and there’s room for speakers. If you’d like to give a talk, contact any of the organizers.
November 13, 2026 at 9:58 PM
burrmill's demo is online: drop a csv, json or parquet file in the tab and run sql over it, nothing leaves the browser. it's the sql engine inside nuthatch, built on datafusion

burrmill-demo.vercel.app
November 16, 2026 at 9:04 AM
Do we still have to denormalize? Have your schema and search it too | Discussion
Do We Still Have to Denormalize? Have Your Schema and Search It Too
How ParadeDB accelerates search and aggregate joins over normalized Postgres schemas using DataFusion, Rank Joins, co-partitioning, and tuple tagging.
www.paradedb.com
November 16, 2026 at 2:40 AM
👋 Bay Area: Apache DataFusion meetup in SF, Tue Nov 10, 5:30 PM PT.

Andrew Lamb on reading Apache Parquet, plus Oleander, Oodle AI, and LangChain on lineage, serverless logs, and AI search. Food arrives in record batches.

Register for the address: https://bit.ly/3VMvWgV
San Francisco Apache DataFusion Meetup · Luma
Join us for an evening of talks and community discussion about Apache DataFusion and its growing role in modern data infrastructure. This meetup will spotlight…
luma.com
November 9, 2026 at 11:54 PM
contrived a reason to use github.com/apache/dataf... in my sim because it’s neat and i wanted to lol
GitHub - apache/datafusion: Apache DataFusion SQL Query Engine
Apache DataFusion SQL Query Engine. Contribute to apache/datafusion development by creating an account on GitHub.
github.com
November 6, 2026 at 1:46 AM
and there is no sql database backing it ;) datafusion is so cool
November 6, 2026 at 5:24 PM
You can represent search queries as SQL (JOIN, ORDER BY, LIMIT, GROUP BY), but the performance is 🤮; This results in systems like Elastic/OpenSearch. Neat blog from @paradedb.com about making the SQL version fast enough to avoid the Galapagos effect www.paradedb.com/blog/joins
Do We Still Have to Denormalize? Have Your Schema and Search It Too
How ParadeDB accelerates search and aggregate joins over normalized Postgres schemas using DataFusion, Rank Joins, co-partitioning, and tuple tagging.
www.paradedb.com
November 4, 2026 at 11:55 AM
Most graph traversals over tabular data require repeated joins. lance-csr takes another route: build a compressed sparse row index and turn each hop into an array lookup. An interesting path toward Cypher directly over Lance and Arrow.

thedataquarry.com/blog/lance-c...
Fast graph queries on Lance (1): Introducing lance-csr
Lance-csr is a fast graph query engine that adds a CSR adjacency index on Lance datasets, with Apache DataFusion at its core.
thedataquarry.com
November 7, 2026 at 12:02 AM
We made single-table search fast. But a search join (one query across tables, ranked by combined relevance) still meant denormalizing, or letting Postgres join and score millions of rows to return 20.

Now a 3-table search join runs 1000x faster.

www.paradedb.com/blog/joins
Do We Still Have to Denormalize? Have Your Schema and Search It Too
How ParadeDB accelerates search and aggregate joins over normalized Postgres schemas using DataFusion, Rank Joins, co-partitioning, and tuple tagging.
www.paradedb.com
November 3, 2026 at 4:11 PM
At @paradedb.com, we've spent the last few months leaning in to tackle one of the hardest first questions we hear from our users: "Our data is normalized, so: Is it fast over joins?"

There is always more work to do, but it's safe to say that the answer is now: "yes": www.paradedb.com/blog/joins
Do We Still Have to Denormalize? Have Your Schema and Search It Too
How ParadeDB accelerates search and aggregate joins over normalized Postgres schemas using DataFusion, Rank Joins, co-partitioning, and tuple tagging.
www.paradedb.com
November 3, 2026 at 3:59 PM
Do We Still Have to Denormalize? Have Your Schema and Search It Too
tech_blogs_parade_db | Author: Stu Hood

#Databases
Do We Still Have to Denormalize? Have Your Schema and Search It Too
How ParadeDB accelerates search and aggregate joins over normalized Postgres schemas using DataFusion, Rank Joins, co-partitioning, and tuple tagging.
www.paradedb.com
November 3, 2026 at 5:30 AM
The agent I’ve been working with and I have a habit of saluting each other.
October 31, 2026 at 7:27 PM
Apache DataFusion San Francisco Meetup Nov 10: luma.com/k04h0om1. I will be talking about Advancing the State of the Art of Apache Parquet Readers
San Francisco Apache DataFusion Meetup · Luma
Join us for an evening of talks and community discussion about Apache DataFusion and its growing role in modern data infrastructure. This meetup will spotlight…
luma.com
October 30, 2026 at 9:25 AM
In the last day I have requested changes on more pull requests in Apache DataFusion than I think I have in the entire 6 years with the project. We need to slow things down to reduce the chaos and spend our time actually thinking and planning the code.
October 29, 2026 at 2:33 PM
Up to 8x faster point queries with Apache DataFusion with the right tuning: datafusion.apache.org/blog/2026/10...
October 25, 2026 at 3:57 PM
Worth a read: Nuno Faria writes about best practices for single row lookups on the Apache DataFusion blog.

Also: Flapping Airplanes raised a $1B Series A. And Gimlet Labs is hiring a compiler engineer in SF.

theconsensus.dev/g/2026/10/26/
October 26, 2026 at 1:39 PM
#DTS
A simulation framework aligns and fuses event data recorder, GPS, and video evidence, overcoming single-source limitations for precise traffic accident reconstruction.
Details: https://www.maxapress.com/article/doi/10.48130/dts-0026-0018
#AccidentReconstruction #EDR #DataFusion #VehicleSafety
October 25, 2026 at 5:46 AM
I've been working on a feature for Streamling, and I needed to filter out data based on statistics (with min/max/null counts).

DataFusion (which Streamling uses) already has built-in support for this - PruningPredicate! It can rewrite expressions if they're compatible with available statistics.
October 20, 2026 at 4:29 PM
🍻 tpcgen-cli 🍻

Fast TPC-H and TPC-DS data generator

🔗 https://github.com/datafusion-contrib/tpcgen-rs

#homebrew #newpkg #macos #linux #formula
October 20, 2026 at 5:27 PM
Join the Apache DataFusion community in San Francisco for an evening of lightning talks, mingling, and snacks.

🎙️ Speaker lineup coming soon
📍 November 12 | 5:30 PM-8:30 PM

Come for the talks, stay for the community, leave with new ideas.

Save the date: https://bit.ly/3VMvWgV
October 20, 2026 at 7:40 PM
Cool project that's worth checking out if you're running Flink at scale: github.com/datafusion-c...
GitHub - datafusion-contrib/StreamFusion: World's First OSS Flink Accelerator built on Apache DataFusion
World's First OSS Flink Accelerator built on Apache DataFusion - datafusion-contrib/StreamFusion
github.com
October 16, 2026 at 4:06 PM
Polars 2.0

Polars 2.0 introduces initial out-of-core spill-to-disk support, major performance improvements, first-class SQL, a new Map dtype, and stricter dtype handling. With the SQL and performance work, Polars reports leading DataFusion and DuckDB on TPC…

https://pola.rs/posts/release-polars-2/
October 16, 2026 at 4:15 AM