everyone asks can you fsync, when you fsync but no one asks how you fsync deferworks.org/posts/can-yo...
Can you even fsync correctly | Opinions on Software.
Ramblings on databases, compilers and software matters.
deferworks.org
November 6, 2026 at 12:13 AM
everyone asks can you fsync, when you fsync but no one asks how you fsync deferworks.org/posts/can-yo...
I re-read and realized I misread the question. The context is that people have started to adopt VSR over Raft because VSRR lets you not fsync prepare/accepts as long as you fsync leader votes, but that’s only the case because there’s no similar rejoin protocol written down for Raft and Paxos.
November 8, 2026 at 11:00 PM
I re-read and realized I misread the question. The context is that people have started to adopt VSR over Raft because VSRR lets you not fsync prepare/accepts as long as you fsync leader votes, but that’s only the case because there’s no similar rejoin protocol written down for Raft and Paxos.
Remember that time I cut 2,0
#Linux #Kernel #Storage #Ext4
https://mustafaerbay.com.tr/en/blog/technology/2000-flushi-sifira-indirdim-fsync-yine-basarili-dondu/
#Linux #Kernel #Storage #Ext4
https://mustafaerbay.com.tr/en/blog/technology/2000-flushi-sifira-indirdim-fsync-yine-basarili-dondu/
I Cut 2,000 Flushes to Zero and fsync Still Returned Success
One line into write_cache: the 2,000 flushes from 1,000 fsync calls dropped to zero, and fsync kept returning success. Measurements and the single if.
mustafaerbay.com.tr
November 2, 2026 at 4:00 PM
Remember that time I cut 2,0
#Linux #Kernel #Storage #Ext4
https://mustafaerbay.com.tr/en/blog/technology/2000-flushi-sifira-indirdim-fsync-yine-basarili-dondu/
#Linux #Kernel #Storage #Ext4
https://mustafaerbay.com.tr/en/blog/technology/2000-flushi-sifira-indirdim-fsync-yine-basarili-dondu/
The Ceph Days Chicago schedule is now live. Talks include State of the #Cephalopod, #RADOS fsync, CRUSH data movement, and Releasing the #Kraken.
Registration is still open: t.ly/CDChicago26
#Ceph #CephDays
Registration is still open: t.ly/CDChicago26
#Ceph #CephDays
November 5, 2026 at 12:01 PM
The Ceph Days Chicago schedule is now live. Talks include State of the #Cephalopod, #RADOS fsync, CRUSH data movement, and Releasing the #Kraken.
Registration is still open: t.ly/CDChicago26
#Ceph #CephDays
Registration is still open: t.ly/CDChicago26
#Ceph #CephDays
O 2.000 flush'ı sıf
#Linux #Cekirdek #Depolama #Ext4
https://mustafaerbay.com.tr/blog/technology/2000-flushi-sifira-indirdim-fsync-yine-basarili-dondu/
#Linux #Cekirdek #Depolama #Ext4
https://mustafaerbay.com.tr/blog/technology/2000-flushi-sifira-indirdim-fsync-yine-basarili-dondu/
2.000 Flush'ı Sıfıra İndirdim, fsync Yine Başarılı Döndü
write_cache dosyasına tek satır yazdım: 1.000 fsync'in diske gönderdiği 2.000 flush sıfıra indi, fsync hâlâ başarı döndürdü. Ölçümler ve kaynaktaki tek if.
mustafaerbay.com.tr
November 2, 2026 at 4:00 PM
O 2.000 flush'ı sıf
#Linux #Cekirdek #Depolama #Ext4
https://mustafaerbay.com.tr/blog/technology/2000-flushi-sifira-indirdim-fsync-yine-basarili-dondu/
#Linux #Cekirdek #Depolama #Ext4
https://mustafaerbay.com.tr/blog/technology/2000-flushi-sifira-indirdim-fsync-yine-basarili-dondu/
tbf when I ask the kernel to fsync a file I wrote I really do not know what it is doing underneath . the effect has been summarized to me and Im happy submitting any bugs I find to the mailing list. we are already adjusting our resolution of effects all the time!
November 1, 2026 at 7:26 PM
tbf when I ask the kernel to fsync a file I wrote I really do not know what it is doing underneath . the effect has been summarized to me and Im happy submitting any bugs I find to the mailing list. we are already adjusting our resolution of effects all the time!
it’s funny this is exactly as mystical as ‘does fsync do what you think fsync does’ in a seemingly simpler time
October 31, 2026 at 7:31 PM
it’s funny this is exactly as mystical as ‘does fsync do what you think fsync does’ in a seemingly simpler time
A CSI driver with shared storage definitely solves the scheduling trap. But I have to strongly disagree on that last point for databases. As noted in the article, running Postgres over NFS is usually a recipe for disaster. That 'network hop' absolutely murders fsync latency on the WAL.
November 1, 2026 at 7:42 PM
A CSI driver with shared storage definitely solves the scheduling trap. But I have to strongly disagree on that last point for databases. As noted in the article, running Postgres over NFS is usually a recipe for disaster. That 'network hop' absolutely murders fsync latency on the WAL.
like honestly what the fuck do you mean KDE as a project has intentionally had data loss for several years because they, with full knowledge, made the technical tradeoff "better for users to lose data irrecoverably, than it is to be slow"
this is not what i want from my desktop and FILE MANAGER.
this is not what i want from my desktop and FILE MANAGER.
November 4, 2026 at 6:07 PM
like honestly what the fuck do you mean KDE as a project has intentionally had data loss for several years because they, with full knowledge, made the technical tradeoff "better for users to lose data irrecoverably, than it is to be slow"
this is not what i want from my desktop and FILE MANAGER.
this is not what i want from my desktop and FILE MANAGER.
I also prefer to immediately ack before durability! when the disk decides to fsync is none of my business
October 28, 2026 at 11:29 PM
I also prefer to immediately ack before durability! when the disk decides to fsync is none of my business
4. at rotation, fsync the old wal, commit the metadata atomically, then write the new wal
5. checkpointing at recovery is safe only if buffered versions cant be rolled back
5. checkpointing at recovery is safe only if buffered versions cant be rolled back
October 29, 2026 at 5:07 PM
4. at rotation, fsync the old wal, commit the metadata atomically, then write the new wal
5. checkpointing at recovery is safe only if buffered versions cant be rolled back
5. checkpointing at recovery is safe only if buffered versions cant be rolled back
Computing I/O operations rely on buffering to improve performance, but this creates an asymmetry between reading and writing. While reads report errors immediately, buffered writes delay system calls, often hiding errors until close or requiring fsync.
Reading and writing are less symmetric than you (probably) think (107)
I am talking about the I/O operations as used in computing here. A typical example of how this kind of thing is exposed are the POSIX syscalls read(2) and write(2), which have the following C function prototypes: ssize_t read(int fd, void *buf, size_t count); ssize_t write(int fd, const void *buf, size_t count); Now these are…
news.ycombinator.com
November 1, 2026 at 7:46 AM
Computing I/O operations rely on buffering to improve performance, but this creates an asymmetry between reading and writing. While reads report errors immediately, buffered writes delay system calls, often hiding errors until close or requiring fsync.
when consuming atproto data it seems obvious that if you somehow lost a couple seconds of data to an outage, your consumer would also fail to advance, and you get a chance to replay . lots of non-atproto data streams offer this capability too! your approach reminds me much of kafka or fjall
Why Apache Kafka doesn't need fsync to be safe — Jack Vanlightly
TLDR: Apache Kafka doesn’t need fsyncs to be safe because it includes recovery in its replication protocol. It is a real-world distributed system that uses asynchronous log writing + recovery with som...
jack-vanlightly.com
October 28, 2026 at 11:58 PM
when consuming atproto data it seems obvious that if you somehow lost a couple seconds of data to an outage, your consumer would also fail to advance, and you get a chance to replay . lots of non-atproto data streams offer this capability too! your approach reminds me much of kafka or fjall
Building an xAPI 2.0 library and LRS for .NET: a conformance suite whose exit code wraps at 256, Utf8JsonReader for Native AOT, fsync on macOS, SERIALIZABLE on PostgreSQL, and await in Unity WebGL. hubs.li/Q04BtNzH0 #dotnet #csharp
What I learned building an xAPI 2.0 library and LRS for .NET
xAPI (the Experience API, now IEEE 9274.1.1) is how e-learning content reports what a learner did:...
hubs.li
October 27, 2026 at 4:13 PM
Building an xAPI 2.0 library and LRS for .NET: a conformance suite whose exit code wraps at 256, Utf8JsonReader for Native AOT, fsync on macOS, SERIALIZABLE on PostgreSQL, and await in Unity WebGL. hubs.li/Q04BtNzH0 #dotnet #csharp
- Wait time in slow query logs
- Slow query logs on vlstorage nodes
- #Unix domain socket support
- New #fsync latency panel
Explore the new features and learn how to get started 👇
victoriametrics.com/blog/victori...
#Monitoring #DevOps #LogManagement #Logs
- Slow query logs on vlstorage nodes
- #Unix domain socket support
- New #fsync latency panel
Explore the new features and learn how to get started 👇
victoriametrics.com/blog/victori...
#Monitoring #DevOps #LogManagement #Logs
October 22, 2026 at 9:15 AM
- Wait time in slow query logs
- Slow query logs on vlstorage nodes
- #Unix domain socket support
- New #fsync latency panel
Explore the new features and learn how to get started 👇
victoriametrics.com/blog/victori...
#Monitoring #DevOps #LogManagement #Logs
- Slow query logs on vlstorage nodes
- #Unix domain socket support
- New #fsync latency panel
Explore the new features and learn how to get started 👇
victoriametrics.com/blog/victori...
#Monitoring #DevOps #LogManagement #Logs
NFS without writeback caching is doing a synchronous commit per write — the guest does not see the ACK until the server has it on stable storage, so every fsync becomes a round trip. iSCSI hides it because the block layer can reorder and coalesce behind the guest. Practical split: keep iSCSI …
October 19, 2026 at 10:19 AM
NFS without writeback caching is doing a synchronous commit per write — the guest does not see the ACK until the server has it on stable storage, so every fsync becomes a round trip. iSCSI hides it because the block layer can reorder and coalesce behind the guest. Practical split: keep iSCSI …
We saved the config by writing straight over the old file.
The process was killed mid write, and the next start read a truncated file that no longer parsed.
A half written file is not a saved file.
Write to a temp file, fsync it, then rename it over the original.
The process was killed mid write, and the next start read a truncated file that no longer parsed.
A half written file is not a saved file.
Write to a temp file, fsync it, then rename it over the original.
October 18, 2026 at 5:00 AM
We saved the config by writing straight over the old file.
The process was killed mid write, and the next start read a truncated file that no longer parsed.
A half written file is not a saved file.
Write to a temp file, fsync it, then rename it over the original.
The process was killed mid write, and the next start read a truncated file that no longer parsed.
A half written file is not a saved file.
Write to a temp file, fsync it, then rename it over the original.
Direct-attached NVMe is the detail worth more than the 2 GB. Fsync latency drops a lot versus network storage, and the price is that a dead host takes the disk with it, so replicas carry the safety.
October 15, 2026 at 3:46 PM
Direct-attached NVMe is the detail worth more than the 2 GB. Fsync latency drops a lot versus network storage, and the price is that a dead host takes the disk with it, so replicas carry the safety.
In our Sysbench tests, switching WAL commits from fsync to fdatasync and prewriting zero-filled journal regions improved Dolt’s write performance by 10–40%.
Aaron explains why the change works, where it doesn’t, and some of the tradeoffs: https://ow.ly/1O2I50ZXKye
Aaron explains why the change works, where it doesn’t, and some of the tradeoffs: https://ow.ly/1O2I50ZXKye
fdatasync vs. fsync for WAL commits: When is it faster and what are the tradeoffs?
Dolt's WAL commit implementation on Linux recently changed from fsync to fdatasync. Internal sysbench runs showed a big improvement. What are the tradeoffs? When would a user's workflow see similar gains?
ow.ly
October 14, 2026 at 5:46 PM
In our Sysbench tests, switching WAL commits from fsync to fdatasync and prewriting zero-filled journal regions improved Dolt’s write performance by 10–40%.
Aaron explains why the change works, where it doesn’t, and some of the tradeoffs: https://ow.ly/1O2I50ZXKye
Aaron explains why the change works, where it doesn’t, and some of the tradeoffs: https://ow.ly/1O2I50ZXKye
PostgreSQL 18, one connection, single-row inserts: synchronous_commit = off did 47,000 commits/s and fsync = off did 36,000. With fsync off each backend still makes one WAL write per commit; async commit leaves that to the WAL writer too.
October 13, 2026 at 3:41 AM
PostgreSQL 18, one connection, single-row inserts: synchronous_commit = off did 47,000 commits/s and fsync = off did 36,000. With fsync off each backend still makes one WAL write per commit; async commit leaves that to the WAL writer too.
PostgreSQL 18 on a Mac: 14,400 single-row commits/s with the default wal_sync_method, 260 with fsync_writethrough, the only method that makes the drive flush. Measured with pgbench.
https://importstatic.com/databases/postgres-synchronous-commit-fsync-cost
https://importstatic.com/databases/postgres-synchronous-commit-fsync-cost
PostgreSQL's default wal_sync_method on a Mac skips the drive flush
PostgreSQL's default wal_sync_method on macOS never forces the drive to flush. With fsync_writethrough, 14,400 commits a second became 260. Crash test included.
importstatic.com
October 12, 2026 at 3:29 PM
PostgreSQL 18 on a Mac: 14,400 single-row commits/s with the default wal_sync_method, 260 with fsync_writethrough, the only method that makes the drive flush. Measured with pgbench.
https://importstatic.com/databases/postgres-synchronous-commit-fsync-cost
https://importstatic.com/databases/postgres-synchronous-commit-fsync-cost
Someone needs to spec out a lightweight rejoin protocol for Raft (e.g. don't vote until your log is caught up to the leader), as it seems only safer to not fsync with VSR because we have no "rejoining" state specified for Raft. It's possible PreVote does almost the necessary work already?
October 9, 2026 at 7:26 PM
Someone needs to spec out a lightweight rejoin protocol for Raft (e.g. don't vote until your log is caught up to the leader), as it seems only safer to not fsync with VSR because we have no "rejoining" state specified for Raft. It's possible PreVote does almost the necessary work already?
I bet patching Bionic to nop out fsync would fix this. 🤡
October 4, 2026 at 6:25 PM
I bet patching Bionic to nop out fsync would fix this. 🤡