Discover TernFS — the revolutionary distributed file system from XTX Markets. Designed for immutable data, multi-region resilience & trillion-file scale. Open-sourced in 2025, it’s redefining storage ...

XTX Markets as one of the largest algorithmic trading firms that handles $250 billion in daily traded volume and relies on around 650+ petabytes of storage for its price forecasts and other algorithmic trading data has open-sourced its Linux file-system. XTX developed TernFS for distributed storage after they outgrew their original NFS usage and other file-system alternatives...

XTX Markets is a leading algorithmic trading firm which uses state-of-the-art machine learning technology to produce price forecasts for over 50,000 financial instruments across equities, fixed income...

TernFS – An exabyte scale, multi-region distributed filesystem

TernFS es el sistema de archivos distribuido de código abierto diseñado para exabyte-scale, pensado para operaciones de alto rendimiento, big data, inteligencia artificial y trading algorítmico. Optim...

A distributed file system. Contribute to XTXMarkets/ternfs development by creating an account on GitHub.

XTX Markets is a leading algorithmic trading firm which uses state-of-the-art machine learning technology to produce price forecasts for over 50,000 financial instruments across equities, fixed…

XTX Markets as one of the largest algorithmic trading firms that handles $250 billion in daily traded volume and relies on around 650+ petabytes of storage for its price forecasts and other algorithmic trading data has open-sourced its Linux file-system. XTX developed TernFS for distributed storage after they outgrew their original NFS usage and other file-system alternatives...

XTX Markets is a leading algorithmic trading firm which uses state-of-the-art machine learning technology to produce price forecasts for over 50,000 financial instruments across equities, fixed income, currencies, commodities and crypto. It uses those forecasts to trade on exchanges and alternative trading venues, and to offer differentiated liquidity directly to clients worldwide. The firm trades over $250bn a day across 35 countries and has over 250 employees based in London, Singapore, New York, Paris, Bristol, Mumbai, Yerevan and Kajaani.

XTX Markets is a leading algorithmic trading firm which uses state-of-the-art machine learning technology to produce price forecasts for over 50,000 financial instruments across equities, fixed income...

XTX Markets as one of the largest algorithmic trading firms that handles $250 billion in daily traded volume and relies on around 650+ petabytes of storage for its price forecasts and other algorithmic trading data has open-sourced its Linux file-system

Et encore un article un peu technique pour finir la journée en beauté ! Si je vous disais que votre serveur Linux pouvait gérer 10 exaoctets de données sans broncher ? Vous ne me croiriez pas je pense… D’ailleurs c’est quoi 10 exaoctets ?? Et bien ça correspond à 10 millions de To. C’est pas mal hein ? Hé bien c’est exactement ce que permet de gérer TernFS, le système de fichiers qu’XTX Markets vient de libérer après trois ans d’utilisation intensive. XTX Markets est une boîte d’algo-trading qui brasse 250 milliards de dollars par jour et j’avoue que c’est un joli cadeau de presque-Noël qu’elle vient de nous faire…

Discover how XTX Markets is revolutionizing data storage with TernFS, their open-sourced exabyte-scale distributed filesystem. Explore its features now. The post XTX Markets Open-Sources TernFS, Its Exabyte-Scale Distributed Filesystem appeared first on Linux Today.

Comments

XTX Markets as one of the largest algorithmic trading firms that handles $250 billion in daily traded volume and relies on around 650+ petabytes of storage for its price forecasts and other algorithmi...

Discover how XTX Markets is revolutionizing data storage with TernFS, their open-sourced exabyte-scale distributed filesystem. Explore its features now.

XTX Markets has announced that its in-house filesystem, TernFS, is now available as open-source software, under the GPL-2.0-or-later license for its core filesystem components and Apache-2.0 with LLVM exception for protocol definitions and client libraries Originally built to handle the company’s massive data needs, TernFS is designed for exabyte-scale storage, supporting trillions of files and millions of clients at once. And now, these capabilities are publicly available to everyone, as TernFS enters the landscape, joining projects like Ceph, Lustre, and GlusterFS that have long dominated the field. But before we continue, let’s shed some more light on the filesystem itself. TernFS has been in use at XTX since mid-2023, managing over 500 petabytes of data across three data centers, backed by approximately 30,000 hard disks and 10,000 flash drives. The system is built to scale far beyond that—up to 10 exabytes of logical storage, trillions of files, and millions of concurrent clients. The design of TernFS emphasizes immutability, durability, and multi-region operation. Files are immutable once written, which simplifies replication and consistency at scale. Metadata is distributed across multiple shards, removing single points of failure, while replication and Reed–Solomon coding provide redundancy. Key components include: * Registry for cluster metadata and service tracking * Metadata shards using RocksDB with consensus via LogsDB * Block services to store file contents redundantly across nodes * Cross-Directory Coordinator for operations that span directories Access is supported through a Linux kernel module, a FUSE client, and a minimal S3 API implementation. The system is optimized for large, immutable files—typically larger than a few megabytes. That means it is not well-suited for small files, frequent updates, or directory operations that require rapid changes. For more information, see the announcement or visit the newly published GitHub repository.