#HDF5
LightNews LightNews
dd86k.space
Hire me. I understand SQLite3, HDF5, and DCOM #OpenForHire
July 27, 2026 at 3:35 PM
fmrib-steve.bsky.social
Preprint on PANDORA by Aslan Abivardi:

A massive archive of UK Biobank brain imaging from 82K subjects. For each of 98 sub-modalities (e.g., FA from dMRI), the images are collated into a convenient subjectsXvoxels HDF5 file.

pages.fmrib.ox.ac.uk/pandora/web/

www.medrxiv.org/content/10.6...
January 7, 2026 at 8:15 AM
preibischs.bsky.social
Major update of #BigStitcher & #BigStitcherSpark:

- 10x faster fusion
- cloud support (S3 & GS)
- improved interest-point alignment
- worst error visualization
- improved virtual image splitting
- Zarr/N5/HDF5 export
- consolidated menu structure

#OSSI @HHMIJanelia @FijiSc
November 8, 2024 at 9:35 PM
permadeath.com
i have been an admin for mysql, postgres, doltdb, cassandra, mongo, redis, redshift, snowflake, sqlite, on-disk hdf5, and probably a dozen more

ask me for a database and you're getting managed postgres until you convince me it won't work
why.bsky.world Why @why.bsky.world · Apr 3
I try not to foist my kinks on others these days but yes im absolutely a postgres sicko
April 3, 2026 at 1:15 PM
joshmoore.bsky.social
ICYMI: A MATLAB implementation of @zarr.dev has dropped from @mathworks.bsky.social 🚀

github.com/mathworks/MA...
GitHub - mathworks/MATLAB-support-for-Zarr-files: This contribution provides an interface in MATLAB to read and write Zarr files. Zarr is a file format for scientific data (like HDF5 and netCDF) which...
This contribution provides an interface in MATLAB to read and write Zarr files. Zarr is a file format for scientific data (like HDF5 and netCDF) which is optimized for cloud storage and parallel co...
github.com
March 18, 2025 at 10:41 PM
dcjones.bsky.social
Agree, but with the caveat that these binary formats should nearly always be based on some existing lower level open format like parquet, hdf5, zarr, etc.
August 5, 2025 at 6:05 PM
orhun.dev
Working with scientific data in the terminal!

🔬 h5v — A HDF5 explorer TUI for visualizing and inspecting datasets

💯 Browse files, plot charts, explore matrices & heatmaps, preview images

🦀 Written in Rust & built w/ @ratatui.rs

⭐ GitHub: github.com/DanielHauge/...

#rustlang #ratatui #tui
September 2, 2026 at 5:07 PM
hikikomorphism.bsky.social
the thing about this is, a lot of the steps in the chain show horrible safety practices and embarrassing failures on the part of the evaluators, sure, but the agent swarm also did find and exploit zero-days - maybe not insanely impressive ones, but not an open-door 'password was admin' thing either
The AI models exploited a zero-day vulnerability in JFrog’s product to elevate their privileges, then moved laterally to an internet-connected system. The confirmation came one day after JFrog announced patches for nine Artifactory vulnerabilities, crediting OpenAI for finding “previously unknown zero-day vulnerabilities in self-hosted Artifactory installations that could be exploited to gain unintended internet access.” JFrog said that OpenAI immediately disclosed the security defects responsibly, but did not specifically mention that the zero-days were exploited in the Hugging Face incident. “We developed, validated, and released a fix for all JFrog customers, self-hosted and cloud alike,” JFrog’s CTO Yoav Landman says, underlining the need to address newly uncovered bugs faster in the AI era. Reports on the July 2026 OpenAI agent intrusion have highlighted HDF5, as a feature of the format was used in the attack. The following summarizes our knowledge as of July 30. During a cybersecurity evaluation, an autonomous agent escaped its environment, gained internet access, and attacked Hugging Face’s production infrastructure. Hugging Face identified approximately 17,600 agent actions across multiple systems and trust boundaries. One of the agent’s initial techniques used HDF5. The agent uploaded valid HDF5 files with datasets referencing raw data stored on the processing server. When Hugging Face’s dataset service opened these files, it read local files, including /proc/self/environ, and returned their contents through its API.
September 14, 2026 at 7:02 PM
lshaw-ironarray.bsky.social
📢Blosc2 3.10.2 Released! 📢

Lazy evaluation for ALL Blosc2 functions in version 3.10!

Blosc2 powers:
- Zarr/HDF5 computations (see image and blosc.org/posts/tensor...)
- Server compuation via Cat2Cloud (demo site here cat2.cloud/demo/)!

#Zarr
#HDF5
#Python
October 15, 2025 at 11:58 AM
skepteis.bsky.social
Nice (fresh) open source project I discovered:
A hdf5/h5 file viewer github.com/DanielHauge/...
GitHub - DanielHauge/h5v: HDF5 Terminal Viewer
HDF5 Terminal Viewer. Contribute to DanielHauge/h5v development by creating an account on GitHub.
github.com
August 22, 2025 at 3:41 PM
lgmartelotto.bsky.social
www.biorxiv.org/content/10.1...
anndataR improves interoperability between R and Python in single-cell transcriptomics
Many single-cell transcriptomics datasets are stored in the HDF5-backed AnnData (H5AD) file format, as popularised by the Python scverse ecosystem. However, accessing these datasets from R, allowing u...
www.biorxiv.org
August 22, 2025 at 2:57 PM
@russpoldrack.org convincing you to prefer zarr over HDF5 in about 10 lines of code. Thanks Russ!
January 13, 2026 at 8:23 PM
mdsumner.bsky.social
do you worry that #RStats was left out of new cool python stuff for arrays? meet vrefs, an array chunk virtualizer leveraging #GDAL multidim #VRT and HDF5 github.com/mdsumner/vrefs
GitHub - mdsumner/vrefs: builds virtual Zarr reference stores from collections of NetCDF4/HDF5 files
builds virtual Zarr reference stores from collections of NetCDF4/HDF5 files - mdsumner/vrefs
github.com
June 3, 2026 at 12:48 PM
giswqs.bsky.social
Working with Planet Tanager Hyperspectral Data (426 Bands) in QGIS with HyperCoast

In this tutorial, you will explore freely available Planet Tanager open data and learn how to stream imagery, download HDF5 datasets, inspect spectral signatures, and perform visualization directly inside QGIS.
May 25, 2026 at 2:56 PM
robp.bsky.social
Ehh.... that's not *always* worse. HDF5 is pretty bad. I wonder if it would be possible to shove an HDF5 blob in the final column of a record in a GTF file ... 🤔
August 6, 2025 at 8:23 PM
dd86k.space
Speaking of sqlite3, I have tried HDF5 and it.. works well. You will be overwhelmed at first lol

Didn't try 2.0 though. At oldjob, someone misused a multidimensional matrix and didn't want to fix their code 🍮
March 3, 2026 at 3:05 AM
saramcardle.bsky.social
Chaotic good! I honestly spent 5 minutes debating whether hdf5 or ome.zarr was more chaotic
August 8, 2025 at 6:53 PM
ironarray.io
Unlock the power of your HDF5 data in the cloud! ☁️

Cat2Cloud now offers native, out-of-the-box support for HDF5 files. Upload and process your HDF5 data as seamlessly as any other format. ✨

Ready to see it in action? Try our online demo server today! 👉 cat2cloud.com/demo/

#HDF5 #CloudComputing
November 24, 2025 at 12:47 PM
francescalted.bsky.social
We're making HDF5, the standard for scientific data 📊, faster ⚡ and more versatile 🛠️. By integrating Blosc2 with HDF5, our Caterva2 platform allows you to natively serve ☁️, share 🤝, visualize 👀, and compute 💻 your HDF5 datasets with greater efficiency and ease.

Slides:
www.blosc.org/docs/2025-Eu...
Caterva2 displaying the result of a computed tomography of a large HDF5 dataset in real-time.
May 27, 2025 at 4:46 PM
inggrid.bsky.social
New software descriptor published on ing.grid! "h5RDMtoolbox - A Python Toolbox for FAIR Data Management around HDF5" by Matthias Probst and Balazs Pritz www.inggrid.org/article/id/4... #RDM #HDF5 #python #FAIR #ResearchSoftware
h5RDMtoolbox - A Python Toolbox for FAIR Data Management around HDF5
Sustainable data management is fundamental to efficient and successful scientific research. The FAIR principles (Findable, Accessible, Interoperable and Reusable) have been proven to be successful gui...
www.inggrid.org
August 27, 2024 at 8:58 AM
aki.lethalbit.net
So i'm still thinking about computational electrodynamics stuff, for some reason.

And i'm yearning for a decent visualization framework for that kinda stuff for Rust, something that gives me a vulkan canvas and hot reloading with some QoL stuff, rather than having to write out VTK or HDF5 files
January 15, 2026 at 12:13 PM
developmentseed.org
TiTiler-CMR
Dynamic tiling for Earth data

→ Pulls data straight from NASA’s CMR
→ Works with NetCDF, HDF5, COGs
→ Powers VEDA’s IMERG + HLS products
→ Fully open source

Blog: www.earthdata.nasa.gov/news/blog/vi...
Repo: developmentseed.org/titiler-cmr/...
Demo: www.loom.com/share/af6fef...
June 13, 2025 at 10:39 PM
yuton.org
あらゆる巨大データ、HDF5で保存しておいてほしい
January 9, 2025 at 12:26 PM
hughagraham.bsky.social
Still a WIP but I've been building a new #rstats 📦 for cloud optimised reads of #GEDI and IceSat-2 data. It's 20 x quicker than direct download and saves 10-100s of GiB on disk. The #rust powered hdf5 parser is crazy fast - with help from the 🤖.
belian-earth.github.io/spacelaser/ #rspatial #rextendr
GitHub - belian-earth/spacelaser: An R package for cloud optimised partial reading of GEDI and IceSat data.
An R package for cloud optimised partial reading of GEDI and IceSat data. - belian-earth/spacelaser
github.com
April 17, 2026 at 6:47 PM
dd86k.space
Depends! I could even throw in HDF5! Make all the things cursed!
November 26, 2025 at 10:21 PM