We built a pure-Rust HDF5 implementation from scratch. No libhdf5, no C toolchain, no build scripts. Files round-trip with h5py.
HDF5 has had exactly one implementation for 30 years — the C library from The HDF Group. Every language binding (Python's h5py, Julia's HDF5.jl, the existing Rust hdf5 crate) wraps it. That creates a hard dependency on a C toolchain, breaks cross-compilation, and makes no_std use impossible.
rustyhdf5 replaces it entirely:
55× faster file open (377µs vs 20.9ms) via mmap — no global state init, no symbol table loading
Zero-copy contiguous reads — data served directly from OS page cache, no allocation
no_std support — rustyhdf5-format compiles without std, works in firmware/WASM/embedded
h5py compatible — full round-trip fidelity including chunked datasets, compressed data, compound types, variable-length strings
SHINES provenance — built-in SHA-256 content hashing + creator/timestamp metadata baked into the file
We also built EdgeHDF5 on top of it — a single-file agent memory store with adaptive vector search (Scalar → SIMD → BLAS → Accelerate → GPU → IVF-PQ), hybrid BM25+vector retrieval, and Product Quantization for 8× storage compression. The whole thing is one .h5 file, no daemon required.
GitHub:
https://github.com/rustystack/rustyhdf5https://github.com/rustystack/edgehdf5
Happy to answer questions about the format implementation, the no_std approach, or the benchmark methodology.
We built a pure-Rust HDF5 implementation from scratch. No libhdf5, no C toolchain, no build scripts. Files round-trip with h5py. HDF5 has had exactly one implementation for 30 years — the C library from The HDF Group. Every language binding (Python's h5py, Julia's HDF5.jl, the existing Rust hdf5 crate) wraps it. That creates a hard dependency on a C toolchain, breaks cross-compilation, and makes no_std use impossible. rustyhdf5 replaces it entirely:
55× faster file open (377µs vs 20.9ms) via mmap — no global state init, no symbol table loading Zero-copy contiguous reads — data served directly from OS page cache, no allocation no_std support — rustyhdf5-format compiles without std, works in firmware/WASM/embedded h5py compatible — full round-trip fidelity including chunked datasets, compressed data, compound types, variable-length strings SHINES provenance — built-in SHA-256 content hashing + creator/timestamp metadata baked into the file
We also built EdgeHDF5 on top of it — a single-file agent memory store with adaptive vector search (Scalar → SIMD → BLAS → Accelerate → GPU → IVF-PQ), hybrid BM25+vector retrieval, and Product Quantization for 8× storage compression. The whole thing is one .h5 file, no daemon required. GitHub: https://github.com/rustystack/rustyhdf5 https://github.com/rustystack/edgehdf5 Happy to answer questions about the format implementation, the no_std approach, or the benchmark methodology.