DISTORT

Disaggregated Storage over RDMA orchestrated in Kubernetes

Direct RDMA Data Path

Connects consumer nodes to remote NVMe storage over RDMA. Once a volume is mounted, application I/O does not pass through the management controller.

Kubernetes-Native Schedulers

Driven by a declarative Custom Resource Definition (CRD) state machine to coordinate device discovery, node health, partition claims, and volume bindings.

SPDK User-Space Polling

Supports SPDK for target-side user-space NVMe access and polling, alongside a Linux kernel target backend. Consumer nodes use the kernel NVMe initiator.

Welcome to DISTORT

DISTORT (DISaggregated STorage Over RDMA Transport) manages physical NVMe storage for Kubernetes workloads over NVMe-over-Fabrics/RDMA. Administrators explicitly claim unused devices; applications request filesystem volumes using StorageClasses and PersistentVolumeClaims.

Kubernetes custom resources coordinate discovery, allocation, exports, and consumer ownership. Node agents configure SPDK or Linux kernel targets, while the CSI driver connects and mounts volumes on consumer nodes. After setup, application I/O travels directly over RDMA without passing through the manager.

Project Status

DISTORT is an alpha-stage project moving toward beta. Version 0.5.0 is a development release and is not recommended for production use. The project has been tested with K3s 1.35.4 in its three-node Vagrant lab and with Kubernetes 1.35.5 in a three-node kubeadm cluster.

These are tested configurations, not a compatibility certification. The current driver supports ext4/XFS mounted single-node-writer volumes. Raw block, multi-node access, snapshots, cloning, expansion, replication, multipath failover, and an NVMe/TCP fallback are not implemented. See the roadmap for proposed work.

Control and data paths

flowchart LR Kubernetes["Kubernetes PVC and Pod"] --> CSI["CSI lifecycle operations"] CSI --> Resources["DISTORT resources"] Resources --> Controllers["Manager and node agents"] Controllers -.->|Configure| Target["SPDK or kernel target"] App["Application / consumer NVMe initiator"] <-->|NVMe-oF / RDMA| Target Target <--> Disk["Physical NVMe"]

Getting Started

Explore the different sections of the documentation to understand and consume DISTORT:

  • Architecture: Read the design, component roles, control and data paths, and implementation layout.
  • Project Internals: Follow the controller, agent, CSI, and storage data paths in detail, including recovery boundaries.
  • Using DISTORT: Learn how to discover underlying storage controllers, claim hardware drives using NVMeDeviceClaim specs, and request StorageClasses.
  • Local Testing Lab: Keep a three-node Vagrant/K3s cluster running for fast redeploys, manual storage checks, and automated E2E tests.
  • Testing Strategy: Run the host-side, regression, and hardware test suites.
  • Contributing: Build, generate, validate, and submit changes using the repository’s supported workflows.

Community

Ask questions and propose changes through GitHub Issues, or contact the maintainers. Follow the Code of Conduct and use the private security reporting process for vulnerabilities. The governance policy describes how maintainers make decisions.