Files
millerson-overlay.nix/docs/therock-rocm.md
T
alex 16166f115b feat(rocm): Vendor SDK recipe and update to 10.1
Maintain the gfx1151 SDK locally using the official AMD archive. Use the TheRock patchelf pin and adapt LLVM auxiliary tools for pure Nix builds.

Verify SDK installation, HIP and rocBLAS GPU operations, and rebuild its consumers. Document the internal rc3 stamp and upstream migration findings.
2026-11-12 23:40:48 +03:00

4.7 KiB

TheRock ROCm SDK

The therock-rocm package is maintained in this repository. packages/therock-rocm/package.nix pins the version, GPU target and archive hash; sdk.nix installs and adapts the official AMD binary SDK to Nix. It was adapted from hellas-ai/nix-strix-halo at b2969bd. It no longer imports that project's ROCm recipe or source metadata. The flake input remains necessary for Gufo's XRT package.

The SDK archive is therock-dist-linux-gfx1151-10.1.0.tar.gz. Its manifest declares rocm_version = 10.1.0 and an internal rocm_package_version = 10.1.0rc3. Both are checked during installation; the internal build stamp is also exposed as therock-rocm.packageVersion. Hipfire (including its kernels), Strata, Crucible and Strixite share this package. Crucible retains its official engine binary compiled with ROCm 7.2.4; its runtime comes from this SDK. Kyojin, Gufo and the llama.cpp forks retain their own upstream toolchain choices.

As reviewed on 2026-11-12, nixpkgs PR #579527 is open and targets staging. It is the final part of a series replacing the 7.2.4 scope with ROCm 10.1, including LLVM 24. The author reports hardware checks on gfx1102 and gfx906; this does not validate gfx1151. The PR also adds hipfile for torch/vLLM and libdrm for torch. Those source-package changes are not needed for our standalone inference engines' bundled SDK.

Issue #563157 discusses the TheRock/ROCm naming transition, LLVM 24 prerequisites, composable-kernel changes, OpenMP/profiler integration and build resource requirements. It remains open. We keep the SDK tarball approach while the source-package migration is under review, without changing the repository's nixpkgs pins to an unmerged PR.

The TheRock environment guide requires the PHDR fix from patchelf #544 when rewriting kpack split ELF libraries. Nixpkgs' patchelf 0.19.2 includes that fix, but both 0.19.2 and the newer release 0.19.3 reject this SDK's librocsparse.so.1.0 with section content is not naturally aligned when extending its RPATH. The recipe therefore uses TheRock's pinned patchelf commit d0f70eea5397606c486857e0a105e53ec123904a, built locally with its test suite. It includes the PHDR fix. This override is confined to the SDK and exposed as therock-rocm.patchelf. The global patchelf package is unchanged. The recipe patches split libraries instead of retaining the old workaround that skipped late PHDR tables. SDK install checks inspect the HSA runtime's program headers and execute the compiler and HIP configuration tools. All top-level ELF tools in the SDK's compiler/bin directories are patched, including LLVM 24's llvm-offload-binary, which clang invokes as a subprocess. Its stock /lib64/ld-linux-x86-64.so.2 interpreter cannot run in a pure Nix build.

The upstream review also checked TheRock #6712 and the pin in the current manylinux Dockerfile: both select the same d0f70ee patchelf. A local build of release 0.19.3 passed 69 tests with 6 skipped, but still failed the exact same librocsparse RPATH rewrite that the TheRock pin passes. TheRock #2787 already added llvm-offload-binary to the compiler tool list; our archive contains it and only needs its interpreter adapted to Nix. The recent PyTorch wheel RPATH change #9515 addresses the wheel layout rather than this standalone SDK tarball.

To update, change the SDK version and obtain the flat archive hash with nix store prefetch-file URL --json. Keep the official target-specific tarball URL and pin the returned hash. Inspect share/therock/therock_manifest.json in the result as well as the archive name: the old 10.0.0 archive reported an rc4 package version internally, and the official stable 10.1.0 archive reports rc3.

Build nix build .#therock-rocm .#therock-rocm.tests.hip, then run nix run .#therock-rocm.tests.hip on a host with GPU access. The HIP test compiles through the SDK's Nix-aware compiler wrapper and checks vector addition and rocBLAS matrix multiplication on the device. Rebuild the four direct consumers and hipfire-kernels; also run nix run .#strixite.tests.gpu and the flake evaluation checks. Kernel checks do not replace model inference testing.