feat(crucible): Add Strix Halo inference package

This commit is contained in:
2026-11-12 21:41:56 +03:00
parent a30c7d6c3a
commit 3d04ce81ad
5 changed files with 120 additions and 0 deletions
+19
View File
@@ -44,6 +44,7 @@ A custom Nix overlay and flake providing additional packages not found in upstre
| `xrt-amdxdna` | XRT with the amdxdna plugin, built against gufo's nixpkgs | AI Inference |
| `therock-rocm` | TheRock 10.0 SDK for gfx1151 | AI Inference |
| `strata` | Strata inference engine, API server and CPU vision encoder with ROCm 10 HIP support for gfx1151 | AI Inference |
| `crucible-strix-halo` | Crucible EXL3 inference engine and OpenAI-compatible server for gfx1151 with the ROCm 10 runtime | AI Inference |
| `pwilkin-llama-cpp` | pwilkin llama.cpp fork with ROCm for gfx1151 | AI Inference |
| `strix-llama-cpp` | halo-box llama.cpp fork with Vulkan | AI Inference |
| `loop` | Corporate messenger for your team | Communication |
@@ -149,6 +150,24 @@ uses `exe`, `args`, `cwd`, `tokenizer` and `model_name`. Set `exe` to the packag
The ROCm 7.14.1 hipBLASLt tuning table from the reference recipe is not enabled
with ROCm 10; the engine falls back to hipBLAS when no matching table is supplied.
Crucible ships `crucible-server` (also `nix run .#crucible-strix-halo`) and
`crucible-engine`. It uses upstream's release engine compiled with ROCm 7.2.4,
patched to use this overlay's TheRock ROCm 10.0 runtime. Upstream requires the
7.2.4 compiler for source builds. The Python server comes from the matching tag.
The package targets the Ryzen AI Max+ 395 / Radeon 8060S (`gfx1151`) with 128 GB
RAM. Supply a compatible Qwen3.8-Flash-Next EXL3 pack separately:
```bash
nix run .#crucible-strix-halo -- /path/to/exl3-pack --ctx 32768
```
Use `--help` for serving options; the default endpoint is `http://127.0.0.1:8790`.
Automatic release checks are disabled by default; update through Nix. The
upstream checkout-management CLI (`setup`, `update`, and service installation)
is not included. Model downloads and conversation state belong outside the Nix
store. See the upstream [setup guide](https://github.com/jaedync/crucible-strix-halo/blob/v0.37.0/docs/QUICKSTART.md)
for GPU memory configuration and model requirements.
### nftablesbuilder Runtime Notes
`nftablesbuilder` is a web interface for managing nftables. It consists of a