feat(crucible): Add Strix Halo inference package
This commit is contained in:
@@ -44,6 +44,7 @@ A custom Nix overlay and flake providing additional packages not found in upstre
|
||||
| `xrt-amdxdna` | XRT with the amdxdna plugin, built against gufo's nixpkgs | AI Inference |
|
||||
| `therock-rocm` | TheRock 10.0 SDK for gfx1151 | AI Inference |
|
||||
| `strata` | Strata inference engine, API server and CPU vision encoder with ROCm 10 HIP support for gfx1151 | AI Inference |
|
||||
| `crucible-strix-halo` | Crucible EXL3 inference engine and OpenAI-compatible server for gfx1151 with the ROCm 10 runtime | AI Inference |
|
||||
| `pwilkin-llama-cpp` | pwilkin llama.cpp fork with ROCm for gfx1151 | AI Inference |
|
||||
| `strix-llama-cpp` | halo-box llama.cpp fork with Vulkan | AI Inference |
|
||||
| `loop` | Corporate messenger for your team | Communication |
|
||||
@@ -149,6 +150,24 @@ uses `exe`, `args`, `cwd`, `tokenizer` and `model_name`. Set `exe` to the packag
|
||||
The ROCm 7.14.1 hipBLASLt tuning table from the reference recipe is not enabled
|
||||
with ROCm 10; the engine falls back to hipBLAS when no matching table is supplied.
|
||||
|
||||
Crucible ships `crucible-server` (also `nix run .#crucible-strix-halo`) and
|
||||
`crucible-engine`. It uses upstream's release engine compiled with ROCm 7.2.4,
|
||||
patched to use this overlay's TheRock ROCm 10.0 runtime. Upstream requires the
|
||||
7.2.4 compiler for source builds. The Python server comes from the matching tag.
|
||||
The package targets the Ryzen AI Max+ 395 / Radeon 8060S (`gfx1151`) with 128 GB
|
||||
RAM. Supply a compatible Qwen3.8-Flash-Next EXL3 pack separately:
|
||||
|
||||
```bash
|
||||
nix run .#crucible-strix-halo -- /path/to/exl3-pack --ctx 32768
|
||||
```
|
||||
|
||||
Use `--help` for serving options; the default endpoint is `http://127.0.0.1:8790`.
|
||||
Automatic release checks are disabled by default; update through Nix. The
|
||||
upstream checkout-management CLI (`setup`, `update`, and service installation)
|
||||
is not included. Model downloads and conversation state belong outside the Nix
|
||||
store. See the upstream [setup guide](https://github.com/jaedync/crucible-strix-halo/blob/v0.37.0/docs/QUICKSTART.md)
|
||||
for GPU memory configuration and model requirements.
|
||||
|
||||
### nftablesbuilder Runtime Notes
|
||||
|
||||
`nftablesbuilder` is a web interface for managing nftables. It consists of a
|
||||
|
||||
Reference in New Issue
Block a user