feat(strata): bump 0.2.13 -> 0.2.15
Strata 0.2.14 runs the prompt on its own kernels and vendors llama.cpp (third_party/strata-llama), so the separate llama.cpp fetch for the vision encoder goes. The build keeps upstream's halo flags: MMQ stays out (STRATA_MMQ defaults to OFF), 47 of 48 UD-Q5_K_XL layers run on the fused kernels, and the ROCm 10.1 hipBLASLt table still ships. UD-Q5_K_XL on ms-s1-max, 16K cold prompts: 715 -> ~810 tok/s on the defaults, ~1180 tok/s with the gfx1151 prompt set (unchanged). Bead: nix-overlay-ydg
This commit is contained in:
@@ -150,11 +150,14 @@ projectors are supplied separately.
|
||||
Run `nix run .#strata -- --help` for server options, or invoke the engine directly
|
||||
with `nix shell .#strata -c strata --help`. For inference, pass
|
||||
`strata-server --engine strata --config /path/to/strata.json`; the upstream
|
||||
[server configuration](https://github.com/Niko1221/Strata/blob/v0.2.13/serve/server.py)
|
||||
[server configuration](https://github.com/Niko1221/Strata/blob/v0.2.15/serve/server.py)
|
||||
uses `exe`, `args`, `cwd`, `tokenizer` and `model_name`. Set `exe` to the packaged
|
||||
`strata` executable and use writable model/log paths outside the Nix store.
|
||||
The ROCm 7.14.1 hipBLASLt tuning table from the reference recipe is not enabled
|
||||
with ROCm 10; the engine falls back to hipBLAS when no matching table is supplied.
|
||||
Upstream's gfx1151 hipBLASLt table was calibrated on TheRock 7.14.1; its solution
|
||||
ids do not exist in ROCm 10.1's hipBLASLt, which reports the same version. The
|
||||
package ships a table recalibrated against the shared ROCm 10.1 SDK in
|
||||
`bin/tuning/`, where the engine finds it without `STRATA_HIPBLASLT_TUNING`;
|
||||
recalibrate it (`tools/hip/tune_hipblaslt`) when the SDK changes.
|
||||
|
||||
Crucible ships `crucible-server` (also `nix run .#crucible-strix-halo`) and
|
||||
`crucible-engine`. Upstream built the release engine with ROCm 7.2.4 and tested
|
||||
|
||||
Reference in New Issue
Block a user