Commit Graph

5 Commits

Author SHA1 Message Date
alex 7bb05ea685 feat(strata): build the MMQ prompt path
CI / check (push) Has been cancelled
Strata 0.2.14 leaves llama.cpp's MMQ prompt kernels out by default, so
STRATA_MMQ_KQ does nothing and UD-Q5_K_XL's Q6_K layer runs on Strata's
int8 kernel. On ms-s1-max, 0.2.15 with the gfx1151 prompt set and
STRATA_MMQ_KQ=1 gives first-token logits bit-identical to 0.2.13 over
15 cold prompts at the same speed (~1180 tok/s at 16K); without MMQ an
agent prompt reached KL 0.199 against 0.2.13. The engine grows from 39
to 54 MB.

Bead: nix-overlay-481
2026-11-16 21:42:33 +03:00
alex b3287c926f feat(strata): bump 0.2.13 -> 0.2.15
Strata 0.2.14 runs the prompt on its own kernels and vendors llama.cpp
(third_party/strata-llama), so the separate llama.cpp fetch for the
vision encoder goes. The build keeps upstream's halo flags: MMQ stays
out (STRATA_MMQ defaults to OFF), 47 of 48 UD-Q5_K_XL layers run on the
fused kernels, and the ROCm 10.1 hipBLASLt table still ships.

UD-Q5_K_XL on ms-s1-max, 16K cold prompts: 715 -> ~810 tok/s on the
defaults, ~1180 tok/s with the gfx1151 prompt set (unchanged).

Bead: nix-overlay-ydg
2026-11-16 20:51:14 +03:00
alex d14f6d5e37 fix(strata): ship a hipBLASLt table calibrated for ROCm 10.1
Upstream's gfx1151-hipblaslt-100401.txt was calibrated on TheRock 7.14.1.
therock-rocm 10.1.0rc3 reports the same hipBLASLt version, so the engine
accepted the table, but its solution ids do not exist in that build and
every prompt GEMM fell back to the heuristic. The table now comes from
tools/hip/tune_hipblaslt run on gfx1151 against therock-rocm 10.1 and is
installed in bin/tuning/, where the engine finds it by itself.

UD-Q5_K_XL, 17.5K cold prompt on ms-s1-max: 418 -> 700-725 tok/s.

Bead: nix-overlay-2yv
2026-11-16 20:22:23 +03:00
alex f2abdc96f9 fix(rocm): Address package review findings
CI / check (push) Has been cancelled
2026-11-12 23:55:09 +03:00
alex a30c7d6c3a feat(strata): Add ROCm 10 build for gfx1151 2026-11-12 21:28:24 +03:00