Strata 0.2.14 leaves llama.cpp's MMQ prompt kernels out by default, so STRATA_MMQ_KQ does nothing and UD-Q5_K_XL's Q6_K layer runs on Strata's int8 kernel. On ms-s1-max, 0.2.15 with the gfx1151 prompt set and STRATA_MMQ_KQ=1 gives first-token logits bit-identical to 0.2.13 over 15 cold prompts at the same speed (~1180 tok/s at 16K); without MMQ an agent prompt reached KL 0.199 against 0.2.13. The engine grows from 39 to 54 MB. Bead: nix-overlay-481
This commit is contained in:
@@ -52,6 +52,9 @@ stdenv.mkDerivation (finalAttrs: {
|
||||
"-DSTRATA_ENABLE_HIP=ON"
|
||||
"-DSTRATA_ENABLE_CUDA=OFF"
|
||||
"-DSTRATA_PREFILL_MMQ=ON"
|
||||
# Off upstream since 0.2.14. With it, STRATA_MMQ_KQ=1 keeps UD-Q5_K_XL's
|
||||
# Q6_K layer on MMQ: the same logits as 0.2.13, at the same prompt speed.
|
||||
"-DSTRATA_MMQ=ON"
|
||||
"-DSTRATA_BUILD_TESTS=OFF"
|
||||
"-DSTRATA_PORTABLE=ON"
|
||||
"-DCMAKE_HIP_ARCHITECTURES=gfx1151"
|
||||
|
||||
Reference in New Issue
Block a user