fix(voicestudio): run the backend from a Nix-built Python env

The app's first-run setup downloads uv's generic CPython into
~/.config/VoiceStudio/runtime, which NixOS cannot execute (stub-ld).

Do the same frozen `uv sync` at build time against upstream's nixpkgs
python, in a fixed-output derivation, and point the app at it through
OMNIVOICE_BACKEND_CMD so it no longer manages a runtime of its own.
The ctranslate2 executable-stack flag is cleared at build time with
upstream's own repair, since the store is read-only.

CPU torch only. Adds an import-probe check.

Refs: nix-overlay-2r3
This commit is contained in:
2026-11-20 20:26:10 +03:00
parent 5d96850f8a
commit fc657f4d55
4 changed files with 159 additions and 12 deletions
+1
View File
@@ -4,6 +4,7 @@
{"id":"nix-overlay-2gf","title":"Update all packages to latest versions","description":"## Why\\nUser request: update every package in the overlay to its latest upstream version.\\n\\n## What\\nBump version + src hash (and cargoHash for Rust) in each packages/<name>/package.nix, update README table if needed, verify builds.\\n\\n## Acceptance\\n- Every package builds with nix build .#<name>\\n- README reflects new versions\\n- Changes committed and pushed","notes":"All packages updated and built successfully. Commit 1f3f0f468bdd on local main. PUSH BLOCKED: git.millerson.name:22002 and DoltHub unreachable from this network (SSH connect timeout). Re-run git push + bd dolt push when network is back, then close.","status":"closed","priority":2,"issue_type":"task","assignee":"Alexander Miroshnichenko","owner":"alex@millerson.name","created_at":"2026-08-07T07:22:50Z","created_by":"Alexander Miroshnichenko","updated_at":"2026-08-07T13:17:34Z","closed_at":"2026-08-07T13:17:34Z","close_reason":"All packages updated to latest versions, verified building, pushed to origin","source_repo":".","compaction_level":0,"original_size":0}
{"id":"nix-overlay-2p0","title":"Verify kyojin 1.8.26 on remote builder or CI","description":"Run nix build .#kyojin .#kyojin.tests.cpu .#kyojin.tests.jit on a remote builder or CI only. This workstation lacks resources; user explicitly forbids local builds. Local evaluation previously reported 1.8.26; build was interrupted, success unconfirmed. Downloaded/built dependencies were removed (232 paths, 21.7 GiB).","status":"open","priority":2,"issue_type":"task","created_at":"2026-11-17T18:18:48.554328412Z","created_by":"minder","updated_at":"2026-11-17T18:18:48.554328412Z","source_repo":"nix-overlay","compaction_level":0,"original_size":0}
{"id":"nix-overlay-2q7","title":"Add opt-in direct writes for large Kyojin prefix segments","description":"Retain the standard torch checkpoint and ordinary reads, small records and storage admission. Let the Flash-Next agent write large KV segments with aligned O_DIRECT to reduce page-cache pressure on bcachefs. Preserve CRC, mmap restore, unsupported-filesystem fallback and error handling. Verify native launcher, fs behavior, repeated agent timing and CPU tests.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-18T22:28:42.144819734Z","created_by":"minder","updated_at":"2026-11-18T22:34:07.087715703Z","closed_at":"2026-11-18T22:34:07.085648146Z","close_reason":"Verified on native 128K save and fresh-process restore, strict CPU checks and repeated real agent continuations.","source_repo":"nix-overlay","compaction_level":0,"original_size":0,"comments":[{"id":22,"issue_id":"nix-overlay-2q7","author":"minder","text":"Scope packages/kyojin. Opt-in environment for KV segments >=64 MiB only, 2 MiB staging, ordinary torch ZIP and mmap loader. Validate source-identical candidate in a real private engine.","created_at":"2026-11-18T22:28:52Z"},{"id":23,"issue_id":"nix-overlay-2q7","author":"minder","text":"ms-s1-max, bcachefs. Strict 22 filesystem/queue tests passed. Packaged CPU suite: 547 passed, 189 skipped, 349 subtests passed. Both local overlay and independent source tree build /nix/store/7ahzb4yyqj6wgqxpvcn13cpzs71y4qx6-kyojin-1.8.26. Native process opened a 3.76 GiB segment with O_DIRECT, reported 4 KiB peak residency, and served three correct receipts from 131072 cached tokens without restaging. Fresh native process using the on-disk segment and deepest checkpoint restored all 131072 tokens and returned emerald-88, 212 fresh tokens, TTFT 3.455 s, total 4.09 s. ABBA 128K/500-token JSON powers-of-two snapshots: buffered TTFT 3.861/2.772 s and total 19.907/39.741 s, direct 1.496/1.465 and 13.194/13.086 s. Identical answer and reasoning hashes. Direct serialization itself 19.4/28.7 s vs buffered 6.7/25.9 s. Archive: /DATA/LLMs/benchmark_results/kyojin-checkpoint-io-20261119.","created_at":"2026-11-18T22:34:06Z"}]}
{"id":"nix-overlay-2r3","title":"voicestudio: run backend from a Nix-built Python env, not ~/.config runtime","description":"nix run .#voicestudio fails on NixOS: the app downloads uv's CPython into ~/.config/VoiceStudio/runtime (stub-ld). Build the locked Python env in the store and point the app at it via OMNIVOICE_BACKEND_CMD so nothing is downloaded into the home directory.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-20T14:16:02.684043167Z","created_by":"minder","updated_at":"2026-11-20T17:26:09.671430323Z","closed_at":"2026-11-20T17:26:09.670768064Z","close_reason":"Completed: backend runs from a Nix-built Python env via OMNIVOICE_BACKEND_CMD; e2e in scratch HOME reached stage=ready with no runtime download; FOD reproducible (--rebuild)","source_repo":"nix-overlay","compaction_level":0,"original_size":0}
{"id":"nix-overlay-2yv","title":"strata: ship hipBLASLt tuning table calibrated for therock ROCm 10.1","description":"Upstream gfx1151-hipblaslt-100401.txt was calibrated on TheRock 7.14.1; therock 10.1.0rc3 reports the same hipBLASLt 100401 but its solution ids differ (HCD_EXACT: solution -1), so the table is a no-op and prefill runs ~418 tok/s. Recalibrated with tune_hipblaslt on ms-s1-max (gfx1151, ROCm 10.1): 17.5K cold prompt UD-Q5_K_XL 418 -> ~720 tok/s. Ship it in $out/bin/tuning/ so the engine finds it by itself.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-16T15:49:56.904695144Z","created_by":"minder","updated_at":"2026-11-16T17:22:22.934929696Z","closed_at":"2026-11-16T17:22:22.934303630Z","close_reason":"table calibrated, shipped and measured","source_repo":"nix-overlay","compaction_level":0,"original_size":0,"comments":[{"id":8,"issue_id":"nix-overlay-2yv","author":"minder","text":"Built on ms-s1-max (nix build --store ssh-ng): /nix/store/vv7j7ffdhcpi804pzhximszz959impgm-strata-0.2.13 ships bin/tuning/gfx1151-hipblaslt-100401.txt (rows identical to the tune_hipblaslt output). UD-Q5_K_XL 17.5K cold prompt: 418 -> 700-725 tok/s with the table; HCD_EXACT solution -1 warning gone.","created_at":"2026-11-16T17:22:22Z"}]}
{"id":"nix-overlay-3c1","title":"Split oversized paged KV saves when two segments can respect the disk floor","description":"Kyojin on ms-s1-max: a single cold 200K prompt misses the SSD cache when a monolithic 5.87 GiB segment needs twice its bytes above the 46 GiB floor. Try an opt-in two-segment fallback with total cap admission, per-write floor rechecks, independent indexes and retry. Validate immediate and restarted real tool turns before enabling.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-19T06:57:35.016699443Z","created_by":"minder","updated_at":"2026-11-19T07:33:29.292246978Z","closed_at":"2026-11-19T07:33:29.291538581Z","close_reason":"Optional bounded segments and contiguous copies verified on native long-context inference, failed-write retry, scattered pages, cap and floor boundaries.","source_repo":"nix-overlay","compaction_level":0,"original_size":0,"comments":[{"id":24,"issue_id":"nix-overlay-3c1","author":"minder","text":"Work scope packages/kyojin new patch and package patch list. Keep normal single-file path if it fits; test refusal and partial failures. No external push.","created_at":"2026-11-19T06:57:35Z"},{"id":25,"issue_id":"nix-overlay-3c1","author":"minder","text":"Original two-part writer passed 27 CPU tests and one full native 200K restart, but a small free-space change prevented the second 400-page file in a repeat. Final candidate bounds split segments at 256 pages, uses contiguous copies only when the physical rows allow, and preserves the per-write gate. Hardware and full-suite results recorded in /DATA/LLMs/benchmark_results/kyojin-first-200k-20261119.","created_at":"2026-11-19T07:28:29Z"},{"id":26,"issue_id":"nix-overlay-3c1","author":"minder","text":"Final packaged writer stored 256+256+256+32 pages from one cold 204901-token initial HTTP request. Reload used exactly 204800 cached tokens and 187 new, answered opal-91, first token 5.162 s (control 152.599 s). Native staging 889 ms. Exact direct segment fd, minimum disk free 48.178 GiB at a 46 GiB floor. Strict suite 29 passed, original build red in 3 cases. Nix CPU suite 554 passed, 189 skipped, 349 subtests passed. Native and CPU build paths in bounded/package.path and bounded/tests.out.","created_at":"2026-11-19T07:33:29Z"}]}
{"id":"nix-overlay-3k7","title":"Bound cold prefill deferral beside a resident agent reply","description":"Opt-in Kyojin generator optimization on ms-s1-max. Give an older highly cached reply a short fixed opportunity to finish while a second fresh long prefill arrives. Preserve cold throughput, output budget, expiry and existing scheduler behavior. Compare native HTTP timings and exact tool outputs.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-19T07:49:09.998243985Z","created_by":"minder","updated_at":"2026-11-19T08:02:32.035285104Z","closed_at":"2026-11-19T08:02:32.031438908Z","close_reason":"Opt-in bounded scheduling verified with real warm agent replies, long output and deadline/bypass regressions.","source_repo":"nix-overlay","compaction_level":0,"original_size":0,"comments":[{"id":27,"issue_id":"nix-overlay-3k7","author":"minder","text":"File scope: packages/kyojin/package.nix, new warm-reply-grace.patch and tracker export. Private server and isolated CPU regression.","created_at":"2026-11-19T07:49:46Z"},{"id":28,"issue_id":"nix-overlay-3k7","author":"minder","text":"Native 200K resident/33378 fresh with 2 s grace, 4 HTTP runs: last content around 1.3 s, completion 3.896–4.010 s vs 24.222–24.578 s stock; no forced decode limit. The 257-token tool receipt was exact, the independent request started before its completion. Red old build (7 failed), green strict policy 7 passed; full CPU Nix build 561 passed, 189 skipped, 349 subtests. nixfmt and staged diff checks pass. Host policy and 128K/200K matrix tracked in nixos-config-7hss.","created_at":"2026-11-19T08:02:31Z"}]}
+1 -1
View File
@@ -58,7 +58,7 @@ A custom Nix overlay and flake providing additional packages not found in upstre
| `relay-free-llm` | RESTful API to route user prompts to various AI model providers with automatic failover and intent-based routing | AI LLM Gateway |
| `stakpak` | DevOps AI agent that generates infrastructure code, debugs Kubernetes, configures CI/CD, and automates deployments | AI Agents |
| `traycer` | Open-source AI orchestration app for agentic coding | AI Coding Agents |
| `voicestudio` | Fully-local voice studio — cloning, design, dubbing, dictation, transcription and audiobook creation in 646 languages | AI Audio |
| `voicestudio` | Fully-local voice studio — cloning, design, dubbing, dictation, transcription and audiobook creation in 646 languages; Python backend built by Nix (CPU torch), no runtime download on first launch | AI Audio |
## Usage
+73 -11
View File
@@ -3,21 +3,83 @@
inputs,
...
}:
# Upstream's flake builds the app against its own pinned nixpkgs (see the
# comment in flake.nix), so there is nothing to re-derive here. Only the first
# run touches the network: the backend bootstraps itself with uv into
# ~/.local/share, exactly like the published AppImage.
# Upstream builds the desktop package for x86_64-linux only.
let
package = inputs.voicestudio.packages.${pkgs.stdenv.hostPlatform.system}.default or null;
system = pkgs.stdenv.hostPlatform.system;
app = inputs.voicestudio.packages.${system}.default or null;
in
if package == null then
if app == null then
# Upstream builds the desktop package for x86_64-linux only.
{
meta.platforms = [ "x86_64-linux" ];
}
else
package.overrideAttrs (_: {
passthru = (package.passthru or { }) // {
category = "AI Audio";
let
# Python and its libraries come from the nixpkgs the app itself is built against, so
# the closure holds one glibc and one libstdc++: upstream's launcher already puts its
# gcc libstdc++ on LD_LIBRARY_PATH, which the backend inherits.
upstream = inputs.voicestudio.inputs.nixpkgs.legacyPackages.${system};
python = upstream.callPackage ./runtime.nix { src = inputs.voicestudio; };
resources = "${app}/libexec/electron/resources";
# The manylinux wheels dlopen libstdc++ and libz, which NixOS has no /usr/lib for.
libraryPath = upstream.lib.makeLibraryPath [
upstream.stdenv.cc.cc.lib
upstream.zlib
];
# The packaged backend command from electron/src/main/backend.ts, run from the store
# environment. The shell passes the port through OMNIVOICE_PORT.
# shortcut: no bytecode cache (-B, the store is read-only), so the heavy imports (torch and
# co.) take about 15 s longer on every launch; drop -B and set PYTHONPYCACHEPREFIX to a user
# cache dir if that hurts.
backend = upstream.writeShellApplication {
name = "voicestudio-backend";
# Opt-in engines build their own venvs with `uv venv --python 3.11`; ffmpeg and uv
# are already on PATH from upstream's launcher.
runtimeInputs = [ upstream.python311 ];
text = ''
export LD_LIBRARY_PATH="${libraryPath}''${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
# Never fetch an interpreter: uv's own CPython cannot run here.
export UV_PYTHON_DOWNLOADS=never UV_PYTHON_PREFERENCE=only-system
cd ${resources}
# -s -B: no user site, and the sources live in the read-only store.
exec ${python}/bin/python -s -B -m uvicorn main:app \
--app-dir backend --host 127.0.0.1 --port "''${OMNIVOICE_PORT:-3900}"
'';
};
})
# The app's own import probe (electron/src/main/runtime-import-probe.ts), run in the
# backend's environment: fails when a wheel cannot find a system library.
importProbe = upstream.runCommand "voicestudio-imports" { } ''
export HOME=$TMPDIR LD_LIBRARY_PATH="${libraryPath}"
cd ${resources}
${python}/bin/python -s -B -c "import fastapi, uvicorn, omnivoice, faster_whisper, sentencepiece, torch, torchaudio, torchvision"
touch $out
'';
in
# With OMNIVOICE_BACKEND_CMD set the shell treats the backend as external: it neither
# stages sources nor runs `uv sync` into ~/.config/VoiceStudio. --set-default keeps the
# variable usable as an override.
upstream.runCommand "voicestudio"
{
nativeBuildInputs = [ upstream.makeWrapper ];
meta = app.meta;
passthru = {
category = "AI Audio";
inherit backend python;
tests.imports = importProbe;
};
}
''
makeWrapper ${app}/bin/voicestudio $out/bin/voicestudio \
--set-default OMNIVOICE_BACKEND_CMD ${backend}/bin/voicestudio-backend
# The menu entry has to start this launcher, not the bare app.
cp -r ${app}/share $out/share
chmod -R u+w $out/share
substituteInPlace $out/share/applications/VoiceStudio.desktop \
--replace-fail "Exec=${app}/bin/voicestudio" "Exec=$out/bin/voicestudio"
''
+84
View File
@@ -0,0 +1,84 @@
{
lib,
runCommand,
cacert,
python311,
uv,
src,
}:
# The app's own setup runs `uv sync` into ~/.config/VoiceStudio with a CPython that uv
# downloads, and NixOS cannot execute that interpreter (stub-ld). This is the same frozen
# sync done at build time against nixpkgs' CPython, so the backend environment is a store
# path and the running app downloads no interpreter and no packages.
#
# The torch build is CPU: uv.lock keeps the "cpu" and "cuda" groups as conflicting
# alternatives, and the app itself picks CPU on a host without an NVIDIA driver.
let
version = (lib.importJSON "${src}/package.json").version;
# A fixed-output derivation may not reference store paths, so it holds site-packages
# only: no console scripts (their shebangs name the build directory) and no pyvenv.cfg
# (it names the interpreter). The venv below supplies those.
sitePackages =
runCommand "voicestudio-site-packages-${version}"
{
nativeBuildInputs = [
uv
python311
];
# Nix empties SSL_CERT_FILE in the sandbox and uv then trusts nothing.
SSL_CERT_FILE = "${cacert}/etc/ssl/certs/ca-bundle.crt";
outputHashMode = "recursive";
# Changes with uv.lock: build once, copy the hash from the mismatch error.
outputHash = "sha256-qnRSxapQBCuJa8DYtyrAywS/9US5Or54N7dcN3sdL8s=";
}
''
mkdir project
cd project
cp ${src}/pyproject.toml ${src}/uv.lock ${src}/README.md .
export HOME=$TMPDIR
export UV_CACHE_DIR=$TMPDIR/uv-cache
export UV_PROJECT_ENVIRONMENT=$TMPDIR/venv
export UV_PYTHON_DOWNLOADS=never
export UV_PYTHON_PREFERENCE=only-system
export UV_LINK_MODE=copy
# Nine locked packages are sdists, and uv fetches their build backends unpinned:
# a new setuptools release would change the wheels built from them, and this hash.
# The cutoff pins those backends. The project itself is not installed, the app
# ships its own sources.
uv sync --frozen --no-dev --no-default-groups --group cpu --no-install-project \
--python ${python311}/bin/python3.11 --exclude-newer 2026-11-12T00:00:00Z
mkdir -p $out
cp -r $TMPDIR/venv/lib/python3.11/site-packages/. $out/
# CTranslate2 4.4.0, the last release whisperx allows on Python 3.11, asks for an
# executable stack and current glibc refuses to dlopen it ("cannot enable executable
# stack"). The app repairs this in place at run time, which a read-only store cannot
# allow, so apply its own repair (backend/core/execstack.py) here. Fails the build
# if the libraries move.
python3.11 -B - $out <<'EOF'
import glob, sys
sys.path.insert(0, "${src}/backend")
from core.execstack import clear_execstack, has_execstack
libs = glob.glob(sys.argv[1] + "/ctranslate2*/libctranslate2*.so*")
assert libs, "no CTranslate2 libraries found"
for lib in libs:
clear_execstack(lib)
assert has_execstack(lib) is False, lib
EOF
# RECORD still lists the console scripts left behind, and their hashes depend on
# the build directory named in each shebang. Drop those lines so a different
# `sandbox-build-dir` cannot change this output.
find $out -name RECORD -exec sed -i '/^\.\.\/\.\.\/\.\.\/bin\//d' {} +
'';
in
runCommand "voicestudio-python-${version}" { passthru = { inherit sitePackages; }; } ''
${python311}/bin/python3.11 -m venv --without-pip $out
rm -r $out/lib/python3.11/site-packages
ln -s ${sitePackages} $out/lib/python3.11/site-packages
''