fix(voicestudio): run the backend from a Nix-built Python env
The app's first-run setup downloads uv's generic CPython into ~/.config/VoiceStudio/runtime, which NixOS cannot execute (stub-ld). Do the same frozen `uv sync` at build time against upstream's nixpkgs python, in a fixed-output derivation, and point the app at it through OMNIVOICE_BACKEND_CMD so it no longer manages a runtime of its own. The ctranslate2 executable-stack flag is cleared at build time with upstream's own repair, since the store is read-only. CPU torch only. Adds an import-probe check. Refs: nix-overlay-2r3
This commit is contained in:
@@ -4,6 +4,7 @@
|
||||
{"id":"nix-overlay-2gf","title":"Update all packages to latest versions","description":"## Why\\nUser request: update every package in the overlay to its latest upstream version.\\n\\n## What\\nBump version + src hash (and cargoHash for Rust) in each packages/<name>/package.nix, update README table if needed, verify builds.\\n\\n## Acceptance\\n- Every package builds with nix build .#<name>\\n- README reflects new versions\\n- Changes committed and pushed","notes":"All packages updated and built successfully. Commit 1f3f0f468bdd on local main. PUSH BLOCKED: git.millerson.name:22002 and DoltHub unreachable from this network (SSH connect timeout). Re-run git push + bd dolt push when network is back, then close.","status":"closed","priority":2,"issue_type":"task","assignee":"Alexander Miroshnichenko","owner":"alex@millerson.name","created_at":"2026-08-07T07:22:50Z","created_by":"Alexander Miroshnichenko","updated_at":"2026-08-07T13:17:34Z","closed_at":"2026-08-07T13:17:34Z","close_reason":"All packages updated to latest versions, verified building, pushed to origin","source_repo":".","compaction_level":0,"original_size":0}
|
||||
{"id":"nix-overlay-2p0","title":"Verify kyojin 1.8.26 on remote builder or CI","description":"Run nix build .#kyojin .#kyojin.tests.cpu .#kyojin.tests.jit on a remote builder or CI only. This workstation lacks resources; user explicitly forbids local builds. Local evaluation previously reported 1.8.26; build was interrupted, success unconfirmed. Downloaded/built dependencies were removed (232 paths, 21.7 GiB).","status":"open","priority":2,"issue_type":"task","created_at":"2026-11-17T18:18:48.554328412Z","created_by":"minder","updated_at":"2026-11-17T18:18:48.554328412Z","source_repo":"nix-overlay","compaction_level":0,"original_size":0}
|
||||
{"id":"nix-overlay-2q7","title":"Add opt-in direct writes for large Kyojin prefix segments","description":"Retain the standard torch checkpoint and ordinary reads, small records and storage admission. Let the Flash-Next agent write large KV segments with aligned O_DIRECT to reduce page-cache pressure on bcachefs. Preserve CRC, mmap restore, unsupported-filesystem fallback and error handling. Verify native launcher, fs behavior, repeated agent timing and CPU tests.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-18T22:28:42.144819734Z","created_by":"minder","updated_at":"2026-11-18T22:34:07.087715703Z","closed_at":"2026-11-18T22:34:07.085648146Z","close_reason":"Verified on native 128K save and fresh-process restore, strict CPU checks and repeated real agent continuations.","source_repo":"nix-overlay","compaction_level":0,"original_size":0,"comments":[{"id":22,"issue_id":"nix-overlay-2q7","author":"minder","text":"Scope packages/kyojin. Opt-in environment for KV segments >=64 MiB only, 2 MiB staging, ordinary torch ZIP and mmap loader. Validate source-identical candidate in a real private engine.","created_at":"2026-11-18T22:28:52Z"},{"id":23,"issue_id":"nix-overlay-2q7","author":"minder","text":"ms-s1-max, bcachefs. Strict 22 filesystem/queue tests passed. Packaged CPU suite: 547 passed, 189 skipped, 349 subtests passed. Both local overlay and independent source tree build /nix/store/7ahzb4yyqj6wgqxpvcn13cpzs71y4qx6-kyojin-1.8.26. Native process opened a 3.76 GiB segment with O_DIRECT, reported 4 KiB peak residency, and served three correct receipts from 131072 cached tokens without restaging. Fresh native process using the on-disk segment and deepest checkpoint restored all 131072 tokens and returned emerald-88, 212 fresh tokens, TTFT 3.455 s, total 4.09 s. ABBA 128K/500-token JSON powers-of-two snapshots: buffered TTFT 3.861/2.772 s and total 19.907/39.741 s, direct 1.496/1.465 and 13.194/13.086 s. Identical answer and reasoning hashes. Direct serialization itself 19.4/28.7 s vs buffered 6.7/25.9 s. Archive: /DATA/LLMs/benchmark_results/kyojin-checkpoint-io-20261119.","created_at":"2026-11-18T22:34:06Z"}]}
|
||||
{"id":"nix-overlay-2r3","title":"voicestudio: run backend from a Nix-built Python env, not ~/.config runtime","description":"nix run .#voicestudio fails on NixOS: the app downloads uv's CPython into ~/.config/VoiceStudio/runtime (stub-ld). Build the locked Python env in the store and point the app at it via OMNIVOICE_BACKEND_CMD so nothing is downloaded into the home directory.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-20T14:16:02.684043167Z","created_by":"minder","updated_at":"2026-11-20T17:26:09.671430323Z","closed_at":"2026-11-20T17:26:09.670768064Z","close_reason":"Completed: backend runs from a Nix-built Python env via OMNIVOICE_BACKEND_CMD; e2e in scratch HOME reached stage=ready with no runtime download; FOD reproducible (--rebuild)","source_repo":"nix-overlay","compaction_level":0,"original_size":0}
|
||||
{"id":"nix-overlay-2yv","title":"strata: ship hipBLASLt tuning table calibrated for therock ROCm 10.1","description":"Upstream gfx1151-hipblaslt-100401.txt was calibrated on TheRock 7.14.1; therock 10.1.0rc3 reports the same hipBLASLt 100401 but its solution ids differ (HCD_EXACT: solution -1), so the table is a no-op and prefill runs ~418 tok/s. Recalibrated with tune_hipblaslt on ms-s1-max (gfx1151, ROCm 10.1): 17.5K cold prompt UD-Q5_K_XL 418 -> ~720 tok/s. Ship it in $out/bin/tuning/ so the engine finds it by itself.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-16T15:49:56.904695144Z","created_by":"minder","updated_at":"2026-11-16T17:22:22.934929696Z","closed_at":"2026-11-16T17:22:22.934303630Z","close_reason":"table calibrated, shipped and measured","source_repo":"nix-overlay","compaction_level":0,"original_size":0,"comments":[{"id":8,"issue_id":"nix-overlay-2yv","author":"minder","text":"Built on ms-s1-max (nix build --store ssh-ng): /nix/store/vv7j7ffdhcpi804pzhximszz959impgm-strata-0.2.13 ships bin/tuning/gfx1151-hipblaslt-100401.txt (rows identical to the tune_hipblaslt output). UD-Q5_K_XL 17.5K cold prompt: 418 -> 700-725 tok/s with the table; HCD_EXACT solution -1 warning gone.","created_at":"2026-11-16T17:22:22Z"}]}
|
||||
{"id":"nix-overlay-3c1","title":"Split oversized paged KV saves when two segments can respect the disk floor","description":"Kyojin on ms-s1-max: a single cold 200K prompt misses the SSD cache when a monolithic 5.87 GiB segment needs twice its bytes above the 46 GiB floor. Try an opt-in two-segment fallback with total cap admission, per-write floor rechecks, independent indexes and retry. Validate immediate and restarted real tool turns before enabling.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-19T06:57:35.016699443Z","created_by":"minder","updated_at":"2026-11-19T07:33:29.292246978Z","closed_at":"2026-11-19T07:33:29.291538581Z","close_reason":"Optional bounded segments and contiguous copies verified on native long-context inference, failed-write retry, scattered pages, cap and floor boundaries.","source_repo":"nix-overlay","compaction_level":0,"original_size":0,"comments":[{"id":24,"issue_id":"nix-overlay-3c1","author":"minder","text":"Work scope packages/kyojin new patch and package patch list. Keep normal single-file path if it fits; test refusal and partial failures. No external push.","created_at":"2026-11-19T06:57:35Z"},{"id":25,"issue_id":"nix-overlay-3c1","author":"minder","text":"Original two-part writer passed 27 CPU tests and one full native 200K restart, but a small free-space change prevented the second 400-page file in a repeat. Final candidate bounds split segments at 256 pages, uses contiguous copies only when the physical rows allow, and preserves the per-write gate. Hardware and full-suite results recorded in /DATA/LLMs/benchmark_results/kyojin-first-200k-20261119.","created_at":"2026-11-19T07:28:29Z"},{"id":26,"issue_id":"nix-overlay-3c1","author":"minder","text":"Final packaged writer stored 256+256+256+32 pages from one cold 204901-token initial HTTP request. Reload used exactly 204800 cached tokens and 187 new, answered opal-91, first token 5.162 s (control 152.599 s). Native staging 889 ms. Exact direct segment fd, minimum disk free 48.178 GiB at a 46 GiB floor. Strict suite 29 passed, original build red in 3 cases. Nix CPU suite 554 passed, 189 skipped, 349 subtests passed. Native and CPU build paths in bounded/package.path and bounded/tests.out.","created_at":"2026-11-19T07:33:29Z"}]}
|
||||
{"id":"nix-overlay-3k7","title":"Bound cold prefill deferral beside a resident agent reply","description":"Opt-in Kyojin generator optimization on ms-s1-max. Give an older highly cached reply a short fixed opportunity to finish while a second fresh long prefill arrives. Preserve cold throughput, output budget, expiry and existing scheduler behavior. Compare native HTTP timings and exact tool outputs.","status":"closed","priority":2,"issue_type":"task","assignee":"minder","created_at":"2026-11-19T07:49:09.998243985Z","created_by":"minder","updated_at":"2026-11-19T08:02:32.035285104Z","closed_at":"2026-11-19T08:02:32.031438908Z","close_reason":"Opt-in bounded scheduling verified with real warm agent replies, long output and deadline/bypass regressions.","source_repo":"nix-overlay","compaction_level":0,"original_size":0,"comments":[{"id":27,"issue_id":"nix-overlay-3k7","author":"minder","text":"File scope: packages/kyojin/package.nix, new warm-reply-grace.patch and tracker export. Private server and isolated CPU regression.","created_at":"2026-11-19T07:49:46Z"},{"id":28,"issue_id":"nix-overlay-3k7","author":"minder","text":"Native 200K resident/33378 fresh with 2 s grace, 4 HTTP runs: last content around 1.3 s, completion 3.896–4.010 s vs 24.222–24.578 s stock; no forced decode limit. The 257-token tool receipt was exact, the independent request started before its completion. Red old build (7 failed), green strict policy 7 passed; full CPU Nix build 561 passed, 189 skipped, 349 subtests. nixfmt and staged diff checks pass. Host policy and 128K/200K matrix tracked in nixos-config-7hss.","created_at":"2026-11-19T08:02:31Z"}]}
|
||||
|
||||
@@ -58,7 +58,7 @@ A custom Nix overlay and flake providing additional packages not found in upstre
|
||||
| `relay-free-llm` | RESTful API to route user prompts to various AI model providers with automatic failover and intent-based routing | AI LLM Gateway |
|
||||
| `stakpak` | DevOps AI agent that generates infrastructure code, debugs Kubernetes, configures CI/CD, and automates deployments | AI Agents |
|
||||
| `traycer` | Open-source AI orchestration app for agentic coding | AI Coding Agents |
|
||||
| `voicestudio` | Fully-local voice studio — cloning, design, dubbing, dictation, transcription and audiobook creation in 646 languages | AI Audio |
|
||||
| `voicestudio` | Fully-local voice studio — cloning, design, dubbing, dictation, transcription and audiobook creation in 646 languages; Python backend built by Nix (CPU torch), no runtime download on first launch | AI Audio |
|
||||
|
||||
## Usage
|
||||
|
||||
|
||||
@@ -3,21 +3,83 @@
|
||||
inputs,
|
||||
...
|
||||
}:
|
||||
# Upstream's flake builds the app against its own pinned nixpkgs (see the
|
||||
# comment in flake.nix), so there is nothing to re-derive here. Only the first
|
||||
# run touches the network: the backend bootstraps itself with uv into
|
||||
# ~/.local/share, exactly like the published AppImage.
|
||||
# Upstream builds the desktop package for x86_64-linux only.
|
||||
let
|
||||
package = inputs.voicestudio.packages.${pkgs.stdenv.hostPlatform.system}.default or null;
|
||||
system = pkgs.stdenv.hostPlatform.system;
|
||||
app = inputs.voicestudio.packages.${system}.default or null;
|
||||
in
|
||||
if package == null then
|
||||
if app == null then
|
||||
# Upstream builds the desktop package for x86_64-linux only.
|
||||
{
|
||||
meta.platforms = [ "x86_64-linux" ];
|
||||
}
|
||||
else
|
||||
package.overrideAttrs (_: {
|
||||
passthru = (package.passthru or { }) // {
|
||||
category = "AI Audio";
|
||||
let
|
||||
# Python and its libraries come from the nixpkgs the app itself is built against, so
|
||||
# the closure holds one glibc and one libstdc++: upstream's launcher already puts its
|
||||
# gcc libstdc++ on LD_LIBRARY_PATH, which the backend inherits.
|
||||
upstream = inputs.voicestudio.inputs.nixpkgs.legacyPackages.${system};
|
||||
|
||||
python = upstream.callPackage ./runtime.nix { src = inputs.voicestudio; };
|
||||
|
||||
resources = "${app}/libexec/electron/resources";
|
||||
|
||||
# The manylinux wheels dlopen libstdc++ and libz, which NixOS has no /usr/lib for.
|
||||
libraryPath = upstream.lib.makeLibraryPath [
|
||||
upstream.stdenv.cc.cc.lib
|
||||
upstream.zlib
|
||||
];
|
||||
|
||||
# The packaged backend command from electron/src/main/backend.ts, run from the store
|
||||
# environment. The shell passes the port through OMNIVOICE_PORT.
|
||||
# shortcut: no bytecode cache (-B, the store is read-only), so the heavy imports (torch and
|
||||
# co.) take about 15 s longer on every launch; drop -B and set PYTHONPYCACHEPREFIX to a user
|
||||
# cache dir if that hurts.
|
||||
backend = upstream.writeShellApplication {
|
||||
name = "voicestudio-backend";
|
||||
# Opt-in engines build their own venvs with `uv venv --python 3.11`; ffmpeg and uv
|
||||
# are already on PATH from upstream's launcher.
|
||||
runtimeInputs = [ upstream.python311 ];
|
||||
text = ''
|
||||
export LD_LIBRARY_PATH="${libraryPath}''${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
|
||||
# Never fetch an interpreter: uv's own CPython cannot run here.
|
||||
export UV_PYTHON_DOWNLOADS=never UV_PYTHON_PREFERENCE=only-system
|
||||
|
||||
cd ${resources}
|
||||
# -s -B: no user site, and the sources live in the read-only store.
|
||||
exec ${python}/bin/python -s -B -m uvicorn main:app \
|
||||
--app-dir backend --host 127.0.0.1 --port "''${OMNIVOICE_PORT:-3900}"
|
||||
'';
|
||||
};
|
||||
})
|
||||
|
||||
# The app's own import probe (electron/src/main/runtime-import-probe.ts), run in the
|
||||
# backend's environment: fails when a wheel cannot find a system library.
|
||||
importProbe = upstream.runCommand "voicestudio-imports" { } ''
|
||||
export HOME=$TMPDIR LD_LIBRARY_PATH="${libraryPath}"
|
||||
cd ${resources}
|
||||
${python}/bin/python -s -B -c "import fastapi, uvicorn, omnivoice, faster_whisper, sentencepiece, torch, torchaudio, torchvision"
|
||||
touch $out
|
||||
'';
|
||||
in
|
||||
# With OMNIVOICE_BACKEND_CMD set the shell treats the backend as external: it neither
|
||||
# stages sources nor runs `uv sync` into ~/.config/VoiceStudio. --set-default keeps the
|
||||
# variable usable as an override.
|
||||
upstream.runCommand "voicestudio"
|
||||
{
|
||||
nativeBuildInputs = [ upstream.makeWrapper ];
|
||||
meta = app.meta;
|
||||
passthru = {
|
||||
category = "AI Audio";
|
||||
inherit backend python;
|
||||
tests.imports = importProbe;
|
||||
};
|
||||
}
|
||||
''
|
||||
makeWrapper ${app}/bin/voicestudio $out/bin/voicestudio \
|
||||
--set-default OMNIVOICE_BACKEND_CMD ${backend}/bin/voicestudio-backend
|
||||
|
||||
# The menu entry has to start this launcher, not the bare app.
|
||||
cp -r ${app}/share $out/share
|
||||
chmod -R u+w $out/share
|
||||
substituteInPlace $out/share/applications/VoiceStudio.desktop \
|
||||
--replace-fail "Exec=${app}/bin/voicestudio" "Exec=$out/bin/voicestudio"
|
||||
''
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
{
|
||||
lib,
|
||||
runCommand,
|
||||
cacert,
|
||||
python311,
|
||||
uv,
|
||||
src,
|
||||
}:
|
||||
# The app's own setup runs `uv sync` into ~/.config/VoiceStudio with a CPython that uv
|
||||
# downloads, and NixOS cannot execute that interpreter (stub-ld). This is the same frozen
|
||||
# sync done at build time against nixpkgs' CPython, so the backend environment is a store
|
||||
# path and the running app downloads no interpreter and no packages.
|
||||
#
|
||||
# The torch build is CPU: uv.lock keeps the "cpu" and "cuda" groups as conflicting
|
||||
# alternatives, and the app itself picks CPU on a host without an NVIDIA driver.
|
||||
let
|
||||
version = (lib.importJSON "${src}/package.json").version;
|
||||
|
||||
# A fixed-output derivation may not reference store paths, so it holds site-packages
|
||||
# only: no console scripts (their shebangs name the build directory) and no pyvenv.cfg
|
||||
# (it names the interpreter). The venv below supplies those.
|
||||
sitePackages =
|
||||
runCommand "voicestudio-site-packages-${version}"
|
||||
{
|
||||
nativeBuildInputs = [
|
||||
uv
|
||||
python311
|
||||
];
|
||||
# Nix empties SSL_CERT_FILE in the sandbox and uv then trusts nothing.
|
||||
SSL_CERT_FILE = "${cacert}/etc/ssl/certs/ca-bundle.crt";
|
||||
outputHashMode = "recursive";
|
||||
# Changes with uv.lock: build once, copy the hash from the mismatch error.
|
||||
outputHash = "sha256-qnRSxapQBCuJa8DYtyrAywS/9US5Or54N7dcN3sdL8s=";
|
||||
}
|
||||
''
|
||||
mkdir project
|
||||
cd project
|
||||
cp ${src}/pyproject.toml ${src}/uv.lock ${src}/README.md .
|
||||
|
||||
export HOME=$TMPDIR
|
||||
export UV_CACHE_DIR=$TMPDIR/uv-cache
|
||||
export UV_PROJECT_ENVIRONMENT=$TMPDIR/venv
|
||||
export UV_PYTHON_DOWNLOADS=never
|
||||
export UV_PYTHON_PREFERENCE=only-system
|
||||
export UV_LINK_MODE=copy
|
||||
|
||||
# Nine locked packages are sdists, and uv fetches their build backends unpinned:
|
||||
# a new setuptools release would change the wheels built from them, and this hash.
|
||||
# The cutoff pins those backends. The project itself is not installed, the app
|
||||
# ships its own sources.
|
||||
uv sync --frozen --no-dev --no-default-groups --group cpu --no-install-project \
|
||||
--python ${python311}/bin/python3.11 --exclude-newer 2026-11-12T00:00:00Z
|
||||
|
||||
mkdir -p $out
|
||||
cp -r $TMPDIR/venv/lib/python3.11/site-packages/. $out/
|
||||
|
||||
# CTranslate2 4.4.0, the last release whisperx allows on Python 3.11, asks for an
|
||||
# executable stack and current glibc refuses to dlopen it ("cannot enable executable
|
||||
# stack"). The app repairs this in place at run time, which a read-only store cannot
|
||||
# allow, so apply its own repair (backend/core/execstack.py) here. Fails the build
|
||||
# if the libraries move.
|
||||
python3.11 -B - $out <<'EOF'
|
||||
import glob, sys
|
||||
sys.path.insert(0, "${src}/backend")
|
||||
from core.execstack import clear_execstack, has_execstack
|
||||
|
||||
libs = glob.glob(sys.argv[1] + "/ctranslate2*/libctranslate2*.so*")
|
||||
assert libs, "no CTranslate2 libraries found"
|
||||
for lib in libs:
|
||||
clear_execstack(lib)
|
||||
assert has_execstack(lib) is False, lib
|
||||
EOF
|
||||
|
||||
# RECORD still lists the console scripts left behind, and their hashes depend on
|
||||
# the build directory named in each shebang. Drop those lines so a different
|
||||
# `sandbox-build-dir` cannot change this output.
|
||||
find $out -name RECORD -exec sed -i '/^\.\.\/\.\.\/\.\.\/bin\//d' {} +
|
||||
'';
|
||||
in
|
||||
runCommand "voicestudio-python-${version}" { passthru = { inherit sitePackages; }; } ''
|
||||
${python311}/bin/python3.11 -m venv --without-pip $out
|
||||
rm -r $out/lib/python3.11/site-packages
|
||||
ln -s ${sitePackages} $out/lib/python3.11/site-packages
|
||||
''
|
||||
Reference in New Issue
Block a user