feat(packages): add freetoken 0.1.2 NVIDIA MoE inference runtime
Some checks failed
CI / check (push) Has been cancelled

Package FreeToken (FlashML-org/FreeToken) — edge-native MoE serving
engine with OpenAI/Anthropic-compatible APIs, serving the `ft` CLI.

- torch-bin 2.11 (CUDA 12.9 wheel build) satisfies the torch>=2.11,<2.12
  pin; extensions link the same cudaPackages.cuda_cudart via a synthetic
  CUDA_HOME (cudart headers + nvcc crt/ headers, no nvcc needed).
- flashlib==0.3.0 vendored from the PyPI wheel (Triton-only; freetoken
  never imports the CuTeDSL GEMM backends, so nvidia-cutlass-dsl is
  omitted via pythonRemoveDeps).
- apache-tvm-ffi overridden to the pinned 0.1.13.post3 with a vendored
  cython 3.3.0 (build needs >=3.2.8, nixpkgs has 3.2.4); pytest skipped
  (upstream runs pytest-xdist with GPU tests).
- Unfree (CUDA EULA) is self-scoped: the package re-imports nixpkgs with
  config.allowUnfree so no flake-level or user config change is needed.
- Optional accel extras (flashinfer/sglang-kernel) and the kernel-cache
  wheel are not packaged; runtime falls back to pure-Triton kernels.
- Verified: nix build .#freetoken, ft --version, python imports check,
  extension RPATHs.
This commit is contained in:
2026-08-30 20:51:15 +03:00
parent e827a10680
commit ab88b09ceb
5 changed files with 251 additions and 1 deletions

View File

@@ -18,6 +18,7 @@ A custom Nix overlay and flake providing additional packages not found in upstre
| `desloppify` | Multi-language codebase health scanner and technical debt tracker for AI agents | AI Coding Agents |
| `ds4` | DeepSeek 4 Flash and PRO local inference engine for ROCm (Strix Halo) | AI Inference |
| `freebuff` | The world's strongest free coding agent | AI Coding Agents |
| `freetoken` | Local MoE-offload LLM inference runtime with OpenAI- and Anthropic-compatible APIs | AI Inference |
| `graphify` | Turn any folder of code, docs, papers, images, or videos into a queryable knowledge graph | AI Coding Agents |
| `haivemind` | Multi-model AI consensus, aggregation, and fusion runner for popular AI CLIs | AI Coding Agents |
| `hipengine` | ROCm-native local LLM inference engine with torch-free runtime for AMD RDNA GPUs | AI Inference |
@@ -150,6 +151,7 @@ nix-overlay/
│ ├── ds4/ # DeepSeek V4 Flash/PRO inference engine (ROCm)
│ ├── flake-inputs/ # Utility for caching flake inputs
│ ├── freebuff/ # Free coding agent (Codebuff)
│ ├── freetoken/ # Local MoE inference runtime (NVIDIA CUDA)
│ ├── graphify/ # Knowledge graph generator for code folders
│ ├── haivemind/ # Multi-model AI consensus runner
│ ├── hipengine/ # ROCm-native LLM inference engine for AMD GPUs