feat(packages): add freetoken 0.1.2 NVIDIA MoE inference runtime
Some checks failed
CI / check (push) Has been cancelled
Some checks failed
CI / check (push) Has been cancelled
Package FreeToken (FlashML-org/FreeToken) — edge-native MoE serving engine with OpenAI/Anthropic-compatible APIs, serving the `ft` CLI. - torch-bin 2.11 (CUDA 12.9 wheel build) satisfies the torch>=2.11,<2.12 pin; extensions link the same cudaPackages.cuda_cudart via a synthetic CUDA_HOME (cudart headers + nvcc crt/ headers, no nvcc needed). - flashlib==0.3.0 vendored from the PyPI wheel (Triton-only; freetoken never imports the CuTeDSL GEMM backends, so nvidia-cutlass-dsl is omitted via pythonRemoveDeps). - apache-tvm-ffi overridden to the pinned 0.1.13.post3 with a vendored cython 3.3.0 (build needs >=3.2.8, nixpkgs has 3.2.4); pytest skipped (upstream runs pytest-xdist with GPU tests). - Unfree (CUDA EULA) is self-scoped: the package re-imports nixpkgs with config.allowUnfree so no flake-level or user config change is needed. - Optional accel extras (flashinfer/sglang-kernel) and the kernel-cache wheel are not packaged; runtime falls back to pure-Triton kernels. - Verified: nix build .#freetoken, ft --version, python imports check, extension RPATHs.
This commit is contained in:
@@ -18,6 +18,7 @@ A custom Nix overlay and flake providing additional packages not found in upstre
|
||||
| `desloppify` | Multi-language codebase health scanner and technical debt tracker for AI agents | AI Coding Agents |
|
||||
| `ds4` | DeepSeek 4 Flash and PRO local inference engine for ROCm (Strix Halo) | AI Inference |
|
||||
| `freebuff` | The world's strongest free coding agent | AI Coding Agents |
|
||||
| `freetoken` | Local MoE-offload LLM inference runtime with OpenAI- and Anthropic-compatible APIs | AI Inference |
|
||||
| `graphify` | Turn any folder of code, docs, papers, images, or videos into a queryable knowledge graph | AI Coding Agents |
|
||||
| `haivemind` | Multi-model AI consensus, aggregation, and fusion runner for popular AI CLIs | AI Coding Agents |
|
||||
| `hipengine` | ROCm-native local LLM inference engine with torch-free runtime for AMD RDNA GPUs | AI Inference |
|
||||
@@ -150,6 +151,7 @@ nix-overlay/
|
||||
│ ├── ds4/ # DeepSeek V4 Flash/PRO inference engine (ROCm)
|
||||
│ ├── flake-inputs/ # Utility for caching flake inputs
|
||||
│ ├── freebuff/ # Free coding agent (Codebuff)
|
||||
│ ├── freetoken/ # Local MoE inference runtime (NVIDIA CUDA)
|
||||
│ ├── graphify/ # Knowledge graph generator for code folders
|
||||
│ ├── haivemind/ # Multi-model AI consensus runner
|
||||
│ ├── hipengine/ # ROCm-native LLM inference engine for AMD GPUs
|
||||
|
||||
Reference in New Issue
Block a user