# Build/packaging requirements: `sitbon/magg` Companion to `docs/research_magg_upstream.md` (upstream fact sheet). This file answers only the build/packaging questions an implementer needs answered before picking a build helper. Evidence snapshot (all verified, nothing inferred): | item | value | source | | --- | --- | --- | | upstream | , default branch `main`, head tree `dafa9c9ccbfc7209f7bd8e5f303d51598d19f59c` | `GET /repos/sitbon/magg/git/trees/main?recursive=1` (146 entries) | | latest release / PyPI | `v1.3.0` (published 2026-09-26) == PyPI `magg` 1.3.0 | `GET /repos/sitbon/magg/releases/latest`, `GET /pypi/magg/json` | | language | Python only (no Rust/Go/Node sources, no `Makefile`, no `Cargo.toml`, no `go.mod`, no `package.json` anywhere in the 146-entry tree) | git tree listing | | nixpkgs pin used for version checks | `nixos-unstable` rev `eaad089433ca2bb662274377d33df3d0e51ef28b` (== this overlay's `nixpkgs` **and** `nixpkgs-latest`, see `flake.lock` nodes `nixpkgs`/`nixpkgs-latest`) | `flake.lock` | --- ## 1. Which nixpkgs build helper **`python3Packages.buildPythonApplication` with `pyproject = true` and `build-system = [ hatchling ]`.** Evidence: - `pyproject.toml:120-122` — `[build-system] requires = ["hatchling"] / build-backend = "hatchling.build"`. The pin is unpinned (`"hatchling"`, no version specifier), so no `sed` pin-relax hack like `packages/graphify/package.nix:48-50` is needed. - Pure-python wheel: PyPI ships `magg-1.3.0-py3-none-any.whl` (62 files, no compiled extension of its own) and an sdist `magg-1.3.0.tar.gz` (121 entries). - Version is static in `pyproject.toml` (no `dynamic = ["version"]`, no `uv-dynamic-versioning`), so no `writableTmpDirAsHomeHook`/setuptools-scm dance is required. - None of the other helpers apply: no `Cargo.toml` (→ not `buildRustPackage`), no `package.json`/`bun.lock` (→ not `buildNpmPackage`/`bun2nix`), no `go.mod`/`go.sum` (→ not `buildGoModule`), no `Makefile` (→ not `stdenv.mkDerivation`). - In-repo precedent to copy verbatim: `packages/skillsmcp/package.nix:7-30` already does `python3Packages.buildPythonApplication` + `build-system = [ hatchling ]` + `python3Packages.fastmcp`. `packages/nftables-analyzer/package.nix` and `packages/freetoken/package.nix:82-104` are the same shape. Secondary option (deliberately **not** recommended): the repo commits `uv.lock`, so `uv2nix` / `pyproject-nix` (`pyproject.buildApplication` + `pyparsix`/`fetchPypiTree`) is technically possible. It buys nothing here because every dependency exists in nixpkgs (see §3), and the overlay has no `uv2nix` precedent. ## 2. Lockfiles - **Committed: `uv.lock`** (387 038 bytes at head, also included in the PyPI sdist). Header: `version = 1`, `revision = 3`, `requires-python = ">=3.12"`, 92 resolved `[[package]]` entries, all from `https://pypi.org/simple` with sdist+wheel hashes. - **Not present:** `Cargo.lock`, `package-lock.json`, `bun.lock`, `pnpm-lock.yaml`, `go.sum`, `requirements.txt`, `poetry.lock`. - Consequence: there is **no `cargoHash` / `npmDepsHash` / `vendorHash` equivalent to fill in** when using `buildPythonApplication` — dependencies come from `python3Packages`, so the only hash in the derivation is the `src` hash. The offline build is automatic (no network at build time). `uv.lock` would only matter for a `uv sync --locked`-based build (uv2nix route), where each of the 92 locked artifacts must be prefetched — another argument for the nixpkgs-deps route. ## 3. Toolchain version requirements Declared upstream: - `pyproject.toml:4` → `requires-python = ">=3.12"`; classifiers list 3.12 / 3.13 / 3.14. - `.python-version` → `3.14` (upstream's own preferred interpreter). - `.github/workflows/test.yml` matrix → `python-version: ['3.12', '3.13', '3.14']`, installed via `astral-sh/setup-uv@v8.2.0` + `uv python install`. - No Rust / Go / Node toolchain requirement. `nodejs`/`npm` appear only in `dockerfile:10` (`apk add … nodejs npm`) so that *runtime*-mounted `npx` MCP servers work inside the container image — not a build requirement. Fit against this overlay's pinned nixpkgs (`eaad0894…`): - `pkgs/top-level/all-packages.nix:4484` at that rev says `python3 = python314;`, and `pkgs/development/interpreters/python/cpython/` ships `3.11 … 3.15`. So the default `python3Packages` already matches upstream's `.python-version = 3.14`. **No `nixpkgs-latest`, `nixpkgs-rust195` or `nixpkgs-torch211` instance is needed.** Direct-dependency floors vs. what the pinned nixpkgs ships (versions read from `pkgs/development/python-modules//default.nix` at `eaad0894…`): | magg requirement (`pyproject.toml:31-42`) | nixpkgs attr | nixpkgs version | ok | | --- | --- | --- | --- | | `fastmcp>=3.4,<4` | `fastmcp` (thin wrapper over `fastmcp-slim`) | 3.4.7 | ✅ | | `mcp>=1.28` | `mcp` | 1.29.0 | ✅ | | `aiohttp>=3.12.13` | `aiohttp` | 3.14.3 | ✅ | | `art>=6.5` | `art` | 6.5 | ✅ (exactly at the floor) | | `cryptography>=45.0.4` | `cryptography` | 50.0.0 | ✅ | | `pydantic>=2.11.7` | `pydantic` | 2.13.4 | ✅ | | `pydantic-settings>=2.10.0` | `pydantic-settings` | 2.12.0 | ✅ | | `pyjwt>=2.10.1` | `pyjwt` | 2.13.0 | ✅ | | `prompt-toolkit>=3.0.51` | `prompt_toolkit` | 3.0.52 | ✅ | | `rich>=14.0.0` | `rich` | 15.0.0 | ✅ | | `watchdog>=6.0.0` | `watchdog` | 6.0.0 | ✅ (exactly at the floor) | | test extra: `pytest>=8.4`, `pytest-asyncio>=1.0` | `pytest`, `pytest-asyncio` | 9.1.1, 1.4.0 | ✅ | All 11 runtime deps + both test deps exist as `pkgs/development/python-modules//`. `fastmcp-slim` is present and its own `doCheck = pythonAtLeast "3.14"` confirms 3.14 support. **Bonus evidence that the nixpkgs dep set reproduces upstream's locked environment** — the versions `uv.lock` resolved on upstream's own machine are the same ones nixpkgs ships: | package | `uv.lock` @ head | nixpkgs `eaad0894…` | | --- | --- | --- | | pytest | 9.1.1 | 9.1.1 | | pytest-asyncio | 1.4.0 | 1.4.0 | | mcp | 1.29.0 | 1.29.0 | | rich | 15.0.0 | 15.0.0 | | aiohttp | 3.14.3 | 3.14.3 | | art | 6.5 | 6.5 | | watchdog | 6.0.0 | 6.0.0 | | fastmcp | 3.4.5 | 3.4.7 (only drift, still inside `<4`) | This is a good signal that `doCheck = true` is realistic rather than optimistic. Undeclared-but-imported modules to keep in mind (they arrive transitively via `fastmcp`/`mcp`/`rich`, so no extra `dependencies` entry is normally needed, but they explain a `pythonImportsCheck` failure if a transitive dep is ever cut): `httpx`, `pygments`, `readline`/`tomllib` (stdlib). ## 4. Pre-build steps / build-time network **None.** Verified: - No frontend/JS asset, no protobuf/gRPC/ANTLR codegen, no WASM, no `Makefile`, no `build.sh`, no asset download script in the tree. `scripts/` holds only `bump_version.py` and `validate_manual_release.py` (release tooling; CI uses the former to check `server.json` ↔ `pyproject.toml` version parity — irrelevant to the build). - Upstream's own build is `uv sync --locked` + `uv build`/`uv publish` (`.github/workflows/publish.yml:86-110`), i.e. plain hatchling. - Hatchling is a pure-python backend; with `dependencies` supplied from `python3Packages` the build is fully offline. Nothing to prefetch (no `cargoHash`-style extra fetch). - The only "asset" handling is hatchling copying non-`.py` files that live inside the `magg/` package directory (see §6). `[tool.hatch.build.targets.sdist]` (`pyproject.toml:100-119`) only affects the sdist, not the wheel. ## 5. Test suite & `doCheck` policy Upstream has a real suite: `test/magg/*.py` — 35 files, 328 `def test_*` (180 `async def`). CI (`.github/workflows/test.yml`) runs `uv sync --all-groups --locked` then `uv run pytest -v` (no `-m` filter) on 3.12/3.13/3.14; last run of that workflow: **success** (2026-09-26T16:28Z, `GET /actions/workflows/test.yml/runs`). No `conftest.py`; `[tool.pytest.ini_options]` (`pyproject.toml:86-89`) declares only the `integration` marker. Sandbox-safety audit of the suite: - **No outbound network in tests.** The discovery clients that *do* hit the network (`magg/discovery/search.py:32` `registry.modelcontextprotocol.io`, `:144` `glama.ai`, `:264` `api.github.com`, `:318` `registry.npmjs.org`) are exercised only through canned fixtures (`test/magg/test_registry_search.py:7-56` `SAMPLE_RESPONSE`) and `monkeypatch`ed methods (`test/magg/test_cli_fixes.py:118-124`). No `httpx`/`aiohttp`/`requests` call appears in `test/`. - **Loopback only.** The two e2e tests bind `localhost:54322` (hard-coded, `test/magg/test_e2e_simple.py:65,91`, `test/magg/test_e2e_mounting.py:129`) and poll for up to 10 s. Loopback works in the Nix sandbox, but the fixed port + wall-clock polling is the main flake risk. - **Subprocess tests need `python` on PATH**: `test/magg/test_integration.py:37` and `test/magg/test_mounting_real.py` spawn `python ` stdio MCP backends; `test/magg/test_server_robustness.py:374` and `test/magg/test_cli_fixes.py:140` run `sys.executable -m magg …`. `buildPythonPackage`'s `checkPhase` runs inside its venv, so `python` resolves and `magg` is importable — satisfied automatically. - **`$HOME` is read, not written**: `test/magg/test_config.py:95`, `test_auth.py:25`, `test_config_safety.py:76` assert `Path.home()/".magg"` / `Path.home()/".ssh"/"magg"` defaults; `test_cli_fixes.py:25` monkeypatches `HOME` to a tmp dir. Scratch space is `tempfile.TemporaryDirectory()` (→ `$TMPDIR`, writable in the sandbox). - **`sh` is required**: `test/magg/test_cli_fixes.py:78` runs `subprocess.run(["sh", "-c", …])` (bash provides `bin/sh` in the build env — fine). - Timing-sensitive groups: `test_config_reload.py` (watchdog/polling), `test_server_robustness.py` (hung-backend timeouts). Recommended policy (mirrors the `packages/marmel/package.nix:33-60` comment style — explain each workaround and what would let it be dropped): ```nix nativeCheckInputs = with python3Packages; [ pytestCheckHook pytest-asyncio ]; # Upstream runs the whole suite (`.github/workflows/test.yml`: `uv run pytest -v`), but the three # `integration`-marked tests (test/magg/test_basic.py:97, test_e2e_mounting.py:18, # test_e2e_simple.py:16) spawn real stdio/HTTP backends and hard-code localhost:54322 with a # 10 s poll loop (test_e2e_simple.py:65,91), which is the only flake source in an otherwise # network-free suite (registry/glama/github/npm clients are monkeypatched in the unit tests). # Drop this once upstream picks an ephemeral port. pytestOptions = [ "-m" "not integration" ]; preCheck = '' # pydantic-settings defaults read $HOME (magg/settings.py:72,203) and the config-reload tests # poll files under $TMPDIR; keep both inside the writable sandbox tmp dir. export MAGG_AUTO_RELOAD=false mkdir -p /tmp/magg-check export TMPDIR=/tmp/magg-check export HOME=/tmp/magg-check ''; pythonImportsCheck = [ "magg" ]; # import magg pulls fastmcp/mcp/aiohttp/cryptography ``` Notes for the implementer: - `pytest-asyncio` is mandatory in `nativeCheckInputs`: essentially every async test carries `@pytest.mark.asyncio` (178 marker lines for 180 `async def test_*` definitions) and pytest-asyncio defaults to strict mode, so unmarked async tests are only skipped. The 2 apparently unmarked ones are therefore silently skipped upstream too — do not add `-W error`. - Only the `[project.optional-dependencies] test` extra is needed; the `[dependency-groups] dev` group additionally pulls `keyring`, `anthropic`, `ruff` — do **not** add those. - If the suite turns out to be flaky in practice, the repo-wide fallback is `doCheck = false; pythonImportsCheck = [ "magg" ];` as in `packages/skillsmcp/package.nix:33-34`. ## 6. Install phase Standard wheel install — **no custom `installPhase` needed.** - The published wheel contains exactly `magg/**` + `magg-1.3.0.dist-info/` (62 files), so `buildPythonApplication` puts `$out/lib/python3.14/site-packages/magg/` plus two console scripts from `entry_points.txt`: `magg = magg.cli:main`, `mbro = magg.mbro.cli:main` (same as `pyproject.toml:65-67`) → `$out/bin/magg`, `$out/bin/mbro`. Set `mainProgram = "magg";`. - **Runtime-loaded data files live inside the package dir and must not be stripped**: `magg/contrib/magg.mbro`, `magg/contrib/readme.md`, `magg/contrib/kit.d/example.json` (confirmed present in the wheel). `magg/contrib` is a **namespace package with no `__init__.py`**; hatchling still ships it, and it is discovered at runtime via `magg/util/paths.py:12-15` (`contrib.__path__`), which feeds the `MAGG_PATH` search list at `magg/settings.py:203` (`./.magg`, `~/.magg`, contrib dirs) used for `config.json` and `kit.d/`. ⇒ Do not add `disallowedReReferences`/`find -delete` rules for `*.json`/`*.mbro`, and do not "fix" the missing `__init__.py`. - Everything else in the repo is *not* installed and must not be: `server.json` (MCP-registry manifest), `compose.yaml`, `dockerfile`, `docs/`, `examples/`, `scripts/`, and `etc/glama-mcp-api.json` (a static API dump — `grep -rn "glama-mcp-api" magg/` has no hits; the live code calls `https://glama.ai/api/mcp/v1/servers` instead). - No `share/` payloads, no shell completions, no man pages, no `py.typed`. - Hatchling needs `readme.md` and `license.md` present (`pyproject.toml:6-7`); both are in git and in the sdist. ## 7. Runtime dependencies - The Python interpreter wrapper comes from `buildPythonApplication` (shebang points at `python3`'s `$out`), plus the 11 declared python deps and their transitive closure (`openssl` via `cryptography`, `readline` — `magg/mbro/arepl.py` imports `readline`, and nixpkgs' CPython already carries it, `pcre2`/`multidict`/`yarl` C extensions from `aiohttp`). - **TLS trust store** for the search features: `magg/discovery/search.py:40` builds `aiohttp.ClientSession(trust_env=True)` against `registry.modelcontextprotocol.io`, `glama.ai`, `api.github.com`, `registry.npmjs.org`. nixpkgs' OpenSSL/CPython resolve the bundle from `SSL_CERT_FILE` / `/etc/ssl/certs`, so it works out of the box on NixOS; on non-NixOS hosts users need `SSL_CERT_FILE` (same class of issue documented in `packages/marmel/package.nix:33-45`). Adding `cacert` to the *runtime* wrapper is optional, not required. - **Writable state at runtime**: `CatalogManager` writes `/.magg/search_cache.json` (`magg/discovery/catalog.py:15-16`) and the config lives at `./.magg/config.json` or `~/.magg/config.json` (`magg/settings.py:203`, `MAGG_CONFIG_PATH`, `MAGG_PATH`, `MAGG_READ_ONLY`). Worth a `meta` note / NixOS-module hint, nothing build-time. - **Optional external programs** (only when a user mounts such a server): `npx`/`node`, `uvx`, `python` for stdio backends — spawned by `fastmcp`, not by magg itself (`grep -rn "subprocess|Popen" magg/` → no hits). Upstream's image installs `nodejs`/`npm` (`dockerfile:10`) for that reason. Not a build or hard runtime dep. - `magg --version` reads `importlib.metadata.version("magg")` (`magg/__init__.py:9-12`) → works because the dist-info is installed; no source-tree assumption. ## 8. Prior art **`magg` is not packaged in nixpkgs, and upstream ships no Nix expression.** - Complete (non-truncated) listing of `pkgs/development/python-modules` at `eaad0894…` → 9949 entries, none named `magg` (nearest neighbours in that listing: `magic`, `magic-filter`, `magicgui`, `magic-wormhole`, `magika`). - `pkgs/top-level/all-packages.nix` at that rev → `grep -i magg` returns nothing (so it is not a top-level tool either). - The magg repo (146-entry tree) contains no `.nix` file and no `flake.nix`; upstream's documented install paths are `uv tool install magg`, `uvx --from git+https://github.com/sitbon/magg.git magg`, `pip`/`poetry`, and GHCR Docker images. - Nothing reusable to mirror; write from scratch. Best in-repo template: `packages/skillsmcp/package.nix` (hatchling + `python3Packages.fastmcp`), with `packages/nftables-analyzer/package.nix` as the `fetchFromGitHub` + dated-version variant. ## 9. Update mechanism Use the plain form (repo-wide list style, cf. `packages/agnix/package.nix:34-38`): ```nix passthru.updateScript = [ "nix-update" "--flake" ".#magg" ]; ``` - Upstream **does** tag releases: `v1.3.0` (2026-09-26) matching PyPI `magg 1.3.0`, and `pyproject.toml` carries a static `version`, so nix-update's default version detection works — no `--version=branch=main` (that form is only used in this repo for untagged upstreams, e.g. `packages/ds4/package.nix:78`, `packages/skillsmcp/package.nix:39`). - **Caveat if you use `fetchFromGitHub` + `tag = "v${version}"`:** the full tag list (30 tags) also contains two-part alias tags — `v1.3`, `v1.2`, `v1.1`, `v0.10`, `v0.9`, `v0.8`, `v0.7`, `v0.6` — plus non-version tags `root`, `latest-release`, `latest-publish` and legacy `magg/v0.5.1.53`-style tags, so a tag-based bump can pick the `v1.3` alias (a valid tag, but an ugly version string) or a junk tag. Two clean options: 1. **Preferred:** build from PyPI — `src = fetchPypi { pname = "magg"; inherit version; format = "pyproject"; }`. The sdist is complete (121 entries: `magg/`, `test/`, `pyproject.toml`, `uv.lock`, `readme.md`, `license.md`), so `checkPhase` still works, and nix-update's PyPI updater resolves clean versions deterministically. Ready-made hash for 1.3.0: `sha256-2jzBtka4YWp4yEGvJorHN/JQpI6sWCrmRu/TtvRnqpU=` (sha256 `da3cc1b646b8616a78c841af268ac737f250a48eac582ae646efd3b6f467aa95`). 2. Keep `fetchFromGitHub` (rev = tag `v1.3.0`) and pin the updater with `--version=tag`, then verify the picked version is the 3-part tag. --- ### One-paragraph summary for the implementer `magg` is a pure-python hatchling project: use `python3Packages.buildPythonApplication` with `pyproject = true`, `build-system = [ python3Packages.hatchling ]`, and all 11 runtime deps taken from `python3Packages` (all present and new enough on the overlay's pinned `nixos-unstable`, whose default `python3` is 3.14 — exactly upstream's `.python-version`). No lockfile hash, no vendorHash, no pre-build codegen, no asset fetch, no custom install phase; two console scripts (`magg`, `mbro`) and the `magg/contrib/**` data files come along via the wheel. Run the test suite (`pytest` + `pytest-asyncio`) with `-m "not integration"` and a tmp `$HOME`/`$TMPDIR` in `preCheck`, keep `pythonImportsCheck = [ "magg" ]`, set `mainProgram = "magg"`, license `agpl3Plus` (`license = {text = "AGPL-3.0-or-later"}`), and update with plain `nix-update --flake .#magg` (prefer `fetchPypi` over `fetchFromGitHub` because upstream also publishes `v1.3`-style alias tags). MISSION COMPLETE (t-002)