Files
millerson-overlay.nix/docs/research_magg_upstream.md
T
alex 44e6d90a77
CI / check (push) Has been cancelled
feat(magg): add magg 1.3.0 MCP aggregator package
Add magg, an MCP aggregator / meta-MCP server, to the overlay. It is a
Python package on the hatchling backend, built with
python3Packages.buildPythonApplication from the v1.3.0 source tarball,
so no upstream release artifacts are fetched. Ships both console
scripts declared upstream: magg (the aggregator CLI/server) and mbro
(the MCP browser). License is AGPL-3.0-or-later.

The install check runs the upstream pytest suite with
-m "not integration", because the integration-marked tests need a live
server. A forced nix build .#magg --rebuild passes with 329 passed,
3 skipped, 3 deselected, 0 failed, and both binaries smoke-test clean.

Also adds the README and AGENTS.md package rows, the upstream and build
research notes, and the verification report in docs/magg_build_report.md.
2026-11-08 12:48:30 +03:00

344 lines
18 KiB
Markdown

# Upstream fact sheet: `sitbon/magg`
Task: t-001 — upstream research for a Nix derivation.
Researched: 2026-11-08 (all facts fetched live from GitHub/PyPI on that date).
**Headline:** The repository **exists and is public**: <https://github.com/sitbon/magg>
(GitHub repo id `1005453886`, `private: false`, `archived: false`, 143 stars, 25 forks).
It is a **pure-Python, PEP 621 / hatchling / uv** project published to PyPI as `magg`,
with GitHub Releases + git tags. Latest release: **v1.3.0**.
---
## 1. What is `magg`?
**Short description for `meta.description`** (129 chars):
```
MCP aggregator server that manages, aggregates and proxies other MCP servers so LLMs can discover, add and manage tools at runtime
```
Alternatives (all sourced verbatim from upstream):
| Source | Text |
| --- | --- |
| GitHub repo `description` | `Magg: The MCP Aggregator` |
| `pyproject.toml` → `project.description` | `MCP Aggregator` |
| `server.json` → `description` | `Aggregates and proxies other MCP servers, letting LLMs discover, add, and manage tools at runtime.` |
| `readme.md` (line 12, intro) | `A Model Context Protocol server that manages, aggregates, and proxies other MCP servers, enabling LLMs to dynamically extend their own capabilities.` |
| `readme.md` (heading) | `🧲 Magg - The MCP Aggregator` |
**Longer explanation (upstream's own words, `readme.md` § "What is Magg?"):**
Magg is a *meta-MCP server* acting as a central hub for managing multiple MCP servers. It exposes
MCP tools that let an LLM search for new MCP servers online, add/configure them dynamically,
enable/disable them, aggregate their tools under unified prefixes, and persist configuration
across sessions (`.magg/config.json`). Upstream describes it as "a 'package manager for LLM
tools' - it lets AI assistants install and manage their own capabilities at runtime". It supports
stdio / HTTP / hybrid transports, optional RSA-JWT bearer auth for HTTP mode, dynamic config
reload (file watch + SIGHUP), "kits" (bundles of servers), and ships a second CLI, `mbro`
("MCP Browser") for interactive inspection.
Evidence: `https://raw.githubusercontent.com/sitbon/magg/v1.3.0/readme.md`,
`https://raw.githubusercontent.com/sitbon/magg/v1.3.0/server.json`,
`https://api.github.com/repos/sitbon/magg`.
---
## 2. Implementation language & build system
- **Language: Python only.** GitHub language breakdown returns exactly
`{"Python": 739242}` (100 %). Source tree contains only `*.py` + markdown/JSON.
Evidence: `https://api.github.com/repos/sitbon/magg/languages`.
- **Build system: PEP 621 `pyproject.toml` with the `hatchling` build backend**, project managed
with **uv** (a committed `uv.lock` and `.python-version` are present). There is **no**
`Cargo.toml`, `package.json`, `go.mod`, `Makefile`, `justfile`, or `setup.py`.
Evidence: root listing of tag `v1.3.0` via
`https://api.github.com/repos/sitbon/magg/contents/?ref=v1.3.0` and
`https://raw.githubusercontent.com/sitbon/magg/v1.3.0/pyproject.toml`.
Root of `v1.3.0` (full listing):
`.env.example`, `.gitignore`, `.python-version`, `compose.yaml`, `contributing.md`, `dockerfile`,
`license.md`, `pyproject.toml`, `readme.md`, `security.md`, `server.json`, `uv.lock`,
plus dirs `.github/`, `docs/`, `etc/`, `examples/`, `magg/`, `scripts/`, `test/`.
Key manifest facts (`pyproject.toml` @ v1.3.0):
```toml
[project]
name = "magg"
version = "1.3.0"
requires-python = ">=3.12"
description = "MCP Aggregator"
license = {text = "AGPL-3.0-or-later"}
license-files = ["license.md"]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project.optional-dependencies]
test = ["pytest>=8.4.0", "pytest-asyncio>=1.0.0"]
[tool.pytest.ini_options]
markers = ["integration: marks tests as integration tests (deselect with '-m \"not integration\"')"]
```
Runtime dependencies (`[project.dependencies]`, v1.3.0):
`fastmcp>=3.4,<4`, `mcp>=1.28`, `aiohttp>=3.12.13`, `pydantic>=2.11.7`,
`pydantic-settings>=2.10.0`, `rich>=14.0.0`, `prompt-toolkit>=3.0.51`,
`cryptography>=45.0.4`, `pyjwt>=2.10.1`, `watchdog>=6.0.0`, `art>=6.5`.
Python support: `requires-python = ">=3.12"`; classifiers list CPython 3.12 / 3.13 / 3.14
(README prose says "Python 3.12 or higher (3.13+ recommended)"; `.python-version` pins `3.14`
for upstream development).
---
## 3. Versioning & release state
**Yes — upstream publishes both GitHub Releases and annotated git tags**, and also publishes to
PyPI (`https://pypi.org/project/magg/`, 23 releases; PyPI latest = `1.3.0`, uploaded
`2026-09-26T16:29:42Z`, i.e. in lock-step with the GitHub release).
**Latest release: `v1.3.0`**
| Field | Value | Evidence |
| --- | --- | --- |
| Release tag name | `v1.3.0` (`🧲 Magg Release v1.3.0`) | `https://api.github.com/repos/sitbon/magg/releases/latest` |
| Tag object (annotated) | `cc9963ea784fcc2e465c318ab9f0f24e5059c1d1` (`type: tag`) | `.../git/refs/tags/v1.3.0` |
| **Peeled commit SHA** | **`1b18b87fdcda9628381888aea25fefa60fe1f923`** | `.../commits/1b18b87f…` |
| Commit date | `2026-09-26T16:28:09Z` (authored `2026-09-26 09:28:09 -0700`) — "Robustness fixes: config data loss, unmounting, hung backends, CLI and mbro bugs (#21)" | GitHub API / `git log` |
| Release created / published | `2026-09-26T16:29:30Z` / `2026-09-26T16:29:41Z`; `prerelease: false`, `draft: false`, `target_commitish: main` | `.../releases/tags/v1.3.0` |
| Release assets | `magg-1.3.0-py3-none-any.whl` (143 372 B), `magg-1.3.0.tar.gz` (340 217 B) | same |
| Release author | `github-actions[bot]` (automated release on tag) | same |
Release-asset hashes (computed locally from the GitHub release downloads):
```
sha256 magg-1.3.0.tar.gz da3cc1b646b8616a78c841af268ac737f250a48eac582ae646efd3b6f467aa95
sha256 magg-1.3.0-py3-none-any.whl 6360f04b49fdce6d8ec48aa6865848f8a610a60a3e433fcd539383299958c933
```
Tag hygiene (from `https://api.github.com/repos/sitbon/magg/tags?per_page=15`): upstream emits
both a patch tag and a minor tag per release, both pointing at the same commit —
`v1.3.0` **and** `v1.3` → `1b18b87f…`; `v1.2.1` → `419e8e07…`; `v1.2.0` → `5af7cad8…`;
`v1.2` → `419e8e07…`; `v1.1.0`/`v1.1` → `79923f85…`; `v0.10.1`/`v0.10` → `c237d4c1…`; etc.
A moving tag **`latest-publish`** also points at `1b18b87f…` (marks the last PyPI publish).
⇒ **Use the dotted tag `v1.3.0`** for `src.version`/`rev`.
Default branch / HEAD (in case `main` is preferred over the tag):
- Default branch: **`main`**
- HEAD: **`dafa9c9ccbfc7209f7bd8e5f303d51598d19f59c`**, committed `2026-09-27T19:05:55Z`
("Add security policy"). Verified as a descendant of `v1.3.0` (the only extra commit adds
`security.md`), so **`v1.3.0` is the right packaging rev**.
- Other branches: `beta` (`774636eb…`, feeds the `ghcr.io/sitbon/magg:beta` image), plus two
stale `claude/*` branches.
- Repo `pushed_at`: `2026-09-27T19:05:55Z`.
---
## 4. License
**SPDX: `AGPL-3.0-or-later`** → Nix: `meta.licenses = lib.licenses.agpl3Plus;`
Evidence:
- `pyproject.toml`: `license = {text = "AGPL-3.0-or-later"}`, `license-files = ["license.md"]`,
classifier `"License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)"`.
- `license.md` (34 260 B) is the verbatim **GNU AFFERO GENERAL PUBLIC LICENSE Version 3,
19 November 2007** text (incl. the §13 "remote network interaction" clause).
- PyPI metadata: `License: AGPL-3.0-or-later`; wheel `METADATA` has
`License: AGPL-3.0-or-later` + `License-File: license.md`.
- GitHub API reports `spdx_id: "AGPL-3.0"` (GitHub's coarse bucket; the manifest is more precise
with `-or-later`).
- Dockerfile OCI label: `org.opencontainers.image.licenses=AGPL-3.0-or-later`.
**Flag:** *not* dual-licensed, but it **is AGPL** — strong copyleft with a network-use clause
(free per DFSG, but relevant for redistribution policy). Note the file is `license.md`
(lowercase markdown), not `LICENSE`; hatchling ships it into
`magg-<ver>.dist-info/licenses/license.md`, so a `fetchPypi`-style build finds it automatically.
There is no separate `NOTICE`/dual-license file.
---
## 5. Binaries produced (`meta.mainProgram`)
**Two console scripts** — declared in `pyproject.toml` `[project.scripts]` and confirmed by
`magg-1.3.0.dist-info/entry_points.txt` inside the released wheel:
```ini
[console_scripts]
magg = magg.cli:main
mbro = magg.mbro.cli:main
```
- `meta.mainProgram = "magg";`
- Second binary: **`mbro`** (the "MCP Browser" interactive CLI, documented in `docs/mbro.md`).
Set `meta.mainProgram` to `magg` and expect both `bin/magg` and `bin/mbro` in the output.
- `python -m magg` and `python -m magg.mbro` also work (`magg/__main__.py`,
`magg/mbro/__main__.py` exist in the wheel).
- Upstream install docs' primary entry point is `uv tool install magg` then `magg serve`.
---
## 6. Repo layout relevant to packaging
- **App is at the repo root, single flat package** — importable package dir `magg/` sits at the
top level next to `pyproject.toml` (hatchling auto-detects it; no `[tool.hatch.build]` wheel
`packages` override). **No Cargo/npm workspace, no `src/` layout, no sub-crate.**
- Package subpackages inside `magg/`: `contrib/`, `discovery/`, `logs/`, `mbro/`, `proxy/`,
`server/`, `util/` — 66 files total in `magg/`.
- **Package data shipped in the wheel** (must not be stripped): `magg/contrib/magg.mbro`,
`magg/contrib/readme.md`, `magg/contrib/kit.d/example.json` (default kit directory).
Wheel contains 62 files: 56 `.py` + those 3 data files + `dist-info`.
- **`etc/glama-mcp-api.json` (19 729 B) is NOT shipped** in the wheel *or* the sdist — the
`[tool.hatch.build.targets.sdist]` include list omits `etc/`. It is a vendored API spec used
only for upstream development/discovery. Same for `.github/` (explicitly excluded).
- **No git submodules** — `https://raw.githubusercontent.com/sitbon/magg/v1.3.0/.gitmodules` → HTTP 404.
- **No Git LFS** — `.gitattributes` → HTTP 404; the git tree lists all 145 entries as plain
blobs (`git/trees/v1.3.0?recursive=1`, `truncated: false`), largest being `uv.lock` (387 038 B),
`magg/cli.py` (41 731 B), `magg/server/server.py` (37 036 B), `license.md` (34 260 B).
Repo size ≈ 1.3 MB — cheap to fetch.
- **Tests are shipped in the sdist** (`magg-1.3.0/test/…`, 35 files, incl. `test/magg/`) —
a `checkPhase` can run `pytest` from the unpacked sdist. `pyproject.toml` registers the
`integration` marker; 3 tests in the tree carry `@pytest.mark.integration`, so
`-m "not integration"` is the safe deselection (upstream's CI runs the full suite on
`.github/workflows/test.yml`, `uv run pytest -v`).
- Other root artifacts: `compose.yaml` (Docker Compose), `dockerfile`, `server.json`
(MCP Registry manifest, `io.github.sitbon/magg`, version kept in sync with `pyproject.toml`
by `scripts/bump_version.py --check`), `docs/` (11 markdown files, `docs/index.md` is the hub),
`examples/` (6 python examples), `scripts/` (`bump_version.py`, `validate_manual_release.py`).
---
## 7. Upstream build / install instructions (from `readme.md` § Installation)
**Prerequisites (upstream's words):** "Python 3.12 or higher (3.13+ recommended)" and
"`uv` (recommended) - Install from astral.sh/uv".
```bash
# 1. Recommended: install from PyPI as a uv tool
uv tool install magg
magg serve # stdio transport (Claude Desktop, Cline, …)
magg serve --http # HTTP transport, system-wide
# 2. Run straight from GitHub (no install)
uvx --from git+https://github.com/sitbon/magg.git magg
uvx --from git+https://github.com/sitbon/magg.git magg serve --http
# 3. Local development
git clone https://github.com/sitbon/magg.git && cd magg
uv sync --dev # or: poetry install --with dev
magg --help
# 4. Docker (ghcr images)
docker run -p 8000:8000 ghcr.io/sitbon/magg:latest # prod, WARNING log level
docker run -p 8000:8000 ghcr.io/sitbon/magg:beta # beta, INFO log level
docker compose up # prod/staging/dev via compose.yaml
```
Other run modes: `magg serve --http --port 8000`, `magg serve --hybrid [--port N]`,
`mbro connect local-magg magg serve`.
**Required runtime env vars:** none are mandatory. All are optional `MAGG_*` (see `.env.example`
and README § Environment Variables):
`MAGG_CONFIG_PATH` (default `.magg/config.json`, cwd-relative), `MAGG_LOG_LEVEL`
(DEBUG/INFO/WARNING/ERROR/CRITICAL, default INFO), `MAGG_STDERR_SHOW=1`,
`MAGG_AUTO_RELOAD` (default true), `MAGG_RELOAD_POLL_INTERVAL` (default 1.0),
`MAGG_RELOAD_USE_WATCHDOG`, `MAGG_READ_ONLY=true`, `MAGG_SELF_PREFIX` (default `magg`),
`MAGG_PREFIX_SEP` (default `_`), `MAGG_BACKEND_INIT_TIMEOUT` (default 30, `0` = wait forever),
`MAGG_PRIVATE_KEY` / `MAGG_JWT` (only for authenticated HTTP mode; key generated with
`magg auth private-key --oneline` / `magg auth jwt -q`). Docker sets
`MAGG_CONFIG_PATH`, `MAGG_READ_ONLY`.
**Runtime deps beyond CPython:** the Magg package itself needs only the PyPI deps above
(`cryptography` is the only binary-wheel-dependent one). **`node`/`npx` and `uvx` are needed at
*runtime* only to launch proxied backend MCP servers** — Magg explicitly supports backend
commands `python`, `node`, `npx`, `uvx` (`magg/util/transport.py`, `magg/util/transports.py`),
and deliberately defers the "npx not on PATH" failure to connection time rather than startup.
Upstream's own `dockerfile` (alpine 3.24 base) installs `tini bash curl nano nodejs npm` plus `uv`
in a venv — a good hint at what to wrap in `buildInputs`/`makeWrapper` if you want out-of-the-box
npx backends. **No ffmpeg, no binaries vendored, no network needed at build time** (but note
`magg_search_servers` hits GitHub/Glama/MCP-registry APIs at *runtime*).
**CI reference for a `checkPhase`** (`.github/workflows/test.yml`): matrix on Python
`3.12`, `3.13`, `3.14`, `ubuntu-latest`, `uv sync --all-groups --locked` then
`uv run pytest -v`. Dev group additionally pulls `keyring`, `anthropic`, `packaging`, `ruff`
(`ruff` config: `line-length = 120`, lint `select = ["F","I"]`).
---
## 8. Homepage / documentation / changelog URLs
From `pyproject.toml` `[project.urls]` (verbatim) and the wheel `METADATA` `Project-URL` lines:
| Key | URL |
| --- | --- |
| Homepage | `https://github.com/sitbon/magg` |
| Repository | `https://github.com/sitbon/magg.git` |
| Documentation | `https://github.com/sitbon/magg#readme` |
| Issues | `https://github.com/sitbon/magg/issues` |
| Release Notes | `https://github.com/sitbon/magg/releases` |
- **`meta.homepage = "https://github.com/sitbon/magg";`** (matches `server.json.websiteUrl`).
Note: the GitHub repo's own `homepage` field points to `https://pypi.org/project/magg/` —
use the GitHub URL for `meta.homepage` (per manifest) and add
`https://pypi.org/project/magg/` as a source/download location.
- **`meta.changelog = "https://github.com/sitbon/magg/releases";`** — there is **no
`CHANGELOG`/`HISTORY` file anywhere in the repo** (verified via the full recursive git tree of
`v1.3.0`; the only "changelog" mention is inside `.github/workflows/readme.md`). Release notes
are generated per tag by GitHub Actions; per-tag URL form:
`https://github.com/sitbon/magg/releases/tag/v1.3.0`.
- In-repo docs: `docs/index.md` (hub), plus `docs/mbro.md`, `docs/authentication.md`,
`docs/config-reload.md`, `docs/examples.md`, `docs/kits.md`, `docs/mcp-registry.md`,
`docs/mcp-stateless.md`, `docs/messaging.md`, `docs/proxy.md`, `docs/proxy-spec.md`.
These are shipped in the sdist (not the wheel). Third-party AI docs: `https://deepwiki.com/sitbon/magg`.
- MCP Registry listing: `io.github.sitbon/magg` on `https://registry.modelcontextprotocol.io`.
---
## 9. Supported platforms
- **Declared: OS-independent, CPython only.** `pyproject.toml` classifiers include
`"Operating System :: OS Independent"` and `"Programming Language :: Python :: 3 :: Only"` /
`Implementation :: CPython`; `requires-python = ">=3.12"` (3.12/3.13/3.14 listed).
No `platform_machine`, no macOS/x86-only markers in the dependency set.
- **CI only exercises Linux** (`runs-on: ubuntu-latest`, Python 3.12/3.13/3.14).
- **Platform-specific code (all guarded, none exclusionary):**
- `magg/server/runner.py:102-109,127-135` — SIGHUP-triggered config reload is
**Unix-only**, guarded by `if hasattr(signal, "SIGHUP")`; comment at line 118:
"No loop signal support (e.g. Windows): hand the signal to the loop thread-safely".
README: "SIGHUP signal: `kill -HUP <pid>` … (Unix-like systems)".
- `magg/util/uri.py:30-31` — Windows path handling: `if os.name == "nt" and path.startswith("/") …`.
- `magg/util/system.py:24` / `magg/mbro/cli.py:596` / `magg/util/terminal.py:39` — TTY
detection (`os.isatty`, `sys.stdout.isatty`) to degrade the `rich`/`prompt-toolkit` REPL
when not interactive.
- `magg/mbro/arepl.py:110` prints `f"asyncio REPL {sys.version} on {sys.platform}"` (cosmetic).
- **musl/Alpine:** upstream ships a working Alpine image and applies a
"Fix for Python 3.12 extension suffix mismatch on Alpine" in the `dockerfile` — relevant if
you target `x86_64-linux` with libc-agnostic Python; nothing in pure-Python Magg itself is
libc-specific (only `cryptography`/`aiohttp` wheels matter).
- Termux/Android is implicitly considered (commit `1b18b87f` notes that Termux sets env vars
like `PREFIX`/`URI`/`KEY_PATH`, which Magg's config models no longer read).
- **Practical conclusion for `meta.platforms`:** `lib.platforms.all`-ish (pure `py3-none-any`
wheel, `Operating System :: OS Independent`); upstream *tests* Linux only, and the interactive
`mbro` REPL + SIGHUP reload are nicest on Unix.
---
## Quick derivation cheat-sheet (facts, not a proposal)
```
pname = "magg"; version = "1.3.0";
src = GitHub owner="sitbon" repo="magg" rev="v1.3.0"
sha256 = da3cc1b646b8616a78c841af268ac737f250a48eac582ae646efd3b6f467aa95 # magg-1.3.0.tar.gz
# or PyPI pname="magg" version="1.3.0" (same sha256 family; wheel:
# 6360f04b49fdce6d8ec48aa6865848f8a610a60a3e433fcd539383299958c933)
tag commit = 1b18b87fdcda9628381888aea25fefa60fe1f923 (annotated tag obj cc9963ea784fcc2e465c318ab9f0f24e5059c1d1)
build system = hatchling (PEP 517) ; python = 3.12..3.14
license = AGPL-3.0-or-later ; mainProgram = "magg" (also installs "mbro")
tests = pytest (sdist ships test/); deselect -m "not integration"
```
MISSION COMPLETE (t-001)