RFC: pnpm agent — server-side resolution for faster installs#9
Open
zkochan wants to merge 1 commit into
Open
Conversation
zkochan
added a commit
to pnpm/pnpm
that referenced
this pull request
Apr 14, 2026
gluxon
approved these changes
Apr 14, 2026
gluxon
left a comment
gluxon
left a comment
Member
There was a problem hiding this comment.
This is a great idea. Definitely in support of the RFC.
zkochan
added a commit
to pnpm/pnpm
that referenced
this pull request
Apr 16, 2026
zkochan
added a commit
to pnpm/pnpm
that referenced
this pull request
Apr 16, 2026
zkochan
added a commit
to pnpm/pnpm
that referenced
this pull request
Apr 17, 2026
Member
Author
|
This will be part of pnpr: https://github.com/pnpm/pnpm/tree/main/pnpr/npm/pnpr#pnpmpnpr |
zkochan
added a commit
to pnpm/pnpm
that referenced
this pull request
May 29, 2026
Port the pnpm-agent proof of concept from TypeScript onto pacquet's resolver and content-addressable store, exposed as additive, opt-in, versioned endpoints alongside pnpr's npm-compatible API: - POST /v1/install resolves a project server-side (pacquet Install in lockfile-only mode against a throwaway temp project), fetches only the uncached packages into the shared store, and streams an NDJSON response: D lines (file digests the client is missing), I lines (pre-packed msgpackr-records store-index entries), and a final L line with the lockfile and stats. - POST /v1/files serves a batch of files by digest as a gzip binary stream the client writes straight into its CAFS. The pacquet runtime (a single leaked Config plus a shared HTTP client) is held lazily per server in router state, so servers that never receive an agent request pay nothing and each server in a multi-server test process keeps its own store. Adds a public Lockfile::load_wanted_from_dir mirroring pnpm's readWantedLockfile(dir). Deferred (documented in the module): multi-project workspaces, incremental resolution from a client lockfile, overrides, per-request minimumReleaseAge, and true response streaming. Refs pnpm/rfcs#9 --- Written by an agent (Claude Code, claude-opus-4-8).
Member
Author
|
I am going to license the new server part with a source-available license instead of MIT: pnpm/pnpm#12082 So I am not sure if the rfc can be merged here or it should be a separate process for RFCs related to pnpr. |
zkochan
added a commit
to pnpm/pnpm
that referenced
this pull request
May 30, 2026
Port the pnpm-agent proof of concept from TypeScript onto pacquet's resolver and content-addressable store, exposed as additive, opt-in, versioned endpoints alongside pnpr's npm-compatible API: - POST /v1/install resolves a project server-side (pacquet Install in lockfile-only mode against a throwaway temp project), fetches only the uncached packages into the shared store, and streams an NDJSON response: D lines (file digests the client is missing), I lines (pre-packed msgpackr-records store-index entries), and a final L line with the lockfile and stats. - POST /v1/files serves a batch of files by digest as a gzip binary stream the client writes straight into its CAFS. The pacquet runtime (a single leaked Config plus a shared HTTP client) is held lazily per server in router state, so servers that never receive an agent request pay nothing and each server in a multi-server test process keeps its own store. Adds a public Lockfile::load_wanted_from_dir mirroring pnpm's readWantedLockfile(dir). Deferred (documented in the module): multi-project workspaces, incremental resolution from a client lockfile, overrides, per-request minimumReleaseAge, and true response streaming. Refs pnpm/rfcs#9 --- Written by an agent (Claude Code, claude-opus-4-8).
zkochan
added a commit
to pnpm/pnpm
that referenced
this pull request
May 31, 2026
…lient + CLI) (#12077) ## What Adds an opt-in **`pnprServer`** setting that offloads the slow part of an install — dependency resolution and computing which files the local store is missing — to a [pnpr](https://github.com/pnpm/pnpm/tree/main/pnpr) server, which streams back only the missing files. `node_modules` is still linked **locally** from the server-produced lockfile (like server-side rendering: the compute runs remotely, the result is materialized locally). Realizes the agent concept from [RFC #9](pnpm/rfcs#9), reworked around how it's actually used and rewritten in Rust on pacquet + pnpr. ## How it works 1. `pacquet install` (with `pnprServer` set) handshakes the server — `GET /-/pnpr` — to negotiate a protocol version. 2. It `POST`s `/v1/install` with the project's dependencies, the integrities already in its store, and **its own registry config** (default `registry`, `namedRegistries`, `overrides`, `minimumReleaseAge`). 3. The server resolves against *those* registries, fetches any uncached packages into its store, and streams NDJSON: `D` (missing file digests), `I` (pre-packed store-index entries), `L` (lockfile + stats). 4. The client downloads the missing files from `/v1/files` (gzip binary), writes them into its CAFS **by digest** (no re-hashing), writes the index entries, and runs a frozen install to link `node_modules` from the server's lockfile. ## Pieces - **Server (`pnpr`)** — `GET /-/pnpr` handshake + `POST /v1/install` (NDJSON) + `POST /v1/files` (gzip), additive and opt-in alongside the npm-compatible API. Resolves against the client-sent registries, interning a `&'static Config` per distinct client config to bound the leak. - **Client (`pacquet-pnpr-client`)** — `PnprClient`: reads store integrities, negotiates the protocol version, sends the registry config, parses the stream, materializes files + index entries, returns the lockfile. Rejects unrequested file entries and repairs truncated CAFS files. - **CLI** — the `pnprServer` setting (`--pnpr-server`, `pnprServer:` in `pnpm-workspace.yaml`, `PNPM_CONFIG_PNPR_SERVER`). When set, `pacquet install` routes through the client and then links locally — pnpm's `install()` → `installFromPnpmRegistry` shape. `trustPolicy: no-downgrade` is refused (the server can't enforce it), matching pnpm's `TRUST_POLICY_INCOMPATIBLE_WITH_AGENT`. ## Design notes - **A distinct URL, not the registry.** The server resolves from the registries the client sends, so it's a compute service — not "a registry that resolves from itself" — which is why it's a separate `pnprServer` URL rather than reusing `registry`. The same server works for any client's registry setup, and a single pnpr can be both registry and `pnprServer`. - **Handshake = version negotiation + fail-fast.** Explicit opt-in, so there's no silent fallback to local resolution; a non-pnpr server (404) or a version mismatch errors clearly. - **Naming:** everything is `pnpr`; "agent" survives only in upstream citations (`@pnpm/agent.client`, the pnpm-agent PoC, pnpm's `TRUST_POLICY_INCOMPATIBLE_WITH_AGENT` error code). ## Tests - `pacquet-pnpr-client`: resolve + download, multi-file package, warm-store no-op, and handshake rejection. The pnpr server's own uplink is left at the default, so resolution provably uses the **client-sent** registry. - `pacquet-cli`: a real `pacquet install --pnpr-server <url>` against an in-process pnpr (resolving from the mocked fixtures registry) links `node_modules`. - `pnpr`: `/v1/files` binary-framing round-trip + handshake route. Full suites green; clippy / dylint (Perfectionist) / fmt / taplo / `cargo doc -D warnings` clean. ## Deferred Auth/credential forwarding (so private/scoped registries resolve), `pacquet add` / `remove` via `pnprServer`, multi-project workspaces, and true streaming (responses are buffered today). Refs pnpm/rfcs#9
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This RFC proposes a pnpm agent — an opt-in server that resolves dependencies server-side and streams only the files missing from the client's store.
Reference implementation: pnpm/pnpm#11251
Key ideas
install()with SQLite-backed metadata cache (~1s resolution vs ~3.4s)Configuration
See the full RFC text for details.