nix: shrink daemon install with @vercel/nft tracing (#966)
* nix: share npmDeps FOD between paseo and paseo-desktop
The daemon and desktop derivations share package-lock.json, so their
npmDeps FODs contain byte-identical content. But Nix names the FOD
store path with the consuming pname prefix, so paseo-<v>-npm-deps and
paseo-desktop-<v>-npm-deps resolve to different store paths despite
identical bytes. Building both runs prefetch-npm-deps twice in
parallel; on a clean store this downloads the entire registry
(~1-2 GB of tarballs) twice over the wire.
Wire paseo through as a callPackage arg of the desktop drv and
inherit (paseo) npmDeps. One FOD, one fetch, one store path.
Override path becomes paseo.override { npmDepsHash = "..."; } — the
desktop drv picks up the overridden npmDeps transitively. Downstream
flakes that previously did paseo-desktop.override { npmDepsHash }
need to either drop the desktop-side override (recommended) or pass
the overridden daemon through as paseo-desktop.override { paseo }.
* nix: trace daemon runtime closure with @vercel/nft
The daemon Nix installPhase used to `cp -a node_modules $out/lib/paseo/`,
shipping every package in the hoisted root — Expo, React Native, Metro,
Electron, ML stacks, ~1700 third-party deps the daemon never `require()`s
at runtime. Result: ~2.6 GB store path for a daemon whose actual runtime
closure is a tiny fraction of that.
Replace it with static module-graph tracing via @vercel/nft (the same
library Vercel and Next.js use for serverless bundling). The new
installPhase runs `node scripts/trace-daemon.mjs`, which:
1. Calls nodeFileTrace on three entry points — cli/dist/index.js,
server/dist/scripts/supervisor-entrypoint.js, and the forked
server/dist/server/terminal/terminal-worker-process.js. The worker
needs to be a separate entry because nft does not follow fork()
process boundaries.
2. Unions the trace result with an explicit list of non-JS runtime
inputs nft does not detect: shell-integration assets read via
readFileSync, the Silero VAD ONNX model loaded by the sherpa
provider, .env.example, the CLI shebang script, and node-pty's
compiled prebuilt binary for the host platform.
3. Ignores cross-platform native variants we explicitly do not ship
(sherpa-onnx-*, @mariozechner/clipboard-*) and node-fetch's
optional `encoding` peer.
installPhase consumes the file list, copies each entry into
$out/lib/paseo/ preserving its repo-relative path, and wraps two bin
entries (paseo, paseo-server) via makeWrapper.
Verified end-to-end:
- daemon $out drops from ~2.6 GB to ~131 MB (≈95% reduction)
- paseo --version, paseo --help work from the new $out
- paseo-server boots, HTTP server reaches "Server listening" within
~40 ms, no missing-module errors in the bootstrap log
- speech features degrade gracefully when sherpa-onnx-* / model files
are absent (the documented behaviour for the Nix build, unchanged)
@vercel/nft added to root devDependencies (1.5.0); used only at
build time by the Nix derivation.
* fix(nix): gitignore result
* fix(nix): refresh npm deps hash post-rebase
The conflict resolution during rebase kept the trace-daemon hash, but
the merged package-lock.json combines upstream's lockfile updates with
the @vercel/nft addition, so the hash needed to be recomputed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matan Bendix Shenhav
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
5ea68dcdfa
commit
e309d82b41
@@ -9,10 +9,11 @@
|
||||
makeDesktopItem,
|
||||
electron,
|
||||
libuv,
|
||||
# Shares the daemon's npm-deps hash — same package-lock.json, same fetcher.
|
||||
# Override via `.override { npmDepsHash = "..."; }` if your nixpkgs computes a
|
||||
# different value.
|
||||
npmDepsHash ? lib.fileContents ./npm-deps.hash,
|
||||
# Reuse the daemon's prebuilt npm-deps FOD. Same lockfile, same content —
|
||||
# without this, the desktop drv produces a separately-named store path
|
||||
# (`paseo-desktop-<v>-npm-deps`) and refetches the entire registry. Override
|
||||
# the upstream hash via `paseo.override { npmDepsHash = "..."; }`.
|
||||
paseo,
|
||||
}:
|
||||
|
||||
buildNpmPackage rec {
|
||||
@@ -21,7 +22,8 @@ buildNpmPackage rec {
|
||||
|
||||
src = lib.cleanSourceWith {
|
||||
src = ./..;
|
||||
filter = path: type:
|
||||
filter =
|
||||
path: type:
|
||||
let
|
||||
baseName = builtins.baseNameOf path;
|
||||
relPath = lib.removePrefix (toString ./..) path;
|
||||
@@ -42,7 +44,7 @@ buildNpmPackage rec {
|
||||
};
|
||||
|
||||
nodejs = nodejs_22;
|
||||
inherit npmDepsHash;
|
||||
inherit (paseo) npmDeps;
|
||||
|
||||
# Prevent onnxruntime-node's install script from running during automatic
|
||||
# npm rebuild. We manually rebuild only node-pty in buildPhase.
|
||||
|
||||
+1
-1
@@ -1 +1 @@
|
||||
sha256-vBudtcS/3V5tbtAug++V9F5BZknsTFdsMyv914gfy9Q=
|
||||
sha256-SRax+Frqf0OhDhfWICSlhOnHBGsoHiXQc6P+zo5VjN8=
|
||||
|
||||
+15
-37
@@ -88,47 +88,25 @@ buildNpmPackage rec {
|
||||
installPhase = ''
|
||||
runHook preInstall
|
||||
|
||||
# Compute the daemon's runtime closure by static module-graph tracing
|
||||
# (@vercel/nft from supervisor-entrypoint.js, cli/dist/index.js, and the
|
||||
# forked terminal-worker-process.js) plus an explicit list of non-JS
|
||||
# assets read at runtime. The trace script is the single source of
|
||||
# truth for what the daemon needs at $out — auditable in plain JS, no
|
||||
# npm hoisting / .bin / workspace-symlink footguns.
|
||||
mkdir -p $out/lib/paseo
|
||||
node scripts/trace-daemon.mjs > daemon-files.txt
|
||||
|
||||
# Copy root package metadata
|
||||
while IFS= read -r path; do
|
||||
[ -z "$path" ] && continue
|
||||
mkdir -p "$out/lib/paseo/$(dirname "$path")"
|
||||
cp -a "$path" "$out/lib/paseo/$path"
|
||||
done < daemon-files.txt
|
||||
|
||||
# Root package.json lets node resolve the workspace layout when the
|
||||
# CLI/server bin starts from $out.
|
||||
cp package.json $out/lib/paseo/
|
||||
|
||||
# Copy node_modules (preserving workspace symlinks)
|
||||
cp -a node_modules $out/lib/paseo/
|
||||
|
||||
# Auto-detect which @getpaseo/* packages were built by build:daemon
|
||||
# (they'll have a dist/ directory). Copy those and remove the rest.
|
||||
for link in $out/lib/paseo/node_modules/@getpaseo/*; do
|
||||
name=$(basename "$link")
|
||||
if [ -d "packages/$name/dist" ]; then
|
||||
mkdir -p "$out/lib/paseo/packages/$name"
|
||||
cp "packages/$name/package.json" "$out/lib/paseo/packages/$name/"
|
||||
cp -a "packages/$name/dist" "$out/lib/paseo/packages/$name/"
|
||||
if [ -d "packages/$name/node_modules" ]; then
|
||||
cp -a "packages/$name/node_modules" "$out/lib/paseo/packages/$name/"
|
||||
fi
|
||||
else
|
||||
rm -f "$link"
|
||||
fi
|
||||
done
|
||||
|
||||
# Copy CLI bin entry
|
||||
mkdir -p $out/lib/paseo/packages/cli/bin
|
||||
cp packages/cli/bin/paseo $out/lib/paseo/packages/cli/bin/
|
||||
|
||||
# Copy extra server files referenced at runtime
|
||||
for f in agent-prompt.md .env.example; do
|
||||
if [ -f packages/server/$f ]; then
|
||||
cp packages/server/$f $out/lib/paseo/packages/server/
|
||||
fi
|
||||
done
|
||||
|
||||
# Copy server scripts (including supervisor-entrypoint) needed by CLI
|
||||
if [ -d packages/server/dist/scripts ]; then
|
||||
mkdir -p $out/lib/paseo/packages/server/dist/scripts
|
||||
cp -a packages/server/dist/scripts/* $out/lib/paseo/packages/server/dist/scripts/
|
||||
fi
|
||||
|
||||
# Create wrapper for the server entry point (for systemd / direct use)
|
||||
mkdir -p $out/bin
|
||||
makeWrapper ${nodejs}/bin/node $out/bin/paseo-server \
|
||||
|
||||
Reference in New Issue
Block a user