docs: research gVisor supervisor topology

goals/research/gvisor-supervisor-topology.md

@@ -0,0 +1,228 @@
+# gVisor supervisor topology for Linux deployments
+
+## Decision
+
+Build the POC around **one root-owned host service that invokes `runsc` directly against OCI bundles**. The service exposes a Paseo-specific Unix socket and nothing else. A host-installed Paseo daemon connects to that socket directly. The official Paseo Docker container receives only a bind mount of that socket and a deployment identifier; the host service maps that identifier to root-owned configuration containing the real host workspace paths.
+
+Do not put `runsc` inside the Paseo container. Do not give Paseo, the Paseo container, or an agent sandbox `docker.sock`, `containerd.sock`, the runsc state directory, arbitrary OCI configuration, or the supervisor's executable environment.
+
+This is the smallest topology that has the same sandbox owner in both deployments and does not make Docker or containerd part of Paseo's agent control plane.
+
+```text
+Host-installed deployment
+
+  paseo daemon (user) ── narrow RPC ──> /run/paseo-sandbox/supervisor.sock
+                                            │ root-owned supervisor
+                                            ├─ builds validated OCI bundle
+                                            ├─ runsc --root=/run/paseo-sandbox/runsc ...
+                                            └─ owns PTY/pipes and lifecycle records
+
+Official Docker deployment
+
+  paseo container (uid 1000)
+    └─ bind-mounted supervisor.sock ───────> same host supervisor and runsc
+                                             │
+  host bind source ── supervisor mapping ────┴─> sandbox mount at /workspace
+
+Agent sandbox
+  ├─ versioned read-only rootfs
+  ├─ only approved workspace/credential mounts
+  ├─ no supervisor socket
+  └─ no Docker/containerd socket
+```
+
+## Primary-source findings
+
+### `runsc` is usable as the low-level boundary
+
+`runsc` is an OCI runtime. Its direct quick start constructs a bundle containing `config.json` and a `rootfs`, then runs that bundle with `sudo runsc run <id>`.[1] The OCI runtime specification defines durable container identity and the `create`, `start`, `state`, `kill`, and `delete` operations. Returned state includes the container ID, lifecycle status, runtime PID, and absolute bundle path.[2] This is enough for a supervisor to own container lifecycle without Docker or containerd.
+
+The OCI configuration controls the process user, capabilities, `noNewPrivileges`, read-only root filesystem, additional mounts, namespace set, and resource limits. Bind-mount sources name paths on the runtime host, while mount destinations name paths inside the container.[3] The supervisor must therefore construct `config.json`; a daemon inside Docker cannot safely or correctly submit its own container-visible paths as host mount sources.
+
+`runsc exec` starts a **new** process in an existing container. Its implementation supports a process JSON file, detached execution, a PID file, and a console socket that transfers the master PTY file descriptor.[4] It is not an API for attaching new stdio to an already-running process. `runsc list` and `runsc state` load containers from the selected runtime root, so a restarted supervisor can reconcile runtime state when it uses the same root directory.[5]
+
+**Design consequence:** the supervisor, not the Paseo daemon, must retain PTY or pipe ownership and multiplex daemon connections. Daemon reconnect is feasible. A supervisor crash can recover container identity and status, but cannot be assumed to recover the original stdio file descriptors.
+
+### gVisor narrows, but does not erase, the host boundary
+
+gVisor's Sentry implements the application-facing system call API rather than passing application system calls directly to the host. Host filesystem access is mediated by a Gofer over a connected socket; the normal Sentry runs in an empty mount namespace and has a restricted host syscall surface.[6] The `runsc` configuration source states that `--directfs` lets the Sentry access the filesystem directly with elevated privileges, whereas the normal mode keeps the Gofer in the data path.[7]
+
+**POC rule:** use the default Gofer-backed filesystem path and do not enable `--directfs`. A writable workspace still gives the sandbox authority to alter every file mounted there. gVisor protects the host kernel boundary; it does not turn an overly broad mount into a safe mount.
+
+### Rootless `runsc` does not fit this lifecycle
+
+gVisor documents three rootless patterns. The convenient `runsc --rootless` mode maps only the caller's UID and explicitly does not support `runsc create`; it also lacks save/restore and Netstack external connectivity, ignores cgroup configuration errors, and requires host networking for external access.[8] The native OCI user-namespace route can use multiple mappings through `newuidmap`/`newgidmap`, but currently lacks network namespacing; strictly unprivileged nested environments collapse to a single UID/GID mapping and cannot faithfully unpack ordinary multi-owner root filesystems.[8]
+
+The `runsc` source also warns that defense-in-depth is weaker in rootless mode because Sentry and Gofer are root inside a user namespace mapped to the caller.[7]
+
+**Decision:** retain the required root-owned supervisor for the POC. Rootless mode removes capabilities needed by the intended `create`/`start` lifecycle and produces different behavior between host and Docker deployments.
+
+### The Unix socket can be a narrow authorization boundary
+
+On Linux, pathname Unix sockets honor parent-directory permissions; connecting to a stream socket requires write permission on the socket. Ownership and mode can be changed with `chown`/`chmod`. `SO_PEERCRED` returns the credentials captured for the connected peer.[9] Abstract sockets have no meaningful filesystem permissions, so they are inappropriate here.[9]
+
+Use a pathname socket such as `/run/paseo-sandbox/supervisor.sock`, owned by `root:paseo-sandbox` with mode `0660`, in a root-owned non-writable directory. The supervisor must also verify `SO_PEERCRED`; filesystem mode is not the whole authorization policy. For Docker, Compose must add the socket's numeric host GID as a supplementary group because Unix permission checks use numeric credentials.
+
+The socket is mounted only into the outer Paseo container. It is never included among OCI mounts generated for an agent. Linux mount namespaces isolate mount views, and network namespaces isolate network resources.[10] A filesystem pathname that is absent from the sandbox mount namespace is not reachable merely because another process on the host can see it.
+
+### Docker and containerd integration are valid tests, but the wrong control plane
+
+gVisor's Docker integration registers `runsc` in `/etc/docker/daemon.json`, restarts Docker, and selects it with `docker run --runtime=runsc`.[11] Its containerd integration installs `containerd-shim-runsc-v1`, registers `io.containerd.runsc.v1` in `config.toml`, restarts containerd, and then uses `ctr` or CRI clients that communicate with containerd.[12]
+
+Those integrations solve image management, network setup, and lifecycle orchestration. They also place the Docker/containerd API between Paseo and `runsc`. Exposing either manager socket would grant a much broader container-management authority than Paseo needs. Even if only the root supervisor held that socket, the POC would inherit daemon configuration, version, restart, plugin, and sibling-container behavior that direct OCI operation avoids.
+
+Docker bind mounts name a source on the Docker daemon host and a destination in the outer container. They are writable by default and are tied to the host's directory structure.[13] This confirms why the Docker topology needs an explicit host-side mapping: `/workspace` inside Paseo is not a trustworthy host path.
+
+## Alternatives
+
+| Alternative | Host-installed daemon | Official Docker deployment | Privilege and socket consequences | Decision |
+| --- | --- | --- | --- | --- |
+| **Host root supervisor + direct `runsc` OCI** | Direct Unix socket | Bind-mount only the same Unix socket; host config maps deployment roots | One small root service owns bundles, runsc state, namespaces, PTYs, and mounts. No general manager socket. | **Choose for POC** |
+| Root supervisor driving Docker with `--runtime=runsc` | Requires Docker even for local installs | Creates sibling gVisor containers | Paseo remains off `docker.sock`, but the supervisor holds root-equivalent Docker authority and inherits Docker lifecycle/network semantics. | Keep as a diagnostic fallback, not the POC |
+| Root supervisor driving containerd's runsc shim | Adds containerd to local installs | Creates sibling containers through containerd | Same broad-manager issue; also adds shim/plugin/CNI configuration and a daemon restart prerequisite. | Reject for POC |
+| Run `runsc` inside the official Paseo container | Nested only | Same container | Conflicts with the image's non-root daemon, needs substantial namespace/cgroup/device privilege, and rootless paths lack required lifecycle/network behavior. A privileged outer container would enlarge the host attack surface. | Reject |
+| Daemon invokes rootless `runsc` | Different behavior and limits | Nested rootless constraints | No restricted root supervisor; `create` and useful isolated networking are not available in the simple rootless mode. | Reject |
+
+## Recommended POC topology
+
+### Host service and protocol
+
+Install `paseo-sandbox-supervisor` as a system service running as root. Give it private directories:
+
+- `/run/paseo-sandbox/` for the socket, PID/lease data, and a single explicit runsc `--root`;
+- `/var/lib/paseo-sandbox/bundles/<sandbox-id>/` for generated bundles and immutable metadata;
+- `/var/lib/paseo-sandbox/rootfs/<version>/` for versioned, read-only base root filesystems;
+- a root-owned deployment registry mapping an opaque deployment ID to allowed host workspace and credential roots.
+
+The RPC should expose domain operations such as `createSandbox`, `startProcess`, `writeInput`, `subscribeOutput`, `signalProcess`, `inspectSandbox`, and `deleteSandbox`. It must not expose argv for `runsc`, raw OCI JSON, arbitrary mount source/destination pairs, hooks, capabilities, host paths, or generic `exec` as root. Requests identify a registered workspace by opaque handle and a fixed provider launch profile. The supervisor resolves paths beneath configured roots using descriptor-relative operations, rejects symlinks and mount escapes, generates the bundle, and invokes a pinned `runsc` binary with fixed flags.
+
+The supervisor records the requesting peer identity, deployment ID, Paseo sandbox ID, runsc container ID, bundle version, process status, and output cursor. Agent sandboxes receive no copy of the control socket.
+
+### Host-installed Paseo
+
+The installer:
+
+1. installs signed/pinned `runsc` artifacts and the supervisor;
+2. creates the supervisor group and adds the daemon user;
+3. registers allowed workspace roots through an operator-owned command or root-owned config;
+4. starts the root service;
+5. leaves the Paseo daemon unprivileged.
+
+The daemon connects to the pathname socket and refers to host workspaces through registered handles. Arbitrary user-selected directories require an explicit registration policy; allowing every path the daemon user can name would recreate the daemon's current authority inside a root service and invites symlink/mount confusion.
+
+### Official Docker deployment
+
+The outer Paseo container remains non-root. Its deployment adds only:
+
+- a read-write bind mount of `/run/paseo-sandbox/supervisor.sock` at the same path;
+- the socket's numeric group through Compose `group_add`;
+- an opaque supervisor deployment ID;
+- the existing workspace and home bind mounts.
+
+The host operator registers that deployment ID with mappings such as:
+
+```text
+container /workspace  -> host /srv/paseo/workspace
+container /home/paseo -> host /srv/paseo/home
+```
+
+The daemon sends only the deployment ID plus container-relative workspace identity. The supervisor substitutes the preconfigured host source and mounts it at the expected sandbox destination. It rejects client-provided host paths. The outer container gets neither the bundle directory nor runsc state.
+
+The sandbox rootfs cannot be taken implicitly from the running outer container without involving Docker's API. The POC therefore needs a separately installed versioned rootfs containing Node and one test provider CLI. A later packaging ticket must decide how official and user-derived agent images produce matching sandbox rootfs artifacts without giving Paseo an image-manager socket.
+
+### Process I/O and reconnect
+
+For each provider process, the supervisor creates and retains a PTY master (via runsc's console-socket support) or retained pipes, drains output continuously into a bounded sequence buffer, and accepts multiple narrow subscribers. Paseo reconnects with `(sandbox-id, process-id, last-sequence)` and receives missed output before live data.
+
+On daemon restart, the supervisor remains alive, so the provider process and file descriptors remain owned. On supervisor restart, reconcile with `runsc list/state` and persisted records. Do not claim full reattachment: OCI state recovers lifecycle metadata, while `runsc exec` creates a new process and cannot reacquire an old PTY master. The POC should report such a process as `running-unattached` and either leave it alive for inspection or terminate/recreate it according to explicit policy. A stronger crash-recovery design would need a persistent in-sandbox I/O relay or tested checkpoint/restore and belongs in a later ticket.
+
+### Networking
+
+Use two POC profiles:
+
+1. `--network=none` for filesystem, process, lifecycle, and reconnect tests. The runsc source defines this as loopback-only Netstack.[7]
+2. `--network=sandbox` for networked tests, with the root supervisor creating a dedicated network namespace, veth, addressing, DNS inputs, and host NAT/firewall rules before start. gVisor describes sandbox networking as its isolated internal stack.[7]
+
+Do not use `--network=host` as the default. gVisor says host passthrough disables the userspace stack and decreases host isolation.[14] It is acceptable only as a clearly labeled bring-up diagnostic. Agent sandboxes created for a Docker-hosted Paseo daemon are siblings at the host network layer, not members of the outer container's Compose network; reaching Compose-only services requires host-published endpoints or a separately designed bridge.
+
+## POC prerequisites and test matrix
+
+### Prerequisites
+
+- Linux 5.6 or newer on `amd64` or `arm64`, matching gVisor's documented support floor.[15]
+- Pinned `runsc` and required companion binaries installed on the host, not inside Paseo.
+- A versioned OCI rootfs prepared without a live Docker/containerd socket in the Paseo control path.
+- A root-owned system service, private state directories, dedicated group, and pathname Unix socket.
+- For Docker: Linux Docker Engine, host bind-source registration, socket bind mount, and matching supplementary GID.
+- POC starts with the default systrap platform; KVM and `/dev/kvm` are not prerequisites.
+
+### Required tests
+
+Run every applicable case once with a host-installed daemon and once with the official Docker daemon:
+
+1. create/start a sandbox and prove gVisor is running;
+2. read/write only the registered workspace; deny adjacent host paths, supervisor state, and the socket;
+3. reject `..`, symlink, bind-submount, stale-handle, and deployment-ID substitution attempts;
+4. prove the sandbox has no Docker/containerd/supervisor socket and cannot add mounts or capabilities;
+5. stream stdin/stdout/stderr, disconnect Paseo, reconnect by output sequence, and continue the same process;
+6. restart Paseo while work runs and preserve the session;
+7. restart the supervisor and verify state reconciliation plus the documented unattached-I/O outcome;
+8. signal, wait, delete, and garbage-collect stopped and half-created containers;
+9. exercise `network=none`, then sandbox Netstack DNS and outbound HTTPS; verify inbound and host-loopback behavior explicitly;
+10. verify uid/gid behavior on bind-mounted workspaces for host users other than 1000 and for Docker's default 1000:1000;
+11. enforce CPU, memory, PID, and output-buffer bounds;
+12. test both supported architectures in CI or dedicated Linux runners.
+
+## Trust boundaries
+
+- **Root supervisor:** trusted with host root, runsc lifecycle, configured source roots, network namespaces, and resource controls. Its RPC parser and path resolver are security-critical.
+- **Paseo daemon:** trusted to request operations within pre-registered deployment/workspace policy, but not trusted with root, runsc flags, OCI hooks, raw host paths, or general container management.
+- **Outer official Docker container:** same authority as its Paseo daemon plus mounted workspace/home; possessing the narrow socket must not expand it beyond the RPC policy.
+- **gVisor Sentry/Gofer:** trusted isolation layer. Only explicitly mounted data is available; Gofer-backed mounts remain part of the host-facing trusted computing base.
+- **Agent process and its output:** untrusted. It receives no control socket and cannot ask the supervisor directly.
+- **Docker/containerd:** used to run the outer official deployment if the operator chooses Docker, but outside the agent sandbox control path.
+
+## Open uncertainties
+
+1. **Supervisor-crash I/O recovery:** confirm runsc's exact behavior when the process holding the PTY master dies. Current APIs support state reconciliation and new exec, not attaching to existing stdio.
+2. **Rootfs/toolchain distribution:** decide how provider CLIs, native libraries, CA certificates, git/ssh, and user-installed custom agents enter a signed/versioned sandbox rootfs for both deployment modes.
+3. **Credential mounts:** define per-provider read/write requirements. Mounting all of `/home/paseo` would defeat credential separation.
+4. **Workspace identity and path safety:** choose the registration UX and prove descriptor-relative resolution across symlinks, nested mounts, renamed roots, and Docker bind-source changes.
+5. **Networking policy:** choose CNI versus supervisor-owned veth/NAT, DNS handling, IPv6, egress filtering, inbound ports, and access to host or Compose services.
+6. **Resource controls:** verify cgroup v2 ownership and cleanup under systemd across supported distributions.
+7. **Provider compatibility:** test PTY versus pipes, subprocess trees, file watching, git credential helpers, browsers, and agents that launch their own sandboxes.
+8. **Multi-user authorization:** decide whether one supervisor serves multiple Paseo users and how peer credentials partition state and registered roots.
+
+## Later work unblocked
+
+This decision provides the topology needed to scope later tickets for:
+
+- the restricted supervisor RPC and authorization model;
+- OCI bundle generation, mount policy, and runsc lifecycle adapter;
+- daemon-side sandbox transport and reconnectable I/O;
+- host installer/systemd packaging and rootfs distribution;
+- official Docker Compose/socket/GID/deployment-root wiring;
+- network namespace and egress policy;
+- credential and workspace mount profiles;
+- lifecycle garbage collection and supervisor-restart behavior;
+- dual-deployment security and compatibility test harnesses.
+
+It does **not** settle rootfs packaging, production networking, credential partitioning, or supervisor-crash stream recovery.
+
+## Sources
+
+1. gVisor, [OCI Quick Start](https://gvisor.dev/docs/user_guide/quick_start/oci/).
+2. Open Container Initiative, [Runtime Specification: state, lifecycle, and operations](https://github.com/opencontainers/runtime-spec/blob/main/runtime.md).
+3. Open Container Initiative, [Runtime Configuration Specification](https://github.com/opencontainers/runtime-spec/blob/main/config.md).
+4. gVisor source, [`runsc/cmd/exec.go`](https://github.com/google/gvisor/blob/master/runsc/cmd/exec.go).
+5. gVisor source, [`runsc/cmd/list.go`](https://github.com/google/gvisor/blob/master/runsc/cmd/list.go) and [`runsc/cmd/state.go`](https://github.com/google/gvisor/blob/master/runsc/cmd/state.go).
+6. gVisor, [Security Model](https://gvisor.dev/docs/architecture_guide/security/).
+7. gVisor source, [`runsc/config/config.go`](https://github.com/google/gvisor/blob/master/runsc/config/config.go).
+8. gVisor, [Rootless](https://gvisor.dev/docs/user_guide/rootless/).
+9. Linux man-pages, [`unix(7)`](https://man7.org/linux/man-pages/man7/unix.7.html).
+10. Linux man-pages, [`namespaces(7)`](https://man7.org/linux/man-pages/man7/namespaces.7.html).
+11. gVisor, [Docker Quick Start](https://gvisor.dev/docs/user_guide/quick_start/docker/).
+12. gVisor, [Containerd Quick Start](https://gvisor.dev/docs/user_guide/containerd/quick_start/).
+13. Docker, [Bind mounts](https://docs.docker.com/engine/storage/bind-mounts/).
+14. gVisor, [Networking: network passthrough](https://gvisor.dev/docs/user_guide/networking/#network-passthrough).
+15. gVisor, [Installation](https://gvisor.dev/docs/user_guide/install/).