Isolated application/Convex restore rehearsal (T01)

This is a runbook and evidence template, not an executed restore. It is intended for a disposable self-hosted Convex deployment and synthetic users only. Never point it at the persistent development deployment, copy a live .env, or use production Infisical credentials.

Scope and ownership

T01 owns the application/Compose release procedure, Convex data and instance credential backup/restore, configuration/key backup handling, and post-restore session/grant invalidation evidence. T03 owns immutable reference artifact rollback and profile-compatible PREVIOUS handling; this runbook does not change or claim that path.

The current repository has no automated Convex migration rollback command and no implemented global post-restore session/grant invalidation migration. Those are explicit blockers below, not steps that may be silently skipped.

Prerequisites and isolation

A clean T01 worktree at a recorded commit; Bun 1.3.14 and Docker/Compose.
A unique project name, for example eden-restore-$USER-$RANDOM, and unique volume names. Keep published ports on loopback and use a disposable backend.
At least 2x the current Convex data volume bytes plus 10% free disk for source volume, backup archive, and restored volume. Record du -sb before starting.
Budget 2 CPUs, 4 GiB RAM, and 30 minutes for the disposable backend and application rehearsal. A reference rebuild is a separate T03 operation; the reviewed production worker budget is 2 CPUs/4 GiB and must not be assumed as Convex restore evidence.
A temporary mode-0600 directory outside the repository for synthetic keys, backup archives, logs, and evidence. Do not print key values or put them in command arguments.

set -euo pipefail
repo="$PWD"
project="eden-restore-${USER:-operator}-$(date +%s)-$$"
work="$(mktemp -d "${TMPDIR:-/tmp}/eden-restore.XXXXXX")"
chmod 700 "$work"
trap 'docker compose -p "$project" down -v >/dev/null 2>&1 || true; rm -rf "$work"' EXIT
(cd "$work" && bun "$repo/scripts/generate-keys.ts")
chmod 600 "$work/.env.keys"

Load only those synthetic values into a temporary shell environment. Set APP_MODE=demo, APP_RELEASE_ID=restore-fixture, APP_CAPABILITY_ALLOWLIST=asset_search,asset_valuation,asset_sync,assistant_dispatch,session_token,identity,subscriptions, and an issuer such as http://127.0.0.1:18080. Do not use a production callback or provider key.

Backup rehearsal

Start a disposable backend with a unique Compose project and record the backend image digest, git commit, volume name, and container ID.

export COMPOSE_PROJECT_NAME="$project"
docker compose -p "$project" up -d backend
volume="${project}_convex-data"
mount="$(docker volume inspect -f '{{ .Mountpoint }}' "$volume")"
du -sb "$mount"

Configure the synthetic Convex environment using the temporary key file, synthetic issuer/JWKS, bridge secret, and allowlist. Use the existing bun run convex:configure path only with the disposable URL/admin key; never pass a persistent admin key through this procedure.
Run the existing synthetic write smoke against this backend, with SMOKE_ALLOW_WRITE=1, and record its output without recording secrets. It exercises login/session, staged publication, failed publication preserving the previous generation, tenant isolation, JWT audience checks, and local revocation.
Create a consistent filesystem backup while the disposable backend is stopped. The Convex data volume contains the backend data and instance credential material; back up it as one unit. A host administrator may need root access to the Docker volume mount.
```
docker compose -p "$project" stop backend
tar --xattrs --acls --numeric-owner -C "$mount" \
  -czf "$work/convex-data.tar.gz" .
sha256sum "$work/convex-data.tar.gz" > "$work/convex-data.tar.gz.sha256"
cp --preserve=mode "$work/.env.keys" "$work/application-keys.synthetic"
sha256sum "$work/application-keys.synthetic" > "$work/application-keys.synthetic.sha256"
```
Record archive byte size, SHA-256, volume size, and stop/start timestamps. Do not include raw keys in the evidence bundle; retain the mode-0600 file in the protected temporary directory until cleanup.

Restore and isolation checks

Create a second unique Compose project and volume. Extract only the recorded archive into the new volume, preserving ownership and xattrs.

restore_project="${project}-restored"
restore_volume="${restore_project}_convex-data"
docker volume create "$restore_volume" >/dev/null
restore_mount="$(docker volume inspect -f '{{ .Mountpoint }}' "$restore_volume")"
tar --xattrs --acls --numeric-owner -C "$restore_mount" \
  -xzf "$work/convex-data.tar.gz"

Start the restored backend under the second project and configure the web container with only the synthetic application keys. Confirm that the restored instance credential is the backed-up credential, not a newly generated one; record only a redacted fingerprint/metadata, never its value.
Repeat the synthetic Convex smoke with fresh synthetic IDs. Verify:
- restored schema/functions start without a migration error;
- a newly created synthetic user cannot read the source project's records;
- restored records remain tenant-isolated;
- the restored reference/config volume is separate and mounted read-only by web;
- the old pre-backup session/JWT is rejected after the approved invalidation step; and
- the old encrypted grant cannot service synchronization until explicit reauthorization.
Current blocker: the repository does not yet provide a global post-restore session/grant invalidation command. Do not claim this check passed. The operator must choose and approve one of: a Convex migration that revokes all sessions and quarantines token records, a deployment-specific credential-version fence, or an equivalent bounded mechanism. Record the maximum stale-authority window and the decision before reopening traffic.
Test application rollback by stopping the restored web service, starting the immediately previous digest-pinned web image against the restored backend, and exercising read-only synthetic status/assets paths. Do not use a mutable latest tag. If the schema is not backward compatible, record rollback as blocked; do not mutate the source volume to make it pass.

Migration rollback rehearsal

There is no migration history/rollback command in this baseline. A valid rehearsal therefore requires a parent-approved migration fixture and a chosen rollback mechanism. At minimum:

back up the disposable volume before migration;
deploy the migration to the disposable backend;
run schema/functions and tenant-isolation checks;
restore the pre-migration volume into a fresh project (never in place);
start the previous application image and rerun the checks; and
record whether the migration is backward compatible and which data loss, RPO, and RTO limits apply.

A docker compose down -v or a successful process restart is not migration rollback evidence. T03's reference artifact rollback is a separate test and must verify profile, importer version, manifest integrity, and live-reader compatibility.

Evidence template

Copy this table into the release record and attach only redacted logs/checksums:

Field	Record
Worktree commit / image digests
Source and restored Compose project names
Convex source/restored volume names
Synthetic key IDs/fingerprints (not values)
Backup archive bytes / SHA-256
Pre/post volume byte counts
Schema/migration version and rollback mechanism
Session/grant invalidation decision and stale window
Commands and timestamps (UTC)
Synthetic smoke result
Tenant-isolation result
Old-session rejection result
Old-grant sync rejection result
Browser/provider/restore steps not exercised
Cleanup confirmation and retained artifacts/expiry
Operator approval and unresolved blockers

The evidence must label mock, fixture, disposable-backend, and consenting-live observations separately. Passing local tests, CI, or a fixture container does not constitute a restore, browser, provider, or production-readiness claim.

8.3 KiB Raw Blame History