# Isolated application/Convex restore rehearsal (T01)

This is a runbook and evidence template, not an executed restore. It is
intended for a disposable self-hosted Convex deployment and synthetic users
only. Never point it at the persistent development deployment, copy a live
`.env`, or use production Infisical credentials.

## Scope and ownership

T01 owns the application/Compose release procedure, Convex data and instance
credential backup/restore, configuration/key backup handling, and post-restore
session/grant invalidation evidence. T03 owns immutable reference artifact
rollback and profile-compatible `PREVIOUS` handling; this runbook does not
change or claim that path.

The current repository has no automated Convex migration rollback command and
no implemented global post-restore session/grant invalidation migration. Those
are explicit blockers below, not steps that may be silently skipped.

## Prerequisites and isolation

- A clean T01 worktree at a recorded commit; Bun 1.3.14 and Docker/Compose.
- A unique project name, for example `eden-restore-$USER-$RANDOM`, and unique
  volume names. Keep published ports on loopback and use a disposable backend.
- At least **2x the current Convex data volume bytes plus 10%** free disk for
  source volume, backup archive, and restored volume. Record `du -sb` before
  starting.
- Budget **2 CPUs, 4 GiB RAM, and 30 minutes** for the disposable backend and
  application rehearsal. A reference rebuild is a separate T03 operation;
  the reviewed production worker budget is 2 CPUs/4 GiB and must not be
  assumed as Convex restore evidence.
- A temporary mode-0600 directory outside the repository for synthetic keys,
  backup archives, logs, and evidence. Do not print key values or put them in
  command arguments.

```sh
set -euo pipefail
repo="$PWD"
project="eden-restore-${USER:-operator}-$(date +%s)-$$"
work="$(mktemp -d "${TMPDIR:-/tmp}/eden-restore.XXXXXX")"
chmod 700 "$work"
trap 'docker compose -p "$project" down -v >/dev/null 2>&1 || true; rm -rf "$work"' EXIT
(cd "$work" && bun "$repo/scripts/generate-keys.ts")
chmod 600 "$work/.env.keys"
```

Load only those synthetic values into a temporary shell environment. Set
`APP_MODE=demo`, `APP_RELEASE_ID=restore-fixture`,
`APP_CAPABILITY_ALLOWLIST=asset_search,asset_valuation,asset_sync,assistant_dispatch,session_token,identity,subscriptions`,
and an issuer such as `http://127.0.0.1:18080`. Do not use a production
callback or provider key.

## Backup rehearsal

1. Start a disposable backend with a unique Compose project and record the
   backend image digest, git commit, volume name, and container ID.

   ```sh
   export COMPOSE_PROJECT_NAME="$project"
   docker compose -p "$project" up -d backend
   volume="${project}_convex-data"
   mount="$(docker volume inspect -f '{{ .Mountpoint }}' "$volume")"
   du -sb "$mount"
   ```

2. Configure the synthetic Convex environment using the temporary key file,
   synthetic issuer/JWKS, bridge secret, and allowlist. Use the existing
   `bun run convex:configure` path only with the disposable URL/admin key;
   never pass a persistent admin key through this procedure.

3. Run the existing synthetic write smoke against this backend, with
   `SMOKE_ALLOW_WRITE=1`, and record its output without recording secrets. It
   exercises login/session, staged publication, failed publication preserving
   the previous generation, tenant isolation, JWT audience checks, and local
   revocation.

4. Create a consistent filesystem backup while the disposable backend is
   stopped. The Convex data volume contains the backend data and instance
   credential material; back up it as one unit. A host administrator may need
   root access to the Docker volume mount.

   ```sh
   docker compose -p "$project" stop backend
   tar --xattrs --acls --numeric-owner -C "$mount" \
     -czf "$work/convex-data.tar.gz" .
   sha256sum "$work/convex-data.tar.gz" > "$work/convex-data.tar.gz.sha256"
   cp --preserve=mode "$work/.env.keys" "$work/application-keys.synthetic"
   sha256sum "$work/application-keys.synthetic" > "$work/application-keys.synthetic.sha256"
   ```

   Record archive byte size, SHA-256, volume size, and stop/start timestamps.
   Do not include raw keys in the evidence bundle; retain the mode-0600 file in
   the protected temporary directory until cleanup.

## Restore and isolation checks

1. Create a second unique Compose project and volume. Extract only the
   recorded archive into the new volume, preserving ownership and xattrs.

   ```sh
   restore_project="${project}-restored"
   restore_volume="${restore_project}_convex-data"
   docker volume create "$restore_volume" >/dev/null
   restore_mount="$(docker volume inspect -f '{{ .Mountpoint }}' "$restore_volume")"
   tar --xattrs --acls --numeric-owner -C "$restore_mount" \
     -xzf "$work/convex-data.tar.gz"
   ```

2. Start the restored backend under the second project and configure the web
   container with **only** the synthetic application keys. Confirm that the
   restored instance credential is the backed-up credential, not a newly
   generated one; record only a redacted fingerprint/metadata, never its
   value.

3. Repeat the synthetic Convex smoke with fresh synthetic IDs. Verify:

   - restored schema/functions start without a migration error;
   - a newly created synthetic user cannot read the source project's records;
   - restored records remain tenant-isolated;
   - the restored reference/config volume is separate and mounted read-only by
     web;
   - the old pre-backup session/JWT is rejected after the approved invalidation
     step; and
   - the old encrypted grant cannot service synchronization until explicit
     reauthorization.

4. **Current blocker:** the repository does not yet provide a global
   post-restore session/grant invalidation command. Do not claim this check
   passed. The operator must choose and approve one of: a Convex migration that
   revokes all sessions and quarantines token records, a deployment-specific
   credential-version fence, or an equivalent bounded mechanism. Record the
   maximum stale-authority window and the decision before reopening traffic.

5. Test application rollback by stopping the restored web service, starting the
   immediately previous digest-pinned web image against the restored backend,
   and exercising read-only synthetic status/assets paths. Do not use a mutable
   `latest` tag. If the schema is not backward compatible, record rollback as
   blocked; do not mutate the source volume to make it pass.

## Migration rollback rehearsal

There is no migration history/rollback command in this baseline. A valid
rehearsal therefore requires a parent-approved migration fixture and a chosen
rollback mechanism. At minimum:

1. back up the disposable volume before migration;
2. deploy the migration to the disposable backend;
3. run schema/functions and tenant-isolation checks;
4. restore the pre-migration volume into a fresh project (never in place);
5. start the previous application image and rerun the checks; and
6. record whether the migration is backward compatible and which data loss,
   RPO, and RTO limits apply.

A `docker compose down -v` or a successful process restart is not migration
rollback evidence. T03's reference artifact rollback is a separate test and
must verify profile, importer version, manifest integrity, and live-reader
compatibility.

## Evidence template

Copy this table into the release record and attach only redacted logs/checksums:

| Field | Record |
|---|---|
| Worktree commit / image digests | |
| Source and restored Compose project names | |
| Convex source/restored volume names | |
| Synthetic key IDs/fingerprints (not values) | |
| Backup archive bytes / SHA-256 | |
| Pre/post volume byte counts | |
| Schema/migration version and rollback mechanism | |
| Session/grant invalidation decision and stale window | |
| Commands and timestamps (UTC) | |
| Synthetic smoke result | |
| Tenant-isolation result | |
| Old-session rejection result | |
| Old-grant sync rejection result | |
| Browser/provider/restore steps not exercised | |
| Cleanup confirmation and retained artifacts/expiry | |
| Operator approval and unresolved blockers | |

The evidence must label mock, fixture, disposable-backend, and consenting-live
observations separately. Passing local tests, CI, or a fixture container does
not constitute a restore, browser, provider, or production-readiness claim.