Add containerized staging deployment
This commit is contained in:
@@ -0,0 +1,198 @@
|
||||
# Container and staging deployment
|
||||
|
||||
SHD.build is packaged as a Node 24 container using the official SvelteKit Node
|
||||
adapter. The image contains application code, migration and notification
|
||||
scripts, and native runtime dependencies. It does not contain an environment
|
||||
file, database, backups, or user uploads.
|
||||
|
||||
## Persistent data
|
||||
|
||||
The staging compose project uses the named volume `shd-build-stage-data`,
|
||||
mounted at `/app/data`. It contains:
|
||||
|
||||
- `shd-build-staging.db` and its embedded-database support files;
|
||||
- timestamped JSON backups under `backups/`;
|
||||
- normalized user avatars under `avatars/`;
|
||||
- uploaded game-data icons under `game-icons/`.
|
||||
|
||||
Never remove this volume during routine deployments. `docker compose down`
|
||||
without `--volumes` preserves it. Before any manual backup or restore, stop
|
||||
both the `app` and `scheduler` services because both access the embedded
|
||||
database.
|
||||
|
||||
## Staging environment
|
||||
|
||||
Staging defaults to:
|
||||
|
||||
- Public origin: `https://stage.shd.build`
|
||||
- Reverse-proxy upstream: `http://127.0.0.1:5569`
|
||||
- Database: `/app/data/shd-build-staging.db`
|
||||
- Passkey relying-party ID: `shd.build`
|
||||
- Adapter canonical origin: `https://stage.shd.build`
|
||||
|
||||
Edit its encrypted configuration locally:
|
||||
|
||||
```sh
|
||||
npm run env:decrypt:staging
|
||||
# Fill in Discord and SMTP values.
|
||||
npm run env:encrypt:staging
|
||||
```
|
||||
|
||||
Add `https://stage.shd.build/auth/discord/callback` to the Discord application's
|
||||
allowed OAuth redirect URIs. Point the staging DNS record at the deployment
|
||||
host and configure the host's TLS reverse proxy to forward
|
||||
`stage.shd.build` to `127.0.0.1:5569`.
|
||||
|
||||
The deployment decrypts `.env.staging` on the trusted staging runner into a
|
||||
mode-`0600` runtime file. The decryption key is held by Gitea Actions, not
|
||||
committed or included in the container image.
|
||||
|
||||
The staging container disables default-user seeding. On a brand-new staging
|
||||
database, the existing bootstrap policy grants Admin to the first real account
|
||||
that signs in instead of writing generated credentials into deployment logs.
|
||||
|
||||
## Gitea Actions runner
|
||||
|
||||
Gitea Actions requires an `act_runner`. Register a trusted runner on the
|
||||
staging host from the repository's **Settings → Actions → Runners** page. The
|
||||
host needs Node/npm, Docker Engine, and Docker Compose v2. The runner account
|
||||
must be able to use Docker and write to the deployment directory.
|
||||
|
||||
Use runner labels that provide an isolated validation job and a host deployment
|
||||
job:
|
||||
|
||||
```text
|
||||
ubuntu-latest:docker://node:24-bookworm,stage-deploy:host
|
||||
```
|
||||
|
||||
The `stage-deploy:host` label is intentional: the deployment step needs the
|
||||
staging host's Docker daemon and persistent named volume. Only attach this
|
||||
trusted runner to repositories allowed to deploy to that host. Gitea warns that
|
||||
Docker daemon access is effectively host-level authority.
|
||||
|
||||
Prepare the default deployment directory:
|
||||
|
||||
```sh
|
||||
sudo install -d -o act_runner -g act_runner -m 0750 /opt/shd-build-stage
|
||||
sudo usermod -aG docker act_runner
|
||||
```
|
||||
|
||||
Restart the runner session after adding it to the Docker group.
|
||||
|
||||
## Repository Actions configuration
|
||||
|
||||
In `https://git.jb.codes/jb/shd.build/settings/actions`:
|
||||
|
||||
1. Add repository secret `DOTENV_PRIVATE_KEY_STAGING`, using the matching value
|
||||
from the ignored local `.env.keys`.
|
||||
2. Optionally add repository variable `STAGE_DEPLOY_PATH`. It defaults to
|
||||
`/opt/shd-build-stage`.
|
||||
3. Optionally add repository variable `STAGE_PORT`. It defaults to `5569`.
|
||||
4. Ensure Actions are enabled and the `stage-deploy` runner is online.
|
||||
|
||||
The workflow at `.gitea/workflows/deploy-staging.yml` runs on every push to
|
||||
`main` and can also be started manually. It does not need registry credentials
|
||||
because the trusted staging runner builds a commit-tagged image locally.
|
||||
|
||||
## Deployment lifecycle
|
||||
|
||||
The workflow:
|
||||
|
||||
1. installs dependencies, runs `npm run check`, and produces the Node build;
|
||||
2. builds the candidate container while the current site remains online;
|
||||
3. stops only the staging application and notification scheduler;
|
||||
4. creates a timestamped JSON backup if the staging database exists;
|
||||
5. applies additive, repeatable migrations;
|
||||
6. starts the application and scheduler with the new image;
|
||||
7. waits for `/api/health/database` to report healthy;
|
||||
8. retains the logs and exits unsuccessfully if health verification fails.
|
||||
|
||||
The short outage is limited to backup, migration, and container startup. This
|
||||
follows the database-preservation policy while minimizing data-entry downtime.
|
||||
The scheduler runs automated system messages, Oversight consolidation, and
|
||||
email digest checks once per minute.
|
||||
|
||||
## Initial development-data bootstrap
|
||||
|
||||
The first staging release can be initialized from the current development
|
||||
database, including users, roles, builds, guides, proposals, audit history,
|
||||
avatars, and uploaded game-data icons. This is deliberately a one-time flow:
|
||||
the staging importer refuses to run after users, builds, audits, or proposals
|
||||
exist in its target database.
|
||||
|
||||
First stop this project's development server. The embedded database must not be
|
||||
open while its complete transfer bundle is created:
|
||||
|
||||
```sh
|
||||
npm run stage:bootstrap:export
|
||||
```
|
||||
|
||||
The command writes a timestamped, ignored JSON bundle under `data/backups/`.
|
||||
It contains user records and password hashes, so transfer it over an encrypted
|
||||
channel and never add it to Git. Copy it to the staging host, then run:
|
||||
|
||||
```sh
|
||||
/opt/shd-build-stage/bootstrap-staging.sh \
|
||||
/secure/path/shd-stage-bootstrap-YYYY-MM-DDTHH-MM-SSZ.json
|
||||
```
|
||||
|
||||
The bootstrap flow stops the app and scheduler, creates a safety backup of the
|
||||
target, verifies that staging is unused, imports the entire development
|
||||
database and uploads, and restarts both services. Delete the transfer copy from
|
||||
the staging host after verifying the imported record counts and critical
|
||||
entities. The copy under the local ignored backup directory remains the
|
||||
recovery source.
|
||||
|
||||
Because this is a full initial clone, development session tokens and account
|
||||
data also enter staging. Staging and development remain independent from that
|
||||
point forward.
|
||||
|
||||
## Pulling canonical ISAC data into development
|
||||
|
||||
Use the manually triggered **Export staging ISAC data** workflow in Gitea
|
||||
Actions. It briefly stops staging database writers, exports only canonical
|
||||
builder-source tables plus uploaded game-data icons, restarts staging, and
|
||||
publishes a `shd-stage-isac-data` artifact for 14 days.
|
||||
|
||||
Download `isac-stage.json` from that workflow run. Then stop this project's
|
||||
development server and run:
|
||||
|
||||
```sh
|
||||
npm run isac:import -- /path/to/isac-stage.json --confirm-replace-isac-data
|
||||
```
|
||||
|
||||
The importer automatically creates both a complete development database backup
|
||||
and an ISAC-only snapshot before replacing canonical data. It does **not**
|
||||
replace local users, roles, sessions, builds, guides, comments, notifications,
|
||||
proposals, contributor agreements, audit history, or system-message settings.
|
||||
Restart development only after the command reports both backup paths and the
|
||||
imported row/icon counts.
|
||||
|
||||
For a local read-only snapshot without importing:
|
||||
|
||||
```sh
|
||||
npm run isac:export
|
||||
```
|
||||
|
||||
## Manual staging operations
|
||||
|
||||
From the deployment directory:
|
||||
|
||||
```sh
|
||||
docker compose --env-file .env.deploy -f compose.staging.yml ps
|
||||
docker compose --env-file .env.deploy -f compose.staging.yml logs -f app
|
||||
docker compose --env-file .env.deploy -f compose.staging.yml stop scheduler app
|
||||
docker compose --env-file .env.deploy -f compose.staging.yml run --rm --no-deps app node scripts/db-backup.mjs
|
||||
docker compose --env-file .env.deploy -f compose.staging.yml up -d app scheduler
|
||||
```
|
||||
|
||||
Do not use `docker compose down --volumes` for staging. That deletes the named
|
||||
volume containing the embedded database and uploads.
|
||||
|
||||
## Production promotion
|
||||
|
||||
Production should use a separate compose project, volume, database filename,
|
||||
runtime environment, port, and deployment workflow. Do not point staging and
|
||||
production at the same embedded database or storage volume. Once staging is
|
||||
proven, the same container image can be tagged for production so promotion does
|
||||
not rebuild application code.
|
||||
Reference in New Issue
Block a user