shd.build/docs/DEPLOYMENT.md

Container, staging, and production deployment

SHD.build runs in a Node 24 container through the official SvelteKit Node adapter. The image includes application code, migration and notification scripts, and native runtime dependencies. Environment files, databases, backups, and user uploads remain outside the image.

Persistent data

The staging compose project uses the named volume shd-build-stage-data, mounted at /app/data. It contains:

• shd-build-staging.db and its embedded-database support files;
• timestamped JSON backups under backups/;
• normalized user avatars under avatars/;
• uploaded game-data icons under game-icons/.

Keep this volume during routine deployments. docker compose down without --volumes preserves it. Before a manual backup or restore, stop the app and scheduler services because both access the embedded database.

Staging environment

Staging defaults to:

• Public origin: https://stage.shd.build
• Reverse-proxy upstream: host port 5569
• Database: /app/data/shd-build-staging.db
• Passkey relying-party ID: shd.build
• Adapter canonical origin: https://stage.shd.build

Edit its encrypted configuration locally:

npm run env:decrypt:staging
# Fill in Discord and SMTP values.
npm run env:encrypt:staging

Add https://stage.shd.build/auth/discord/callback to the Discord application's allowed OAuth redirect URIs. Point the staging DNS record at the deployment host and configure the host's TLS reverse proxy to forward stage.shd.build to port 5569 on the staging host. Docker publishes this port on the host interfaces so a containerized reverse proxy or tunnel can reach it; firewall the port to trusted LAN/tunnel sources when the host is reachable from other networks.

The trusted staging runner decrypts .env.staging into a mode-0600 runtime file. Gitea Actions holds the decryption key; Git and the container image do not.

The staging container disables default-user seeding. On a brand-new staging database, the existing bootstrap policy grants Admin to the first real account that signs in instead of writing generated credentials into deployment logs.

Gitea Actions runner

Gitea Actions requires an act_runner. Register a trusted runner on the staging host from the repository's Settings → Actions → Runners page. The host needs Node/npm, Docker Engine, and Docker Compose v2. The runner account must be able to use Docker and write to the deployment directory.

Use runner labels that provide an isolated validation job and a host deployment job:

ubuntu-latest:docker://node:24-bookworm,stage-deploy:host

The stage-deploy:host label is intentional: the deployment step needs the staging host's Docker daemon and persistent named volume. Only attach this trusted runner to repositories allowed to deploy to that host. Gitea warns that Docker daemon access is effectively host-level authority.

Prepare the default deployment directory:

sudo install -d -o act_runner -g act_runner -m 0750 /opt/shd-build-stage
sudo usermod -aG docker act_runner

Restart the runner session after adding it to the Docker group.

Repository Actions configuration

In https://git.jb.codes/jb/shd.build/settings/actions:

1. Add repository secret DOTENV_PRIVATE_KEY_STAGING, using the matching value from the ignored local .env.keys.
2. Optionally add repository variable STAGE_DEPLOY_PATH. It defaults to /opt/shd-build-stage.
3. Optionally add repository variable STAGE_PORT. It defaults to 5569.
4. Ensure Actions are enabled and the stage-deploy runner is online.

.gitea/workflows/deploy-staging.yml runs for pushes to main and manual dispatches. The trusted staging runner builds each commit-tagged image locally, so the workflow needs no registry credentials. Gitea assigns gitea.run_number to SHD_BUILD_NUMBER; each new workflow run receives the next integer, while retries of the same run retain their number.

Deployment lifecycle

The workflow:

1. installs dependencies, runs npm run check, and produces the Node build;
2. builds the candidate container while the current site remains online;
3. stops only the staging application and notification scheduler;
4. creates a timestamped JSON backup if the staging database exists;
5. applies additive, repeatable migrations;
6. starts the application and scheduler with the new image;
7. waits for /api/health/database to report healthy;
8. retains the logs and exits unsuccessfully if health verification fails.

The outage covers backup, migration, and container startup. Migrations may also remove records that use retired data formats when a release explicitly ends compatibility. The deployment backup remains the recovery source for those records; the application does not repair or reinterpret them at runtime. The scheduler calls a bearer-authenticated endpoint on the application service once per minute. Scheduled message delivery therefore uses the web process's existing embedded-database connection instead of opening the locked database from a second process. The endpoint runs automated system messages, Oversight consolidation, and email digest checks and is not published through a host port.

Initial development-data bootstrap

Initialize the first staging release from the development database, including users, roles, builds, guides, proposals, audit history, avatars, and uploaded game-data icons. The staging importer refuses a target database that contains users, builds, audits, or proposals. It also requires the source and target table sets to match exactly; snapshots with retired or unknown tables are rejected.

First stop this project's development server. The embedded database must not be open while its complete transfer bundle is created:

npm run stage:bootstrap:export

The command writes a timestamped, ignored JSON bundle under data/backups/. The bundle includes user records and password hashes. Transfer it over an encrypted channel, keep it out of Git, then copy it to the staging host and run:

/opt/shd-build-stage/bootstrap-staging.sh \
  /secure/path/shd-stage-bootstrap-YYYY-MM-DDTHH-MM-SSZ.json

The bootstrap flow stops the app and scheduler, creates a safety backup of the target, verifies that staging is unused, imports the entire development database and uploads, and restarts both services. Delete the transfer copy from the staging host after verifying the imported record counts and critical entities. The copy under the local ignored backup directory remains the recovery source.

Because this is a full initial clone, development session tokens and account data also enter staging. Staging and development remain independent from that point forward.

Pulling canonical ISAC data into development

Use the manually triggered Export staging ISAC data workflow in Gitea Actions. It briefly stops staging database writers, exports only canonical builder-source tables plus uploaded game-data icons, restarts staging, and publishes a shd-stage-isac-data artifact for 14 days.

Download isac-stage.json from that workflow run, stop this project's development server, and run:

npm run isac:import -- /path/to/isac-stage.json --confirm-replace-isac-data

The importer automatically creates both a complete development database backup and an ISAC-only snapshot before replacing canonical data. It does not replace local users, roles, sessions, builds, guides, comments, notifications, proposals, contributor agreements, audit history, or system-message settings. Restart development only after the command reports both backup paths and the imported row/icon counts.

For a local read-only snapshot without importing:

npm run isac:export

Manual staging operations

From the deployment directory:

docker compose --env-file .env.deploy -f compose.staging.yml ps
docker compose --env-file .env.deploy -f compose.staging.yml logs -f app
docker compose --env-file .env.deploy -f compose.staging.yml stop scheduler app
docker compose --env-file .env.deploy -f compose.staging.yml run --rm --no-deps app node scripts/db-backup.mjs
docker compose --env-file .env.deploy -f compose.staging.yml up -d app scheduler

Do not use docker compose down --volumes for staging. That deletes the named volume containing the embedded database and uploads.

Production promotion

Production uses a separate compose project, volume, database filename, runtime environment, port, and deployment workflow:

• Public origin: https://shd.build
• Reverse-proxy upstream: host port 3102
• Deployment directory: /opt/shd-build-production
• Database: /app/data/shd-build-production.db
• Persistent volume: shd-build-production-data

Routine pushes leave production data unchanged. The production deploy stops its application and scheduler, creates a timestamped JSON backup, applies repeatable migrations, restarts, and verifies database health. Keep docker compose down --volumes away from production.

Add these repository Action secrets:

• DOTENV_PRIVATE_KEY_PRODUCTION, matching the ignored local .env.keys
• SCHEDULER_TOKEN_PRODUCTION, a long random value used only by the internal scheduler endpoint

Optional repository variables are PRODUCTION_DEPLOY_PATH (default /opt/shd-build-production) and PRODUCTION_PORT (default 3102). The deploy-production.yml workflow deploys production on pushes to main and manual dispatches using the trusted stage-deploy host runner. Its Gitea workflow run number supplies the production SHD_BUILD_NUMBER.

Prepare the host directory once from an interactive SSH session:

sudo install -d -o jb -g jb -m 0750 /opt/shd-build-production

Generate the scheduler secret locally without printing it into a shared log, then enter it directly in Gitea's repository-secret form. Do not commit it or reuse the staging scheduler token.

Initial production bootstrap

The production bootstrap clones current development data and uploads. Stop this project's development server before creating the bundle:

npm run production:bootstrap:export

Copy the resulting ignored JSON bundle to the deployment host over SSH, then:

/opt/shd-build-production/bootstrap-production.sh \
  /secure/path/shd-production-bootstrap-YYYY-MM-DDTHH-MM-SSZ.json

The importer verifies the bundle target, creates a production safety backup, and refuses to replace a production database containing users, builds, audit events, or proposals. Remove the remote transfer copy after verification.

Pull production ISAC data into development

Manually run Export production ISAC data in Gitea Actions and download its shd-production-isac-data artifact. Stop the local development server, then:

npm run isac:import -- /path/to/isac-production.json --confirm-replace-isac-data

The importer creates a complete local database backup and an ISAC-only snapshot before replacing canonical game data. It leaves accounts, content, proposals, notifications, settings, and audit history untouched.

Pull production ISAC data into staging

Manually run Replace staging ISAC data from production in Gitea Actions. The workflow exports a consistent production snapshot, restarts production, then stops staging and imports it. The staging importer creates full and ISAC-only safety backups before replacement. Deployments do not run this flow.

Promote managed images without replacing other data

Image Library assets use stable IDs and can be promoted additively from local development. Export a bundle after finishing local uploads and metadata edits:

Set the server-only REMOVE_BG_API_KEY in each encrypted environment that accepts ISAC uploads. It is sent only from the server in the X-Api-Key header. Transparent icon uploads skip remove.bg. Opaque sources use its preview output, then pass through local trimming, white normalization, WebP conversion, and storage.

The application stores a remove.bg capacity snapshot and refreshes it from the internal scheduler. REMOVE_BG_CREDIT_CHECK_HOURS controls the account lookup interval and defaults to six hours. Low-capacity alerts require both remaining pools to cross their thresholds: REMOVE_BG_LOW_CREDIT_THRESHOLD defaults to five paid credits, and REMOVE_BG_LOW_FREE_CALL_THRESHOLD defaults to ten free calls. Alerts are mandatory ISAC notifications for active Administrators and are deduplicated for each warning or critical episode.

npm run images:export

The command writes a timestamped JSON bundle under data/backups/. It contains only managed-image metadata and processed WebP files. Transfer that bundle to the deployment host, stop the target app and scheduler services so the embedded database is unlocked, and run the merge using the target image:

# Staging
cd /opt/shd-build-stage
docker compose -f compose.staging.yml stop scheduler app
docker compose -f compose.staging.yml run --rm \
  -v /secure/path:/import:ro app \
  node scripts/managed-images-import.mjs /import/managed-images.json --confirm-merge
docker compose -f compose.staging.yml up -d app scheduler

# Production
cd /opt/shd-build-production
docker compose -f compose.production.yml stop scheduler app
docker compose -f compose.production.yml run --rm \
  -v /secure/path:/import:ro app \
  node scripts/managed-images-import.mjs /import/managed-images.json --confirm-merge
docker compose -f compose.production.yml up -d app scheduler

Each import creates a full target database backup before changes. The importer adds new stable IDs, skips identical assets, and blocks conflicting records. After reviewing an intentional metadata or file replacement, add --update-existing. Imported changes append image.imported or image.import_updated audit events. Remove the transferred bundle after verifying both environments because it may contain unpublished visual assets and creator identifiers.

The .env.staging and .env.production npm import commands are intended for maintenance performed directly in an environment where those database and storage paths are mounted:

npm run images:import:staging -- /path/to/bundle.json --confirm-merge
npm run images:import:production -- /path/to/bundle.json --confirm-merge