feat: add durable asset synchronization worker
This commit is contained in:
@@ -44,11 +44,12 @@ Use [the live deployment guide](docs/live-setup.md) for `dev.evebot.app`, the re
|
||||
bunx convex dev --once
|
||||
```
|
||||
|
||||
`APP_AUTH_ISSUER`, `APP_AUTH_JWKS` and `CONVEX_BRIDGE_SECRET` must be set in **Convex's environment**, not merely the Docker process environment. The configure script handles these three. Custom JWT audience is `eve-intelligence`; the public JWKS data URI avoids a boot dependency on the web server.
|
||||
`APP_AUTH_ISSUER`, `APP_AUTH_JWKS`, `CONVEX_BRIDGE_SECRET` and `CONVEX_SYNC_WORKER_SECRET` must be set in **Convex's environment**, not merely the Docker process environment. The direct configure command reads the local private `.env`, including `CONVEX_SELF_HOSTED_ADMIN_KEY`; never put that admin key in a web or worker environment. Custom JWT audience is `eve-intelligence`; the public JWKS data URI avoids a boot dependency on the web server.
|
||||
|
||||
6. Register an EVE application and exact callback at <https://developers.eveonline.com/>. Set `EVE_CLIENT_ID`, `EVE_CALLBACK_URL`, and `EVE_CLIENT_SECRET` for a confidential web client. A public PKCE client leaves the secret blank. Initial login requests `esi-assets.read_assets.v1`; later dataset upgrades preserve only server-allowlisted existing scopes.
|
||||
7. Set `ESI_USER_AGENT` with your actual operator contact. Optional `TYPESAFE_API_KEY` enables server-side JEV routing. Set public/internal Convex URLs as described in `.env.example`.
|
||||
8. Build the fixture reference database and start `bun run dev`. Connect a character, synchronize it, and add more characters with **Connect EVE Character**. Each connects explicitly to the current application user.
|
||||
9. In a second terminal, create a private `.env.worker` containing only `CONVEX_URL`, `CONVEX_SYNC_WORKER_SECRET`, `TOKEN_ENCRYPTION_KEY`, `EVE_CLIENT_ID`, optional `EVE_CLIENT_SECRET`, `EVE_CALLBACK_URL`, `ESI_USER_AGENT`, and worker tuning values, then start the worker with `bun --env-file=.env.worker run sync:worker`. Do not source the full web `.env`; the worker launcher intentionally rejects bridge/admin/JWT/JEV credentials.
|
||||
|
||||
**Important:** the fixture DB contains only a handful of types. Live assets absent from it remain explicitly unpriced/unknown. Do not describe a fixture-backed report as a complete real-character valuation. See [full-import handoff](docs/reference-operations.md) before ingesting real releases.
|
||||
|
||||
@@ -60,10 +61,11 @@ Compose uses Convex's default SQLite backend; it introduces no other datastore.
|
||||
|
||||
- `backend`: persistent Convex SQLite volume, API bound to loopback by default (the configured Pangolin deployment explicitly overrides this).
|
||||
- `web`: Bun web service with read-only reference volume.
|
||||
- `sync-worker`: dedicated durable asset-sync worker with a bundled runtime and worker-only secret.
|
||||
- `reference-worker`: dedicated one-shot heavy-ingestion process (`tools` profile).
|
||||
- `dashboard`: optional loopback-only management UI (`admin` profile).
|
||||
|
||||
For local container use, set `APP_ORIGIN` and `APP_AUTH_ISSUER` to `http://localhost:3000` and the EVE callback to `http://localhost:3000/auth/eve/callback`. Repeat `convex:configure` after changing issuer.
|
||||
For local container use, set `APP_ORIGIN` and `APP_AUTH_ISSUER` to `http://localhost:3000` and the EVE callback to `http://localhost:3000/auth/eve/callback`. Repeat `bun run convex:configure` after changing issuer.
|
||||
|
||||
```sh
|
||||
docker compose up -d backend
|
||||
@@ -71,6 +73,8 @@ docker compose up -d backend
|
||||
# Explicit production archive import (~109 MiB for the reviewed releases):
|
||||
docker compose --profile tools run --rm --build reference-worker
|
||||
docker compose up -d --build web
|
||||
# Start the explicitly profiled durable asset worker; Compose forwards only its worker environment:
|
||||
docker compose --profile worker up -d --build sync-worker
|
||||
# Optional management UI:
|
||||
docker compose --profile admin up -d dashboard
|
||||
```
|
||||
@@ -109,9 +113,9 @@ docs/research/ current official-source evidence and captured ESI con
|
||||
|
||||
## Operational limits
|
||||
|
||||
- The configured development deployment has passed live TypeSafe routing and public Convex WebSocket checks; the operator has confirmed real EVE sign-in, asset synchronization, and matching Assets/chat valuation totals for one character. Live multi-character linking and revocation/transfer scenarios remain separate checks.
|
||||
- This checkpoint records local/disposable prototype verification only. It makes no claim of deployed/live ESI verification or production readiness; live multi-character linking, revocation and transfer scenarios remain deferred.
|
||||
- Production imports use reviewed source adapters and immutable SQLite artifacts. CCP NPC station IDs, system/type/owner/position facts are imported when `npcStations.jsonl` is present; station names remain optional and the production profile explicitly marks station coverage incomplete until an isolated measured replay. Twenty-one dangling blueprint references in the reviewed release are preserved explicitly rather than fabricated. See reference operations and the verification record for executed import checks and limitations.
|
||||
- Asset synchronization runs on demand in a single Bun web process. Convex leases/CAS protect staging and token rotation, but durable background character-sync job recovery and scheduled refresh are future work.
|
||||
- Asset synchronization is enqueued by the web process and completed by the separate durable `sync-worker`. Convex leases/CAS protect staging and token rotation; start the worker explicitly with the worker-scoped launcher before testing live synchronization.
|
||||
- ESI page collections are not transactional snapshots. Stability checks reduce risk; cache rollover/mutation may require a later retry.
|
||||
- Location IDs are shown explicitly as unresolved. Fitted/nested assets are distinct ESI items; no contents multiplier is applied to ship prices.
|
||||
- Conversations/execution summaries persist for authenticated live users; this first UI does not yet provide a history browser. A pending guest request is kept only in tab `sessionStorage` and rechecked after login/sync.
|
||||
|
||||
Reference in New Issue
Block a user