85 lines
4.7 KiB
Markdown
85 lines
4.7 KiB
Markdown
# T07 durable asset synchronization
|
|
|
|
Status: prototype implementation. This record describes the local, disposable
|
|
queue and does not claim production readiness.
|
|
|
|
## Boundaries
|
|
|
|
`POST /api/sync` authenticates the application session and only enqueues
|
|
server-derived asset jobs. It does not refresh OAuth credentials or call ESI.
|
|
Partial enqueue results carry per-character safe skip reasons; an all-failed
|
|
queue response is not reported as `202`. The assets and chat surfaces poll
|
|
`/api/sync/status?job=<id>`, expose per-character queued/terminal state and
|
|
cancellation, and rerun the report only after every queued job succeeds.
|
|
`POST /api/sync/cancel` fences a job for its owning user. The Convex bridge
|
|
checks the session proof, character ownership, active grant and exact dataset
|
|
scope again when enqueueing and claiming.
|
|
|
|
Jobs are user-owned `asset_sync` rows. The server derives the dedupe key
|
|
`user:character:assets`; only `queued` and `running` rows deduplicate. A claim
|
|
captures grant revision, authorization epoch, credential version and scopes.
|
|
Claims have random lease tokens and a ten-minute default lease, renewed every
|
|
minute by the worker. Expired claims are recovered by the next claim
|
|
transaction. Selection is bounded to 100 due rows and prefers users with no
|
|
running work before due/creation order, providing per-user fairness without a
|
|
second scheduler.
|
|
|
|
Prototype budgets are conservative and explicit (worker lease/poll values are environment-configurable): five attempts per job,
|
|
250 assets per staged chunk, 4,096 chunks, 50,000 ESI rows, at most eight
|
|
active jobs per user, a ten-minute sync lease, and a 24-hour maximum retry
|
|
delay. ESI request/page/response limits are
|
|
owned by the T06 transport. Queue backpressure is visible as a bounded failed
|
|
job, never a silently truncated generation.
|
|
|
|
## Fencing and delivery
|
|
|
|
A worker starts a generation only through the job lease, stages through a
|
|
job-token mutation, and commits success or failure through a mutation that
|
|
rechecks the lease, job fence, character authorization and generation fence.
|
|
Success and failure therefore cannot mutate state after cancellation, scope
|
|
loss, credential rotation, quarantine or lease expiry. Staging is contiguous;
|
|
identical duplicate chunk delivery is idempotent while a changed duplicate is
|
|
rejected. A successful commit is idempotent if a response is lost. Failed
|
|
snapshots preserve the previous complete generation.
|
|
|
|
`isSingleton` is validated by the T06 ESI parser and is carried unchanged in
|
|
every stored asset row; absent legacy values remain unknown.
|
|
|
|
## Refresh coordination
|
|
|
|
Refresh leases are durable per user/character rows, not process maps. The
|
|
worker decrypts a grant only inside the dedicated worker, performs exactly one
|
|
remote rotating-token refresh, then commits the replacement with the expected
|
|
credential version. The committed credential version/revision/epoch are the
|
|
only authority values used for ESI and publication. A lost, timed-out, or
|
|
otherwise uncertain refresh result is never replayed: the character becomes
|
|
`reconnect_required`, credentials are removed, active work is quarantined, and
|
|
all private datasets are blocked until explicit reconnect.
|
|
|
|
The worker receives only its dedicated `CONVEX_SYNC_WORKER_SECRET`,
|
|
token-encryption, EVE OAuth and ESI user-agent credentials. Worker mutations do
|
|
not accept the general `CONVEX_BRIDGE_SECRET`; login/session/tenant operations
|
|
remain on the web-only bridge. It does not receive application JWT signing
|
|
keys, JEV keys, reference paths/artifacts, or deployment-admin credentials. The
|
|
web process no longer performs request-bound refresh or asset paging. The image
|
|
build bundles `workers/sync.ts` to `build/sync-worker.js`, so the runtime never
|
|
executes a raw `$lib`-aliased TypeScript entry. Compose starts that artifact
|
|
through `scripts/run-sync-worker.ts`; the launcher rejects bridge, admin,
|
|
JWT-signing and JEV credentials even when `bun run sync:worker` is invoked
|
|
directly. Start it with `bun run secrets:run --worker -- docker compose --profile worker up -d
|
|
--build sync-worker`, never with the general web/deployment secret modes. The Compose
|
|
service is explicitly profiled so plain `docker compose up` cannot launch a worker
|
|
with empty credentials.
|
|
|
|
## Cooldowns and recovery
|
|
|
|
Provider cooldowns are persisted in `syncCooldowns`, keyed by character
|
|
context and shared provider group. Asset jobs carry the provider group, and
|
|
claim selection plus the worker both honor the group's eligible-at timestamp
|
|
before ESI work. ESI remains the single transport retry/cooldown
|
|
implementation. No public-feed scheduler is introduced by this task.
|
|
|
|
Retention, migrations, production quotas, remote revocation, reference
|
|
extraction and public market orchestration remain deferred as required by the
|
|
parallel-worktree plan.
|