shd.build/docs/CONTENT_MODERATION.md

# Community content moderation

SHD.build applies one moderation lifecycle to public comments, profile
biographies, custom avatars, and Guides. The shared server service lives in
`src/lib/server/community-moderation.ts`. Content save actions must use that
service before publication.

Public database records also host one moderated discussion thread. The thread
uses the existing `comment` content type and accepts only visible comments from
authenticated Agents. Community moderators may pin one visible comment as the
record's Community Note. Pinning and unpinning append `community_note.pinned`
and `community_note.unpinned` audit events; a pinned comment remains ordinary
community text and follows the same removal rules as every other comment.

## Publication decisions

The moderation service combines local rules with the configured provider result:

- Clean content publishes without human review when automatic publication is
  enabled.
- Banned terms, PII checks, link limits, provider flags, and configured score
  thresholds send content to Community Oversight.
- Provider failures send content to Oversight when fail-closed behavior is
  enabled.
- An allow term can suppress the matching local term rule. It cannot override a
  provider flag or threshold.
- A user report sends visible content to Oversight. One report does not hide the
  content because reports can be mistaken or abusive.

The service records every automated scan in `content_moderation_scans`.
Automated scans do not write to the human ISAC audit log. Human policy changes
and review decisions use the shared audit utility.

## Content storage

Comments use an explicit status:

- `visible`: public and approved
- `pending_review`: hidden while Oversight decides
- `removed`: unavailable after author removal or an Oversight decision

Long-form content uses immutable pending revisions. Profile biographies keep
text candidates in `moderated_content_revisions`. Guides use the complete
structured snapshot in `article_revisions`, including title, deck, document,
tags, media references, and Build snapshots. The public article keeps its last
approved revision while another revision waits for review. Rejection retains the
submitted revision and evidence without publishing it.

News articles do not use automated content moderation. The `news.manage`
permission gates News creation, editing, publication, and deletion. Comments on
News articles remain community content and use the standard comment moderation
lifecycle.

Custom avatar uploads remain active after a clean image scan. A flagged image or
fail-closed scan error creates an `avatar_reviews` item and is unavailable
through the public avatar route until approval. An Agent report also creates a
review item, but leaves an already-published avatar visible while Oversight
decides.

## ISAC responsibilities

ISAC Site Controls stores the site-wide policy:

- provider scanning and per-content enablement;
- clean-content and provider-failure behavior;
- provider model;
- PII, link, and duplicate checks;
- category review and priority thresholds;
- allowed and banned local terms.

The term registry accepts single rules and batch imports. Batch imports use one
term or phrase per line, accept commas and semicolons as separators, infer word
or phrase matching, and add no more than 250 unique rules. Word rules match
exact forms: `kill` does not match `kills`, `killed`, or `killing`. The import
skips duplicates, records each created rule in the ISAC audit log, and leaves
the batch editor open for the next list. The registry provides server-side
search and displays 25 rules per page by default.

Each Site Controls category has its own save action. A save validates and
updates only that category, increments the moderation policy version when
applicable, and records a specific ISAC audit event with before and after
evidence. Unsaved changes in other categories remain local drafts.

Site Controls requires `site.settings.manage`. Community Oversight handles text
decisions through the existing comment and profile permissions. Avatar Oversight
handles custom images through `avatars.review`.

Moderators must record a reason when they approve, reject, remove, restore,
dismiss, or resolve a queued item. The action must append an audit event after
the content and review records change.

## User reports

Authenticated Agents can report visible comments, biographies, and avatars they
do not own. Registered and anonymous visitors can report visible Guides. A Guide
report records the exact published `article_revision_id`; a decision cannot hide
a newer revision. Anonymous reports use a random HTTP-only browser token whose
domain-separated SHA-256 hash supplies duplicate suppression. The server applies
rate limits and requests Turnstile verification after repeated anonymous
activity. SHD.build does not use device fingerprinting or retain a raw IP
address as the reporter identity.

A reviewer either dismisses the report and keeps the content or resolves the
report through a moderation action. Report details must not copy authentication
secrets or unnecessary personal information into the audit log.

## Adding a public UGC field

Guide authoring and later public text fields must complete this checklist:

1. Normalize and validate the submitted text before scanning.
2. Call `moderateCommunityText` with a supported content type.
3. Place the shared `ModerationDisclosure` beside the submission control.
4. Publish only an `approved` result.
5. Store held long-form text as an immutable revision so the last approved
   version remains public.
6. Call `recordCommunityModerationScan` with the stable content ID, exact
   article revision, author, hash, policy version, provider evidence, and
   local-rule matches.
7. Add the content type to the Community Oversight queue and permission checks.
8. Add the shared report control to the public rendering.
9. Delay follower, owner, or reply notifications until the content becomes
   public.
10. Add approval, rejection, report dismissal, author notification, and audit
    tests.
11. Confirm every public query selects approved content explicitly.

Do not add a route-specific word filter, provider client, threshold table, or
moderation status vocabulary. Extend the shared service and schema when a new
content type needs more evidence or a different review presentation.

## Operations and calibration

The API key stays in `OPENAI_MODERATION_API_KEY`. The server can use
`OPENAI_MECHANIC_API_KEY` as a temporary fallback, but production should use a
separate moderation credential.

Public submission notices link to the Terms & Privacy page instead of
summarizing provider data handling beside each form. The Terms & Privacy page
must not promise that OpenAI never retains or uses moderation inputs. It states
that OpenAI controls downstream processing under its then-current terms, data
controls, and account settings. If SHD.build enables an OpenAI data-sharing
option, update the legal copy before sending user content under that setting.

Threshold changes increment the policy version. Stored scans retain their model
and policy version so reviewers can interpret older decisions. Test threshold
changes against representative Division terminology before activation, including
weapon names, damage descriptions, kills, status effects, and build strategy
language.

Database migrations that add moderation states or revision storage follow the
repository database-preservation procedure: stop this project's development
server, verify the database path, create and verify a JSON backup, migrate,
check foreign keys and critical record counts, then restart and health-check.