shd.build/docs/CONTENT_MODERATION.md

Community content moderation

SHD.build applies one moderation lifecycle to public comments, profile biographies, custom avatars, and Guides. The shared server service lives in src/lib/server/community-moderation.ts. Content save actions must use that service before publication.

Public database records also host one moderated discussion thread. The thread uses the existing comment content type and accepts only visible comments from authenticated Agents. Community moderators may pin one visible comment as the record's Community Note. Pinning and unpinning append community_note.pinned and community_note.unpinned audit events; a pinned comment remains ordinary community text and follows the same removal rules as every other comment.

Publication decisions

The moderation service combines local rules with the configured provider result:

• Clean content publishes without human review when automatic publication is enabled.
• Banned terms, PII checks, link limits, provider flags, and configured score thresholds send content to Community Oversight.
• Provider failures send content to Oversight when fail-closed behavior is enabled.
• An allow term can suppress the matching local term rule. It cannot override a provider flag or threshold.
• A user report sends visible content to Oversight. One report does not hide the content because reports can be mistaken or abusive.

The service records every automated scan in content_moderation_scans. Automated scans do not write to the human ISAC audit log. Human policy changes and review decisions use the shared audit utility.

Content storage

Comments use an explicit status:

• visible: public and approved
• pending_review: hidden while Oversight decides
• removed: unavailable after author removal or an Oversight decision

Long-form content uses immutable pending revisions. Profile biographies keep text candidates in moderated_content_revisions. Guides use the complete structured snapshot in article_revisions, including title, deck, document, tags, media references, and Build snapshots. The public article keeps its last approved revision while another revision waits for review. Rejection retains the submitted revision and evidence without publishing it.

News articles do not use automated content moderation. The news.manage permission gates News creation, editing, publication, and deletion. Comments on News articles remain community content and use the standard comment moderation lifecycle.

Custom avatar uploads remain active after a clean image scan. A flagged image or fail-closed scan error creates an avatar_reviews item and is unavailable through the public avatar route until approval. An Agent report also creates a review item, but leaves an already-published avatar visible while Oversight decides.

ISAC responsibilities

ISAC Site Controls stores the site-wide policy:

• provider scanning and per-content enablement;
• clean-content and provider-failure behavior;
• provider model;
• PII, link, and duplicate checks;
• category review and priority thresholds;
• allowed and banned local terms.

The term registry accepts single rules and batch imports. Batch imports use one term or phrase per line, accept commas and semicolons as separators, infer word or phrase matching, and add no more than 250 unique rules. Word rules match exact forms: kill does not match kills, killed, or killing. The import skips duplicates, records each created rule in the ISAC audit log, and leaves the batch editor open for the next list. The registry provides server-side search and displays 25 rules per page by default.

Each Site Controls category has its own save action. A save validates and updates only that category, increments the moderation policy version when applicable, and records a specific ISAC audit event with before and after evidence. Unsaved changes in other categories remain local drafts.

Site Controls requires site.settings.manage. Community Oversight handles text decisions through the existing comment and profile permissions. Avatar Oversight handles custom images through avatars.review.

Moderators must record a reason when they approve, reject, remove, restore, dismiss, or resolve a queued item. The action must append an audit event after the content and review records change.

User reports

Authenticated Agents can report visible comments, biographies, and avatars they do not own. Registered and anonymous visitors can report visible Guides. A Guide report records the exact published article_revision_id; a decision cannot hide a newer revision. Anonymous reports use a random HTTP-only browser token whose domain-separated SHA-256 hash supplies duplicate suppression. The server applies rate limits and requests Turnstile verification after repeated anonymous activity. SHD.build does not use device fingerprinting or retain a raw IP address as the reporter identity.

A reviewer either dismisses the report and keeps the content or resolves the report through a moderation action. Report details must not copy authentication secrets or unnecessary personal information into the audit log.

Adding a public UGC field

Guide authoring and later public text fields must complete this checklist:

1. Normalize and validate the submitted text before scanning.
2. Call moderateCommunityText with a supported content type.
3. Place the shared ModerationDisclosure beside the submission control.
4. Publish only an approved result.
5. Store held long-form text as an immutable revision so the last approved version remains public.
6. Call recordCommunityModerationScan with the stable content ID, exact article revision, author, hash, policy version, provider evidence, and local-rule matches.
7. Add the content type to the Community Oversight queue and permission checks.
8. Add the shared report control to the public rendering.
9. Delay follower, owner, or reply notifications until the content becomes public.
10. Add approval, rejection, report dismissal, author notification, and audit tests.
11. Confirm every public query selects approved content explicitly.

Do not add a route-specific word filter, provider client, threshold table, or moderation status vocabulary. Extend the shared service and schema when a new content type needs more evidence or a different review presentation.

Operations and calibration

The API key stays in OPENAI_MODERATION_API_KEY. The server can use OPENAI_MECHANIC_API_KEY as a temporary fallback, but production should use a separate moderation credential.

Public submission notices link to the Terms & Privacy page instead of summarizing provider data handling beside each form. The Terms & Privacy page must not promise that OpenAI never retains or uses moderation inputs. It states that OpenAI controls downstream processing under its then-current terms, data controls, and account settings. If SHD.build enables an OpenAI data-sharing option, update the legal copy before sending user content under that setting.

Threshold changes increment the policy version. Stored scans retain their model and policy version so reviewers can interpret older decisions. Test threshold changes against representative Division terminology before activation, including weapon names, damage descriptions, kills, status effects, and build strategy language.

Database migrations that add moderation states or revision storage follow the repository database-preservation procedure: stop this project's development server, verify the database path, create and verify a JSON backup, migrate, check foreign keys and critical record counts, then restart and health-check.