Teach & Mimic — internals

The routed flows live in references/commands/teach.md, references/commands/mimic.md, and the refine section of that mimic command file. This file holds the deep material those flows link to: card anatomy, sample requirements, scoring, baselines, statistics, and failure modes.

Reconstruction runs under detection's constitution: a mimic or refined output that reintroduces slop is a failure. Every artifact here is deterministic and testable; the model is used only to generate prose, never to score itself.

Teach — the artifacts

teach distills a writer's samples into a reusable voice, stored under .unslop/voice/<name>/ (gitignored by default):

scripts/voice_profile.py samples/ -o profile.json — the machine fingerprint (char 3-grams, function-word delta, sentence-length EMD, punctuation, contractions, MTLD, word-length, impostor z-scores, GI rank). This is the referee the scorer uses.
scripts/voice_card.py --profile profile.json --samples samples/ --out . --name <name> — the layered voice card the generating model actually follows. --name labels the card (# Voice card: <name>); default voice. voice_card recomputes the profile from --samples and refuses (exit 2, named field) if the supplied --profile does not describe those samples, so a stale profile can never silently drive a card.
Add --provenance to write provenance.json (per-sample sha256, word counts, doc count, genre note, low-confidence flag) so a teach run is auditable.

Sample files. Both scripts read only .txt and .md files, recursively, under the samples directory (nothing else — a directory of .docx or extensionless files reads as empty and both scripts exit 2 with a diagnostic naming the requirement). Convert samples to .txt/.md first.

Sample requirements. Use at least 5 documents, 2-3k words total, in the same genre as the target. Fewer than ~2000 words or cross-genre samples set low_confidence in the profile and provenance; the card still builds but its claims are noisier. Surface low_confidence to the user — tell them the voice is provisional and ask for more same-genre samples before trusting a mimic. Cross-domain style attribution degrades sharply, so do not teach on blog posts and score legal memos.

The voice card (layered)

The card is a directory, not one file, so the generator loads only what the current writing task needs:

card.md — the always-loaded core, kept under 300 words: rhythm (median sentence length, IQR, burstiness), contraction habits with real examples, punctuation the writer uses and avoids, top sentence openers, a Never list of at/near-zero features, and an index table — "when writing X, read card/X.md".
card/<situation>.md — one sheet per covered situation from the taxonomy (explaining-technical, anecdote, argument, disagreement, praise, hedging-uncertainty, numbers-data, addressing-reader, openings, closings). Each sheet gives how this author handles that situation, 1-3 verbatim sample snippets, and measured markers.

No fabrication. A dimension with no sample evidence gets no sheet; it is named under "Uncovered" in card.md. The card never invents a voice the samples cannot support.

Coverage classification. voice_card.py --coverage emits a deterministic lexical coverage matrix over the taxonomy. It is intentionally coarse: it DRIVES which sheets get written and which teach prompts to ask the user, and a misclassified sentence can only add or drop a sheet — it can never change a card claim, because claims come from measured facts and verbatim snippets. So misclassification is low-stakes by construction. A sample set missing numeric writing, for instance, leaves numbers-data uncovered and named as a gap.

Coverage → prompt procedure. When a dimension the target writing will need is uncovered, ask the user for a short sample that exercises it. Only ask for dimensions the task requires — do not fish for all ten. Templates:

Dimension	Prompt to the user
explaining-technical	"Share something you wrote that explains how a thing works or why it behaves the way it does."
anecdote	"Share a few sentences where you tell a small story about something that happened to you."
argument	"Share something you wrote that argues a position — a claim you defended."
disagreement	"Share something where you pushed back on or disagreed with an idea."
praise	"Share something where you praised or recommended something you liked."
hedging-uncertainty	"Share something where you were unsure and said so — thinking out loud."
numbers-data	"Share something you wrote that works with numbers, quantities, or measurements."
addressing-reader	"Share something written directly to a reader — instructions or a note to someone."

openings and closings are structural (first/last sentence of every document) and are covered as soon as there is one document, so they need no prompt.

The card is pack-sized (~1-2k tokens loaded) so it rides in the generation context cheaply. Same inputs give byte-identical files.

`--refine` — the internal hill-climb

evals/run_mimic_refine.py iterates when one pass is not enough. It is worth the cost only when a single mimic keeps landing short of the target voice; most users teach once and mimic cheaply thereafter.

Protocol (climb on A, accept on DEV, report on a sealed split):

Split samples by document (seeded): A (~60%, retrieval/context pool) and DEV (~40%, acceptance). Fewer than 5 docs → refuse (exit 2).
LIVE (default): each iteration assembles B prompts — the draft, the A-split voice card, the k=2 nearest-A samples by char-3gram cosine, and the current directives — and invokes --generate-cmd (default claude -p) once per beam, piping the prompt on stdin and reading the candidate on stdout. --baseline zero|few|retrieval selects which samples ride in the prompt (none / first-k / k nearest-A) and generates its candidate set through this same path, so the honest retrieval-few-shot competitor and the refine loop share one pipeline.
DRY-RUN: pass --candidates-dir DIR and the loop reads DIR/iter<i>/*.md as that iteration's batch instead of generating them, so the loop logic runs with no model calls (this is how the MIMIC-* rows exercise it; the LIVE path is covered by a mock generator).
Hard gates discard a candidate regardless of score: banned-phrase clean, structure clean, draft→candidate preservation, no copy-gate violation against A, and a 150-word floor.
Survivors are scored against DEV (and recorded against A for the divergence guard) with the full voice_score composite — 0.5·(1−GI) + 0.5· clipped weighted impostor-z distance against a same-genre impostor pool (--impostors, default the committed pool), seeded. This is deliberately NOT a raw weighted distance: a marker-stuffed candidate can minimize a raw distance and clear every hard gate yet be exposed by GI, so raw-distance acceptance would accept it over honest prose. The generator never scores itself.
Accept an iteration only if the DEV composite improves by ≥ min-delta; the best-so-far never regresses.

Stopping and the divergence guard

Earliest of: iteration cap, patience (N iterations without a DEV gain), or divergence — the A composite improving while DEV worsens for two consecutive iterations. Divergence is the reward-hacking signature (the model is fitting the retrieval pool's topics, not the author's style); the loop halts immediately and sets reward_hacking_warning.

Directives and card refinement

Each iteration derives up to four ranked directives from the DEV metric deltas, and each directive carries a proposed card amendment line. The best candidate is copied to OUT/final.md and the amended card to OUT/voice-card.refined.md.

Scoring

The voice_score composite the refine loop accepts on is 0.5·(1−GI) + 0.5·zsum (lower = more author-like), where GI is the General Impostors rank — the fraction of random feature-subset trials in which the candidate beats every sampled impostor at looking like the profile — and zsum is the clipped, weighted impostor-z distance (each per-feature distance normalized by the impostor pool's mean/std, clipped to ±3, weighted by the WP10a research weights). Both halves are calibrated against a same-genre impostor pool, so the number rewards beating plausible other authors, not merely matching aggregate surface stats. That is what defeats a marker-stuffed candidate: it can drive a raw weighted distance down and still be far worse under GI. Meaning preservation vs the original draft is a separate hard gate, never blended into the composite. The composite is a guide, not an oracle — even strong authorship verification tops out near a 93-94% ceiling, so treat per-candidate deltas as the signal and under-claim.

Baselines

Score refine output against paired baselines on the same drafts: zero-shot ("write like this person"), static few-shot, and retrieval few-shot (k nearest-A samples — the honest competitor). Refine has to beat retrieval few-shot to earn its cost.

Statistics

evals/mimic_stats.py runs the small-n discipline: per-item paired deltas, a BCa bootstrap CI (n=2000, seeded), and a sign-flip permutation test (exact for ≤12 items, else 20k seeded). Claim a win only when the CI lower bound > 0 and p < 0.05. Anything degenerate (too few items, zero variance) reports not-improved.

Cost

Step	Calls
teach (profile + card)	free / deterministic
mimic	one strong generation call
refine	iterations × beam strong calls, plus free gate/score passes

Failure modes and the gate that catches each

Failure	Symptom	Caught by
Topic bleed	importing what samples are about, not how they read	divergence guard (A improves, DEV worsens)
Verbatim copying	lifting sample phrasing	copy-gate (4-gram / LCS vs A)
Register lock	latching one sample's register	DEV composite vs held-out split
Reward hacking	self-assessed quality rising while DEV falls	divergence guard + `reward_hacking_warning`
Slop reintroduced	AI tells creep back into voiced prose	banned-phrase / structure gates

For chunking long documents and the tiered execution architecture around these steps, see pipeline.md.

10 KiB Raw Blame History Unescape Escape