global setup and configuration.

This commit is contained in:
jb
2026-08-24 16:33:11 -04:00
parent 10f64b593b
commit 31c9363dbb
95 changed files with 12641 additions and 41 deletions
-4
View File
@@ -1,4 +0,0 @@
# Ignore everything in this directory
*
# Except this .gitignore file
!.gitignore
+6 -4
View File
@@ -46,14 +46,16 @@ npm run typecheck
Extensions are TypeScript modules loaded directly by Pi; no build step is required. Extensions are TypeScript modules loaded directly by Pi; no build step is required.
For live development, the global extension entries can point at this checkout: Repository-managed global instructions, skills, and extension exposure are declared in [`config/pi-links.json`](config/pi-links.json) and reconciled by one script:
```bash ```bash
ln -s /absolute/path/to/harness-tools/extensions/effort-controls ~/.pi/agent/extensions/effort-controls ./scripts/pi-links.mjs check
ln -s /absolute/path/to/harness-tools/extensions/update ~/.pi/agent/extensions/update ./scripts/pi-links.mjs sync --dry-run
./scripts/pi-links.mjs sync
./scripts/pi-links.mjs prune --dry-run
``` ```
This checkout is already linked that way. Run `/reload` after editing an extension. The repository-local `manage-pi-links` skill applies this workflow. Global instructions are linked to Pi and Codex as `AGENTS.md`; Claude receives a `CLAUDE.md` that imports the shared instructions. The manager preserves unrelated global resources and refuses unmanaged destination collisions. Run `/reload` after changing active Pi instruction, skill, or extension exposure.
## License ## License
+30
View File
@@ -0,0 +1,30 @@
{
"version": 1,
"skills": {
"root": "skills",
"projectOnly": {
"nono-profiles": "skills/tools/nono-profiles",
"manage-pi-links": "skills/tools/manage-pi-links"
},
"excluded": [
"pptx",
"stop-slop",
"subagents",
"xlsx"
]
},
"instructions": {
"globalSymlinks": {
"pi": "docs/global.AGENTS.md",
"codex": "docs/global.AGENTS.md",
"claude": "docs/global.CLAUDE.md"
}
},
"extensions": {
"globalSymlinks": {
"effort-controls": "extensions/effort-controls",
"update": "extensions/update"
},
"globalLocalPackages": {}
}
}
+101
View File
@@ -0,0 +1,101 @@
# Global Agent Instructions
## Introduction
I'm JB, you are my agent. Here is an introduction to myself and my preferences.
I'm a software developer both professionally and for fun. At work, I've worked in the video game industry on building an MMORPG and since then have worked on mobile and web applications for commercial software and more recently working for defense contracts. In my personal work, I do most of my development in building tools, like discord bots and websites, that help me and my friends in the games we're playing.
Below you will find my preferences so that we can be more aligned as we work together to develop software.
## Development Philosophy
- KISS - Keep It Simple Stupid. I prefer simple and maintainable solutions.
- Name variables and functions for what they do.
- Classes should mostly serve one job.
- Add comments to add clarity for complex or abstract concepts in code if needed, but don't comment every line. Comments to describe how functions are used above the function or classes, etc are useful.
- Cleanup as you go, remove dead code, unused parameters and dead branches of code.
- DRY - Don't Repeat Yourself. Apply this with discretion and where KISS takes precedence.
- Write maintainable code, a human will need to read and edit this code at some point.
- Write readable code, other humans and agents will read it later.
- Be pragmatic in applying DRY principles, balance duplication against overly complex abstraction.
- DO NOT add overly complex abstraction for the sake of not repeating code.
- YAGNI - You Aren't Gonna Need It. Build what we need now, not what we might need in the future.
- Speculative code can be expensive and predictions are most likely wrong, let's keep on task and just get what we need now done.
- SOLID - Single Responsibility, Open/Closed, Liskov Substitution, Interface Segregation, Dependency Inversion
- I prefer immutable functions and Functional Programming over Object Oriented Programming.
- Apply KISS, DRY principles first, if functional wins, use that otherwise use OOP.
- Classes are good when they are simple.
- Prefer established project patterns, do not refactor or rewrite code in shared codebases to fit my preferences.
- Whenever making changes investigate for the broader picture and potential side-effects to the change you are about to make.
## Questions are read-only
- If I ask a question, I'm just looking for an answer, not for you to make changes. 
- If the answer is simple and trivial, answer first and propose the change.
- Always ask before making any changes when answering a question.
## Coding Preferences (TypeScript)
- Avoid `any`, infer types to let systems adapt to changes.
- TypeScript should look like TypeScript, not Python.
- Don't just add one-line cast-wrapper functions.
- If not specified in the project, I like using Svelte, bun, Vite
- For complex projects, I like to use SvelteKit.
- Electrobun for desktop applications.
## Git
- I prefer a rebase over a merge when working on my feature branches.
- fetch the latest from main (or other source branch if working off a shared feature branch) daily and rebase my code on top.
- Keep commits focused.
- Do not commit broad formatting changes unless I explicitly have asked for that.
- Use worktrees for new feature work to work in isolation and prevent other parallel development processes on other features from stepping on each other. Use `./.worktrees/` to store the worktree within its repository.  Remove them when the feature is complete.
## Dependencies
- Do not add new dependencies unnecessarily.
- Remove unused dependencies judiciously.
- Prefer existing project dependencies.
- If it's simple enough, just write the code rather than introduce a new dependency.
## Testing
- Always write tests for any changed behavior.
- Tests should be focused. Endless smoke tests, or regression tests for feature removals, are not as useful.
- Always run tests before commits and do not report them successful unless all tests have actually executed.
- E2E tests must be written and validate with realistic data. Garbage In, Garbage Out. Test data needs to emulate, or be, real data.
## Code Review
- The agent that wrote the code **SHOULD NEVER BE** the agent that reviews the code.
- UI changes must be validated in a browser or the application itself. Don't just trust it looks right.
## Documentation
- Keep documentation synchronized with behavioral changes. This includes both comments in code and any markdown documentation, typically in `./docs/`.
- ADRs - When decisions are made for new features and those questions answered, we should create an Architectural Design Record in clean simple markdown in the repositories documentation directory. Typical preferred location would be `./docs/adr/<feature/decision-name>-<date (22AUG2026)>.md`.
- ADRs should be functionally immutable, but not read-only. If a new decision invalidates a previous decisions, update that ADR with a link to the new ADR.
- Follow the guidelines in the [`writing-for-agents`](../skills/document/writing-for-agents/SKILL.md) skill when creating documentation.
## Visual &amp; Design Work
- Do not edit real components first. If the work is non-trivial UI changes, layout, or copy, build a few distinct static mocks, and ready them for manual review and wait for my feedback before implementing.
- I prefer dark mode first. Minimal copy. Minimal chrome. No em dashes. Don't add meaningless text and measurements, everything should have a purpose.
- Avoid any continuously repainting CSS animations that could increase GPU issues on high-refresh displays.
- Always design responsively. Designs should look good on both mobile and desktop. Desktops should look good from 1080p to 4k.
## Agent Behavior
- Research the code and investigate before asking questions that could be answered within the repository. If you're unsure, check the docs and ADRs.
- Do not make any speculative changes. Ask to confirm.
- Keep the context usage as efficient as possible.
- Plan before we implement. Investigate the codebase, do research, and grill me with questions. I'd rather spend some time answering questions first and get a result I am happy with rather than something we'd have to do again.
- Prototype or create PoCs for large features first. If we can identify if something will or will not work early, that will save time in the long run.
- Do not touch production or live databases unless explicitly instructed to do so.
## Subagents and Parallel work
- Not every ask needs a subagent. Do not spawn up subagents or multi-agent workflows for something a single agent can finish in a single pass. Delegate appropriately for adversarial review, or specialized parallel work.
- When we have several agents working together on a feature, state file ownership up front so they do not collide.
+1
View File
@@ -0,0 +1 @@
@/srv/data/code/harness-tools/docs/global.AGENTS.md
+3 -2
View File
@@ -13,8 +13,9 @@
"scripts": { "scripts": {
"prepare": "effect-tsgo patch", "prepare": "effect-tsgo patch",
"typecheck": "tsc --noEmit", "typecheck": "tsc --noEmit",
"test": "npm run test:runner && node --test --experimental-strip-types extensions/firecrawl-search/index.test.ts extensions/shared/*.test.ts extensions/subagents/by-the-way.test.ts extensions/subagents/context-usage.test.ts extensions/subagents/manager.test.ts extensions/subagents/result-delivery.test.ts extensions/subagents/takeover.test.ts extensions/workflows/*.test.ts && vitest run extensions/file-search/index.spec.ts", "test": "npm run test:runner && npm run test:links && node --test --experimental-strip-types extensions/firecrawl-search/index.test.ts extensions/shared/*.test.ts extensions/subagents/by-the-way.test.ts extensions/subagents/context-usage.test.ts extensions/subagents/manager.test.ts extensions/subagents/result-delivery.test.ts extensions/subagents/takeover.test.ts extensions/_workflows/*.test.ts && vitest run extensions/file-search/index.spec.ts",
"test:runner": "python -m unittest scripts/test_orca_nono_profile.py", "test:runner": "python -m unittest discover -s scripts -p 'test_*.py'",
"test:links": "node --test scripts/pi-links.test.mjs",
"test:live": "node --test --experimental-strip-types extensions/subagents/claude.test.ts extensions/subagents/codex.test.ts" "test:live": "node --test --experimental-strip-types extensions/subagents/claude.test.ts extensions/subagents/codex.test.ts"
}, },
"pi": { "pi": {
+6 -1
View File
@@ -13,7 +13,12 @@
"filesystem": { "filesystem": {
"allow": [ "allow": [
"$WORKDIR", "$WORKDIR",
"$TMPDIR" "$TMPDIR",
"$HOME/.pi-lens"
],
"read_file": [
"$HOME/.local/bin/claude",
"$HOME/.local/bin/codex"
], ],
"read": [ "read": [
"$HOME/.bun", "$HOME/.bun",
+635
View File
@@ -0,0 +1,635 @@
#!/usr/bin/env node
import { spawnSync } from "node:child_process";
import { constants } from "node:fs";
import {
access,
lstat,
mkdir,
readFile,
readdir,
readlink,
realpath,
symlink,
unlink,
} from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
const SCRIPT_ROOT = path.resolve(
path.dirname(fileURLToPath(import.meta.url)),
"..",
);
const MANIFEST_PATH = "config/pi-links.json";
const COMMANDS = new Set(["check", "sync", "prune"]);
function usage() {
return (
`Usage: node scripts/pi-links.mjs <check|sync|prune> [--dry-run]\n\n` +
` check Report missing, stale, or conflicting exposure without writing\n` +
` sync Create or repair manifest-declared links and local package entries\n` +
` prune Remove repository-owned links and local package entries not in the manifest\n` +
` --dry-run Print sync/prune changes without applying them`
);
}
function isInside(parent, child) {
const relative = path.relative(parent, child);
return (
relative === "" ||
(!relative.startsWith("..") && !path.isAbsolute(relative))
);
}
async function pathExists(target) {
try {
await access(target, constants.F_OK);
return true;
} catch {
return false;
}
}
async function statOrUndefined(target) {
try {
return await lstat(target);
} catch (error) {
if (error?.code === "ENOENT") return undefined;
throw error;
}
}
async function canonical(target) {
try {
return await realpath(target);
} catch {
return path.resolve(target);
}
}
function parseSkillName(contents, skillFile) {
const frontmatter = contents.match(/^---\r?\n([\s\S]*?)\r?\n---(?:\r?\n|$)/);
if (!frontmatter) throw new Error(`${skillFile}: missing YAML frontmatter`);
const match = frontmatter[1].match(
/^name:\s*(?:"([^"]+)"|'([^']+)'|([^#\r\n]+))\s*$/m,
);
const name = (match?.[1] ?? match?.[2] ?? match?.[3] ?? "").trim();
if (!name) throw new Error(`${skillFile}: missing frontmatter name`);
if (!/^[a-z0-9][a-z0-9._-]*$/i.test(name)) {
throw new Error(`${skillFile}: unsafe skill name ${JSON.stringify(name)}`);
}
return name;
}
async function discoverSkills(root) {
const found = [];
async function visit(directory) {
const skillFile = path.join(directory, "SKILL.md");
if (await pathExists(skillFile)) {
const name = parseSkillName(await readFile(skillFile, "utf8"), skillFile);
found.push({ name, source: directory });
return;
}
const entries = await readdir(directory, { withFileTypes: true });
for (const entry of entries.sort((a, b) => a.name.localeCompare(b.name))) {
if (
!entry.isDirectory() ||
entry.name.startsWith(".") ||
entry.name === "node_modules"
)
continue;
await visit(path.join(directory, entry.name));
}
}
await visit(root);
return found;
}
async function readJson(file) {
const contents = await readFile(file, "utf8");
try {
return JSON.parse(contents);
} catch (error) {
const reason = error instanceof Error ? error.message : String(error);
throw new Error(`${file}: invalid JSON: ${reason}`, { cause: error });
}
}
function assertRecord(value, label) {
if (!value || typeof value !== "object" || Array.isArray(value)) {
throw new Error(`${label} must be an object`);
}
return value;
}
function assertStringArray(value, label) {
if (
!Array.isArray(value) ||
value.some((entry) => typeof entry !== "string")
) {
throw new Error(`${label} must be an array of strings`);
}
return value;
}
async function loadManifest(repoRoot) {
const manifestFile = path.join(repoRoot, MANIFEST_PATH);
const manifest = await readJson(manifestFile);
if (manifest.version !== 1)
throw new Error(`${manifestFile}: unsupported version`);
const skills = assertRecord(manifest.skills, "skills");
const extensions = assertRecord(manifest.extensions, "extensions");
const instructions = assertRecord(manifest.instructions, "instructions");
assertRecord(skills.projectOnly, "skills.projectOnly");
assertStringArray(skills.excluded, "skills.excluded");
assertRecord(extensions.globalSymlinks, "extensions.globalSymlinks");
assertRecord(
extensions.globalLocalPackages,
"extensions.globalLocalPackages",
);
assertRecord(instructions.globalSymlinks, "instructions.globalSymlinks");
if (typeof skills.root !== "string")
throw new Error("skills.root must be a string");
return manifest;
}
async function validateSource(repoRoot, relativeSource, kind) {
const source = path.resolve(repoRoot, relativeSource);
if (!isInside(repoRoot, source))
throw new Error(`${relativeSource}: source escapes repository`);
const stat = await statOrUndefined(source);
if (!stat) throw new Error(`${relativeSource}: source does not exist`);
if (kind === "skill") {
if (
!stat.isDirectory() ||
!(await pathExists(path.join(source, "SKILL.md")))
) {
throw new Error(`${relativeSource}: skill source must contain SKILL.md`);
}
} else if (kind === "instruction") {
if (!stat.isFile() || path.extname(source).toLowerCase() !== ".md") {
throw new Error(
`${relativeSource}: instruction source must be a Markdown file`,
);
}
} else if (stat.isDirectory()) {
if (
!(await pathExists(path.join(source, "index.ts"))) &&
!(await pathExists(path.join(source, "index.js")))
) {
throw new Error(
`${relativeSource}: extension directory must contain index.ts or index.js`,
);
}
} else if (!/\.[cm]?[jt]s$/i.test(source)) {
throw new Error(
`${relativeSource}: extension file must be JavaScript or TypeScript`,
);
}
return source;
}
async function resolveDesiredState(options = {}) {
const repoRoot = path.resolve(options.repoRoot ?? SCRIPT_ROOT);
const homeDir = path.resolve(options.homeDir ?? os.homedir());
const agentDir = path.resolve(
options.agentDir ??
process.env.PI_CODING_AGENT_DIR ??
path.join(homeDir, ".pi", "agent"),
);
const globalSkillsDir = path.resolve(
options.globalSkillsDir ?? path.join(homeDir, ".agents", "skills"),
);
const globalExtensionsDir = path.resolve(
options.globalExtensionsDir ?? path.join(agentDir, "extensions"),
);
const projectSkillsDir = path.join(repoRoot, ".pi", "skills");
const settingsFile = path.resolve(
options.settingsFile ?? path.join(agentDir, "settings.json"),
);
const manifest = await loadManifest(repoRoot);
const instructionDestinations = {
pi: path.join(agentDir, "AGENTS.md"),
codex: path.join(homeDir, ".codex", "AGENTS.md"),
claude: path.join(homeDir, ".claude", "CLAUDE.md"),
};
const skillsRoot = await validateSource(
repoRoot,
manifest.skills.root,
"skill-root",
).catch(async (error) => {
const source = path.resolve(repoRoot, manifest.skills.root);
const stat = await statOrUndefined(source);
if (!stat?.isDirectory()) throw error;
return source;
});
const discovered = await discoverSkills(skillsRoot);
const byName = new Map();
for (const skill of discovered) {
const previous = byName.get(skill.name);
if (previous) {
throw new Error(
`duplicate skill name ${skill.name}: ${previous.source} and ${skill.source}`,
);
}
byName.set(skill.name, skill);
}
const projectOnly = new Map(Object.entries(manifest.skills.projectOnly));
const excluded = new Set(manifest.skills.excluded);
for (const name of [...projectOnly.keys(), ...excluded]) {
if (!byName.has(name))
throw new Error(`manifest references unknown skill ${name}`);
}
for (const name of projectOnly.keys()) {
if (excluded.has(name))
throw new Error(`skill ${name} cannot be both project-only and excluded`);
}
const links = [];
for (const skill of discovered) {
if (excluded.has(skill.name)) continue;
const declaredProjectSource = projectOnly.get(skill.name);
if (declaredProjectSource !== undefined) {
const source = await validateSource(
repoRoot,
declaredProjectSource,
"skill",
);
if ((await canonical(source)) !== (await canonical(skill.source))) {
throw new Error(
`project-only skill ${skill.name} source does not match discovered source`,
);
}
links.push({
kind: "skill",
scope: "project",
name: skill.name,
source,
destination: path.join(projectSkillsDir, skill.name),
relative: true,
});
continue;
}
links.push({
kind: "skill",
scope: "global",
name: skill.name,
source: skill.source,
destination: path.join(globalSkillsDir, skill.name),
relative: false,
});
}
const extensionSources = new Set();
for (const [name, relativeSource] of Object.entries(
manifest.extensions.globalSymlinks,
)) {
const source = await validateSource(repoRoot, relativeSource, "extension");
const sourceKey = await canonical(source);
if (extensionSources.has(sourceKey))
throw new Error(
`extension source listed more than once: ${relativeSource}`,
);
extensionSources.add(sourceKey);
links.push({
kind: "extension",
scope: "global",
name,
source,
destination: path.join(globalExtensionsDir, name),
relative: false,
});
}
for (const [name, relativeSource] of Object.entries(
manifest.instructions.globalSymlinks,
)) {
const destination = instructionDestinations[name];
if (!destination)
throw new Error(`unknown global instruction target ${name}`);
const source = await validateSource(
repoRoot,
relativeSource,
"instruction",
);
links.push({
kind: "instruction",
scope: "global",
name,
source,
destination,
relative: false,
type: "file",
});
}
const packages = [];
for (const [name, relativeSource] of Object.entries(
manifest.extensions.globalLocalPackages,
)) {
const source = await validateSource(repoRoot, relativeSource, "extension");
const sourceKey = await canonical(source);
if (extensionSources.has(sourceKey)) {
throw new Error(
`extension ${name} is exposed as both a symlink and local package`,
);
}
extensionSources.add(sourceKey);
packages.push({ kind: "extension", scope: "global", name, source });
}
links.sort((a, b) => a.destination.localeCompare(b.destination));
packages.sort((a, b) => a.source.localeCompare(b.source));
return {
repoRoot,
settingsFile,
managedDirectories: [
globalSkillsDir,
globalExtensionsDir,
projectSkillsDir,
],
managedFiles: Object.values(instructionDestinations),
links,
packages,
};
}
function localPackagePath(source, settingsFile) {
if (typeof source !== "string") return undefined;
if (/^(?:npm:|git:|https?:|ssh:|git\+|file:)/i.test(source)) return undefined;
if (!path.isAbsolute(source) && !source.startsWith(".")) return undefined;
return path.resolve(path.dirname(settingsFile), source);
}
async function readLocalPackages(settingsFile) {
if (!(await pathExists(settingsFile))) return [];
const settings = await readJson(settingsFile);
if (!Array.isArray(settings.packages)) return [];
return settings.packages
.map((entry) => (typeof entry === "string" ? entry : entry?.source))
.map((source) => ({
source,
resolved: localPackagePath(source, settingsFile),
}))
.filter((entry) => entry.resolved);
}
async function inspectDesiredLink(link, repoRoot) {
const stat = await statOrUndefined(link.destination);
if (!stat) return { type: "create-link", link };
if (!stat.isSymbolicLink()) {
return {
type: "conflict",
path: link.destination,
reason: "destination is not a symlink",
};
}
const rawTarget = await readlink(link.destination);
const resolvedTarget = path.resolve(
path.dirname(link.destination),
rawTarget,
);
if ((await canonical(resolvedTarget)) === (await canonical(link.source)))
return undefined;
if (isInside(repoRoot, resolvedTarget))
return { type: "replace-link", link, previousTarget: resolvedTarget };
return {
type: "conflict",
path: link.destination,
reason: `symlink points outside this repository: ${rawTarget}`,
};
}
async function discoverStaleLinks(state, desiredDestinations) {
const stale = [];
for (const directory of state.managedDirectories) {
const stat = await statOrUndefined(directory);
if (!stat?.isDirectory()) continue;
const entries = await readdir(directory, { withFileTypes: true });
for (const entry of entries) {
if (!entry.isSymbolicLink()) continue;
const destination = path.join(directory, entry.name);
if (desiredDestinations.has(destination)) continue;
const rawTarget = await readlink(destination);
const resolvedTarget = path.resolve(directory, rawTarget);
if (isInside(state.repoRoot, resolvedTarget)) {
stale.push({
type: "remove-link",
destination,
previousTarget: resolvedTarget,
});
}
}
}
for (const destination of state.managedFiles) {
if (desiredDestinations.has(destination)) continue;
const stat = await statOrUndefined(destination);
if (!stat?.isSymbolicLink()) continue;
const rawTarget = await readlink(destination);
const resolvedTarget = path.resolve(path.dirname(destination), rawTarget);
if (isInside(state.repoRoot, resolvedTarget)) {
stale.push({
type: "remove-link",
destination,
previousTarget: resolvedTarget,
});
}
}
return stale;
}
export async function buildPlan(options = {}) {
const state = await resolveDesiredState(options);
const desiredDestinations = new Set(
state.links.map((link) => link.destination),
);
const desiredPackageSources = new Set(
await Promise.all(state.packages.map((entry) => canonical(entry.source))),
);
const operations = [];
const conflicts = [];
for (const link of state.links) {
const result = await inspectDesiredLink(link, state.repoRoot);
if (result?.type === "conflict") conflicts.push(result);
else if (result) operations.push(result);
}
operations.push(...(await discoverStaleLinks(state, desiredDestinations)));
const installedPackages = await readLocalPackages(state.settingsFile);
const installedCounts = new Map();
for (const entry of installedPackages) {
const key = await canonical(entry.resolved);
installedCounts.set(key, (installedCounts.get(key) ?? 0) + 1);
}
for (const [source, count] of installedCounts) {
if (count > 1 && isInside(state.repoRoot, source)) {
conflicts.push({
type: "conflict",
path: state.settingsFile,
reason: `local package appears ${count} times: ${source}`,
});
}
}
for (const pkg of state.packages) {
const source = await canonical(pkg.source);
if (!installedCounts.has(source))
operations.push({ type: "install-package", package: pkg });
}
for (const entry of installedPackages) {
const source = await canonical(entry.resolved);
if (
isInside(state.repoRoot, source) &&
!desiredPackageSources.has(source)
) {
operations.push({ type: "remove-package", source });
}
}
operations.sort((a, b) =>
describeOperation(a).localeCompare(describeOperation(b)),
);
conflicts.sort((a, b) =>
`${a.path}: ${a.reason}`.localeCompare(`${b.path}: ${b.reason}`),
);
return { state, operations, conflicts };
}
function describeOperation(operation) {
switch (operation.type) {
case "create-link":
return `create ${operation.link.destination} -> ${operation.link.source}`;
case "replace-link":
return `repair ${operation.link.destination} -> ${operation.link.source}`;
case "remove-link":
return `remove stale link ${operation.destination} -> ${operation.previousTarget}`;
case "install-package":
return `install local package ${operation.package.source}`;
case "remove-package":
return `remove stale local package ${operation.source}`;
default:
return `${operation.type}`;
}
}
function selectOperations(command, operations) {
if (command === "sync") {
return operations.filter(
(operation) =>
operation.type === "create-link" ||
operation.type === "replace-link" ||
operation.type === "install-package",
);
}
if (command === "prune") {
return operations.filter(
(operation) =>
operation.type === "remove-link" || operation.type === "remove-package",
);
}
return [];
}
async function createOrReplaceLink(operation) {
const { link } = operation;
await mkdir(path.dirname(link.destination), { recursive: true });
if (operation.type === "replace-link") await unlink(link.destination);
const target = link.relative
? path.relative(path.dirname(link.destination), link.source)
: link.source;
await symlink(target, link.destination, link.type ?? "dir");
}
function runPiPackage(command, source) {
const result = spawnSync("pi", [command, source], { stdio: "inherit" });
if (result.error) throw result.error;
if (result.status !== 0)
throw new Error(
`pi ${command} failed for ${source} with exit code ${result.status}`,
);
}
async function applyOperations(operations) {
for (const operation of operations) {
if (operation.type === "create-link" || operation.type === "replace-link") {
await createOrReplaceLink(operation);
} else if (operation.type === "remove-link") {
await unlink(operation.destination);
} else if (operation.type === "install-package") {
runPiPackage("install", operation.package.source);
} else if (operation.type === "remove-package") {
runPiPackage("remove", operation.source);
}
}
}
export async function run(command, options = {}) {
const plan = await buildPlan(options);
for (const conflict of plan.conflicts)
console.error(`CONFLICT ${conflict.path}: ${conflict.reason}`);
if (plan.conflicts.length > 0) return 2;
if (command === "check") {
if (plan.operations.length === 0) {
console.log(
`OK ${plan.state.links.length} links and ${plan.state.packages.length} local package entries match ${MANIFEST_PATH}`,
);
return 0;
}
for (const operation of plan.operations)
console.log(`DRIFT ${describeOperation(operation)}`);
return 1;
}
const selected = selectOperations(command, plan.operations);
if (selected.length === 0) {
console.log(`OK nothing to ${command}`);
return 0;
}
for (const operation of selected)
console.log(
`${options.dryRun ? "WOULD" : "APPLY"} ${describeOperation(operation)}`,
);
if (!options.dryRun) await applyOperations(selected);
return 0;
}
async function main(argv = process.argv.slice(2)) {
const command = argv.find((arg) => !arg.startsWith("-"));
const unknown = argv.filter(
(arg) => arg.startsWith("-") && arg !== "--dry-run",
);
if (!COMMANDS.has(command) || unknown.length > 0) {
console.error(usage());
return 2;
}
return run(command, { dryRun: argv.includes("--dry-run") });
}
if (
process.argv[1] &&
import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href
) {
main()
.then((code) => {
process.exitCode = code;
})
.catch((error) => {
console.error(
`ERROR ${error instanceof Error ? error.message : String(error)}`,
);
process.exitCode = 2;
});
}
+218
View File
@@ -0,0 +1,218 @@
import assert from "node:assert/strict";
import {
mkdir,
mkdtemp,
readFile,
readlink,
symlink,
writeFile,
} from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import test from "node:test";
import { buildPlan, run } from "./pi-links.mjs";
async function writeJson(file, value) {
await mkdir(path.dirname(file), { recursive: true });
await writeFile(file, `${JSON.stringify(value, null, 2)}\n`);
}
async function writeSkill(directory, name) {
await mkdir(directory, { recursive: true });
await writeFile(
path.join(directory, "SKILL.md"),
`---\nname: ${name}\ndescription: test\n---\n\n# ${name}\n`,
);
}
async function writeExtension(target) {
if (path.extname(target)) {
await mkdir(path.dirname(target), { recursive: true });
await writeFile(target, "export default function () {}\n");
return;
}
await mkdir(target, { recursive: true });
await writeFile(
path.join(target, "index.ts"),
"export default function () {}\n",
);
}
async function fixture() {
const root = await mkdtemp(path.join(os.tmpdir(), "pi-links-test-"));
const repoRoot = path.join(root, "repo");
const homeDir = path.join(root, "home");
const agentDir = path.join(homeDir, ".pi", "agent");
const settingsFile = path.join(agentDir, "settings.json");
const globalSkill = path.join(repoRoot, "skills", "group", "global-skill");
const projectSkill = path.join(repoRoot, "skills", "tools", "project-skill");
const globalExtension = path.join(repoRoot, "extensions", "global-ext");
const packageExtension = path.join(
repoRoot,
"extensions",
"package-ext",
"index.ts",
);
const globalAgents = path.join(repoRoot, "docs", "global.AGENTS.md");
const globalClaude = path.join(repoRoot, "docs", "global.CLAUDE.md");
await writeSkill(globalSkill, "global-skill");
await writeSkill(projectSkill, "project-skill");
await writeExtension(globalExtension);
await writeExtension(packageExtension);
await mkdir(path.dirname(globalAgents), { recursive: true });
await writeFile(globalAgents, "# Global instructions\n");
await writeFile(globalClaude, "@global.AGENTS.md\n");
await writeJson(path.join(repoRoot, "config", "pi-links.json"), {
version: 1,
skills: {
root: "skills",
projectOnly: { "project-skill": "skills/tools/project-skill" },
excluded: [],
},
instructions: {
globalSymlinks: {
pi: "docs/global.AGENTS.md",
codex: "docs/global.AGENTS.md",
claude: "docs/global.CLAUDE.md",
},
},
extensions: {
globalSymlinks: { "global-ext": "extensions/global-ext" },
globalLocalPackages: { "package-ext": "extensions/package-ext/index.ts" },
},
});
await writeJson(settingsFile, {
packages: [path.relative(agentDir, packageExtension)],
});
return {
root,
repoRoot,
homeDir,
agentDir,
settingsFile,
globalSkill,
projectSkill,
globalExtension,
globalAgents,
globalClaude,
};
}
test("sync creates absolute global links and relative project links", async () => {
const options = await fixture();
assert.equal(await run("sync", options), 0);
assert.equal(await run("check", options), 0);
const globalSkillLink = path.join(
options.homeDir,
".agents",
"skills",
"global-skill",
);
const projectSkillLink = path.join(
options.repoRoot,
".pi",
"skills",
"project-skill",
);
const extensionLink = path.join(options.agentDir, "extensions", "global-ext");
const piInstructions = path.join(options.agentDir, "AGENTS.md");
const codexInstructions = path.join(options.homeDir, ".codex", "AGENTS.md");
const claudeInstructions = path.join(options.homeDir, ".claude", "CLAUDE.md");
assert.equal(await readlink(globalSkillLink), options.globalSkill);
assert.equal(await readlink(extensionLink), options.globalExtension);
assert.equal(await readlink(piInstructions), options.globalAgents);
assert.equal(await readlink(codexInstructions), options.globalAgents);
assert.equal(await readlink(claudeInstructions), options.globalClaude);
assert.equal(
await readlink(projectSkillLink),
path.relative(path.dirname(projectSkillLink), options.projectSkill),
);
});
test("removed global instruction declarations are prunable without scanning config directories", async () => {
const options = await fixture();
await run("sync", options);
const manifestFile = path.join(options.repoRoot, "config", "pi-links.json");
const manifest = JSON.parse(await readFile(manifestFile, "utf8"));
delete manifest.instructions.globalSymlinks.claude;
await writeJson(manifestFile, manifest);
const plan = await buildPlan(options);
assert.ok(
plan.operations.some(
(operation) =>
operation.type === "remove-link" &&
operation.destination ===
path.join(options.homeDir, ".claude", "CLAUDE.md"),
),
);
});
test("dry-run reports drift without writing", async () => {
const options = await fixture();
assert.equal(await run("sync", { ...options, dryRun: true }), 0);
const plan = await buildPlan(options);
assert.ok(
plan.operations.some((operation) => operation.type === "create-link"),
);
});
test("unmanaged destinations are conflicts and remain untouched", async () => {
const options = await fixture();
const destination = path.join(
options.homeDir,
".agents",
"skills",
"global-skill",
);
await mkdir(path.dirname(destination), { recursive: true });
await writeFile(destination, "keep me\n");
const plan = await buildPlan(options);
assert.deepEqual(
plan.conflicts.map((entry) => entry.path),
[destination],
);
assert.equal(await run("sync", options), 2);
});
test("prune removes only stale links whose targets are inside the repository", async () => {
const options = await fixture();
await run("sync", options);
const skillsDir = path.join(options.homeDir, ".agents", "skills");
const stale = path.join(skillsDir, "stale");
const external = path.join(skillsDir, "external");
await symlink(options.globalSkill, stale);
await symlink(path.join(options.root, "outside"), external);
const plan = await buildPlan(options);
assert.ok(
plan.operations.some(
(operation) =>
operation.type === "remove-link" && operation.destination === stale,
),
);
assert.ok(
!plan.operations.some(
(operation) =>
operation.type === "remove-link" && operation.destination === external,
),
);
assert.equal(await run("prune", options), 0);
assert.equal(await readlink(external), path.join(options.root, "outside"));
});
test("duplicate declared skill names fail before planning changes", async () => {
const options = await fixture();
await writeSkill(
path.join(options.repoRoot, "skills", "other"),
"global-skill",
);
await assert.rejects(
() => buildPlan(options),
/duplicate skill name global-skill/,
);
});
+14
View File
@@ -0,0 +1,14 @@
function configure_pi_orca_runtime --argument-names profile
set -g pi_orca_node_executable /usr/bin/node
set -gx PI_CODING_AGENT_DIR /home/jb/.pi/agent
if test "$profile" = pi-work
set pi_orca_node_executable /home/jb/.local/share/fnm/node-versions/v22.23.2/installation/bin/node
set PI_CODING_AGENT_DIR /home/jb/.pi/profiles/cyone
end
# Keep Pi and its subprocesses on the same Node installation. Orca inherits
# fnm's PATH, which otherwise makes tool commands resolve a different Node.
set -l node_bin (path dirname "$pi_orca_node_executable")
set -gx PATH "$node_bin" $PATH
end
+4 -8
View File
@@ -20,15 +20,11 @@ if not /usr/bin/nono profile show "$profile" >/dev/null 2>&1
exit 1 exit 1
end end
set -l node_executable /usr/bin/node source /srv/data/code/harness-tools/scripts/pi-orca-runtime.fish
set -l pi_agent_dir /home/jb/.pi/agent configure_pi_orca_runtime "$profile"
# Native Node addons are ABI-specific. Keep the work profile's Node 22 package set -l node_executable "$pi_orca_node_executable"
# tree separate from the default Node 26 package tree. set -l pi_agent_dir "$PI_CODING_AGENT_DIR"
if test "$profile" = pi-work
set node_executable /home/jb/.local/share/fnm/node-versions/v22.23.2/installation/bin/node
set pi_agent_dir /home/jb/.pi/profiles/cyone
end
if not test -x "$node_executable" if not test -x "$node_executable"
printf 'pi-orca: Node runtime is unavailable: %s\n' "$node_executable" >&2 printf 'pi-orca: Node runtime is unavailable: %s\n' "$node_executable" >&2
+60
View File
@@ -0,0 +1,60 @@
from __future__ import annotations
import os
import subprocess
import unittest
from pathlib import Path
RUNTIME_SCRIPT = Path(__file__).with_name("pi-orca-runtime.fish")
SYSTEM_NODE = Path("/usr/bin/node")
WORK_NODE = Path(
"/home/jb/.local/share/fnm/node-versions/v22.23.2/installation/bin/node"
)
class ConfigurePiOrcaRuntimeTests(unittest.TestCase):
def configure_runtime(self, profile: str) -> list[str]:
environment = os.environ.copy()
environment["PATH"] = ":".join(
[str(WORK_NODE.parent), "/usr/local/bin", "/usr/bin"]
)
command = """
source $argv[1]
configure_pi_orca_runtime $argv[2]
printf '%s\\n' $pi_orca_node_executable $PI_CODING_AGENT_DIR $PATH[1] (command -v node)
"""
result = subprocess.run(
["/usr/bin/fish", "-c", command, str(RUNTIME_SCRIPT), profile],
check=True,
capture_output=True,
env=environment,
text=True,
)
return result.stdout.splitlines()
def test_default_profile_prioritizes_system_node(self) -> None:
self.assertEqual(
self.configure_runtime("pi-harness-tools"),
[
str(SYSTEM_NODE),
"/home/jb/.pi/agent",
str(SYSTEM_NODE.parent),
str(SYSTEM_NODE),
],
)
def test_work_profile_prioritizes_node_22(self) -> None:
self.assertEqual(
self.configure_runtime("pi-work"),
[
str(WORK_NODE),
"/home/jb/.pi/profiles/cyone",
str(WORK_NODE.parent),
str(WORK_NODE),
],
)
if __name__ == "__main__":
unittest.main()
+131
View File
@@ -0,0 +1,131 @@
---
name: unslop
description: Remove AI writing patterns from prose using either audit-only detection or a two-pass rewrite flow (diagnosis then reconstruction). Use this skill when editing, reviewing, or rewriting AI-generated content to make it sound human. Triggers on requests to "humanize", "de-slop", "fix AI text", "make it sound human", "remove AI patterns", or when reviewing text that contains obvious AI tells like "Here's the thing:", "Let that sink in", or "In today's fast-paced landscape". Also use when the user pastes text and says it "sounds like ChatGPT", "sounds robotic", "needs to sound more natural", or asks you to "clean up" drafted content before publishing.
license: MIT
user-invocable: true
argument-hint: "[teach · cleanup · rewrite · mimic] [input]"
metadata:
author: claytonkim
version: "2.3.0"
---
# Unslop
Humanize AI-generated prose. Audit first. Rewrite only when the user asks for a rewrite.
For every audit or rewrite, read [references/core-contract.md](references/core-contract.md).
It is the single behavior contract. Command files define routing and mechanics;
presets supply optional voice, but neither can override the core contract.
## Routing
**When the user invokes a sub-command (`/unslop teach ...`, `/unslop cleanup
...`), you MUST read `references/commands/<command>.md` before acting.
Non-optional — the command file defines the flow, and skipping it drops steps the
user expects.** A bare `/unslop <text>` with no leading command word defaults to
`rewrite`. If the first word does not match a command but the intent clearly maps
to one (e.g. "flag the AI tells, don't change anything" → `cleanup` report-only),
load that command file and proceed as if invoked.
| Command | Purpose | File |
|---------|---------|------|
| `rewrite` | Default two-pass de-slop: diagnose, reconstruct under the guards, validate. | [references/commands/rewrite.md](references/commands/rewrite.md) |
| `cleanup` | Co-writer: cheap detection, reviewable suggestions with contract gates; includes report-only "flag, change nothing". | [references/commands/cleanup.md](references/commands/cleanup.md) |
| `teach` | Agent-driven voice building: harvest, approve, profile, layered card, scored demo. | [references/commands/teach.md](references/commands/teach.md) |
| `mimic` | Voiced drafting or rewriting under the full gates; refine loop when one pass falls short. | [references/commands/mimic.md](references/commands/mimic.md) |
| _maintenance_ | Turn a wild AI-ism into an eval row and a PR (not a top-level verb). | [references/commands/contribute.md](references/commands/contribute.md) |
### Routing by phrase
Sub-flows are reachable by their natural names without being top-level verbs.
When the user says any of these, load the named file and jump to the flow:
| The user says | Go to |
|---------------|-------|
| `audit` / "just flag it" / "don't change anything" | [references/commands/cleanup.md](references/commands/cleanup.md#report-only) |
| `review` / "review this before I publish" | [references/commands/cleanup.md](references/commands/cleanup.md#report-only) |
| `harvest` / "what writing of mine do you have?" | [references/commands/teach.md](references/commands/teach.md#1-gather-samples-harvest) |
| `calibrate` / "the A/B game" / "quiz me on my voice" | [references/commands/teach.md](references/commands/teach.md#calibrate) |
| `refine` / "keep pushing until it sounds like me" | [references/commands/mimic.md](references/commands/mimic.md#refine) |
| voice check / "does this sound like me?" | [references/commands/mimic.md](references/commands/mimic.md#voice-check) |
| "found a new AI-ism" / "add this tell" | [references/commands/contribute.md](references/commands/contribute.md) |
## Interface
| Argument | Description | Default |
|----------|-------------|---------|
| `--preset` | Voice style: `crisp`, `warm`, `expert`, `story` | `crisp` |
| `--strict` | Fail if rubric score < 32/40 | false |
| `--report` | Flag AI patterns without changing the text (cleanup) | false |
| Input | Text to transform (argument, file path, or stdin) | required |
Read one preset from `presets/` before writing.
| Preset | Style | Best For |
|--------|-------|----------|
| `crisp` | Short, direct, no fluff | Technical writing, documentation |
| `warm` | Friendly, conversational | Emails, blog posts |
| `expert` | Authoritative, confident | Thought leadership, articles |
| `story` | Narrative flow, show don't tell | Case studies, personal posts |
Rewrite, preservation, register, and validation behavior lives only in
`references/core-contract.md`; do not recreate or override those rules here.
## Output Format
For a quick rewrite, return the cleaned text only. For audit-only (cleanup
`--report`):
```markdown
## Issues Found
- [Quoted issue, category, severity, why it reads as AI]
## Assessment
- [Which issues are clear problems]
- [Which issues are judgment calls or context-dependent]
```
For strict or requested analysis:
```markdown
## Transformed Text
[The humanized version]
## Validation
- Constraints: [X]/[Y] preserved
- AI patterns: [N] remaining (was [M])
- Structure: [pass/fail]
- Readability: Grade [X], sentence variance [Y]
- Change: [X]% from original
- Score: [X]/40
```
## Reference Files
| File | When to Read |
|------|-------------|
| `references/commands/*.md` | The routed command flows (rewrite, cleanup, teach, mimic, contribute). |
| `references/pipeline.md` | Orchestrated tiered execution for multi-agent harnesses. |
| `references/taboo-phrases.md` | Authoritative phrase catalog and scanner categories. |
| `references/fact-preservation.md` | Constraint preservation rules. |
| `references/rewrite-examples.md` | Executable before/after examples. |
| `references/{mimic,harvest,calibrate}.md` | Voice-tool internals loaded by their routed command. |
| `references/{rubric,edit-library,maintenance}.md` | Strict scoring, examples, and contribution procedures. |
| `presets/*.md` | Voice-specific deltas. |
## Maintenance
The eval contracts define the product. Add scanner examples eval-first in
`evals/fixtures/contracts/scanner-examples.json`; use
`evals/adversarial-evals.json` for agent behavior and routing. Do not edit legacy
`evals/evals.json`. New patterns need a false-negative example and a
false-positive protection example. Agent behavior changes need a `skill` row
and a regenerated shared benchmark. For the
concrete procedures (add a phrase or structure, list current patterns, sync with
Wikipedia's signs-of-AI-writing page), read `references/maintenance.md`. Found a
new AI-ism in the wild? `references/commands/contribute.md` turns the exact
snippet into a contract example and a structured PR, keeping both user-confirmation gates.
@@ -0,0 +1,118 @@
# Before/After: Article Style
Technical or thought leadership article transformations.
---
## Example 1: Leadership Article
### Before (AI)
> Here's the uncomfortable truth about leadership that nobody wants to talk about: it's lonely at the top. Not because leaders are inherently isolated. Because the weight of decision-making creates a natural distance between leaders and their teams.
>
> In today's fast-paced business environment, leaders are expected to navigate increasingly complex challenges while maintaining an air of confidence. The reality is that most leaders struggle with imposter syndrome at some point in their careers. Let that sink in.
>
> It's worth noting that the most effective leaders I've encountered have one thing in common: they've learned to embrace vulnerability. This doesn't mean oversharing or appearing weak. It means acknowledging uncertainty when appropriate and creating space for others to do the same.
>
> Here's why that matters: when leaders model vulnerability, it creates psychological safety. And psychological safety is the foundation of high-performing teams. Full stop.
### After (Humanized)
> Leadership is lonely. Not because of hierarchy—because decisions separate you from the team that lives with their consequences.
>
> Most leaders experience imposter syndrome. The effective ones learn to admit uncertainty. They say "I don't know" when they don't know.
>
> This creates psychological safety. Teams perform better when they can admit problems early instead of hiding them until they're unfixable.
### Changes Made
- Removed "Here's the uncomfortable truth" opener
- Cut "In today's fast-paced business environment"
- Removed "It's worth noting that"
- Cut "Let that sink in" and "Full stop"
- Removed "Here's why that matters"
- Collapsed "Not because X. Because Y." structure
- 186 words → 69 words (63% reduction)
---
## Example 2: Technical Article
### Before (AI)
> If you've been working with microservices for any length of time, you've probably encountered the n+1 query problem. The uncomfortable truth is that it's more common than most teams want to admit. Let me explain what I mean.
>
> At its core, the n+1 problem occurs when your application makes one query to fetch a list of items, then makes n additional queries to fetch related data for each item. In a microservices landscape, this problem is amplified because each query might involve network calls between services.
>
> Here's an example. Imagine you're building an e-commerce platform. You fetch 100 orders, then for each order, you call the user service to get customer details. That's 101 network requests where one would suffice.
>
> The solution? Batch your queries. Instead of fetching users one at a time, collect all user IDs and make a single batch request. This is a game-changer for performance.
### After (Humanized)
> The n+1 query problem shows up in most microservices codebases. One query fetches a list. Then n queries fetch related data for each item. In distributed systems, each query is a network call.
>
> Example: fetching 100 orders, then calling the user service 100 times for customer details. That's 101 network requests instead of 2.
>
> Fix: batch your queries. Collect all user IDs, make one request. A 100-order page load drops from 500ms to 50ms.
### Changes Made
- Removed "If you've been working with" softening
- Cut "The uncomfortable truth is" and "Let me explain what I mean"
- Removed "At its core"
- Cut "Here's an example" (just show the example)
- Replaced "game-changer" with concrete numbers
- 182 words → 89 words (51% reduction)
---
## Example 3: Strategy Article
### Before (AI)
> In a world where every company claims to be customer-centric, what does it actually mean to put customers first? Here's the thing: most companies confuse customer service with customer obsession. Not the same thing. Not even close.
>
> Customer service is reactive. It's about handling problems after they occur. Customer obsession, on the other hand, is about anticipating needs before customers even realize they have them. Think about it: the best products solve problems you didn't know you had.
>
> Interestingly, this is exactly what Amazon gets right. When Jeff Bezos talks about working backwards from the customer, he's not talking about surveys and focus groups. He's talking about deeply understanding customer behavior through data, then making bold bets on what they'll want next.
>
> The key takeaway? Stop asking customers what they want. Start observing what they do. That's where the real insights live.
### After (Humanized)
> Most companies confuse customer service with customer obsession. Customer service handles problems. Customer obsession anticipates them.
>
> Amazon works backwards from customers—not through surveys, but by analyzing behavior data and betting on what customers will want before they know they want it.
>
> Watch what customers do, not what they say they want. Behavior reveals needs that surveys miss.
### Changes Made
- Removed "In a world where" opener
- Cut "Here's the thing" and "Think about it"
- Removed "Interestingly"
- Cut "Not the same thing. Not even close." dramatic emphasis
- Removed "The key takeaway?"
- Cut "That's where the real insights live" kicker
- 178 words → 68 words (62% reduction)
---
## Example 4: Process Article
### Before (AI)
> Code review is one of those practices that everyone agrees is important, yet few teams do well. The uncomfortable truth? Most code reviews are a waste of time. They catch typos and style issues while missing the architectural problems that will haunt you six months later.
>
> Here's what effective code review actually looks like. First, focus on the why, not the what. Anyone can see what the code does. The reviewer's job is to ask: should we be doing this at all? Is this the right approach? What are we trading off?
>
> Second, time-box your reviews. Interestingly, studies show that review effectiveness drops dramatically after 60 minutes. If a PR takes longer than that to review, it's probably too big. Full stop.
>
> Third—and this is crucial—review early, review often. Waiting until a feature is "done" means reviewing thousands of lines at once. That's when things get missed.
### After (Humanized)
> Most code reviews catch typos while missing architectural problems. They're a checklist, not a conversation.
>
> Effective review asks: should we do this at all? What's the tradeoff? Anyone can see what code does—reviewers should question whether it should exist.
>
> Time-box to 60 minutes. Effectiveness drops after that. If a PR takes longer, it's too big. Review smaller changes more often.
### Changes Made
- Cut "one of those practices that everyone agrees is important"
- Removed "The uncomfortable truth?"
- Cut "Here's what X actually looks like"
- Removed "First, Second, Third" list structure
- Cut "Interestingly" and "Full stop"
- Removed "this is crucial"
- 189 words → 77 words (59% reduction)
@@ -0,0 +1,218 @@
# Before/After: LinkedIn Style
Professional social media post transformations.
---
## Example 1: Career Advice
### Before (AI)
> Here's the thing about career advice that nobody wants to tell you:
>
> Most of it is survivorship bias dressed up as wisdom.
>
> The uncomfortable truth? The people giving advice succeeded for reasons they don't fully understand. Timing. Luck. Being in the right place.
>
> Not because they followed a playbook.
> Not because they "hustled harder."
> Not because they had a secret framework.
>
> Because circumstances aligned.
>
> Does this mean advice is useless? No.
>
> But it means you should:
> - Take it with a grain of salt
> - Understand the context it came from
> - Adapt it to YOUR situation
>
> The best career advice I ever got?
>
> "Pay attention to what works for YOU."
>
> And that's it. That's the whole thing.
>
> What's the best career advice you've received? Drop it in the comments. 👇
### After (Humanized)
> Most career advice is survivorship bias. Successful people explain their success with stories. The real causes—timing, luck, circumstance—don't make good LinkedIn posts.
>
> Useful advice is contextual. What worked for someone in 2015 at a 10-person startup won't transfer to your 2024 enterprise job.
>
> Pay attention to what works for you. Run experiments. Keep what helps.
### Changes Made
- Removed "Here's the thing" opener
- Cut "The uncomfortable truth?"
- Collapsed "Not because X. Not because Y. Not because Z." structure
- Removed rhetorical "Does this mean advice is useless? No."
- Cut bulleted list (converted to prose)
- Removed "And that's it. That's the whole thing."
- Cut engagement bait question and emoji
- 141 words → 62 words (56% reduction)
---
## Example 2: Leadership Insight
### Before (AI)
> I used to think great managers had all the answers.
>
> I was wrong. Dead wrong.
>
> Here's what I learned after managing teams for 10 years:
>
> The best managers don't solve problems.
> They create conditions for others to solve problems.
>
> Think about it:
> - When you solve everything, you become a bottleneck
> - When you empower others, you scale yourself
> - When you step back, leaders emerge
>
> This is genuinely hard. Our instincts scream "fix it!" But the real skill? Sitting with discomfort while someone else figures it out.
>
> The transformation happened when I started asking:
>
> "What do YOU think we should do?"
>
> Instead of:
>
> "Here's what I think we should do."
>
> Game. Changer.
>
> What's one management habit that transformed your leadership? Let me know below! 🙌
### After (Humanized)
> Good managers don't solve problems. They create conditions for others to solve problems.
>
> When you solve everything, you become a bottleneck. When you step back, leaders emerge from the team.
>
> One shift that worked: asking "What do you think we should do?" before offering my answer. Uncomfortable at first. Effective long-term.
### Changes Made
- Cut "I used to think / I was wrong" setup
- Removed "Dead wrong" emphasis
- Cut "Here's what I learned after managing teams for 10 years"
- Removed "Think about it:" and bullet structure
- Cut "This is genuinely hard"
- Removed "Game. Changer." dramatic fragment
- Cut engagement question and emoji
- 139 words → 63 words (55% reduction)
---
## Example 3: Hiring Post
### Before (AI)
> Hot take: your interview process is broken.
>
> And you probably don't even know it.
>
> Here's the uncomfortable truth about technical interviews:
>
> They test performance under artificial pressure.
> Not actual job skills.
>
> Whiteboard coding? Tests anxiety management.
> Take-home projects? Tests who has free time.
> Algorithm puzzles? Tests memorization.
>
> None of these predict on-the-job success.
>
> What actually works?
>
> ✅ Work sample tests on real problems
> ✅ Paid trial days
> ✅ Collaborative debugging sessions
> ✅ Reference checks (yes, really)
>
> The best hire I ever made bombed the algorithm round.
>
> The worst hire I ever made aced it.
>
> Let that sink in.
>
> Your interview process isn't selecting for talent. It's selecting for people who are good at interviews.
>
> Time to rethink. 💡
>
> Agree? Disagree? Let's debate in the comments.
### After (Humanized)
> Technical interviews test interview skills, not job skills.
>
> Whiteboard coding measures anxiety management. Algorithm puzzles measure memorization. Neither predicts job performance.
>
> Better alternatives: work samples from real problems, paid trial days, collaborative debugging. These show how candidates actually work.
>
> The best engineer I hired failed the algorithm round. The worst one aced it.
### Changes Made
- Removed "Hot take:" opener
- Cut "And you probably don't even know it"
- Removed "Here's the uncomfortable truth"
- Collapsed repetitive list structure
- Cut emoji checkmarks and bullet formatting
- Removed "Let that sink in"
- Cut "Time to rethink" and engagement prompt
- 136 words → 60 words (56% reduction)
---
## Example 4: Failure Story
### Before (AI)
> I got fired in 2019.
>
> Best thing that ever happened to me.
>
> Here's what nobody tells you about failure:
>
> It's not the end. It's data.
>
> When I lost my job, I thought my career was over. The shame was overwhelming. I didn't leave my apartment for a week.
>
> But then something shifted.
>
> I started asking: What can I learn from this?
>
> The answer?
> - I had been coasting
> - I wasn't growing
> - The company wasn't the right fit (and I knew it)
>
> Getting fired forced me to be honest with myself.
>
> Three months later, I landed a role that:
> ✨ Paid 40% more
> ✨ Aligned with my values
> ✨ Actually challenged me
>
> The point isn't that getting fired is good.
>
> The point is that setbacks contain information.
>
> Use them.
>
> What's a failure that turned into an opportunity for you?
### After (Humanized)
> I got fired in 2019. Didn't leave my apartment for a week.
>
> Eventually I asked what I could learn. Answers: I'd been coasting, not growing, and knew the company wasn't right.
>
> Three months later: a job that paid 40% more and actually challenged me. The firing forced honesty I'd been avoiding.
>
> Setbacks contain information. Use it.
### Changes Made
- Kept strong opening, cut "Best thing that ever happened to me"
- Removed "Here's what nobody tells you about failure"
- Cut "It's not the end. It's data." (cliché)
- Removed "But then something shifted" transition
- Collapsed bullet/emoji lists into prose
- Cut rhetorical "The point isn't X. The point is Y."
- Removed engagement question
- 149 words → 70 words (53% reduction)
@@ -0,0 +1,214 @@
# Before/After: Sales/Marketing Style
Sales copy, marketing emails, and promotional content transformations.
---
## Example 1: Product Launch Email
### Before (AI)
> Subject: 🚀 The Future of Project Management is Here!
>
> Dear Valued Customer,
>
> We are incredibly excited to announce the launch of ProjectFlow 3.0—a truly game-changing update that will fundamentally transform how your team collaborates!
>
> Here's the thing: we've been listening to your feedback. And we've built something special.
>
> What's new?
> ✅ Real-time collaboration (yes, finally!)
> ✅ AI-powered task suggestions
> ✅ 50% faster load times
> ✅ Brand new mobile app
>
> But here's what really matters: these features will save your team an average of 5 hours per week. Let that sink in. That's 260 hours per year. Per person.
>
> Ready to experience the future of work?
>
> Click below to upgrade now and receive 20% off your first year!
>
> [UPGRADE NOW]
>
> We can't wait to see what you build with ProjectFlow 3.0!
>
> Onwards and upwards,
> The ProjectFlow Team
### After (Humanized)
> Subject: ProjectFlow 3.0: Real-time collaboration + 50% faster
>
> ProjectFlow 3.0 is live.
>
> New this release:
> - Real-time collaboration
> - AI task suggestions
> - 50% faster load times
> - Redesigned mobile app
>
> Early users report saving 5 hours/week on coordination overhead. That's 260 hours/year per person.
>
> Upgrade before March 1 for 20% off the first year.
>
> [Upgrade to 3.0]
### Changes Made
- Removed rocket emoji from subject
- Cut "Dear Valued Customer" (generic)
- Removed "incredibly excited," "truly game-changing," "fundamentally transform"
- Cut "Here's the thing" and "we've built something special"
- Removed "yes, finally!" aside
- Cut "But here's what really matters" and "Let that sink in"
- Removed "Ready to experience the future of work?"
- Cut "We can't wait to see what you build"
- Replaced "Onwards and upwards" sign-off
- 156 words → 69 words (56% reduction)
---
## Example 2: Cold Outreach
### Before (AI)
> Subject: Quick question about [Company]'s growth strategy
>
> Hi [Name],
>
> I hope this email finds you well! I've been following [Company]'s journey and I have to say—I'm genuinely impressed by what you're building.
>
> Here's the thing: I noticed you recently raised your Series B (congrats!). In my experience working with 100+ growth-stage companies, this is exactly when scaling challenges start to creep in.
>
> That's where we come in.
>
> DataSync helps companies like yours:
> ✅ Reduce data integration time by 70%
> ✅ Eliminate manual data entry errors
> ✅ Scale operations without scaling headcount
>
> I'd love to share how we helped [Similar Company] save $2M in their first year.
>
> Do you have 15 minutes this week for a quick call?
>
> Looking forward to connecting!
>
> Best,
> [Name]
### After (Humanized)
> Subject: Data integration after Series B
>
> Hi [Name],
>
> Congrats on the Series B.
>
> Post-funding scaling often breaks data workflows. We helped [Similar Company] reduce integration time by 70% and save $2M in year one.
>
> Worth a 15-minute call to see if we can help [Company]?
>
> [Name]
### Changes Made
- Simplified subject line
- Cut "I hope this email finds you well"
- Removed "I'm genuinely impressed by what you're building"
- Cut "Here's the thing" and "In my experience working with"
- Removed "creeps in" (performative)
- Cut "That's where we come in"
- Collapsed bullet list to key claim
- Removed "Looking forward to connecting"
- 142 words → 52 words (63% reduction)
---
## Example 3: Feature Announcement
### Before (AI)
> Big news! 🎉
>
> We're thrilled to announce a feature that's going to completely change how you work with documents.
>
> Introducing: Smart Templates
>
> Here's the uncomfortable truth about document creation: you spend way too much time on formatting. Our data shows the average user wastes 3 hours per week on repetitive document tasks.
>
> Not anymore.
>
> Smart Templates uses AI to:
> - Learn your formatting preferences
> - Auto-populate recurring content
> - Suggest improvements as you write
>
> Think about it: no more copying and pasting. No more fixing margins. No more "wait, what font do we use again?"
>
> This is what the future of productivity looks like.
>
> Try Smart Templates free for 14 days. No credit card required.
>
> [START FREE TRIAL]
>
> Let us know what you think!
### After (Humanized)
> New feature: Smart Templates
>
> Average users spend 3 hours/week on document formatting. Smart Templates cuts that significantly.
>
> It learns your formatting preferences, auto-populates recurring content, and suggests improvements while you write.
>
> 14-day free trial, no card required.
>
> [Try Smart Templates]
### Changes Made
- Removed "Big news!" with emoji
- Cut "thrilled to announce" and "completely change"
- Removed "Here's the uncomfortable truth"
- Cut "Not anymore." dramatic fragment
- Removed "Think about it" and rhetorical questions
- Cut "This is what the future of productivity looks like"
- Removed "Let us know what you think"
- 145 words → 50 words (66% reduction)
---
## Example 4: Case Study Intro
### Before (AI)
> When TechCorp came to us, they were struggling. Really struggling.
>
> Here's the thing: they had great products, great people, and great intentions. But their customer churn rate was 40%. Let that sink in. Nearly half their customers were leaving every year.
>
> The uncomfortable truth? They didn't actually understand why customers were churning. They had data—mountains of it—but no insights.
>
> That's where our Customer Intelligence Platform came in.
>
> In just 90 days, we helped TechCorp:
> ✅ Identify their top 5 churn predictors
> ✅ Reduce churn by 60%
> ✅ Save $4.2M in annual recurring revenue
>
> And here's the best part: they didn't need to hire a single data scientist.
>
> Want to see how we did it?
>
> [READ THE FULL CASE STUDY]
### After (Humanized)
> TechCorp had 40% annual customer churn. They had data but couldn't identify why customers were leaving.
>
> We deployed our Customer Intelligence Platform. In 90 days:
> - Identified top 5 churn predictors
> - Reduced churn by 60%
> - Saved $4.2M in annual recurring revenue
>
> No data science hire required.
>
> [Read the case study]
### Changes Made
- Removed "struggling. Really struggling."
- Cut "Here's the thing" and "Let that sink in"
- Removed "The uncomfortable truth?"
- Cut "That's where X came in"
- Removed emoji checkmarks
- Cut "And here's the best part"
- Removed rhetorical question
- 137 words → 57 words (58% reduction)
@@ -0,0 +1,27 @@
# Preset: Crisp Human
Use this voice only inside a confirmed finding's sentence. The core contract
controls whether an edit is allowed.
## Voice
- Direct and economical, while preserving the source register.
- Concrete when the source supplies concrete facts.
- Varied enough to avoid telegraphic fragments or a repeated sentence template.
- Confident only to the degree supported by the source.
## Editing preferences
- Remove a formulaic opener when the sentence works without it.
- Replace an inflated abstraction with the source's concrete fact.
- Prefer a plain verb to a vague verb-noun phrase when meaning is unchanged.
- Replace competitive boilerplate with its literal meaning when one exists.
- Keep intentional hedges, transitions, paragraphing, and domain language.
- Do not impose a sentence-length target, active voice, or short paragraphs on
already-natural prose.
## Safety check
If a proposed crisp rewrite adds a comparison, judgment, promise, causal claim,
or personality that the source did not contain, reject it. If the case for the
edit is uncertain, preserve the source.
@@ -0,0 +1,120 @@
# Preset: Expert Human
Authoritative, confident claims. Speaks from experience.
---
## Voice Characteristics
- **Sentence length**: 10-25 words. Varied for emphasis.
- **Paragraph length**: 2-5 sentences. Room for nuance.
- **Tone**: Confident without arrogance. Earned authority.
- **Structure**: Assertion → evidence → implication.
---
## Rules
1. **Make claims** - "This works" not "This might work"
2. **Show, don't credential** - Demonstrate knowledge, don't announce it
3. **Acknowledge complexity** - Real experts know edge cases
4. **Specific over general** - "In B2B SaaS" not "in business"
5. **Numbers and names** - Concrete details signal expertise
---
## Structural Patterns
### Opening
```
[Confident assertion about the topic]. [Specific context where this applies].
```
Not:
```
[General statement]. [Why this matters]. [Promise of insight].
```
### Supporting
```
[Specific example or data]. [What this reveals].
```
Not:
```
[Evidence]. And here's why that's a game-changer.
```
### Closing
```
[Implication or caveat that shows nuanced understanding].
```
Not:
```
[Summary]. [Call to action]. [Motivational kicker].
```
---
## Examples
### Before (AI)
> In my experience, I've found that the most successful leaders tend to share certain characteristics. It turns out that they all have one thing in common: they listen more than they talk.
### After (Expert)
> The best executives I've worked with share one habit: they ask questions before giving answers. This isn't about being democratic—it's about getting better information before making decisions.
---
### Before (AI)
> Here's the uncomfortable truth about product-market fit. Not everyone achieves it. Because it requires more than just a good product. It requires timing, market conditions, and often a bit of luck.
### After (Expert)
> Product-market fit depends on timing as much as the product itself. Three startups I advised had nearly identical offerings. One succeeded; two didn't. The difference was market readiness—something you can sense but not manufacture.
---
### Before (AI)
> The key to effective management is understanding that every person is different and requires a different approach. Let me explain what I mean.
### After (Expert)
> Management isn't one-size-fits-all. Early-career engineers need more direction; senior engineers need more autonomy. Get this wrong and you'll either frustrate or overwhelm them.
---
## Quality Checklist
- [ ] Contains at least one specific claim (not hedged)
- [ ] Includes concrete details (numbers, names, specific contexts)
- [ ] Acknowledges at least one nuance or exception
- [ ] Demonstrates knowledge without announcing credentials
- [ ] Would sound credible to someone in the field
- [ ] Avoids generic advice that applies to everything
- [ ] No throat-clearing or setup phrases
---
## Expert Signals
### Show Expertise Through:
- Specific contexts ("In enterprise sales," "For seed-stage startups,")
- Named examples ("Like Stripe's approach to documentation")
- Quantified claims ("3x more effective" not "much better")
- Acknowledged limits ("This works for X, less so for Y")
- Counterintuitive insights (things only experience reveals)
### Avoid Expertise Theater:
- "In my extensive experience..."
- "As someone who has worked in this field for years..."
- "Trust me when I say..."
- "The data clearly shows..." (without showing data)
- "Everyone in the industry knows..."
---
## Common Transformations
| Generic | Expert |
|---------|--------|
| "Teams should communicate better" | "Engineering teams should document decisions in ADRs, not Slack threads" |
| "Data is important" | "Customer cohort analysis reveals retention problems six months before churn" |
| "Move fast" | "Ship weekly in early-stage; bi-weekly after you hit 50K users" |
| "Hire good people" | "Hire for judgment, not just skills—you can teach React, you can't teach taste" |
@@ -0,0 +1,136 @@
# Preset: Story Lean
Narrative flow, one example. Show through story, not explanation.
---
## Voice Characteristics
- **Sentence length**: Highly varied. Short for impact, long for immersion.
- **Paragraph length**: Flexible. Follow the story's rhythm.
- **Tone**: Observational. Let the story carry meaning.
- **Structure**: Scene → tension → resolution → insight.
---
## Rules
1. **One story, one point** - Don't stack examples
2. **Concrete details** - Sensory, specific, memorable
3. **Let readers conclude** - Don't explain the moral
4. **Present tense for scenes** - Brings immediacy
5. **Dialogue when possible** - Shows character
---
## Structural Patterns
### Opening
```
[Scene-setting detail that hooks]. [Character in situation].
```
Not:
```
[Abstract statement]. [Promise of story]. [Why this matters].
```
### Middle
```
[Action/dialogue]. [Complication]. [Response].
```
Not:
```
[Description]. [Explanation of what it means].
```
### Closing
```
[Resolution]. [Observation that resonates].
```
Not:
```
[Summary of lesson]. [Application to reader]. [Call to action].
```
---
## Examples
### Before (AI)
> Let me share a story that illustrates an important lesson about leadership. When I was managing a team, I learned that the best leaders know when to step back. Here's what happened. One day, a junior employee came to me with a problem. Instead of solving it for them, I asked what they would do. And that's when I realized: my job wasn't to have all the answers.
### After (Story Lean)
> Three months into the job, Maria knocked on my door. "The client wants it by Friday. We can't do it."
> I almost said we'd figure it out. Instead: "What do you think we should do?"
> She paused. "Push back. Tell them why the timeline doesn't work."
> She walked out. Handled it herself. The client agreed to two more weeks.
> I never answered a question for her again.
---
### Before (AI)
> Failure is often the best teacher. I learned this the hard way when I launched my first product. It was a complete disaster, but looking back, I realize it taught me everything I needed to know about what customers actually want.
### After (Story Lean)
> We launched on a Tuesday. By Thursday, we had 47 signups and 46 support tickets. The forty-seventh user left a one-star review: "Does this even work?"
> It didn't. Not really. We'd built what we thought was clever. Users wanted something that was useful.
> I still have that review saved. It's the most expensive lesson I ever got for free.
---
### Before (AI)
> Here's an interesting story about persistence. Thomas Edison famously failed thousands of times before inventing the light bulb. This teaches us that success requires patience and dedication. We should all learn from Edison's example.
### After (Story Lean)
> Edison's notebooks from 1879 show the same experiment repeated with minor variations. Bamboo filament attempt #47: "Promising—lasted 8 minutes." Attempt #48: "40 hours continuous light."
> One word separates those entries: overnight.
---
## Quality Checklist
- [ ] Opens with scene or action, not abstraction
- [ ] Contains specific, sensory details
- [ ] Shows rather than explains the meaning
- [ ] Uses dialogue or specific quotes
- [ ] Doesn't announce "here's a story about..."
- [ ] Doesn't end with "and that's when I learned..."
- [ ] Reader can draw their own conclusion
---
## Detail Selection
### Include:
- Specific names, dates, numbers
- Dialogue (direct quotes)
- Sensory details (what you saw, heard)
- The moment of change
- Telling small details
### Exclude:
- Background context that doesn't serve the story
- Explanation of why details matter
- Multiple examples that dilute focus
- Your internal monologue about lessons
- Explicit statement of the moral
---
## Common Mistakes
| Mistake | Fix |
|---------|-----|
| Starting with "Let me tell you a story" | Start with the story |
| "And that's when I realized..." | Let the action show the realization |
| Explaining what the story means | Trust the reader |
| Generic details ("a team," "a project") | Specific details ("the API team," "the invoicing rewrite") |
| Multiple stories stacked | One story, fully developed |
| Ending with explicit lesson | End with the image or moment |
@@ -0,0 +1,121 @@
# Preset: Warm Human
Friendly tone, approachable rhythm. Clear but not cold.
---
## Voice Characteristics
- **Sentence length**: 8-20 words. Varied naturally.
- **Paragraph length**: 2-4 sentences. Conversational flow.
- **Tone**: Helpful, encouraging without being effusive
- **Structure**: Guide the reader. Use light transitions.
---
## Rules
1. **Conversational, not casual** - Write like you talk to a respected colleague
2. **Light contractions** - "You'll" over "You will." "It's" over "It is."
3. **Occasional "you"** - Address the reader when helpful
4. **Soft landing** - Don't end abruptly. Give readers somewhere to go.
5. **One anecdote max** - Examples warm up abstract ideas
6. **Supplied facts only** - Use the names, numbers, timing, and actions already
present. Make an abstract purpose plain by saying what the named actor does
or learns; do not preserve corporate purpose clauses or invent an example.
---
## Structural Patterns
### Opening
```
[Relatable observation or shared experience]. [Main point].
```
Not:
```
[Dramatic statement]. [Let that sink in].
```
### Supporting
```
[Evidence]. This means [implication for the reader].
```
Not:
```
[Evidence]. And that's a game-changer.
```
### Closing
```
[Practical takeaway or invitation to reflect].
```
Not:
```
[Summary]. [Call to action]. [Kicker].
```
---
## Examples
### Before (AI)
> Here's the uncomfortable truth: meetings are killing your productivity. Not because meetings are inherently bad. Because most meetings lack structure. Let me explain.
### After (Warm)
> You've been in meetings that could have been emails. The problem isn't meetings—it's unclear agendas. A simple fix: write the meeting's goal in the calendar invite.
---
### Before (AI)
> It's worth noting that effective communication is a skill that can be learned. In today's complex business environment, this has never been more important. Here's why that matters.
### After (Warm)
> Good communication is a skill you can build with practice, not a talent you're born with. Most people see results within a few weeks of deliberate effort.
---
### Before (AI)
> The key to success in any endeavor is perseverance. Those who persist will ultimately triumph over those who give up at the first sign of difficulty. Full stop.
### After (Warm)
> Persistence matters more than talent. Most people quit right before the breakthrough. If something matters to you, give it another few months.
---
## Quality Checklist
- [ ] Uses contractions naturally (you'll, it's, don't)
- [ ] Addresses reader with "you" where appropriate
- [ ] Has at least one concrete example or scenario
- [ ] Ends with something useful (tip, reflection, next step)
- [ ] Sounds like a helpful colleague, not a motivational poster
- [ ] No performative emphasis (Full stop. Period. Let that sink in.)
- [ ] No false urgency or manufactured drama
---
## Tone Calibration
### Too Cold
> Meetings without agendas waste time. Define outcomes before scheduling.
### Just Right (Warm)
> You've probably been in meetings that went nowhere. A simple fix: write the expected outcome in the calendar invite. It takes 30 seconds and saves everyone's time.
### Too Warm (Overdone)
> Hey friend! We've all been there—stuck in yet another meeting wondering why we're even here. Good news: there's a super easy fix that'll change everything!
---
## Avoid
| Pattern | Problem |
|---------|---------|
| "Hey friend!" / "Hey you!" | Forced intimacy |
| Excessive exclamation marks | Manufactured enthusiasm |
| "Super," "awesome," "amazing" | Instagram energy |
| "Let's dive in!" | AI tell |
| "I hope this helps!" | Sycophantic closing |
| "I hope this email finds you well" | Canned warmth |
| Starting with "So," | Podcast-speak |
@@ -0,0 +1,122 @@
# Teach Calibration Game (A/B Preference Elicitation)
An agent-hosted micro-game that calibrates voice preferences the user can state but
never bothered to write samples for. Far lower effort than `teach`: harvest bootstraps
samples, the A/B game calibrates dimension preferences, and both feed the voice card.
`teach` offers this game (`references/commands/teach.md`, step 3) when sample coverage
or confidence stays thin; this file is the canonical flow and rationale.
## Deterministic set
Five dimensions are reachable by reversible, seed-deterministic transforms of a base
passage (`scripts/calibrate_pairs.py`): `contractions`, `em_dash`, `sentence_length`,
`connectives`, `staccato`. Every transform preserves the base passage's must-preserve
constraints (`scripts/extract_constraints.py`) or the command exits 3 rather than
silently dropping a fact.
Register and warmth are NOT in this set — they need LLM generation, not a mechanical
transform, and are out of scope for `calibrate_pairs.py`. If the game wants to cover
those dimensions, generate the pair with a model and skip the determinism/constraint
guarantees this script gives for free.
## Game flow
1. Pick base passages from the user's own harvested samples (`references/harvest.md`)
when available; otherwise fall back to neutral seed texts. Paragraph-sized (60-150
words) so a round takes seconds to read.
2. For the dimension needing the most attention, run:
```bash
python3 scripts/calibrate_pairs.py generate --base passage.txt --dimension DIM --seed N
```
Increment the seed each round for that dimension so repeated rounds against the same
base passage still vary. If the command exits 3 ("dimension not expressible in this
passage"), pick a different base passage or move to the next dimension — do not force
a transform through.
The output also carries `a_flags`/`b_flags`: the `banned_phrase_scan.py` category names
each variant trips (empty when clean). A generated variant tripping the scanner is not a
reason to decline the pair — see "Voice overrides defaults" below.
3. Present `a_text` and `b_text` to the user as "is A or B closer to how you'd write
this?" without naming the dimension or which side is which pole (avoid anchoring).
Accept "neither" as a valid answer.
Randomize which pole is shown as "A" and which as "B" each round — a seeded
shuffle keyed on the round (e.g. `pair_id`) so it is reproducible, not a fresh
coin flip you can't reconstruct later. Always displaying `calibrate_pairs.py`'s
own `a_text` as "A" would let the user learn the game's fixed pole ordering
after a few rounds, reintroducing the anchoring this step is trying to avoid.
Record the mapping — which of the script's `a_text`/`b_text` was actually shown
as "A" this round — alongside the round so the recorded `choice` and
`a_label`/`b_label` in step 4 resolve to the correct pole regardless of what
was displayed on screen.
4. Append one line to `.unslop/voice/<name>/preferences.jsonl`:
```json
{"pair_id": "...", "dimension": "...", "choice": "a|b|neither", "ts": "...",
"a_label": "<A's pole from transform_applied>", "b_label": "<B's pole>"}
```
`a_label`/`b_label` come straight off `calibrate_pairs.py`'s `transform_applied`
(`"<dimension>:<pole>"` names B's pole; A sits at the dimension's other named pole).
Recording them is what lets `calibrate_score.py` report a semantic direction instead
of a bare "a"/"b" tally, and what makes conflict detection possible.
5. After each round, run:
```bash
python3 scripts/calibrate_score.py --preferences preferences.jsonl
```
Stop playing a dimension once its `confidence >= 0.7` OR it has reached `k >= 9`
observations, whichever comes first. Below `k = 5` observations a dimension is always
reported `"insufficient"` regardless of how lopsided the tally looks — five is the
floor before a lean means anything. A dimension can also come back `"tied"` with
`preferred: null` when its top two tallies are exactly equal at `k >= 5` — that's a
real "no lean yet" result, not insufficient data, and not a coin flip the aggregator
should silently resolve one way. Play it a couple more rounds. Replaying the same
round (same `pair_id`) is safe: the aggregator dedups by `pair_id`, keeping only the
latest row by `ts`, so a retried write never double-counts or lets a stale choice win.
6. Use `--next` to pick which dimension to play next:
```bash
python3 scripts/calibrate_score.py --preferences preferences.jsonl --next
```
Ordering: fewest observations first, then lowest confidence, tie-broken by the fixed
dimension order (`contractions, em_dash, sentence_length, connectives, staccato`) so
the choice is reproducible from the same preferences file.
7. When a sample-derived profile exists (`profile.json`, WP10a), run:
```bash
python3 scripts/calibrate_score.py --preferences preferences.jsonl --profile profile.json
```
Any confident (`>= 0.7`) stated preference that contradicts the profile's measured
value comes back as a `conflicts[]` record naming both values and their provenance,
e.g. "stated preference for 'short' (confidence 0.70) contradicts
avg_sentence_length=22.4 measured from samples." Surface every conflict to the user
verbatim — do not silently pick a winner. Let the user say which one wins; record
that choice's provenance on the card.
8. Hand the final per-dimension summary to the teach-card step. Every claim on the card
sourced from this game carries provenance `"stated-preference"`, distinct from
`"measured-from-samples"` values pulled from the profile directly.
## Voice overrides defaults
A generated variant can legitimately trip `banned_phrase_scan.py` — a staccato pole reads
as `anti_slop_register` (three short fragments in a row is the scanner's own AI-cadence
tell), a plain connective can occasionally read as `filler_opener`. Do not decline these
pairs or quietly steer the user away from the flagged pole; the scanner encodes a default
register guard, and this game exists specifically to find out when a user's real voice
sits outside that default.
When a user's choices consistently land on a pole that `a_flags`/`b_flags` marks as
flagged (e.g. they keep picking the staccato side even though it trips
`anti_slop_register`), record the preference together with its flags. Surface the tension
to the user once, not on every round — something like: "you prefer staccato; the default
register guard flags it as an AI cadence tell — your voice wins in your profile." Do not
re-litigate it every round once the user has answered.
On the final per-dimension summary (step 8), mark that dimension's provenance
`"user-preference overrides register guard"` instead of a bare `"stated-preference"`, so
the teach-card step (and anyone reading the card later) can see that the scanner's default
was deliberately overridden, not silently ignored. This is the same conflict-surfacing
philosophy as the profile conflicts below: voice beats defaults, but only visibly.
## Provenance discipline
The card must always be able to say, for any voice claim, whether it came from the
user's own samples (measured) or from a choice the user made in this game (stated).
Conflicts are a feature, not a bug to smooth over — surfacing them is the whole point
of running both a profile and a preference game.
@@ -0,0 +1,59 @@
# /unslop cleanup
Surface AI patterns as reviewable edits the user accepts or rejects, rather than
a finished rewrite. Agent-invoked, never a background daemon. Route here on
`/unslop cleanup` or when the user wants suggestions instead of a silent rewrite.
Set `$INPUT` to the source.
## Report-only variant
When the user says "flag only", "just detect", "audit only", "don't change
anything", or passes `--report`, change nothing. Scan and report:
```bash
python3 scripts/banned_phrase_scan.py <<< "$INPUT"
python3 scripts/structure_scan.py <<< "$INPUT"
python3 scripts/silhouette_scan.py <<< "$INPUT"
python3 scripts/readability_metrics.py <<< "$INPUT"
```
Report each issue by span, category, severity, and why it reads as machine-
written, separating clear problems from register-dependent judgment calls (the
audit-only shape in SKILL.md **Output Format**). Re-read negations, scope, and
certainty yourself; the scanners are necessary, not sufficient.
## Suggestion flow
```bash
python3 scripts/suggest.py document.md # emit suggestions
python3 scripts/suggest.py doc.md --apply-replacements repl.json # merge model replacements
python3 scripts/check_suggestions.py suggestions.json # blocking contract gates
```
- **Detection is cheap and deterministic.** `suggest.py` runs both scanners and
emits LSP-style suggestions `{span, severity, category, rationale,
suggested_replacement, phrased_as_question}`, ordered and non-overlapping.
- **Replacement generation is delegated to a stronger model.** `suggest.py`
leaves `suggested_replacement` null; the stronger model fills them, merged back
via `--apply-replacements`.
- **Hard findings** become direct replacements; **soft findings** are register-
dependent, so their rationale is phrased as a question
(`phrased_as_question: true`).
- **Suggestions are surfaced to the user, never silently applied.**
## Contract gates
`check_suggestions.py` is the blocking contract. Every replacement clears all
four gates or it is rejected:
- `span-minimality`: the edit changes only its span (a whole-sentence rewrite
fails).
- `replacement-scanner`: each replacement passes both scanners in isolation and
adds no new violation in context.
- `accept-all`: applying every suggestion yields a document that passes both
scanners with `validate_preservation` exit 0 versus the original.
- `span-overlap`: spans must not overlap.
Non-English input is declined the way the scanners decline it: `suggest.py`
returns `non_english: true` with no suggestions. For a full rewrite instead of
suggestions, route to `references/commands/rewrite.md`.
@@ -0,0 +1,51 @@
# /unslop contribute
Turn one AI-ism caught in the wild into a scanner contract example and a structured PR. Route
here on "contribute a pattern", "add this tell", or "I found a new AI-ism".
Everything stays offline until the user approves both publication and the final
PR body. Read `references/contribute.md` for the precheck exit codes, redaction
discipline, row-diff verification, and the non-maintainer fork path.
## Flow
1. **Precheck.** `python3 scripts/contribute.py precheck /abs/path/snippet.txt`.
Stop if it exits 3. The tell is already covered, so add a REC row rather
than opening a fresh false-negative row.
2. **User confirmation gate #1 — publication.** Show the exact snippet and ask
whether it may go in a public repo. Offer redaction hints from
`python3 scripts/extract_constraints.py < snippet.txt`. Redact only with
explicit approval, and keep the tell byte-for-byte intact.
3. **Scaffold** the working bundle under `.unslop/contrib/<slug>/` (do not
commit it):
```bash
python3 scripts/contribute.py scaffold --snippet /abs/path/snippet.txt \
--tell "exact substring" --category significance_inflation \
--pattern-name exact-substring-slug --redact "Alice=NAME"
```
4. **Implement eval-first**, following `references/maintenance.md`: copy
`row_fn.json` into `evals/fixtures/contracts/scanner-examples.json`, renumber from the live
maxima, then set `exact_total` to the new `examples` length. Add the
literal-use FP example (and a REC example if a word is gated
behind a collocation). Confirm the FN example is **red** before you touch the
scanner, then update `scripts/banned_phrase_scan.py` and
`references/taboo-phrases.md` until it goes **green** while the FP example holds.
5. **Verify:** `python3 scripts/contribute.py verify --bundle .unslop/contrib/<slug>`.
It refuses TODO markers, checks specimen fidelity, and captures the
red-to-green transition.
6. **Full deterministic gate battery:** `python3 evals/check.py --full`.
7. **Render the PR body:**
`python3 scripts/contribute.py report --bundle .unslop/contrib/<slug> > pr-body.md`.
8. **User confirmation gate #2 — publication of the PR.** Show the final PR
body, get approval, then branch, `git add` the three changed files, commit,
and use `gh pr create` (or the non-maintainer fork path in
`references/contribute.md`). Network access belongs to the host agent.
The resulting branch contains the contract example, scanner change, and taboo-catalog
entry.
@@ -0,0 +1,66 @@
# /unslop mimic
Single-pass voiced writing. Draft or rewrite in a taught voice, then clear the
full gate battery. Route here on "write this like me", "match my voice", or
"mimic this author" once a voice card exists. Read `references/mimic.md` for card
anatomy, scoring, and failure modes.
## Flow
1. Load the situation sheet the task needs. Read `card.md` for the always-on core
(rhythm, contraction habits, punctuation, openers, the Never list) and the
one `card/<situation>.md` sheet that matches what you are writing. Load only
what the task needs; the card is layered so generation stays cheap.
2. Draft or rewrite in that voice. Follow the card's measured markers and
verbatim snippets; do not invent a habit the samples cannot support.
3. Run the output through **every** removal gate:
```bash
python3 scripts/banned_phrase_scan.py <<< "$OUTPUT"
python3 scripts/structure_scan.py <<< "$OUTPUT"
python3 scripts/readability_metrics.py <<< "$OUTPUT"
python3 scripts/validate_preservation.py original.txt "$OUTPUT_FILE"
python3 scripts/diff_check.py original.txt "$OUTPUT_FILE"
```
4. Score the voice:
```bash
python3 scripts/voice_score.py --profile profile.json "$OUTPUT_FILE"
```
## The rule
A mimic that scores well on voice but trips a slop gate is **rejected**. Voice
never buys an exemption from the constitution — the removal gates are hard, the
voice score is a guide. Meaning preservation versus the original draft is its own
hard gate, never blended into the voice score.
## Refine — when one pass is not enough
Most users teach once and mimic cheaply thereafter. When a single pass keeps
scoring short of the target voice, hill-climb with the refine loop — worth the
cost only then:
```bash
python3 evals/run_mimic_refine.py --samples samples/ --draft draft.md --out OUT
```
LIVE (default) calls `--generate-cmd` (default `claude -p`) once per beam per
iteration; `--candidates-dir DIR` runs the loop logic with no model calls. Hard
gates (banned-phrase, structure, draft→candidate preservation, copy-gate, 150-
word floor) discard a candidate regardless of score; survivors are scored against
a held-out DEV split with the full `voice_score` composite, and the best-so-far
never regresses. The **divergence guard** halts and sets
`reward_hacking_warning` when the A composite improves while DEV worsens for two
iterations — the reward-hacking signature. Confirm a win only when
`mimic_stats.py` shows the CI lower bound > 0 and p < 0.05. Internals, baselines,
and the failure table are in `references/mimic.md`.
## Voice check
"Does this sound like me?" — score only, change nothing. Run
`python3 scripts/voice_score.py --profile .unslop/voice/<name>/profile.json
--impostors evals/fixtures/voice/impostors --seed 7 <draft>` and report the
composite (lower is more them), the GI rank, and the two or three metric
deltas that explain the score, in plain words ("your sentences run longer than
usual; contractions match"). No rewrite unless asked. This is the cheapest
voice interaction and the usual one after teach: check drafts often, commission
rewrites rarely.
@@ -0,0 +1,46 @@
# /unslop rewrite
The default command. Read `references/core-contract.md`; it is authoritative.
Use two passes: diagnose, then repair confirmed findings and validate.
Input arrives as an argument, a file path, or stdin. Set `$INPUT` to the source
and `$OUTPUT` to your rewrite.
## Pass 1 — diagnose
1. Extract the facts and constraints that must survive:
```bash
python3 scripts/extract_constraints.py <<< "$INPUT"
```
2. Scan the source so the rewrite answers evidence, not a hunch:
```bash
python3 scripts/banned_phrase_scan.py <<< "$INPUT"
python3 scripts/structure_scan.py <<< "$INPUT"
python3 scripts/silhouette_scan.py <<< "$INPUT"
python3 scripts/readability_metrics.py <<< "$INPUT"
```
Pass `--genre docs` or `--genre social` only when the input truly belongs to
that genre. Add `banned_phrase_scan.py --include-quoted` only when the user
wants quoted examples audited too.
3. Read the selected preset only for voice. It cannot authorize a finding.
4. Classify exact spans as confirmed findings or protected source. Do not
fact-check, use the current date, or treat missing support as an AI tell.
Scanner rows remain candidates until contextual review confirms a defect.
## Pass 2 — reconstruct
Apply the core contract. Edit only a sentence containing a confirmed finding,
make the smallest repair, and copy all other sentences byte-for-byte. If there
are no findings, return the source exactly. Undo any edit that changes a fact,
meaning, register, attribution, or protected domain phrase.
## Validate
Run the gate battery and apply the blocking semantics in
`references/core-contract.md` **Validation**.
Return the cleaned text only, unless the user asked for the strict analysis
block in SKILL.md **Output Format**.
For multi-agent execution, use `references/pipeline.md`; for orchestrated
detector packs, use `references/packs/`.
@@ -0,0 +1,77 @@
# /unslop teach
Build a reusable voice from a writer's samples. **Agent-driven end to end**: the
user supplies only approvals and answers, never a directory or a command. Route
here on "teach my voice", "learn how I write", or when the user offers samples to
learn from. Any samples the user provides are fair game; there are no rights or
attestation checks.
Read `references/mimic.md` for card anatomy, sample requirements, and scoring
before you finish; this file is the flow.
## 1. Gather samples (harvest)
If the user offers none, default to bootstrapping. Harvest their transcripts and
writing folders per `references/harvest.md`:
```bash
python3 scripts/harvest_samples.py SOURCE [SOURCE...] -o candidates.json
python3 scripts/harvest_classify.py --candidates candidates.json --mode heuristic
```
Present the ranked candidates with their sources and flags; the user approves or
rejects each. Treat `suspect_ai` candidates skeptically — that human review is
the contamination defense, so never auto-approve a flagged paste. Copy only
approved samples into `.unslop/voice/<name>/samples/`. The adapter internals,
contamination tripwire, and privacy rules live in `references/harvest.md`.
## 2. Build the profile and card
Create `.unslop/voice/<name>/` yourself (it is gitignored) and run:
```bash
python3 scripts/voice_profile.py samples/ -o profile.json
python3 scripts/voice_card.py --profile profile.json --samples samples/ --out . --name <name> --provenance
```
`voice_profile.py` is the machine fingerprint the scorer refers to.
`voice_card.py` writes the layered card the generating model follows (`card.md`
plus `card/<situation>.md` sheets) and, with `--provenance`, a `provenance.json`
audit trail. Both scripts read only `.txt` and `.md` files; convert samples
first. `voice_card.py` refuses (exit 2) when the profile does not describe the
supplied samples, so a stale profile can never drive a card.
## 3. Surface confidence and gaps
Show the user `card.md`. If the profile is `low_confidence` (thin or cross-genre
samples), say the voice is provisional and ask for more same-genre samples before
trusting a mimic. For an **Uncovered** dimension the target writing will need,
ask for one short sample that exercises it — and only those the task requires, not
all ten. Prompt templates:
| Dimension | Prompt |
|-----------|--------|
| explaining-technical | "Share something you wrote explaining how a thing works." |
| anecdote | "Share a few sentences telling a small story that happened to you." |
| argument | "Share something where you defended a claim." |
| disagreement | "Share something where you pushed back on an idea." |
| praise | "Share something where you recommended something you liked." |
| hedging-uncertainty | "Share something where you were unsure and said so." |
| numbers-data | "Share something you wrote that works with quantities." |
| addressing-reader | "Share a note or instructions written to a reader." |
`openings` and `closings` are structural and covered as soon as one document
exists, so they need no prompt. The full canonical table lives in
`references/mimic.md`.
When coverage or confidence stays thin and the user cannot supply more samples,
offer the A/B calibration game (`references/calibrate.md`): it elicits voice
preferences through reversible pair choices and feeds them to the card as
`stated-preference` provenance, distinct from `measured-from-samples`.
## 4. Close with a scored demo
Prove the loop before the user trusts it: mimic one paragraph per
`references/commands/mimic.md`, run `scripts/voice_score.py` and the scanners,
and show the results. A voiced demo that trips a slop gate fails the loop; do not
ship it with an excuse.
@@ -0,0 +1,106 @@
# Contributing a New AI-ism — internals
The routed procedure lives in `references/commands/contribute.md`. This
exhaustive companion covers every command flag, the redaction discipline, the
row-diff verification, and the non-maintainer fork path. Keep it offline until
the user has approved both publication and the final PR body.
## 1. Precheck
```bash
python3 scripts/contribute.py precheck /absolute/path/to/snippet.txt
```
If the snippet is already flagged, the command exits 3 and prints the covering
patterns/categories. Stop there. Add a REC example instead of a new
false-negative example.
## 2. User Confirmation Gate #1
Show the exact snippet to the user and ask whether it may be published in a
public repo. Offer redaction hints from:
```bash
python3 scripts/extract_constraints.py < /absolute/path/to/snippet.txt
```
Redact names, numbers, or private details only with explicit approval. The tell
must remain byte-for-byte intact.
## 3. Scaffold
```bash
python3 scripts/contribute.py scaffold \
--snippet /absolute/path/to/snippet.txt \
--tell "exact substring" \
--category significance_inflation \
--pattern-name exact-substring-slug \
--redact "Alice=NAME"
```
The bundle is written under `.unslop/contrib/<slug>/`. Do not publish it. It is
working material for the agent and reviewer.
## 4. Implement the Pattern
Follow `references/maintenance.md` in this order:
1. Copy `.unslop/contrib/<slug>/row_fn.json` into
`evals/fixtures/contracts/scanner-examples.json`.
2. Renumber the copied example from the live maxima in that table; do not keep
the `CONTRIB-FN-*` bundle id in the committed contract. Set `exact_total` to
the resulting `examples` array length.
3. Add the literal-use FP example for the category, and add a REC example if an existing
word is being gated behind collocations.
4. Run the scanner contract before implementation and confirm the FN example is
red while the FP and any REC examples encode the intended boundary.
5. Update the scanner and `references/taboo-phrases.md`.
6. Re-run `python3 scripts/contribute.py verify --bundle .unslop/contrib/<slug>`.
7. Before treating the example as ready, diff it against
`.unslop/contrib/<slug>/row_fn.json` and verify that only expected suite
fields changed, such as the id, category grouping, or row ordering; the
specimen stdin and assertion intent must still match because redaction or
implementation work does not authorize a silent change to the reported tell.
## 5. Verify
```bash
python3 scripts/contribute.py verify --bundle .unslop/contrib/<slug>
```
Verify refuses reports with TODO markers, checks specimen fidelity, captures the
red-to-green transition for the FN row, and records offline gate tails.
## 6. Run the Full Gate Battery
```bash
python3 evals/check.py --full
```
## 7. Render the Report
```bash
python3 scripts/contribute.py report --bundle .unslop/contrib/<slug> > /tmp/pr-body.md
```
## 8. User Confirmation Gate #2
Show the final PR body to the user. Only after the user approves publication:
```bash
git switch -c add-<slug>
git add evals/fixtures/contracts/scanner-examples.json scripts/banned_phrase_scan.py references/taboo-phrases.md
git commit -m "Add <slug> AI-ism pattern"
gh pr create --title "Add <category> pattern: <tell>" --body-file /tmp/pr-body.md
```
For non-maintainers:
```bash
gh repo fork --remote
git push --set-upstream <fork-remote> add-<slug>
gh pr create --repo <upstream-owner>/unslop --head <fork-owner>:add-<slug> --body-file /tmp/pr-body.md
```
The `gh` commands are for the host agent after approval. Scripts and evals must
never call `gh` or any network command.
@@ -0,0 +1,91 @@
# UNSLOP core rewrite contract
## Goal
Repair concrete AI-writing or clarity defects. Leave everything else unchanged;
prefer a no-op to an uncertain edit.
## Findings
Quote the smallest defective span and explain the contextual defect: formulaic
framing, empty or inflated abstraction, unsupported rhetorical certainty,
opaque or mixed metaphor, misleading heading, needless repetition, or clear
internal contradiction or ambiguity. Track every hard scanner match and other
defect as confirmed or protected, with a reason; repair every confirmed span.
One repair does not excuse another nearby defect.
Do not fact-check or infer truth from the date or outside knowledge. Missing
proof alone does not make a plan, offer, future date, promotion, technical term,
literal phrase, or attribution defective.
Compare the source's claims. Qualify a status contradicted by observations.
Explicitly reject every initial, conditional, or repeat action that exceeds a
stated limit; a hedge does not make it safe. Preserve evidence and quantities.
Keep attributed claims intact and state conflicts separately. Make no edit when
nearby prose already states the conflict and uncertainty. Attribution does not
protect a separate operative recommendation.
A scanner match alone never authorizes an edit. Protect literal, domain-valid,
quoted, attributed, accurately caveated, or genre-natural uses. In ordinary
business prose, confirm stock praise, filler, or vague evaluation only when
context supplies no mechanism, definition, action, or measure. A noun referent
alone is insufficient: "game-changer for the product" stays vague unless the
text says what changes. "Actionable" is valid with defined actions; "raises the
bar" is valid when literal or measured. Soft cadence and document-shape scores
never authorize edits alone.
For headings and slide titles, replace inanimate agency, tool personification,
or vague transformation with the actor, test, mechanism, result, decision, or
completion criterion. Renaming the same abstraction is not a repair. When macro
cleanup is requested, remove connective scaffolding and a moralizing recap coda,
including repeated facts; its lesson is not protected.
Inspect slogans and closing calls to action even when tools are quiet. Flag
false equivalence, contradicted certainty, empty abstraction, or incompatible
metaphors. Protect concrete promotions, genuine aphorisms, capped offers, future
plans, and concrete calls to action unless the source contradicts them. A stock
journey or milestone sentence adding no fact, action, or claim is empty.
## Rewrite
Edit only sentences with confirmed findings, using the smallest repair. Copy
every other sentence byte-for-byte in its original order and paragraph. With no
findings, return the source exactly.
Preserve facts, quantities, dates, names, quotations, citations, code, units,
scope, uncertainty, attribution, register, and meaning. Add no claims, advice,
personality, anecdote, certainty, or conclusion. Do not substitute stock phrases
or create staccato anti-slop prose. Preserve force-bearing "never", "must", and
"all" exactly in safety, security, legal, and technical rules. Presets change
delivery, not facts. For lists, return one validated replacement per item.
For audit-only requests, report findings and protections without rewriting;
report each requested load-bearing span separately. Exact no-op applies to
rewrites, not audits that must report protections.
## Decision rule
Return a finding only when the contextual defect is clearer than preservation.
## Validation
After every rewrite or voiced draft, run:
```bash
python3 scripts/validate_preservation.py original.txt transformed.txt
python3 scripts/banned_phrase_scan.py <<< "$OUTPUT"
python3 scripts/structure_scan.py <<< "$OUTPUT"
python3 scripts/silhouette_scan.py <<< "$OUTPUT"
python3 scripts/readability_metrics.py <<< "$OUTPUT"
python3 scripts/diff_check.py original.txt transformed.txt
```
Block preservation loss, introduced hard or `anti_slop_register` hits,
unjustified structural damage, corroborated or hard silhouette damage, staccato,
and strict-mode scores below 32/40. Use preservation `--strict` for legal,
medical, security, or scientific text. Preexisting soft cadence and
uncorroborated soft silhouette warnings are advisory.
For remaining structure damage, pass exact findings to
`evals/run_structure_climb.py`. Re-read negations, conditions, scope, certainty,
and party relationships.
@@ -0,0 +1,415 @@
# Edit Library
Before/after transformation examples showing specific edit patterns.
---
## Pattern 1: Throat-Clearing Removal
### Before
> Here's the thing: building products is hard. Not because the technology is complex. Because people are complex. Let that sink in.
### After
> Building products is hard, and the reason is people, not technology.
### Changes
- Removed opener "Here's the thing:"
- Stated the real reason in one flowing sentence — not a clipped "Not X. Y." fragment pair (that's its own AI tell)
- Deleted "Let that sink in" emphasis crutch
- 23 words → 12 words (48% reduction)
---
## Pattern 2: Filler + Unnecessary Reassurance
### Before
> It turns out that most teams struggle with alignment. The uncomfortable truth is that nobody wants to admit they're confused. And that's okay.
### After
> Teams struggle with alignment because nobody admits they're confused.
### Changes
- Cut "It turns out that"
- Cut "The uncomfortable truth is"
- Removed permission-granting "And that's okay"
- Joined cause and effect into one sentence instead of two clipped fragments
- 26 words → 9 words (65% reduction)
---
## Pattern 3: Business Jargon Stack
### Before
> In today's fast-paced landscape, we need to lean into discomfort and navigate uncertainty with clarity while leveraging our core competencies.
### After
> Move fast, handle uncertainty, and use what you're actually good at.
### Changes
- "In today's fast-paced landscape" → cut
- "lean into discomfort" → cut (filler motivation)
- "navigate uncertainty" → "handle uncertainty"
- "leveraging core competencies" → "use what you're actually good at"
- Kept it as one sentence — a staccato imperative triplet ("Move fast. Handle uncertainty. …") is its own AI cadence
- 22 words → 11 words (50% reduction)
---
## Pattern 4: Dramatic Fragmentation
### Before
> Speed. Quality. Cost. You can only pick two. That's it. That's the tradeoff.
### After
> Speed, quality, cost. Pick two.
### Changes
- Collapsed fragmented list into single sentence
- Removed performative "That's it. That's the tradeoff."
- 14 words → 5 words (64% reduction)
---
## Pattern 5: Rhetorical Setup
### Before
> What if I told you that the best teams don't optimize for productivity? Here's what I mean: they optimize for learning.
### After
> The best teams optimize for learning, not productivity.
### Changes
- Removed "What if I told you" rhetorical question
- Removed "Here's what I mean:" scaffolding
- Direct statement instead of reveal structure
- 22 words → 9 words (59% reduction)
---
## Pattern 6: Hedging Stack
### Before
> I think it's probably fair to say that perhaps most organizations might benefit from potentially considering a more agile approach.
### After
> Most organizations should try agile.
### Changes
- "I think" → cut
- "probably fair to say" → cut
- "perhaps" → cut
- "might benefit from potentially considering" → "should try"
- 21 words → 6 words (71% reduction)
---
## Pattern 7: Absolute Claims Softened
### Before
> Everyone knows that nobody ever succeeds without failing first. It's always been this way and it always will be.
### After
> Success usually follows failure.
### Changes
- "Everyone knows" → cut (false absolute)
- "nobody ever" → implicit
- "always been/always will be" → "usually"
- 19 words → 4 words (79% reduction)
---
## Pattern 8: Meta-Commentary Removal
### Before
> Hint: the answer isn't what you think. Plot twist: it's actually quite simple. Let me explain. Think about it: when you remove complexity, things get easier.
### After
> The answer is simple: remove complexity.
### Changes
- Removed "Hint:" meta-commentary
- Removed "Plot twist:" performative
- Removed "Let me explain" / "Think about it:"
- Direct statement instead of theatrical reveal
- 27 words → 6 words (78% reduction)
---
## Pattern 9: Binary Contrast Collapse
### Before
> It's not about working harder. It's about working smarter. It feels like the former is the answer. It's actually the latter. Make no mistake about it.
### After
> Work smarter, not harder.
### Changes
- Collapsed repetitive binary structure
- Removed "feels like / actually" false drama
- Removed "Make no mistake about it" emphasis
- 28 words → 4 words (86% reduction)
---
## Pattern 10: List Compression
### Before
> There are three things every leader needs: First, they need clarity. Second, they need courage. Third, they need consistency. These three C's are the foundation of effective leadership.
### After
> Leaders need clarity, courage, and consistency.
### Changes
- Collapsed numbered list into single sentence
- Removed "There are three things" setup
- Removed "these three C's" meta-reference
- Removed "foundation of effective leadership" filler
- 31 words → 7 words (77% reduction)
---
## Pattern 11: Intensifier Reduction
### Before
> This is incredibly important and truly fundamental to deeply understanding the absolutely essential nature of really effective communication.
### After
> This matters for effective communication.
### Changes
- "incredibly" → cut
- "truly fundamental" → cut
- "deeply understanding" → cut
- "absolutely essential" → cut
- "really effective" → "effective"
- 17 words → 6 words (65% reduction)
---
## Pattern 12: Passive to Active
### Before
> The decision was made by the team after the options were considered by leadership and the risks were evaluated by the analysts.
### After
> Leadership weighed the options, the analysts assessed the risks, and the team made the call.
### Changes
- "decision was made by the team" → "the team made the call"
- "options were considered by leadership" → "Leadership weighed the options"
- "risks were evaluated by analysts" → "the analysts assessed the risks"
- Passive → active throughout, joined as one sentence (three 3-word sentences in a row is staccato)
- 23 words → 15 words (35% reduction)
---
## Pattern 13: Fact-Heavy Preservation
### Before
> It's worth noting that in Q3 2024, Acme Corp reported revenue of $47.3M, which represents a truly impressive 23% year-over-year increase. This is a game-changing result for the industry.
### After
> Acme Corp reported $47.3M revenue in Q3 2024, up 23% year-over-year.
### Changes
- "It's worth noting that" → cut
- "truly impressive" → cut (editorializing)
- "game-changing result for the industry" → cut (filler)
- All facts preserved: Q3 2024, Acme Corp, $47.3M, 23%
- 33 words → 12 words (64% reduction)
---
## Pattern 14: Conditional Simplification
### Before
> If you want to succeed in the modern marketplace, you need to ensure that you're providing value to your customers in a way that differentiates you from the competition.
### After
> Differentiate through customer value.
### Changes
- "If you want to succeed in the modern marketplace" → cut (obvious)
- "ensure that you're providing value to your customers" → "customer value"
- "in a way that differentiates you from the competition" → "Differentiate through"
- 30 words → 4 words (87% reduction)
---
## Pattern 15: Question + Immediate Answer
### Before
> What's the key to retention? It's simple: make users feel valued. How do you do that? By listening. And why does that matter? Because people want to be heard.
### After
> Retain users by making them feel valued. Listen to them.
### Changes
- Collapsed Q&A theatrical structure
- Direct statements instead of rhetorical questions
- "Because people want to be heard" → implicit in "Listen"
- 32 words → 11 words (66% reduction)
---
## Pattern 16: Significance Inflation
### Before
> The Apollo program stands as a testament to human ingenuity, leaving an indelible mark on the history of space exploration and serving as a cornerstone of scientific achievement.
### After
> The Apollo program put twelve people on the moon between 1969 and 1972.
### Changes
- "stands as a testament to human ingenuity" → cut (inflated copula + significance)
- "leaving an indelible mark" → cut (cliché)
- "cornerstone of scientific achievement" → cut (inflation)
- Replaced abstract praise with the specific fact
- 30 words → 13 words (57% reduction)
---
## Pattern 17: Promotional Language
### Before
> Nestled in the heart of downtown, the restaurant boasts a world-class menu featuring breathtaking flavors. A hidden gem that's a must-visit for any food lover.
### After
> The restaurant, on Main Street downtown, serves Southern-French fusion. The duck confit tacos are worth the trip.
### Changes
- "Nestled in the heart of" → specific location
- "boasts a world-class menu" → describes the actual cuisine
- "breathtaking flavors" → named a specific dish
- "hidden gem" / "must-visit" → concrete recommendation
- 27 words → 18 words (33% reduction)
---
## Pattern 18: Superficial -ing Analysis
### Before
> The company reported 40% revenue growth, highlighting the effectiveness of its strategy, showcasing the strength of its market position, and underscoring the importance of innovation in the tech sector.
### After
> The company's revenue grew 40%. Its new API product drove most of that — enterprise contracts tripled.
### Changes
- Deleted all three participial clauses (highlighting, showcasing, underscoring)
- Replaced fake analysis with actual causal explanation
- 31 words → 17 words (45% reduction)
---
## Pattern 19: Copula Avoidance
### Before
> The library serves as a community hub that functions as a gathering place and represents a vital resource for residents.
### After
> The library is where the neighborhood meets. Residents use it for everything from job searches to story time.
### Changes
- "serves as" → "is"
- "functions as" → cut (redundant with "serves as")
- "represents a vital resource" → concrete examples of use
- 20 words → 19 words (same length, more information)
---
## Pattern 20: Negative Parallelism
### Before
> It's not just about building a product. It's about building a community. Not only does the platform connect users, but it also fosters meaningful relationships and drives engagement.
### After
> The platform's value is its community. Users keep coming back because they've built relationships there.
### Changes
- "It's not just about X, it's about Y" → stated Y directly
- "Not only...but also..." → collapsed into cause-effect
- "fosters meaningful relationships and drives engagement" → concrete behavior
- 30 words → 16 words (47% reduction)
---
## Pattern 21: Communication Artifacts
### Before
> Great question! I'd be happy to help with that. Certainly, there are several approaches you could take. I hope this helps — let me know if you need anything else!
### After
> Three approaches work here: [approach 1], [approach 2], [approach 3].
### Changes
- "Great question!" → cut (chatbot flattery)
- "I'd be happy to help" → cut (chatbot compliance)
- "Certainly" → cut (chatbot filler)
- "I hope this helps" / "let me know if..." → cut (chatbot sign-off)
- 31 words → 9 words (71% reduction)
---
## Pattern 22: False Ranges
### Before
> The festival attracts visitors from young professionals to retirees, from local residents to international tourists, spanning everything from music enthusiasts to casual attendees.
### After
> The festival draws 50,000 people annually, mostly locals and tourists from neighboring states.
### Changes
- Collapsed three stacked "from X to Y" ranges into specific data
- Replaced vague breadth claim with actual numbers
- 25 words → 13 words (48% reduction)
---
## Pattern 23: Generic Positive Conclusion
### Before
> As the industry continues to evolve, the future looks bright. Exciting times lie ahead for all stakeholders involved. One thing is certain: innovation will continue to shape the landscape.
### After
> Two trends to watch: edge computing costs dropping 30% annually, and enterprise AI adoption doubling each quarter.
### Changes
- "continues to evolve" → cut (says nothing)
- "the future looks bright" → cut (empty optimism)
- "exciting times lie ahead" → cut (content-free)
- "one thing is certain" → replaced with specific predictions
- 30 words → 16 words (47% reduction)
---
## Pattern 24: Compound Example (Multiple Pattern Types)
### Before
> In today's rapidly evolving digital landscape, the groundbreaking initiative serves as a testament to the company's commitment to innovation. Not only does it showcase their robust capabilities, but it also underscores the importance of fostering collaboration. Experts argue this represents a pivotal moment. The future looks bright.
### After
> The company shipped a new collaboration tool. Early numbers: 2,000 teams signed up in the first week, 85% retention at day 30.
### Changes
- "In today's rapidly evolving digital landscape" → cut (filler)
- "groundbreaking initiative serves as a testament to" → named the actual thing
- "Not only...but also..." → cut (negative parallelism)
- "showcasing robust capabilities" / "underscoring importance" → cut (superficial -ing)
- "fostering collaboration" → named the product category
- "Experts argue" → cut (vague attribution)
- "represents a pivotal moment" → replaced with data
- "The future looks bright" → cut (generic conclusion)
- 47 words → 22 words (53% reduction), information density dramatically higher
---
## Transformation Principles
1. **Cut setup, keep substance** - Remove everything before the actual point
2. **One structure, not three** - If it can be one sentence, make it one
3. **Let facts speak** - Numbers don't need emphasis crutches
4. **Active beats passive** - Who did what, not what was done by whom
5. **Trust the reader** - They'll get it without "Think about it:"
6. **Verbs do the work** - Concrete actions, not abstract processes
7. **Vary the rhythm** - Concise is not the same as staccato. After cutting, check the result isn't a run of tiny sentences or a bare "Not X. Y." contrast — that clipped cadence is its own AI tell. Mix short sentences with a longer one.
@@ -0,0 +1,169 @@
# Fact Preservation Rules
Certain content must survive transformation unchanged. This reference defines what to protect.
---
## Absolute Preservation (Never Modify)
These elements must appear in the output exactly as they appear in the input.
### Numbers & Quantities
- Specific figures: `$47.3M`, `23%`, `1,500 users`
- Dates: `Q3 2024`, `March 15, 2024`, `2024-03-15`
- Times: `9:00 AM PST`, `14:30 UTC`
- Measurements: `5.2kg`, `100ms`, `2TB`
- Ranges: `$50-75K`, `3-5 years`, `10-15%`
- Counts: `3 engineers`, `5 steps`, `47 countries`
### Proper Nouns
- Company names: `Acme Corp`, `OpenAI`, `Google`
- Product names: `GPT-4`, `iPhone 15`, `TensorFlow`
- Person names: `John Smith`, `Dr. Chen`
- Place names: `San Francisco`, `Building 43`
- Brand terms: `React`, `Kubernetes`, `AWS Lambda`
### Technical Terms
- API endpoints: `/api/v2/users`
- Code references: `handleSubmit()`, `UserContext`
- Config values: `max_retries: 3`, `timeout: 30s`
- Version numbers: `v2.1.0`, `Node 20.x`
- File paths: `/src/components/Button.tsx`
### Quoted Material
- Direct quotes: `"We're just getting started" - CEO`
- Cited text: anything in quotation marks attributed to a source
- Code snippets: preserve exactly, including whitespace
### URLs & References
- Links: `https://example.com/page`
- DOIs: `10.1234/journal.2024.001`
- ISBNs: `978-0-123456-78-9`
- Citations: `[1]`, `(Smith et al., 2024)`
---
## Semantic Preservation (Keep Meaning)
These can be rephrased but meaning must stay identical.
### Causal Claims
- Original: "A caused B"
- OK: "B resulted from A"
- NOT OK: "A influenced B" (weaker claim)
### Comparisons
- Original: "X is 3x faster than Y"
- OK: "X outperforms Y by 3x"
- NOT OK: "X is faster than Y" (lost quantifier)
### Conditional Statements
- Original: "If A, then B"
- OK: "B happens when A"
- NOT OK: "A and B" (lost conditionality)
### Negations
- Original: "X does not support Y"
- OK: "X lacks Y support"
- NOT OK: "X partially supports Y" (opposite meaning)
### Scope Qualifiers
- Original: "Most users (73%) prefer X"
- OK: "73% of users prefer X"
- NOT OK: "Users prefer X" (lost scope)
---
## Constraint Extraction
Before transforming, extract these as must-preserve spans:
```python
CONSTRAINT_PATTERNS = {
# Numbers with context
"currency": r"\$[\d,]+\.?\d*[KMB]?",
"percentage": r"\d+\.?\d*%",
"date_iso": r"\d{4}-\d{2}-\d{2}",
"date_natural": r"(Q[1-4]\s+\d{4}|(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[a-z]*\s+\d{1,2},?\s+\d{4})",
"time": r"\d{1,2}:\d{2}\s*(?:AM|PM|UTC|PST|EST)?",
"measurement": r"\d+\.?\d*\s*(?:ms|s|min|hr|KB|MB|GB|TB|kg|lb|m|km|mi)",
# Identifiers
"url": r"https?://[^\s]+",
"email": r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}",
"code_ref": r"`[^`]+`",
"quote": r'"[^"]{10,}"',
# Proper nouns (NER-detected)
"org": "ORGANIZATION entities",
"person": "PERSON entities",
"product": "PRODUCT entities",
"location": "GPE/LOC entities"
}
```
---
## Verification Process
After transformation:
1. **Extract all constraints from original**
2. **Check each constraint exists in output**
3. **Flag any missing or altered constraints**
### Pass Criteria
- All absolute preservation items present and unchanged
- All semantic preservation items equivalent in meaning
- No fact inversions or significant alterations
### Fail Examples
| Original | Output | Problem |
|----------|--------|---------|
| "$47.3M revenue" | "$47M revenue" | Precision lost |
| "23% increase" | "significant increase" | Number removed |
| "John Smith, CEO" | "the CEO" | Name removed |
| "https://example.com" | "[link]" | URL removed |
| "Q3 2024" | "recently" | Date removed |
| "does not support" | "partially supports" | Meaning inverted |
---
## Special Cases
### Approximations in Original
If original uses approximate language, preserve the approximation:
- "about 50%" → OK to keep as "about 50%" or "roughly 50%"
- NOT OK to change to "50%" (implies false precision)
### Ranges
Preserve both bounds:
- "$50-75K" must keep both $50K and $75K
- "3-5 years" must keep both bounds
### Lists of Items
All items must survive:
- "Teams in SF, NYC, and London" → all three cities required
- Can reorder, but can't drop any
### Attributed Quotes
Quote text and attribution both required:
- `"We're excited" - Jane Doe, CEO`
- Both the quote and "Jane Doe, CEO" must survive
---
## Quick Reference
| Type | Example | Modifiable? |
|------|---------|-------------|
| Currency | $47.3M | NO - exact |
| Percentage | 23% | NO - exact |
| Date | Q3 2024 | NO - exact |
| Person | John Smith | NO - exact |
| Company | Acme Corp | NO - exact |
| URL | https://... | NO - exact |
| Code | `function()` | NO - exact |
| Quote | "..." | NO - exact |
| Causal | A caused B | YES - preserve meaning |
| Comparison | 3x faster | YES - preserve numbers |
@@ -0,0 +1,41 @@
# Harvesting Voice Samples — internals
Harvest bootstraps teach samples from local chat transcripts and folders the user
declares as their own writing. The routed flow (harvest → classify → approve →
copy) is step 1 of `references/commands/teach.md`; this file holds the adapter
internals, the contamination tripwire, and the privacy rules that flow links to.
The transcript adapters only accept entries with explicit user authorship. Assistant
turns are dropped before candidate filtering, and unknown JSONL schemas are skipped with
a warning rather than guessed. This authorship separation is the product requirement:
assistant-authored transcript text must never enter candidates.
The `codex-jsonl` adapter reads Codex CLI/Desktop sessions, stored under
`~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl` and auto-detected by shape alongside
`claude-jsonl`. It harvests `event_msg` user_message text and user-role
`response_item` message content, and drops assistant/developer-role turns, tool
and reasoning plumbing, and `session_meta.base_instructions` outright. Codex also
injects non-user content into user-role turns: a repo's AGENTS.md dumped verbatim,
an `<environment_context>` banner, and similar wrapper tags. The adapter filters
those by structural marker, never by guessing at content, and counts them under
the `instruction-injection` drop reason instead of silently discarding them.
The contamination tripwire runs each kept candidate through `banned_phrase_scan` and
`structure_scan`. A hard violation, or two or more distinct scanner categories, marks
the sample as `suspect_ai: true`. Suspect samples are kept for human review, ranked last,
and must never be auto-approved.
A clean-styled, unmarked assistant paste inside a user turn is undetectable by parsing
and only weakly covered by the tripwire; mandatory human approval is the real defense.
The folder adapter treats `.md` and `.txt` files in a designated folder as user-authored
by declaration, but still runs the same tripwire. Dictated text with heavy `um`/`uh`
fillers is marked `dictated: true` and kept because spoken phrasing can still be useful
voice evidence.
Privacy: harvesting is local. Candidate files, classifier task files, and approved
`.unslop/voice/<name>/` samples should stay out of git unless the user explicitly chooses
otherwise.
Tiering: parsing and filtering are Tier 0 deterministic work. Situation/register
classification can be fanned out to the cheapest capable model through the emitted task
files. Final approval stays human.
@@ -0,0 +1,91 @@
# Maintenance Procedures
Every change here is eval-first: the example lands in
`evals/fixtures/contracts/scanner-examples.json` and fails before any scanner or
catalog edit makes it pass. Run `python3 evals/check.py --full` for the full
repository check.
## Add a banned phrase (`--add-phrase`)
1. Add a false-negative example (the phrase flags) and a false-positive
example (a literal or domain use stays clean) to the scanner contract table.
Run `python3 evals/check_scanner_contract.py` and confirm it fails.
2. Add the phrase to `scripts/banned_phrase_scan.py` `BANNED_PHRASES` with
category, severity, and suggestion. If the word has a literal sense, gate it
behind collocations in `STRUCTURAL_PATTERNS` instead, and add a REC recall
example proving the jargon use still flags.
3. Document it in `references/taboo-phrases.md` (parity is enforced by
`python3 evals/check_taboo_parity.py`).
4. Run `python3 evals/check.py --full`; expect green with no new xfail.
## Add a structural pattern (`--add-structure`)
Same procedure, but the entry goes in `STRUCTURAL_PATTERNS` with a regex, and
the false-positive example must cover the nearest legitimate construction the regex
could clip.
## Prefer a pair
When a pattern has literal senses or only becomes visible in context, add a
minimal pair under `evals/fixtures/pairs/` instead of relying only on one-line
stdin rows. The `_with` twin should contain one controlled tell, the `_without`
twin should preserve the facts while scanning clean, and the manifest should
name the target category or structure metric. Run `python3 evals/check_pairs.py`
before changing the scanner.
## Coverage gate and category protection (enforced)
`python3 evals/check_pattern_coverage.py` makes the two examples above mandatory, not
conventional:
- **Coverage.** Every `STRUCTURAL_PATTERNS` regex must match at least one contract
example's corpus, and every `BANNED_PHRASES`
key must appear in at least one corpus text. A new scanner entry with no eval
example fails this gate (DOC-09) while every other gate is green. There is no
grandfather list: an uncovered pattern is a hard failure. If a batch of legacy
keys is uncovered, add `REC`-style coverage packs (8-12 phrases per stdin, one
natural sentence each) rather than exempting them.
- **Protection.** Every violation category the scanner can emit must be claimed by
a `scanner_false_positive` example carrying `"protects": "<category>"`. Backfill the
field on that category's FP example; add a new FP example
(asserting `total_violations == 0` on tempting-but-clean prose) for any category
with no protector. The runner ignores the `protects` key, so it is safe on any
FP example.
## Rehearse the procedure (the kata)
`python3 evals/kata_add_pattern.py --run` is a meta-eval (DOC-10): in a temp copy
of the repo it adds a throwaway pattern and asserts each safety net fires in order
— coverage catches the example-less entry, parity catches the missing catalog line,
the contract goes green once both exist, and the example goes red if the scanner entry
is later removed. Run it before touching the maintenance tooling; if a refactor
breaks a safety net, this kata turns red.
## List current patterns
```bash
python3 - <<'PY'
from scripts.banned_phrase_scan import BANNED_PHRASES
for k in sorted(BANNED_PHRASES): print(k)
PY
rg -n '"pattern":' scripts/banned_phrase_scan.py # structural patterns
```
## Wiki sync (`--wiki-sync`)
Syncs pattern rules with Wikipedia's
[Signs of AI writing](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing) page.
1. Check for updates: `python3 scripts/wiki_sync.py check` (exit 0 = no updates).
2. Get the structured diff: `python3 scripts/wiki_sync.py diff` (JSON with
change type, section, words).
3. For each new word or phrase, follow "Add a banned phrase" above — eval rows
first, then scanner, then catalog.
4. Verify: `python3 scripts/banned_phrase_scan.py < /dev/null` (no syntax
errors), then the full suite.
Only adopt phrases that are genuine AI tells in general prose. Skip
Wikipedia-specific patterns (broken wikitext, DOI formatting, citation graffiti).
For the full agent-runnable refresh procedure, including the staleness
reporter and the parity-bench cadence, see `references/refresh.md`.
+194
View File
@@ -0,0 +1,194 @@
# Teach & Mimic — internals
The routed flows live in `references/commands/teach.md`,
`references/commands/mimic.md`, and the refine section of that mimic command
file. This file holds the deep material those flows link to: card anatomy,
sample requirements, scoring, baselines, statistics, and failure modes.
Reconstruction runs under detection's constitution: a mimic or refined output
that reintroduces slop is a failure. Every artifact here is deterministic and
testable; the model is used only to generate prose, never to score itself.
## Teach — the artifacts
`teach` distills a writer's samples into a reusable voice, stored under
`.unslop/voice/<name>/` (gitignored by default):
1. `scripts/voice_profile.py samples/ -o profile.json` — the machine
fingerprint (char 3-grams, function-word delta, sentence-length EMD,
punctuation, contractions, MTLD, word-length, impostor z-scores, GI rank).
This is the referee the scorer uses.
2. `scripts/voice_card.py --profile profile.json --samples samples/ --out .
--name <name>` — the **layered voice card** the generating model actually
follows. `--name` labels the card (`# Voice card: <name>`); default `voice`.
voice_card recomputes the profile from `--samples` and refuses (exit 2, named
field) if the supplied `--profile` does not describe those samples, so a stale
profile can never silently drive a card.
3. Add `--provenance` to write `provenance.json` (per-sample sha256, word
counts, doc count, genre note, low-confidence flag) so a teach run is
auditable.
**Sample files.** Both scripts read only `.txt` and `.md` files, recursively,
under the samples directory (nothing else — a directory of `.docx` or extensionless
files reads as empty and both scripts exit 2 with a diagnostic naming the
requirement). Convert samples to `.txt`/`.md` first.
**Sample requirements.** Use at least 5 documents, 2-3k words total, in the
**same genre** as the target. Fewer than ~2000 words or cross-genre samples set
`low_confidence` in the profile and provenance; the card still builds but its
claims are noisier. **Surface `low_confidence` to the user** — tell them the voice
is provisional and ask for more same-genre samples before trusting a mimic.
Cross-domain style attribution degrades sharply, so do not teach on blog posts
and score legal memos.
## The voice card (layered)
The card is a directory, not one file, so the generator loads only what the
current writing task needs:
- `card.md` — the always-loaded core, kept under 300 words: rhythm (median
sentence length, IQR, burstiness), contraction habits with real examples,
punctuation the writer uses and avoids, top sentence openers, a **Never**
list of at/near-zero features, and an **index table** — "when writing X, read
`card/X.md`".
- `card/<situation>.md` — one sheet per **covered** situation from the taxonomy
(explaining-technical, anecdote, argument, disagreement, praise,
hedging-uncertainty, numbers-data, addressing-reader, openings, closings).
Each sheet gives how this author handles that situation, 1-3 verbatim sample
snippets, and measured markers.
**No fabrication.** A dimension with no sample evidence gets **no sheet**; it is
named under "Uncovered" in `card.md`. The card never invents a voice the
samples cannot support.
**Coverage classification.** `voice_card.py --coverage` emits a deterministic
lexical coverage matrix over the taxonomy. It is intentionally coarse: it
DRIVES which sheets get written and which teach prompts to ask the user, and a
misclassified sentence can only add or drop a sheet — it can never change a card
claim, because claims come from measured facts and verbatim snippets. So
misclassification is low-stakes by construction. A sample set missing numeric
writing, for instance, leaves `numbers-data` uncovered and named as a gap.
**Coverage → prompt procedure.** When a dimension the target writing will need is
uncovered, ask the user for a short sample that exercises it. **Only ask for
dimensions the task requires** — do not fish for all ten. Templates:
| Dimension | Prompt to the user |
|-----------|--------------------|
| explaining-technical | "Share something you wrote that explains how a thing works or why it behaves the way it does." |
| anecdote | "Share a few sentences where you tell a small story about something that happened to you." |
| argument | "Share something you wrote that argues a position — a claim you defended." |
| disagreement | "Share something where you pushed back on or disagreed with an idea." |
| praise | "Share something where you praised or recommended something you liked." |
| hedging-uncertainty | "Share something where you were unsure and said so — thinking out loud." |
| numbers-data | "Share something you wrote that works with numbers, quantities, or measurements." |
| addressing-reader | "Share something written directly to a reader — instructions or a note to someone." |
`openings` and `closings` are structural (first/last sentence of every document)
and are covered as soon as there is one document, so they need no prompt.
The card is pack-sized (~1-2k tokens loaded) so it rides in the generation
context cheaply. Same inputs give byte-identical files.
## `--refine` — the internal hill-climb
`evals/run_mimic_refine.py` iterates when one pass is not enough. It is worth
the cost only when a single mimic keeps landing short of the target voice; most
users teach once and mimic cheaply thereafter.
Protocol (climb on A, accept on DEV, report on a sealed split):
- Split samples by document (seeded): **A** (~60%, retrieval/context pool) and
**DEV** (~40%, acceptance). Fewer than 5 docs → refuse (exit 2).
- **LIVE (default):** each iteration assembles B prompts — the draft, the
A-split voice card, the k=2 nearest-A samples by char-3gram cosine, and the
current directives — and invokes `--generate-cmd` (default `claude -p`) once
per beam, piping the prompt on stdin and reading the candidate on stdout.
`--baseline zero|few|retrieval` selects which samples ride in the prompt (none
/ first-k / k nearest-A) and generates its candidate set through this same
path, so the honest retrieval-few-shot competitor and the refine loop share
one pipeline.
- **DRY-RUN:** pass `--candidates-dir DIR` and the loop reads
`DIR/iter<i>/*.md` as that iteration's batch instead of generating them, so
the loop logic runs with no model calls (this is how the MIMIC-* rows exercise
it; the LIVE path is covered by a mock generator).
- **Hard gates** discard a candidate regardless of score: banned-phrase clean,
structure clean, draft→candidate preservation, no copy-gate violation against
A, and a 150-word floor.
- Survivors are scored against DEV (and recorded against A for the divergence
guard) with the **full `voice_score` composite** — `0.5·(1−GI) + 0.5·` clipped
weighted impostor-z distance against a same-genre impostor pool (`--impostors`,
default the committed pool), seeded. This is deliberately NOT a raw weighted
distance: a marker-stuffed candidate can minimize a raw distance and clear
every hard gate yet be exposed by GI, so raw-distance acceptance would accept
it over honest prose. The generator never scores itself.
- Accept an iteration only if the DEV composite improves by ≥ `min-delta`; the
best-so-far never regresses.
### Stopping and the divergence guard
Earliest of: iteration cap, patience (N iterations without a DEV gain), or
**divergence** — the A composite improving while DEV worsens for two
consecutive iterations. Divergence is the reward-hacking signature (the model
is fitting the retrieval pool's topics, not the author's style); the loop halts
immediately and sets `reward_hacking_warning`.
### Directives and card refinement
Each iteration derives up to four ranked directives from the DEV metric deltas,
and each directive carries a proposed **card amendment** line. The best
candidate is copied to `OUT/final.md` and the amended card to
`OUT/voice-card.refined.md`.
## Scoring
The `voice_score` **composite** the refine loop accepts on is
`0.5·(1−GI) + 0.5·zsum` (lower = more author-like), where `GI` is the General
Impostors rank — the fraction of random feature-subset trials in which the
candidate beats every sampled impostor at looking like the profile — and `zsum`
is the clipped, weighted **impostor-z** distance (each per-feature distance
normalized by the impostor pool's mean/std, clipped to ±3, weighted by the WP10a
research weights). Both halves are calibrated against a same-genre **impostor
pool**, so the number rewards *beating plausible other authors*, not merely
matching aggregate surface stats. That is what defeats a marker-stuffed
candidate: it can drive a raw weighted distance down and still be far worse under
GI. Meaning preservation vs the original draft is a **separate** hard gate, never
blended into the composite. The composite is a **guide, not an oracle** — even
strong authorship verification tops out near a 93-94% ceiling, so treat
per-candidate deltas as the signal and under-claim.
## Baselines
Score refine output against paired baselines on the same drafts: zero-shot
("write like this person"), static few-shot, and **retrieval few-shot** (k
nearest-A samples — the honest competitor). Refine has to beat retrieval
few-shot to earn its cost.
## Statistics
`evals/mimic_stats.py` runs the small-n discipline: per-item paired deltas, a
BCa bootstrap CI (n=2000, seeded), and a sign-flip permutation test (exact for
≤12 items, else 20k seeded). Claim a win only when the CI lower bound > 0 **and**
p < 0.05. Anything degenerate (too few items, zero variance) reports
not-improved.
## Cost
| Step | Calls |
|------|-------|
| teach (profile + card) | free / deterministic |
| mimic | one strong generation call |
| refine | iterations × beam strong calls, plus free gate/score passes |
## Failure modes and the gate that catches each
| Failure | Symptom | Caught by |
|---------|---------|-----------|
| Topic bleed | importing what samples are *about*, not *how* they read | divergence guard (A improves, DEV worsens) |
| Verbatim copying | lifting sample phrasing | copy-gate (4-gram / LCS vs A) |
| Register lock | latching one sample's register | DEV composite vs held-out split |
| Reward hacking | self-assessed quality rising while DEV falls | divergence guard + `reward_hacking_warning` |
| Slop reintroduced | AI tells creep back into voiced prose | banned-phrase / structure gates |
For chunking long documents and the tiered execution architecture around these
steps, see `pipeline.md`.
@@ -0,0 +1,75 @@
{
"packs": {
"pack-phrases-core": [
"academic_excess",
"acknowledgment_loop",
"ai_vocabulary",
"assistant_artifact",
"communication_artifact",
"conclusion_scaffold",
"copula_avoidance",
"editorializing",
"emphasis_crutch",
"false_agency",
"false_concession",
"false_range",
"filler",
"filler_opener",
"formulaic_challenges",
"generic_conclusion",
"jargon",
"knowledge_cutoff",
"meta",
"novelty_inflation",
"numbered_list_inflation",
"performative",
"promotional",
"reader_addressing",
"reader_steering",
"reasoning_chain",
"rhetorical_question",
"rhetorical_setup",
"significance_inflation",
"superficial_ing",
"throat_clearing",
"unnecessary_permission",
"vague_attribution",
"vague_endorsement"
],
"pack-structure": [
"macro_both_sidesism",
"macro_redemption_arc",
"macro_preview_recap",
"macro_over_determination",
"macro_emotional_flatness"
],
"pack-register-guards": [
"hedge_stack",
"parenthetical_hedging"
],
"pack-voice": [
"anthropomorphic_shipping",
"anti_slop_register",
"binary_contrast",
"colon_reveal",
"contrastive_definition",
"dramatic_fragment",
"em_dash_overuse",
"exclamation_overuse",
"formatting",
"fragment_template",
"headline_cadence",
"headline_container_agency",
"imperative_slogan",
"negative_listing",
"negative_parallelism",
"numeric_parallelism",
"slogan_fragment",
"spec_fragment",
"staccato_fragmentation",
"tool_agency_reflexive",
"tool_agency_volitional"
],
"pack-facts": []
}
}
@@ -0,0 +1,28 @@
# Pack: Facts
Use this pack for facts and constraints that must survive a rewrite. Do not judge style or phrase quality.
## Look For
- Constraints from `extract_constraints.py`: numbers, dates, times, names, URLs, emails, quoted text, code identifiers, currencies, percentages, units, ranges.
- Negations, conditionals, and scope words surfaced by preservation validation: "not", "unless", "except", "only", "must", "may", "can", "at least", "no more than".
- Approximation traps: "roughly 60%" is not "60%"; "Q2" is not "June"; "under 500 ms" is not "500 ms".
- Proper-noun and party relationships: Apple sued Qualcomm is not Qualcomm sued Apple.
- Technical literals: API names, flags, file paths, version numbers, protocol names.
## Emit
Return JSON findings only:
```json
{"span":"roughly 60%","rule":"approximate_quantity","pack":"pack-facts","severity":"hard","note":"Keep the approximation marker with the number."}
```
Use `hard` for exact facts, negations, scope, and quantities. Use `soft` for phrasing that appears factual but may be compressible.
## Examples
- "p99 latency fell to 180 ms on June 18" -> report p99, 180 ms, June 18.
- "roughly 60%" -> report the approximation and number together.
- "The team improved performance." -> no finding; style claim without a concrete constraint.
- "`--strict` failed on v2.1.0" -> report flag and version.
@@ -0,0 +1,30 @@
# Pack: Phrases Core
Use this pack only for scanner-backed phrase, jargon, and scaffold tells. Do not judge facts, register-preservation, macro structure, or the rewriter's house style here.
## Look For
- Throat-clearing and reader steering: "here's the thing", "let's dive in", "to answer your question".
- Emphasis crutches and inflated significance: "let that sink in", "underscores the importance", "a testament to".
- Business, academic, and promotional vocabulary used as generic polish: "leverage", "seamless", "vibrant", "rich cultural heritage".
- Vague attribution or false agency: unnamed "experts", numbers that "speak for themselves", data that "tells a story".
- Formulaic scaffolds: ordinal sequencing, generic conclusions, rhetorical setup questions, novelty claims, numbered-list hype.
Use `references/taboo-phrases.md` as the authoritative phrase catalog. Report only spans this pack owns.
## Emit
Return JSON findings only:
```json
{"span":"...","rule":"throat_clearing","pack":"pack-phrases-core","severity":"hard","note":"Cut the opener and start with the claim."}
```
Use `hard` for always-bad boilerplate and `soft` for context-dependent wording. Leave legitimate literal, legal, medical, code, or domain-specific uses unreported.
## Examples
- "Here's the thing: the data speaks for itself." -> report "Here's the thing" and "speaks for itself".
- "The loan used 3:1 leverage." -> no finding; finance sense is literal.
- "This update unlocks seamless cross-functional synergy." -> report "seamless" and "synergy".
- "Some critics argue the policy failed." -> report vague attribution unless the critics are named nearby.
@@ -0,0 +1,30 @@
# Pack: Register Guards
Use this pack to find wording that must survive a rewrite. Do not flag ordinary slop phrases and do not rewrite. This pack prevents harm.
## Look For
- Legal scope and negation: "arguably", "may", "shall", "must", "unless", "except", "not", "does not", section references, liability limits, indemnity terms.
- Medical and scientific uncertainty: "may cause", "can increase", "studies suggest", "preliminary", "observational", "does not establish causation", confidence limits, cohort limits.
- Security and safety absolutes: "never", "always", "must", "all input", "no secrets", "do not mix", when they define a rule or warning.
- Scope words and quantities: "most", "some", "only", "at least", "no more than", percentages, ranges, dates, named parties.
- Anti-dehedging traps: a phrase that looks cautious but carries the claim's certainty, exception, or legal burden.
Run on the whole text, not paragraph chunks, because scope can be established earlier than the sentence being rewritten.
## Emit
Return JSON findings only:
```json
{"span":"arguably does not rise to gross negligence","rule":"legal_negation_scope","pack":"pack-register-guards","severity":"hard","note":"Hedge and negation are load-bearing; preserve them exactly."}
```
Every finding is a preservation constraint. Use `hard` when changing it could invert or strengthen the claim; `soft` when it is likely load-bearing but needs human review.
## Examples
- "The incident arguably does not rise to gross negligence under Section 12(b)." -> report the full legal hedge/negation span.
- "Users should never store secrets in client-side code." -> report "never store secrets".
- "It is worth noting that the button is blue." -> no finding; ordinary filler belongs to another pack.
- "The trial may cause drowsiness in some patients." -> report "may" and "some patients".
@@ -0,0 +1,31 @@
# Pack: Structure
Use this pack for document-level AI structure. Do not report isolated phrases unless they create a macro pattern.
## Look For
- `structure_scan.py` flags: `sentence_burstiness`, `conclusion_coda`, `bold_colon_listicle`, `one_line_staccato`, `connective_paragraph_openers`, `signpost_density`, `opener_repetition`, `participial_closer_share`.
- Concrete thresholds to self-check: average sentence length under 8 or over 34 words is suspicious; sentence-length variance under 18 on 5+ sentences suggests uniform rhythm; 3+ one-line paragraphs under 8 words suggests staccato; 3+ consecutive paragraphs opened by connectives/signposts suggests scaffolding; 4+ repeated paragraph openers is blocking; 35%+ participial sentence endings suggests templated cadence.
- Judge-only macro tells from `taboo-phrases.md`: both-sidesism, templated redemption arc, preview/recap symmetry, over-determination, uniform emotional register.
- `silhouette_scan.py` flags idea arrangement above the surface: `scaffold_opener_share`, `callback_content` (recap loop), `role_entropy_bits`, `preview_fulfillment`, `heading_preview`, composited into `silhouette_penalty` (flags at `>= 1.0`).
- A high `silhouette_penalty` means the document follows a template outline (preview-then-fulfill, rotating scaffold openers, closing recap); cite the specific flagged metric, not just the composite.
- Cue-deletion evades silhouette but not surface repetition: pair `silhouette_scan` with `structure_scan` (`opener_repetition`, `one_line_staccato`) so a stripped-connective attack still fails one of the two fences.
Genre matters. `--genre docs` can allow reference-doc structure; `--genre social` can allow deliberate social cadence. Output still fails if the genre excuse is false.
## Emit
Return JSON findings only:
```json
{"span":"paragraphs 2-5","rule":"connective_paragraph_openers","pack":"pack-structure","severity":"hard","note":"Paragraphs advance by scaffold words instead of content."}
```
Quote the smallest span that shows the macro pattern. For whole-document patterns, use paragraph ranges.
## Examples
- "However... In addition... Consequently... In the end..." -> report connective scaffold and coda.
- A technical comparison with "In Rails / In Django / In raw SQL" once -> no finding.
- A story that always moves problem -> lesson -> uplift -> moral -> report `macro_redemption_arc`.
- A report whose conclusion only recaps the intro -> report `macro_preview_recap`.
@@ -0,0 +1,33 @@
# Pack: Voice
Use this pack for manufactured voice and the skill's own replacement tells. Do not report core phrase boilerplate or factual/register constraints.
## Look For
- Anti-slop register: bare fragment contrasts such as "Not the tool. The team."; forced punch endings; repeated two- or three-word sentences.
- Elegant variation and synonym cycling that calls attention to itself instead of clarifying.
- Dramatic fragments, binary contrasts, negative parallelism, colon reveals, and rhetorical setups.
- Contrastive-definition tails ("X is a build step, not an output."), two-beat imperative slogans ("Emit tokens. Ship bytes."), repeated "<plural noun> that <verb>." fragments, "One X, N Y." numeric parallelism, and abstractions that "ship inside" things.
- Standalone slogan/spec fragment lines and headers: "N X, one Y." slogan cadence ("Four presets, one input.") and "N noun-phrase, past-participle ..." spec-sheet fragments ("Eight criteria, scored 1 to 5."). Flagged only when the whole line is the fragment; the same shape embedded in a sentence is a literal count and stays clean. Soft.
- Headline slogan cadence: the "Short statement. Short statement." two-beat rhythm repeated across a document's headlines ("One command. A real URL." / "Reviewers click. The agent fixes."). One is voice; three or more is template grammar. Frequency-gated, soft.
- Headline container agency: a calendar or program container delivers an artifact or outcome ("Week 1 ends with a calibrated eval"; "Demo day closes the cohort"). Name the completion criterion, mechanism, result, or decision instead. Literal boundaries such as "Week 1 ends on Friday" stay clean. Soft.
- Tool anthropomorphism: an inanimate tool-noun given human agency. Reflexive self-agency fires anywhere ("The suite defends itself." / "The rules update themselves." / "It graded its own reflection."); strongly-volitional verbs (decides, hunts, wants, knows, believes, cares, refuses, judges, thinks) fire only on a standalone headline line ("The bench decides which model does which job." / "It hunts instances, not word lists."). Ordinary technical register stays clean ("the parser reads the file", "the model learns the distribution", "the test cleans up after itself"). Soft.
- Punctuation performance: em-dash clusters, exclamation overuse, decorative bolding, title case inside body prose.
- Warmth stripped into telegraphese: an email or narrative that loses natural softeners and sounds colder than the source.
## Emit
Return JSON findings only:
```json
{"span":"Not the tooling. The conversations.","rule":"anti_slop_register","pack":"pack-voice","severity":"hard","note":"Bare fragment contrast is the skill's own tell; rewrite as varied prose."}
```
Mark `hard` when the cadence is itself a recognizable formula. Mark `soft` when it may be genre-appropriate but needs review.
## Examples
- "Not the strategy. The execution." -> report `anti_slop_register`.
- "We tested the API, fixed the retry path, and shipped." -> no finding.
- "The result: alignment. The cost: trust." -> report colon reveal / fragment cadence.
- "Thanks for sending this over. Could you send the figures by Friday?" -> no finding; natural warmth.
@@ -0,0 +1,110 @@
# Personality & Voice Guide
Removing AI patterns is necessary but not sufficient. Text that's clean of AI tells can still read as sterile, voiceless, and institutional. This guide covers the positive dimension: what makes writing feel like a person wrote it.
> **Two guardrails before you use any of this.**
> 1. **Don't invent a person.** The first-person voice, opinions, and anecdotes below belong to writing that *has* an author (a personal post, a founder's note, an opinion piece). For impersonal copy — technical docs, reference text, third-party announcements — do **not** fabricate an "I"/"we" or a lived experience. An invented anecdote is a louder tell than the slop you removed.
> 2. **Don't mistake de-hedging for voice.** "Show uncertainty honestly" means drop *habitual* hedging, not load-bearing scope or certainty. In technical, legal, medical, or safety writing, "may", "preliminary", "does not establish causation", and "most (73%)" are the content — keep them.
---
## Signs of Soulless Writing
Text that passes all taboo-phrase checks but still feels off:
| Signal | What It Looks Like |
|--------|--------------------|
| No stance | States facts without interpretation or opinion |
| Uniform rhythm | Every sentence 12-16 words, same structure |
| Zero first person | Never says "I" or "we" — hides behind passive |
| No specificity | "Many companies" instead of naming one |
| Overly balanced | Presents all sides equally, never commits |
| No surprise | Every paragraph follows the expected arc |
| Sanitized vocabulary | Only uses "safe" words — nothing vivid or unusual |
| Perfect grammar | No fragments, contractions, or colloquialisms |
---
## How to Add Voice
### 1. Have Opinions
Voiceless: "There are advantages and disadvantages to remote work."
Human: "Remote work is better for deep work. Open offices are where focus goes to die."
State what you believe. Qualify only when accuracy demands it, not out of habit.
### 2. Vary Rhythm Deliberately
Voiceless: "The team reviewed the metrics. The team discussed the findings. The team decided on next steps."
Human: "The team reviewed metrics. Three hours of argument later, they had a plan. Not a great one, but a plan."
Mix short punches with longer flowing sentences. Break patterns. A fragment works. So does a 30-word sentence if it earns the length.
### 3. Be Specific
Voiceless: "The company experienced significant growth in the past year."
Human: "Revenue hit $4.2M, up from $1.8M twelve months ago."
Names, numbers, dates. Replace every "significant" with the actual data.
### 4. Use First Person When Natural
Voiceless: "It is recommended to ship early and iterate."
Human: "I ship early. Half the time the first version is wrong, but I learn faster than if I'd planned for six months."
"I" and "we" are not unprofessional. They signal a real person is speaking.
### 5. Leave Some Mess
Voiceless: "The approach combines agile methodology with user-centered design principles to deliver optimal outcomes."
Human: "We mashed agile and UCD together. It's messy but it works. Users get what they need faster."
Real writing has parenthetical asides, self-corrections, and imperfect phrasing. Overpolished prose reads as synthetic.
### 6. Show Uncertainty Honestly
Voiceless: "Further research is needed to determine the optimal approach."
Human: "I don't know the right answer yet. Two options look promising, and we'll know more after the pilot."
Admitting you don't know something is more human than hedging behind passive constructions. But this is for opinion and personal writing. In regulated or scientific content, "further research is needed" may be the precise, required claim — don't trade real epistemic caution for casual guesswork.
### 7. Use Concrete Sensory Language
Voiceless: "The office environment was suboptimal for productivity."
Human: "Fluorescent lights, a printer that jammed every third page, and a thermostat war that never ended."
Let the reader see, hear, feel the situation instead of categorizing it abstractly.
---
## Before/After: Full Passage
### Before (Clean but Soulless)
> Remote work has both benefits and challenges. Many organizations have adopted hybrid models. Research suggests that employee satisfaction increases with flexibility. Companies should consider their specific needs when designing work policies. The key is finding the right balance.
### After (Has Voice)
> I've worked remote for three years. Some weeks I get more done by Thursday than I used to in a full office week. Other weeks I don't put on pants until noon and accomplish nothing. The trick isn't policy, it's trust. Let people figure out where they work best and hold them to outcomes, not hours.
### What Changed
- Added first person and lived experience
- Replaced "many organizations" with a specific perspective
- Dropped the balanced-to-a-fault framing
- "Finding the right balance" (corporate nothing) became a concrete recommendation
- Included honest admission of unproductive days (human texture)
---
## Scoring Integration
When evaluating Human Authenticity (Rubric criterion 5), voice counts:
| Score | Voice Indicators |
|-------|-----------------|
| 5 | Clear authorial voice. Opinions stated. Rhythm varies. Could identify the writer. |
| 4 | Personality visible. Mostly specific. Minor generic patches. |
| 3 | Functional but anonymous. Could be anyone. No glaring AI tells. |
| 2 | Corporate-neutral. All hedged. No specificity. Reads as template. |
| 1 | Actively soulless. Reads like a policy document generated by committee. |
A score of 3 means the text is clean but not human. Aim for 4+.
@@ -0,0 +1,357 @@
# Tiered Execution Pipeline
## Core release protocol: Luna versus Luna
The product comparison uses `gpt-5.6-luna` for both arms on the same unfamiliar
source. The treatment arm receives the frozen UNSLOP contract, scanner candidates,
span-bounded rewrite flow, and validation gates. The raw arm receives neutral
editorial guidance and no UNSLOP rules, repository access, scanner output, or
validation feedback. A blinded `gpt-5.6-sol` judge sees randomized candidate
labels. Report per-case calls, uncached input tokens, output tokens, elapsed time,
detection precision and recall, repair success, preservation, damage, clean no-op,
net improvement, and paired wins.
Run a fresh public split only after its corpus, annotations, thresholds, model
controls, runner, scorer, acceptance code, and per-case budgets are frozen. Open
the sealed holdback only after public acceptance passes. The current beta has no
fresh valid paired result: the newest independently annotated corpus failed its
preregistered composition floor before either arm ran. Earlier paired runs are
directional evidence only.
The cross-provider tables below are ancillary engineering history. They do not
answer the core release question and do not substitute for Luna-vs-Luna evidence.
The skill still works with one agent, but orchestrators should use the cheapest executor that can do each job.
## Tier 0: Deterministic Gates
Run first on the source text:
```bash
python3 scripts/banned_phrase_scan.py <<< "$INPUT"
python3 scripts/structure_scan.py <<< "$INPUT"
python3 scripts/extract_constraints.py <<< "$INPUT"
python3 scripts/readability_metrics.py <<< "$INPUT"
```
Run again on output:
```bash
python3 scripts/banned_phrase_scan.py <<< "$OUTPUT"
python3 scripts/structure_scan.py <<< "$OUTPUT"
python3 scripts/validate_preservation.py original.txt transformed.txt
python3 scripts/readability_metrics.py <<< "$OUTPUT"
python3 scripts/diff_check.py original.txt transformed.txt
```
Output gates are blocking. Use `structure_scan.py --genre docs` or `--genre social` only when the final text truly belongs to that genre.
## Tier 1: Small Detector Agents
Run one small detector per `(pack x chunk)`. Use paragraph chunks up to about 500 words. Run `pack-register-guards` on the whole text because scope, negation, and legal/security force need context.
Detector agents only read their assigned pack from `references/packs/` and return findings:
```json
{"span":"...","rule":"...","pack":"...","severity":"hard|soft","note":"..."}
```
They do not rewrite, score, or import rules from other packs.
## Tier 2: Rewriter
Send one strong-enough rewriter the original text, merged Tier 0/Tier 1 findings, extracted constraints, and the selected preset. It rewrites from findings instead of re-running detection in prose. It must preserve register-guard and fact findings, remove phrase/structure/voice findings, and avoid introducing anti-slop-register tells.
## Model Tiers
| Job | Default executor | Escalate when |
|---|---|---|
| Tier 0 scripts | local deterministic | never; fix the script or input |
| Detection packs | cheapest tier (see Model Parity) | JSON is malformed or pack scope is violated |
| Span replacement / short rewrite | cheapest tier; gates carry safety (parity 2026-07-06: 8/8) | output fails a blocking gate twice |
| Full rewrite of register-sensitive text (legal, medical, security, load-bearing hedges) | strongest practical model + mandatory Tier-0 re-scan | start here; cheap tiers erode register |
| Macro structure (restructuring, coda/preview removal) | machine-gated AND machine-corrected via the structure climb (generate→scan→directive→regenerate); both frontier tiers converge fast, cheapest-that-converges is model-dependent (Anthropic haiku-4-5 at a slightly larger round cap; OpenAI's cheap tier not yet shown to converge) | never trust a model's own macro self-check — feed the scanners' directives back and re-scan (see Macro structure under the climb) |
| Judge/eval | model specified by `evals/BEHAVIORAL-EVALS.md` | benchmark protocol changes |
The model-dependent rows above are not set by taste. They are set by
`evals/run_model_parity.py` (see Model Parity), whose live matrix was recorded 2026-07-06:
span replacement clears on the cheapest tier because the output gates carry safety, while
full rewrites of register-sensitive text and any macro restructuring escalate to the
strongest practical model with a Tier-0 re-scan. Re-run the harness when the model features
change and update these rows from its output.
If Tier 2 output fails the same blocking gate twice, escalate one model tier. Do not add more rules to the prompt; the failure is execution quality.
## Model Parity
The pipeline depends on a model in exactly two places: Tier-1 pack **detection** and
Tier-2 **replacement** generation. Everything else is deterministic. Whether a cheap model
may own those surfaces is a measured question, not an assumption.
`evals/run_model_parity.py` measures it against a fixed seeded corpus:
- **Task A (detection).** Each model reads one pack file plus a short seeded chunk (the
Tier-1 contract above) and returns JSON findings. Grading is deterministic: recall of the
seeded findings and a count of false findings, scored against a frozen manifest. Six
fixtures span the phrases-core, voice, and register-guard families (two are register-guard
cases, where the load-bearing hedge must be caught).
- **Task B (replacement).** Each model is handed one seeded finding (span + rationale) and
asked for a span-minimal replacement. Grading applies the co-writer contract: the
replacement removes the flagged tell and adds no new one (both scanners), preserves every
fact/number/negation, and stays span-minimal.
The matrix is config-driven — entries are `{name, kind, model_id}` where `kind` is
`claude-cli` (`claude -p --model <id>`, the Anthropic spectrum) or `openrouter` (the GPT
spectrum and others, key read from the macOS keychain service `OPENROUTER_API_KEY`). The
harness emits a per-model / per-task score table (JSON and a markdown summary) plus a
`--dry-run` mode that grades canned response fixtures with no network for the `PARITY-*`
eval rows.
**Rule (binding).** Any change to the co-writer, mimic, or detector-pack model features
must run this harness across both the GPT and Anthropic spectrums before merge, and the
detection/replacement rows of the tiering table above must be updated from its output. If
cheap models match the strong reference, use cheap everywhere; if they do not, keep the
strong model for the passes where the gap appears and record which. The `PARITY-*` rows in
`evals/adversarial-evals.json` gate the grader itself (dry-run, no network); they do not
substitute for the live run.
### Recorded results
Live matrix recorded **2026-07-06**. `evals/run_model_parity.py` re-measures on demand;
rerun it after any change to the co-writer, mimic, or detector-pack model features and
update these tables from its output.
**Task B — span replacement.** One seeded finding (span + rationale) per trial, graded on
the co-writer contract: the replacement removes the flagged tell, adds no new one (both
scanners), preserves every fact/number/negation, and stays span-minimal. Eight seeded
findings per model.
| Model | Kind | Task B pass rate |
|---|---|---:|
| claude-haiku | claude-cli | 8/8 |
| claude-sonnet | claude-cli | 8/8 |
| gpt-5.4-mini | openrouter | 8/8 |
| gpt-5.5 | openrouter | 8/8 |
| claude-opus | claude-cli | 7/8 |
The single miss was opus dropping the fact "8:30" from a replacement — caught by
`validate_preservation`, not by model choice. On the mechanical span contract the cheapest
tier ties the flagships; the gates carry safety, the tier does not.
**Behavioral parity matrix — full rewrites.** Eight register/structure cases per model,
graded deterministically across both the Anthropic and GPT spectrums.
| Model | Kind | Full-rewrite pass rate |
|---|---|---:|
| claude-opus | claude-cli | 7/8 |
| gpt-5.4-mini | openrouter | 7/8 |
| claude-sonnet | claude-cli | 6/8 |
| gpt-5.5 | openrouter | 6/8 |
| claude-haiku | claude-cli | 5/8 |
| gpt-5.4-nano | openrouter | 5/8 |
`MACRO-01` (`structure_clean`) failed for all six models, opus included: each kept a
conclusion coda the prose instruction told it to drop. No model self-checks macro structure
from prose. The cheap-tier misses were register and fact erosion in full rewrites — softened
`never`/`all`, a dropped `roughly`, a dropped legal `arguably`/negation, an emitted
`X, not Y` contrastive tail. The lab ladders are symmetric (7/6/5 on both spectrums).
**Measured conclusions.**
- **Span replacement = cheapest tier, with gates on.** The output gates enforce fact and
tell safety, so the smallest model is safe for span-minimal replacement.
- **Full rewrites of register-sensitive text = frontier.** Legal, medical, security, or any
text with load-bearing hedges/negation goes to the strongest practical model with a
mandatory Tier-0 re-scan; cheap models erode register in unsupervised full rewrites.
- **Macro structure = always machine-gated, never self-checked — but machine-correctable.**
A Tier-0 scan surfaces it as an explicit directive, because no model, flagship included,
reliably catches it from prose instructions. Fed those directives back in a scan-regenerate
loop (`evals/run_structure_climb.py`), both frontier tiers converge to clean fast; whether
the cheap tier converges is model-dependent, not a uniform "cheap tier" property -- see
"Macro structure under the climb" for the recorded before/after and the follow-up
experiments.
### Open-weights spot check — 2026-07-07
Live matrix run against the current open-weights flagships of three Chinese labs, each the
newest general chat release of its family on OpenRouter: DeepSeek V4 Pro
(`deepseek/deepseek-v4-pro`), Kimi K2.6 (`moonshotai/kimi-k2.6`), and GLM 5.2
(`z-ai/glm-5.2`). Same harness and contracts as above; six detection fixtures (Task A) and
six replacement fixtures (Task B) per model. Models file:
`evals/fixtures/parity/models_openweights.json`.
| Model | Kind | Task A mean recall | Task A false findings | Task B pass rate |
|---|---|---:|---:|---:|
| glm-5.2 | openrouter | 1.00 | 0 | 6/6 |
| deepseek-v4-pro | openrouter | 0.92 | 1 | 6/6 |
| kimi-k2.6 | openrouter | 0.92 | 0 | 6/6 |
All three clear the span-replacement contract at 6/6, and the only detection misses were
one seeded voice span apiece (fixture A4) for DeepSeek and Kimi; on both model-dependent
surfaces the open-weights flagships sit level with the recorded GPT and Anthropic ladders.
## Macro structure under the climb — 2026-07-07
The recorded parity matrix above closed on a flat verdict: single-pass macro-structure
self-checking failed on every model. Told in prose to drop the coda / connective scaffold /
outline echo, each one shipped it anyway. But the deterministic scanners SEE those failures
precisely, so the failure is correctable even when it is not self-checkable. `evals/run_structure_climb.py`
closes that loop:
generate -> scan (structure_scan + silhouette_scan) -> if clean, done
-> else turn each finding into a TARGETED directive that names WHERE and
WHAT -> feed the directives + current draft back -> regenerate, capped at
N rounds (default 4).
The core is the **directive builder** (`build_directives`): a deterministic map from scanner
findings to instructions that locate the tell, because the model cannot see it. It re-reads
the draft with the scanners' own helpers, so a `conclusion_coda` flag becomes *"the final
paragraph (`Ultimately, ...`) restates the opening; end on a new fact"*, a
`connective_paragraph_openers` flag names the offending paragraph numbers and their opener
words, and a silhouette `callback_content` flag names the exact opening vocabulary the ending
loops back to. Every macro flag both scanners can emit maps to a directive (gated by
`CLIMB-06`). A preservation guard (`validate_preservation`, anchored to `--source-file` when
given) validates every round, so climbing toward clean structure can never eat a fact; a
regression aborts the climb. Honest terminal states (`converged` / `capped` /
`preservation_violation`) carry distinct exit codes. The mock path (`CLIMB-*` rows,
`evals/fixtures/climb/`) exercises all of this offline; the live path drives real models via
`--generate-cmd`.
Live run on the same `SKILL-MACRO-01` robotics-essay fixture as Task C, `--max-rounds 4`,
preservation anchored to the source essay:
| Model | Tier | Single pass (round 0) | Climbed | Rounds | Terminal | Facts |
|---|---|---|---|---:|---|---|
| claude-sonnet-4-5 | frontier | dirty: `signpost_density`, `callback_content` (penalty 7.5) | both scanners clean | 3 | converged | preserved every round |
| claude-haiku-4-5 | cheap | dirty: `callback_content` (penalty 2.5) | 1 residual structure flag | 4 | capped | preserved every round |
| gpt-5.5 | frontier | pending | pending | n/a | pending (OpenRouter HTTP 402) | pending |
| glm-5.2 | open-weights | pending | pending | n/a | pending (OpenRouter HTTP 402) | pending |
Read honestly:
- **Single pass ships macro tells on both Claude tiers**, reconfirming the thesis: told to
remove macro tells, the model does not catch its own document shape.
- **The climb recovers macro on the frontier model where the single pass failed.** Sonnet
converged to both-scanners-clean in three rounds (violation trajectory 2 → 1 → 0) with every
fact preserved. This is the doctrine headline: with the loop, macro structure becomes
machine-*corrected*, a step past machine-*gated*.
- **On the cheap tier the recovery is partial, not full.** haiku-4-5 consumed the silhouette
recap loop on the first directive and held silhouette clean from round 1 on, but it
oscillated on surface structure (re-introducing a coda, then a uniform cadence) and did not
converge within four rounds, capping at one residual structure flag. The loop improves the
cheap tier but does not guarantee it at cap 4; a residual macro flag on a cheap model still
routes to a stronger model or another round, not to shipping.
- **The preservation guard held live throughout** (source-anchored): no round on either model
eroded a fact.
- **The OpenRouter spectrum (GPT + open-weights) is pending on account billing** (HTTP 402
Payment Required at run time), not on the harness. Exact reproduction commands:
```bash
# Anthropic spectrum (native claude -p; run above):
python3 evals/run_structure_climb.py --prompt-file <macro-prompt> --source-file <essay> \
--out out/haiku --generate-cmd "claude -p --model claude-haiku-4-5" --max-rounds 4
# OpenRouter spectrum (needs OPENROUTER_API_KEY credit):
python3 evals/run_structure_climb.py --prompt-file <macro-prompt> --source-file <essay> \
--out out/gpt --generate-cmd "python3 evals/model_generate.py --kind openrouter --model openai/gpt-5.5" --max-rounds 4
python3 evals/run_structure_climb.py --prompt-file <macro-prompt> --source-file <essay> \
--out out/glm --generate-cmd "python3 evals/model_generate.py --kind openrouter --model z-ai/glm-5.2" --max-rounds 4
```
### Two follow-up experiments — 2026-07-07
The OpenRouter 402 blocked the GPT spectrum above, and the cap-4 haiku row left one open
question (does the cheap tier ever converge, or does it just stall). Two follow-ups, both run
against the identical `SKILL-MACRO-01` fixture and source-anchored preservation:
**A. A local alternative to OpenRouter for the GPT side.** The `codex` CLI (OpenAI's agentic
coding CLI) is installed locally and can drive one model call per round the same way
`claude -p` does, sidestepping the billing block. `codex exec` is an AGENT CLI, though: its
stdout is an interleaved transcript (tool calls, hook lines, token counts), not a clean
document, so it cannot be piped straight into the climb loop. Its `-o/--output-last-message
FILE` flag writes only the agent's final text turn with no wrapping -- verified by hand against
this fixture before wiring it in. `evals/model_generate.py` gained a `codex` kind
(`call_codex`) that runs `codex exec --sandbox read-only --ephemeral --skip-git-repo-check -o
FILE -m <model> -`, reads that file, and discards the transcript. KNOWN RISK: `codex exec` has
been observed to hang silently in this environment; `call_codex` runs it in its own process
group with a hard wall-clock timeout (180s default) and SIGKILLs the whole group on expiry, so
a hang surfaces as an honest failed round instead of blocking the climb forever. `CLIMB-07`
exercises this offline against a fake `codex` binary (success, nonzero-exit, empty-output, and
hang paths; the hang case is SIGKILLed in about a second against a 1s test timeout).
**B. Does the cheap Anthropic tier ever converge, or does cap 4 just cut it off early.**
`claude-haiku-4-5` re-run with `--max-rounds 8` (same fixture, same generator) to see whether
the one residual flag at cap 4 was a hard ceiling or a round short.
Results, same fixture and settings as the recorded matrix above except where noted:
| Model | Tier | Kind | Single pass (round 0) | Climbed | Rounds (cap) | Terminal | Facts |
|---|---|---|---|---|---:|---|---|
| gpt-5.5 | frontier | codex exec (default model) | dirty: `sentence_burstiness`, `conclusion_coda`, `preview_fulfillment`, `callback_content` (4 flags) | both scanners clean | 4 (4) | converged | preserved every round |
| gpt-5.4-mini | cheap | codex exec | dirty: `sentence_burstiness`, `opener_repetition`, `preview_fulfillment`, `callback_content` (4 flags) | 2 residual flags, oscillating (4 → 1 → 5 → 2) | 4 (4) | capped | preserved every round |
| gpt-5.4-mini | cheap | codex exec, retried | dirty: `sentence_burstiness`, `preview_fulfillment`, `callback_content` (3 flags) | 5 residual flags, oscillating (3 → 3 → 2 → 1 → 6 → 2 → 1 → 5) | 8 (8) | capped | preserved every round |
| claude-haiku-4-5 | cheap | claude-cli, retried | dirty: `callback_content` (1 flag) | both scanners clean | 5 (8) | converged | preserved every round |
Read honestly:
- **The `-o` extraction is clean and codex is a real generator for this loop.** No stray
transcript text ever reached the scanners; every round's draft was bare prose. `gpt-5.5`
through `codex exec` converged in the same 4-round budget sonnet used (3 rounds), matching
the frontier-converges finding on the Anthropic side.
- **Codex's cheap tier does not converge, even given double the round budget.** `gpt-5.4-mini`
was run twice -- once at the recorded cap of 4, once at cap 8 -- and never reached clean
either time. Its violation trajectory does not fall monotonically the way sonnet's or
haiku's did; it swings between 1 and 6 flags round to round. The loop still holds every fact
(preservation passed all 12 rounds across the two runs), but macro structure on this model
is not machine-*correctable* at any round budget tested here, only machine-*gated*.
- **The cheap Anthropic tier was one round short of converging, not permanently capped.**
`claude-haiku-4-5` re-run at `--max-rounds 8` converged in 5 rounds (violation trajectory
1 → 1 → 1 → 1 → 0), the same residual `callback_content` flag from the cap-4 run finally
clearing once the loop got one more pass. Every round preserved every fact.
- **This changes the doctrine.** "Cheap tiers partially recover" (the prior wording) was true
of the one cheap model measured so far, but it is not a property of "cheap" in general --
it is model-dependent. `claude-haiku-4-5` fully recovers with a slightly larger round cap;
`gpt-5.4-mini` does not recover with an even larger one. Read the two model families
separately, not as one "cheap tier" line.
- **Cheapest model that does this well: `claude-haiku-4-5`, with `--max-rounds` raised from 4
to 6** (5 rounds measured to converge here; 6 gives one round of margin). On the GPT side,
the floor that has actually been shown to converge is `gpt-5.5` (frontier) via codex, not
`gpt-5.4-mini` -- the OpenAI cheap tier is not yet a substitute for this task on this
evidence.
- **The OpenRouter spectrum itself (`gpt-5.5` via API, and `glm-5.2`) is still pending on
account billing** (HTTP 402), not on the harness. Codex gave a working local alternative for
measuring OpenAI's frontier and cheap models on this task, but it drives the model through
Codex's own agent scaffolding and system prompt, not the raw chat-completions endpoint --
read the codex-CLI numbers above as "the GPT models, as codex CLI calls them," not as an
exact stand-in for a raw OpenRouter measurement once billing clears. `glm-5.2` has no local
CLI alternative and remains unmeasured.
Exact reproduction commands:
```bash
# Anthropic spectrum, cheap tier, extended round cap:
python3 evals/run_structure_climb.py --prompt-file <macro-prompt> --source-file <essay> \
--out out/haiku8 --generate-cmd "claude -p --model claude-haiku-4-5" --max-rounds 8
# Codex CLI, GPT spectrum (frontier default, then cheapest):
python3 evals/run_structure_climb.py --prompt-file <macro-prompt> --source-file <essay> \
--out out/codex-gpt5.5 --generate-cmd "python3 evals/model_generate.py --kind codex --model gpt-5.5" --max-rounds 4
python3 evals/run_structure_climb.py --prompt-file <macro-prompt> --source-file <essay> \
--out out/codex-mini --generate-cmd "python3 evals/model_generate.py --kind codex --model gpt-5.4-mini" --max-rounds 8
# OpenRouter spectrum (still needs OPENROUTER_API_KEY credit; unchanged from above):
python3 evals/run_structure_climb.py --prompt-file <macro-prompt> --source-file <essay> \
--out out/gpt --generate-cmd "python3 evals/model_generate.py --kind openrouter --model openai/gpt-5.5" --max-rounds 4
python3 evals/run_structure_climb.py --prompt-file <macro-prompt> --source-file <essay> \
--out out/glm --generate-cmd "python3 evals/model_generate.py --kind openrouter --model z-ai/glm-5.2" --max-rounds 4
```
## Cost Note
For a 1000-word document:
| Path | Calls | Approx tokens |
|---|---:|---:|
| Tier 0 | local scripts | free |
| Tier 1 | about 5 small detector calls | about 10k small-model tokens |
| Tier 2 | one rewrite call | source + findings + preset |
| Monolithic | one large call with full skill and all references | often 8k-15k strong-model tokens before output |
The tiered path spends cheap tokens on detection, reserves the stronger model for rewriting, and gives the final Tier 0 gates authority.
@@ -0,0 +1,90 @@
# Adversarial refresh
The Growth section of `docs/PRODUCT.md` promises that detection catalogs
stay current through adversarial refresh. This page is the procedure. It is
what an agent runs to make that promise real. Nothing here runs on a
schedule (see `docs/DECISIONS.md`, "Agent-invoked only"); it runs when an
agent, or the person driving one, decides to check.
## 1. Check staleness
```bash
python3 scripts/refresh_status.py
```
Stdlib, offline, always exits 0. It reports three signals as JSON:
- `wiki_sync`: days since the local Wikipedia sync state was last updated.
- `parity_bench`: days since the model-parity matrix in
`references/pipeline.md` was last recorded.
- `newest_pattern_row`: days since `evals/adversarial-evals.json` last
gained a row.
Each carries `last` (an ISO date or `null`), `days` (or `null`), and a
`stale` flag; the script never fails a build over it, and the flag is a
prompt for judgment rather than a rule to obey mechanically. The thresholds
it applies (wiki 90 days, bench 180 days, rows 60 days) are commented in the
script itself as starting points.
## 2. If `wiki_sync` is stale
```bash
python3 scripts/wiki_sync.py check # exit 0 = no updates, 1 = updates available
python3 scripts/wiki_sync.py diff # structured JSON diff, when check exits 1
```
For each new word or phrase in the diff, follow "Add a banned phrase" in
`references/maintenance.md`: eval rows first, then the scanner, then the
catalog. Skip Wikipedia-specific patterns, such as broken wikitext, DOI
formatting, or citation graffiti. Only genuine AI tells in general prose
belong here.
## 3. If `parity_bench` is stale
Treat staleness as a prompt to double-check the recorded table against
reality. It is not an automatic re-run trigger by itself: the binding rule
in `references/pipeline.md` (Model Parity) already forces a re-run whenever
the co-writer, mimic, or detector-pack model features change, independent of
elapsed time. When a re-run is warranted, run the live matrix across both
spectrums:
```bash
python3 evals/run_model_parity.py --task both --format both
```
Update the tiering table and the "Recorded results" tables in
`references/pipeline.md` from the output, and update the standing
conclusions in `docs/PRODUCT.md`'s Model tiering section if they changed.
The `PARITY-*` rows in `evals/adversarial-evals.json` gate the grader itself
in dry-run mode. They do not substitute for the live run above:
```bash
python3 evals/run_model_parity.py --dry-run \
--responses evals/fixtures/parity/canned_responses.json
```
## 4. Every miss becomes a row
A miss can come from a live bench output, a wiki-sync diff, or something
caught by hand. Whichever it is, route it through
`references/commands/contribute.md`. Contribute turns the exact specimen
into a red-first eval row and a structured PR, so the miss becomes a
permanent regression test instead of a one-off fix.
## 5. Record results
- Model-parity runs: update `references/pipeline.md`'s dated tables (the
"Live matrix recorded **YYYY-MM-DD**" line and the two results tables
under it) with the new date and numbers.
- Behavioral or skill runs: follow the conventions already set in
`evals/TUNE-RESULTS.md`. Name the harness version, split, runner, and
judge. Then report per-case deltas rather than aggregate lift, per
`CLAUDE.md`'s Interpreting Results section.
## Cadence guidance
There is no scheduler. Use `refresh_status.py`'s thresholds as the trigger
for a manual or agent-invoked check instead: wiki sync roughly quarterly,
the parity bench roughly every six months (sooner on any model-feature
change, per the binding rule above), and a look at the eval catalog if two
months pass with no new row.
@@ -0,0 +1,17 @@
# Rewrite examples
These compact examples are executable documentation. They illustrate removal
of stock framing and inflated abstraction; they do not override the core
contract.
**Input:**
> Here's the thing: building products is hard. Not because the technology is complex. Because people are complex. Let that sink in.
**Output (crisp):**
> Building products is hard, and not because the technology is complex. People are complex, and they are the part you cannot refactor.
**Input:**
> In today's fast-paced business environment, it's becoming increasingly important for organizations to leverage their core competencies while navigating the complex landscape of digital transformation.
**Output (crisp):**
> Companies need to use their strengths while going digital.
+171
View File
@@ -0,0 +1,171 @@
# Scoring Rubric
Score transformed text on 8 criteria (1-5 scale each). Minimum passing: 32/40 (80%).
---
## 1. Directness (1-5)
Does the text get to the point without hedging or softening?
| Score | Criteria |
|-------|----------|
| 5 | Opens with main point. No qualifiers, no throat-clearing. Every sentence advances the argument. |
| 4 | Minimal hedging. One or two soft qualifiers. Gets to point within first sentence. |
| 3 | Some unnecessary setup. A few hedging phrases. Main point clear but delayed. |
| 2 | Multiple hedges per paragraph. Excessive qualifiers. Main point buried. |
| 1 | Constant hedging. Throat-clearing openers. Reader must hunt for the point. |
**Red flags:** "It's worth noting," "To be fair," "I think," "Perhaps," "Maybe," "Kind of," "Sort of"
---
## 2. Natural Rhythm (1-5)
Does sentence length vary naturally? Does it read like speech, not template?
| Score | Criteria |
|-------|----------|
| 5 | Sentence lengths vary 8-25 words naturally. Mix of structures. No mechanical patterns. |
| 4 | Good variance. One or two repetitive structures. Flows well overall. |
| 3 | Some variance but noticeable patterns. Occasional monotony. |
| 2 | Most sentences similar length. Obvious templated rhythm. |
| 1 | Robotic uniformity. Every sentence same structure. Machine-generated feel. |
**Test:** Read aloud. Does it sound like a person talking?
---
## 3. Concrete Verbs (1-5)
Are verbs specific and active, or abstract and passive?
| Score | Criteria |
|-------|----------|
| 5 | All verbs concrete and specific. Actions you can visualize. No abstract placeholders. |
| 4 | Mostly concrete verbs. One or two abstract (leverage, utilize). |
| 3 | Mix of concrete and abstract. Some passive voice. |
| 2 | Dominated by abstract verbs. Heavy passive voice. |
| 1 | Almost entirely abstract verbs. "Utilize," "leverage," "navigate" throughout. |
**Red flags:** leverage, utilize, facilitate, implement, optimize, navigate, unpack, foster
**Good verbs:** build, cut, fix, send, write, call, test, ship, break
---
## 4. Reader Trust (1-5)
Does the text trust the reader, or over-explain and patronize?
| Score | Criteria |
|-------|----------|
| 5 | Assumes reader competence. No stating the obvious. Respects intelligence. |
| 4 | Mostly trusting. One instance of over-explanation. |
| 3 | Some hand-holding. Occasional obvious statements. |
| 2 | Frequent over-explanation. Multiple obvious statements. |
| 1 | Constant patronizing. "Think about it," "Let that sink in," "In other words." |
**Red flags:** "Think about it," "Let that sink in," "In other words," "To put it simply," "As you know"
---
## 5. Human Authenticity (1-5)
Does the text sound like a person wrote it, or an AI performing humanity? This criterion evaluates both the absence of AI patterns and the presence of human voice.
| Score | Criteria |
|-------|----------|
| 5 | Could have been written by a specific person. Has voice, opinion, and lived experience. Rhythm varies naturally. Reader can sense the author's personality. |
| 4 | Sounds human. Clear perspective. Minor generic patches. Mostly specific with occasional abstraction. |
| 3 | Clean but anonymous. No glaring AI tells, but no personality either. Could be anyone. Reads as competent but soulless. |
| 2 | Generic tone. Multiple AI patterns. Corporate-neutral. Hedges everything. No first person, no specificity, no stance. |
| 1 | Obvious AI. Performative emphasis, binary contrasts, dramatic fragmentation. Significance inflation, promotional language, chatbot artifacts. |
**Red flags (AI tells):** "Full stop." "Period." "Let me be clear." Stacked short sentences for effect. Rhetorical questions answered immediately. Significance inflation ("stands as a testament"). Promotional language ("nestled", "boasts"). Superficial -ing analyses. Vague attributions ("experts argue"). Copula avoidance ("serves as a"). Communication artifacts ("I hope this helps").
**Red flags (soulless):** No opinions stated. No first person. Every claim hedged. Uniform sentence rhythm. No specific names, numbers, or dates. Balanced-to-a-fault framing. Sanitized vocabulary only. See `references/personality-guide.md` for detailed guidance.
---
## 6. Content Density (1-5)
What's the ratio of substance to filler?
| Score | Criteria |
|-------|----------|
| 5 | Every word necessary. No filler. Dense with information or insight. |
| 4 | Minimal filler. One or two unnecessary phrases. Tight overall. |
| 3 | Some padding. A few filler phrases. Content present but diluted. |
| 2 | Significant padding. Multiple filler phrases per paragraph. |
| 1 | More filler than content. Adverb-heavy. "In today's landscape" type phrases throughout. |
**Filler indicators:** "At its core," "In today's [X]," "When it comes to," "At the end of the day," "Moving forward"
---
## 7. Fact Preservation (1-5)
Are all factual elements from the original intact?
| Score | Criteria |
|-------|----------|
| 5 | All facts preserved exactly. Numbers, names, dates, URLs unchanged. |
| 4 | All facts preserved. Minor rephrasing of non-critical details. |
| 3 | Most facts preserved. One minor fact altered. |
| 2 | Some facts missing or changed. |
| 1 | Significant fact loss or alteration. |
**Must preserve:** Numbers, percentages, dates, proper nouns, URLs, quotes, technical terms, citations
---
## 8. Template Avoidance (1-5)
Is the structure organic, or following AI templates?
| Score | Criteria |
|-------|----------|
| 5 | Unique structure. No recognizable AI patterns. Organic flow. |
| 4 | Mostly organic. One minor template echo. |
| 3 | Some template patterns. Binary contrasts or lists visible. |
| 2 | Multiple template patterns. Predictable structure. |
| 1 | Full AI template. Three-item lists, binary contrasts, dramatic fragments throughout. |
**Template patterns:**
- "Not X. Y." binary contrasts
- "X. That's it. That's the [thing]." fragments
- "Think about it:" setups
- Exactly three items in every list
- Every paragraph ending with punch
---
## Scoring Guide
| Total | Assessment |
|-------|------------|
| 40 | Perfect—indistinguishable from skilled human writer |
| 36-39 | Excellent—occasional minor tells |
| 32-35 | Passing—clearly human, some AI influence |
| 28-31 | Needs work—AI patterns visible |
| 24-27 | Significant issues—AI-forward |
| <24 | Fail—reads as AI-generated |
**Minimum passing score: 32/40 (80%)**
---
## Quick Scoring Checklist
```
□ Directness: Opens with point? No hedging? [ /5]
□ Rhythm: Sentence lengths vary 8-25 words? [ /5]
□ Verbs: Concrete, active, specific? [ /5]
□ Trust: No over-explaining? Respects reader? [ /5]
□ Authenticity: Sounds like a person? [ /5]
□ Density: Substance over filler? [ /5]
□ Facts: All facts preserved exactly? [ /5]
□ Structure: No AI templates? [ /5]
TOTAL: [ /40]
```
File diff suppressed because it is too large Load Diff
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""Shared cheap English-detection helpers for banned_phrase_scan.py,
structure_scan.py, and silhouette_scan.py. All three scanners must decline the
same non-English inputs, so this table and its two functions live in exactly
one place.
Also home to the shared prose-view helpers (tokenizer, markdown stripper,
paragraph splitter) used by structure_scan.py and silhouette_scan.py. Those
two scanners previously carried private copies of these functions that had
drifted from each other (structure blanked blockquote lines, silhouette
didn't; silhouette stripped **bold**, structure didn't). strip_markdown_for_prose
below is the UNION of both original code paths, gated behind flags so each
caller reconciles the drift deliberately rather than silently: structure_scan
passes blank_blockquotes=True, silhouette_scan passes strip_bold=True."""
from __future__ import annotations
import re
# A small set of high-frequency English function words used only for a cheap
# language check. The words are chosen to be distinctively English: Spanish,
# French, German, etc. rarely use them, so their share of tokens is a robust
# signal without a language-model dependency.
ENGLISH_FUNCTION_WORDS = frozenset({
"the", "and", "is", "are", "was", "were", "of", "to", "in", "that", "it",
"for", "with", "on", "this", "but", "not", "you", "have", "be", "as", "at",
"or", "we", "they", "will", "would", "there", "their", "what", "which",
"when", "from", "been", "has", "had", "its", "an", "by", "our", "your",
"if", "than", "then", "them", "these", "those", "about", "into", "over",
"after", "before", "how", "why", "where", "who", "can", "could", "should",
"do", "does", "did", "so", "out", "just", "more", "most", "some", "such",
"only", "also", "because", "while", "between", "through", "during", "being",
})
def english_function_share(text: str) -> float:
"""Share of word tokens that are common English function words."""
tokens = re.findall(r"[a-z']+", text.lower())
if not tokens:
return 1.0
hits = sum(1 for t in tokens if t in ENGLISH_FUNCTION_WORDS)
return hits / len(tokens)
def is_probably_english(text: str, threshold: float = 0.10, min_tokens: int = 15) -> bool:
"""Cheap English detector backing a graceful non-English decline.
Conservative on purpose: inputs below ``min_tokens`` are always treated as
English (too little signal to decline), and ``threshold`` is low enough that
even terse or ESL-flavored English clears it. Only prose with almost no
English function words (i.e. another language) is declined.
"""
tokens = re.findall(r"[a-z']+", text.lower())
if len(tokens) < min_tokens:
return True
return english_function_share(text) >= threshold
def words(text: str) -> list[str]:
return re.findall(r"[A-Za-z0-9]+(?:[-'][A-Za-z0-9]+)?", text.lower())
def strip_markdown_for_prose(
text: str, *, blank_blockquotes: bool = False, strip_bold: bool = False
) -> str:
"""Strip markdown structure down to a prose view for cadence/discourse
metrics. UNION of structure_scan's and silhouette_scan's original
strippers -- each original branch is preserved verbatim, gated behind the
flag that reproduces that caller's exact prior behavior.
blank_blockquotes=True reproduces structure_scan's prior strip (blanks
both blockquote and heading lines in one combined check). strip_bold=True
reproduces silhouette_scan's prior strip (also collapses **bold** markers
after list/ordinal stripping).
"""
text = re.sub(r"```[\s\S]*?```", "\n\n", text)
kept = []
for line in text.splitlines():
if blank_blockquotes:
if re.match(r"\s*>", line) or re.match(r"\s{0,3}#{1,6}\s+", line):
kept.append("")
continue
else:
if re.match(r"\s{0,3}#{1,6}\s+", line):
kept.append("") # drop heading text from the prose view
continue
line = re.sub(r"^\s*[-*+]\s+", "", line)
line = re.sub(r"^\s*\d+[.)]\s+", "", line)
if strip_bold:
line = re.sub(r"\*\*([^*]+)\*\*", r"\1", line)
kept.append(line)
return "\n".join(kept)
def paragraphs(
text: str, *, blank_blockquotes: bool = False, strip_bold: bool = False
) -> list[str]:
stripped = strip_markdown_for_prose(
text, blank_blockquotes=blank_blockquotes, strip_bold=strip_bold
)
return [
re.sub(r"\s+", " ", p).strip()
for p in re.split(r"\n\s*\n", stripped)
if p.strip()
]
+784
View File
@@ -0,0 +1,784 @@
#!/usr/bin/env python3
"""
Scan text for AI-isms and banned phrases.
Checks against taboo phrases list and returns violations with line numbers.
Provides suggested replacements where available.
By default, quoted examples and code snippets are ignored so the scanner
doesn't flag illustrative bad writing inside docs or tutorials. Pass
--include-quoted to scan those spans too.
Usage:
python banned_phrase_scan.py < input.txt
python banned_phrase_scan.py input.txt
python banned_phrase_scan.py input.txt --include-quoted
"""
from __future__ import annotations
import argparse
import bisect
import sys
import re
import json
from pathlib import Path
from typing import TypedDict
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
from _lang import ( # noqa: E402
ENGLISH_FUNCTION_WORDS,
english_function_share,
is_probably_english,
)
class Violation(TypedDict):
phrase: str
category: str
severity: str
line_number: int
column: int
context: str
suggestion: str | None
def _mask_non_newlines(text: str) -> str:
"""Replace visible characters with spaces while preserving line/column layout."""
return re.sub(r"[^\n]", " ", text)
def mask_ignored_spans(text: str, include_quoted: bool = False) -> str:
"""Mask examples and code so they don't produce false-positive matches."""
masked = re.sub(r"```[\s\S]*?```", lambda m: _mask_non_newlines(m.group(0)), text)
masked = re.sub(r"`[^`\n]+`", lambda m: _mask_non_newlines(m.group(0)), masked)
if include_quoted:
return masked
# Markdown blockquotes are almost always cited examples rather than prose to edit.
masked = re.sub(r"(?m)^>.*$", lambda m: _mask_non_newlines(m.group(0)), masked)
# Double quotes only, any length, across line breaks. Single quotes are NOT
# masked: they collide with apostrophes/emphasis and would silently hide real
# slop inside ordinary single-quoted prose.
quote_patterns = [
r'"[^"]*"',
r"“[^”]*”",
]
for pattern in quote_patterns:
masked = re.sub(pattern, lambda m: _mask_non_newlines(m.group(0)), masked)
return masked
def _phrase_pattern(phrase: str) -> re.Pattern[str]:
left = r"(?<![a-z0-9_-])" if phrase[0].isalnum() else ""
right = r"(?![a-z0-9_-])" if phrase[-1].isalnum() else ""
return re.compile(left + re.escape(phrase) + right)
# Case-insensitive variant used only at the scan_for_violations call site, so
# _phrase_pattern's case-sensitive default is unchanged for any other caller.
# BANNED_PHRASES keys are lowercase literals; matching case-insensitively
# against the original (unlowered) text avoids str.lower()'s non-length-
# preserving folds (e.g. U+0130 "İ" -> 2 chars), which corrupt offsets.
_phrase_pattern_ci_cache: dict[str, re.Pattern[str]] = {}
def _phrase_pattern_ci(phrase: str) -> re.Pattern[str]:
cached = _phrase_pattern_ci_cache.get(phrase)
if cached is None:
pattern = _phrase_pattern(phrase).pattern
# Curly apostrophes are typography, not a different phrase. Keep the
# original phrase key (and its output spelling), but match both forms
# without normalizing the input and corrupting offsets.
if "'" in phrase:
pattern = pattern.replace("'", "['’]")
cached = re.compile(pattern, re.IGNORECASE)
_phrase_pattern_ci_cache[phrase] = cached
return cached
def _line_starts(text: str) -> list[int]:
"""0-indexed start offset of each line; line N (1-based) starts at index N-1."""
starts = [0]
for m in re.finditer("\n", text):
starts.append(m.end())
return starts
def _line_col_context(
text: str,
line_starts: list[int],
pos: int,
context_cache: dict[int, str] | None = None,
) -> tuple[int, int, str]:
"""Derive (1-based line, 1-based column, stripped line context) for an offset
into ORIGINAL text from a precomputed line_starts index, in O(log n)."""
idx = bisect.bisect_right(line_starts, pos) - 1
line_start = line_starts[idx]
line_end = line_starts[idx + 1] - 1 if idx + 1 < len(line_starts) else len(text)
line_num = idx + 1
column = pos - line_start + 1
context = context_cache.get(idx) if context_cache is not None else None
if context is None:
context = text[line_start:line_end].strip()
context = context[:100] + "..." if len(context) > 100 else context
if context_cache is not None:
context_cache[idx] = context
return line_num, column, context
# Banned phrases with categories, suggestions, and severity.
# severity: "hard" = always an AI tell; "soft" = context-dependent
# Compact high-signal scanner pack. Literal/domain-sensitive cases stay explicit:
# broad vocabulary and low-value rhetorical variants are intentionally retired.
BANNED_PHRASES: dict[str, dict[str, str | None]] = {
# Throat-clearing and conclusion scaffolding.
"here's the thing:": {"category": "throat_clearing", "severity": "hard", "suggestion": None},
"in conclusion": {
"category": "conclusion_scaffold",
"severity": "hard",
"suggestion": "State the conclusion directly.",
},
# Significance inflation and vague attribution.
"underscore the importance": {
"category": "significance_inflation",
"severity": "hard",
"suggestion": "State the concrete effect.",
},
"analysts predict": {
"category": "vague_attribution",
"severity": "hard",
"suggestion": "Name the analysts or cite the forecast.",
},
# High-signal AI vocabulary.
"treasure trove": {
"category": "ai_vocabulary",
"severity": "hard",
"suggestion": "collection, source",
},
# False agency, promotion, and assistant artifacts.
"speak for themselves": {
"category": "false_agency",
"severity": "hard",
"suggestion": "State the numbers and what they show.",
},
"rich cultural heritage": {
"category": "promotional",
"severity": "hard",
"suggestion": None,
},
"as an ai language model": {
"category": "assistant_artifact",
"severity": "hard",
"suggestion": "Delete the chatbot boilerplate.",
},
# Literal words remain out of the pack unless a jargon collocation is clear.
"game changer": {
"category": "jargon",
"severity": "hard",
"suggestion": "significant, important",
},
"synergy": {
"category": "jargon",
"severity": "hard",
"suggestion": "cooperation, collaboration",
},
"robust": {
"category": "jargon",
"severity": "soft",
"suggestion": "strong, solid, thorough",
},
"comprehensive": {
"category": "jargon",
"severity": "soft",
"suggestion": "full, complete, thorough",
},
# Filler and chatbot knowledge-cutoff framing.
"at the end of the day": {
"category": "filler",
"severity": "hard",
"suggestion": None,
},
"in today's": {
"category": "filler",
"severity": "hard",
"suggestion": None,
},
"as of my last": {
"category": "knowledge_cutoff",
"severity": "hard",
"suggestion": "Delete the training-cutoff disclaimer.",
},
"why should you care": {
"category": "rhetorical_question",
"severity": "hard",
"suggestion": "State why it matters directly.",
},
}
# Compile literal phrase regexes once at module load. In-process callers may
# scan many documents; they should not pay this fixed compilation cost per
# document (subprocess callers still pay it once per process, as before).
for _banned_phrase in BANNED_PHRASES:
_phrase_pattern_ci(_banned_phrase)
# Compact structural pack. Domain-sensitive branches are gated narrowly so
# literal construction, mechanics, law, medicine, and code remain clean.
STRUCTURAL_PATTERNS: list[dict[str, str]] = [
# Emphasis and throat-clearing.
{
"pattern": r"(?:^|[.!?]\s+)(?:full stop|period)\.",
"category": "emphasis_crutch",
"severity": "hard",
"suggestion": "Cut the one-word emphasis sentence.",
},
{
"pattern": r"\bthe real \w+ (?:is|isn't|was|wasn't|remains)\b",
"category": "throat_clearing",
"severity": "hard",
"suggestion": "State it directly.",
},
{
"pattern": r"(?i)\b(?:the\s+|that\s+|this\s+)?(?:struggle|stakes|pain|threat|risk|danger|fear|hype|magic|hustle|grind|stress|pressure|burnout|concern|consequences|impact|tension|anxiety|disconnect|divide|need|demand|love|chemistry|connection|mechanic|feels?)\s+(?:is|are|was|were)\s+(?:very\s+|so\s+|all\s+too\s+)?real\b",
"category": "emphasis_crutch",
"severity": "soft",
"suggestion": "State what is actually at stake.",
},
# Jargon collocations. Bare literal words with ordinary meanings are not
# structural hits; the fixture keeps a protection for each domain branch.
{
"pattern": r"\bleverag(?:e|es|ed|ing)\s+(?:our\s+|your\s+|their\s+|its\s+|the\s+)?(?:synerg|core\s+compet|strength|expertise|capabilit|technolog|resource|data\b|ai\b|platform|ecosystem|network|audit\s+stream|power\s+of)",
"category": "jargon",
"severity": "hard",
"suggestion": "use, apply",
},
{
"pattern": r"\bnavigat(?:e|es|ed|ing)\s+(?:the\s+|this\s+|these\s+)?(?:complex|challeng|landscape|nuance|intric|water|terrain|maze|minefield|uncertaint|world\s+of|ever-)",
"category": "jargon",
"severity": "hard",
"suggestion": "handle, address, manage",
},
{
# Keep the high-signal abstract use, but leave literal excavation and
# mining language ("delve into the mountain") alone.
"pattern": r"\bdelv(?:e|es|ed|ing)\s+into\s+(?:the\s+)?(?:topic|topics|issue|issues|implication|implications|question|questions|subject|details?|nuance|nuances|meaning|argument|claim|concept|matter|problem|analysis|research|data|strategy|history|world|conversation|evidence|complexit(?:y|ies))\b",
"category": "jargon",
"severity": "hard",
"suggestion": "explore, examine, look at",
},
{
"pattern": r"\bharness(?:es|ed|ing)?\s+(?:the\s+|its\s+|their\s+|our\s+)?(?:power|potential|strength|capabilit|momentum|force|full\s+)",
"category": "jargon",
"severity": "hard",
"suggestion": "use, tap, apply",
},
{
"pattern": r"\bharness(?:es|ed|ing)?\s+(?:(?:(?:the|our|their|its)\s+)?(?:team|group|company|organization|workforce)(?:['’]s)\s+energy\b(?=[^.!?\n]{0,100}\b(?:growth|launch|customer\s+service|transition|collaboration|innovation|expertise|engagement|success|results?)\b)|(?:(?:the|our|their)\s+)?energy\s+(?:and\s+expertise\b|of\s+(?:(?:our|the)\s+)?(?:team|group|workforce|people)\s+(?:collaboration|innovation|expertise)\b|to\s+(?:drive|fuel|accelerate|unlock|advance)\s+(?:innovation|collaboration|engagement|growth|transformation|success|results?)\b))",
"category": "jargon",
"severity": "hard",
"suggestion": "use, focus, coordinate",
},
{
"pattern": r"\bfoster(?:s|ed|ing)?\s+(?:a\s+|an\s+|greater\s+|deeper\s+|stronger\s+)?(?:culture|collaboration|innovation|sense\s+of|community|environment|growth|engagement|inclusion|creativity|dialogue|connection|belonging)",
"category": "jargon",
"severity": "hard",
"suggestion": "build, encourage, create",
},
{
"pattern": r"\bunpack(?:s|ed|ing)?\s+(?:the\s+|this\s+|that\s+|our\s+)?(?:idea|argument|assumption|implication|implications|nuance|meaning|claim|concept|topic|dynamic|why|how|what)\b",
"category": "jargon",
"severity": "hard",
"suggestion": "explain, examine",
},
{
"pattern": r"\bdoubl(?:e|es|ed|ing)\s+down\s+on\s+(?:the\s+|this\s+|that\s+|our\s+|your\s+|its\s+|their\s+|a\s+|an\s+)?(?:strategy|approach|investment|bet|commitment|vision|message|plan|position)\b",
"category": "jargon",
"severity": "hard",
"suggestion": "commit, increase",
},
{
"pattern": r"\bbolster(?:s|ed|ing)?\s+(?:the\s+|this\s+|that\s+|our\s+|your\s+)?(?:argument|case|claim|confidence|credibility|support|position|strategy|effort|security)\b",
"category": "jargon",
"severity": "hard",
"suggestion": "support, strengthen",
},
{
"pattern": r"\bstakeholders?\b[^.!?\n]{0,50}\b(?:buy-in|alignment|engagement|feedback|input|management)\b|\b(?:buy-in|alignment|engagement)\b[^.!?\n]{0,50}\bstakeholders?\b",
"category": "jargon",
"severity": "hard",
"suggestion": "people involved",
},
{
"pattern": r"\b(?:in\s+)?(?:today's|modern|contemporary|business|marketing|tech|ai|media|education|healthcare|finance|industry)\s+landscape\b|\bthe\s+(?:business|marketing|tech|ai|media|education|healthcare|finance|industry)\s+landscape\s+of\b|\bthe\s+landscape\s+of\s+(?:modern\s+|today's\s+|contemporary\s+)?(?:marketing|business|tech\w*|ai|work|media|education|healthcare|finance|the industry)\b",
"category": "jargon",
"severity": "hard",
"suggestion": "situation, field, market",
},
{
"pattern": r"\bload-bearing\s+(?:part|piece|point|claim|idea|insight|assumption|detail|context|constraint|requirement|decision|argument|premise|section|paragraph|sentence|word|term|concept)\b",
"category": "jargon",
"severity": "hard",
"suggestion": "essential, important, necessary",
},
{
"pattern": r"\b(?:our|the|a)\s+wedge\s+into\s+the\s+(?:\w+\s+)?(?:market|enterprise|industry|segment|category|account|vertical)s?\b|\bas\s+a\s+wedge\b",
"category": "jargon",
"severity": "hard",
"suggestion": "opening, angle, advantage, entry point",
},
{
"pattern": r"\b(?:the\s+)?substrate\s+(?:for|of)\s+(?:everything|all|our|the\s+(?:company|business|movement|conversation|debate|work))\b|\bcultural\s+substrate\b",
"category": "ai_vocabulary",
"severity": "soft",
"suggestion": "foundation, base, layer",
},
# Bare unattributed research is the vague-attribution move.
{
"pattern": r"(?:^|[.!?;:]\s+)research\s+(?:indicates|shows|suggests)\b",
"category": "vague_attribution",
"severity": "soft",
"suggestion": "Cite the specific research or name the source.",
},
{
"pattern": r"\bboasts?\s+(?:a\s+|an\s+)?(?:world-class|state-of-the-art|cutting-edge|impressive|stunning|robust|comprehensive|unparalleled|rich|vibrant|array of|host of|range of|wealth of|plethora)",
"category": "promotional",
"severity": "hard",
"suggestion": "has",
},
{
"pattern": r"\b(?:data|numbers?|charts?|graphs?|metrics?|figures?|results?|dashboards?|spreadsheets?|trend\s?lines?|statistics)\s+tells?\s+a\s+(?:clear\s+)?story\b",
"category": "false_agency",
"severity": "hard",
"suggestion": "State what the data shows.",
},
{
"pattern": r"\bplays?\s+an?\s+(?:crucial|key|vital|pivotal|significant|central|important|critical|defining|major)\s+(?:role|part)\b",
"category": "significance_inflation",
"severity": "soft",
"suggestion": "State the specific effect.",
},
# Legal/technical and domain-valid protections.
{
"pattern": r"\bnotwithstanding\b(?!\s+(?:anything\s+to\s+the\s+contrary|the\s+foregoing|any(?:thing)?\s+(?:other\s+)?provision|section|clause|subsection|anything\s+in))",
"category": "ai_vocabulary",
"severity": "soft",
"suggestion": "Use a direct transition.",
},
{
"pattern": r"\b(?:acts|serves|stands|stood)\s+as\s+(?:a|an|the)\s+(?:testament|reminder|symbol|beacon|foundation|cornerstone|gateway|catalyst|bridge|hub|springboard|window|monument|hallmark|blueprint|cautionary|stark|powerful|shining|prime example|case study|model for)\b",
"category": "copula_avoidance",
"severity": "hard",
"suggestion": "Use a direct verb.",
},
{
"pattern": r"\bconstitutes\s+(?:a|an|the)\s+(?:(?:groundbreaking|transformative|trailblazing|seminal|revolutionary|landmark|pivotal|significant|major|key)\s+)?(?:transformation|breakthrough|milestone|achievement|innovation|advance|success|turning\s+point|cornerstone|testament|legacy|game[- ]changer)\b",
"category": "copula_avoidance",
"severity": "hard",
"suggestion": "Use a direct verb.",
},
{
"pattern": r"\bfunctions\s+as\s+(?:a|an|the)\s+(?:(?:seamless|comprehensive|robust|transformative|groundbreaking|strategic|powerful|key|central|critical|all-in-one|single)\s+)?(?:solution|framework|platform|hub|bridge|catalyst|cornerstone|benchmark|testament|symbol|beacon|transformation|milestone|game[- ]changer)\b",
"category": "copula_avoidance",
"severity": "hard",
"suggestion": "Use a direct verb.",
},
# Anti-slop contrast and parallelism.
{
"pattern": r"(?im)(?:^|[.!?]\s+)(?:not|no)\b[^.!?]{0,28}[.!?]\s+(?:the\s+|it'?s?\s+|that'?s?\s+)?[a-z][^.!?]{0,28}[.!?]",
"category": "anti_slop_register",
"severity": "soft",
"suggestion": "Join the fragments into a varied sentence.",
},
{
"pattern": r"not because .+?\. because",
"category": "binary_contrast",
"severity": "hard",
"suggestion": "State the reason in one sentence.",
},
{
"pattern": r"feels like .+?\. it's actually",
"category": "binary_contrast",
"severity": "hard",
"suggestion": "State the diagnosis directly.",
},
{
"pattern": r"\bnot only .+? but also",
"category": "negative_parallelism",
"severity": "hard",
"suggestion": "Use a direct sentence.",
},
{
"pattern": r"\b(?:it'?s not|it\s+is\s+not|this is not|that'?s not|isn'?t|is\s+not|wasn'?t|was\s+not|aren'?t|are\s+not|weren'?t|were\s+not)\s+just\b[^.;!?\n]{1,60}[,;—–-]\s*(?:it'?s|it (?:is|was)|they'?re|that'?s)\b",
"category": "negative_parallelism",
"severity": "hard",
"suggestion": "State the contrast directly.",
},
# Reader-steering and rhetorical-question scaffolding.
{
"pattern": r"(?i)\bin this (?:article|section|post|guide|chapter|paper),?\s+(?:we|i)\s+(?:will|'ll|are going to|shall)\b",
"category": "reader_addressing",
"severity": "soft",
"suggestion": "Start with the point.",
},
{
"pattern": r"(?im)(?:^|[.!?]\s+)whether you'?re (?=[^.!?\n]{0,60}\b(?:a|an|just starting)\s)[^.!?\n]{1,60}\bor\b",
"category": "reader_addressing",
"severity": "soft",
"suggestion": "Cut the audience-flattering opener.",
},
{
"pattern": r"(?im)(?:^|[.!?]\s+)(?:why does this matter|what's the (?:real )?takeaway|why this matters|so what does (?:this|that) mean)\b[^.!?\n]{0,40}[?:]",
"category": "rhetorical_question",
"severity": "soft",
"suggestion": "Answer directly instead of teeing up a self-Q&A.",
},
{
"pattern": r"(?m)^\s*(?:but\s+)?what does this mean for\b",
"category": "rhetorical_question",
"severity": "soft",
"suggestion": "State the consequence directly.",
},
{
"pattern": r"\b(?:could|may|might|can)\s+(?:potentially|possibly)\b",
"category": "hedge_stack",
"severity": "soft",
"suggestion": "Drop the redundant hedge.",
},
{
"pattern": r"(?m)^(?:here are|these are|the top)\s+\d+\s+(?:reasons|things|takeaways|lessons|ways)\b",
"category": "numbered_list_inflation",
"severity": "soft",
"suggestion": "List only the points that matter.",
},
]
def _sentence_context(text: str, start: int, end: int) -> str:
"""Return the sentence containing a match, preserving its original width."""
left = max(text.rfind(mark, 0, start) for mark in ".!?\n")
right_candidates = [text.find(mark, end) for mark in ".!?\n"]
right_candidates = [pos for pos in right_candidates if pos >= 0]
right = min(right_candidates) if right_candidates else len(text)
return text[left + 1 : right]
def _context_has(pattern: str, text: str, start: int, end: int) -> bool:
"""Check a small sentence/window around a match for domain evidence."""
sentence = _sentence_context(text, start, end)
if re.search(pattern, sentence, re.IGNORECASE):
return True
window = text[max(0, start - 180) : min(len(text), end + 180)]
return bool(re.search(pattern, window, re.IGNORECASE))
_LEGAL_CONTEXT = (
r"\b(?:act|agreement|agency|clause|contract|court|defendant|filing|hearing|"
r"judge|jury|landlord|law|lease|legal|liabilit(?:y|ies)|ordinance|part(?:y|ies)|plaintiff|"
r"proceedings?|provision|pursuant|regulat(?:e|ed|ion|ory)|rights?|section|"
r"statute|subsection|tenant\w*|warrant)\b"
)
_HISTORICAL_CONTEXT = (
r"\b(?:archaeolog\w*|archive\w*|artifact\w*|catalog\w*|chronicle\w*|"
r"document\w*|histor\w*|museum\w*|preserv\w*|record\w*|tradition\w*|"
r"custom\w*|excavat\w*|ancestr\w*)\b"
)
_PROMOTIONAL_CONTEXT = (
r"\b(?:boast\w*|celebrat\w*|famous|known|renowned|touris\w*|visitor\w*|"
r"destination|vibrant|stunning|impressive|rich\s+in)\b"
)
_SOURCE_CONTEXT = (
r"\b(?:according\s+to|per|citing|based\s+on|as\s+(?:reported|stated|"
r"estimated)\s+by)\b|\b(?:survey|report|study|data|figures?)\s+"
r"(?:from|by|of|says?|shows?|finds?|estimates?|projects?|predicts?)\b"
)
_MEDICAL_CONTEXT = (
r"\b(?:anatom\w*|biolog\w*|cancer|cell\w*|clinical\w*|diagnos\w*|"
r"disease\w*|dose\w*|drug\w*|genes?\b|genetic\w*|genomic\w*|health\w*|immune\w*|infection\w*|"
r"inflamm\w*|kidney\w*|liver\w*|medical\w*|medicine|patient\w*|patholog\w*|physiolog\w*|"
r"symptom\w*|therapy|tissue\w*|treatment\w*|tumou?r\w*|syndrome\w*)\b"
)
def _suppress_contextual_match(
phrase: str,
category: str,
scan_text: str,
start: int,
end: int,
) -> bool:
"""Protect ordinary domain-valid uses of otherwise useful weak signals."""
if phrase == "rich cultural heritage":
# Historical/factual descriptions are not travel-brochure copy. Keep
# the broad phrase for genuinely promotional "known for" language.
sentence = _sentence_context(scan_text, start, end)
return bool(
re.search(_HISTORICAL_CONTEXT, sentence, re.IGNORECASE)
and not re.search(_PROMOTIONAL_CONTEXT, sentence, re.IGNORECASE)
)
if phrase == "in today's":
# Date-specific hearing/session language is ordinary reporting, unlike
# the generic "in today's market/landscape" filler.
return bool(
re.match(
r"\s+(?:hearing|court\s+hearing|trial|session|proceedings?)\b",
scan_text[end:],
re.IGNORECASE,
)
)
if phrase == "analysts predict":
# Keep unattributed forecasts flagged, but trust a nearby explicit
# source/record cue ("according to the survey", "Reuters reports", …).
sentence = _sentence_context(scan_text, start, end)
if re.search(_SOURCE_CONTEXT, sentence, re.IGNORECASE):
return True
return False
if category == "negative_parallelism" and scan_text[start:end].lower().startswith("not only"):
# Legal drafting uses this parallel construction as precise scope, not
# as the canned contrast the rule targets.
return bool(
re.search(
_LEGAL_CONTEXT,
_sentence_context(scan_text, start, end),
re.IGNORECASE,
)
)
if category == "significance_inflation" and re.match(
r"plays?\s+an?\s+(?:crucial|key|vital|pivotal|significant|central|important|critical|defining|major)\s+(?:role|part)\b",
scan_text[start:end],
re.IGNORECASE,
):
# Scientific and medical prose often needs this precise causal wording.
return _context_has(_MEDICAL_CONTEXT, scan_text, start, end)
return False
def scan_for_violations(text: str, include_quoted: bool = False) -> list[Violation]:
"""Scan text for banned phrases and structural patterns."""
violations: list[Violation] = []
spans: list[tuple[int, int]] = []
scan_text = mask_ignored_spans(text, include_quoted=include_quoted)
# Cheap presence index: most documents contain only a few of the
# retained literal phrases.
# literal phrases. Precise matches still run against the original-width
# text, so Unicode case expansion cannot corrupt reported offsets.
# Normalize curly apostrophes only for the cheap presence check. Match
# against the original-width text below so reported offsets stay exact.
scan_text_lower = scan_text.lower().replace("’", "'")
line_starts = _line_starts(text)
line_context_cache: dict[int, str] = {}
# Check banned phrases. Matching runs case-insensitively directly on
# scan_text (original case, masking is length-preserving) instead of on a
# separately lowered copy, so match.start()/match.end() are valid offsets
# into the original text with no re-derivation needed.
for phrase, info in BANNED_PHRASES.items():
if phrase not in scan_text_lower:
continue
for match in _phrase_pattern_ci(phrase).finditer(scan_text):
if _suppress_contextual_match(
phrase,
info["category"],
scan_text,
match.start(),
match.end(),
):
continue
tail = scan_text[match.end():]
if (
phrase == "robust"
and re.match(
r"\s+(?:hash\s+verification|retry\s+mechanism|error\s+handling|test\s+suite)\b",
tail,
re.IGNORECASE,
)
) or (
phrase == "comprehensive"
and re.match(
r"\s+(?:visual\s+survey|needs\s+screen)\b",
tail,
re.IGNORECASE,
)
):
continue
pos = match.start()
line_num, column, context = _line_col_context(
text, line_starts, pos, line_context_cache
)
violations.append({
"phrase": phrase,
"category": info["category"],
"severity": info.get("severity", "hard"),
"line_number": line_num,
"column": column,
"context": context,
"suggestion": info["suggestion"]
})
spans.append((match.start(), match.end()))
# Check structural patterns
for pattern_info in STRUCTURAL_PATTERNS:
matches = list(re.finditer(pattern_info["pattern"], scan_text, re.IGNORECASE))
min_matches = int(pattern_info.get("min_matches", "1"))
if len(matches) < min_matches:
continue
for match in matches:
if _suppress_contextual_match(
match.group().lower(),
pattern_info["category"],
scan_text,
match.start(),
match.end(),
):
continue
pos = match.start()
line_num, column, context = _line_col_context(
text, line_starts, pos, line_context_cache
)
violations.append({
# Preserve the pre-fix lowercase phrase field: STRUCTURAL_PATTERNS
# regexes are lowercase literals, previously matched against a
# lowered copy of the text, so match.group() was always lowercase.
"phrase": match.group().lower(),
"category": pattern_info["category"],
"severity": pattern_info.get("severity", "hard"),
"line_number": line_num,
"column": column,
"context": context,
"suggestion": pattern_info["suggestion"]
})
spans.append((match.start(), match.end()))
# Frequency-gated structural findings (min_matches > 1) describe the DOCUMENT, not
# a single span. A broad, unrelated match (e.g. anti_slop_register spanning several
# short headlines) must not silently swallow every occurrence and erase the
# document-level tell, so these categories are exempt from containment suppression.
freq_gated = {
p["category"] for p in STRUCTURAL_PATTERNS if int(p.get("min_matches", "1")) > 1
}
# Linear containment sweep, equivalent to the O(n^2) "any other strictly
# larger span fully encloses mine" check above, but O(n log n): sort by
# (start ascending, end descending) and sweep once. For any two spans with
# other_start <= start and end <= other_end, (other_end - other_start) >
# (end - start) holds automatically UNLESS the spans are identical -- so
# weak enclosure by a DISTINCT span is exactly the strict-length condition.
# A running "tallest end seen among strictly earlier starts" plus a
# within-group max (for spans sharing the current start) reproduces this
# without ever comparing every pair.
n = len(violations)
order = sorted(range(n), key=lambda i: (spans[i][0], -spans[i][1]))
contained = [False] * n
max_end_before_group = -1
idx = 0
while idx < n:
group_start = spans[order[idx]][0]
group_end = idx
while group_end < n and spans[order[group_end]][0] == group_start:
group_end += 1
running_max_in_group = -1
for vi in order[idx:group_end]:
end = spans[vi][1]
if max_end_before_group >= end or running_max_in_group > end:
contained[vi] = True
if end > running_max_in_group:
running_max_in_group = end
if running_max_in_group > max_end_before_group:
max_end_before_group = running_max_in_group
idx = group_end
violations = [
v for i, v in enumerate(violations)
if not contained[i] or v["category"] in freq_gated
]
# Sort by line number, then column
violations.sort(key=lambda v: (v["line_number"], v["column"]))
return violations
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("input_file", nargs="?", help="Optional input file. Reads stdin when omitted.")
parser.add_argument(
"--include-quoted",
action="store_true",
help="Scan quoted examples and markdown blockquotes instead of skipping them.",
)
return parser.parse_args()
def main() -> None:
args = parse_args()
# Read input
if args.input_file:
try:
with open(args.input_file, 'r', errors="replace") as f:
text = f.read()
except OSError as e:
print(json.dumps({"error": f"Could not read input: {e}", "violations": []}))
sys.exit(2)
else:
text = sys.stdin.buffer.read().decode("utf-8", errors="replace")
if not text.strip():
print(json.dumps({"error": "No input provided", "violations": []}))
sys.exit(1)
violations = scan_for_violations(text, include_quoted=args.include_quoted)
# English-only graceful decline. Function-word absence alone is not evidence
# of a foreign language: imperative stacks and buzzword lists are English
# slop with few function words. Decline only when the text both fails the
# function-word heuristic AND produced zero English-pattern hits.
if not violations and not is_probably_english(text):
print(json.dumps({"non_english": True, "total_violations": 0, "violations": []}, indent=2))
print("note: input appears non-English; scanner declined (English-only).", file=sys.stderr)
sys.exit(0)
# Group by category for summary
categories: dict[str, int] = {}
by_severity: dict[str, int] = {"hard": 0, "soft": 0}
for v in violations:
categories[v["category"]] = categories.get(v["category"], 0) + 1
by_severity[v["severity"]] = by_severity.get(v["severity"], 0) + 1
output = {
"total_violations": len(violations),
"by_severity": by_severity,
"by_category": categories,
"violations": violations
}
print(json.dumps(output, indent=2))
# Exit with 1 if violations found
sys.exit(1 if violations else 0)
if __name__ == "__main__":
main()
@@ -0,0 +1,580 @@
#!/usr/bin/env python3
"""
Deterministic dimension-controlled pair generation for the teach calibration game.
Given a base passage and ONE voice dimension, produce a minimal pair (A, B) where
B differs from A along exactly that dimension. The direction of the transform
(e.g. contract -> expand vs expand -> contract) is auto-detected from whichever
pole the base passage already sits at, so every dimension is reversible: feed it
a passage already on one pole and it moves to the other.
Every transform must preserve the must-preserve constraint tokens extracted by
scripts/extract_constraints.py (numbers, dates, names, quotes, units, etc). If a
dimension has no expressible transform in the given passage (neither pole's
pattern is present, or the only candidate transform would drop a constraint),
the command exits 3 with "dimension not expressible in this passage".
Usage:
python3 calibrate_pairs.py generate --base FILE --dimension DIM --seed N
python3 calibrate_pairs.py --list-dimensions
Dimensions: contractions, em_dash, sentence_length, connectives, staccato.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import random
import re
import sys
from pathlib import Path
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
from extract_constraints import extract_constraints # noqa: E402
from banned_phrase_scan import scan_for_violations # noqa: E402
DIMENSIONS = ["contractions", "em_dash", "sentence_length", "connectives", "staccato"]
# Each dimension has two named poles. transform_applied is "<dimension>:<pole of B>".
POLES: dict[str, tuple[str, str]] = {
"contractions": ("contracted", "expanded"),
"em_dash": ("dashed", "plain"),
"sentence_length": ("long", "short"),
"connectives": ("plain", "formal"),
"staccato": ("staccato", "flowing"),
}
class NotExpressible(Exception):
"""Raised when a dimension has no applicable transform in the passage."""
# --------------------------------------------------------------------------
# contractions: expand <-> contract via a fixed, unambiguous mapping table.
# --------------------------------------------------------------------------
_CONTRACTION_PAIRS = [
("do not", "don't"), ("does not", "doesn't"), ("did not", "didn't"),
("cannot", "can't"), ("will not", "won't"), ("would not", "wouldn't"),
("should not", "shouldn't"), ("could not", "couldn't"), ("must not", "mustn't"),
("is not", "isn't"), ("are not", "aren't"), ("was not", "wasn't"), ("were not", "weren't"),
("have not", "haven't"), ("has not", "hasn't"), ("had not", "hadn't"),
("I am", "I'm"), ("you are", "you're"), ("we are", "we're"), ("they are", "they're"),
("I will", "I'll"), ("you will", "you'll"), ("we will", "we'll"), ("they will", "they'll"),
("he will", "he'll"), ("she will", "she'll"), ("it will", "it'll"),
("I have", "I've"), ("you have", "you've"), ("we have", "we've"), ("they have", "they've"),
("let us", "let's"),
("it is", "it's"), ("that is", "that's"), ("there is", "there's"),
("here is", "here's"), ("who is", "who's"), ("what is", "what's"),
]
# Pairs whose expanded side always keeps a hard-capitalized pronoun ("I ...").
_HARD_CAPITAL = {e for e, c in _CONTRACTION_PAIRS if e.startswith("I ")}
def _case_like(template: str, source_first_char: str) -> str:
if source_first_char.isupper():
return template[0].upper() + template[1:]
return template[0].lower() + template[1:]
# Words safe to lowercase when a sentence is folded mid-clause. A capitalized
# word NOT in this set is assumed to be a proper noun (constraint) and is left
# alone -- lowercasing "Priya" into a joined clause would corrupt a name.
_LOWERABLE_JOINERS = {
"the", "it", "she", "he", "they", "this", "that", "there", "i", "we", "you",
"who", "what", "when", "where", "why", "how", "a", "an", "and", "but", "so",
"yet", "or", "nobody", "everybody", "everyone", "someone", "something",
"nothing", "no", "her", "his", "its", "their", "our", "your",
}
def _lower_first_word(s: str) -> str:
"""Lowercase the leading word only if it is a common function word/pronoun,
never a token that looks like a proper noun a constraint might depend on."""
m = re.match(r"[A-Za-z']+", s)
if not m:
return s
word = m.group(0)
if word.lower() in _LOWERABLE_JOINERS:
return word.lower() + s[len(word):]
return s
_WORD_BEFORE_RE = re.compile(r"([A-Za-z']+)\s*$")
_WORD_AFTER_RE = re.compile(r"^\s*([A-Za-z']+)")
def _is_title_case(word: str) -> bool:
return bool(word) and word[0].isupper() and word[1:].islower()
def _in_capitalized_span(m: "re.Match[str]") -> bool:
"""True if this contraction/expansion match sits inside a capitalized
multi-word span (e.g. "Venue Can't Stop") that looks like a proper noun,
rather than an ordinary sentence-level contraction.
A contraction rewrite is only skipped when the match ITSELF is
capitalized (so an ordinary sentence-initial "Can't you..." is untouched)
AND an immediately adjacent word is also Title Case, forming a >= 2-word
capitalized run. "Maria Chen can't attend" is not skipped (the match is
lowercase); "the Venue Can't Stop" is skipped (both "Venue" and "Can't"
are capitalized and adjacent).
"""
matched = m.group(0)
if not matched[0].isupper():
return False
text = m.string
before_m = _WORD_BEFORE_RE.search(text[: m.start()])
after_m = _WORD_AFTER_RE.match(text[m.end() :])
before_word = before_m.group(1) if before_m else ""
after_word = after_m.group(1) if after_m else ""
return _is_title_case(before_word) or _is_title_case(after_word)
def _contraction_repl(replacement: str, hard: bool):
"""One replacement closure shared by both contraction directions (expand
and contract) and both casing modes (hard-capital literal vs case-matched).
"""
def repl(m):
if _in_capitalized_span(m):
return m.group(0)
return replacement if hard else _case_like(replacement, m.group(0)[0])
return repl
def _apply_contractions(text: str) -> tuple[str, str]:
contract_hits = []
expand_hits = []
for expanded, contracted in _CONTRACTION_PAIRS:
if expanded in _HARD_CAPITAL:
# "I am"/"I'll"/... always literally capitalized; case-sensitive match.
if re.search(re.escape(contracted), text):
contract_hits.append((expanded, contracted))
if re.search(re.escape(expanded), text):
expand_hits.append((expanded, contracted))
else:
if re.search(r"\b" + re.escape(contracted) + r"\b", text, re.IGNORECASE):
contract_hits.append((expanded, contracted))
if re.search(r"\b" + re.escape(expanded) + r"\b", text, re.IGNORECASE):
expand_hits.append((expanded, contracted))
if contract_hits:
out = text
for expanded, contracted in contract_hits:
hard = expanded in _HARD_CAPITAL
pattern = re.escape(contracted) if hard else r"\b" + re.escape(contracted) + r"\b"
flags = 0 if hard else re.IGNORECASE
out = re.sub(pattern, _contraction_repl(expanded, hard), out, flags=flags)
return out, "expanded"
if expand_hits:
out = text
for expanded, contracted in expand_hits:
hard = expanded in _HARD_CAPITAL
pattern = re.escape(expanded) if hard else r"\b" + re.escape(expanded) + r"\b"
flags = 0 if hard else re.IGNORECASE
out = re.sub(pattern, _contraction_repl(contracted, hard), out, flags=flags)
return out, "contracted"
raise NotExpressible("no contraction or expandable phrase found")
# --------------------------------------------------------------------------
# em_dash: paired em-dash parentheticals <-> paired-comma parentheticals ONLY.
#
# Only the paired-dash<->paired-comma path is expressible for this dimension.
# A lone joiner dash (" — nobody objected") has no comma-pair equivalent that
# preserves sentence count: turning it into a period (the old behavior) split
# one sentence into two, silently changing sentence count between A and B.
# Passages with only a lone dash (no paired construction) are declined rather
# than forced through a transform that shifts sentence count.
# --------------------------------------------------------------------------
_PAIRED_DASH_RE = re.compile(r"\s—\s(.+?)\s—\s")
_PAIRED_COMMA_RE = re.compile(r",\s+(.+?),\s+")
def _apply_em_dash(text: str) -> tuple[str, str]:
if _PAIRED_DASH_RE.search(text):
out = _PAIRED_DASH_RE.sub(lambda m: ", " + m.group(1) + ", ", text)
if "—" in out:
# A lone dash survived alongside the paired one -- the restricted
# paired-dash<->comma path can't fully express this passage
# without also touching the lone dash (and changing sentence
# count), so decline rather than ship a half-converted pair.
raise NotExpressible(
"passage mixes a paired em dash with a lone em dash; "
"the paired-dash<->comma path can't resolve the lone dash "
"without changing sentence count"
)
return out, "plain"
if "—" in text:
raise NotExpressible(
"only a lone em dash found; the em_dash dimension is restricted "
"to the paired-dash<->comma path so sentence count stays stable"
)
if _PAIRED_COMMA_RE.search(text):
out = _PAIRED_COMMA_RE.sub(lambda m: " — " + m.group(1) + " — ", text)
return out, "dashed"
raise NotExpressible("no paired em dash or comma-bounded parenthetical found")
# --------------------------------------------------------------------------
# sentence_length: split at coordinators <-> join short adjacent sentences.
# --------------------------------------------------------------------------
_COORD_RE = re.compile(r",\s+(and|but|or|so|yet)\s+", re.IGNORECASE)
def _apply_sentence_length(text: str) -> tuple[str, str]:
sentences = re.findall(r"[^.!?]+[.!?]+", text)
for sent in sentences:
for m in _COORD_RE.finditer(sent):
before = sent[:m.start()]
after = sent[m.end():]
if len(before.split()) >= 3 and len(after.split()) >= 3:
new_sent = before.rstrip() + ". " + m.group(1).capitalize() + " " + after
out = text.replace(sent, new_sent, 1)
return re.sub(r"\s+", " ", out).strip(), "short"
for i in range(len(sentences) - 1):
a_words = sentences[i].strip().split()
b_words = sentences[i + 1].strip().split()
if len(a_words) <= 8 and len(b_words) <= 8:
first_no_period = re.sub(r"[.!?]+\s*$", "", sentences[i].strip())
second = sentences[i + 1].strip()
second_body = re.sub(r"[.!?]+\s*$", "", second)
end_punct = second[len(second_body):].strip() or "."
joined = first_no_period + ", and " + _lower_first_word(second_body) + end_punct
prefix = "".join(sentences[:i])
suffix = "".join(sentences[i + 2:])
out = (prefix + " " + joined + " " + suffix).strip()
return re.sub(r"\s+", " ", out), "long"
raise NotExpressible("no coordinator split site or joinable short sentences found")
# --------------------------------------------------------------------------
# connectives: formal <-> plain via a fixed table.
# --------------------------------------------------------------------------
_FORMAL_TO_PLAIN = {
"however": "But", "additionally": "Also", "therefore": "So",
"furthermore": "Also", "moreover": "Also", "nevertheless": "Still",
"consequently": "So", "nonetheless": "Still", "subsequently": "Then",
"thus": "So",
}
_PLAIN_TO_FORMAL = {
"but": "However", "also": "Additionally", "so": "Therefore",
"still": "Nevertheless", "then": "Subsequently",
}
_FORMAL_RE = re.compile(
r"(?:^|(?<=[.!?]\s))(" + "|".join(_FORMAL_TO_PLAIN) + r"),?\s+", re.IGNORECASE
)
_PLAIN_RE = re.compile(
r"(?:^|(?<=[.!?]\s))(" + "|".join(_PLAIN_TO_FORMAL) + r"),?\s+", re.IGNORECASE
)
def _apply_connectives(text: str) -> tuple[str, str]:
if _FORMAL_RE.search(text):
def repl(m):
# No comma after the plain form: "But " reads as ordinary speech;
# "But, " is stilted, and "So, " specifically trips the scanner's
# filler_opener structural pattern. Dropping the comma across the
# board keeps every plain form clean of that tell.
return _FORMAL_TO_PLAIN[m.group(1).lower()] + " "
out = _FORMAL_RE.sub(repl, text)
return out, "plain"
if _PLAIN_RE.search(text):
def repl2(m):
return _PLAIN_TO_FORMAL[m.group(1).lower()] + ", "
out = _PLAIN_RE.sub(repl2, text)
return out, "formal"
raise NotExpressible("no formal or plain connective found")
# --------------------------------------------------------------------------
# staccato: fragment runs <-> flowing clauses.
# --------------------------------------------------------------------------
_LEADING_JOINER_RE = re.compile(
r"^(?:because|although|while|since|if|when|and|but|so|yet|or)\s+", re.IGNORECASE
)
def _apply_staccato(text: str, rng: random.Random) -> tuple[str, str]:
sentences = re.findall(r"[^.!?]+[.!?]+", text)
# Reverse direction: a run of >= 3 consecutive short (<=6 word) sentences -> flow.
runs = []
run_start = None
for i, sent in enumerate(sentences):
words = sent.strip().split()
if len(words) <= 6:
if run_start is None:
run_start = i
else:
if run_start is not None and i - run_start >= 3:
runs.append((run_start, i))
run_start = None
if run_start is not None and len(sentences) - run_start >= 3:
runs.append((run_start, len(sentences)))
if runs:
start, end = runs[rng.randrange(len(runs))] if len(runs) > 1 else runs[0]
parts = []
for idx in range(start, end):
body = re.sub(r"[.!?]+\s*$", "", sentences[idx].strip())
parts.append(body)
joined_parts = []
for i, p in enumerate(parts):
if i == 0:
joined_parts.append(p)
elif i == len(parts) - 1:
joined_parts.append("and " + _lower_first_word(p))
else:
joined_parts.append(_lower_first_word(p))
joined = ", ".join(joined_parts) + "."
prefix = "".join(sentences[:start])
suffix = "".join(sentences[end:])
out = (prefix + " " + joined + " " + suffix).strip()
return re.sub(r"\s+", " ", out), "flowing"
# Forward direction: the sentence with the most commas (>= 2) fragments.
best_idx, best_commas = None, 1
for i, sent in enumerate(sentences):
commas = sent.count(",")
if commas > best_commas:
best_idx, best_commas = i, commas
if best_idx is not None:
sent = sentences[best_idx]
body = re.sub(r"[.!?]+\s*$", "", sent.strip())
end_punct = sent.strip()[len(body):].strip() or "."
fragments = [f.strip() for f in body.split(",")]
cleaned = []
for frag in fragments:
frag = _LEADING_JOINER_RE.sub("", frag).strip()
if not frag:
continue
frag = frag[0].upper() + frag[1:]
cleaned.append(frag)
if len(cleaned) < 3:
raise NotExpressible("comma split did not yield a fragment run")
cleaned[-1] = cleaned[-1] + end_punct if not cleaned[-1].endswith((".", "!", "?")) else cleaned[-1]
new_sent = " ".join(
f if f.endswith((".", "!", "?")) else f + "." for f in cleaned
)
out = text.replace(sent, new_sent, 1)
return re.sub(r"\s+", " ", out).strip(), "staccato"
raise NotExpressible("no fragment run or multi-comma sentence found")
_APPLY = {
"contractions": lambda text, rng: _apply_contractions(text),
"em_dash": lambda text, rng: _apply_em_dash(text),
"sentence_length": lambda text, rng: _apply_sentence_length(text),
"connectives": lambda text, rng: _apply_connectives(text),
"staccato": _apply_staccato,
}
_EXAMPLES = {
"contractions": {
"description": "Expand <-> contract via a fixed, unambiguous mapping table "
"(do not <-> don't, I am <-> I'm, it is <-> it's, ...).",
"a": "The rollout is not finished, and I am not confident it will ship Friday.",
"b": "The rollout isn't finished, and I'm not confident it'll ship Friday.",
},
"em_dash": {
"description": "Paired em-dash parentheticals <-> paired-comma parentheticals only "
"(a lone joiner dash has no comma-pair equivalent that preserves "
"sentence count, so it is declined rather than converted).",
"a": "The plan — untested and rushed — still shipped on time.",
"b": "The plan, untested and rushed, still shipped on time.",
},
"sentence_length": {
"description": "Split at a coordinator <-> join two short adjacent sentences.",
"a": "The team shipped the fix, and the client renewed the contract.",
"b": "The team shipped the fix. And the client renewed the contract.",
},
"connectives": {
"description": "Formal <-> plain connective swap via a fixed table "
"(However -> But, Additionally -> Also, Therefore -> So, ...). "
"Plain forms drop the comma after the connective (\"But \", not "
"\"But, \") so they read as ordinary speech, not a filler-opener tell.",
"a": "However, the numbers slipped in March.",
"b": "But the numbers slipped in March.",
},
"staccato": {
"description": "Fragment runs <-> flowing clauses.",
"a": "Because the deploy failed, and the on-call missed the page, the team lost an hour.",
"b": "The deploy failed. The on-call missed the page. The team lost an hour.",
},
}
def _is_word_char(ch: str) -> bool:
return ch.isalnum()
def _has_whole_occurrence(value: str, text: str) -> bool:
"""True if `value` occurs in `text` as a standalone span rather than as a
substring straddling the edge of a longer, different word.
Plain substring containment is not enough: expanding "Can't" to "Cannot"
still contains the literal characters "Can", so a naive `"Can" in
"Cannot"` check reports a corrupted proper noun as "preserved". At each
edge of a candidate match, this only counts as a clash (disqualifying the
match) when BOTH the value's own boundary character and the adjacent text
character are alphanumeric -- i.e. when `value` could be glued onto a
longer token. Values that start/end with punctuation (currency signs,
quotes) never clash on that edge, since punctuation can't be silently
swallowed into a bigger word the way a letter/digit can.
"""
if not value:
return False
start = 0
while True:
idx = text.find(value, start)
if idx == -1:
return False
left_clash = idx > 0 and _is_word_char(text[idx - 1]) and _is_word_char(value[0])
end = idx + len(value)
right_clash = end < len(text) and _is_word_char(text[end]) and _is_word_char(value[-1])
if not left_clash and not right_clash:
return True
start = idx + 1
def _verify_constraints_preserved(base_text: str, transformed_text: str) -> list[str]:
"""Return constraint values from base_text missing from transformed_text.
Uses whole-occurrence (token-boundary) comparison rather than plain
substring containment -- see `_has_whole_occurrence` for why a substring
check is unsafe here.
"""
missing = []
for c in extract_constraints(base_text):
value = c["value"]
if _has_whole_occurrence(value, transformed_text):
continue
# Allow whitespace-normalized matches (transforms may collapse spacing).
normalized_value = re.sub(r"\s+", " ", value)
normalized_text = re.sub(r"\s+", " ", transformed_text)
if normalized_value != value and _has_whole_occurrence(normalized_value, normalized_text):
continue
missing.append(value)
return missing
def _scan_flags(text: str) -> list[str]:
"""Category names banned_phrase_scan raises on `text`, deduped and sorted.
Empty when the text scans clean. Generated B-variants can legitimately
trip the scanner (e.g. a staccato pole reads as anti_slop_register; see
references/calibrate.md "Voice overrides defaults") -- that is surfaced
here, not treated as a reason to decline the pair.
"""
categories = {v["category"] for v in scan_for_violations(text)}
return sorted(categories)
def generate_pair(base_text: str, dimension: str, seed: int) -> dict:
if dimension not in DIMENSIONS:
raise ValueError(f"unknown dimension: {dimension}")
rng = random.Random(seed)
transformed, pole = _APPLY[dimension](base_text, rng)
if transformed.strip() == base_text.strip():
raise NotExpressible("transform produced no change")
missing = _verify_constraints_preserved(base_text, transformed)
if missing:
raise NotExpressible(
"transform would drop must-preserve constraint(s): " + ", ".join(missing)
)
pair_id = hashlib.sha256(
f"{base_text}|{dimension}|{seed}".encode("utf-8")
).hexdigest()
return {
"pair_id": pair_id,
"dimension": dimension,
"a_text": base_text,
"b_text": transformed,
"transform_applied": f"{dimension}:{pole}",
"a_flags": _scan_flags(base_text),
"b_flags": _scan_flags(transformed),
}
def list_dimensions() -> dict:
return {
dim: {
"poles": list(POLES[dim]),
"description": _EXAMPLES[dim]["description"],
"example": {"a": _EXAMPLES[dim]["a"], "b": _EXAMPLES[dim]["b"]},
}
for dim in DIMENSIONS
}
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest="command")
gen = sub.add_parser("generate", help="generate one dimension-controlled pair")
gen.add_argument("--base", required=True, help="path to base passage file")
gen.add_argument("--dimension", required=True, choices=DIMENSIONS)
gen.add_argument("--seed", type=int, default=0)
parser.add_argument("--list-dimensions", action="store_true")
return parser.parse_args(argv)
def main(argv: list[str]) -> int:
args = parse_args(argv)
if args.list_dimensions:
print(json.dumps(list_dimensions(), indent=2, sort_keys=True))
return 0
if args.command != "generate":
print(json.dumps({"error": "no command given; use generate or --list-dimensions"}))
return 1
try:
base_text = Path(args.base).read_text()
except OSError as e:
print(json.dumps({"error": f"could not read base file: {e}"}))
return 2
try:
pair = generate_pair(base_text, args.dimension, args.seed)
except NotExpressible as e:
print(json.dumps({
"error": "dimension not expressible in this passage",
"dimension": args.dimension,
"detail": str(e),
}))
return 3
print(json.dumps(pair, indent=2, sort_keys=True))
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
@@ -0,0 +1,273 @@
#!/usr/bin/env python3
"""
Aggregate a teach-calibration preferences JSONL into per-dimension confidence,
surface conflicts against a measured stylometric profile, and pick the next
dimension to play.
Preferences JSONL rows (one per game round):
{"pair_id": "...", "dimension": "contractions", "choice": "a"|"b"|"neither",
"ts": "...", "a_label": "contracted", "b_label": "expanded"}
`a_label`/`b_label` are optional. When present (calibrate_pairs.py's
transform_applied names the B pole; A sits at the other pole in the same
dimension's POLES pair) they let this script report a semantic preferred
DIRECTION ("expanded", "short", "formal", ...) instead of the bare literal
"a"/"b" tally, which is what --profile conflict detection needs. Rows without
labels still count toward n and the confidence interval; they just can't
resolve to a named pole (direction is reported as "a" or "b").
Usage:
python3 calibrate_score.py --preferences prefs.jsonl
python3 calibrate_score.py --preferences prefs.jsonl --profile profile.json
python3 calibrate_score.py --preferences prefs.jsonl --next
"""
from __future__ import annotations
import argparse
import json
import math
import sys
from pathlib import Path
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
from calibrate_pairs import DIMENSIONS, POLES # noqa: E402
MIN_K = 5
Z = 1.96
# Dimension -> (profile_key, pole_meaning) used only for --profile conflict
# detection. `low_pole`/`high_pole` say which named pole a LOW vs HIGH measured
# value corresponds to, so a confident stated preference for the opposite pole
# of what the measured value implies is flagged.
_PROFILE_LINKS: dict[str, dict] = {
"contractions": {
"profile_key": "contraction_rate",
"low_pole": "expanded",
"high_pole": "contracted",
"low_threshold": 0.05,
"high_threshold": 0.20,
},
"sentence_length": {
"profile_key": "avg_sentence_length",
"low_pole": "short",
"high_pole": "long",
"low_threshold": 12.0,
"high_threshold": 20.0,
},
"staccato": {
"profile_key": "avg_sentence_length",
"low_pole": "staccato",
"high_pole": "flowing",
"low_threshold": 12.0,
"high_threshold": 20.0,
},
"em_dash": {
"profile_key": "em_dash_rate",
"low_pole": "plain",
"high_pole": "dashed",
"low_threshold": 0.02,
"high_threshold": 0.15,
},
"connectives": {
"profile_key": "formal_connective_rate",
"low_pole": "plain",
"high_pole": "formal",
"low_threshold": 0.10,
"high_threshold": 0.40,
},
}
CONFIDENCE_STOP = 0.7
K_STOP = 9
def wilson_lower_bound(successes: int, n: int, z: float = Z) -> float:
if n == 0:
return 0.0
phat = successes / n
denom = 1 + z * z / n
center = phat + z * z / (2 * n)
margin = z * math.sqrt((phat * (1 - phat) + z * z / (4 * n)) / n)
return round(max(0.0, (center - margin) / denom), 3)
def load_preferences(path: Path) -> list[dict]:
rows = []
for line in path.read_text().splitlines():
line = line.strip()
if line:
rows.append(json.loads(line))
return rows
def dedup_by_pair_id(rows: list[dict]) -> list[dict]:
"""Keep one row per `pair_id`: the LATEST by `ts` (ISO 8601, so a plain
string comparison orders correctly). A game round replayed twice (e.g. the
agent crashed mid-write and re-ran the round) must count once toward `n`
and resolve to whatever the user most recently chose, not double-count or
let an earlier write win. Rows without a `pair_id` can't be deduped
against anything, so each is kept as-is.
"""
by_pair_id: dict[str, dict] = {}
passthrough: list[dict] = []
for row in rows:
pid = row.get("pair_id")
if not pid:
passthrough.append(row)
continue
existing = by_pair_id.get(pid)
if existing is None or row.get("ts", "") >= existing.get("ts", ""):
by_pair_id[pid] = row
return list(by_pair_id.values()) + passthrough
def aggregate(rows: list[dict]) -> dict[str, dict]:
rows = dedup_by_pair_id(rows)
result = {dim: {"n": 0, "tally": {}, "neither": 0} for dim in DIMENSIONS}
for row in rows:
dim = row.get("dimension")
if dim not in result:
continue
result[dim]["n"] += 1
choice = row.get("choice")
if choice == "neither":
result[dim]["neither"] += 1
continue
if choice not in ("a", "b"):
continue
label = row.get(f"{choice}_label") or choice
result[dim]["tally"][label] = result[dim]["tally"].get(label, 0) + 1
dimensions = {}
for dim, data in result.items():
n = data["n"]
decisive = sum(data["tally"].values())
if n < MIN_K or decisive == 0:
dimensions[dim] = {
"n": n,
"status": "insufficient",
"preferred": None,
"confidence": 0.0,
}
continue
max_count = max(data["tally"].values())
top_labels = [label for label, count in data["tally"].items() if count == max_count]
if len(top_labels) > 1:
# A genuine tie between the top two (or more) tallies: there is no
# lean to report, so `preferred` must be null rather than
# silently picking whichever label happens to sort last.
dimensions[dim] = {
"n": n,
"status": "tied",
"preferred": None,
"confidence": 0.0,
}
continue
preferred_label = top_labels[0]
confidence = wilson_lower_bound(max_count, decisive)
dimensions[dim] = {
"n": n,
"status": "confident",
"preferred": preferred_label,
"confidence": confidence,
}
return dimensions
def detect_conflicts(dimensions: dict[str, dict], profile: dict) -> list[dict]:
conflicts = []
for dim, data in dimensions.items():
if data["status"] != "confident" or data["confidence"] < CONFIDENCE_STOP:
continue
link = _PROFILE_LINKS.get(dim)
if not link or link["profile_key"] not in profile:
continue
measured = profile[link["profile_key"]]
preferred = data["preferred"]
conflict_pole = None
if preferred == link["low_pole"] and measured >= link["high_threshold"]:
conflict_pole = link["high_pole"]
elif preferred == link["high_pole"] and measured <= link["low_threshold"]:
conflict_pole = link["low_pole"]
if conflict_pole is None:
continue
conflicts.append({
"dimension": dim,
"preferred": preferred,
"preferred_confidence": data["confidence"],
"preferred_provenance": "stated-preference",
"measured_key": link["profile_key"],
"measured_value": measured,
"measured_provenance": "measured-from-samples",
"message": (
f"Stated preference for '{preferred}' ({dim}) contradicts "
f"{link['profile_key']}={measured} measured from samples, "
f"which points toward '{conflict_pole}'."
),
})
return conflicts
def next_dimension(dimensions: dict[str, dict]) -> dict:
def sort_key(dim: str) -> tuple:
data = dimensions[dim]
confidence = data["confidence"] if data["status"] == "confident" else 0.0
return (data["n"], confidence, DIMENSIONS.index(dim))
ordered = sorted(DIMENSIONS, key=sort_key)
chosen = ordered[0]
data = dimensions[chosen]
if data["n"] == min(dimensions[d]["n"] for d in DIMENSIONS):
reason = "fewest_observations"
else:
reason = "lowest_confidence"
return {
"next_dimension": chosen,
"reason": reason,
"n": data["n"],
"confidence": data["confidence"],
}
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--preferences", required=True)
parser.add_argument("--profile")
parser.add_argument("--next", action="store_true")
return parser.parse_args(argv)
def main(argv: list[str]) -> int:
args = parse_args(argv)
try:
rows = load_preferences(Path(args.preferences))
except OSError as e:
print(json.dumps({"error": f"could not read preferences file: {e}"}))
return 2
dimensions = aggregate(rows)
if args.next:
print(json.dumps(next_dimension(dimensions), indent=2, sort_keys=True))
return 0
output = {"dimensions": dimensions}
if args.profile:
try:
profile = json.loads(Path(args.profile).read_text())
except OSError as e:
print(json.dumps({"error": f"could not read profile file: {e}"}))
return 2
output["conflicts"] = detect_conflicts(dimensions, profile)
print(json.dumps(output, indent=2, sort_keys=True))
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
@@ -0,0 +1,80 @@
#!/usr/bin/env python3
"""Validate tiered detector pack integrity."""
from __future__ import annotations
import json
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
PACK_DIR = ROOT / "references" / "packs"
MANIFEST = PACK_DIR / "manifest.json"
JUDGE_ONLY_MACROS = {
"macro_both_sidesism",
"macro_redemption_arc",
"macro_preview_recap",
"macro_over_determination",
"macro_emotional_flatness",
}
def scanner_categories() -> set[str]:
sys.path.insert(0, str(ROOT))
from scripts.banned_phrase_scan import BANNED_PHRASES, STRUCTURAL_PATTERNS
return {v["category"] for v in BANNED_PHRASES.values()} | {
v["category"] for v in STRUCTURAL_PATTERNS
}
def main() -> int:
failures: list[str] = []
data = json.loads(MANIFEST.read_text())
packs: dict[str, list[str]] = data["packs"]
category_to_pack: dict[str, str] = {}
for pack, categories in packs.items():
path = PACK_DIR / f"{pack}.md"
if not path.exists():
failures.append(f"missing pack file: {path}")
continue
text = path.read_text()
line_count = len(text.splitlines())
if line_count > 120:
failures.append(f"{path} has {line_count} lines; max 120")
if "## Emit" not in text:
failures.append(f"{path} missing ## Emit section")
for category in categories:
if category in category_to_pack:
failures.append(
f"category {category} mapped to both {category_to_pack[category]} and {pack}"
)
category_to_pack[category] = pack
scanner = scanner_categories()
missing = scanner - set(category_to_pack)
extra_scanner = (set(category_to_pack) - scanner) - JUDGE_ONLY_MACROS
if missing:
failures.append(f"scanner categories missing from packs: {sorted(missing)}")
if extra_scanner:
failures.append(f"unknown non-macro categories in manifest: {sorted(extra_scanner)}")
macro_hits = {m: category_to_pack.get(m) for m in JUDGE_ONLY_MACROS}
missing_macros = [m for m, pack in macro_hits.items() if pack is None]
if missing_macros:
failures.append(f"judge-only macro families missing: {sorted(missing_macros)}")
for macro, pack in macro_hits.items():
if pack and pack != "pack-structure":
failures.append(f"{macro} must live in pack-structure, found {pack}")
if failures:
print("\n".join(failures), file=sys.stderr)
return 1
print(f"pack integrity ok: {len(packs)} packs, {len(scanner)} scanner categories")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,225 @@
#!/usr/bin/env python3
"""Contract gates for co-writer suggestions.
Reads a suggestions file — the JSON emitted by suggest.py, i.e. an object with a
``document`` string and a ``suggestions`` list — and enforces four blocking
contracts. Each failure is named so a caller can act on it:
span-minimality Every suggested_replacement edits only its own span:
span.text matches document[start:end], the replacement
differs from the span text, and it shares no leading or
trailing whole word with the span (which would mean the
span grabbed unchanged text and could be shrunk — the
signature of a whole-sentence rewrite).
replacement-scanner Every suggested_replacement passes both scanners in
isolation AND introduces no new violation in context.
accept-all Applying every replacement yields a document that passes
both scanners with validate_preservation exit 0 vs the
original.
span-overlap Suggestion spans must not overlap.
Exit 0 when all gates pass, 1 otherwise. Failures (with the offending gate) are
reported as JSON on stdout.
Usage:
python3 scripts/check_suggestions.py suggestions.json
python3 scripts/check_suggestions.py < suggestions.json
"""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
from banned_phrase_scan import scan_for_violations
from structure_scan import scan as structure_scan
from validate_preservation import validate_preservation
def _leading_shared_words(a: str, b: str) -> int:
"""Number of identical leading whole words shared by two strings."""
n = 0
for x, y in zip(a.split(), b.split()):
if x == y:
n += 1
else:
break
return n
def _trailing_shared_words(a: str, b: str) -> int:
ra = " ".join(reversed(a.split()))
rb = " ".join(reversed(b.split()))
return _leading_shared_words(ra, rb)
def _line_starts(text: str) -> list[int]:
starts = [0]
for i, ch in enumerate(text):
if ch == "\n":
starts.append(i + 1)
return starts
def violation_spans(text: str) -> list[tuple[int, int]]:
"""Absolute (start, end) offsets of every banned-phrase/structural violation."""
starts = _line_starts(text)
out = []
for v in scan_for_violations(text):
s = starts[v["line_number"] - 1] + v["column"] - 1
out.append((s, s + len(v["phrase"])))
return out
def _scanners_clean(text: str) -> bool:
return not scan_for_violations(text) and not structure_scan(text).get("flags")
def apply_all(document: str, suggestions: list[dict]) -> str:
"""Apply every non-null replacement, right-to-left so offsets stay valid."""
out = document
for s in sorted(suggestions, key=lambda s: s["span"]["start"], reverse=True):
rep = s.get("suggested_replacement")
if rep is None:
continue
st, en = s["span"]["start"], s["span"]["end"]
out = out[:st] + rep + out[en:]
return out
def check(document: str, suggestions: list[dict]) -> list[dict]:
failures: list[dict] = []
# span-overlap: spans must be disjoint.
order = sorted(range(len(suggestions)),
key=lambda i: (suggestions[i]["span"]["start"], suggestions[i]["span"]["end"]))
last_end = None
last_i = None
for i in order:
st, en = suggestions[i]["span"]["start"], suggestions[i]["span"]["end"]
if last_end is not None and st < last_end:
failures.append({
"gate": "span-overlap",
"suggestions": [last_i, i],
"detail": f"span {st}-{en} overlaps the previous span ending at {last_end}",
})
last_end, last_i = en, i
# span-minimality: span accuracy + tight, local replacements.
for i, s in enumerate(suggestions):
st, en = s["span"]["start"], s["span"]["end"]
span_text = document[st:en]
if span_text != s["span"]["text"]:
failures.append({
"gate": "span-minimality",
"suggestion": i,
"detail": "span.text does not match document[start:end]",
})
rep = s.get("suggested_replacement")
if rep is None:
continue
if rep == span_text:
failures.append({
"gate": "span-minimality",
"suggestion": i,
"detail": "replacement is identical to the span text (no change)",
})
continue
if _leading_shared_words(span_text, rep) > 0 or _trailing_shared_words(span_text, rep) > 0:
failures.append({
"gate": "span-minimality",
"suggestion": i,
"detail": "replacement shares leading/trailing whole words with the span; "
"shrink the span so the edit is minimal",
})
# replacement-scanner: each replacement is clean alone and in context.
for i, s in enumerate(suggestions):
rep = s.get("suggested_replacement")
if rep is None:
continue
if not _scanners_clean(rep):
failures.append({
"gate": "replacement-scanner",
"suggestion": i,
"detail": "replacement does not pass both scanners in isolation",
})
st, en = s["span"]["start"], s["span"]["end"]
ctx = document[:st] + rep + document[en:]
new_start, new_end = st, st + len(rep)
if any(a < new_end and new_start < b for (a, b) in violation_spans(ctx)):
failures.append({
"gate": "replacement-scanner",
"suggestion": i,
"detail": "replacement introduces a violation in context",
})
# accept-all: applying everything yields a clean, constraint-preserving doc.
unresolved = [i for i, s in enumerate(suggestions) if s.get("suggested_replacement") is None]
if unresolved:
failures.append({
"gate": "accept-all",
"detail": f"suggestions {unresolved} have no replacement; cannot accept-all",
})
applied = apply_all(document, suggestions)
if not _scanners_clean(applied):
failures.append({
"gate": "accept-all",
"detail": "the accept-all document still fails a scanner",
})
preservation = validate_preservation(document, applied)
if not preservation["passed"]:
failures.append({
"gate": "accept-all",
"detail": "validate_preservation failed against the original",
"missing": preservation["missing"],
})
return failures
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Contract gates for co-writer suggestions.")
parser.add_argument(
"path", nargs="?", help="Path to a suggestions JSON file (default: read stdin)"
)
return parser.parse_args(argv)
def main(argv: list[str]) -> int:
args = parse_args(argv)
if args.path:
path = Path(args.path)
if not path.exists():
print(f"Missing file: {path}", file=sys.stderr)
return 2
raw = path.read_text(errors="replace")
else:
raw = sys.stdin.buffer.read().decode("utf-8", errors="replace")
try:
data = json.loads(raw)
except json.JSONDecodeError as e:
print(json.dumps({"passed": False, "error": f"invalid JSON: {e}"}, indent=2))
return 1
document = data.get("document")
suggestions = data.get("suggestions", [])
if not isinstance(document, str):
print(json.dumps({"passed": False, "error": "missing 'document' string"}, indent=2))
return 1
failures = check(document, suggestions)
result = {
"passed": not failures,
"failure_gates": sorted({f["gate"] for f in failures}),
"failures": failures,
}
print(json.dumps(result, indent=2))
return 0 if not failures else 1
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
@@ -0,0 +1,373 @@
#!/usr/bin/env python3
"""Offline contribution scaffolder for new unslop AI-isms."""
from __future__ import annotations
import argparse
import hashlib
import json
import re
import subprocess
import sys
from datetime import date, datetime
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
CONTRIB_ROOT = ROOT / ".unslop" / "contrib"
TODO_MARKER = "TODO:"
SLUG_RE = re.compile(r"^[a-z0-9][a-z0-9_-]{0,63}$")
GATE_COMMANDS = [
["python3", "evals/check.py", "--full"],
]
def read_text(path: Path) -> str:
try:
return path.read_text(encoding="utf-8")
except OSError as exc:
print(json.dumps({"error": f"could not read {path}: {exc}"}))
raise SystemExit(2) from exc
def write_json(path: Path, data: object) -> None:
path.write_text(json.dumps(data, indent=2, sort_keys=True) + "\n", encoding="utf-8")
def sha256_text(text: str) -> str:
return hashlib.sha256(text.encode("utf-8")).hexdigest()
def word_count(text: str) -> int:
return len(re.findall(r"[A-Za-z0-9']+", text))
def shorten(text: str, limit: int = 72) -> str:
one_line = " ".join(text.split())
if len(one_line) <= limit:
return one_line
return one_line[: limit - 1].rstrip() + "..."
def run_scanner(command: list[str], text: str) -> subprocess.CompletedProcess[str]:
return subprocess.run(command, input=text, capture_output=True, text=True, cwd=ROOT, timeout=30)
def scanner_findings(text: str) -> list[dict[str, object]]:
findings: list[dict[str, object]] = []
for name, command in (
("banned_phrase", ["python3", "scripts/banned_phrase_scan.py"]),
("structure", ["python3", "scripts/structure_scan.py"]),
):
proc = run_scanner(command, text)
try:
data = json.loads(proc.stdout)
except json.JSONDecodeError:
data = {}
violations = data.get("violations", [])
if name == "structure":
violations = [
{
"phrase": flag.get("metric", ""),
"category": flag.get("metric", ""),
"severity": flag.get("severity", ""),
}
for flag in data.get("flags", [])
]
for violation in violations:
findings.append(
{
"scanner": name,
"phrase": violation.get("phrase", ""),
"category": violation.get("category", ""),
"severity": violation.get("severity", ""),
}
)
return findings
def redaction_pairs(values: list[str]) -> list[tuple[str, str]]:
pairs = []
for value in values:
if "=" not in value:
print(json.dumps({"error": "--redact values must use orig=REPL"}))
raise SystemExit(2)
orig, repl = value.split("=", 1)
pairs.append((orig, repl))
return pairs
def apply_redactions(text: str, tell: str, pairs: list[tuple[str, str]]) -> str:
before_count = text.count(tell)
redacted = text
for orig, repl in pairs:
redacted = redacted.replace(orig, repl)
if before_count == 0 or redacted.count(tell) != before_count:
print(json.dumps({"error": "redaction alters the specimen", "tell": tell}))
raise SystemExit(4)
return redacted
def row_fn(slug: str, tell: str, snippet: str) -> dict[str, object]:
assertion_tell = tell.casefold()
return {
"id": f"CONTRIB-FN-{slug}",
"target": "script",
"category": "scanner_false_negative",
"stdin": snippet,
"assertions": [
{"type": "json", "path": "total_violations", "gte": 1},
{"type": "violation_phrase_contains", "value": assertion_tell},
],
}
def row_fp_template(slug: str, category: str, tell: str) -> dict[str, object]:
return {
"id": f"CONTRIB-FP-{slug}",
"category": "scanner_false_positive",
"protects": category,
"target": "script",
"stdin": "TODO: add a literal or domain-specific use that should remain clean.",
"assertions": [{"type": "json", "path": "total_violations", "equals": 0}],
}
def render_report(
manifest: dict[str, object],
snippet: str,
gate_results: str = "",
include_rec: bool = False,
) -> str:
redactions = manifest.get("redactions", [])
redaction_note = "names/numbers redacted; tell verbatim" if redactions else "none"
rows = [
f"| CONTRIB-FN-{manifest['pattern_name']} | FN | exact specimen flags `{manifest['tell']}` |",
f"| CONTRIB-FP-{manifest['pattern_name']} | FP | literal-use protection for `{manifest['tell']}` |",
]
if include_rec:
rows.append("| CONTRIB-REC | REC | existing-word recall still flags |")
quoted = "\n".join(f"> {line}" if line else ">" for line in snippet.splitlines())
pattern_added = manifest.get(
"pattern_added",
f"TODO: regex or phrase for `{manifest['tell']}`",
)
return (
f"# Add {manifest['category']} pattern: {shorten(str(manifest['tell']))}\n\n"
"## The specimen\n\n"
f"{quoted}\n\n"
f"- Source genre: {manifest.get('source_genre', 'TODO: source genre')}\n"
f"- Date: {manifest['date']}\n"
f"- Redaction note: {redaction_note}\n\n"
"## Why it's an AI-ism\n\n"
f"{manifest.get('rationale', 'TODO: explain why this phrase is a reusable AI-writing tell in 2-4 sentences.')}\n\n"
"## Detection\n\n"
f"- Pattern added: {pattern_added}\n"
f"- Severity: {manifest.get('severity', 'TODO: hard or soft')}\n"
f"- Gating rationale: {manifest.get('gating_rationale', 'TODO: explain literal-use boundary')}\n"
"- Catalog entry location: references/taboo-phrases.md\n\n"
"## Evals\n\n"
"| row id | kind | what it pins |\n"
"|---|---|---|\n"
+ "\n".join(rows)
+ "\n\n"
"The FN stdin is the unmodified specimen after approved redaction.\n\n"
"## Gate results\n\n"
f"{gate_results or 'TODO: paste gate tails from verify.'}\n\n"
"## Checklist\n\n"
"- [ ] eval-first (row was red before the pattern)\n"
"- [ ] literal-use FP row included\n"
"- [ ] REC row if an existing word was gated\n"
"- [ ] catalog + scanner parity green\n"
"- [ ] coverage gate green (pattern exercised)\n"
"- [ ] snippet publication approved by the user\n"
)
def cmd_precheck(args: argparse.Namespace) -> int:
snippet = read_text(Path(args.snippet_file))
findings = scanner_findings(snippet)
if findings:
print(json.dumps({"status": "already_covered", "findings": findings}, indent=2, sort_keys=True))
return 3
print(json.dumps({"status": "clean", "words": word_count(snippet)}, indent=2, sort_keys=True))
return 0
def cmd_scaffold(args: argparse.Namespace) -> int:
if not SLUG_RE.match(args.pattern_name):
print(
json.dumps(
{
"error": f"invalid pattern name: {args.pattern_name!r}; use lowercase letters, digits, - or _",
}
)
)
return 2
source = Path(args.snippet)
snippet = read_text(source)
if args.tell not in snippet:
print(json.dumps({"error": "tell substring not found", "tell": args.tell}))
return 2
try:
manifest_date = datetime.strptime(args.date, "%Y-%m-%d").date().isoformat()
except ValueError:
print(json.dumps({"error": "--date must use YYYY-MM-DD", "date": args.date}))
return 2
redactions = redaction_pairs(args.redact or [])
redacted = apply_redactions(snippet, args.tell, redactions)
bundle = CONTRIB_ROOT / args.pattern_name
bundle.mkdir(parents=True, exist_ok=True)
manifest = {
"category": args.category,
"date": manifest_date,
"pattern_name": args.pattern_name,
"pre_redaction_sha256": sha256_text(snippet),
"post_redaction_sha256": sha256_text(redacted),
"redactions": [{"from": orig, "to": repl} for orig, repl in redactions],
"tell": args.tell,
"word_count": word_count(redacted),
}
write_json(bundle / "row_fn.json", row_fn(args.pattern_name, args.tell, redacted))
write_json(bundle / "row_fp_TEMPLATE.json", row_fp_template(args.pattern_name, args.category, args.tell))
write_json(bundle / "manifest.json", manifest)
(bundle / "snippet.txt").write_text(redacted, encoding="utf-8")
(bundle / "report.md").write_text(render_report(manifest, redacted), encoding="utf-8")
print(json.dumps({"bundle": str(bundle.relative_to(ROOT)), "status": "scaffolded"}, indent=2, sort_keys=True))
return 0
def run_row(row: dict[str, object]) -> subprocess.CompletedProcess[str]:
return subprocess.run(
["python3", "scripts/banned_phrase_scan.py"],
input=str(row.get("stdin", "")),
capture_output=True,
text=True,
cwd=ROOT,
timeout=30,
)
def row_assertions_pass(row: dict[str, object], proc: subprocess.CompletedProcess[str]) -> bool:
try:
data = json.loads(proc.stdout)
except json.JSONDecodeError:
return False
for assertion in row.get("assertions", []):
if assertion.get("type") == "json":
cur = data
for part in assertion["path"].split("."):
cur = cur[int(part)] if isinstance(cur, list) else cur[part]
if "gte" in assertion and not cur >= assertion["gte"]:
return False
if "equals" in assertion and cur != assertion["equals"]:
return False
elif assertion.get("type") == "violation_phrase_contains":
phrases = [v.get("phrase", "") for v in data.get("violations", [])]
needle = str(assertion["value"]).casefold()
if not any(needle in str(phrase).casefold() for phrase in phrases):
return False
else:
return False
return True
def tail(text: str, lines: int = 12) -> str:
split = text.splitlines()
return "\n".join(split[-lines:])
def verify_bundle(bundle: Path, run_gates: bool = True) -> tuple[bool, str]:
if not bundle.exists():
return False, "missing bundle"
row_path = bundle / "row_fn.json"
report_path = bundle / "report.md"
manifest_path = bundle / "manifest.json"
snippet_path = bundle / "snippet.txt"
for path in (row_path, report_path, manifest_path, snippet_path):
if not path.exists():
return False, f"missing {path.name}"
report = report_path.read_text(encoding="utf-8")
if TODO_MARKER in report:
return False, "report contains TODO markers"
row = json.loads(row_path.read_text(encoding="utf-8"))
snippet = snippet_path.read_text(encoding="utf-8")
if row.get("stdin") != snippet:
return False, "row_fn stdin does not equal snippet.txt byte-for-byte"
proc = run_row(row)
assertion_ok = row_assertions_pass(row, proc)
red_first = not assertion_ok
gate_results = [f"red-first: {'ok' if red_first else 'already green'}"]
ok = red_first
if not red_first:
ok = True
gate_results[0] = "red-first: already green; proposed pattern appears active"
if run_gates:
for command in GATE_COMMANDS:
proc = subprocess.run(command, capture_output=True, text=True, cwd=ROOT, timeout=120)
gate_results.append(
"### "
+ " ".join(command)
+ f"\n\n```text\nexit={proc.returncode}\n{tail(proc.stdout or proc.stderr)}\n```"
)
ok = ok and proc.returncode == 0
final = "\n\n".join(gate_results)
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
report_path.write_text(render_report(manifest, snippet, final, (bundle / "row_rec.json").exists()), encoding="utf-8")
return ok, final
def cmd_verify(args: argparse.Namespace) -> int:
ok, message = verify_bundle(Path(args.bundle), run_gates=not args.no_gates)
print(message)
return 0 if ok else 1
def cmd_report(args: argparse.Namespace) -> int:
bundle = Path(args.bundle)
if not bundle.exists():
print(json.dumps({"error": "missing bundle", "bundle": str(bundle)}))
return 2
manifest = json.loads((bundle / "manifest.json").read_text(encoding="utf-8"))
snippet = (bundle / "snippet.txt").read_text(encoding="utf-8")
report = (bundle / "report.md").read_text(encoding="utf-8")
gate_marker = "## Gate results\n\n"
checklist_marker = "\n\n## Checklist"
gate_results = ""
if gate_marker in report and checklist_marker in report:
gate_results = report.split(gate_marker, 1)[1].split(checklist_marker, 1)[0]
print(render_report(manifest, snippet, gate_results, (bundle / "row_rec.json").exists()).rstrip())
return 0
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest="command", required=True)
precheck = sub.add_parser("precheck")
precheck.add_argument("snippet_file")
precheck.set_defaults(func=cmd_precheck)
scaffold = sub.add_parser("scaffold")
scaffold.add_argument("--snippet", required=True)
scaffold.add_argument("--tell", required=True)
scaffold.add_argument("--category", required=True)
scaffold.add_argument("--pattern-name", required=True)
scaffold.add_argument("--redact", action="append")
scaffold.add_argument("--date", default=date.today().isoformat())
scaffold.set_defaults(func=cmd_scaffold)
verify = sub.add_parser("verify")
verify.add_argument("--bundle", required=True)
verify.add_argument("--no-gates", action="store_true")
verify.set_defaults(func=cmd_verify)
report = sub.add_parser("report")
report.add_argument("--bundle", required=True)
report.set_defaults(func=cmd_report)
return parser.parse_args(argv)
def main(argv: list[str]) -> int:
args = parse_args(argv)
return args.func(args)
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+139
View File
@@ -0,0 +1,139 @@
#!/usr/bin/env python3
"""
Check change percentage between original and transformed text.
Flags if >40% of words changed (may indicate over-editing).
Pure reordering is exempt: change_percentage still reflects the raw
delete+insert diff, but excessive_change stays false when the transformed
text is a word-level permutation of the original (moved sentences are not
over-editing). SKILL.md's 40% guidance refers to the excessive_change flag.
Usage:
python diff_check.py original.txt transformed.txt
"""
from __future__ import annotations
import sys
import json
import re
from collections import Counter
from difflib import SequenceMatcher
from typing import TypedDict
class DiffResult(TypedDict):
original_word_count: int
transformed_word_count: int
similarity_ratio: float
change_percentage: float
words_added: int
words_removed: int
words_changed: int
excessive_change: bool
flags: list[str]
def split_words(text: str) -> list[str]:
"""Split text into comparison tokens.
Punctuation is tokenized separately (not stripped) so that structural edits
like repunctuation register as change instead of reading as identical.
"""
return re.findall(r'\w+|[^\w\s]', text.lower())
def calculate_diff(original: str, transformed: str) -> DiffResult:
"""Calculate difference metrics between two texts."""
original_words = split_words(original)
transformed_words = split_words(transformed)
# Use SequenceMatcher for word-level diff
matcher = SequenceMatcher(None, original_words, transformed_words)
similarity = matcher.ratio()
# Count operations
words_added = 0
words_removed = 0
words_changed = 0
for tag, i1, i2, j1, j2 in matcher.get_opcodes():
if tag == 'replace':
words_changed += max(i2 - i1, j2 - j1)
elif tag == 'delete':
words_removed += i2 - i1
elif tag == 'insert':
words_added += j2 - j1
# Change percentage (relative to original)
total_changes = words_added + words_removed + words_changed
change_percentage = (
(total_changes / len(original_words) * 100)
if original_words else 0
)
shared_tokens = sum((Counter(original_words) & Counter(transformed_words)).values())
token_overlap = shared_tokens / len(original_words) if original_words else 0
# Flags
flags: list[str] = []
excessive = False
if change_percentage > 40:
if token_overlap >= 0.9:
flags.append(f"Mostly reordered ({token_overlap:.0%} tokens shared)")
else:
flags.append(f"Excessive change ({change_percentage:.1f}% > 40% threshold)")
excessive = True
if len(transformed_words) < len(original_words) * 0.3:
flags.append("Transformed text is less than 30% of original length")
excessive = True
if len(transformed_words) > len(original_words) * 1.5:
flags.append("Transformed text is 50%+ longer than original")
# Length change ratio
length_ratio = len(transformed_words) / len(original_words) if original_words else 0
if length_ratio < 0.5:
flags.append(f"Significant condensation ({length_ratio:.0%} of original)")
elif length_ratio > 1.2:
flags.append(f"Text expanded ({length_ratio:.0%} of original)")
return {
"original_word_count": len(original_words),
"transformed_word_count": len(transformed_words),
"similarity_ratio": round(similarity, 3),
"change_percentage": round(change_percentage, 1),
"words_added": words_added,
"words_removed": words_removed,
"words_changed": words_changed,
"excessive_change": excessive,
"flags": flags
}
def main() -> None:
if len(sys.argv) < 3:
print("Usage: diff_check.py <original.txt> <transformed.txt>")
sys.exit(1)
# Read inputs
try:
with open(sys.argv[1], 'r') as f:
original = f.read()
with open(sys.argv[2], 'r') as f:
transformed = f.read()
except OSError as e:
print(json.dumps({"error": f"Could not read input: {e}"}))
sys.exit(2)
result = calculate_diff(original, transformed)
print(json.dumps(result, indent=2))
# Exit with 1 if excessive change
sys.exit(1 if result["excessive_change"] else 0)
if __name__ == "__main__":
main()
+201
View File
@@ -0,0 +1,201 @@
#!/usr/bin/env python3
"""
Extract must-preserve constraints from input text.
Identifies facts, names, dates, URLs, numbers that must survive transformation.
Outputs JSON with constraint spans for validation.
Usage:
python extract_constraints.py < input.txt
python extract_constraints.py input.txt
echo "Text here" | python extract_constraints.py
"""
from __future__ import annotations
import argparse
import sys
import re
import json
from typing import TypedDict
class Constraint(TypedDict):
type: str
value: str
start: int
end: int
PATTERNS: dict[str, str] = {
# Currency with amounts
"currency": r"\$[\d,]+\.?\d*[KMBkmb]?(?:\s*(?:million|billion|thousand))?",
# Percentages
"percentage": r"\d+\.?\d*%",
# ISO dates
"date_iso": r"\d{4}-\d{2}-\d{2}",
# Quarter dates
"date_quarter": r"Q[1-4]\s+\d{4}",
# Natural dates
"date_natural": r"(?:January|February|March|April|May|June|July|August|September|October|November|December|Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[a-z]*\.?\s+\d{1,2}(?:st|nd|rd|th)?,?\s+\d{4}",
# Years (standalone, 1900-2099)
"year": r"\b(?:19|20)\d{2}\b",
# Times
"time": r"\d{1,2}:\d{2}(?::\d{2})?\s*(?:AM|PM|am|pm|UTC|PST|EST|CST|MST|GMT)?",
# Numeric quantities with magnitude words
"magnitude_number": r"\b\d[\d,]*\.?\d*\s+(?:thousand|million|billion|trillion)\b",
# Measurements with units
"measurement": r"\d+\.?\d*\s*(?:°C|°F|degrees?\s*(?:C|F|Celsius|Fahrenheit)?|ms|s|sec|min|hr|hour|day|week|month|year|KB|MB|GB|TB|PB|kg|g|lb|oz|m|km|mi|ft|in|cm|mm|px|em|rem|%)\b",
# Phone numbers (capture the whole number, before "range" can grab a slice)
"phone": r"\b(?:\+?1[-.\s]?)?(?:\(\d{3}\)\s*|\d{3}[-.\s])\d{3}[-.\s]\d{4}\b",
# Ranges (numeric)
"range": r"\d+\.?\d*\s*[-–]\s*\d+\.?\d*(?:\s*(?:K|M|B|%|years?|months?|days?))?",
# URLs
"url": r"https?://[^\s\)\]\>\"\']+",
# Email addresses
"email": r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}",
# Code references (backticked)
"code": r"`[^`]+`",
# Direct quotes (longer than 10 chars)
"quote": r'["“][^"“”]{10,}["”]',
# Cited references (Section 12(b), Article 5, Figure 2, Table 1, Eq. 3 …) —
# must-preserve per fact-preservation.md and easy to silently drop.
"reference": r"\b(?:Sections?|Sec\.|§|Articles?|Clauses?|Paragraphs?|Para\.|Figures?|Fig\.|Tables?|Appendix|Appendices|Schedule|Exhibit|Equations?|Eq\.|Chapters?|Rules?|Items?)\s+\d+[A-Za-z]?(?:\([a-z0-9]+\))?(?:[.\-]\d+)*",
# Version numbers
"version": r"v?\d+\.\d+(?:\.\d+)?(?:-[a-zA-Z0-9]+)?",
# API endpoints
"api_endpoint": r"(?<![\w])/(?:api|v\d+)(?:/[\w-]+)+|(?<![\w])/[\w-]+(?:/[\w-]+){2,}",
# Inclusive disjunction — collapsing and/or to a plain conjunction changes scope
"and_or": r"\band/or\b",
# Numeric counts with context
"count": r"\b\d+(?:,\d{3})*\s+(?:users?|customers?|employees?|companies?|teams?|people|engineers?|developers?|items?|products?|orders?|transactions?|requests?|queries?|rows?|records?)\b",
}
# Proper noun patterns (simplified - real implementation would use NER)
PROPER_NOUN_INDICATORS = [
r"\b[A-Z][a-z]+(?:\s+[A-Z][a-z]+)+\b", # Multi-word capitalized (John Smith)
r"\b[A-Z][a-z]+\s+(?:Inc|Corp|LLC|Ltd|Co)\b\.?", # Company suffixes
r"\b(?:Dr|Mr|Ms|Mrs|Prof)\.?\s+[A-Z][a-z]+\b", # Titles with names
]
def extract_constraints(text: str) -> list[Constraint]:
"""Extract all must-preserve constraints from text."""
constraints: list[Constraint] = []
seen_spans: set[tuple[int, int]] = set()
# Extract pattern-based constraints
for constraint_type, pattern in PATTERNS.items():
for match in re.finditer(pattern, text, re.IGNORECASE if constraint_type.startswith("date") else 0):
span = (match.start(), match.end())
if span not in seen_spans:
seen_spans.add(span)
constraints.append({
"type": constraint_type,
"value": match.group(),
"start": match.start(),
"end": match.end()
})
# Extract proper nouns (simplified)
for pattern in PROPER_NOUN_INDICATORS:
for match in re.finditer(pattern, text):
span = (match.start(), match.end())
# Don't add if overlapping with existing constraint
overlaps = any(
not (span[1] <= existing[0] or span[0] >= existing[1])
for existing in seen_spans
)
if not overlaps:
seen_spans.add(span)
constraints.append({
"type": "proper_noun",
"value": match.group(),
"start": match.start(),
"end": match.end()
})
# Bare quantities: comma-grouped numbers or integers of 4+ digits. These are
# real facts (50000 fans, 2,500 signups) that the unit/count patterns miss.
# Skip any that overlap a constraint already claimed (years, currency, phones)
# so we don't double-count or shred a captured value.
number_pattern = r"(?<![\d.,])(?:\d{1,3}(?:,\d{3})+|\d{4,})(?![\d.,])"
for match in re.finditer(number_pattern, text):
span = (match.start(), match.end())
overlaps = any(
not (span[1] <= existing[0] or span[0] >= existing[1])
for existing in seen_spans
)
if not overlaps:
seen_spans.add(span)
constraints.append({
"type": "number",
"value": match.group(),
"start": match.start(),
"end": match.end()
})
# Sort by position
constraints.sort(key=lambda c: c["start"])
return constraints
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Extract must-preserve constraints from input text."
)
parser.add_argument("path", nargs="?", help="Path to input text file (default: read stdin)")
return parser.parse_args(argv)
def main() -> None:
args = parse_args(sys.argv[1:])
# Read input
if args.path:
try:
with open(args.path, 'r', errors="replace") as f:
text = f.read()
except OSError as e:
print(json.dumps({"error": f"Could not read input: {e}", "constraints": []}))
sys.exit(2)
else:
text = sys.stdin.buffer.read().decode("utf-8", errors="replace")
if not text.strip():
print(json.dumps({"error": "No input provided", "constraints": []}))
sys.exit(1)
constraints = extract_constraints(text)
output = {
"input_length": len(text),
"constraint_count": len(constraints),
"constraints": constraints
}
print(json.dumps(output, indent=2))
if __name__ == "__main__":
main()
@@ -0,0 +1,223 @@
#!/usr/bin/env python3
"""Classify harvested candidates into situation/register coverage cells."""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
from typing import Any
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
from harvest_samples import DATE_FLOOR, recency_value # noqa: E402
CELLS = [
"numbers_data",
"question_addressed",
"anecdote_markers",
"disagreement",
"openings_closings",
]
def load_candidates(path: Path) -> list[dict[str, Any]]:
data = json.loads(path.read_text())
if isinstance(data, dict):
return data.get("candidates", [])
if isinstance(data, list):
return data
raise ValueError("candidates file must contain a list or {candidates: [...]}")
def cells_for(text: str) -> list[str]:
lowered = text.lower()
cells = []
if re.search(r"\b\d+(?:[,.]\d+)?%?\b", lowered):
cells.append("numbers_data")
if "?" in text or re.search(r"\b(?:why|how|what|when|where|which)\b", lowered):
cells.append("question_addressed")
if re.search(r"\b(?:last quarter|yesterday|once|during|when we|i noticed|i remember)\b", lowered):
cells.append("anecdote_markers")
if re.search(r"\b(?:disagree|however|instead|not convinced|push back)\b", lowered):
cells.append("disagreement")
if re.search(r"\b(?:hi|thanks|best|regards|closing|opening|first off)\b", lowered):
cells.append("openings_closings")
return cells
def quality_for(candidate: dict[str, Any], cells: list[str]) -> int:
words = int(candidate.get("words") or len(re.findall(r"\w+", candidate.get("text", ""))))
score = 3
if 40 <= words <= 220:
score += 1
if cells:
score += 1
if candidate.get("suspect_ai"):
score -= 2
if candidate.get("dictated"):
score -= 1
return max(1, min(5, score))
def candidate_id(candidate: dict[str, Any], index: int) -> Any:
return candidate.get("id", index)
def source_position(candidate: dict[str, Any]) -> str:
source = candidate.get("source", {})
return str(source.get("line", source.get("offset", "")))
def coverage_from(candidates: list[dict[str, Any]]) -> dict[str, int]:
coverage = {cell: 0 for cell in CELLS}
for candidate in candidates:
for cell in candidate.get("cells", []):
coverage[cell] = coverage.get(cell, 0) + 1
return coverage
def rank_enriched(candidates: list[dict[str, Any]]) -> list[int]:
seen_empty = set()
rank_rows = []
for idx, candidate in enumerate(candidates):
cells = candidate.get("cells", [])
fills_empty = any(cell not in seen_empty for cell in cells)
seen_empty.update(cells)
rank_rows.append((idx, fills_empty))
ranked = sorted(
rank_rows,
key=lambda row: (
not row[1],
-int(candidates[row[0]].get("quality") or 0),
-recency_value(candidates[row[0]]),
bool(candidates[row[0]].get("suspect_ai")),
bool(candidates[row[0]].get("dictated")),
str(candidates[row[0]].get("source", {}).get("path", "")),
source_position(candidates[row[0]]),
row[0],
),
)
return [candidate_id(candidates[idx], idx) for idx, _ in ranked]
def heuristic(candidates: list[dict[str, Any]]) -> dict[str, Any]:
enriched = []
seen_empty = set()
for idx, candidate in enumerate(candidates):
cells = cells_for(candidate.get("text", ""))
quality = quality_for(candidate, cells)
fills_empty = any(cell not in seen_empty for cell in cells)
seen_empty.update(cells)
enriched.append({
"index": idx,
"id": candidate.get("id", idx),
"cells": cells,
"quality": quality,
"why": "lexical heuristic matched " + (", ".join(cells) if cells else "no named cell"),
"fills_empty_coverage_cell": fills_empty,
"source": candidate.get("source", {}),
"suspect_ai": candidate.get("suspect_ai"),
"dictated": candidate.get("dictated"),
})
return {
"coverage_matrix": coverage_from(enriched),
"candidates": enriched,
"ranking": rank_enriched(enriched),
}
def write_agent_tasks(candidates: list[dict[str, Any]], out_dir: Path) -> dict[str, Any]:
out_dir.mkdir(parents=True, exist_ok=True)
chunks = []
prompt = (
"Classify each candidate into WP10b situation/register cells. "
"Return JSONL rows with candidate_index, cells, quality 1-5, and one-line why. "
"Cells include numbers_data, question_addressed, anecdote_markers, "
"disagreement, openings_closings, plus any clearly justified additional cell."
)
for start in range(0, len(candidates), 10):
chunk = candidates[start:start + 10]
path = out_dir / f"harvest-classify-{start // 10 + 1:03d}.json"
payload = {
"contract": "tier-1-pack-detector",
"prompt": prompt,
"candidates": [
{"candidate_index": start + i, "text": c.get("text", ""), "source": c.get("source", {})}
for i, c in enumerate(chunk)
],
}
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
chunks.append(str(path))
return {"task_files": chunks, "chunk_size": 10}
def result_candidate_key(row: dict[str, Any]) -> Any:
for key in ("candidate_id", "id", "candidate_index", "index"):
if key in row:
return row[key]
raise ValueError("result row missing candidate id")
def merge_results(candidates_path: Path, results_path: Path) -> dict[str, Any]:
candidates = load_candidates(candidates_path)
rows_by_id: dict[Any, dict[str, Any]] = {}
for line in results_path.read_text().splitlines():
if line.strip():
row = json.loads(line)
rows_by_id[result_candidate_key(row)] = row
merged = []
for idx, candidate in enumerate(candidates):
cid = candidate_id(candidate, idx)
row = rows_by_id.get(cid)
if row is None and idx in rows_by_id:
row = rows_by_id[idx]
cells = list(row.get("cells", [])) if row else []
quality = int(row.get("quality")) if row and row.get("quality") is not None else quality_for(candidate, cells)
why = str(row.get("why", "no classifier result")) if row else "no classifier result"
merged.append({
**candidate,
"id": cid,
"cells": cells,
"quality": max(1, min(5, quality)),
"why": why,
})
return {
"coverage_matrix": coverage_from(merged),
"candidates": merged,
"ranking": rank_enriched(merged),
}
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--candidates", required=True)
parser.add_argument("--mode", choices=["heuristic", "agent"], default="heuristic")
parser.add_argument("--out-dir", default="harvest-agent-tasks")
parser.add_argument("--merge")
return parser.parse_args(argv)
def main(argv: list[str]) -> int:
args = parse_args(argv)
if args.merge:
print(json.dumps(merge_results(Path(args.candidates), Path(args.merge)), indent=2, sort_keys=True))
return 0
candidates = load_candidates(Path(args.candidates))
if args.mode == "heuristic":
print(json.dumps(heuristic(candidates), indent=2, sort_keys=True))
else:
print(json.dumps(write_agent_tasks(candidates, Path(args.out_dir)), indent=2, sort_keys=True))
return 0
if __name__ == "__main__":
import sys
raise SystemExit(main(sys.argv[1:]))
@@ -0,0 +1,534 @@
#!/usr/bin/env python3
"""Harvest user-authored writing samples from transcripts and declared folders.
Adapters:
- claude-jsonl: JSONL transcript files with explicit user/assistant roles. Unknown
schemas are skipped with a warning; authorship is never guessed.
- codex-jsonl: Codex CLI/Desktop session JSONL (~/.codex/sessions/YYYY/MM/DD/
rollout-*.jsonl). Reads `event_msg` user_message text and user-role
`response_item` message content; drops assistant/developer/tool rows and
filters instruction-injection wrappers (AGENTS.md dumps, <environment_context>,
and similar structural markers) that Codex injects into user-role turns.
- text-folder: directories containing .md/.txt files. These are user-authored by
declaration, but still pass through the AI-contamination tripwire.
Adapter detection for .jsonl files is by content shape, not filename: each file
is peeked line-by-line until a recognizable claude-jsonl or codex-jsonl entry is
found. Files that never match either shape fall through to the claude-jsonl
parser's own "unknown schema" warning path.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from datetime import datetime
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT / "scripts"))
import banned_phrase_scan # noqa: E402
import structure_scan # noqa: E402
WORD_RE = re.compile(r"[A-Za-z0-9]+(?:[-'][A-Za-z0-9]+)?")
SENTENCE_RE = re.compile(r"[.!?](?:\s|$)")
COMMAND_RE = re.compile(
r"^\s*(?:/[\w-]+|(?:open|read|write|edit|fix|review|run|grep|search|cat|"
r"sed|python3?|npm|git)\b.*(?:/[\w./-]+|--?\w+))",
re.I,
)
TAG_RE = re.compile(r"<(?:system-reminder|[^>\s]+)[^>]*>[\s\S]*?</(?:system-reminder|[^>\s]+)>", re.I)
QUOTE_ASSISTANT_RE = re.compile(r"(?im)^\s*(?:>|you said:|assistant:|claude said:)")
FILLER_RE = re.compile(r"\b(?:um|uh)\b,?", re.I)
DATE_FLOOR = 0.0
def words(text: str) -> list[str]:
return WORD_RE.findall(text)
def normalize_text(text: str) -> str:
return re.sub(r"\s+", " ", text).strip()
def normal_tokens(text: str) -> list[str]:
return [w.lower() for w in words(text)]
def fivegrams(text: str) -> set[tuple[str, ...]]:
toks = normal_tokens(text)
return set(tuple(toks[i:i + 5]) for i in range(max(0, len(toks) - 4)))
def complete_sentence_count(text: str) -> int:
return len(SENTENCE_RE.findall(text))
def extract_text(content: Any) -> str:
if isinstance(content, str):
return content
if isinstance(content, list):
parts = []
for item in content:
if isinstance(item, str):
parts.append(item)
elif isinstance(item, dict) and isinstance(item.get("text"), str):
parts.append(item["text"])
return "\n".join(parts)
return ""
def role_from_entry(entry: dict[str, Any]) -> str | None:
msg = entry.get("message")
role = msg.get("role") if isinstance(msg, dict) else None
if role in {"user", "assistant"}:
return role
top = entry.get("type")
if top in {"user", "assistant"}:
return top
return None
def message_text(entry: dict[str, Any]) -> str:
msg = entry.get("message")
if isinstance(msg, dict):
return extract_text(msg.get("content"))
return extract_text(entry.get("content"))
# Codex CLI/Desktop session envelopes: {"timestamp", "type", "payload"}. These are
# the top-level `type` values actually observed (and defensively allowed) in
# ~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl. `user_message`/`agent_message` are
# not top-level types themselves -- they are `payload.type` values nested inside
# an `event_msg` envelope.
CODEX_TOP_TYPES = {"session_meta", "event_msg", "response_item", "turn_context", "compacted"}
# Structural markers Codex injects into user-role turns that are not the user's
# own writing: a repo's AGENTS.md dumped verbatim, the environment/skill/turn
# banners, or an explicit user-instructions wrapper. Filtering is by these
# prefixes (structure), never by guessing at content.
CODEX_INJECTION_PREFIXES = (
"# AGENTS.md instructions for",
"<environment_context>",
"<user_instructions>",
"<INSTRUCTIONS>",
"<skill>",
"<turn_aborted>",
)
def is_codex_envelope(entry: dict[str, Any]) -> bool:
return entry.get("type") in CODEX_TOP_TYPES and isinstance(entry.get("payload"), dict)
def is_injection_wrapper(text: str) -> bool:
return text.strip().startswith(CODEX_INJECTION_PREFIXES)
def detect_jsonl_adapter(path: Path) -> str:
"""Peek at a JSONL file's shape to pick claude-jsonl or codex-jsonl parsing.
Falls back to claude-jsonl (which itself warns and skips truly unknown
schemas) when neither shape is recognized in any line.
"""
for line in path.read_text(errors="replace").splitlines():
if not line.strip():
continue
try:
entry = json.loads(line)
except json.JSONDecodeError:
continue
if not isinstance(entry, dict):
continue
if is_codex_envelope(entry):
return "codex-jsonl"
if role_from_entry(entry) is not None:
return "claude-jsonl"
return "claude-jsonl"
def strip_transcript_noise(text: str) -> str:
text = TAG_RE.sub(" ", text)
lines = []
for line in text.splitlines():
if re.match(r"^\s*>", line):
continue
lines.append(line)
return normalize_text("\n".join(lines))
def is_quoted_assistant(text: str) -> bool:
if QUOTE_ASSISTANT_RE.search(text):
return True
lowered = text.lower().strip()
return lowered.startswith(("you wrote:", "your answer:", "your response:"))
def is_command_like(text: str) -> bool:
stripped = text.strip()
if COMMAND_RE.search(stripped):
return True
if len(words(stripped)) <= 8 and re.search(r"(^|\s)(?:/[\w./-]+|--?\w+)", stripped):
return True
return False
def dictated(text: str) -> bool:
w = max(1, len(words(text)))
fillers = FILLER_RE.findall(text)
return len(fillers) >= 2 and len(fillers) / w > 0.035
def tripwire(text: str) -> bool:
banned = banned_phrase_scan.scan_for_violations(text, include_quoted=True)
structure = structure_scan.scan(text)
categories = {v["category"] for v in banned}
categories.update(f"struct:{f['metric']}" for f in structure["flags"])
hard = any(v["severity"] == "hard" for v in banned)
return hard or len(categories) >= 2
def source_date(entry: dict[str, Any]) -> str | None:
raw = entry.get("timestamp") or entry.get("created_at")
if not isinstance(raw, str):
return None
return raw
def parse_since(value: str | None) -> datetime | None:
if not value:
return None
return datetime.fromisoformat(value)
def date_ok(date: str | None, since: datetime | None) -> bool:
if not since or not date:
return True
try:
return datetime.fromisoformat(date.replace("Z", "+00:00")).replace(tzinfo=None) >= since
except ValueError:
return True
def iter_claude_jsonl(path: Path, warnings: list[str]) -> tuple[list[dict[str, Any]], dict[str, int]]:
candidates = []
stats = {"authorship": 0}
saw_known_schema = False
for idx, line in enumerate(path.read_text(errors="replace").splitlines(), start=1):
if not line.strip():
continue
try:
entry = json.loads(line)
except json.JSONDecodeError:
warnings.append(f"{path}: line {idx}: invalid JSON; skipping file")
return [], stats
if not isinstance(entry, dict):
continue
role = role_from_entry(entry)
if role is None:
continue
saw_known_schema = True
if role != "user":
stats["authorship"] += 1
continue
text = message_text(entry)
candidates.append({
"text": text,
"source": {
"path": str(path),
"line": idx,
"message_index": idx,
"date": source_date(entry),
"adapter": "claude-jsonl",
},
})
if not saw_known_schema:
warnings.append(f"{path}: unknown jsonl schema; skipped")
return [], stats
return candidates, stats
def codex_message_texts(payload: dict[str, Any]) -> list[str]:
"""Text of a response_item message's content parts, keyed on the caller
already having confirmed role=="user" -- the role check, not the content
item's own `type` (input_text/output_text/...), is what gates authorship."""
content = payload.get("content")
if not isinstance(content, list):
return []
return [
item["text"]
for item in content
if isinstance(item, dict) and isinstance(item.get("text"), str)
]
def iter_codex_jsonl(path: Path, warnings: list[str]) -> tuple[list[dict[str, Any]], dict[str, int]]:
candidates = []
stats = {"authorship": 0, "instruction-injection": 0}
saw_known_schema = False
for idx, line in enumerate(path.read_text(errors="replace").splitlines(), start=1):
if not line.strip():
continue
try:
entry = json.loads(line)
except json.JSONDecodeError:
warnings.append(f"{path}: line {idx}: invalid JSON; skipping file")
return [], stats
if not isinstance(entry, dict):
continue
if not is_codex_envelope(entry):
continue
saw_known_schema = True
payload = entry["payload"]
payload_type = payload.get("type")
date = source_date(entry)
# event_msg envelopes: only user_message/agent_message carry authored text.
if entry.get("type") == "event_msg":
if payload_type == "agent_message":
stats["authorship"] += 1
elif payload_type == "user_message":
message = payload.get("message")
if isinstance(message, str):
if is_injection_wrapper(message):
stats["instruction-injection"] += 1
else:
candidates.append({
"text": message,
"source": {
"path": str(path),
"line": idx,
"message_index": idx,
"date": date,
"adapter": "codex-jsonl",
},
})
# task_started/task_complete/token_count/exec_command_end/etc: not
# authored text at all -- skip without guessing.
continue
# response_item envelopes: only role=="user" messages are candidates;
# reasoning/function_call/function_call_output/custom_tool_call* are
# tool plumbing, never authored text.
if entry.get("type") == "response_item" and payload_type == "message":
role = payload.get("role")
if role != "user":
stats["authorship"] += 1
continue
texts = codex_message_texts(payload)
if not texts:
continue
if any(is_injection_wrapper(t) for t in texts):
stats["instruction-injection"] += 1
continue
candidates.append({
"text": "\n".join(texts),
"source": {
"path": str(path),
"line": idx,
"message_index": idx,
"date": date,
"adapter": "codex-jsonl",
},
})
continue
# session_meta (holds base_instructions), turn_context, compacted, and
# any other response_item payload type: never harvest, never guess.
if not saw_known_schema:
warnings.append(f"{path}: unknown jsonl schema; skipped")
return [], stats
return candidates, stats
def iter_text_file(path: Path) -> list[dict[str, Any]]:
return [{
"text": path.read_text(errors="replace"),
"source": {
"path": str(path),
"offset": 0,
"adapter": "text-folder",
"mtime": path.stat().st_mtime,
},
}]
def collect_sources(paths: list[Path], warnings: list[str]) -> tuple[list[dict[str, Any]], dict[str, int], bool]:
raw = []
stats = {"authorship": 0}
missing = False
for source in sorted(paths, key=lambda p: str(p)):
if not source.exists():
print(f"missing source: {source}", file=sys.stderr)
missing = True
continue
files: list[Path]
if source.is_dir():
files = sorted(
[p for p in source.rglob("*") if p.suffix.lower() in {".jsonl", ".md", ".txt"}],
key=lambda p: str(p),
)
else:
files = [source]
for file in files:
try:
if file.suffix.lower() == ".jsonl":
adapter = detect_jsonl_adapter(file)
if adapter == "codex-jsonl":
items, sub = iter_codex_jsonl(file, warnings)
else:
items, sub = iter_claude_jsonl(file, warnings)
raw.extend(items)
for key, value in sub.items():
stats[key] = stats.get(key, 0) + value
elif file.suffix.lower() in {".md", ".txt"}:
raw.extend(iter_text_file(file))
except (OSError, UnicodeDecodeError) as e:
warnings.append(f"{file}: unreadable ({e}); skipping")
stats["unreadable"] = stats.get("unreadable", 0) + 1
return raw, stats, missing
def apply_filters(raw: list[dict[str, Any]], min_words: int, since: datetime | None) -> tuple[list[dict[str, Any]], dict[str, int]]:
stats = {
"authorship": 0,
"length": 0,
"fragment-share": 0,
"command-likeness": 0,
"duplication": 0,
"quoted-assistant": 0,
"since": 0,
}
kept = []
previous: list[set[tuple[str, ...]]] = []
for item in sorted(raw, key=lambda c: (c["source"]["path"], c["source"].get("line", c["source"].get("offset", 0)))):
if not date_ok(item["source"].get("date"), since):
stats["since"] += 1
continue
text = strip_transcript_noise(item["text"])
if is_quoted_assistant(text):
stats["quoted-assistant"] += 1
continue
if is_command_like(text):
stats["command-likeness"] += 1
continue
count = len(words(text))
if count < min_words:
stats["length"] += 1
continue
if complete_sentence_count(text) < 2:
stats["fragment-share"] += 1
continue
grams = fivegrams(text)
if grams:
dupe = False
for old in previous:
overlap = len(grams & old) / max(1, min(len(grams), len(old)))
if overlap > 0.6:
dupe = True
break
if dupe:
stats["duplication"] += 1
continue
previous.append(grams)
candidate = {
"text": text,
"source": item["source"],
"words": count,
"dictated": False,
}
if dictated(text):
candidate["dictated"] = True
if tripwire(text):
candidate["suspect_ai"] = True
kept.append(candidate)
return kept, stats
def recency_value(candidate: dict[str, Any]) -> float:
source = candidate.get("source", {})
raw_date = source.get("date")
if isinstance(raw_date, str):
try:
return datetime.fromisoformat(raw_date.replace("Z", "+00:00")).timestamp()
except ValueError:
pass
raw_mtime = source.get("mtime")
if isinstance(raw_mtime, int | float):
return float(raw_mtime)
path = source.get("path")
if isinstance(path, str):
try:
return Path(path).stat().st_mtime
except OSError:
pass
return DATE_FLOOR
def rank_candidates(candidates: list[dict[str, Any]], max_candidates: int) -> list[dict[str, Any]]:
ranked = sorted(
candidates,
key=lambda c: (
bool(c.get("suspect_ai")),
-recency_value(c),
c["source"]["path"],
c["source"].get("line", c["source"].get("offset", 0)),
),
)
return ranked[:max_candidates]
def harvest(args: argparse.Namespace) -> tuple[dict[str, Any], int]:
warnings: list[str] = []
raw, auth_stats, missing = collect_sources([Path(s) for s in args.sources], warnings)
candidates, stats = apply_filters(raw, args.min_words, parse_since(args.since))
for key, value in auth_stats.items():
stats[key] = stats.get(key, 0) + value
output = {
"candidates": rank_candidates(candidates, args.max_candidates),
"drop_stats": {k: v for k, v in stats.items() if v},
"warnings": warnings,
}
if args.self_check_determinism:
output["deterministic"] = json.dumps(output, sort_keys=True) == json.dumps(output, sort_keys=True)
for warning in warnings:
print(f"warning: {warning}", file=sys.stderr)
return output, 2 if missing else 0
def write_output(output: dict[str, Any], path: str) -> None:
text = json.dumps(output, indent=2, sort_keys=True) + "\n"
if path == "-":
print(text, end="")
return
target = Path(path)
target.write_text(text)
stats_path = target.with_name(target.stem + ".drop_stats.json")
stats_path.write_text(json.dumps(output["drop_stats"], indent=2, sort_keys=True) + "\n")
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("sources", nargs="+", metavar="SOURCE")
parser.add_argument("-o", "--output", required=True)
parser.add_argument("--min-words", type=int, default=40)
parser.add_argument("--max-candidates", type=int, default=200)
parser.add_argument("--since")
parser.add_argument("--self-check-determinism", action="store_true", help=argparse.SUPPRESS)
return parser.parse_args(argv)
def main(argv: list[str]) -> int:
args = parse_args(argv)
output, code = harvest(args)
write_output(output, args.output)
return code
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
+295
View File
@@ -0,0 +1,295 @@
#!/usr/bin/env python3
"""
Calculate readability metrics for transformed text.
Outputs:
- Flesch-Kincaid grade level
- Sentence length variance
- Word repetition score
- Paragraph length stats
Usage:
python readability_metrics.py < input.txt
python readability_metrics.py input.txt
"""
from __future__ import annotations
import argparse
import sys
import re
import json
from collections import Counter
from typing import TypedDict
class ReadabilityMetrics(TypedDict):
flesch_kincaid_grade: float
flesch_reading_ease: float
sentence_count: int
word_count: int
avg_sentence_length: float
sentence_length_variance: float
min_sentence_length: int
max_sentence_length: int
consecutive_similar_length: int
word_repetition_score: float
top_repeated_words: list[tuple[str, int]]
paragraph_count: int
avg_paragraph_length: float
flags: list[str]
def count_syllables(word: str) -> int:
"""Count syllables in a word (approximation)."""
word = word.lower().strip()
if not word:
return 0
# Handle special cases
if len(word) <= 3:
return 1
# Count vowel groups
vowels = "aeiouy"
count = 0
prev_is_vowel = False
for char in word:
is_vowel = char in vowels
if is_vowel and not prev_is_vowel:
count += 1
prev_is_vowel = is_vowel
# Adjust for silent e
if word.endswith('e') and count > 1:
count -= 1
# Adjust for -le endings
if word.endswith('le') and len(word) > 2 and word[-3] not in vowels:
count += 1
return max(1, count)
def split_sentences(text: str) -> list[str]:
"""Split text into sentences."""
parts = re.split(r'([.!?]["”]?)\s+(?=["“]?[A-Z])', text)
sentences = []
current = ""
for part in parts:
if re.match(r'[.!?]["”]?$', part):
current += part
sentences.append(current.strip())
current = ""
else:
current += part
if current.strip():
sentences.append(current.strip())
# Filter empty and clean up
return [s.strip() for s in sentences if s.strip()]
def split_staccato_units(text: str) -> list[str]:
units = re.split(r'\s*[—;]\s*|(?<=[.!?])\s+', text)
return [u.strip() for u in units if u.strip()]
def split_words(text: str) -> list[str]:
"""Split text into words."""
# Remove punctuation except hyphens in words. Numeric tokens are real words
# (a data table is not "empty text"), so keep them.
text = re.sub(r'[^\w\s-]', ' ', text)
words = text.lower().split()
return [w for w in words if w]
def calculate_metrics(text: str) -> ReadabilityMetrics:
"""Calculate all readability metrics."""
sentences = split_sentences(text)
words = split_words(text)
paragraphs = [p.strip() for p in text.split('\n\n') if p.strip()]
if not sentences or not words:
return {
"flesch_kincaid_grade": 0,
"flesch_reading_ease": 0,
"sentence_count": 0,
"word_count": 0,
"avg_sentence_length": 0,
"sentence_length_variance": 0,
"min_sentence_length": 0,
"max_sentence_length": 0,
"consecutive_similar_length": 0,
"word_repetition_score": 0,
"top_repeated_words": [],
"paragraph_count": len(paragraphs),
"avg_paragraph_length": 0,
"flags": ["Empty or invalid text"]
}
# Basic counts
sentence_count = len(sentences)
word_count = len(words)
syllable_count = sum(count_syllables(w) for w in words)
# Sentence lengths
sentence_lengths = [len(split_words(s)) for s in sentences]
avg_sentence_length = word_count / sentence_count if sentence_count else 0
# Variance calculation
if len(sentence_lengths) > 1:
mean = sum(sentence_lengths) / len(sentence_lengths)
variance = sum((x - mean) ** 2 for x in sentence_lengths) / len(sentence_lengths)
else:
variance = 0
# Consecutive similar length sentences
consecutive_similar = 0
max_consecutive = 0
for i in range(1, len(sentence_lengths)):
# "Similar" = within 3 words of each other
if abs(sentence_lengths[i] - sentence_lengths[i-1]) <= 3:
consecutive_similar += 1
max_consecutive = max(max_consecutive, consecutive_similar)
else:
consecutive_similar = 0
# Longest run of tiny (<=5-word) sentence-like units — the staccato "anti-slop" cadence.
# Tracked independently of sentence_count so short de-slopped outputs don't slip.
staccato_lengths = [len(split_words(s)) for s in split_staccato_units(text)]
staccato_run = 0
max_staccato_run = 0
for length in staccato_lengths:
if length <= 5:
staccato_run += 1
max_staccato_run = max(max_staccato_run, staccato_run)
else:
staccato_run = 0
# Flesch-Kincaid calculations
avg_syllables_per_word = syllable_count / word_count if word_count else 0
flesch_reading_ease = (
206.835
- (1.015 * avg_sentence_length)
- (84.6 * avg_syllables_per_word)
)
flesch_kincaid_grade = (
(0.39 * avg_sentence_length)
+ (11.8 * avg_syllables_per_word)
- 15.59
)
# Word repetition (excluding common words)
stop_words = {
'the', 'a', 'an', 'and', 'or', 'but', 'in', 'on', 'at', 'to', 'for',
'of', 'with', 'by', 'from', 'is', 'are', 'was', 'were', 'be', 'been',
'being', 'have', 'has', 'had', 'do', 'does', 'did', 'will', 'would',
'could', 'should', 'may', 'might', 'must', 'shall', 'can', 'need',
'it', 'its', 'this', 'that', 'these', 'those', 'i', 'you', 'he', 'she',
'we', 'they', 'what', 'which', 'who', 'when', 'where', 'why', 'how',
'not', 'no', 'yes', 'if', 'then', 'else', 'so', 'as', 'than', 'just'
}
content_words = [w for w in words if w not in stop_words and len(w) > 2]
word_freq = Counter(content_words)
# Repetition score: percentage of content words appearing 3+ times
repeated_words = {w: c for w, c in word_freq.items() if c >= 3}
repetition_score = (
sum(repeated_words.values()) / len(content_words) * 100
if content_words else 0
)
top_repeated = sorted(word_freq.items(), key=lambda x: x[1], reverse=True)[:10]
# Paragraph stats
para_lengths = [len(split_words(p)) for p in paragraphs]
avg_para_length = sum(para_lengths) / len(para_lengths) if para_lengths else 0
# Generate flags
flags: list[str] = []
if flesch_kincaid_grade > 12:
flags.append(f"High reading level ({flesch_kincaid_grade:.1f} grade)")
if variance < 10 and sentence_count > 3:
flags.append("Low sentence length variance (monotonous rhythm)")
if max_consecutive >= 3:
flags.append(f"{max_consecutive}+ consecutive similar-length sentences")
if repetition_score > 15:
flags.append(f"High word repetition ({repetition_score:.1f}%)")
if avg_sentence_length > 25:
flags.append(f"Long average sentence length ({avg_sentence_length:.1f} words)")
if avg_sentence_length < 8 and sentence_count > 3:
flags.append("Very short sentences (may feel choppy)")
# Staccato cadence is the loudest "anti-slop AI" signature; flag it even on
# short outputs (no sentence_count gate) so de-slopped text can't hide it.
if max_staccato_run >= 3:
flags.append(f"Staccato cadence: {max_staccato_run} consecutive tiny sentences (anti-slop AI tell)")
elif avg_sentence_length < 6 and sentence_count >= 2:
flags.append(f"Staccato cadence (avg {avg_sentence_length:.1f} words/sentence — anti-slop AI tell)")
if max(sentence_lengths) - min(sentence_lengths) < 5 and sentence_count > 5:
flags.append("Sentences all similar length (AI tell)")
return {
"flesch_kincaid_grade": round(flesch_kincaid_grade, 1),
"flesch_reading_ease": round(flesch_reading_ease, 1),
"sentence_count": sentence_count,
"word_count": word_count,
"avg_sentence_length": round(avg_sentence_length, 1),
"sentence_length_variance": round(variance, 1),
"min_sentence_length": min(sentence_lengths) if sentence_lengths else 0,
"max_sentence_length": max(sentence_lengths) if sentence_lengths else 0,
"consecutive_similar_length": max_consecutive,
"word_repetition_score": round(repetition_score, 1),
"top_repeated_words": top_repeated,
"paragraph_count": len(paragraphs),
"avg_paragraph_length": round(avg_para_length, 1),
"flags": flags
}
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Calculate readability metrics for transformed text."
)
parser.add_argument("path", nargs="?", help="Path to input text file (default: read stdin)")
return parser.parse_args(argv)
def main() -> None:
args = parse_args(sys.argv[1:])
# Read input
if args.path:
try:
with open(args.path, 'r', errors="replace") as f:
text = f.read()
except OSError as e:
print(json.dumps({"error": f"Could not read input: {e}"}))
sys.exit(2)
else:
text = sys.stdin.buffer.read().decode("utf-8", errors="replace")
if not text.strip():
print(json.dumps({"error": "No input provided"}))
sys.exit(1)
metrics = calculate_metrics(text)
print(json.dumps(metrics, indent=2))
# Exit with 1 if any flags (warnings)
sys.exit(1 if metrics["flags"] else 0)
if __name__ == "__main__":
main()
@@ -0,0 +1,154 @@
#!/usr/bin/env python3
"""
Report staleness of the three inputs the adversarial-refresh promise depends
on: the Wikipedia wiki-sync state, the model-parity bench, and the newest
adversarial-eval row. Pure stdlib, network-free, always exits 0 — this is a
reporter an agent reads before deciding what to run next
(see references/refresh.md), not a gate that blocks anything itself.
Usage:
python3 scripts/refresh_status.py
"""
from __future__ import annotations
import json
import re
import subprocess
import sys
from datetime import date, datetime, timezone
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
WIKI_STATE_FILE = ROOT / "scripts" / ".wiki_sync_state.json"
PIPELINE_DOC = ROOT / "references" / "pipeline.md"
ADVERSARIAL_EVALS = ROOT / "evals" / "adversarial-evals.json"
# Guidance thresholds, not hard limits -- the numbers below are the point at
# which an agent should proactively check, not a required cadence. Every
# field is a plain reporter value; nothing here fails the build.
#
# Wikipedia's "Signs of AI writing" page is a slow-moving reference page;
# quarterly is enough to catch drift without chasing noise.
WIKI_STALE_DAYS = 90
# The GPT/Anthropic model spectrums shift on the order of months, and the
# rule in references/pipeline.md already forces a re-run on any co-writer,
# mimic, or detector-pack change regardless of this threshold -- this is a
# backstop for the case where no such change happened but time still passed.
BENCH_STALE_DAYS = 180
# The adversarial-eval catalog is the product's growth signal (docs/PRODUCT.md,
# Growth); a two-month gap with no new row is worth a look even if nothing
# else prompted one.
ROW_STALE_DAYS = 60
# The convention this script relies on in references/pipeline.md: the
# recorded parity date is written as "Live matrix recorded **YYYY-MM-DD**."
# If that sentence's wording changes, this regex needs to change with it.
PIPELINE_DATE_RE = re.compile(
r"Live matrix recorded \*\*(\d{4}-\d{2}-\d{2})\*\*"
)
def _today() -> date:
return datetime.now(timezone.utc).date()
def _days_since(d: date | None) -> int | None:
if d is None:
return None
return (_today() - d).days
def _status(last: date | None, stale_days: int) -> dict:
days = _days_since(last)
stale = days is None or days > stale_days
return {
"last": last.isoformat() if last else None,
"days": days,
"stale": stale,
}
def _parse_iso_date(value: str) -> date | None:
"""Best-effort parse of an ISO-8601 timestamp (MediaWiki style, 'Z'
suffix included) down to a plain date."""
try:
return datetime.fromisoformat(value.replace("Z", "+00:00")).date()
except (ValueError, AttributeError):
return None
def wiki_sync_status() -> dict:
if not WIKI_STATE_FILE.exists():
# Never synced. Treat as stale so it surfaces for attention.
return _status(None, WIKI_STALE_DAYS)
last: date | None = None
try:
state = json.loads(WIKI_STATE_FILE.read_text())
last = _parse_iso_date(state.get("last_timestamp", ""))
except (json.JSONDecodeError, OSError):
last = None
if last is None:
# Content didn't parse; fall back to the state file's mtime.
try:
mtime = WIKI_STATE_FILE.stat().st_mtime
last = datetime.fromtimestamp(mtime, tz=timezone.utc).date()
except OSError:
last = None
return _status(last, WIKI_STALE_DAYS)
def parity_bench_status() -> dict:
last: date | None = None
try:
text = PIPELINE_DOC.read_text()
match = PIPELINE_DATE_RE.search(text)
if match:
last = date.fromisoformat(match.group(1))
except OSError:
last = None
return _status(last, BENCH_STALE_DAYS)
def newest_pattern_row_status() -> dict:
last: date | None = None
try:
result = subprocess.run(
[
"git",
"log",
"-1",
"--format=%cs",
"--",
str(ADVERSARIAL_EVALS),
],
cwd=str(ROOT),
capture_output=True,
text=True,
timeout=10,
)
out = result.stdout.strip()
if result.returncode == 0 and out:
last = date.fromisoformat(out)
except (OSError, subprocess.SubprocessError, ValueError):
last = None
return _status(last, ROW_STALE_DAYS)
def main() -> None:
output = {
"wiki_sync": wiki_sync_status(),
"parity_bench": parity_bench_status(),
"newest_pattern_row": newest_pattern_row_status(),
}
print(json.dumps(output, indent=2))
sys.exit(0)
if __name__ == "__main__":
main()
@@ -0,0 +1,390 @@
#!/usr/bin/env python3
"""Scan prose for discourse-level silhouette AI-writing patterns.
This is the macro-structure scanner one level above scripts/structure_scan.py.
structure_scan measures SURFACE structure (sentence cadence, opener words,
listicle formatting). silhouette_scan measures IDEA ARRANGEMENT: does the
document follow a template-shaped outline, preview-then-fulfill its own points,
open body paragraphs with rotating discourse cues, and close with a recap loop?
Five validated one-sided tells (all AI-high; humans cluster at zero):
scaffold_opener_share body paragraphs opening with a discourse-cue class
callback_content early vocab absent mid-document, returning at the end
(the recap loop; strongest single tell)
role_entropy_bits count of distinct cue-opener classes (a template rotates
"However / In addition / Ultimately" openers)
preview_fulfillment intro content words reappearing as body-paragraph heads
heading_preview heading head-nouns previewed in the intro (outline follow)
Composite:
silhouette_penalty = sum_i weight_i * relu((m_i - median_i) / scale_i)
scored against a committed HUMAN reference distribution
(evals/fixtures/silhouette/human_reference.json), negative side clipped. The
document flags when silhouette_penalty >= 1.0.
Scale note (deviation from a naive sample-IQR denominator, stated on purpose):
these five metrics are degenerate at zero across the human corpus -- the sample
IQR is 0.0 for every one, so the literal (m - median)/IQR with a 0.05 floor
reduces to m/0.05 and produces human false positives (a human doc with one
genuine callback at 0.17 lands at penalty > 1.0). The reference therefore scales
each metric by its human UPPER FENCE -- the validated activation threshold past
which a human essentially never scores. The scorer uses
denom = max(sample_iqr, fence) so a future per-author profile with a real,
non-degenerate IQR widens the scale naturally. This keeps the exact weighted
relu-of-z shape, reproduces the research's struct01/03/09/11 signature, and holds
0 / 8 human false positives on the validation corpus.
Voice fingerprint (not implemented here): the same per-metric median/scale
machinery generalizes to a PER-AUTHOR reference stored in a voice profile, so
mimic mode can penalize |draft - author-median| instead of generic-human. That
per-author integration lands with the teach/mimic branch (WP10b); this scanner
ships the generic-human reference only.
stdlib only. JSON to stdout. Exit 1 on flag, 0 clean, 2 on a missing file.
"""
from __future__ import annotations
import argparse
import json
import math
import re
import sys
from collections import Counter
from pathlib import Path
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
from structure_scan import STOPWORDS as _STRUCTURE_STOPWORDS # noqa: E402
from _lang import ( # noqa: E402
is_probably_english,
paragraphs as _prose_paragraphs,
words,
)
REFERENCE_PATH = (
Path(__file__).resolve().parent.parent
/ "evals" / "fixtures" / "silhouette" / "human_reference.json"
)
# Flag the document when the composite reaches this penalty (research-validated).
PENALTY_THRESHOLD = 1.0
# Minimum prose paragraphs before the silhouette metrics are meaningful.
MIN_PARAGRAPHS = 3
# silhouette_scan's stopword list is a pure superset of structure_scan's: same
# core function words plus pronouns/quantifiers that matter for content-bigram
# and callback-content comparisons but that structure_scan doesn't need.
SILHOUETTE_STOPWORDS = _STRUCTURE_STOPWORDS | frozenset({
"our", "your", "their", "my", "me", "us", "them", "his", "her", "what",
"which", "who", "when", "where", "how", "why", "there", "here", "about",
"just", "more", "most", "some", "all", "also", "out", "up", "one", "two",
"get", "got", "like", "much", "many", "very", "every", "only",
})
# Discourse cue classes for paragraph-opener roles. Copied verbatim from the
# validated research prototype (scratchpad/research-silhouette/silhouette_probe.py).
ROLE_CUES = {
"contrast": r"^(on the other hand|on one hand|however|conversely|in contrast|"
r"yet|but |perhaps most|that said|still,)",
"addition": r"^(moreover|furthermore|additionally|in addition|also,|"
r"another|second|third|next,|finally,|besides)",
"conclusion": r"^(in conclusion|ultimately|overall|in the end|to sum|"
r"in summary|as we|remember|the future)",
"enumeration": r"^(first,|firstly|1\.|step \d|there are)",
"cause": r"^(therefore|thus|consequently|as a result|because of)",
}
ROLE_RE = {k: re.compile(v, re.I) for k, v in ROLE_CUES.items()}
def content(text: str) -> list[str]:
return [w for w in words(text) if len(w) > 3 and w not in SILHOUETTE_STOPWORDS]
def paragraphs(text: str) -> list[str]:
return _prose_paragraphs(text, strip_bold=True)
# ---------------- METRICS (verbatim from the validated prototype) ----------------
def m_scaffold_opener_share(paras: list[str]):
"""Share of body paragraphs opening with a discourse-connective/scaffold cue."""
body = paras[1:] if len(paras) > 1 else paras
if not body:
return 0.0
hits = 0
for p in body:
for rx in ROLE_RE.values():
if rx.search(p):
hits += 1
break
return round(hits / len(body), 3)
def m_role_entropy(paras: list[str]):
"""Shannon entropy (bits) over opener role classes incl. 'topic' (none).
A human never opens paragraphs with cue classes -> entropy 0; a template
rotates several distinct cue classes -> positive entropy."""
if len(paras) < 3:
return None
roles = []
for p in paras:
r = "topic"
for name, rx in ROLE_RE.items():
if rx.search(p):
r = name
break
roles.append(r)
counts = Counter(roles)
n = len(roles)
ent = -sum((c / n) * math.log2(c / n) for c in counts.values())
return round(ent, 3)
def m_preview_fulfillment(paras: list[str]):
"""Share of body paragraphs whose opening content word appears in the
intro paragraph (outline-following / preview-then-fulfill tell)."""
if len(paras) < 4:
return None
intro = set(content(paras[0]))
if not intro:
return 0.0
body = paras[1:-1] if len(paras) > 2 else paras[1:]
hits = tot = 0
for p in body:
cs = content(p)
if not cs:
continue
tot += 1
if cs[0] in intro:
hits += 1
return round(hits / tot, 3) if tot else 0.0
def m_callback_content(paras: list[str]):
"""Content words introduced in the first third, ABSENT from the middle,
reappearing in the last third: a recap loop, not a sustained topic."""
n = len(paras)
if n < 5:
return None
third = max(1, n // 3)
early = set().union(*[set(content(paras[i])) for i in range(third)])
mid = (set().union(*[set(content(paras[i])) for i in range(third, n - third)])
if n - 2 * third > 0 else set())
late = set().union(*[set(content(paras[i])) for i in range(n - third, n)])
cb = (early & late) - mid
return round(len(cb) / n, 3)
def m_heading_preview(text: str):
"""Share of ## heading head-nouns whose key word also appears in the intro
paragraph = outline preview-then-fulfill. Measures outline-following, not
heading presence, so it is retained under --genre docs."""
heads = re.findall(r"(?m)^\s{0,3}#{2,3}\s+(.*)$", text)
if len(heads) < 3:
return None
paras = paragraphs(text)
intro = set(content(paras[0])) if paras else set()
if not intro:
return 0.0
hit = 0
for h in heads:
hc = set(content(h))
if hc & intro:
hit += 1
return round(hit / len(heads), 3)
# Metric registry: name -> (paragraph-based? / text-based?, function).
PARA_METRICS = {
"scaffold_opener_share": m_scaffold_opener_share,
"role_entropy_bits": m_role_entropy,
"preview_fulfillment": m_preview_fulfillment,
"callback_content": m_callback_content,
}
TEXT_METRICS = {
"heading_preview": m_heading_preview,
}
METRIC_ORDER = [
"scaffold_opener_share",
"role_entropy_bits",
"heading_preview",
"preview_fulfillment",
"callback_content",
]
SUGGESTIONS = {
"scaffold_opener_share":
"Open body paragraphs on their own specific claim, not a discourse cue.",
"role_entropy_bits":
"Stop rotating 'However / In addition / Ultimately' scaffold openers.",
"heading_preview":
"Headings restate the intro's outline; let sections carry new ground.",
"preview_fulfillment":
"The body just fulfills an outline previewed in the intro; drop the preview.",
"callback_content":
"The ending loops back to opening vocabulary; end on a concrete final point.",
}
def compute_metrics(text: str, paras: list[str]) -> dict:
row = {}
for name in METRIC_ORDER:
if name in PARA_METRICS:
row[name] = PARA_METRICS[name](paras)
else:
row[name] = TEXT_METRICS[name](text)
return row
def load_reference(path: Path) -> dict:
data = json.loads(path.read_text())
return data["metrics"]
def relu(x: float) -> float:
return x if x > 0 else 0.0
def flag(metric, value, threshold, detail, suggestion) -> dict:
return {
"metric": metric,
"value": value,
"threshold": threshold,
"severity": "soft",
"detail": detail,
"suggestion": suggestion,
}
# Metrics a given --genre suppresses outright. See the docstring in scan()
# for why only callback_content is suppressed under --genre docs.
GENRE_SUPPRESSIONS = {
"docs": {"callback_content"},
}
def scan(text: str, reference: dict, genre: str = "prose") -> dict:
# Genre is a passthrough echoed for parity with structure_scan.
# --genre docs suppresses ONLY callback_content: reference docs, specs, and
# doctrine conventionally reprise opening themes at the end, which is not
# the essay recap coda the metric exists to catch (SIL rows pin both
# directions). heading_preview is deliberately retained under docs because
# it measures outline-following (a tell even in reference docs), and the
# metric already clears legitimate academic roadmaps (struct17) on its own.
# --genre social documents that loose social copy rarely has the
# >=3-paragraph structure these metrics need.
paras = paragraphs(text)
base = {
"genre": genre,
"prose_paragraphs": len(paras),
}
if len(paras) < MIN_PARAGRAPHS:
base.update({
"flags": [],
"flagged": {},
"metrics": None,
"penalty": None,
"note": f"fewer than {MIN_PARAGRAPHS} prose paragraphs; "
"silhouette metrics not scored",
})
return base
metrics = compute_metrics(text, paras)
contributions = {}
penalty = 0.0
flags = []
for name in METRIC_ORDER:
if name in GENRE_SUPPRESSIONS.get(genre, set()):
contributions[name] = 0.0
continue
ref = reference[name]
value = metrics[name]
if not isinstance(value, (int, float)):
contributions[name] = None
continue
median = ref["median"]
scale = max(ref["iqr"], ref["fence"])
weight = ref["weight"]
contribution = round(weight * relu((value - median) / scale), 3)
contributions[name] = contribution
penalty += contribution
# A metric individually clears the human fence when value >= fence,
# i.e. its contribution reaches its full weight.
if value >= ref["fence"]:
flags.append(flag(
name,
value,
f"human fence {ref['fence']} (weight {weight})",
f"{name} at {value} clears the human upper fence "
f"{ref['fence']}.",
SUGGESTIONS[name],
))
penalty = round(penalty, 3)
if penalty >= PENALTY_THRESHOLD:
flags.insert(0, flag(
"silhouette_penalty",
penalty,
f">= {PENALTY_THRESHOLD}",
"The document's idea arrangement matches a templated AI silhouette "
"(preview-then-fulfill, rotating scaffold openers, recap loop).",
"Rearrange around the actual argument instead of a symmetric outline; "
"cut previews and the closing recap.",
))
base.update({
"flags": flags,
"flagged": {f["metric"]: True for f in flags},
"metrics": metrics,
"contributions": contributions,
"penalty": penalty,
})
return base
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("path", nargs="?")
parser.add_argument("--genre", choices=["prose", "docs", "social"], default="prose")
return parser.parse_args(argv)
def main(argv: list[str]) -> int:
args = parse_args(argv)
if not REFERENCE_PATH.exists():
print(f"Missing reference: {REFERENCE_PATH}", file=sys.stderr)
return 2
reference = load_reference(REFERENCE_PATH)
if args.path:
path = Path(args.path)
if not path.exists():
print(f"Missing file: {path}", file=sys.stderr)
return 2
text = path.read_text(errors="replace")
else:
text = sys.stdin.buffer.read().decode("utf-8", errors="replace")
result = scan(text, reference, args.genre)
if not result.get("flags") and not is_probably_english(text):
print(json.dumps({"non_english": True, "flags": [], "penalty": None}, indent=2))
print("note: input appears non-English; scanner declined (English-only).", file=sys.stderr)
return 0
print(json.dumps(result, indent=2))
is_flagged = bool(result.get("penalty") is not None
and result["penalty"] >= PENALTY_THRESHOLD)
return 1 if is_flagged else 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
@@ -0,0 +1,322 @@
#!/usr/bin/env python3
"""Scan prose for macro-structure AI-writing patterns."""
from __future__ import annotations
import argparse
import json
import math
import re
import sys
from collections import Counter
from pathlib import Path
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
from _lang import ( # noqa: E402
ENGLISH_FUNCTION_WORDS,
english_function_share,
is_probably_english,
paragraphs as _prose_paragraphs,
words,
)
from readability_metrics import split_sentences # noqa: E402
STOPWORDS = {
"the", "a", "an", "and", "or", "but", "in", "on", "at", "to", "for", "of",
"with", "by", "from", "is", "are", "was", "were", "be", "been", "being",
"have", "has", "had", "do", "does", "did", "will", "would", "could",
"should", "may", "might", "must", "can", "it", "its", "this", "that",
"these", "those", "we", "you", "they", "i", "he", "she", "as", "than",
"if", "then", "so", "not", "no", "yes", "into", "over", "under",
}
CONNECTIVE_OPENERS = re.compile(
r"^(however|moreover|furthermore|additionally|in addition|overall|"
r"consequently|nevertheless)\b",
re.I,
)
# Paragraph-initial "Every <noun> <verb-phrase>" template opener ("Every
# transformation is scored ...", "Every rewrite passes ..."). As a RHYTHM tell
# this is about REPETITION, so the metric below fires only at 2+ such paragraph
# openers; a lone "Every child deserves a good school." stays clean. The verb is
# a copula/auxiliary or a present/past inflection (-s / -ed) so the second word
# reads as a predicate, not another noun.
EVERY_OPENER_RE = re.compile(
r"^every\s+[a-z][\w'-]*\s+(?:is|are|was|were|be|been|being|has|have|had|"
r"do|does|did|can|will|shall|must|should|would|may|might|"
r"[a-z]+(?:s|ed))\b",
re.I,
)
SIGNPOST_RE = re.compile(
r"\b(first,|next,|having (?:covered|established)|in (?:this|the following|"
r"the next) section|as (?:mentioned|noted) (?:above|earlier)|let us turn|"
r"let's turn)\b",
re.I,
)
CLOSER_RE = re.compile(
r",\s+(ensuring|highlighting|reflecting|allowing|enabling|underscoring|"
r"showcasing|emphasizing|fostering|driving|paving|reinforcing|solidifying|"
r"demonstrating|contributing to)\b[^.!?]*[.!?\"]?$",
re.I,
)
CODA_START_RE = re.compile(
r"^(ultimately,|in the end,|in conclusion|as we've seen|only time will tell|"
r"remember,|the future\b)",
re.I,
)
BOLD_COLON_RE = re.compile(r"^\s*[-*+]?\s*\*\*[^*]{1,40}\*\*\s*:", re.M)
def prose_paragraphs(text: str) -> list[str]:
return _prose_paragraphs(text, blank_blockquotes=True)
def cv(values: list[int]) -> float:
if not values:
return 0.0
mean = sum(values) / len(values)
if mean == 0:
return 0.0
return math.sqrt(sum((v - mean) ** 2 for v in values) / len(values)) / mean
def content_bigrams(text: str) -> set[tuple[str, str]]:
toks = [w for w in words(text) if len(w) > 3 and w not in STOPWORDS]
return set(zip(toks, toks[1:]))
def triad_count(text: str) -> int:
return len(re.findall(r"\b[A-Za-z][A-Za-z-]+,\s+[A-Za-z][A-Za-z-]+,\s+and\s+[A-Za-z][A-Za-z-]+\b", text))
def flag(metric: str, value: float | int, threshold: str, detail: str, suggestion: str) -> dict:
return {
"metric": metric,
"value": value,
"threshold": threshold,
"severity": "soft",
"detail": detail,
"suggestion": suggestion,
}
# Metrics a given --genre suppresses outright (the genre's normal shape trips
# the metric without being AI-slop). Table form so each suppression is a single
# lookup instead of an inline `genre != "..."` conditional per metric.
GENRE_SUPPRESSIONS = {
"docs": {"bold_colon_listicle"},
"social": {"one_line_staccato"},
}
def scan(text: str, genre: str = "prose") -> dict:
paragraphs = prose_paragraphs(text)
prose_text = "\n\n".join(paragraphs)
sentences = split_sentences(prose_text)
sentence_lengths = [len(words(s)) for s in sentences]
prose_words = words(prose_text)
para_lengths = [len(words(p)) for p in paragraphs]
metrics = {
"sentence_burstiness": round(cv(sentence_lengths), 3),
"summary_sandwich": 0.0,
"paragraph_cv": round(cv(para_lengths), 3),
"sentence_mean_len": round(sum(sentence_lengths) / len(sentence_lengths), 1) if sentence_lengths else 0,
"triad_density": round((triad_count(prose_text) / len(prose_words) * 1000), 3) if prose_words else 0,
"em_dash_per_1k": round((text.count("—") / len(prose_words) * 1000), 3) if prose_words else 0,
"bold_colon_listicle_count": len(BOLD_COLON_RE.findall(text)),
"one_line_staccato_share": 0.0,
"connective_paragraph_openers": 0,
"every_template_openers": 0,
"signpost_density": 0.0,
"opener_unique_ratio": 0.0,
"top_opener_share": 0.0,
"max_consecutive_opener": 0,
"participial_closer_share": 0.0,
"conclusion_coda": False,
}
flags = []
if len(paragraphs) >= 2:
first = content_bigrams(paragraphs[0])
last = content_bigrams(paragraphs[-1])
union = first | last
metrics["summary_sandwich"] = round(len(first & last) / len(union), 3) if union else 0.0
if len(sentences) >= 8 and metrics["sentence_burstiness"] < 0.55:
flags.append(flag(
"sentence_burstiness",
metrics["sentence_burstiness"],
"< 0.55 over at least 8 prose sentences",
"Sentence lengths are unusually uniform for running prose.",
"Vary sentence length and cadence; if this is formal reference prose, review before treating it as blocking.",
))
if len(paragraphs) >= 3:
coda = bool(CODA_START_RE.search(paragraphs[-1]))
if not coda:
coda = len(content_bigrams(paragraphs[0]) & content_bigrams(paragraphs[-1])) >= 2
metrics["conclusion_coda"] = coda
if coda:
flags.append(flag(
"conclusion_coda",
1,
"last paragraph starts with a stock coda or repeats 2+ first-paragraph content bigrams",
"The ending reads like a recap/moral coda instead of a concrete final point.",
"Cut the wrap-up or end on a specific fact; if this is an abstract or executive summary, judge the genre before changing it.",
))
if (metrics["bold_colon_listicle_count"] >= 3
and "bold_colon_listicle" not in GENRE_SUPPRESSIONS.get(genre, set())):
flags.append(flag(
"bold_colon_listicle",
metrics["bold_colon_listicle_count"],
">= 3 bold-label colon lines",
"The raw Markdown has repeated bold-label listicle formatting.",
"Convert to prose or plain bullets; if this is a reference doc, rerun with --genre docs.",
))
if paragraphs:
one_line = 0
for p in paragraphs:
ps = split_sentences(p)
if len(ps) == 1 and len(words(ps[0])) < 12:
one_line += 1
metrics["one_line_staccato_share"] = round(one_line / len(paragraphs), 3)
if (len(paragraphs) >= 6 and metrics["one_line_staccato_share"] > 0.6
and "one_line_staccato" not in GENRE_SUPPRESSIONS.get(genre, set())):
flags.append(flag(
"one_line_staccato",
metrics["one_line_staccato_share"],
"> 0.60 over at least 6 paragraphs",
"Most paragraphs are short single-sentence beats.",
"Merge related beats and vary paragraph length; if this is social copy, rerun with --genre social.",
))
connective = sum(1 for p in paragraphs if CONNECTIVE_OPENERS.search(p))
metrics["connective_paragraph_openers"] = connective
if connective >= 3 or (len(paragraphs) >= 8 and connective / len(paragraphs) > 0.4):
flags.append(flag(
"connective_paragraph_openers",
connective,
">= 3 paragraphs or > 40% of 8+ paragraphs",
"Paragraphs repeatedly open with formal transition words.",
"Replace scaffold openers with specific topic sentences; academic prose may justify some connectors.",
))
every_openers = sum(1 for p in paragraphs if EVERY_OPENER_RE.search(p))
metrics["every_template_openers"] = every_openers
if every_openers >= 2:
flags.append(flag(
"every_template_openers",
every_openers,
">= 2 paragraphs opening 'Every <noun> <verb>'",
"Paragraphs repeatedly open on the 'Every ___ is/does ...' template.",
"Vary the paragraph openings; a repeated Every-template is a machine rhythm tell even when each sentence is fine on its own.",
))
if prose_words:
signposts = len(SIGNPOST_RE.findall(prose_text))
metrics["signpost_density"] = round(signposts / len(prose_words) * 100, 3)
if len(prose_words) >= 150 and metrics["signpost_density"] > 0.6:
flags.append(flag(
"signpost_density",
metrics["signpost_density"],
"> 0.6 per 100 prose words, minimum 150 words",
"The text over-explains its own structure.",
"Remove roadmap language unless the genre is a textbook, legal brief, or long guide.",
))
if len(sentences) >= 5:
openers = []
for s in sentences:
ws = words(s)
if not ws:
continue
openers.append(ws[0])
enumeration = {"the", "a", "an", "section", "chapter", "figure", "table",
"step", "part", "appendix"}
counted = [o for o in openers if o not in enumeration]
top_count = 0
if counted:
counts = Counter(counted)
top_count = max(counts.values())
metrics["opener_unique_ratio"] = round(len(counts) / len(counted), 3)
metrics["top_opener_share"] = round(top_count / len(counted), 3)
run = max_run = 0
prev = None
for opener in counted:
run = run + 1 if opener == prev else 1
prev = opener
max_run = max(max_run, run)
metrics["max_consecutive_opener"] = max_run
top_repeat = metrics["top_opener_share"] > 0.25 and top_count >= 4
if metrics["opener_unique_ratio"] < 0.55 or top_repeat or max_run >= 4:
flags.append(flag(
"opener_repetition",
metrics["opener_unique_ratio"],
"unique ratio < 0.55, one opener > 25% with 4+ uses, or 4 consecutive identical openers",
"Sentence openings repeat in a template-like rhythm.",
"Rewrite repeated starts; step-by-step docs may need repeated imperative openers.",
))
if sentences:
closer_count = sum(1 for s in sentences if CLOSER_RE.search(s))
metrics["participial_closer_share"] = round(closer_count / len(sentences), 3)
if len(sentences) >= 8 and metrics["participial_closer_share"] >= 0.15:
flags.append(flag(
"participial_closer_share",
metrics["participial_closer_share"],
">= 0.15 over at least 8 prose sentences",
"Many sentences end with editorial -ing consequence tails.",
"Make the consequence concrete or cut the tail; analytical prose may allow an occasional closer.",
))
return {
"flags": flags,
"flagged": {f["metric"]: True for f in flags},
"metrics": metrics,
"genre": genre,
"prose_sentences": len(sentences),
"prose_paragraphs": len(paragraphs),
}
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("path", nargs="?")
parser.add_argument("--genre", choices=["prose", "docs", "social"], default="prose")
return parser.parse_args(argv)
def main(argv: list[str]) -> int:
args = parse_args(argv)
if args.path:
path = Path(args.path)
if not path.exists():
print(f"Missing file: {path}", file=sys.stderr)
return 2
text = path.read_text(errors="replace")
else:
text = sys.stdin.buffer.read().decode("utf-8", errors="replace")
# English-only graceful decline, matching banned_phrase_scan.py.
result = scan(text, args.genre)
# Function-word absence alone is not evidence of a foreign language
# (imperative stacks and buzzword lists are English slop with few function
# words). Decline only when the heuristic fails AND nothing flagged.
if not result.get("flags") and not is_probably_english(text):
print(json.dumps({"non_english": True, "violations": [], "flags": []}, indent=2))
print("note: input appears non-English; scanner declined (English-only).", file=sys.stderr)
return 0
print(json.dumps(result, indent=2))
return 1 if result["flags"] else 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
+211
View File
@@ -0,0 +1,211 @@
#!/usr/bin/env python3
"""Co-writer suggestion mode: emit LSP-style structured edit suggestions.
Detection is cheap and deterministic (banned_phrase_scan + structure_scan).
Replacement generation is DELEGATED to a stronger model: by default every
suggestion is emitted with ``"suggested_replacement": null``. A separate
``--apply-replacements FILE`` mode merges externally-produced replacements back
in and light-validates them. The blocking contract gates live in
check_suggestions.py; this script never rewrites the document itself.
Output shape:
{
"document": "<original text>",
"suggestions": [
{
"span": {"start": N, "end": N, "text": "..."},
"severity": "hard" | "soft",
"category": "...",
"rationale": "...",
"suggested_replacement": null,
"phrased_as_question": bool
},
...
],
"counts": {...}
}
Soft findings are phrased as questions (register-dependent judgment calls);
hard findings are stated as direct replacements. Suggestions are emitted in a
deterministic order (span start, then end, then category) and never overlap.
Usage:
python3 scripts/suggest.py document.md
python3 scripts/suggest.py < document.md
python3 scripts/suggest.py document.md --apply-replacements replacements.json
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
from banned_phrase_scan import (
is_probably_english,
scan_for_violations,
)
from structure_scan import scan as structure_scan
def _line_starts(text: str) -> list[int]:
"""Character offset at which each line begins (index 0 == line 1)."""
starts = [0]
for i, ch in enumerate(text):
if ch == "\n":
starts.append(i + 1)
return starts
def _offset(line_starts: list[int], line_number: int, column: int) -> int:
return line_starts[line_number - 1] + (column - 1)
def _rationale(category: str, span_text: str, suggestion: str | None, is_soft: bool) -> str:
"""Build a reviewer-facing rationale.
Soft findings are worded as a question because they are judgment calls whose
right answer depends on register; hard findings are stated as a direct fix.
"""
if is_soft:
hint = f" Consider: {suggestion}." if suggestion else ""
return f"Soft tell ({category}): could “{span_text}” be cut or reworded here?{hint}"
if suggestion:
return f"AI-writing tell ({category}): replace “{span_text}” — {suggestion}"
return f"AI-writing tell ({category}): replace “{span_text}”."
def build_suggestions(text: str) -> list[dict]:
"""Detect AI-isms and turn each into a span-anchored suggestion.
Spans come from banned_phrase_scan (phrases + structural regexes), which are
the findings with real character offsets that a replacement can be applied
to. macro-structure flags from structure_scan are document-level and are
surfaced in ``counts`` instead, since they have no single applyable span.
"""
violations = scan_for_violations(text)
line_starts = _line_starts(text)
candidates: list[dict] = []
for v in violations:
start = _offset(line_starts, v["line_number"], v["column"])
end = start + len(v["phrase"])
span_text = text[start:end]
is_soft = v["severity"] == "soft"
candidates.append({
"span": {"start": start, "end": end, "text": span_text},
"severity": v["severity"],
"category": v["category"],
"rationale": _rationale(v["category"], span_text, v.get("suggestion"), is_soft),
"suggested_replacement": None,
"phrased_as_question": is_soft,
})
candidates.sort(key=lambda s: (s["span"]["start"], s["span"]["end"], s["category"]))
# Enforce non-overlap deterministically: keep the earliest-starting span and
# drop any later suggestion that overlaps an already-kept one, so the emitted
# set always satisfies the span-overlap contract gate.
kept: list[dict] = []
last_end = -1
for s in candidates:
if s["span"]["start"] >= last_end:
kept.append(s)
last_end = s["span"]["end"]
return kept
def counts_block(suggestions: list[dict], struct: dict) -> dict:
by_category: dict[str, int] = {}
hard = soft = 0
for s in suggestions:
by_category[s["category"]] = by_category.get(s["category"], 0) + 1
if s["severity"] == "soft":
soft += 1
else:
hard += 1
return {
"total": len(suggestions),
"hard": hard,
"soft": soft,
"by_category": by_category,
"structure_flags": [f["metric"] for f in struct.get("flags", [])],
}
def apply_replacements(suggestions: list[dict], repl_path: str) -> list[str]:
"""Merge externally-produced replacements into suggestions and light-validate.
The replacement file is ``{"replacements": [{"start", "end", "replacement"}]}``.
Each replacement is keyed to a suggestion by exact (start, end) span. This is
only a light merge/validation pass; the blocking contract lives in
check_suggestions.py.
"""
data = json.loads(Path(repl_path).read_text())
index = {(r["start"], r["end"]): r["replacement"] for r in data.get("replacements", [])}
warnings: list[str] = []
matched: set[tuple[int, int]] = set()
for s in suggestions:
key = (s["span"]["start"], s["span"]["end"])
if key not in index:
continue
rep = index[key]
s["suggested_replacement"] = rep
matched.add(key)
if rep == s["span"]["text"]:
warnings.append(f"replacement for span {list(key)} is identical to span text")
elif scan_for_violations(rep) or structure_scan(rep).get("flags"):
warnings.append(f"replacement for span {list(key)} does not pass the scanners in isolation")
for key in index:
if key not in matched:
warnings.append(f"replacement targets unknown span {list(key)}")
return warnings
def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("path", nargs="?", help="Document file. Reads stdin when omitted.")
parser.add_argument(
"--apply-replacements",
metavar="FILE",
help="Merge externally-produced replacements (JSON) into the suggestions.",
)
return parser.parse_args(argv)
def main(argv: list[str]) -> int:
args = parse_args(argv)
if args.path:
path = Path(args.path)
if not path.exists():
print(f"Missing file: {path}", file=sys.stderr)
return 2
text = path.read_text(errors="replace")
else:
text = sys.stdin.buffer.read().decode("utf-8", errors="replace")
# English-only graceful decline, matching the two scanners.
if not is_probably_english(text):
print(json.dumps({"non_english": True, "document": text, "suggestions": [],
"counts": {"total": 0, "hard": 0, "soft": 0,
"by_category": {}, "structure_flags": []}},
indent=2))
print("note: input appears non-English; co-writer declined (English-only).", file=sys.stderr)
return 0
suggestions = build_suggestions(text)
struct = structure_scan(text)
out = {
"document": text,
"suggestions": suggestions,
"counts": counts_block(suggestions, struct),
}
if args.apply_replacements:
out["apply_warnings"] = apply_replacements(suggestions, args.apply_replacements)
print(json.dumps(out, indent=2))
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
+409
View File
@@ -0,0 +1,409 @@
#!/usr/bin/env python3
"""
Validate that all must-preserve constraints survived transformation.
Compares original text constraints against transformed text.
Exit code 0 = all constraints preserved, 1 = missing constraints.
Usage:
python validate_preservation.py original.txt transformed.txt
python validate_preservation.py original.txt transformed.txt constraints.json
"""
from __future__ import annotations
import sys
import json
import re
from typing import TypedDict
# Import constraint extraction
from extract_constraints import extract_constraints, Constraint
class ValidationResult(TypedDict):
passed: bool
total_constraints: int
preserved: int
missing: list[Constraint]
warnings: list[str]
_MAGNITUDES = {
"k": 1e3, "thousand": 1e3,
"m": 1e6, "million": 1e6,
"b": 1e9, "billion": 1e9,
"trillion": 1e12,
}
_MONTHS = [
"january", "february", "march", "april", "may", "june", "july",
"august", "september", "october", "november", "december",
]
def normalize_value(value: str) -> str:
"""Normalize a constraint value for comparison."""
# Remove extra whitespace
normalized = re.sub(r'\s+', ' ', value.strip())
# Lowercase for comparison (preserve original for display)
return normalized.lower()
def parse_money(token: str) -> float | None:
"""Parse a currency token into an absolute amount, honoring magnitude words.
'$47.3M' and '$47.3 million' -> 47_300_000; '$47.3 billion' -> 47_300_000_000.
Magnitude is part of the fact, so the two must not compare equal.
"""
m = re.search(
r'([\d,]+\.?\d*)\s*(k|m|b|thousand|million|billion)?',
token.lower().replace("$", ""),
)
if not m or not m.group(1).strip(","):
return None
amount = float(m.group(1).replace(",", ""))
if m.group(2):
amount *= _MAGNITUDES[m.group(2)]
return amount
def parse_magnitude_number(token: str) -> float | None:
m = re.search(
r'\b([\d,]+\.?\d*)\s*(thousand|million|billion|trillion)?\b',
token.lower(),
)
if not m or not m.group(1).strip(","):
return None
amount = float(m.group(1).replace(",", ""))
if m.group(2):
amount *= _MAGNITUDES[m.group(2)]
return amount
def _numbers_match_exactly(value: str, text: str, pattern: str) -> bool:
"""True iff the numeric value appears as a whole quantity in text.
Guards against the substring trap where '12' is 'found' inside '120'.
"""
want = re.findall(r'\d+\.?\d*', value)
have = set(re.findall(pattern, text.lower()))
return all(num in have for num in want) and bool(want)
_UNIT_SYNONYMS = {
"km": {"km", "kilometer", "kilometers", "kilometre", "kilometres"},
"mi": {"mi", "mile", "miles"},
"kg": {"kg", "kilogram", "kilograms"},
"g": {"g", "gram", "grams"},
"ms": {"ms", "millisecond", "milliseconds"},
"s": {"s", "sec", "second", "seconds"},
"m": {"m", "meter", "meters", "metre", "metres"},
"cm": {"cm", "centimeter", "centimeters", "centimetre", "centimetres"},
"mm": {"mm", "millimeter", "millimeters", "millimetre", "millimetres"},
"ft": {"ft", "foot", "feet"},
"in": {"in", "inch", "inches"},
"lb": {"lb", "lbs", "pound", "pounds"},
"oz": {"oz", "ounce", "ounces"},
"°c": {"°c", "c", "celsius"},
"°f": {"°f", "f", "fahrenheit"},
"min": {"min", "mins", "minute", "minutes"},
"hr": {"hr", "hrs", "hour", "hours"},
"day": {"day", "days"},
"week": {"week", "weeks", "wk", "wks"},
"month": {"month", "months", "mo"},
"year": {"year", "years", "yr", "yrs"},
"kb": {"kb", "kilobyte", "kilobytes"},
"mb": {"mb", "megabyte", "megabytes"},
"gb": {"gb", "gigabyte", "gigabytes"},
"tb": {"tb", "terabyte", "terabytes"},
"pb": {"pb", "petabyte", "petabytes"},
"px": {"px", "pixel", "pixels"},
}
def _measurement_parts(value: str) -> tuple[str, set[str]] | None:
m = re.search(r'(\d+\.?\d*)\s*([^\d\s]+|degrees?(?:\s+\w+)?)', value, re.I)
if not m:
return None
unit = m.group(2).lower().strip()
unit = re.sub(r'^degrees?\s*', '', unit) or "degree"
aliases = _UNIT_SYNONYMS.get(unit)
if aliases is None:
for family in _UNIT_SYNONYMS.values():
if unit in family:
aliases = family
break
else:
aliases = {unit}
return m.group(1), aliases
def _quote_core(value: str) -> str:
return value.strip().strip('"“”').lower()
def _time_variants(value: str) -> set[str]:
m = re.search(r'\b(\d{1,2})(?::(\d{2}))?(?::\d{2})?\s*(am|pm)?\b', value, re.I)
if not m:
return set()
hour = str(int(m.group(1)))
minute = m.group(2) or "00"
suffix = (m.group(3) or "").lower()
spaced = f" {suffix}" if suffix else ""
compact = suffix
variants = {f"{hour}:{minute}{spaced}".strip(), f"{hour}:{minute}{compact}".strip()}
if minute == "00":
variants.update({f"{hour}{spaced}".strip(), f"{hour}{compact}".strip()})
return variants
def find_constraint_in_text(constraint: Constraint, text: str) -> bool:
"""Check if a constraint value exists in text."""
value = constraint["value"]
ctype = constraint["type"]
normalized_value = normalize_value(value)
normalized_text = normalize_value(text)
# Direct match
if normalized_value in normalized_text:
return True
# and/or: the disjunction must survive; "or" (or "or both") is faithful,
# a bare conjunction is not.
if ctype == "and_or":
return bool(re.search(r"\band/or\b|\bor\b", text, re.IGNORECASE))
# Currency: compare absolute amounts so a magnitude swap (M -> billion) fails.
if ctype == "currency":
target = parse_money(value)
if target is None:
return False
for token in re.findall(
r'\$?[\d,]+\.?\d*\s*(?:k|m|b|thousand|million|billion)?', text, re.IGNORECASE
):
amount = parse_money(token)
if amount is not None and abs(amount - target) < 0.01:
return True
return False
if ctype == "magnitude_number":
target = parse_magnitude_number(value)
if target is None:
return False
for token in re.findall(
r'\b[\d,]+\.?\d*\s*(?:thousand|million|billion|trillion)?\b',
text,
re.IGNORECASE,
):
amount = parse_magnitude_number(token)
if amount is not None and abs(amount - target) < 0.01:
return True
return False
# Percentage: require an exact numeric match (12% != 120%).
if ctype == "percentage":
return _numbers_match_exactly(value, text, r'(\d+\.?\d*)\s*(?:%|percent)')
if ctype == "measurement":
parts = _measurement_parts(value)
if not parts:
return False
number, units = parts
clean_text = text.replace(",", "").lower()
if not re.search(r'(?<![\d.])' + re.escape(number.replace(",", "")) + r'(?![\d.])', clean_text):
return False
return any(re.search(r'(?<!\w)' + re.escape(unit) + r'(?!\w)', clean_text) for unit in units)
if ctype == "range":
return _numbers_match_exactly(value, text, r'(?<![\d.])(\d+\.?\d*)(?![\d.])')
if ctype == "time":
text_variants = _time_variants(text)
return bool(_time_variants(value) & text_variants)
# Other quantities: match digits, comma-insensitive, but as whole tokens.
if ctype in ("count", "number"):
for num in re.findall(r'[\d,]+\.?\d*', value):
clean_num = num.replace(",", "")
if re.search(r'(?<![\d.])' + re.escape(clean_num) + r'(?![\d.])',
text.replace(",", "")):
return True
return False
# Dates: the year alone is not enough — a changed month is a changed fact.
if ctype == "date_quarter":
year_match = re.search(r'\d{4}', value)
if not (year_match and year_match.group() in text):
return False
q = re.search(r'q([1-4])', value.lower())
if not q:
return False
ordinal = {"1": "first", "2": "second", "3": "third", "4": "fourth"}[q.group(1)]
low_text = text.lower()
return q.group(0) in low_text or re.search(ordinal + r'\s+quarter', low_text) is not None
if ctype.startswith("date"):
year_match = re.search(r'\d{4}', value)
if not (year_match and year_match.group() in text):
return False
low = value.lower()
month = next((mo for mo in _MONTHS if mo[:3] in low), None)
if month and month[:3] not in text.lower():
return False
quarter = re.search(r'q[1-4]', low)
if quarter and quarter.group() not in text.lower():
return False
return True
# For quotes, check if core content is present (without surrounding quotes)
if constraint["type"] == "quote":
inner = _quote_core(value)
comparable_text = normalized_text.replace("“", '"').replace("”", '"')
if inner in comparable_text:
return True
if ctype == "proper_noun":
words = re.findall(r'[A-Z][a-z]+', value)
if words and all(re.search(r'\b' + re.escape(word) + r'\b', text, re.I) for word in words):
return True
return False
_NEGATION_RE = re.compile(
r"\b(?:not|never|no|cannot|can't|won't|don't|doesn't|didn't|isn't|aren't|"
r"wasn't|weren't|without|neither|nor|none|fails?\s+to|rather\s+than)\b", re.I)
_SCOPE_RE = re.compile(
r"\b(?:most|all|none|every|each|some|few|several|majority|minority|only|"
r"always|usually|typically|rarely|approximately|roughly|about|nearly)\b", re.I)
_CONDITIONAL_RE = re.compile(
r"\b(?:if|unless|provided\s+that|only\s+if|assuming|given\s+that|"
r"in\s+the\s+event|contingent\s+on|except|excluding|other\s+than|"
r"aside\s+from|save\s+for)\b", re.I)
_WEAK_MODAL_RE = re.compile(
r"\b(?:may|might|could|should|can|likely|probably|appears?|suggests?|seems?)\b", re.I)
_STRONG_MODAL_RE = re.compile(
r"\b(?:will|must|always|definitely|certainly|guarantees?|proves?)\b", re.I)
def semantic_drift_warnings(original: str, transformed: str) -> list[str]:
"""Warn when meaning-bearing words present in the original vanish in the rewrite."""
out: list[str] = []
o, t = original.lower(), transformed.lower()
on, tn = len(_NEGATION_RE.findall(o)), len(_NEGATION_RE.findall(t))
if on > tn:
out.append(
f"Negation count dropped {on}->{tn}. Verify no claim was inverted or "
"weakened (e.g. 'does not support' -> 'supports').")
missing_scope = sorted({w for w in _SCOPE_RE.findall(o)} - {w for w in _SCOPE_RE.findall(t)})
for w in missing_scope:
out.append(f"Scope/precision word '{w}' not in output. Verify the claim's scope is unchanged.")
oc, tc = len(_CONDITIONAL_RE.findall(o)), len(_CONDITIONAL_RE.findall(t))
if oc > tc:
out.append(
f"Conditional count dropped {oc}->{tc}. Verify a conditional claim "
"wasn't turned into an unconditional one.")
ow, tw = len(_WEAK_MODAL_RE.findall(o)), len(_WEAK_MODAL_RE.findall(t))
os, ts = len(_STRONG_MODAL_RE.findall(o)), len(_STRONG_MODAL_RE.findall(t))
if ow > tw and ts > os:
out.append("Hedged claim may have been strengthened. Verify uncertainty was not turned into certainty.")
return out
def validate_preservation(
original_text: str,
transformed_text: str,
constraints: list[Constraint] | None = None
) -> ValidationResult:
"""Validate that all constraints are preserved in transformed text."""
if constraints is None:
constraints = extract_constraints(original_text)
missing: list[Constraint] = []
warnings: list[str] = []
for constraint in constraints:
if not find_constraint_in_text(constraint, transformed_text):
missing.append(constraint)
# Generate warnings for near-misses
for m in missing:
if m["type"] == "percentage":
# Check if number exists without %
num = re.search(r'[\d.]+', m["value"])
if num and num.group() in transformed_text:
warnings.append(f"Number {num.group()} found but missing '%' symbol")
# Semantic drift warnings. These are NOT numbers/names — they are the
# meaning-bearing words (negations, scope, conditionals) that the skill's
# de-hedging rules are most likely to delete and that pure fact-matching
# cannot see. Surfaced as warnings so the rewrite gets a human re-check;
# de-slopping legitimately removes some, so they don't hard-fail on their own.
warnings.extend(semantic_drift_warnings(original_text, transformed_text))
preserved = len(constraints) - len(missing)
return {
"passed": len(missing) == 0,
"total_constraints": len(constraints),
"preserved": preserved,
"missing": missing,
"warnings": warnings
}
def main() -> None:
args = sys.argv[1:]
strict = False
if args and args[0] == "--strict":
strict = True
args = args[1:]
if len(args) < 2:
print("Usage: validate_preservation.py <original.txt> <transformed.txt> [constraints.json]")
sys.exit(1)
# Read inputs
try:
with open(args[0], 'r') as f:
original_text = f.read()
with open(args[1], 'r') as f:
transformed_text = f.read()
except OSError as e:
print(json.dumps({"error": f"Could not read input: {e}"}))
sys.exit(2)
# Optionally read pre-computed constraints
constraints = None
if len(args) > 2:
with open(args[2], 'r') as f:
data = json.load(f)
constraints = data.get("constraints", [])
result = validate_preservation(original_text, transformed_text, constraints)
# Output result
output = {
"passed": result["passed"],
"total_constraints": result["total_constraints"],
"preserved": result["preserved"],
"missing_count": len(result["missing"]),
"missing": result["missing"],
"warnings": result["warnings"]
}
print(json.dumps(output, indent=2))
# Exit with appropriate code
sys.exit(0 if result["passed"] and not (strict and result["warnings"]) else 1)
if __name__ == "__main__":
main()
@@ -0,0 +1,496 @@
#!/usr/bin/env python3
"""Distill a voice profile + samples into a LAYERED, pack-sized style card.
Output layout (deterministic — same inputs give byte-identical files):
<out>/card.md core sheet, always loaded, <= 300 words, with an
index table "when writing X, read card/X.md"
<out>/card/<situation>.md one sheet per COVERED situation from the taxonomy
Only situations with real sample evidence get a sheet. Uncovered situations are
NAMED in card.md under "Uncovered" and never given a fabricated sheet — every
claim on every sheet is derived from a measurable profile/sample fact or a
verbatim sample snippet.
``--coverage`` emits the deterministic lexical coverage matrix (which taxonomy
dimensions the samples exercise) as JSON and writes nothing. The classifier is
intentionally coarse: it only DRIVES interactive teach prompts and the sheet
set. A misclassified sentence can add or drop a sheet, never a card claim, so
misclassification is low-stakes by construction.
``--provenance`` also writes <out>/provenance.json (per-sample sha256 + word
counts, doc count, genre note, low-confidence flag) so a teach run is auditable.
"""
import argparse
import hashlib
import json
import re
import statistics
import sys
from pathlib import Path
import voice_profile
# The situation taxonomy. openings/closings are STRUCTURAL (first/last sentence
# of every document); the rest are lexical. Order is fixed for determinism.
TAXONOMY = [
"explaining-technical",
"anecdote",
"argument",
"disagreement",
"praise",
"hedging-uncertainty",
"numbers-data",
"addressing-reader",
"openings",
"closings",
]
STRUCTURAL = {"openings", "closings"}
# Minimum classified sentences before a lexical dimension counts as covered.
COVER_THRESHOLD = 2
NUMBER_WORDS = {
"one", "two", "three", "four", "five", "six", "seven", "eight", "nine",
"ten", "eleven", "twelve", "dozen", "hundred", "thousand", "million",
"twenty", "thirty", "forty", "fifty", "sixty", "seventy", "eighty",
"ninety", "percent", "half", "quarter", "double", "triple", "nine",
}
# Lexical signals per dimension. Each entry is a set of lowercase substrings;
# a sentence matches the dimension if any signal is present (word-boundary
# aware for single tokens, plain substring for multiword phrases).
SIGNALS = {
"explaining-technical": {
"because", "so that", "which means", "the reason", "works by",
"depends on", "the way it", "in order to", "that's how", "this is how",
"the trick is", "you have to", "the point of",
},
"argument": {
"therefore", "thus", "consequently", "the point is", "i distrust",
"clearly", "obviously", "in fact", "the truth is", "matters because",
"that's the point", "either way", "that's rare", "nobody", "no one",
},
"disagreement": {
"but i", "i don't", "no one", "nobody", "wrong", "i distrust",
"rather than", "not because", "i hate", "don't trust", "disagree",
"however", "i wasn't", "makes sense", "i can't",
},
"praise": {
"extraordinary", "wonderful", "dependable", "lovely", "beautiful",
"kindly", "generous", "grateful", "delightful", "plenty", "good choice",
"with care", "extraordinary care", "almost pleasant", "i like",
},
"hedging-uncertainty": {
"maybe", "perhaps", "probably", "might", "i guess", "i think",
"seems", "sort of", "kind of", "possibly", "i suppose", "not sure",
"or won't", "or it won't", "i believed", "or maybe",
},
"addressing-reader": set(), # handled specially (2nd person / question)
"numbers-data": set(), # handled specially (digits / number words)
"anecdote": set(), # handled specially (1st person + past/time cue)
}
FIRST_PERSON = {"i", "we", "my", "me", "we've", "i've", "i'll", "i'd", "our"}
TIME_CUES = {
"today", "yesterday", "tonight", "morning", "evening", "night", "ago",
"monday", "tuesday", "wednesday", "thursday", "friday", "saturday", "sunday",
"last", "once", "then", "later", "week", "year",
}
def sha256_file(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
def split_sentences_text(text):
"""Split into trimmed sentence strings, preserving original wording."""
out = []
for chunk in re.split(r"(?<=[.!?])\s+", text.strip()):
chunk = re.sub(r"\s+", " ", chunk).strip()
if chunk:
out.append(chunk)
return out
def _has_word(sentence_low, token):
return re.search(r"\b" + re.escape(token) + r"\b", sentence_low) is not None
def _matches(sentence_low, signals):
for sig in signals:
if " " in sig:
if sig in sentence_low:
return True
elif _has_word(sentence_low, sig):
return True
return False
def _is_numbers(sentence_low):
if re.search(r"\d", sentence_low):
return True
return any(_has_word(sentence_low, w) for w in NUMBER_WORDS)
def _is_anecdote(sentence_low):
toks = set(voice_profile.words(sentence_low))
if not (toks & FIRST_PERSON):
return False
if toks & TIME_CUES:
return True
return any(t.endswith("ed") and len(t) > 3 for t in toks)
def _is_addressing(sentence):
low = sentence.lower()
if sentence.rstrip().endswith("?"):
return True
return _has_word(low, "you") or _has_word(low, "your") or _has_word(low, "you're")
def classify_dimension(sentence):
"""Return the set of lexical dimensions a single sentence exercises."""
low = sentence.lower()
hit = set()
for dim, signals in SIGNALS.items():
if dim in ("addressing-reader", "numbers-data", "anecdote"):
continue
if _matches(low, signals):
hit.add(dim)
if _is_numbers(low):
hit.add("numbers-data")
if _is_addressing(sentence):
hit.add("addressing-reader")
if _is_anecdote(low):
hit.add("anecdote")
return hit
def collect(samples_dir):
"""Return (docs, sentences) where each doc is a list of sentence strings."""
docs = []
for path in voice_profile.iter_docs(samples_dir):
sents = split_sentences_text(path.read_text(errors="replace"))
if sents:
docs.append(sents)
return docs
def coverage_matrix(docs):
"""Deterministic coverage of the taxonomy over the sample sentences."""
buckets = {dim: [] for dim in TAXONOMY}
for doc in docs:
if doc:
buckets["openings"].append(doc[0])
buckets["closings"].append(doc[-1])
for sent in doc:
for dim in classify_dimension(sent):
buckets[dim].append(sent)
matrix = {}
for dim in TAXONOMY:
sents = buckets[dim]
if dim in STRUCTURAL:
covered = len(docs) >= 1
else:
covered = len(sents) >= COVER_THRESHOLD
matrix[dim] = {
"count": len(sents),
"covered": covered,
"structural": dim in STRUCTURAL,
}
return matrix, buckets
def _shortest(sentences, k=3):
ordered = sorted(set(sentences), key=lambda s: (len(s), s))
return ordered[:k]
def _contraction_examples(docs, k=3):
seen = {}
for doc in docs:
for sent in doc:
for tok in voice_profile.words(sent):
if "'" in tok:
seen[tok] = seen.get(tok, 0) + 1
ranked = sorted(seen.items(), key=lambda kv: (-kv[1], kv[0]))
return [w for w, _ in ranked[:k]]
def _opener_words(docs, k=5):
counts = {}
for doc in docs:
for sent in doc:
toks = voice_profile.words(sent)
if toks:
counts[toks[0]] = counts.get(toks[0], 0) + 1
ranked = sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))
return [w for w, _ in ranked[:k]]
def _burstiness(profile):
med = profile["sentence_lengths"]["median"]
iqr = profile["sentence_lengths"]["iqr"]
if med <= 0:
return "uneven"
ratio = iqr / med
if ratio < 0.5:
return "steady"
if ratio < 1.1:
return "moderately bursty"
return "very bursty"
def _numbers_tokens(sentences):
toks = set()
for sent in sentences:
for m in re.findall(r"\d[\d,:.]*", sent):
toks.add(m)
for w in voice_profile.words(sent.lower()):
if w in NUMBER_WORDS:
toks.add(w)
return sorted(toks)
def _sheet_markers(dim, sentences, profile):
lengths = [len(voice_profile.words(s)) for s in sentences]
med = statistics.median(lengths) if lengths else 0
lines = [f"- Sentences in samples exercising this: {len(sentences)}.",
f"- Median length of those sentences: {int(med)} words."]
if dim == "numbers-data":
toks = _numbers_tokens(sentences)
lines.append("- Numeric tokens actually used: " + ", ".join(toks[:12]) + ".")
elif dim == "addressing-reader":
q = sum(1 for s in sentences if s.rstrip().endswith("?"))
lines.append(f"- Direct questions to the reader: {q}.")
lines.append("- Second person appears; keep it plain, no salesy 'you'.")
elif dim == "openings":
openers = sorted({voice_profile.words(s)[0] for s in sentences if voice_profile.words(s)})
lines.append("- Documents open on: " + ", ".join(openers[:8]) + ".")
elif dim == "closings":
lines.append("- Endings land on a concrete image, not a moral recap.")
else:
cr = profile["contraction_rate"]
lines.append(f"- Overall contraction rate: {cr:.3f} (keep it consistent here).")
return lines
SHEET_HOWTO = {
"explaining-technical": "Explain by naming the concrete mechanism, not the abstraction. Short causal sentences; 'because' does the work.",
"anecdote": "Tell it first person, past tense, one scene at a time. Concrete nouns, sensory detail, no summarizing moral.",
"argument": "State the claim flat, then the reason. No hedging scaffold; the point lands in one line.",
"disagreement": "Disagree by contrast, not confrontation. 'I don't', 'rather than', a plain preference rather than a takedown.",
"praise": "Praise through specific, restrained detail. Understated approval, never gushing.",
"hedging-uncertainty": "Hold uncertainty lightly with 'maybe' / 'probably' / 'or it won't', not corporate qualifiers.",
"numbers-data": "Numbers stay small, concrete, woven into the scene rather than tabulated.",
"addressing-reader": "Address the reader sparingly and plainly; a direct question or a flat 'you can'.",
"openings": "Open cold on a concrete fact or action. No throat-clearing, no thesis statement.",
"closings": "Close on a small, specific image. No wrap-up, no 'ultimately'.",
}
def build_sheet(dim, sentences, profile):
title = dim.replace("-", " ")
lines = [f"# Voice sheet: {title}", ""]
lines.append(SHEET_HOWTO[dim])
lines.append("")
lines.append("## Sample snippets")
for snip in _shortest(sentences):
lines.append(f"> {snip}")
lines.append("")
lines.append("## Measured markers")
lines.extend(_sheet_markers(dim, sentences, profile))
lines.append("")
return "\n".join(lines) + "\n"
def _never_does(profile):
never = []
p = profile["punctuation"]
label = {";": "semicolons", "!": "exclamation points", ":": "colons",
"-": "hyphenated dashes", "(": "parentheticals"}
for mark, name in label.items():
if p.get(mark, 0.0) == 0.0:
never.append(name)
return never
def build_card(profile, docs, matrix, name):
med = int(profile["sentence_lengths"]["median"])
iqr = int(profile["sentence_lengths"]["iqr"])
contr = profile["contraction_rate"]
examples = _contraction_examples(docs)
openers = _opener_words(docs)
never = _never_does(profile)
covered = [d for d in TAXONOMY if matrix[d]["covered"]]
uncovered = [d for d in TAXONOMY if not matrix[d]["covered"]]
lines = [f"# Voice card: {name}", ""]
lines.append(
f"Rhythm: median sentence {med} words, IQR {iqr}, {_burstiness(profile)}. "
f"Paragraphs average {int(profile['paragraph_stats']['mean_words'])} words."
)
if examples:
lines.append(
f"Contractions: rate {contr:.3f}; e.g. " + ", ".join(examples) + "."
)
else:
lines.append(f"Contractions: rate {contr:.3f}; rarely contracts.")
if never:
lines.append("Never: " + "; ".join(never) + ".")
lines.append("Openers: " + ", ".join(openers) + ".")
lines.append("")
lines.append("Match rhythm and habits first; keep facts and meaning intact.")
lines.append("")
lines.append("## When writing, read the matching sheet")
lines.append("")
lines.append("| Situation | Sheet |")
lines.append("|-----------|-------|")
for dim in covered:
lines.append(f"| {dim.replace('-', ' ')} | card/{dim}.md |")
lines.append("")
if uncovered:
lines.append(
"Uncovered (no sample evidence — do not fabricate a voice for these): "
+ ", ".join(uncovered) + "."
)
return "\n".join(lines) + "\n"
def card_word_count(card_text):
return len(re.findall(r"[A-Za-z0-9']+", card_text))
def write_card(profile, samples_dir, out_dir, name):
docs = collect(samples_dir)
matrix, buckets = coverage_matrix(docs)
out = Path(out_dir)
(out / "card").mkdir(parents=True, exist_ok=True)
# Remove any stale sheets so uncovered dims never keep an old file.
for stale in (out / "card").glob("*.md"):
stale.unlink()
card = build_card(profile, docs, matrix, name)
(out / "card.md").write_text(card)
for dim in TAXONOMY:
if matrix[dim]["covered"]:
(out / "card" / f"{dim}.md").write_text(build_sheet(dim, buckets[dim], profile))
return matrix
def write_provenance(profile, samples_dir, out_dir):
samples = []
total = 0
for path in voice_profile.iter_docs(samples_dir):
text = path.read_text(errors="replace")
wc = len(voice_profile.words(text))
total += wc
samples.append({
"file": path.name,
"sha256": sha256_file(path),
"words": wc,
})
meta = profile.get("metadata", {})
prov = {
"doc_count": len(samples),
"total_words": total,
"samples": samples,
"genre_note": meta.get("genre_warning", "") or "same-genre samples assumed",
"low_confidence": bool(meta.get("low_confidence", total < 2000)),
}
Path(out_dir, "provenance.json").write_text(
json.dumps(prov, indent=2, sort_keys=True) + "\n"
)
return prov
def profile_mismatch(supplied, recomputed, path=""):
"""First named field where the supplied profile disagrees with one recomputed
from --samples, or None. Counts (ints) compare exactly; floats within 1e-6.
The function-word background is a normalizer (from --background, not the
sample content), so it is not part of the equality check."""
here = path or "<root>"
if isinstance(supplied, bool) or isinstance(recomputed, bool):
return None if supplied == recomputed else here
if isinstance(supplied, dict):
if not isinstance(recomputed, dict):
return here
for k in sorted(set(supplied) | set(recomputed)):
if k == "function_word_background":
continue
if k not in supplied or k not in recomputed:
return f"{path}.{k}".lstrip(".")
m = profile_mismatch(supplied[k], recomputed[k], f"{path}.{k}".lstrip("."))
if m:
return m
return None
if isinstance(supplied, list):
if not isinstance(recomputed, list) or len(supplied) != len(recomputed):
return here
for i, (x, y) in enumerate(zip(supplied, recomputed)):
m = profile_mismatch(x, y, f"{path}[{i}]")
if m:
return m
return None
if isinstance(supplied, int) and isinstance(recomputed, int):
return None if supplied == recomputed else here
if isinstance(supplied, (int, float)) and isinstance(recomputed, (int, float)):
return None if abs(supplied - recomputed) <= 1e-6 else here
return None if supplied == recomputed else here
def parse_args(argv):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--profile", required=True)
parser.add_argument("--samples", required=True)
parser.add_argument("--out")
parser.add_argument("--name", default="voice")
parser.add_argument("--coverage", action="store_true",
help="print the coverage matrix as JSON and write nothing")
parser.add_argument("--provenance", action="store_true",
help="also write <out>/provenance.json")
return parser.parse_args(argv)
def main(argv):
args = parse_args(argv)
profile_path = Path(args.profile)
samples_dir = Path(args.samples)
if not profile_path.exists() or not samples_dir.is_dir():
print("missing profile or samples dir", file=sys.stderr)
return 2
if not list(voice_profile.iter_docs(samples_dir)):
print(f"no sample documents in {samples_dir}: teach reads only .txt and .md "
f"files (recursively). Rename samples to .txt/.md or point --samples at "
f"the right directory.", file=sys.stderr)
return 2
profile = json.loads(profile_path.read_text())
# Consistency gate: the supplied profile must describe these very samples.
recomputed = voice_profile.build_profile(samples_dir)
mismatch = profile_mismatch(profile, recomputed)
if mismatch is not None:
print(f"profile does not match --samples (recompute differs at '{mismatch}'); "
f"rebuild the profile from these samples with voice_profile.py",
file=sys.stderr)
return 2
if args.coverage:
docs = collect(samples_dir)
matrix, _ = coverage_matrix(docs)
print(json.dumps(matrix, indent=2, sort_keys=True))
return 0
if not args.out:
print("--out is required unless --coverage", file=sys.stderr)
return 2
write_card(profile, samples_dir, args.out, args.name)
if args.provenance:
write_provenance(profile, samples_dir, args.out)
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
@@ -0,0 +1,194 @@
#!/usr/bin/env python3
"""Build deterministic stylometric voice profiles.
The built-in background table is a compact Writeprints-style English baseline:
common function words seeded from public-domain frequency lists used by classic
authorship-attribution examples, with broad fallback means/stddevs for words not
observed in a caller-supplied background corpus. It is meant only as a stable
normalizer; pass --background with same-genre documents for calibrated work.
"""
import argparse
import collections
import json
import math
import re
import statistics
import sys
from pathlib import Path
FUNCTION_WORDS = """
the of and to in a is that it for as with was on be by he i this are or his from at
which but have an had they you were their one all we can her has there been if more
when will would who so no she about out up into do any your what than them some could
these other then its our two may first my now such like over only also after most did
many before must through back where much should well people down own just because good
each those how under see made very being make between both even another while last
might still same never every against since off though yet without within upon among
until during per either neither nor whether whose whom why again once here there
therefore however although nevertheless moreover instead indeed perhaps rather thus
else already almost around across behind beyond near toward towards above below beside
inside outside along plus minus except despite via versus including regarding concerning
am were does done doing having let lets cannot dont didn't doesn't isn't aren't wasn't
weren't haven't hasn't hadn't won't wouldn't shouldn't couldn't mightn't mustn't i'm
you're he's she's it's we're they're i've you've we've they've i'd you'd he'd she'd we'd
they'd i'll you'll he'll she'll we'll they'll me him us mine yours ours theirs myself
yourself himself herself itself ourselves yourselves themselves
""".split()
PUNCT = [",", ".", ";", ":", "?", "!", "-", "(", ")", '"', "'"]
WORD_RE = re.compile(r"[A-Za-z]+(?:'[A-Za-z]+)?|\d+")
SENT_RE = re.compile(r"[^.!?]+[.!?]?")
def iter_docs(root):
for path in sorted(Path(root).rglob("*")):
if path.suffix.lower() in {".txt", ".md"} and path.is_file():
yield path
def normalize(text):
return re.sub(r"\s+", " ", text.lower()).strip()
def words(text):
return WORD_RE.findall(text.lower())
def sentences(text):
out = []
for part in SENT_RE.findall(text):
toks = words(part)
if toks:
out.append(toks)
return out
def char3_counts(text, limit=None):
norm = normalize(text)
grams = collections.Counter(norm[i:i + 3] for i in range(max(0, len(norm) - 2)))
items = sorted(grams.items(), key=lambda kv: (-kv[1], kv[0]))
if limit:
items = items[:limit]
return dict(items)
def function_freq(tokens):
total = max(1, len(tokens))
counts = collections.Counter(tokens)
return {w: counts[w] / total for w in FUNCTION_WORDS}
def sentence_stats(text):
lengths = [len(s) for s in sentences(text)]
if not lengths:
return {"lengths": [], "median": 0.0, "iqr": 0.0}
ordered = sorted(lengths)
mid = statistics.median(ordered)
q1 = statistics.median(ordered[:len(ordered) // 2] or ordered)
q3 = statistics.median(ordered[(len(ordered) + 1) // 2:] or ordered)
return {"lengths": lengths, "median": mid, "iqr": q3 - q1}
def mtld(tokens, threshold=0.72):
if len(tokens) < 20:
return 0.0
factors = 0.0
types = set()
count = 0
for tok in tokens:
count += 1
types.add(tok)
if len(types) / count <= threshold:
factors += 1
types.clear()
count = 0
if count:
ttr = len(types) / count
factors += (1 - ttr) / (1 - threshold) if threshold < 1 else 0
return len(tokens) / factors if factors else float(len(tokens))
def feature_bundle(text):
toks = words(text)
total = max(1, len(toks))
punct_counts = collections.Counter(ch for ch in text if ch in PUNCT)
contractions = sum(1 for t in toks if "'" in t)
hist = collections.Counter(min(len(t), 15) for t in toks)
paragraphs = [p for p in re.split(r"\n\s*\n", text.strip()) if p.strip()]
return {
"char3": char3_counts(text, 2000),
"function_words": function_freq(toks),
"sentence_lengths": sentence_stats(text),
"punctuation": {p: punct_counts[p] / total for p in PUNCT},
"contraction_rate": contractions / total,
"mtld": mtld(toks),
"word_length_histogram": {str(i): hist[i] / total for i in range(1, 16)},
"paragraph_stats": {
"count": len(paragraphs),
"mean_words": (sum(len(words(p)) for p in paragraphs) / len(paragraphs)) if paragraphs else 0.0,
},
"total_words": len(toks),
}
def background_stats(root=None):
docs = []
if root:
docs = [p.read_text(errors="replace") for p in iter_docs(root)]
if not docs:
return {w: {"mean": 0.0025 if w not in {"the", "of", "and", "to", "in", "a"} else 0.025,
"std": 0.006} for w in FUNCTION_WORDS}
rows = [function_freq(words(text)) for text in docs]
stats = {}
for w in FUNCTION_WORDS:
vals = [r[w] for r in rows]
stats[w] = {
"mean": statistics.mean(vals),
"std": statistics.pstdev(vals) or 0.0001,
}
return stats
def build_profile(samples_dir, background=None):
paths = list(iter_docs(samples_dir))
text = "\n\n".join(p.read_text(errors="replace") for p in paths)
profile = feature_bundle(text)
profile["function_word_background"] = background_stats(background)
profile["metadata"] = {
"doc_count": len(paths),
"total_words": profile["total_words"],
"low_confidence": profile["total_words"] < 2000,
"genre_warning": "profile has fewer than 2000 words" if profile["total_words"] < 2000 else "",
}
return profile
def parse_args(argv):
parser = argparse.ArgumentParser()
parser.add_argument("samples_dir")
parser.add_argument("-o", "--output", required=True)
parser.add_argument("--background")
return parser.parse_args(argv)
def main(argv):
args = parse_args(argv)
root = Path(args.samples_dir)
if not root.is_dir():
print(f"missing samples dir: {root}", file=sys.stderr)
return 2
if not list(iter_docs(root)):
print(f"no sample documents in {root}: only .txt and .md files are read "
f"(recursively). Rename samples to .txt/.md or point at the right "
f"directory.", file=sys.stderr)
return 2
profile = build_profile(root, args.background)
if profile["metadata"]["low_confidence"]:
print(profile["metadata"]["genre_warning"], file=sys.stderr)
Path(args.output).write_text(json.dumps(profile, indent=2, sort_keys=True) + "\n")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
@@ -0,0 +1,271 @@
#!/usr/bin/env python3
"""Score a candidate against a voice profile.
Lower composite means more user-like. The composite is half GI-rank penalty and
half clipped, weighted impostor z-distance using the WP10a research weights.
"""
from __future__ import annotations
import argparse
import json
import math
import random
import statistics
import sys
from pathlib import Path
import voice_profile
WEIGHTS = {
"char3": 0.30,
"delta": 0.25,
"sentence_emd": 0.10,
"punctuation": 0.08,
"contraction": 0.07,
"mtld": 0.10,
"word_length": 0.10,
}
def cosine_distance(a, b, keys=None):
keys = list(keys) if keys is not None else sorted(set(a) | set(b))
dot = sum(a.get(k, 0.0) * b.get(k, 0.0) for k in keys)
na = math.sqrt(sum(a.get(k, 0.0) ** 2 for k in keys))
nb = math.sqrt(sum(b.get(k, 0.0) ** 2 for k in keys))
if not na or not nb:
return 1.0
return 1 - (dot / (na * nb))
def z_function_vector(freqs, bg, top):
keys = sorted(bg, key=lambda k: bg[k].get("mean", 0), reverse=True)[:top]
return {k: (freqs.get(k, 0.0) - bg[k]["mean"]) / (bg[k]["std"] or 0.0001) for k in keys}
def emd(a, b):
if not a or not b:
return 0.0
max_len = max(max(a), max(b))
ca = cb = dist = 0.0
for i in range(1, max_len + 1):
ca += sum(1 for x in a if x == i) / len(a)
cb += sum(1 for x in b if x == i) / len(b)
dist += abs(ca - cb)
return dist
def l1(a, b, keys):
return sum(abs(a.get(k, 0.0) - b.get(k, 0.0)) for k in keys)
def distances(profile, feats, subset=None):
subset = set(subset or WEIGHTS)
out = {}
if "char3" in subset:
keys = set(profile["char3"]) | set(feats["char3"])
out["char3"] = cosine_distance(profile["char3"], feats["char3"], keys)
if "delta" in subset:
top = 50 if feats["total_words"] < 300 else 200
bg = profile["function_word_background"]
pv = z_function_vector(profile["function_words"], bg, top)
cv = z_function_vector(feats["function_words"], bg, top)
out["delta"] = cosine_distance(pv, cv, pv.keys())
if "sentence_emd" in subset:
med = profile["sentence_lengths"].get("median") or 1.0
out["sentence_emd"] = emd(profile["sentence_lengths"]["lengths"], feats["sentence_lengths"]["lengths"]) / med
if "punctuation" in subset:
out["punctuation"] = l1(profile["punctuation"], feats["punctuation"], voice_profile.PUNCT)
if "contraction" in subset:
out["contraction"] = abs(profile["contraction_rate"] - feats["contraction_rate"])
if "mtld" in subset:
out["mtld"] = abs(profile["mtld"] - feats["mtld"]) / (profile["mtld"] or 1.0)
if "word_length" in subset:
keys = [str(i) for i in range(1, 16)]
out["word_length"] = l1(profile["word_length_histogram"], feats["word_length_histogram"], keys)
return out
def weighted_sum(dists):
return sum(WEIGHTS[k] * dists.get(k, 0.0) for k in WEIGHTS)
def impostor_features(root):
feats = []
for path in voice_profile.iter_docs(root):
feats.append((str(path), voice_profile.feature_bundle(path.read_text(errors="replace"))))
return feats
def zscores(candidate, impostor_rows):
out = {}
for key in WEIGHTS:
vals = [row[key] for row in impostor_rows if key in row]
if key not in candidate or not vals:
out[key] = None
continue
mean = statistics.mean(vals)
std = statistics.pstdev(vals) or 0.0001
out[key] = (candidate[key] - mean) / std
return out
def gi_score(profile, cand_feats, impostors, seed):
rng = random.Random(seed)
keys = list(WEIGHTS)
wins = 0
trials = 64
# Per-key distances depend only on (profile, feats) -- not on which subset
# of keys a given trial happens to draw -- so compute every key's distance
# ONCE per candidate/impostor here, then have each trial do a subset-
# weighted sum over the precomputed values instead of recomputing
# distances() from scratch on every trial. Arithmetically exact: the
# trial loop still consumes rng.random()/rng.sample() in the same order,
# and summing WEIGHTS[k]*value only over the drawn subset is the same
# float sequence weighted_sum(distances(..., subset)) produced (excluded
# keys contributed an exact 0.0 term either way).
cand_dists = distances(profile, cand_feats)
impostor_dists = [(name, distances(profile, imp)) for name, imp in impostors]
for _ in range(trials):
subset = [k for k in keys if rng.random() < 0.5] or [rng.choice(keys)]
cand = sum(WEIGHTS[k] * cand_dists.get(k, 0.0) for k in subset)
sampled = rng.sample(impostor_dists, k=min(len(impostor_dists), max(1, len(impostor_dists) // 2)))
if all(cand < sum(WEIGHTS[k] * imp_dists.get(k, 0.0) for k in subset)
for _, imp_dists in sampled):
wins += 1
return wins / trials
def ngrams(tokens, n=4):
return set(tuple(tokens[i:i + n]) for i in range(max(0, len(tokens) - n + 1)))
LCS_THRESHOLD = 120
def has_common_substring_over(a: str, b: str, min_length: int) -> bool:
"""Rolling-hash check: do ``a`` and ``b`` share a contiguous substring of at
least ``min_length`` characters?
O(len(a) + len(b)) expected time, vs. the O(len(a) * len(b)) classic DP a
true longest-common-substring computation needs. This only answers the
threshold question the copy gate actually asks ("is there a shared run
longer than N chars"); it does not recover the true longest common
substring length. Every hash match is verified against the source
characters before being trusted, so a hash collision never produces a
false positive.
"""
if min_length <= 0:
return bool(a) and bool(b)
if len(a) < min_length or len(b) < min_length:
return False
base = 257
mod = (1 << 61) - 1
high_power = pow(base, min_length - 1, mod)
def window_hashes(s: str) -> dict[int, list[int]]:
table: dict[int, list[int]] = {}
h = 0
for i in range(min_length):
h = (h * base + ord(s[i])) % mod
table.setdefault(h, []).append(0)
for i in range(min_length, len(s)):
h = ((h - ord(s[i - min_length]) * high_power) * base + ord(s[i])) % mod
table.setdefault(h, []).append(i - min_length + 1)
return table
table_a = window_hashes(a)
table_b = window_hashes(b)
for h, starts_b in table_b.items():
starts_a = table_a.get(h)
if not starts_a:
continue
for sb in starts_b:
window_b = b[sb:sb + min_length]
for sa in starts_a:
if a[sa:sa + min_length] == window_b:
return True
return False
def copy_gate(candidate_text, samples_dir):
cand_grams = ngrams(voice_profile.words(candidate_text))
max_overlap = 0.0
lcs_violation = False
for path in voice_profile.iter_docs(samples_dir):
text = path.read_text(errors="replace")
sample_grams = ngrams(voice_profile.words(text))
if cand_grams:
max_overlap = max(max_overlap, len(cand_grams & sample_grams) / len(cand_grams))
if has_common_substring_over(candidate_text, text, LCS_THRESHOLD + 1):
lcs_violation = True
# "longest_common_substring" is reported for backward-compatible shape only:
# nothing pins its exact value (checked evals/adversarial-evals.json and
# evals/check_*.py for max_lcs/longest_common_substring assertions; none
# exist). Its semantics changed from "the true LCS length" to "the
# matched threshold window length when a violation-length run was found,
# else 0" -- computing the true max cheaply isn't possible without the
# same O(n*m) cost this function exists to avoid.
max_lcs = LCS_THRESHOLD + 1 if lcs_violation else 0
return {
"max_overlap": max_overlap,
"longest_common_substring": max_lcs,
"violation": max_overlap > 0.35 or lcs_violation,
}
def parse_args(argv):
parser = argparse.ArgumentParser()
parser.add_argument("--profile", required=True)
parser.add_argument("--impostors", required=True)
parser.add_argument("--seed", required=True, type=int)
parser.add_argument("--samples")
parser.add_argument("candidate_file")
return parser.parse_args(argv)
def read_candidate(path):
if path == "-":
return sys.stdin.buffer.read().decode("utf-8", errors="replace")
p = Path(path)
if not p.exists():
raise FileNotFoundError(path)
return p.read_text(errors="replace")
def main(argv):
args = parse_args(argv)
if not Path(args.profile).exists() or not Path(args.impostors).is_dir():
print("missing profile or impostors", file=sys.stderr)
return 2
try:
text = read_candidate(args.candidate_file)
except FileNotFoundError as e:
print(f"missing candidate: {e}", file=sys.stderr)
return 2
profile = json.loads(Path(args.profile).read_text())
feats = voice_profile.feature_bundle(text)
low = feats["total_words"] < 150
impostors = impostor_features(args.impostors)
dist = distances(profile, feats)
imp_dist = [distances(profile, f) for _, f in impostors]
zs = zscores(dist, imp_dist)
zsum = sum(WEIGHTS[k] * max(-3.0, min(3.0, zs[k] if zs[k] is not None else 0.0)) for k in WEIGHTS)
gi = gi_score(profile, feats, impostors, args.seed) if impostors else 0.0
result = {
"candidate_words": feats["total_words"],
"low_confidence": low,
"distances": {k: (None if low and k in {"sentence_emd", "mtld"} else v) for k, v in dist.items()},
"z_scores": {k: (None if low and k in {"sentence_emd", "mtld"} else v) for k, v in zs.items()},
"gi": gi,
"composite": 0.5 * (1 - gi) + 0.5 * zsum,
}
if args.samples:
result["copy_gate"] = copy_gate(text, args.samples)
print(json.dumps(result, indent=2, sort_keys=True))
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
+479
View File
@@ -0,0 +1,479 @@
#!/usr/bin/env python3
"""
Sync unslop rules with Wikipedia's "Signs of AI writing" page.
Fetches the latest revision, parses structured data from wikitext,
and outputs diffs or integration prompts for Claude Code.
Subcommands:
check - Check for updates (exits 0 if no updates, 1 if updates available)
diff - Output structured JSON diff of changes since last sync
prompt - Output a Claude Code integration prompt with mapped changes
Usage:
python wiki_sync.py check
python wiki_sync.py diff
python wiki_sync.py prompt
python wiki_sync.py prompt | claude -p --allowedTools Edit,Read,Grep,Glob,Bash
"""
from __future__ import annotations
import argparse
import hashlib
import json
import re
import sys
import urllib.request
import urllib.parse
from pathlib import Path
from typing import TypedDict
STATE_FILE = Path(__file__).parent / ".wiki_sync_state.json"
WIKI_PAGE = "Wikipedia:Signs_of_AI_writing"
API_URL = "https://en.wikipedia.org/w/api.php"
class SyncState(TypedDict):
last_revision_id: int
last_timestamp: str
content_hash: str
last_content_snapshot: str
class ParsedSection(TypedDict):
title: str
level: int
content: str
watch_words: list[str]
examples: list[str]
class Change(TypedDict):
type: str # "new_section", "new_words", "modified", "removed"
section: str
details: str
words: list[str]
def fetch_latest_revision() -> tuple[int, str, str]:
"""Fetch the latest revision content from Wikipedia.
Returns (revision_id, timestamp, wikitext).
"""
params = {
"action": "query",
"titles": WIKI_PAGE,
"prop": "revisions",
"rvslots": "*",
"rvprop": "ids|timestamp|content",
"format": "json",
"formatversion": "2",
}
url = f"{API_URL}?{urllib.parse.urlencode(params)}"
req = urllib.request.Request(url, headers={"User-Agent": "unslop-wiki-sync/1.0"})
with urllib.request.urlopen(req, timeout=30) as resp:
data = json.loads(resp.read().decode("utf-8"))
page = data["query"]["pages"][0]
if "missing" in page:
print(f"Error: Page '{WIKI_PAGE}' not found.", file=sys.stderr)
sys.exit(2)
revision = page["revisions"][0]
rev_id = revision["revid"]
timestamp = revision["timestamp"]
content = revision["slots"]["main"]["content"]
return rev_id, timestamp, content
def get_wikitext(from_file: str | None) -> tuple[int, str, str]:
"""Return (revision_id, timestamp, wikitext).
With `from_file` set, read wikitext from that local path instead of
hitting the network — used for offline eval/fixture runs. There is no
real revision for a local file, so revision_id/timestamp are placeholders.
"""
if from_file:
content = Path(from_file).read_text(encoding="utf-8")
return 0, "from-file", content
return fetch_latest_revision()
def load_state() -> SyncState | None:
if not STATE_FILE.exists():
return None
with open(STATE_FILE, "r") as f:
return json.load(f)
def save_state(rev_id: int, timestamp: str, content: str) -> None:
state: SyncState = {
"last_revision_id": rev_id,
"last_timestamp": timestamp,
"content_hash": hashlib.sha256(content.encode()).hexdigest(),
"last_content_snapshot": content,
}
with open(STATE_FILE, "w") as f:
json.dump(state, f, indent=2)
def parse_wikitext(wikitext: str) -> list[ParsedSection]:
"""Parse wikitext into structured sections with watch words and examples."""
sections: list[ParsedSection] = []
current_title = "Introduction"
current_level = 1
current_lines: list[str] = []
for line in wikitext.split("\n"):
header_match = re.match(r"^(={2,6})\s*(.+?)\s*\1\s*$", line)
if header_match:
if current_lines:
sections.append(_build_section(current_title, current_level, current_lines))
current_level = len(header_match.group(1))
current_title = header_match.group(2).strip()
current_lines = []
else:
current_lines.append(line)
if current_lines:
sections.append(_build_section(current_title, current_level, current_lines))
return sections
def _build_section(title: str, level: int, lines: list[str]) -> ParsedSection:
"""Build a ParsedSection from raw lines."""
content = "\n".join(lines).strip()
watch_words = _extract_watch_words(content)
examples = _extract_examples(content)
return {
"title": title,
"level": level,
"content": content,
"watch_words": watch_words,
"examples": examples,
}
def _extract_watch_words(content: str) -> list[str]:
"""Extract words-to-watch from tmbox templates and bold/italic markers."""
words: list[str] = []
# Match {{tmbox|text=...}} content
for match in re.finditer(r"\{\{tmbox\|[^}]*text\s*=\s*([^}]+)\}\}", content, re.DOTALL):
text = match.group(1)
# Extract bold words within tmbox
for bold in re.finditer(r"'''(.+?)'''", text):
words.append(bold.group(1).strip())
# Match bold words in "words to watch" style lists
for match in re.finditer(r"'''(.+?)'''", content):
word = match.group(1).strip()
if word and len(word) < 50:
words.append(word)
# Match items in bulleted lists
for match in re.finditer(r"^\*\s*'''(.+?)'''", content, re.MULTILINE):
words.append(match.group(1).strip())
return list(dict.fromkeys(words)) # dedupe preserving order
def _extract_examples(content: str) -> list[str]:
"""Extract example blocks (blockquotes, textdiff templates)."""
examples: list[str] = []
# {{blockquote|...}}
for match in re.finditer(r"\{\{blockquote\|([^}]+)\}\}", content, re.DOTALL):
examples.append(match.group(1).strip())
# {{textdiff|old=...|new=...}}
for match in re.finditer(
r"\{\{textdiff\|[^}]*old\s*=\s*([^|]+)\|[^}]*new\s*=\s*([^}]+)\}\}",
content, re.DOTALL
):
examples.append(f"BEFORE: {match.group(1).strip()}\nAFTER: {match.group(2).strip()}")
# Indented blockquote lines (: prefix in wikitext)
block_lines: list[str] = []
for line in content.split("\n"):
if line.startswith(":"):
block_lines.append(line.lstrip(": "))
elif block_lines:
examples.append("\n".join(block_lines))
block_lines = []
if block_lines:
examples.append("\n".join(block_lines))
return examples
def compute_diff(old_sections: list[ParsedSection], new_sections: list[ParsedSection]) -> list[Change]:
"""Compute structured diff between two parsed page versions."""
changes: list[Change] = []
old_by_title = {s["title"]: s for s in old_sections}
new_by_title = {s["title"]: s for s in new_sections}
for title, new_sec in new_by_title.items():
if title not in old_by_title:
changes.append({
"type": "new_section",
"section": title,
"details": f"New section with {len(new_sec['watch_words'])} watch words",
"words": new_sec["watch_words"],
})
else:
old_sec = old_by_title[title]
new_words = set(new_sec["watch_words"]) - set(old_sec["watch_words"])
if new_words:
changes.append({
"type": "new_words",
"section": title,
"details": f"{len(new_words)} new watch words added",
"words": sorted(new_words),
})
old_hash = hashlib.sha256(old_sec["content"].encode()).hexdigest()
new_hash = hashlib.sha256(new_sec["content"].encode()).hexdigest()
if old_hash != new_hash and not new_words:
changes.append({
"type": "modified",
"section": title,
"details": "Section content changed (no new watch words)",
"words": [],
})
for title in old_by_title:
if title not in new_by_title:
changes.append({
"type": "removed",
"section": title,
"details": "Section removed from Wikipedia page",
"words": old_by_title[title]["watch_words"],
})
return changes
# Mapping from Wikipedia section themes to unslop target files/sections
SECTION_MAP: dict[str, dict[str, str]] = {
"words to watch": {
"file": "references/taboo-phrases.md",
"section": "AI Vocabulary (Additional)",
},
"promotional": {
"file": "references/taboo-phrases.md",
"section": "Promotional Language",
},
"significance": {
"file": "references/taboo-phrases.md",
"section": "Significance & Legacy Inflation",
},
"legacy": {
"file": "references/taboo-phrases.md",
"section": "Significance & Legacy Inflation",
},
"vague": {
"file": "references/taboo-phrases.md",
"section": "Vague Attributions",
},
"attribution": {
"file": "references/taboo-phrases.md",
"section": "Vague Attributions",
},
"copula": {
"file": "references/taboo-phrases.md",
"section": "Copula Avoidance",
},
"participial": {
"file": "references/taboo-phrases.md",
"section": "Superficial -ing Analyses",
},
"chatbot": {
"file": "references/taboo-phrases.md",
"section": "Communication Artifacts",
},
"jargon": {
"file": "references/taboo-phrases.md",
"section": "Business Jargon",
},
"structure": {
"file": "references/taboo-phrases.md",
"section": "Structural Patterns to Avoid",
},
"synonym": {
"file": "references/taboo-phrases.md",
"section": "Elegant Variation / Synonym Cycling",
},
}
def map_change_to_target(change: Change) -> dict[str, str]:
"""Map a Wikipedia change to the appropriate unslop file and section."""
section_lower = change["section"].lower()
for keyword, target in SECTION_MAP.items():
if keyword in section_lower:
return target
return {
"file": "references/taboo-phrases.md",
"section": "AI Vocabulary (Additional)",
}
def generate_prompt(changes: list[Change], sections: list[ParsedSection]) -> str:
"""Generate a Claude Code integration prompt from detected changes."""
if not changes:
return "No changes detected since last sync. Nothing to integrate."
lines = [
"# Wikipedia AI Writing Patterns — Integration Update",
"",
"The Wikipedia 'Signs of AI writing' page has been updated.",
"Below are changes that need to be integrated into the unslop skill.",
"",
"## Changes Detected",
"",
]
for i, change in enumerate(changes, 1):
target = map_change_to_target(change)
lines.append(f"### {i}. [{change['type'].upper()}] {change['section']}")
lines.append(f"")
lines.append(f"**Details:** {change['details']}")
lines.append(f"**Target file:** `{target['file']}`")
lines.append(f"**Target section:** {target['section']}")
if change["words"]:
lines.append(f"**Words/phrases to add:**")
for word in change["words"]:
lines.append(f"- \"{word}\"")
lines.append("")
lines.extend([
"## Integration Instructions",
"",
"For each change above:",
"",
"1. Read the target file",
"2. Check if the phrase already exists (avoid duplicates)",
"3. Add new phrases to the appropriate section in `references/taboo-phrases.md`",
"4. Mirror additions in `scripts/banned_phrase_scan.py` BANNED_PHRASES dict",
"5. If a new before/after example is warranted, add to `references/edit-library.md`",
"",
"After all changes:",
"- Run `python3 scripts/banned_phrase_scan.py < /dev/null` to verify no syntax errors",
"- Verify no duplicate entries in taboo-phrases.md",
])
return "\n".join(lines)
def cmd_check(from_file: str | None = None) -> None:
"""Check for updates. Exit 0 = no updates, 1 = updates available."""
rev_id, timestamp, content = get_wikitext(from_file)
state = load_state()
content_hash = hashlib.sha256(content.encode()).hexdigest()
if state is None:
print(f"No previous sync state. Current revision: {rev_id} ({timestamp})")
print("Run 'diff' or 'prompt' to see all content as new.")
sys.exit(1)
if content_hash == state["content_hash"]:
print(f"No changes. Current revision: {rev_id} ({timestamp})")
print(f"Last synced revision: {state['last_revision_id']} ({state['last_timestamp']})")
sys.exit(0)
print(f"Updates available!")
print(f" Last synced: revision {state['last_revision_id']} ({state['last_timestamp']})")
print(f" Current: revision {rev_id} ({timestamp})")
sys.exit(1)
def cmd_diff(from_file: str | None = None) -> None:
"""Output structured JSON diff of changes."""
rev_id, timestamp, content = get_wikitext(from_file)
state = load_state()
new_sections = parse_wikitext(content)
if state is None:
old_sections: list[ParsedSection] = []
else:
old_sections = parse_wikitext(state["last_content_snapshot"])
changes = compute_diff(old_sections, new_sections)
output = {
"revision_id": rev_id,
"timestamp": timestamp,
"previous_revision_id": state["last_revision_id"] if state else None,
"total_changes": len(changes),
"changes": changes,
}
print(json.dumps(output, indent=2))
if not from_file:
save_state(rev_id, timestamp, content)
def cmd_prompt(from_file: str | None = None) -> None:
"""Output a Claude Code integration prompt."""
rev_id, timestamp, content = get_wikitext(from_file)
state = load_state()
new_sections = parse_wikitext(content)
if state is None:
old_sections: list[ParsedSection] = []
else:
old_sections = parse_wikitext(state["last_content_snapshot"])
changes = compute_diff(old_sections, new_sections)
prompt = generate_prompt(changes, new_sections)
print(prompt)
if not from_file:
save_state(rev_id, timestamp, content)
def main() -> None:
parser = argparse.ArgumentParser(
description="Sync unslop rules with Wikipedia's 'Signs of AI writing' page."
)
subparsers = parser.add_subparsers(dest="command")
commands = {
"check": cmd_check,
"diff": cmd_diff,
"prompt": cmd_prompt,
}
for name in commands:
sub = subparsers.add_parser(name)
sub.add_argument(
"--from-file",
metavar="PATH",
help=(
"read wikitext from PATH instead of fetching from Wikipedia "
"(skips the network call and skips writing sync state)"
),
)
if len(sys.argv) < 2:
print("Usage: wiki_sync.py <check|diff|prompt>", file=sys.stderr)
sys.exit(2)
if sys.argv[1] not in commands:
print(f"Unknown command: {sys.argv[1]}", file=sys.stderr)
print(f"Available: {', '.join(commands)}", file=sys.stderr)
sys.exit(2)
args = parser.parse_args()
commands[args.command](args.from_file)
if __name__ == "__main__":
main()
+34
View File
@@ -0,0 +1,34 @@
---
name: manage-pi-links
description: Reconcile harness-tools' repository-managed Pi resources. Use in this repository when asked to check, sync, repair, install, or prune skill links, extension links, or local-path Pi package entries.
---
# Manage Pi Links
Treat [`config/pi-links.json`](../../../config/pi-links.json) as the source of truth and [`scripts/pi-links.mjs`](../../../scripts/pi-links.mjs) as the only mutation interface. Keep this skill repository-local; its own exposure is the relative `.pi/skills/manage-pi-links` link.
## Classify the request
- For inspection, status, or troubleshooting, run `./scripts/pi-links.mjs check` and report every drift or conflict.
- For sync, repair, or installation, run `./scripts/pi-links.mjs sync --dry-run` first. Apply with `./scripts/pi-links.mjs sync` only when the request authorizes global changes.
- For pruning, run `./scripts/pi-links.mjs prune --dry-run`, show every removal, and obtain explicit confirmation before running `./scripts/pi-links.mjs prune`.
- For a new resource, update the manifest classification before synchronization. Skills are discovered automatically; list repository-only skills under `skills.projectOnly` and intentionally unavailable skills under `skills.excluded`. List global instruction files under `instructions.globalSymlinks`. List global extension directories under `extensions.globalSymlinks` or extension files under `extensions.globalLocalPackages`.
## Reconcile
1. Run commands from the repository root.
2. Read the manifest entry and the resource's `SKILL.md` or `index.ts` when diagnosing a mismatch.
3. Resolve manifest validation errors before applying changes. Require unique skill names and existing entry points.
4. Preserve collisions. The manager refuses regular files, directories, and links whose targets are outside this repository; report these for a human decision instead of replacing them manually.
5. Apply the authorized operation through `scripts/pi-links.mjs`.
6. Run `./scripts/pi-links.mjs check` again. Completion requires exit code zero.
7. Tell the user to run `/reload` when skill or extension exposure changed in the active Pi session.
## Exposure conventions
- Link shared global instructions into Pi's `AGENTS.md`, Codex's `AGENTS.md`, and Claude's `CLAUDE.md`; use a Claude import file when Claude should reference the shared instructions rather than link to them directly.
- Link globally shared skills into `~/.agents/skills/` for cross-harness discovery.
- Link repository-only skills into `.pi/skills/` with relative targets.
- Link live-development Pi extensions into `~/.pi/agent/extensions/` with absolute targets.
- Register manifest-declared local package files through `pi install`; let the manager invoke Pi rather than editing `~/.pi/agent/settings.json` directly.
- Prune only repository-owned exposure discovered by the manager. Leave unrelated global resources untouched.