Fix fork-mode lifecycle test and default test model to gpt-5.6-luna

The fork test asked the model to call subagent with fork: true and no
agent, which the tool schema has never supported - it only passed when a
model improvised. It now spawns a real fork-mode agent (test-fork with
session-mode: fork).

The harness default model is now openai-codex/gpt-5.6-luna. The previous
active-session-model fallback broke under pi -ne, where extension-registered
providers do not resolve.
This commit is contained in:
jb
2026-08-30 15:55:15 -04:00
parent 208cd6f4e7
commit 0b7af601a3
3 changed files with 26 additions and 12 deletions
+13
View File
@@ -0,0 +1,13 @@
---
name: test-fork
description: Integration test agent — fork-mode child that inherits the parent session
model: anthropic/claude-haiku-4-5
tools: read, bash
session-mode: fork
auto-exit: true
disable-model-invocation: true
---
You are a test agent. Complete the task given to you immediately. Be direct and concise.
When asked to write content to a file, do it right away using the bash tool.
Do not ask questions. Do not explain. Just execute the task.
+8 -7
View File
@@ -73,13 +73,14 @@ const EXTENSION_SOURCE = join(PROJECT_ROOT, "pi-extension", "subagents", "index.
// ── Configuration ──
/** Model used for integration tests. Override with PI_TEST_MODEL env var. */
const activePiModel =
process.env.PI_PROVIDER && process.env.PI_MODEL
? `${process.env.PI_PROVIDER}/${process.env.PI_MODEL}`
: undefined;
export const TEST_MODEL =
process.env.PI_TEST_MODEL ?? activePiModel ?? "anthropic/claude-haiku-4-5";
/**
* Model used for integration tests. Override with PI_TEST_MODEL env var.
*
* The default must be a provider that exists under `pi -ne`: startPi() launches
* with extensions disabled, so extension-registered providers (and therefore
* the parent session's active model) may not resolve in the child.
*/
export const TEST_MODEL = process.env.PI_TEST_MODEL ?? "openai-codex/gpt-5.6-luna";
/** Per-test timeout in ms. Override with PI_TEST_TIMEOUT env var. */
export const PI_TIMEOUT = Number(process.env.PI_TEST_TIMEOUT ?? "120000");
+5 -5
View File
@@ -1,18 +1,18 @@
/**
* Integration tests for the full subagent lifecycle.
*
* These tests spawn REAL pi sessions with REAL LLM calls (haiku by default).
* These tests spawn REAL pi sessions with REAL LLM calls (gpt-5.6-luna by default).
* Each test creates a terminal surface, runs pi with a task that uses the subagent
* tool, and verifies the outcome via marker files and screen output.
*
* Costs: ~$0.01-0.05 per test run (haiku).
* Costs: ~$0.01-0.05 per test run.
* Duration: ~30-90s per test.
*
* Run inside Herdr, inside an Orca terminal, or inside tmux:
* tmux new 'npm run test:integration'
*
* Configuration:
* PI_TEST_MODEL — model for all pi sessions (default: anthropic/claude-haiku-4-5)
* PI_TEST_MODEL — model for all pi sessions (default: openai-codex/gpt-5.6-luna)
* PI_TEST_TIMEOUT — per-test timeout in ms (default: 120000)
*/
import { describe, it, before, after } from "node:test";
@@ -253,9 +253,9 @@ for (const backend of backends) {
const task = [
`Call the subagent tool with these EXACT parameters:`,
` name: "Fork-${id}"`,
` fork: true`,
` agent: "test-fork"`,
` task: "Run this bash command: echo 'FORK_OK_${id}' > '${markerFile}'"`,
`Do not set the agent parameter. Just set name, fork, and task.`,
`Do not do anything else. Just call the subagent tool once.`,
`After you receive the result, say FORK_COMPLETE.`,
].join("\n");