Fix fork-mode lifecycle test and default test model to gpt-5.6-luna
The fork test asked the model to call subagent with fork: true and no agent, which the tool schema has never supported - it only passed when a model improvised. It now spawns a real fork-mode agent (test-fork with session-mode: fork). The harness default model is now openai-codex/gpt-5.6-luna. The previous active-session-model fallback broke under pi -ne, where extension-registered providers do not resolve.
This commit is contained in:
@@ -0,0 +1,13 @@
|
||||
---
|
||||
name: test-fork
|
||||
description: Integration test agent — fork-mode child that inherits the parent session
|
||||
model: anthropic/claude-haiku-4-5
|
||||
tools: read, bash
|
||||
session-mode: fork
|
||||
auto-exit: true
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
You are a test agent. Complete the task given to you immediately. Be direct and concise.
|
||||
When asked to write content to a file, do it right away using the bash tool.
|
||||
Do not ask questions. Do not explain. Just execute the task.
|
||||
@@ -73,13 +73,14 @@ const EXTENSION_SOURCE = join(PROJECT_ROOT, "pi-extension", "subagents", "index.
|
||||
|
||||
// ── Configuration ──
|
||||
|
||||
/** Model used for integration tests. Override with PI_TEST_MODEL env var. */
|
||||
const activePiModel =
|
||||
process.env.PI_PROVIDER && process.env.PI_MODEL
|
||||
? `${process.env.PI_PROVIDER}/${process.env.PI_MODEL}`
|
||||
: undefined;
|
||||
export const TEST_MODEL =
|
||||
process.env.PI_TEST_MODEL ?? activePiModel ?? "anthropic/claude-haiku-4-5";
|
||||
/**
|
||||
* Model used for integration tests. Override with PI_TEST_MODEL env var.
|
||||
*
|
||||
* The default must be a provider that exists under `pi -ne`: startPi() launches
|
||||
* with extensions disabled, so extension-registered providers (and therefore
|
||||
* the parent session's active model) may not resolve in the child.
|
||||
*/
|
||||
export const TEST_MODEL = process.env.PI_TEST_MODEL ?? "openai-codex/gpt-5.6-luna";
|
||||
|
||||
/** Per-test timeout in ms. Override with PI_TEST_TIMEOUT env var. */
|
||||
export const PI_TIMEOUT = Number(process.env.PI_TEST_TIMEOUT ?? "120000");
|
||||
|
||||
@@ -1,18 +1,18 @@
|
||||
/**
|
||||
* Integration tests for the full subagent lifecycle.
|
||||
*
|
||||
* These tests spawn REAL pi sessions with REAL LLM calls (haiku by default).
|
||||
* These tests spawn REAL pi sessions with REAL LLM calls (gpt-5.6-luna by default).
|
||||
* Each test creates a terminal surface, runs pi with a task that uses the subagent
|
||||
* tool, and verifies the outcome via marker files and screen output.
|
||||
*
|
||||
* Costs: ~$0.01-0.05 per test run (haiku).
|
||||
* Costs: ~$0.01-0.05 per test run.
|
||||
* Duration: ~30-90s per test.
|
||||
*
|
||||
* Run inside Herdr, inside an Orca terminal, or inside tmux:
|
||||
* tmux new 'npm run test:integration'
|
||||
*
|
||||
* Configuration:
|
||||
* PI_TEST_MODEL — model for all pi sessions (default: anthropic/claude-haiku-4-5)
|
||||
* PI_TEST_MODEL — model for all pi sessions (default: openai-codex/gpt-5.6-luna)
|
||||
* PI_TEST_TIMEOUT — per-test timeout in ms (default: 120000)
|
||||
*/
|
||||
import { describe, it, before, after } from "node:test";
|
||||
@@ -253,9 +253,9 @@ for (const backend of backends) {
|
||||
const task = [
|
||||
`Call the subagent tool with these EXACT parameters:`,
|
||||
` name: "Fork-${id}"`,
|
||||
` fork: true`,
|
||||
` agent: "test-fork"`,
|
||||
` task: "Run this bash command: echo 'FORK_OK_${id}' > '${markerFile}'"`,
|
||||
`Do not set the agent parameter. Just set name, fork, and task.`,
|
||||
`Do not do anything else. Just call the subagent tool once.`,
|
||||
`After you receive the result, say FORK_COMPLETE.`,
|
||||
].join("\n");
|
||||
|
||||
|
||||
Reference in New Issue
Block a user