Skip unrelated required checks without losing their names (#2542)

* fix(ci): skip unrelated required checks cleanly

Matrix jobs had to start no-op runners to preserve interpolated required-check names. Static named jobs let GitHub report genuine skipped conclusions while keeping those names stable.

* fix(ci): gate changes to install patching script

* fix(ci): route cross-package test contracts narrowly

* fix(ci): keep routed contracts merge-blocking

* fix(server): exclude nested test utilities from builds

* fix(ci): cover app-owned native and browser domains

* refactor(ci): align tests with runtime domains

Own browser specs, desktop specs, and Electron-only app code by stable directories so PR routing does not depend on filename conventions. Keep desktop integration coverage inside the required desktop check.

* fix(ci): keep routing and desktop suites isolated

The routing regression must run before dependency installation, and desktop Vitest must not collect Playwright-owned specs.

* fix(ci): isolate desktop e2e runtime

* perf(test): isolate Playwright workers

* perf(test): parallelize Playwright within specs

* fix(test): keep Playwright file scheduling

* fix(test): share dynamic Metro fixture

* fix(test): preserve own-host restart fixtures

* fix(ci): preserve routed contracts after rebase

* Stabilize Playwright wheel-scroll assertions
This commit is contained in:
Mohamed Boudra
2026-08-02 00:56:05 +08:00
committed by GitHub
parent 429155b536
commit 5a30affd7f
256 changed files with 2693 additions and 2674 deletions
+17 -8
View File
@@ -124,14 +124,14 @@ PASEO_DESKTOP_SMOKE_ARTIFACT_DIR=/tmp/paseo-desktop-smoke \
npm run build:desktop -- --publish never --linux --x64 --dir
```
### Browser tab bridge regression
### Desktop browser regression
The desktop browser tab bridge E2E launches an isolated real daemon, Metro, and Electron app. It forces workspace LRU eviction to reparent the original tab and replace its guest `WebContents`, then makes one MCP call each for tab listing, snapshot, and click against that original browser id. A final MCP wait proves the real target page received the click.
The desktop browser E2E launches an isolated real daemon, Metro, and Electron app. It forces workspace LRU eviction to reparent the original tab and replace its guest `WebContents`, then makes one MCP call each for tab listing, snapshot, and click against that original browser id. A final MCP wait proves the real target page received the click.
Run it locally with the same command owned by the Ubuntu leg of the existing `desktop-tests` CI check:
Run it locally with the same command owned by the Ubuntu `desktop-tests` required check:
```bash
npm run test:e2e:browser-tab-bridge --workspace=@getpaseo/desktop
npm run test:e2e:browser-tabs --workspace=@getpaseo/desktop
```
## Test organization
@@ -154,7 +154,7 @@ Vitest picks up tests by suffix. The suffix tells the runner which category it b
| `*.real.e2e.test.ts` | E2E that hits a real provider (Claude/Codex/Copilot/OpenCode/Pi) — needs creds in `packages/server/.env.test` | `npm run test:integration:real` / `test:e2e:real` |
| `*.local.e2e.test.ts` | E2E that needs a local-only resource | `npm run test:integration:local` / `test:e2e:local` |
App-level Playwright browser E2E lives in `packages/app/e2e/*.spec.ts` and runs via `npm run test:e2e --workspace=@getpaseo/app` (separate from Vitest E2E). App Playwright specs that hit real providers use `*.real.spec.ts` and run through `npm run test:e2e:real --workspace=@getpaseo/app`; the default app E2E project ignores that suffix so CI does not need provider credentials.
Browser Playwright specs live in `packages/app/e2e/browser/`. Desktop Playwright and real-Electron E2E live in `packages/desktop/e2e/`. Harness code shared by both suites lives in `packages/app/e2e/support/`; neither suite may place specs there. App Playwright specs that hit real providers use `*.real.spec.ts` and run through `npm run test:e2e:real --workspace=@getpaseo/app`; the default browser project ignores that suffix so CI does not need provider credentials.
Live provider smoke tests belong in `*.real.e2e.test.ts`, not `*.test.ts`, even when guarded by environment variables. Default unit suites must use deterministic provider adapters/fakes so missing credits, auth outages, and upstream model drift do not block normal CI.
@@ -175,9 +175,18 @@ Test suites in this repo are heavy. Running them in bulk freezes the machine, es
- Never re-run a suite another agent already reported green.
- For full-suite confidence, push to CI and check GitHub Actions.
- Never run the full Playwright E2E suite locally — defer whole-suite verification to CI. Targeted Playwright specs are allowed when you changed or need to prove that specific flow.
- App Playwright specs share one isolated daemon per run. Helpers that create projects or workspaces must remove the daemon project record during cleanup, not only delete the temp directory. Agent helpers must pass the intended `workspaceId` through to agent creation; never infer ownership from `cwd`.
- CI can shard app Playwright across multiple jobs; each shard still owns a full isolated daemon/relay/Metro stack from global setup. Helpers that restart the daemon must preserve the global setup environment, including disabled speech/local-model settings, so a restart does not change the tested surface or start background downloads.
- Global setup starts Metro before Wrangler, assigns Wrangler explicit distinct relay and inspector ports, and accepts Metro as ready only when `/status` returns `packager-status:running`. A generic TCP listener is not sufficient readiness evidence.
- App Playwright shares one warmed Metro server per run and gives every Playwright worker its own isolated daemon and `PASEO_HOME`. Spec files run concurrently without exposing one file's projects, agents, terminals, history, or provider configuration to another worker; tests within a file remain together so file-level setup is not repeated.
- Helpers that create projects or workspaces own those records until cleanup. Their clients remove the daemon project on close, and an automatic fixture fails any test that still leaks a project record. Deleting only the temporary directory is not cleanup. Agent helpers pass the intended `workspaceId` through to agent creation; they never infer ownership from `cwd`.
- Tests whose subject is daemon-global state, such as an empty history or daemon restart, start a dedicated host explicitly. Filenames and directories describe product behavior, never execution order or isolation mechanics.
- Global setup accepts Metro as ready only when `/status` returns `packager-status:running`, then fetches the document's scripts so the cold bundle compilation finishes before Playwright's per-test timeout starts. A generic TCP listener is not sufficient readiness evidence. The browser suite uses direct local daemon connections and does not start a relay.
## Pull-request test routing
PR checks are routed by the behavior each suite proves, using `.github/ci-paths.yml`. A package does not inherit every test suite of its runtime consumers: app changes do not run CLI or Electron-wrapper tests, and protocol changes do not run every package that imports the protocol. Cross-package static compatibility belongs to `typecheck`; full integration coverage runs after merge on main and in manual CI runs.
Required matrix legs are declared as statically named jobs. Their shared steps use YAML anchors, while job-level `if` conditions let GitHub report an unaffected leg as genuinely skipped without allocating a runner or losing the exact required-check name.
The smallest meaningful contract wins over package ownership. Tiny structural invariants such as daemon launch supervision run unconditionally in the always-running routing job instead of maintaining a transitive file list; this check reads source entrypoints and builds no product. Routed integration contracts use stable domain directories. Browser changes select the required Playwright shards; desktop changes select the existing required desktop jobs, with renderer, real-Electron, and packaged-app coverage together in the Ubuntu leg. CLI-side Hub changes select one focused test inside the existing required server jobs. Repository scripts and the shared Vitest configuration run every PR contract because they are cross-cutting toolchain inputs.
## Agent authentication in tests