diff --git a/docs/architecture.md b/docs/architecture.md index ff920a58d..167e74737 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -141,8 +141,22 @@ Electron wrapper for macOS, Linux, and Windows. > **In-app browser profile.** Every browser guest uses one stable persistent Electron session, so cookies, authentication, cache, and site storage are shared across tabs, workspaces, and desktop windows and survive tab or app closure. Browser identity is independent of that storage partition: after `did-attach`, the renderer explicitly registers its browser id, workspace id, and guest `WebContents` id, and main accepts the registration only when that guest belongs to the calling renderer and the shared profile. Settings > General > Clear browser data is the sole profile-deletion path; it clears the shared session and reloads live guests without deleting saved tabs or URLs. > > **In-app browser window opens.** Ordinary link opens, including Shift-clicked links, become Paseo workspace tabs. Script-created opens with popup features or a named window target and POST-backed opens remain secured Electron child windows in the shared browser profile, preserving `window.opener`, `postMessage`, named-window reuse, request bodies, and `window.close()` for OAuth, payment, and similar popup protocols. Unsupported URL schemes are denied before either path. +> +> **In-app browser ownership.** Each registered guest records its owning host window. The active browser is keyed by `(host window, workspace)`, and application-menu Reload / Force Reload resolve only within the window Electron supplies to the menu callback. A non-null active update must name a browser owned by that host; a null update clears only that host/workspace. Browser automation continues to target explicit browser ids returned by `browser_new_tab` or `browser_list_tabs`. +> +> **Browser keyboard boundary.** Guest pages receive renderer-published shortcuts first. `Cmd/Ctrl+L` and `Cmd/Ctrl+R` are explicit guest-shell reservations; ordinary Paseo shortcuts run only after the page declines them. The sandboxed guest preload runs in every frame so focused iframes use the same boundary, while Node integration remains disabled. Human guest input disables Electron's menu fallback for plain keys. Agent-generated keys use guest `sendInputEvent` with `skipIfUnhandled`, so an unhandled Enter stops at the guest instead of reaching the host composer. Main selects the preload; it exposes no APIs to guest pages. -> **In-app browser targets are not yet per-window.** Browser webviews are still tracked by one process-global registry that keeps a single current `WebContents` per browser id. Human focus records the workspace-active browser for UI state and `list_tabs` reporting, while agent automation targets explicit browser ids returned by `browser_new_tab` or `browser_list_tabs`. Explicit attached-guest registration prevents concurrent windows from swapping different browser ids, but rendering the same saved browser tab in multiple windows can still make menu actions target the most recently registered guest. Making the registry window-scoped remains a follow-up. +```text +Human key -> guest WebContents + |-- Cmd/Ctrl+T/L/R ----------> reserved browser-shell action + `-- page keydown + |-- page prevents ------> page owns it + `-- published shortcut -> guest preload -> IPC(browserId) -> Paseo resolver + +Agent browser_keypress -> guest sendInputEvent(skipIfUnhandled) + |-- guest handles ------------> page owns it + `-- guest does not handle ----> stop; never redispatch to the host window +``` ### `packages/website` — Marketing site diff --git a/docs/browser-capture-harness.md b/docs/browser-capture-harness.md index 5907b1a76..983801c7e 100644 --- a/docs/browser-capture-harness.md +++ b/docs/browser-capture-harness.md @@ -13,7 +13,16 @@ It validates the compositor behavior that unit tests cannot see: - both viewport `capturePage` and full-page CDP screenshots return real pixels from the permanent production parking state; - guest background throttling can be disabled once at attach without per-capture - renderer coordination. + renderer coordination; +- the real-Electron host-composer sentinel proves guest Enter cannot submit a focused + host composer; +- the automation group loads the compiled production keyboard boundary and guest + preload, then proves that initial page window handlers get first refusal, unhandled + shortcuts synchronously suppress editable browser defaults before crossing the host + boundary, shortcuts marked unavailable in editable targets retain the browser field's + native behavior, handlers registered after preload still get first refusal, focused + iframes share the same boundary, digit wildcard shortcuts cross, and background automation + stays in the guest. Run it with the repo Electron: @@ -21,9 +30,11 @@ Run it with the repo Electron: npm run capture-harness --workspace=@getpaseo/desktop ``` -Run the browser automation fixture with: +Build the desktop main process before the automation group so its production guest +preload is available: ```bash +npm run build:main --workspace=@getpaseo/desktop PASEO_CAPTURE_HARNESS_GROUP=automation npm run capture-harness --workspace=@getpaseo/desktop ``` @@ -43,6 +54,9 @@ ARIA-like snapshot text includes headings, static text, and controls; refs survi `pushState` when the element still matches; same-URL rerenders stale old refs; and a file-input ref can be resolved to a CDP backend node id for upload. It also verifies page-context evaluation, including passing a resolved ref element as the function argument. +Keyboard containment runs last because the host-composer sentinel intentionally leaves +native focus in the host. It reuses an existing fixture button: adding a test-only control +changes the inline fixture geometry exercised by the earlier actionability checks. On macOS the harness process must set `app.setActivationPolicy("accessory")` and hide the Dock icon before creating any window. `showInactive()` only prevents window diff --git a/packages/app/src/components/browser-webview-resident.browser.test.ts b/packages/app/src/components/browser-webview-resident.browser.test.ts index cca532a44..c4f4de663 100644 --- a/packages/app/src/components/browser-webview-resident.browser.test.ts +++ b/packages/app/src/components/browser-webview-resident.browser.test.ts @@ -3,11 +3,16 @@ import { type BrowserWebviewProfileHost, clearResidentBrowserWebviewsForTests, ensureResidentBrowserWebview, + getResidentBrowserWebview, prepareBrowserWebview, releaseResidentBrowserWebview, removeResidentBrowserWebview, takeResidentBrowserWebview, } from "./browser-webview-resident"; +import { + setCommandCenterFocusRestoreElement, + takeCommandCenterFocusRestoreElement, +} from "../utils/command-center-focus-restore"; const RESIDENT_HOST_ID = "paseo-browser-resident-webviews"; const attachedBrowsers: Array<{ @@ -254,6 +259,20 @@ describe("resident browser webviews", () => { expect(document.getElementById(RESIDENT_HOST_ID)).toBeNull(); }); + it("finds the originating browser webview for focus restoration", () => { + const webview = ensureTestBrowser({ + browserId: "browser-focus", + workspaceId: "workspace-focus", + url: "https://example.com", + }); + + setCommandCenterFocusRestoreElement(getResidentBrowserWebview("browser-focus")); + + expect(takeCommandCenterFocusRestoreElement()).toBe(webview); + + expect(getResidentBrowserWebview("browser-missing")).toBeNull(); + }); + it("removes a resident webview when its browser tab closes", () => { const webview = ensureTestBrowser({ browserId: "browser-closed", diff --git a/packages/app/src/components/browser-webview-resident.ts b/packages/app/src/components/browser-webview-resident.ts index 7dfd7c701..a97190018 100644 --- a/packages/app/src/components/browser-webview-resident.ts +++ b/packages/app/src/components/browser-webview-resident.ts @@ -225,6 +225,19 @@ export function ensureResidentBrowserWebview(input: { return webview; } +export function getResidentBrowserWebview(browserId: string): HTMLElement | null { + const normalizedBrowserId = trimNonEmpty(browserId); + if (!normalizedBrowserId) { + return null; + } + const resident = residentWebviewsByBrowserId.get(normalizedBrowserId) ?? null; + if (resident?.isConnected) { + return resident; + } + const ownerDocument = readDocument(); + return ownerDocument ? findBrowserWebview(normalizedBrowserId, ownerDocument) : null; +} + export function takeResidentBrowserWebview(browserId: string): HTMLElement | null { const normalizedBrowserId = trimNonEmpty(browserId); if (!normalizedBrowserId) { diff --git a/packages/app/src/desktop/host.ts b/packages/app/src/desktop/host.ts index bc26b91c5..cb750af12 100644 --- a/packages/app/src/desktop/host.ts +++ b/packages/app/src/desktop/host.ts @@ -1,5 +1,6 @@ import { Platform } from "react-native"; import { getElectronHost } from "@/desktop/electron/host"; +import type { BrowserKeyboardPolicy } from "@/keyboard/browser-shortcuts"; import type { SessionInboundMessage, SessionOutboundMessage } from "@getpaseo/protocol/messages"; type BrowserAutomationExecuteRequest = Extract< @@ -122,10 +123,9 @@ export interface DesktopEventsBridge { on?: (event: string, handler: (payload: unknown) => void) => Promise<() => void> | (() => void); } -export interface DesktopBrowserShortcutEvent { - browserId?: string; - action: "focus-url"; -} +export type DesktopBrowserShortcutEvent = + | { browserId?: string; action: "focus-url" } + | { browserId: string; action: "new-tab" }; export interface DesktopBrowserNewTabRequestEvent { sourceBrowserId: string; @@ -139,6 +139,7 @@ export interface DesktopAttachedBrowserRegistration { } export interface DesktopBrowserBridge { + setShortcutPolicy?: (input: BrowserKeyboardPolicy) => Promise; readonly profilePartition?: string; registerAttachedBrowser?: (input: DesktopAttachedBrowserRegistration) => Promise; unregisterWorkspaceBrowser?: (browserId: string) => Promise; diff --git a/packages/app/src/hooks/use-keyboard-shortcuts.ts b/packages/app/src/hooks/use-keyboard-shortcuts.ts index de6e370c4..ce6155a5a 100644 --- a/packages/app/src/hooks/use-keyboard-shortcuts.ts +++ b/packages/app/src/hooks/use-keyboard-shortcuts.ts @@ -1,17 +1,25 @@ -import { useEffect, useMemo, useRef } from "react"; +import { useCallback, useEffect, useMemo, useRef } from "react"; import { usePathname, useRouter } from "expo-router"; import { getIsElectronRuntime } from "@/constants/layout"; import { useKeyboardShortcutsStore } from "@/stores/keyboard-shortcuts-store"; import { setCommandCenterFocusRestoreElement } from "@/utils/command-center-focus-restore"; +import { getResidentBrowserWebview } from "@/components/browser-webview-resident"; import { navigateToWorkspace } from "@/stores/navigation-active-workspace-store"; import { keyboardActionDispatcher } from "@/keyboard/keyboard-action-dispatcher"; import { type ChordState, + type KeyboardShortcutInput, resolveKeyboardShortcut, buildEffectiveBindings, getWorkspaceIndexJumpModifierKey, } from "@/keyboard/keyboard-shortcuts"; import { resolveKeyboardFocusScope } from "@/keyboard/focus-scope"; +import { + buildBrowserKeyboardPolicy, + parseBrowserShortcutInput, + shouldPublishBrowserShortcutPolicy, +} from "@/keyboard/browser-shortcuts"; +import type { KeyboardFocusScope, KeyboardShortcutPayload } from "@/keyboard/actions"; import { routeKeyboardShortcut, type ShortcutAction, @@ -47,6 +55,8 @@ export function useKeyboardShortcuts({ const resetModifiers = useKeyboardShortcutsStore((s) => s.resetModifiers); const { overrides } = useKeyboardShortcutOverrides(); const bindings = useMemo(() => buildEffectiveBindings(overrides), [overrides]); + const isDesktopApp = getIsElectronRuntime(); + const isMac = getShortcutOs() === "mac"; const chordStateRef = useRef({ candidateIndices: [], step: 0, @@ -56,20 +66,41 @@ export function useKeyboardShortcuts({ const activeWorkspaceSelection = useActiveWorkspaceSelection(); const keyboardWorkspaceSelectionRef = useRef(null); + const publishBrowserShortcutPolicy = useCallback( + (chordState?: ChordState) => { + const policy = + enabled && !isMobile + ? buildBrowserKeyboardPolicy({ + bindings, + chordState, + isMac, + isDesktop: isDesktopApp, + }) + : { menuPrefixes: [], prefixes: [] }; + void getDesktopHost()?.browser?.setShortcutPolicy?.(policy); + }, + [bindings, enabled, isDesktopApp, isMac, isMobile], + ); + useEffect(() => { if (activeWorkspaceSelection) { keyboardWorkspaceSelectionRef.current = activeWorkspaceSelection; } }, [activeWorkspaceSelection]); + useEffect(() => { + if (!isDesktopApp) { + return; + } + + publishBrowserShortcutPolicy(); + }, [isDesktopApp, publishBrowserShortcutPolicy]); + useEffect(() => { if (!enabled) return; if (isNative) return; if (isMobile) return; - const isDesktopApp = getIsElectronRuntime(); - const isMac = getShortcutOs() === "mac"; - // Only the modifier that actually performs the workspace-index jump on this // runtime should reveal the sidebar number badges (Alt on web, Cmd on // desktop Mac, Ctrl on desktop non-Mac). The store ORs altDown/cmdOrCtrlDown @@ -106,7 +137,11 @@ export function useKeyboardShortcuts({ "cycle-theme": cycleTheme, }; - const performShortcutAction = (action: ShortcutAction, event: KeyboardEvent): boolean => { + const performShortcutAction = ( + action: ShortcutAction, + event: KeyboardEvent | null, + browserFocusRestoreElement: HTMLElement | null = null, + ): boolean => { switch (action.kind) { case "none": return false; @@ -138,7 +173,11 @@ export function useKeyboardShortcuts({ return true; case "command-center-toggle": { if (action.nextOpen) { - captureCommandCenterFocusRestore(event); + if (event) { + captureCommandCenterFocusRestore(event); + } else { + setCommandCenterFocusRestoreElement(browserFocusRestoreElement); + } } useKeyboardShortcutsStore.getState().setCommandCenterOpen(action.nextOpen); return true; @@ -149,6 +188,98 @@ export function useKeyboardShortcuts({ } }; + const routeAndPerformShortcut = (input: { + action: string; + payload: KeyboardShortcutPayload; + domEvent: KeyboardEvent | null; + browserFocusRestoreElement?: HTMLElement | null; + }): boolean => { + const store = useKeyboardShortcutsStore.getState(); + const shortcutAction = routeKeyboardShortcut( + { action: input.action, payload: input.payload }, + { + pathname, + isMobile, + sidebarShortcutTargets: store.sidebarShortcutWorkspaceTargets, + navigationActiveWorkspace: + keyboardWorkspaceSelectionRef.current ?? activeWorkspaceSelection, + commandCenterOpen: store.commandCenterOpen, + shortcutsDialogOpen: store.shortcutsDialogOpen, + }, + ); + return performShortcutAction( + shortcutAction, + input.domEvent, + input.browserFocusRestoreElement, + ); + }; + + const resolveAndPerformShortcut = (input: { + event: KeyboardShortcutInput; + focusScope: KeyboardFocusScope; + domEvent: KeyboardEvent | null; + browserFocusRestoreElement?: HTMLElement | null; + }) => { + const store = useKeyboardShortcutsStore.getState(); + const previousChordState = chordStateRef.current; + const result = resolveKeyboardShortcut({ + event: input.event, + context: { + isMac, + isDesktop: isDesktopApp, + focusScope: input.focusScope, + commandCenterOpen: store.commandCenterOpen, + }, + chordState: chordStateRef.current, + onChordReset: () => { + chordStateRef.current = { + candidateIndices: [], + step: 0, + timeoutId: null, + }; + publishBrowserShortcutPolicy(); + }, + bindings, + }); + + chordStateRef.current = result.nextChordState; + if ( + shouldPublishBrowserShortcutPolicy({ + isBrowserInput: "browserId" in input.event, + previousChordState, + nextChordState: result.nextChordState, + }) + ) { + publishBrowserShortcutPolicy(result.nextChordState); + } + + if (result.preventDefault && input.domEvent) { + input.domEvent.preventDefault(); + input.domEvent.stopPropagation(); + } + + if (!result.match) { + return; + } + + const handled = routeAndPerformShortcut({ + action: result.match.action, + payload: result.match.payload, + domEvent: input.domEvent, + browserFocusRestoreElement: input.browserFocusRestoreElement, + }); + if (!handled || !input.domEvent) { + return; + } + + if (result.match.preventDefault) { + input.domEvent.preventDefault(); + } + if (result.match.stopPropagation) { + input.domEvent.stopPropagation(); + } + }; + const handleKeyDown = (event: KeyboardEvent) => { if (!shouldHandle()) { return; @@ -180,60 +311,11 @@ export function useKeyboardShortcuts({ target: event.target, commandCenterOpen: store.commandCenterOpen, }); - const result = resolveKeyboardShortcut({ + resolveAndPerformShortcut({ event, - context: { - isMac, - isDesktop: isDesktopApp, - focusScope, - commandCenterOpen: store.commandCenterOpen, - }, - chordState: chordStateRef.current, - onChordReset: () => { - chordStateRef.current = { - candidateIndices: [], - step: 0, - timeoutId: null, - }; - }, - bindings, + focusScope, + domEvent: event, }); - - chordStateRef.current = result.nextChordState; - - if (result.preventDefault) { - event.preventDefault(); - event.stopPropagation(); - } - - if (!result.match) { - return; - } - - const shortcutAction = routeKeyboardShortcut( - { action: result.match.action, payload: result.match.payload }, - { - pathname, - isMobile, - sidebarShortcutTargets: store.sidebarShortcutWorkspaceTargets, - navigationActiveWorkspace: - keyboardWorkspaceSelectionRef.current ?? activeWorkspaceSelection, - commandCenterOpen: store.commandCenterOpen, - shortcutsDialogOpen: store.shortcutsDialogOpen, - }, - ); - - const handled = performShortcutAction(shortcutAction, event); - if (!handled) { - return; - } - - if (result.match.preventDefault) { - event.preventDefault(); - } - if (result.match.stopPropagation) { - event.stopPropagation(); - } }; const handleKeyUp = (event: KeyboardEvent) => { @@ -252,25 +334,20 @@ export function useKeyboardShortcuts({ window.addEventListener("blur", handleBlurOrHide); document.addEventListener("visibilitychange", handleBlurOrHide); - const forwardedKeySubscription = isElectronRuntime() - ? getDesktopHost()?.events?.on?.("browser-forwarded-key", (payload) => { - if (!payload || typeof payload !== "object") return; - const p = payload as Record; - if (typeof p.key !== "string") return; - window.dispatchEvent( - new KeyboardEvent("keydown", { - key: p.key, - code: typeof p.code === "string" ? p.code : "", - metaKey: p.meta === true, - ctrlKey: p.control === true, - shiftKey: p.shift === true, - altKey: p.alt === true, - bubbles: true, - }), - ); + const browserShortcutSubscription = isElectronRuntime() + ? getDesktopHost()?.events?.on?.("browser-shortcut-input", (payload) => { + const input = parseBrowserShortcutInput(payload); + if (!input) { + return; + } + resolveAndPerformShortcut({ + event: input, + focusScope: "browser", + domEvent: null, + browserFocusRestoreElement: getResidentBrowserWebview(input.browserId), + }); }) : null; - return () => { if (chordStateRef.current.timeoutId !== null) { clearTimeout(chordStateRef.current.timeoutId); @@ -284,10 +361,10 @@ export function useKeyboardShortcuts({ window.removeEventListener("keyup", handleKeyUp, true); window.removeEventListener("blur", handleBlurOrHide); document.removeEventListener("visibilitychange", handleBlurOrHide); - if (typeof forwardedKeySubscription === "function") { - forwardedKeySubscription(); + if (typeof browserShortcutSubscription === "function") { + browserShortcutSubscription(); } else { - void forwardedKeySubscription?.then((dispose) => dispose()); + void browserShortcutSubscription?.then((dispose) => dispose()); } }; }, [ @@ -295,9 +372,12 @@ export function useKeyboardShortcuts({ cycleTheme, enabled, activeWorkspaceSelection, + isDesktopApp, + isMac, isMobile, openProjectPickerAction, pathname, + publishBrowserShortcutPolicy, resetModifiers, router, toggleAgentList, diff --git a/packages/app/src/keyboard/actions.ts b/packages/app/src/keyboard/actions.ts index d0246594c..7f776cd0a 100644 --- a/packages/app/src/keyboard/actions.ts +++ b/packages/app/src/keyboard/actions.ts @@ -3,6 +3,7 @@ export type KeyboardFocusScope = | "message-input" | "command-center" | "editable" + | "browser" | "other"; export type MessageInputKeyboardActionKind = diff --git a/packages/app/src/keyboard/browser-shortcuts.test.ts b/packages/app/src/keyboard/browser-shortcuts.test.ts new file mode 100644 index 000000000..9dd323946 --- /dev/null +++ b/packages/app/src/keyboard/browser-shortcuts.test.ts @@ -0,0 +1,322 @@ +import { describe, expect, it } from "vitest"; +import { + buildBrowserKeyboardPolicy, + parseBrowserShortcutInput, + shouldPublishBrowserShortcutPolicy, +} from "./browser-shortcuts"; +import { buildEffectiveBindings, resolveKeyboardShortcut } from "./keyboard-shortcuts"; + +describe("buildBrowserKeyboardPolicy", () => { + it("publishes only chord starts while no browser chord is pending", () => { + const bindings = buildEffectiveBindings({ + "workspace-tab-new-ctrl-t-non-mac": "Ctrl+Y", + "workspace-terminal-new-ctrl-shift-t-non-mac": "Ctrl+F12 Ctrl+F11", + }); + + const policy = buildBrowserKeyboardPolicy({ bindings, isMac: false, isDesktop: true }); + + expect(policy.prefixes).toContainEqual({ + alt: false, + code: "KeyY", + control: true, + key: "y", + meta: false, + shift: false, + }); + expect(policy.prefixes).toContainEqual({ + alt: false, + code: "F12", + control: true, + meta: false, + shift: false, + }); + expect(policy.prefixes).not.toContainEqual({ + alt: false, + code: "F11", + control: true, + meta: false, + shift: false, + }); + }); + + it("publishes a chord continuation only after its browser start crosses the boundary", () => { + const bindings = buildEffectiveBindings({ + "workspace-terminal-new-ctrl-shift-t-non-mac": "Ctrl+F12 Ctrl+F11", + }); + const chordIndex = bindings.findIndex( + (binding) => binding.id === "workspace-terminal-new-ctrl-shift-t-non-mac", + ); + + const policy = buildBrowserKeyboardPolicy({ + bindings, + chordState: { candidateIndices: [chordIndex], step: 1, timeoutId: null }, + isMac: false, + isDesktop: true, + }); + + expect(policy.prefixes).toEqual([ + { + alt: false, + code: "F11", + control: true, + meta: false, + shift: false, + }, + ]); + expect(policy.menuPrefixes).toContainEqual({ + alt: false, + code: "KeyW", + control: true, + key: "w", + meta: false, + shift: false, + }); + + const result = resolveKeyboardShortcut({ + event: { + altKey: false, + code: "F11", + ctrlKey: true, + key: "F11", + metaKey: false, + repeat: false, + shiftKey: false, + }, + context: { + commandCenterOpen: false, + focusScope: "browser", + isDesktop: true, + isMac: false, + }, + chordState: { candidateIndices: [chordIndex], step: 1, timeoutId: null }, + onChordReset: () => undefined, + bindings, + }); + + expect(result.match?.action).toBe("workspace.terminal.new"); + }); + + it("rejects an entire chord when a continuation cannot cross the browser boundary", () => { + const bindings = buildEffectiveBindings({ + "settings-toggle-ctrl-comma-non-mac": "Ctrl+F10 F9", + }); + + const policy = buildBrowserKeyboardPolicy({ bindings, isMac: false, isDesktop: true }); + + expect(policy.prefixes).not.toContainEqual({ + alt: false, + code: "F10", + control: true, + meta: false, + shift: false, + }); + expect(policy.prefixes).not.toContainEqual({ + alt: false, + code: "F9", + control: false, + meta: false, + shift: false, + }); + }); + + it("publishes Mod bindings for the current shortcut platform", () => { + const bindings = buildEffectiveBindings({ + "workspace-tab-new-cmd-t-mac": "Mod+Y", + }); + + expect( + buildBrowserKeyboardPolicy({ bindings, isMac: true, isDesktop: true }).prefixes, + ).toContainEqual({ + alt: false, + code: "KeyY", + control: false, + key: "y", + meta: true, + shift: false, + }); + }); + + it("keeps Ctrl+W out of the window menu after the tab-close shortcut is remapped", () => { + const bindings = buildEffectiveBindings({ + "workspace-tab-close-current-ctrl-w-non-mac": "Ctrl+Y", + }); + + const policy = buildBrowserKeyboardPolicy({ bindings, isMac: false, isDesktop: true }); + + expect(policy.prefixes).not.toContainEqual({ + alt: false, + code: "KeyW", + control: true, + key: "w", + meta: false, + shift: false, + }); + expect(policy.menuPrefixes).toContainEqual({ + alt: false, + code: "KeyW", + control: true, + key: "w", + meta: false, + shift: false, + }); + }); + + it("leaves macOS browser back and forward shortcuts in the guest", () => { + const bindings = buildEffectiveBindings({}); + + const policy = buildBrowserKeyboardPolicy({ bindings, isMac: true, isDesktop: true }); + + for (const code of ["BracketLeft", "BracketRight"]) { + expect(policy.prefixes).not.toContainEqual({ + alt: false, + code, + control: false, + key: code === "BracketLeft" ? "[" : "]", + meta: true, + shift: false, + }); + } + }); + + it("marks editable-only exclusions for enforcement inside the guest", () => { + const bindings = buildEffectiveBindings({}); + const policy = buildBrowserKeyboardPolicy({ bindings, isMac: true, isDesktop: true }); + + expect(policy.prefixes).toContainEqual({ + alt: false, + code: "ArrowLeft", + control: false, + editable: false, + meta: true, + shift: true, + }); + }); + + it("does not publish plain browser keys", () => { + const bindings = buildEffectiveBindings({}); + const policy = buildBrowserKeyboardPolicy({ bindings, isMac: false, isDesktop: true }); + + expect(policy.prefixes).not.toContainEqual({ + alt: false, + code: "Enter", + control: false, + meta: false, + shift: false, + }); + expect(policy.prefixes).not.toContainEqual({ + alt: false, + code: "Slash", + control: false, + meta: false, + shift: true, + }); + }); + + it("publishes Cmd+B with its logical key for non-QWERTY layouts", () => { + const bindings = buildEffectiveBindings({}); + const policy = buildBrowserKeyboardPolicy({ bindings, isMac: true, isDesktop: true }); + + expect(policy.prefixes).toContainEqual({ + alt: false, + code: "KeyB", + control: false, + key: "b", + meta: true, + shift: false, + }); + }); + + it("publishes the physical code needed for macOS Option shortcuts", () => { + const bindings = buildEffectiveBindings({}); + const policy = buildBrowserKeyboardPolicy({ bindings, isMac: true, isDesktop: true }); + + expect(policy.prefixes).toContainEqual({ + alt: true, + code: "KeyT", + control: false, + key: "t", + meta: true, + shift: false, + }); + }); +}); + +describe("shouldPublishBrowserShortcutPolicy", () => { + it("restores the initial browser policy when a host key resets a pending chord", () => { + expect( + shouldPublishBrowserShortcutPolicy({ + isBrowserInput: false, + previousChordState: { candidateIndices: [1], step: 1, timeoutId: null }, + nextChordState: { candidateIndices: [], step: 0, timeoutId: null }, + }), + ).toBe(true); + }); +}); + +describe("parseBrowserShortcutInput", () => { + it("normalizes browser shortcut input without losing its identity", () => { + expect( + parseBrowserShortcutInput({ + browserId: "browser-1", + key: "t", + code: "KeyT", + meta: false, + control: true, + shift: false, + alt: false, + }), + ).toEqual({ + browserId: "browser-1", + key: "t", + code: "KeyT", + metaKey: false, + ctrlKey: true, + shiftKey: false, + altKey: false, + repeat: false, + }); + }); + + it("keeps browser identities exact", () => { + expect( + parseBrowserShortcutInput({ + browserId: " browser-1 ", + key: "t", + code: "KeyT", + meta: false, + control: true, + shift: false, + alt: false, + }), + ).toMatchObject({ browserId: " browser-1 " }); + }); + + it.each([ + { + name: "a missing browser identity", + payload: { + key: "t", + code: "KeyT", + meta: false, + control: true, + shift: false, + alt: false, + }, + }, + { + name: "a malformed repeat flag", + payload: { + browserId: "browser-1", + key: "t", + code: "KeyT", + meta: false, + control: true, + shift: false, + alt: false, + repeat: "yes", + }, + }, + ])("rejects $name", ({ payload }) => { + expect(parseBrowserShortcutInput(payload)).toBeNull(); + }); +}); diff --git a/packages/app/src/keyboard/browser-shortcuts.ts b/packages/app/src/keyboard/browser-shortcuts.ts new file mode 100644 index 000000000..003790802 --- /dev/null +++ b/packages/app/src/keyboard/browser-shortcuts.ts @@ -0,0 +1,211 @@ +import { + type ChordState, + matchesKeyboardShortcutContext, + type KeyboardShortcutInput, + type ParsedShortcutBinding, +} from "./keyboard-shortcuts"; +import type { KeyCombo } from "./shortcut-string"; + +export interface BrowserShortcutPrefix { + alt: boolean; + code: string; + codeFallback?: true; + control: boolean; + editable?: false; + key?: string; + meta: boolean; + repeat?: false; + shift: boolean; + shiftedKey?: string; +} + +export interface BrowserShortcutInput extends KeyboardShortcutInput { + browserId: string; +} + +export interface BrowserKeyboardPolicy { + menuPrefixes: BrowserShortcutPrefix[]; + prefixes: BrowserShortcutPrefix[]; +} + +interface BrowserShortcutPolicyInput { + bindings: readonly ParsedShortcutBinding[]; + chordState?: ChordState; + isMac: boolean; + isDesktop: boolean; +} + +export function shouldPublishBrowserShortcutPolicy(input: { + isBrowserInput: boolean; + nextChordState: ChordState; + previousChordState: ChordState; +}): boolean { + return ( + input.isBrowserInput || (input.previousChordState.step > 0 && input.nextChordState.step === 0) + ); +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +export function parseBrowserShortcutInput(value: unknown): BrowserShortcutInput | null { + if (!isRecord(value)) { + return null; + } + const { browserId, code, key } = value; + if (typeof browserId !== "string" || browserId.length === 0) { + return null; + } + if (typeof code !== "string" || typeof key !== "string") { + return null; + } + if ( + typeof value.alt !== "boolean" || + typeof value.control !== "boolean" || + typeof value.meta !== "boolean" || + typeof value.shift !== "boolean" || + (value.repeat !== undefined && typeof value.repeat !== "boolean") + ) { + return null; + } + + return { + browserId, + key, + code, + altKey: value.alt, + ctrlKey: value.control, + metaKey: value.meta, + shiftKey: value.shift, + repeat: value.repeat ?? false, + }; +} + +function prefixFromCombo( + combo: KeyCombo, + isMac: boolean, + editable: false | undefined, +): BrowserShortcutPrefix | null { + const prefix: BrowserShortcutPrefix = { + alt: combo.alt === true, + code: combo.code, + control: combo.ctrl === true || (!isMac && combo.mod === true), + meta: combo.meta === true || (isMac && combo.mod === true), + shift: combo.shift === true, + }; + if (combo.codeFallback === true) { + prefix.codeFallback = true; + } + if (editable === false) { + prefix.editable = false; + } + if (combo.key) { + prefix.key = combo.key; + } + if (combo.repeat === false) { + prefix.repeat = false; + } + if (combo.shiftedKey) { + prefix.shiftedKey = combo.shiftedKey; + } + return prefix.meta || prefix.control || prefix.alt ? prefix : null; +} + +function isBrowserNativeNavigationPrefix(prefix: BrowserShortcutPrefix, isMac: boolean): boolean { + return ( + isMac && + prefix.meta && + !prefix.control && + !prefix.alt && + !prefix.shift && + (prefix.code === "BracketLeft" || prefix.code === "BracketRight") + ); +} + +function canCrossBrowserBoundary(binding: ParsedShortcutBinding, isMac: boolean): boolean { + return binding.parsedChord.every((combo) => { + const prefix = prefixFromCombo(combo, isMac, binding.when?.editable); + return prefix !== null && !isBrowserNativeNavigationPrefix(prefix, isMac); + }); +} + +function prefixKey(prefix: BrowserShortcutPrefix): string { + return [ + prefix.code, + prefix.key ?? "", + prefix.shiftedKey ?? "", + prefix.codeFallback ?? "", + prefix.editable ?? "", + prefix.control, + prefix.meta, + prefix.alt, + prefix.shift, + prefix.repeat ?? "", + ].join(":"); +} + +function buildBrowserShortcutPrefixes(input: BrowserShortcutPolicyInput): BrowserShortcutPrefix[] { + const prefixes = new Map(); + const context = { + isMac: input.isMac, + isDesktop: input.isDesktop, + focusScope: "browser" as const, + commandCenterOpen: false, + }; + + const candidates = + input.chordState && input.chordState.step > 0 + ? input.chordState.candidateIndices + : input.bindings.map((_, index) => index); + const step = input.chordState?.step ?? 0; + + for (const index of candidates) { + const binding = input.bindings[index]; + if (!binding) { + continue; + } + if (!matchesKeyboardShortcutContext(binding.when, context)) { + continue; + } + if (!canCrossBrowserBoundary(binding, input.isMac)) { + continue; + } + const combo = binding.parsedChord[step]; + if (!combo) { + continue; + } + const prefix = prefixFromCombo(combo, input.isMac, binding.when?.editable); + if (!prefix) { + continue; + } + prefixes.set(prefixKey(prefix), prefix); + } + + return [...prefixes.values()]; +} + +export function buildBrowserKeyboardPolicy( + input: BrowserShortcutPolicyInput, +): BrowserKeyboardPolicy { + const idlePrefixes = buildBrowserShortcutPrefixes({ ...input, chordState: undefined }); + const prefixes = + input.chordState && input.chordState.step > 0 + ? buildBrowserShortcutPrefixes(input) + : idlePrefixes; + const menuPrefixes = [...idlePrefixes]; + if (!input.isMac) { + const closeWindowGuard: BrowserShortcutPrefix = { + alt: false, + code: "KeyW", + control: true, + key: "w", + meta: false, + shift: false, + }; + if (!menuPrefixes.some((prefix) => prefixKey(prefix) === prefixKey(closeWindowGuard))) { + menuPrefixes.push(closeWindowGuard); + } + } + return { menuPrefixes, prefixes }; +} diff --git a/packages/app/src/keyboard/keyboard-shortcuts.test.ts b/packages/app/src/keyboard/keyboard-shortcuts.test.ts index 5e1c905ec..496288bb8 100644 --- a/packages/app/src/keyboard/keyboard-shortcuts.test.ts +++ b/packages/app/src/keyboard/keyboard-shortcuts.test.ts @@ -7,10 +7,11 @@ import { resolveKeyboardShortcut, type ChordState, type KeyboardShortcutContext, + type KeyboardShortcutInput, type ParsedShortcutBinding, } from "./keyboard-shortcuts"; -function keyboardEvent(overrides: Partial): KeyboardEvent { +function keyboardInput(overrides: Partial): KeyboardShortcutInput { return { key: "", code: "", @@ -20,7 +21,7 @@ function keyboardEvent(overrides: Partial): KeyboardEvent { shiftKey: false, repeat: false, ...overrides, - } as KeyboardEvent; + }; } function shortcutContext( @@ -44,14 +45,14 @@ function initialChordState(): ChordState { } function resolveShortcut(input: { - event: Partial; + event: Partial; context?: Partial; chordState?: ChordState; onChordReset?: () => void; bindings?: readonly ParsedShortcutBinding[]; }) { return resolveKeyboardShortcut({ - event: keyboardEvent(input.event), + event: keyboardInput(input.event), context: shortcutContext(input.context), chordState: input.chordState ?? initialChordState(), onChordReset: input.onChordReset ?? (() => undefined), @@ -60,7 +61,7 @@ function resolveShortcut(input: { } function expectShortcutResolution(input: { - event: Partial; + event: Partial; context?: Partial; action: string; payload?: unknown; @@ -83,7 +84,7 @@ function expectShortcutResolution(input: { } function expectNoShortcutResolution(input: { - event: Partial; + event: Partial; context?: Partial; }) { const result = resolveShortcut({ @@ -98,7 +99,7 @@ function expectNoShortcutResolution(input: { interface MatchingShortcutCase { name: string; - event: Partial; + event: Partial; context?: Partial; action: string; payload?: unknown; @@ -108,7 +109,7 @@ interface MatchingShortcutCase { interface NonMatchingShortcutCase { name: string; - event: Partial; + event: Partial; context?: Partial; } @@ -548,6 +549,14 @@ describe("keyboard-shortcuts", () => { expect(secondResult.nextChordState).toEqual(initialChordState()); }); + it("resolves a browser-origin shortcut with browser focus instead of host focus", () => { + expectShortcutResolution({ + event: { key: "t", code: "KeyT", ctrlKey: true }, + context: { isDesktop: true, focusScope: "browser" }, + action: "workspace.tab.new", + }); + }); + it("schedules a chord reset timeout for advancing candidates", () => { vi.useFakeTimers(); diff --git a/packages/app/src/keyboard/keyboard-shortcuts.ts b/packages/app/src/keyboard/keyboard-shortcuts.ts index e67158475..60d0d2f77 100644 --- a/packages/app/src/keyboard/keyboard-shortcuts.ts +++ b/packages/app/src/keyboard/keyboard-shortcuts.ts @@ -18,6 +18,16 @@ export interface KeyboardShortcutContext { commandCenterOpen: boolean; } +export interface KeyboardShortcutInput { + key: string; + code: string; + altKey: boolean; + ctrlKey: boolean; + metaKey: boolean; + shiftKey: boolean; + repeat: boolean; +} + export interface KeyboardShortcutMatch { action: KeyboardActionId; payload: KeyboardShortcutPayload; @@ -1060,8 +1070,8 @@ export function buildEffectiveBindings(overrides: Record): Parse // --- Matching engine --- -function parseDigit(event: KeyboardEvent): number | null { - const code = event.code ?? ""; +function parseDigit(event: KeyboardShortcutInput): number | null { + const code = event.code; if (code.startsWith("Digit")) { const value = Number(code.slice("Digit".length)); return Number.isFinite(value) && value >= 1 && value <= 9 ? value : null; @@ -1070,14 +1080,14 @@ function parseDigit(event: KeyboardEvent): number | null { const value = Number(code.slice("Numpad".length)); return Number.isFinite(value) && value >= 1 && value <= 9 ? value : null; } - const key = event.key ?? ""; + const key = event.key; if (key >= "1" && key <= "9") { return Number(key); } return null; } -function matchesKeyOrCode(combo: KeyCombo, event: KeyboardEvent): boolean { +function matchesKeyOrCode(combo: KeyCombo, event: KeyboardShortcutInput): boolean { if (combo.key === undefined) { return event.code === combo.code; } @@ -1095,7 +1105,7 @@ function matchesKeyOrCode(combo: KeyCombo, event: KeyboardEvent): boolean { return combo.codeFallback === true && event.code === combo.code; } -function matchesCombo(combo: KeyCombo, event: KeyboardEvent, isMac: boolean): boolean { +function matchesCombo(combo: KeyCombo, event: KeyboardShortcutInput, isMac: boolean): boolean { if (combo.mod) { if (isMac) { if (!event.metaKey) return false; @@ -1118,7 +1128,10 @@ function matchesCombo(combo: KeyCombo, event: KeyboardEvent, isMac: boolean): bo return matchesKeyOrCode(combo, event); } -function matchesWhen(when: ShortcutWhen | undefined, context: KeyboardShortcutContext): boolean { +export function matchesKeyboardShortcutContext( + when: ShortcutWhen | undefined, + context: KeyboardShortcutContext, +): boolean { if (!when) return true; if (when.mac !== undefined && when.mac !== context.isMac) return false; if (when.desktop !== undefined && when.desktop !== context.isDesktop) return false; @@ -1136,7 +1149,7 @@ function matchesWhen(when: ShortcutWhen | undefined, context: KeyboardShortcutCo function resolvePayload( def: ShortcutPayloadDef | undefined, - event: KeyboardEvent, + event: KeyboardShortcutInput, ): KeyboardShortcutPayload { if (!def) return null; switch (def.type) { @@ -1187,7 +1200,7 @@ function helpMatchesPlatform( function buildMatchFromBinding( binding: ParsedShortcutBinding, - event: KeyboardEvent, + event: KeyboardShortcutInput, ): KeyboardShortcutMatch { return { action: binding.action, @@ -1198,7 +1211,7 @@ function buildMatchFromBinding( } function resolveInitialChordStep(input: { - event: KeyboardEvent; + event: KeyboardShortcutInput; context: KeyboardShortcutContext; chordState: ChordState; onChordReset: () => void; @@ -1220,7 +1233,7 @@ function resolveInitialChordStep(input: { if (!matchesCombo(firstCombo, event, context.isMac)) { continue; } - if (!matchesWhen(binding.when, context)) { + if (!matchesKeyboardShortcutContext(binding.when, context)) { continue; } if (binding.parsedChord.length > 1) { @@ -1252,7 +1265,7 @@ function resolveInitialChordStep(input: { } function resolveAdvancingChordStep(input: { - event: KeyboardEvent; + event: KeyboardShortcutInput; context: KeyboardShortcutContext; chordState: ChordState; onChordReset: () => void; @@ -1278,7 +1291,7 @@ function resolveAdvancingChordStep(input: { if (!matchesCombo(combo, event, context.isMac)) { continue; } - if (!matchesWhen(binding.when, context)) { + if (!matchesKeyboardShortcutContext(binding.when, context)) { continue; } if (chordState.step + 1 === binding.parsedChord.length) { @@ -1317,7 +1330,7 @@ function resolveAdvancingChordStep(input: { } export function resolveKeyboardShortcut(input: { - event: KeyboardEvent; + event: KeyboardShortcutInput; context: KeyboardShortcutContext; chordState: ChordState; onChordReset: () => void; diff --git a/packages/desktop/capture-harness/index.html b/packages/desktop/capture-harness/index.html index f603bda64..3befdcd94 100644 --- a/packages/desktop/capture-harness/index.html +++ b/packages/desktop/capture-harness/index.html @@ -38,12 +38,40 @@ +
+ + +