feat(browser): inspect, annotate, and grab page elements for the agent (#1708)

* feat(browser): inspect, annotate, and grab page elements for the agent

Build a design-review flow on top of the in-app Electron browser so users
can send page elements to their coding agent with context.

- Annotate: pick an element, write a comment, choose an intent
  (fix/change/question/approve); the element context + intent + comment go
  to the agent as text, and a cropped screenshot rides along as an image.
- Grab: pick an element to copy its info + screenshot straight to the
  system clipboard (no comment), with a toast on success/failure.
- Hover inspector: in select mode each element shows a floating label with
  tag, id/class, React component name, and pixel size.
- Page markers: annotated elements on the current page get numbered badges
  that track scroll/resize.
- Device sizes: a viewport-size menu (responsive + 13 common device presets)
  renders the page centered in a fixed-size frame.
- Toolbar buttons now expose hover tooltips and no longer get squeezed out
  on narrow panes; the element selector is available to all desktop users
  (previously dev-only).

Screenshots and clipboard writes go through new Electron main-process IPCs
(capturePage + clipboard) so they work regardless of webview focus. Element
screenshots are referenced by id in the workspace attachment store and are
protected from the draft-store attachment GC. All new strings are
translated across the six supported locales.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(browser): write element clipboard exactly once

Resolve the image before writing so a combined text+image grab no longer
does a redundant text-only writeText() first (which flashed an intermediate
clipboard state). Addresses greptile review feedback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* i18n: add browser annotate/grab/devices strings for ja and pt-BR

main added Japanese and Brazilian Portuguese locales after this branch was
created; add the browser annotation, grab, and device-size keys to keep all
locales in sync with en. resources.test.ts parity passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
huiliaoning
2026-06-30 22:58:16 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 2b46caa20e
commit b8b66816ca
17 changed files with 1313 additions and 94 deletions
+19
View File
@@ -427,6 +427,25 @@ export const ja: TranslationResources = {
openDevTools: "ブラウザ開発ツールを開く",
cancelSelector: "要素セレクターをキャンセル",
selectElement: "要素を選択",
grabElement: "要素をクリップボードにコピー",
grabElementLabel: "要素",
grabFailed: "要素をコピーできませんでした",
},
annotate: {
title: "エージェントにフィードバックを送信",
placeholder: "変更すべき内容を記述…",
submit: "添付",
cancel: "キャンセル",
intents: {
fix: "修正",
change: "変更",
question: "質問",
approve: "承認",
},
},
devices: {
label: "デバイスサイズ",
responsive: "レスポンシブ",
},
errors: {
failedToLoad: "ページの読み込みに失敗しました",