desktop-touch-mcp
desktop-touch-mcp is a Windows 10/11 MCP server that gives AI agents full control over a desktop environment through these capabilities:
Screen Observation
screenshot— Capture the desktop, windows, or regions at multiple detail levels (meta, text, image, OCR, Set-of-Marks) and modes (diff, background, dot-by-dot)workspace_snapshot— Full overview of all open windows with thumbnails and UI summariesdesktop_state— Lightweight read of focused window/element, cursor position, and modal state
Mouse & Keyboard Input
mouse_click— Click at coordinates with auto-guard (window identity verification), double/triple-click supportmouse_drag— Drag for sliders, drag-and-drop, canvas drawing, window resizingkeyboard— Type text, send key combos (e.g.ctrl+c), or run multi-step chord sequencesscroll— Scroll by wheel, element name, or with OCR-based text readingclick_element— Click native UI elements by name or AutomationId via UIA (no coordinates needed)
UI Discovery & Interaction
desktop_discover— Observe the desktop and return interactive entities with time-limited leasesdesktop_act— Act on a discovered entity (click, type, drag, select) using its lease for verified targeting
Browser Automation (Chrome/Edge CDP — no Selenium needed)
Open/connect, navigate, click DOM elements, fill forms (including React/Vue/Svelte), execute JavaScript, inspect forms, search elements, and get full page overviews
Shell & Terminal Automation
shell_session— Background command execution with real OS process stdin/stdout/exit codesterminal— Drive a visible terminal window (run, send input, read output)wait_until— Server-side polling for conditions (window appears, focus changes, URL matches, terminal output contains, etc.)
Window Management
Focus windows by title, pin/unpin (always-on-top), dock to screen corners, launch apps and wait for their windows
Workflow Utilities
run_macro— Execute up to 50 tool calls sequentially in one MCP call (eliminates round-trip latency)clipboard— Read/write the Windows clipboard with delivery verificationnotification_show— Show Windows system tray notificationsexcel— Author and run VBA macros against Excel via COM
Safety & Performance
Emergency stop: Move mouse to top-left corner (within 10px of 0,0) to immediately terminate the server
Auto-guard: Verifies window identity and coordinate bounds before every action
Native Rust engine for high performance (UIA ~2ms, image diff ~0.26ms) with PowerShell fallback
Recovery signals guide re-discovery, modal dismissal, or fallback methods on errors
Provides browser automation via Chrome DevTools Protocol (CDP), enabling semantic DOM interaction, navigation, form filling, and evaluation of scripts for Brave browser.
Provides visual fallback (SoM) for Electron applications that lack UI Automation support, using OCR and image analysis to detect clickable elements and enable interaction.
Provides browser automation via Chrome DevTools Protocol (CDP), enabling semantic DOM interaction, navigation, form filling, and evaluation of scripts for Google Chrome.
Provides visual fallback for Progressive Web Apps (PWAs) that lack UI Automation support, using OCR and image analysis to detect clickable elements and enable interaction.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@desktop-touch-mcpTake a screenshot of my desktop and describe the open windows"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
desktop-touch-mcp
Windows 10/11 专用 Computer-use MCP 服务器 — 截图、UI Automation、Chrome CDP、键鼠输入、终端 + 可靠后台 shell_session,原生 Rust 引擎 + PowerShell 回退。
快速开始
方式 A:下载 zip(推荐)
从 Releases 下载 zip
解压到任意文件夹
双击
start.bat— 完事
start.bat首次运行自动安装依赖,之后直接启动。无需 npm/npx/构建工具。
方式 B:npx
npx -y @harusame64/desktop-touch-mcpHTTP 模式
双击 start.bat 时加参数即可开启 HTTP 模式:
start.bat --http --port 23847 --key YOUR_KEY注册到 Claude CLI
在 ~/.claude.json 的 mcpServers 中添加:
{
"desktop-touch": {
"type": "stdio",
"command": "node",
"args": ["C:/Tools/desktop-touch-mcp/dist/index.js"]
}
}默认绑定
0.0.0.0。绑定到非 localhost 地址时 API 密钥为必填,客户端须带Authorization: Bearer <KEY>请求头。健康检查:http://<地址>:<端口>/health。
Related MCP server: pov
核心理念
发现-操作 —
desktop_discover返回带租约的可交互实体(非原始坐标),desktop_act操作你意图的目标而非它曾经的位置感知防护 — 每次操作前自动验证目标窗口身份和边界,防止误窗口输入
Rust 原生加速 — UIA 焦点查询 2ms(160× 加速),图像差分 SSE2 SIMD 13~15× 加速;引擎不可用时透明回退 PowerShell
环境要求
要求 | 版本 |
操作系统 | Windows 10/11(64 位) |
Node.js | v20+(推荐 v22+) |
PowerShell | 5.1+(仅作 Rust 引擎回退) |
VC++ 运行时 | 下载(nut-js 需要) |
工具一览(32+ 个)
分类 | 工具 | 说明 |
🌐 主体路径 |
| 观察桌面,返回带租约的可交互实体 |
| 通过租约验证对实体操作(点击/输入/拖拽/选择) | |
👁️ 观察 |
| 轻量焦点/窗口/光标/注意信号检查 |
| 多模式捕获(text/diff/dotByDot/background) | |
| 截图缓存查询与清理 | |
| 所有窗口缩略图 + UI 摘要 | |
| 原生引擎健康诊断 | |
⌨️ 输入 |
| 键盘输入,支持 IME 旁路 |
| 坐标交互 + 归位 + 强制焦点 | |
| 滚轮 / 定位元素 / 智能列表 / 拼接 | |
| UIA 按名称/ID 点击(回退) | |
🌐 浏览器 CDP |
| 幂等调试启动 + 导航 |
| 跨重绘稳定的 DOM 交互 | |
| JS 执行 / DOM 提取 / SPA 状态 | |
🛠️ 工作流 |
| 可靠后台命令通道(进程 stdin/stdout + exit code) |
| 驱动可见控制台窗口(run/send/read) | |
| 窗口/焦点/文本/URL 状态轮询 | |
| 窗口吸附/置顶/聚焦 | |
| 启动应用 + 自动检测新窗口 | |
| 最多 50 个操作批量执行 | |
| 剪贴板 + 通知 | |
📊 Office |
| VBA 宏写入与运行 |
标准工作流
desktop_state → 定向:焦点窗口/元素、模态、注意信号
desktop_discover → 查找可操作实体(返回 lease + windows[])
desktop_act(lease, …) → 操作实体(返回 attention + post.perception)
desktop_state → 确认世界按预期变化命令执行:选对通道(重要)
场景 | 用哪个 | 原因 |
装软件、跑脚本、文件操作、包管理 |
| 真实 OS 进程 + stdin/stdout + exit code;不依赖焦点/IME/剪贴板/SoM 日志 |
用户必须看见某个控制台窗口 |
| 驱动可见窗口(UIA/WM_CHAR),适合交互演示 |
# 一次性命令(推荐)
shell_session({ action: "exec", input: "whoami" })
# → ok:true 仅当 exitCode===0;非零退出返回 ok:false + stdout/stderr
# 交互式多步
shell_session({ action: "start", id: "ops", shell: "powershell" })
shell_session({ action: "write", id: "ops", input: "Get-ChildItem", waitMs: 500 })
shell_session({ action: "read", id: "ops", clear: true })
shell_session({ action: "stop", id: "ops" })禁止假成功: write 只有 stdin 被 OS 接受才 ok:true;exec 只有 exitCode===0 才 ok:true。
点击优先级
browser_click(selector)— Chrome/Edge CDP(跨重绘稳定)desktop_act(lease)— 原生/对话框/视觉(基于实体)click_element(name | automationId)— UIA 回退mouse_click(x, y, origin?, scale?)— 像素级最后手段
恢复提示
信号 | 处理 |
| 重新 |
|
|
|
|
| 回退到 |
租约 TTL 自适应(上限 60s),softExpiresAtMs 约 60% 处 LLM 应考虑刷新。
截图参数速查
参数 | 效果 | Token |
| PNG/WebP 像素(默认) | ~443 |
| UIA 元素 JSON + clickAt 坐标 | ~100-300 |
| 仅标题+区域 | ~20/窗口 |
| 1:1 WebP,图像像素=屏幕坐标 | ~800 |
| 限制最长边,响应含 scale | — |
| 灰度,文本类减小约 50% | — |
| 窗口局部裁剪 | — |
| 仅变化窗口(P帧) | ~160 |
| UIA 稀疏时自动触发 Windows OCR | — |
推荐流程:
workspace_snapshot() → 全面定位
screenshot(detail="text", windowTitle=X) → 获取 clickAt 坐标
mouse_click(x, y) → 直接点击
screenshot(diffMode=true) → 仅检查变化浏览器 CDP 自动化
无需 Selenium/Playwright,只需启用 Chrome 远程调试端口:
chrome.exe --remote-debugging-port=9222 --user-data-dir=C:\tmp\cdpbrowser_open({launch:{}}) → 启动 CDP Chrome + 列出标签
browser_click({selector:"#submit"}) → 查找+点击一步完成
browser_eval({action:"js", expression:"..."}) → 执行 JS
browser_fill({selector:"#email", value:"..."}) → 填充受控输入(React/Vue/Svelte 安全)
browser_navigate({url:"https://example.com"}) → CDP 导航browser_locate 返回的坐标已含浏览器 UI 偏移和 DPI 缩放,可直接传给 mouse_click。
终端命令完成判定
terminal(action='run') 通过 until 参数控制"命令是否完成":
模式 | 等待内容 | 适用场景 |
| 输出安静持续 | 短命令 |
| 输出匹配预期的字符串/正则 | 有已知结束标记的长命令 |
| 命令真正结束 + 返回退出码 | 需要完成码时 |
exit 模式注入与回显不同的完成标记,彻底解决哨兵误匹配问题:
terminal({
action: 'run', windowTitle: 'pwsh',
input: 'npm run build',
until: { mode: 'exit', shell: 'powershell' },
})
// → { reason: 'exited', exitCode: 0 }支持的 shell:bash、powershell。cmd.exe 尚不支持。不安全输入(未关闭引号等)直接拒绝。
鼠标归位校正
截图获取坐标后窗口可能已移动,归位系统自动校正:
层级 | 启用方式 | 功能 |
1 | 始终可用 | (dx, dy) 偏移修正 |
2 | 传 | 窗口被遮挡时自动前置 |
3 | 传 | UIA 重查缩放后的新坐标 |
mouse_click(x=500, y=300) # 层级 1
mouse_click(x=500, y=300, windowTitle="记事本") # 层级 1+2
mouse_click(x=500, y=300, windowTitle="记事本", elementName="保存") # 层级 1+2+3
mouse_click(x=500, y=300, homing=false) # 关闭归位origin + scale 坐标换算
dotByDot 截图带 dotByDotMaxDimension 时响应含 origin 和 scale,直接传入即可自动换算:
mouse_click(x=640, y=300, origin={x:0, y:120}, scale=0.6667, windowTitle="Chrome")
# 服务器换算: screen = (0 + 640/0.6667, 120 + 300/0.6667) = (960, 570)自动防护
操作工具传入 windowTitle 时自动防护:
✅ 验证目标窗口身份(检测进程重启/HWND 替换)
✅ 确认点击坐标在窗口矩形内
✅ 失败时返回
post.perception.status,LLM 可无截图恢复
状态 | 含义 |
| 防护通过 |
| 未提供 windowTitle |
| 无匹配窗口 |
| 窗口已被替换 |
| 坐标在窗口矩形外 |
| 需用 browser_click 或指定 windowTitle |
unsafe_coordinates 或 identity_changed 时可传 fixId 批准一次性恢复(15 秒过期)。
设置 DESKTOP_TOUCH_AUTO_GUARD=0 可禁用自动防护。
强制焦点
Windows 前台保护可能阻止按键到达目标窗口。mouse_click、keyboard、terminal(send) 均支持 forceFocus: true,通过 AttachThreadInput 绕过:
{ "name": "mouse_click", "arguments": { "x": 500, "y": 300, "windowTitle": "Chrome", "forceFocus": true } }全局默认:设 DESKTOP_TOUCH_FORCE_FOCUS=1。被拒绝时返回 ok:false + code: "ForegroundRestricted",操作被抑制不误投。
自动停靠 CLI
MCP 启动时自动停靠承载 Claude 的终端:
{
"mcpServers": {
"desktop-touch": {
"env": {
"DESKTOP_TOUCH_DOCK_TITLE": "@parent",
"DESKTOP_TOUCH_DOCK_CORNER": "bottom-right",
"DESKTOP_TOUCH_DOCK_WIDTH": "480",
"DESKTOP_TOUCH_DOCK_HEIGHT": "360"
}
}
}
}环境变量 | 默认值 | 说明 |
| — |
|
|
| 四角可选 |
|
| px 或比例(如 |
|
| 置顶 |
| 主显示器 | 显示器 ID |
|
| 屏幕边缘填充(px) |
⚠️ 置顶窗口活动时按键会发到它而非目标。键盘操作前先
focus_window(title=...)。
截图缓存
screenshot 返回廉价的 screenshot://by-ref/{id} 链接而非内联像素,减少 token 消耗。
环境变量 | 默认值 | 说明 |
| 用户缓存目录 | 固定缓存路径 |
|
| 缓存上限 |
|
| 磁盘上限 |
| — | 超龄丢弃(opt-in) |
|
| 新增时自动清理, |
自动感知(Always-on)
每个 desktop_state 和 desktop_act 响应自动附加 attention 信号,操作工具传 windowTitle 时自动防护。lensId 参数在操作工具上保留供高级固定目标使用。
安全机制
紧急停止(唯一安全机制)
将鼠标移至屏幕左上角(0,0 附近 10px 以内)即立即终止 MCP 服务器。
每次工具调用前检查
checkFailsafe()500ms 后台轮询作为长操作后备
触发半径:10px
所有按键组合和应用启动均不受限制。键盘黑名单和应用黑名单已移除。
鼠标移动速度
mouse_click、mouse_drag、scroll 均支持 speed 参数:
值 | 行为 |
省略 | 默认 1500 px/秒 |
| 瞬移,无动画 |
| N px/秒动画 |
全局设置:DESKTOP_TOUCH_MOUSE_SPEED=3000。常用:0=瞬移,1500=柔和,3000=快速,5000=极速。
性能(Rust 原生引擎)
UIA 基准
函数 | Rust 原生 | PowerShell | 加速比 |
| 2.2 ms | 366 ms | 163.9× |
| 106.5 ms | 346 ms | 3.3× |
加权平均 | ~82× |
图像差分(SSE2 SIMD)
函数 | Rust | TypeScript | 加速比 |
| 0.26 ms | 3.8 ms | ~15× |
| 0.09 ms | 1.2 ms | ~13× |
架构
MCP 客户端
│ stdio / HTTP
▼
TypeScript 服务层
├── Rust 原生引擎(.node 插件)
│ ├── UIA: napi-rs + windows-rs,MTA COM 线程
│ └── 图像: SSE2 SIMD 差分 + 感知哈希
└── PowerShell 回退(引擎不可用时自动激活)UI 操作层 V2
desktop_discover / desktop_act 默认开启。设 DESKTOP_TOUCH_DISABLE_FUKUWARAI_V2=1 可禁用回退到 V1。
已知限制
限制 | 变通方法 |
游戏/DirectX 全屏截图可能黑屏 | 用 |
Chrome/WinUI3 UIA 元素为空 | 自动 OCR 回退,或用 |
| 先关 Chrome,用 |
diff 缓冲区 90s 不活动后清除 | 长等待后调 |
置顶窗口活动时键盘发到错误窗口 | 先 |
Chrome 中长破折号/智能引号被拦截 | 用 |
React/Vue/Svelte 受控输入 | 用 |
Token 成本参考
模式 | Token | 用途 |
| ~443 | 一般视觉 |
| ~800 | 精确点击 |
| ~160 | 操作后差异 |
| ~100-300 | UI 交互(无图像) |
| ~2000 | 全会话概览 |
环境变量汇总
变量 | 默认值 | 说明 |
| — | HTTP 模式 API 密钥,绑定非 localhost 时必填 |
|
| HTTP 监听地址 |
|
| 鼠标移动速度(px/秒) |
| — | 设 |
|
| 设 |
| — | 自动停靠标题( |
|
| 停靠角落 |
|
| 停靠尺寸 |
|
| 停靠置顶 |
|
| 停靠边距 |
| — | 设 |
| — | npx 缓存根目录 |
| 用户缓存 | 截图缓存路径 |
|
| 缓存数量上限 |
|
| 缓存大小上限 |
许可证
MIT
Available Tools
30 toolsbrowser_clickA
Click a DOM element in Chrome/Edge. Two ways to target: (1) selector — a CSS selector (combines browser_locate + mouse_click; stable across repaints); or (2) by-axis (semantic) — by:'text'|'regex'|'role'|'ariaLabel' + pattern, so you do not have to build a CSS selector for dynamic-class SPAs. by-axis resolves to a SINGLE actionable element (climbing to a clickable ancestor up to 3 levels, hit-testing for occlusion) and STOPS with code:'BrowserAmbiguousTarget' (candidates[] + next[] hints) when 2+ actionable elements match, or code:'BrowserNoActionableTarget' when matches exist but none is clickable — it never guesses. If the target is behind a modal dialog blocking the page, BOTH targeting modes STOP with code:'BrowserModalBlocking' (context.blockingElement {name, role}) instead of clicking through to the backdrop — dismiss the dialog (its close button or Escape) and retry; a plain navigation drawer does not count as blocking. Optionally add role to filter (by:'text',pattern:'Save',role:'button') and scope to narrow the search. Provide EITHER selector OR by+pattern (not both). Pass tabId+port so the server auto-guards (verifies tab readyState and identity) and returns post.perception.status. lensId is optional for advanced pinned-tab workflows. Caveats: selector mode fails if the element is outside the visible viewport — scroll it into view with browser_eval("document.querySelector('sel').scrollIntoView()") first (by-axis only resolves in-viewport actionable targets). hints.verifyDelivery:{status:'delivered'|'unverifiable', reason, observedSignals:{mutationCount,urlChanged,activeElementChanged}} reports the post-click observation in 2 values: 'delivered' fires only when mutationCount>0 OR urlChanged (activeElementChanged is recorded in observedSignals but intentionally NOT a delivery signal — plain clicks on focusable controls always update focus, treating that as 'delivered' would mask silent-fail regressions); 'unverifiable' reason ∈ {'iframe_context_mismatch','no_dom_mutation','probe_install_failed','probe_read_failed'}. CDP emits 2 values only (focus_only is a UIA-path concept, N/A here). BrowserClickNotDelivered is reserved-only (false-positive risk too high to emit) — degradation reads from 'unverifiable' status.
| Name | Required | Description | Default |
|---|---|---|---|
| by | No | Semantic axis to target by INSTEAD of a CSS selector: 'text' (visible text), 'regex', 'role' (ARIA/implicit role), 'ariaLabel'. Pair with pattern. Resolves to a SINGLE actionable element and STOPS with candidates when ambiguous. | |
| port | No | Chrome/Edge CDP remote debugging port. | |
| role | No | Optional ARIA/implicit-role filter AND-combined with by (e.g. by:'text', pattern:'Save', role:'button'). | |
| fixId | No | Approve a pending suggestedFix (one-shot, 15s TTL). Selector mode only. | |
| scope | No | Optional CSS selector to limit the by-axis search scope (disambiguation). | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| lensId | No | Optional perception lens ID. Guards (target.identityStable) are evaluated before clicking, and a perception envelope is attached to post.perception on success. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. rich includes UIA or browser state diff when supported. | minimal |
| pattern | No | Value matched against the chosen by axis (required when by is set). | |
| selector | No | CSS selector for the target element (e.g. '#submit', '.btn'). Provide EITHER selector OR by+pattern. | |
| caseSensitive | No | Case-sensitive matching for by:'text'/'regex' (default false). | |
| scrollIntoView | No | When true, if the target is outside the viewport, scroll it into view (centered) before clicking, instead of failing with ElementNotInViewport. Default false preserves the explicit scrollIntoView-then-retry workflow. Selector mode only (by-axis resolves only in-viewport actionable targets). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: how by-axis resolves to a single actionable element, handling of ambiguity and modal blocking, delivery verification signals, and scroll behavior. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but well-structured with clear sections. Some parts, like verifyDelivery details, could be more concise, but overall it earns its length given the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 13 parameters and no output schema, the description is remarkably complete, covering return behavior, error states, boundary cases (modal, viewport), and parameter interactions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema coverage, the description adds significant meaning: explains by+pattern pairing, role/scope filtering, scrollIntoView effects, fixId, and verifyDelivery hints. All 13 parameters are meaningfully elaborated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it clicks a DOM element in Chrome/Edge and distinguishes two targeting modes (selector and by-axis), differentiating it from siblings like browser_locate (which only locates) and mouse_click (global).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on when to use each targeting mode, caveats (viewport, modal blocking), and alternative tools (browser_eval for scrolling). Also explains error codes and retry strategies.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_evalA
Purpose: Inspect or operate on a browser tab via 3 actions: 'js' (evaluate JS), 'dom' (get HTML), 'appState' (extract SSR-injected SPA state). Details: action='js' — Run a JS expression. withPerception:true wraps in {ok, result, post}. action='dom' — Return outerHTML of selector (or document.body), truncated to maxLength. action='appState' — Scan Next/Nuxt/Remix/Apollo/GitHub/Redux SSR injected JSON; pass selectors to override defaults. Prefer: Use action='appState' BEFORE 'dom' or 'js' on SPAs where rendered HTML is sparse — single CDP call. Use 'dom' when 'appState' is empty and you need page structure. Use 'js' as the escape hatch for arbitrary scripting. Caveats: DOM nodes cannot be returned from action='js' directly (circular refs are serialized safely). React/Vue/Svelte controlled inputs cannot be set via element.value — use keyboard(action='type') / browser_fill instead. readyState is strictly checked; guard blocks if page is still loading. Typed errors: code:'BrowserNotConnected' on CDP disconnect (re-attach via browser_open); code:'AutoGuardBlocked' when the auto-guard refuses (e.g. page still loading) — the error message preserves the guard's 1-sentence recommended next step (most often wait_until({condition:'ready_state'}) or browser_eval readyState polling, then retry). Examples: browser_eval({action:'js', expression:'document.title'}) → page title browser_eval({action:'dom', selector:'#main', maxLength:5000}) → outerHTML browser_eval({action:'appState'}) → default SPA state probes
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port. | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| action | Yes | Action selector — one of: js, dom, appState. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| lensId | No | Optional perception lens ID. Guards (target.identityStable) are evaluated before eval. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| maxBytes | No | Max bytes per individual payload (default 4000). Larger payloads are truncated. | |
| selector | No | CSS selector for root element. Omit for document.body. | |
| maxLength | No | Max characters of HTML to return (default 10000). | |
| selectors | No | Custom probe selectors. Omit to use the default SPA framework set (__NEXT_DATA__ / __NUXT_DATA__ / __REMIX_CONTEXT__ / __APOLLO_STATE__ / window:__INITIAL_STATE__ etc.). Window globals must be prefixed with 'window:'. | |
| expression | No | JavaScript expression to evaluate. The server automatically wraps snippets in an async IIFE to avoid repeated const/let collisions. For multi-statement snippets, use an explicit final return value. Declarations (const/let/var) are scoped per snippet — use window.* / globalThis.* for persistence. A single eval is bounded by the CDP per-command timeout (~15s): do NOT write in-page polling loops here — use wait_until (element_matches / url_matches / ready_state) to wait for conditions instead. | |
| includeContext | No | When true, append activeTab and readyState context to the response. | |
| withPerception | No | When true, return structured JSON {ok, result, post} with post.perception attached. Default false preserves raw-text return. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description bears full responsibility for disclosing behavior. It details error types (e.g., 'BrowserNotConnected', 'AutoGuardBlocked'), truncation behavior, readyState checks, and limitations like circular reference serialization. This fully informs the agent of the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but well-structured with clear sections (Purpose, Details, Prefer, Caveats, Examples). It is front-loaded with the essential purpose and action list. While some redundancy could be trimmed, the structure and content are appropriate for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (12 parameters, no output schema), the description is thorough. It covers all actions, usage preferences, caveats, error handling, and provides examples. It addresses potential edge cases and integrates with sibling tools, ensuring completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds significant value beyond the schema by explaining parameter interactions (e.g., 'withPerception' returns structured JSON, 'includeContext' appends context) and providing examples. It enhances understanding of parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: inspect or operate on a browser tab via three distinct actions ('js', 'dom', 'appState'). It differentiates from sibling tools by specifying when to use each action and provides examples, making the purpose very clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidelines, including a 'Prefer' section that advises when to use each action (e.g., use 'appState' before 'dom' or 'js' on SPAs). It also includes caveats for when not to use certain actions, such as avoiding direct DOM node return from 'js'. This thoroughly guides the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_fillA
Fill a form input with a value via CDP — works on React/Vue/Svelte controlled inputs that reject browser_eval value assignment. Two ways to target: (1) selector — a CSS selector (use browser_overview / browser_locate to find one); or (2) by-axis (semantic) — by:'text'|'regex'|'role'|'ariaLabel' + pattern (e.g. by:'ariaLabel', pattern:'Email address', or by:'role', pattern:'textbox'), so you do not have to build a CSS selector. by-axis resolves to a SINGLE fillable element and STOPS with code:'BrowserAmbiguousTarget' (candidates[] + next[] hints) when 2+ match, or code:'BrowserNoActionableTarget' when the match is not a fillable input/textarea/contenteditable — it never guesses. Optionally add role to filter and scope to narrow. Provide EITHER selector OR by+pattern (not both). Use this over browser_eval when setting a controlled input's value via JS does not update framework state. Caveats: Requires browser_open (CDP active). actual in the response shows the element's value after fill; verify it matches the intended value. Typed errors: code:'BrowserFillNotDelivered' on post-fill value mismatch — note the false-positive case where a React controlled input's onChange transforms the value (delivery actually succeeded; hints.verifyDelivery.subReason:'controlled_input_transform' for that case; the actual value is authoritative).
| Name | Required | Description | Default |
|---|---|---|---|
| by | No | Semantic axis to target by INSTEAD of a CSS selector: 'text' (visible text), 'regex', 'role' (ARIA/implicit role), 'ariaLabel'. Pair with pattern. Resolves to a SINGLE actionable element and STOPS with candidates when ambiguous. | |
| port | No | Chrome/Edge CDP remote debugging port. | |
| role | No | Optional ARIA/implicit-role filter AND-combined with by (e.g. by:'text', pattern:'Save', role:'button'). | |
| scope | No | Optional CSS selector to limit the by-axis search scope (disambiguation). | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| value | Yes | Text to fill into the input element | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| pattern | No | Value matched against the chosen by axis (required when by is set). | |
| selector | No | CSS selector for the input element. Provide EITHER selector OR by+pattern. | |
| caseSensitive | No | Case-sensitive matching for by:'text'/'regex' (default false). | |
| includeContext | No | When true, append activeTab and readyState context to the response. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses CDP dependency, error codes (BrowserAmbiguousTarget, BrowserNoActionableTarget, BrowserFillNotDelivered), and false-positive case for controlled inputs. Also explains stopping behavior and verification via 'actual' in response. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Despite length, the description is well-structured: it starts with purpose, then targeting methods, usage comparison, caveats, and error handling. Every sentence serves a purpose with no redundancy. Front-loaded with the main use case.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 11 parameters and no output schema, the description covers input semantics, targeting, error codes, and verification. It does not detail the full response shape, but mentions 'actual' and hints. Given the richness of other sections, it is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds significant meaning: explains the by-axis method (text, regex, role, ariaLabel), the mutual exclusivity of selector and by+pattern, and the optional role/scope filters. It also details error conditions tied to parameters, which the schema does not. Therefore, adds value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fills form inputs via CDP, specifically for React/Vue/Svelte controlled inputs where browser_eval fails. It distinguishes from sibling browser_eval by naming the alternative and the condition. The verb 'Fill' and resource 'form input' are precise, and the scope is well-defined.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use over browser_eval: 'Use this over browser_eval when setting a controlled input's value via JS does not update framework state.' Also explains when to use by-axis vs selector, and mentions prerequisites like browser_open. Provides clear context and exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_formA
Inspect all form fields (input, select, textarea, button) within a CSS-selector-specified container and return their name, type, id, current value, hint text, disabled/readOnly state, and associated label text (resolved via for[id], ancestor LABEL, aria-labelledby, aria-label in that order). Use this before browser_fill to discover exact field selectors and avoid accidentally targeting the wrong input (e.g. a global search bar). Caveats: Requires browser_open (CDP active). Hidden inputs (type=hidden) are excluded by default — set includeHidden:true if needed. Value text is truncated at 200 chars.
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port. | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| selector | Yes | CSS selector for the form or container element to inspect (e.g. '#login-form', '.search-bar'). All input, select, textarea, and button descendants are returned. | |
| maxResults | No | Maximum number of form fields to return (default 100). | |
| includeHidden | No | When true, include hidden inputs (type=hidden). Default false to avoid CSRF-token / serialized-state clutter. | |
| includeContext | No | When true, append activeTab and readyState context to the response. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses all key behaviors: requires CDP active, excludes hidden inputs by default, value truncation at 200 chars, label resolution order, and exact return fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is front-loaded with purpose and contains dense information. Every sentence adds value, though it is a single long sentence that could be broken for readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 7 parameters and no output schema, the description covers all essential context: purpose, when to use, prerequisites, caveats, and return fields. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. Description adds marginal value beyond schema for parameters; it contextualizes selector usage and mentions default for includeHidden, but mostly repeats schema info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'inspect' and the resource 'form fields' within a container specified by CSS selector. It lists the fields returned, distinguishing it from sibling tools like browser_fill.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends using this tool before browser_fill to discover exact field selectors and avoid targeting wrong inputs. Also mentions prerequisites (browser_open) and caveats about hidden inputs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_locateA
Find a DOM element by CSS selector and return its physical screen coordinates — compatible directly with mouse_click. Prefer browser_click to find+click in one step. Prefer browser_overview to discover selectors. Caveats: Coordinates are captured at call time; if the page reflows before mouse_click, coords may be stale. Typed errors: code:'BrowserNotConnected' (call browser_open first), code:'ElementNotFound' (selector did not match — re-discover via browser_overview / browser_search).
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port. | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| selector | Yes | CSS selector for the target element (e.g. '#submit', '.btn', 'button[type=submit]'). | |
| includeContext | No | When true, append activeTab and readyState context to the response. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full burden. It discloses that coordinates are captured at call time and may be stale after page reflow. Also lists specific error codes with explanations. Lacks explicit statement that the tool is read-only, but the caveat implies no side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: three sentences covering purpose, usage guidance, caveats, and error types. No extraneous words, well structured with important info front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explains the return type (physical screen coordinates) and compatibility. It also covers common errors. Could be more complete by specifying the exact format of coordinates (e.g., x, y integers), but overall sufficient for an agent to understand and invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already described in schema. The description adds the context that the output is screen coordinates compatible directly with mouse_click, which is not in schema. However, it does not add further detail to parameter meanings beyond what's in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Find', resource 'DOM element by CSS selector', and the output 'physical screen coordinates'. It distinguishes from sibling tools by noting compatibility with mouse_click and referencing alternatives like browser_click and browser_overview.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends alternatives: 'Prefer browser_click to find+click in one step' and 'Prefer browser_overview to discover selectors'. Also includes caveats about stale coordinates, guiding when to use this tool vs others.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_openA
Connect to Chrome/Edge running with --remote-debugging-port and return open tab IDs — required before all other browser_* tools. Pass launch:{} (or with overrides) to auto-spawn a debug-mode browser when no CDP endpoint is live (idempotent: an already-running endpoint is preferred). Returns tabs[] with id, url, title, active — pass tabId to browser_* tools to target a specific tab. Caveats: CDP connection is per-process; if Chrome restarts, call browser_open again to get fresh tab IDs. A Chrome session started without --remote-debugging-port cannot be taken over — close it first or use a separate userDataDir. If the CDP endpoint is unreachable and launch is omitted, returns ok:false (typically code:'BrowserNotConnected' when the fetch surfaces ECONNREFUSED, otherwise code:'ToolError' with error 'Cannot reach Chrome/Edge CDP...'); re-call with launch:{} (idempotent) to auto-spawn or start Chrome manually with --remote-debugging-port=9222.
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port. | |
| launch | No | If set, spawn a debug-mode browser when no CDP endpoint is live on the target port (idempotent: an already-running endpoint is preferred and the spawn step is skipped). Pass {} to use defaults (chrome, C:\tmp\cdp, no initial URL). Omit to perform pure connect. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description fully discloses idempotency, preference for existing endpoint, return structure, per-process connection, and error codes. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph packs all info but is somewhat dense; front-loads key purpose. Every sentence adds value, but could be more scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, so description must explain return (tabs array) and errors, which it does. Missing details on envelope structure but coverable by parameter documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (baseline 3). Description adds value by explaining 'idempotent' launch behavior, killExisting warning, and envelope/raw response shape.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Connect to Chrome/Edge running with --remote-debugging-port and return open tab IDs' and distinguishes it from siblings by noting it is required before all other browser_* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance on when to use this tool (before other browser tools), when to pass launch:{} vs omit, conditions for re-call (Chrome restart), and alternatives if browser lacks debug flag.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_overviewA
List all interactive elements (links, buttons, inputs, ARIA controls) on the current page with CSS selectors, visible text or value for inputs, and viewport status — use before browser_click to discover stable selectors, and prefer this over screenshot when verifying button/toggle state after submission (no image tokens, structured output). scope limits to a CSS subsection (e.g. '.sidebar'). Returns state (checked/pressed/selected/expanded) for ARIA custom controls. Also returns a modal: section — whether a true modal dialog is blocking the page (isModal + blocker {name, role} + the signals it was judged on); it is ALWAYS present (isModal:false when no modal), and a navigation drawer is NOT reported as a modal (only an aria-modal / alertdialog / native showModal dialog, or a backdrop-backed dialog that locks the page, is treated as modal). Caveats: Selectors are CDP-generated snapshots — re-call after page navigates or re-renders. Input text reflects the empty-field hint text when defined (takes priority over typed value) — use browser_eval('document.querySelector(sel).value') to read actual typed content. Typed errors: code:'BrowserNotConnected' (CDP not attached — call browser_open or browser_open({launch:{}})). Note: a non-matching scope CSS selector silently falls back to the full document (does not raise an error) — verify the selector via browser_eval if scoped enumeration is required.
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port. | |
| scope | No | CSS selector to limit the search scope (e.g. '.s-main-slot', '#nav-search-form'). Omit to scan the full page. | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| types | No | Element types to include. Default 'all' returns links, buttons, and inputs. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| maxResults | No | Maximum number of elements to return (default 50). | |
| inViewportOnly | No | When true, only return elements currently visible in the viewport. | |
| includeContext | No | When true, append activeTab and readyState context to the response. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the burden of disclosure. It details that selectors are CDP snapshots, input text reflects hint priority, non-matching scope silently falls back, modal detection logic, and error conditions like BrowserNotConnected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but each sentence adds value. It is front-loaded with main purpose and usage, followed by detailed sections on modal and caveats. Slightly verbose on modal detection but necessary for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters, no output schema, and complex behavior, the description covers return elements, modal detection, caveats, and error handling. It could explicitly list all output fields but the modal section is well specified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining scope usage and the purpose of the modal section, but most parameter details are already in the schema. Extra context like 'scope silently falls back' provides beyond-schema understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool lists interactive elements on the page, differentiates from siblings like browser_click and screenshot by specifying use cases ('use before browser_click', 'prefer over screenshot when verifying button/toggle state').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends usage before browser_click and over screenshot for state verification. Mentions scope for subsection filtering and caveats like re-call after navigation. Alternative tools are implicitly given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_searchA
Grep-like element search across the current page. by: 'text' (literal substring), 'regex', 'role', 'ariaLabel', 'selector' (CSS). Returns results[] sorted by confidence descending — pass results[0].selector to browser_click. Pagination via offset/maxResults. Caveats: Use browser_overview for broad discovery; use browser_search when you know specific text or role to target. Typed errors: code:'BrowserSearchNoResults' (broaden the by:'text' substring or relax the by:'regex' pattern; switch to browser_overview to enumerate selectors), code:'BrowserSearchTimeout' (reduce maxResults / narrow scope), code:'ScopeNotFound' (the scope CSS selector did not match — verify the selector or omit scope), code:'BrowserNotConnected' (call browser_open first, or browser_open({launch:{}}) to auto-spawn).
| Name | Required | Description | Default |
|---|---|---|---|
| by | Yes | Search axis: text/regex/role/ariaLabel/selector | |
| port | No | Chrome/Edge CDP remote debugging port. | |
| scope | No | CSS selector to limit the search scope. | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| offset | No | Offset into the result set (default 0). | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| pattern | Yes | Pattern to match against the chosen axis. | |
| maxResults | No | Max results returned (default 50). | |
| visibleOnly | No | Only visible elements (default true). Set false to include hidden ones with confidence penalty. | |
| caseSensitive | No | Case-sensitive matching for text/regex (default false). | |
| inViewportOnly | No | Only currently-in-viewport elements (default false). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses result ordering (confidence descending), pagination, error behaviors, and scope limitations. However, it does not explicitly state that the tool is read-only or describe side effects, though it is implied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the core purpose and axes. It includes caveats and error details, but is slightly verbose. Every sentence adds value, though a minor reduction in length could improve conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 11 parameters, no output schema, and no annotations, the description covers behavior thoroughly including error handling, usage guidance, and parameter interaction. It could elaborate slightly on the 'include' parameter, but overall it is sufficiently complete for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all parameters. The description adds value beyond the schema by explaining how results[0].selector can be used with browser_click, pagination via offset/maxResults, and error codes. It also clarifies the 'by' axis options and the 'include' parameter's effect.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a 'Grep-like element search across the current page' and lists all search axes (text, regex, role, etc.), making the purpose unambiguous. It distinguishes itself from siblings like browser_overview and browser_click.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is given: 'Use browser_overview for broad discovery; use browser_search when you know specific text or role to target.' Additionally, error codes with actionable next steps are provided, helping the agent decide when and how to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
click_elementA
Invoke a UI element by name or automationId via UIA InvokePattern — no screen coordinates needed. The server auto-guards using windowTitle (verifies identity, foreground, modal) and returns post.perception.status. Prefer over mouse_click for buttons, menu items, and links in native Windows apps. Use desktop_discover first to discover automationIds. Pass fixId from a suggestedFix to re-target after window identity drift. lensId is optional for advanced pinned-lens use. Caveats: Typed errors: code:'InvokePatternNotSupported' — the control does not expose InvokePattern, fall back to mouse_click; code:'ElementDisabled' — the element is in a disabled state, re-check preconditions before retry; code:'GuardFailed' — read the perception envelope (attention / guard fields) and choose recovery (re-focus, wait, or pass the suggestedFix.fixId on the next call). Some custom controls do not expose InvokePattern at all; fall back to mouse_click for those.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). String to avoid 64-bit precision issues. | |
| name | No | Element name/label (partial match, case-insensitive) | |
| fixId | No | Approve a pending suggestedFix (one-shot, 15s TTL). | |
| lensId | No | Optional perception lens ID. Guards (safe.keyboardTarget, target.identityStable) are evaluated before clicking, and a perception envelope is attached to post.perception on success. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. rich includes UIA or browser state diff when supported. | minimal |
| controlType | No | Control type filter, e.g. 'Button', 'MenuItem' | |
| windowTitle | Yes | Partial window title of the target window. Use '@active' for the current foreground window. | |
| automationId | No | Exact AutomationId of the element |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Explains auto-guarding mechanism, return of post.perception.status, typed error codes, and behavior with lensId. Comprehensive disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded purpose and usage guidelines, then caveats. Every sentence adds value but could be more concise. Error details are thorough but lengthy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Handles 9 parameters, 1 required, no output schema. Explains return value, error handling, prerequisites (desktop_discover), and usage patterns. Complete for a complex tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, baseline 3. Description adds context for fixId (one-shot, 15s TTL), lensId (advanced use), include (response shape opt-in), and narrate (narration levels). Enriches beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Invoke a UI element by name or automationId via UIA InvokePattern — no screen coordinates needed.' It specifies the verb (invoke) and resource (UI element), and distinguishes from mouse_click (coordinates) and browser_click (browser).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Prefer over mouse_click for buttons, menu items, and links in native Windows apps. Use desktop_discover first to discover automationIds.' Also provides caveats and fallbacks when InvokePattern is not supported.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clipboardA
Read or write the Windows clipboard. action='read' returns current text content (empty string if non-text). action='write' replaces clipboard with given text and verifies delivery via Get-Clipboard -Raw read-back, comparing the bytes (UTF-16LE) for exact equality. Caveats: Non-text clipboard payloads (images, files) return empty string on read. Overwrites existing clipboard content on write. action='write' delivery-verification failure returns code:'ClipboardWriteNotDelivered' — typical causes: a third-party clipboard manager intercepts SetClipboardData, DLP / endpoint protection blocks the payload, RDP / Citrix clipboard transcoding strips the text, or another process clears the clipboard between Set and the read-back. Recovery: retry the write, or fall back to keyboard(action='type', use_clipboard=false) for short text. Examples: clipboard({action:'write', text:'hello'}) → write+verify; clipboard({action:'read'}) → returns current text.
| Name | Required | Description | Default |
|---|---|---|---|
| text | No | Text to place on the clipboard | |
| action | Yes | Action selector — one of: read, write. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the burden. It discloses all relevant behaviors: non-text clipboard returns empty string, overwriting on write, delivery verification with byte comparison, and detailed failure reasons. This exceeds the minimum required transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is somewhat long but front-loaded with the main purpose. Every sentence contributes useful information; however, details about byte-level verification could be simplified for brevity without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given three parameters and no output schema, the description covers return values, side effects, failure modes, and recovery. It also connects to sibling tools (keyboard) and provides examples. The tool is thoroughly documented for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds value by explaining the action enum values, the text parameter's role in writing, and the optional include parameter for response shaping. It provides context beyond the schema's dry definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool reads or writes the Windows clipboard, with specific verb-resource pairs for each action. It distinguishes itself from siblings like keyboard by focusing solely on clipboard operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use each action and offers recovery strategies for write failures (retry or fallback to keyboard). It does not explicitly list alternatives but the guidance is sufficient for correct usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
desktop_stateA
Purpose: Read-only observation of the current desktop state. Returns focused window/element, modal flag, attention signal from Auto Perception. Phase 4 absorbs former get_active_window / get_cursor_position / get_screen_info / get_document_state via include* flags. Details: Always returns: focusedWindow (title, hwnd, processName), focusedElement (name, type, value, automationId), cursorPos {x,y}, cursorOverElement (name, type), cursorOverWindow, hasModal (boolean), pageState ('ready'|'loading'|'dialog'), attention, visibleWindows count. Optional fields (default off): includeCursor:true → cursor {x,y,monitorId} (richer than cursorPos). includeScreen:true → screen {virtualScreen, displays[], displayCount, primaryIndex}. includeDocument:true → document {url, title, readyState, selection, scroll, viewport} via CDP (silently omitted on non-Chromium foreground). includeSessionContext:true (or include:['sessionContext']) → sessionContext {origin, consoleSessionId, sessionLabel, sessionState, ownWinStation} for Terminal Services session classification (ADR-017, observability-only). Chromium: cursorOverElement is null (UIA sparse); focusedElement may fall back to CDP document.activeElement; hints.focusedElementSource reports which path produced the row ('view' = engine-perception latest_focus, 'uia' = direct UIA query, 'cdp' = document.activeElement). Does NOT enumerate descendants — use desktop_discover for actionable entity list and window list. Prefer: Use after each action to confirm state. Cheapest observation tool — cheaper than any screenshot. attention='ok' means safe to proceed; other values require recovery (see suggest[]). Set include* flags only when you need the extra data (each adds one syscall or CDP round-trip). Caveats: Cannot detect non-UIA elements (custom-drawn UIs, game overlays). hasModal only detects modal dialogs exposed via UIA — browser alert/confirm dialogs may not appear here. includeDocument requires browser_open (CDP active); silently omitted otherwise with hints.documentUnavailable.
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | CDP port for includeDocument (default 9222). | |
| tabId | No | Optional CDP tab id for includeDocument; omit for the focused tab. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| includeCursor | No | When true, add a richer `cursor` field with monitor index alongside the lightweight `cursorPos`. Phase 4: absorbs former get_cursor_position. Default false. | |
| includeScreen | No | When true, add a `screen` field with all connected display info (resolution, position, DPI, scale). Phase 4: absorbs former get_screen_info. Default false. Use the displayId values returned here in screenshot / window_dock(action='dock'). | |
| includeDocument | No | When true, add a `document` field with the focused Chrome tab's url, title, readyState, selection, and scroll position via CDP. Phase 4: absorbs former get_document_state. Default false. Requires browser_open (CDP active); silently omitted on non-Chromium foreground. | |
| includeSessionContext | No | When true, add a `sessionContext` field with the Terminal Services session classification (origin, consoleSessionId, sessionLabel: 'console'|'rdp'|'other', sessionState: 'active'|'connected'|'disconnected'|'locked'|'unknown', ownWinStation). Default false. Equivalent to `include: ['sessionContext']`. Per ADR-017: observability-only — does not gate input. `sessionState: 'locked'` is a heuristic (active + foreground=null + previous sample within 60s saw a non-null foreground); treat it as a generic input-pause signal — it can also fire on secure-desktop transitions (UAC prompt, Credential UI), where the user-visible state is not strictly 'locked' but input is equally unavailable to this session. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description thoroughly explains behavioral traits: it is read-only, returns specific fields, has caveats (non-UIA elements, modal detection limitations, includeDocument requiring browser_open), and details Chromium-specific behavior. It also notes the absorption of former tools, providing full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized into Purpose, Details, Prefer, and Caveats sections. While it is lengthy, every sentence adds necessary detail for effective tool use. It is minimally verbose given the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 optional parameters, no output schema), the description is highly complete. It covers default behavior, optional enrichments, caveats, integration notes with other tools, and Chromium-specific fallbacks. The agent can use this tool correctly without ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining the context and purpose of each include* parameter (e.g., absorbing former tools, default off) and the meaning of port and tabId. This goes beyond basic parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose as 'Read-only observation of the current desktop state' and lists what it returns (focused window/element, modal flag, etc.). It distinguishes itself from sibling tool desktop_discover by noting it does not enumerate descendants, making its scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises using the tool after each action to confirm state and highlights it as the cheapest observation tool. It provides guidance on setting include* flags only when needed. However, it lacks explicit 'when not to use' instructions beyond the mention of desktop_discover for descendant enumeration.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
excelA
Purpose: Author and run VBA macros against Excel via COM late binding (ADR-015). Headline differentiator against Claude for Excel which writes formulas but cannot run VBA.
Details: action='run_vba' authors a Sub in a fresh workbook, saves into the managed Trusted Location (%LOCALAPPDATA%\desktop-touch-mcp\trusted-vba), and Application.Run the macro. Requires HKCU AccessVBOM=1 + VBAWarnings=1 + a registered Trusted Location (all configured by node scripts/enable-access-vbom.mjs). Trust setup: Excel must restart after the CLI runs (values cached at process start). action='check_access_vbom' is a read-only preflight returning {trusted, lockedByPolicy, scope}.
Prefer: Run check_access_vbom first when a workflow depends on macro execution; the remediation hint pre-empts an opaque HRESULT 0x800a03ec failure inside run_vba.
Caveats: macroName MUST appear as Sub <name>(...) in code (else VbaMacroNotFound). VBA Editor UI is structurally bypassed — no UIA tree walk needed. Excel COM is STA: each call serialises through the bridge's worker thread, so long-running macros block subsequent excel() calls on the same MCP server.
Examples:
excel({action:'check_access_vbom'}) → {trusted:true, scope:'hkcu'}
excel({action:'run_vba', code:'Sub DesktopTouchAdHoc()\n Range("A1").Value = "Hello"\nEnd Sub'}) → {ok:true, workbookPath:'...\trusted-vba\dt_vba_.xlsm'}
excel({action:'run_vba', code:'Sub Demo()\n MsgBox "hi"\nEnd Sub', macroName:'Demo', visible:true}) → demo recording path
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description carries full burden. Discloses destructive nature (creates macros), required registry settings, trust location, error conditions, and serialization behavior due to STA. Highly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with headings (Purpose, Details, Prefer, Caveats, Examples) and front-loaded purpose. Slightly verbose but necessary for the complex tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers prerequisites, alternative actions, error handling, and provides multiple examples. No output schema needed given the examples. Complete for an AI agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has no properties; description compensates by explaining 'action', 'code', 'macroName', 'visible' parameters, including valid values and usage. However, no formal parameter definitions in schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states purpose: author and run VBA macros via COM late binding. Distinguishes from similar tools like 'Claude for Excel' and sibling 'run_macro', with specific verb+resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends running 'check_access_vbom' first, provides remediation hints, and notes caveats like macroName format and STA threading. Offers clear when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
focus_windowA
Bring a window to the foreground by partial title match (case-insensitive). Use when a tool does not accept a windowTitle param, or when you need to switch focus before a sequence of actions. Use chromeTabUrlContains to activate a specific Chrome/Edge tab by URL substring before focusing — only the active tab's title appears in the windows list. If CDP is unavailable, chromeTabUrlContains is silently skipped — check response.hints.warnings. Returns WindowNotFound if no match exists; call desktop_discover to see available titles. Caveats: On some apps focus may be immediately stolen back (modal dialogs, UAC prompts) — verify with desktop_state after focusing. Win11 foreground refusal (UIPI cross-elevation / admin-only target / call from a background process or service) returns code:'ForegroundRestricted' ok:false instead of silently failing — recover by switching to a tool that does not require foreground transfer: desktop_act / click_element use UIA InvokePattern (no foreground needed); keyboard BG path bypasses foreground for terminal-class targets only (Windows Terminal / cmd / PowerShell — keyboard with windowTitle on non-terminal apps still hits the same ForegroundRestricted refusal). browser_* tools target by tabId/selector, not windowTitle.
| Name | Required | Description | Default |
|---|---|---|---|
| title | Yes | Partial window title to search for (case-insensitive) | |
| cdpPort | No | CDP port for chromeTabUrlContains (default 9222) | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| forceFocus | No | When set, use AttachThreadInput-based foreground escalation on the first attempt. When omitted (default), focus_window first tries the standard SetForegroundWindow path and auto-escalates to force-focus only if Win11 refused the default attempt (issue #197). Override env: DESKTOP_TOUCH_FORCE_FOCUS=1 sets the implicit default to true. If both default and force paths fail, focus_window now returns ok:false code:'ForegroundRestricted' instead of the previous silent ok:true with windowChanged:false. | |
| chromeTabUrlContains | No | When set, activate the Chrome/Edge tab whose URL contains this substring before focusing the window. Requires Chrome/Edge running with --remote-debugging-port (default 9222). Use this when the target is a Chrome tab that is not currently active — the active tab title is the only one visible in the window title list. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description fully discloses behaviors: returns WindowNotFound, caveats about focus being stolen back, Win11 foreground refusal behavior, forceFocus escalation algorithm, and handling of CDP unavailability. Completely transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is appropriately sized, front-loaded with core functionality. Every sentence adds value with no redundancy. Well-structured, starting with main action then adding details and caveats.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 params, no output schema, and many sibling tools, description is extremely complete: explains return types, error conditions, recovery strategies, and integration with other tools (chromeTabUrlContains, desktop_discover, desktop_act, etc.). No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. Description adds significant context beyond schema: explains purpose of each parameter (title matching, cdpPort default, include response shape, forceFocus algorithm, chromeTabUrlContains usage). Adds value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'bring to foreground', resource 'window', and mechanism 'partial title match (case-insensitive)'. Distinguishes from siblings by specifying when to use this tool vs alternatives like chromeTabUrlContains and other focus methods.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: when a tool does not accept windowTitle param, or when needing to switch focus before a sequence of actions. Provides clear when-not-to-use scenarios and alternatives (desktop_act, click_element, keyboard) for recovery from ForegroundRestricted errors.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
keyboardA
Purpose: Send keyboard input to a window: 'type' for text, 'press' for key combos, 'sequence' for atomic multi-step chords. Details: action='type' inserts text (auto-clipboard for non-ASCII / IME-safe). action='press' sends key combos like 'ctrl+c'/'alt+tab'. action='sequence' runs ordered steps in one keyboard lock — use for Alt+letter, letter mnemonic chains where intermediate tool calls would close the menu. Pass windowTitle to auto-focus and auto-guard (identity, foreground, modal) before input. Omitting windowTitle acts on the active window (unguarded). Prefer: Use windowTitle to auto-focus before injection. Set lensId for perception guards. Use desktop_act({action:'setValue'}) for UIA ValuePattern text fields. Caveats: win+r/win+x/win+s/win+l blocked. action='type' does not handle CJK IME composition — use use_clipboard=true or desktop_act({action:'setValue'}). Non-ASCII text (CJK / emoji / diacritics / smart-quote-class punctuation) auto-clipboards to prevent silent-drop and Chrome accelerator hijack; pass forceKeystrokes:true to disable. Background (PostMessage/WM_CHAR) auto-engages for terminal-class windows (Windows Terminal / cmd / PowerShell); DTM_BG_AUTO=1 enables globally. Foreground non-terminal type runs a per-chunk leash; user focus-steal mid-stream aborts with FocusLostDuringType + context.typed/remaining; pass abortOnFocusLoss:false to disable. BG type verifies WM_CHAR via UIA TextPattern read-back; mismatch returns BackgroundInputNotDelivered (see SUGGESTS for false-positive notes). BG press read-back is scoped to terminal-class + enter/tab/arrow; other combos return verifyDelivery:'unverifiable', failure returns BackgroundKeyNotDelivered. action='sequence' is FG-only (BG/foreground_flash schema-rejected); emits verifyDelivery:'focus_only'; mid-loop focus theft returns MenuFocusLostMidSequence + context.remaining: Step[]. Win11 FG refusal returns ForegroundRestricted — terminal-class targets auto-engage BG; non-terminal switch to desktop_act / click_element. Examples: keyboard({action:'type', text:'hello', windowTitle:'Notepad'}) → text injected (guarded) keyboard({action:'type', text:'hello'}) → text injected (unguarded) keyboard({action:'press', keys:'ctrl+c'}) → copy keyboard({action:'press', keys:'escape', windowTitle:'Dialog'}) → dismiss dialog keyboard({action:'sequence', steps:[{keys:'alt+i', gapMs:100},{keys:'m'}], windowTitle:'Microsoft Visual Basic'}) → Insert > Module (atomic)
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). Obtain from get_windows response (hwnd field). String type to avoid 64-bit precision issues. | |
| keys | No | Key combo string, e.g. 'ctrl+c', 'alt+tab', 'enter', 'ctrl+shift+s'. Note: win+r, win+x, win+s, win+l are blocked for security. | |
| text | No | The text to type (max 10,000 characters) | |
| fixId | No | Approve a pending suggestedFix (one-shot, 15s TTL). Pass the fixId returned by a previous failed keyboard(action='type') to re-attempt with guard-validated args. | |
| steps | No | Ordered list of key-press steps. Min 1, max 16. Total duration must not exceed 5000ms (excludes settleMs and focus acquisition). N=1 is allowed but inherits the sequence verification contract (hints.verifyDelivery.status='focus_only'); if you want the stricter keyboard:press contract, call keyboard({action:'press', keys}) directly (issue #278, matrix doc §3.1). | |
| action | Yes | Action selector — one of: type, press, sequence. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| lensId | No | Optional perception lens ID. Guards (safe.keyboardTarget) are evaluated before typing, and a perception envelope is attached to post.perception on success. | |
| method | No | Input method. background = WM_CHAR PostMessage (no focus change); foreground = SendInput (current default); auto = pick automatically. | auto |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. rich includes UIA or browser state diff when supported. | minimal |
| settleMs | No | Milliseconds to wait before checking post-action state. | |
| forceFocus | No | Bypass Windows foreground-stealing protection before focusing. | |
| replaceAll | No | When true, send Ctrl+A to select all existing text before typing. Equivalent to Ctrl+A → keyboard(action='type') in one call (requires field already focused). Default false. | |
| trackFocus | No | Detect if focus was stolen after the action. | |
| forceImeOff | No | Issue #245 系統②: when true, query the target window's IME open-status via Imm32 before typing; if ON, switch OFF for the duration of this call and restore the prior state in `finally`. Prevents silent romaji conversion when the user's Japanese IME is active but the LLM is typing ASCII commands. Requires `windowTitle` or `hwnd` (otherwise no target to query). Default false — existing use_clipboard auto-promotion still handles non-ASCII symbols transparently. No-op when the addon predates the IMM bridge (call proceeds with whatever IME state is in effect). | |
| windowTitle | No | Partial title of the window that should receive keyboard input. | |
| use_clipboard | No | If true, copy text to clipboard and paste with Ctrl+V instead of simulating keystrokes. Use this when typing URLs, paths, or ASCII text into apps with Japanese IME active — prevents IME from converting characters. Default false. | |
| forceKeystrokes | No | When true, always use keystroke mode even if text contains non-ASCII content (CJK, emoji, diacritics, em-dash, smart quotes, etc.) that would normally trigger auto-clipboard. Default false — auto-clipboard is enabled. | |
| abortOnFocusLoss | No | Focus Leash Phase B: when true, the foreground keystroke send is split into chunks (default 8 chars; override via DTM_LEASH_CHUNK_SIZE env) and the target window's foreground state is verified between chunks. If the user grabs focus mid-stream, the call aborts and returns FocusLostDuringType with context.typed (chars delivered to target) and context.remaining (unsent tail) so the caller can re-focus and retry the unsent portion. Default: true when windowTitle is provided, false otherwise. Has no effect on the clipboard path (atomic Ctrl+V) or the BG (WM_CHAR) path (HWND-targeted, foreground-independent). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses behavioral traits: auto-clipboard for non-ASCII, background input methods, focus loss detection, blocked keys, and detailed error contexts (e.g., 'FocusLostDuringType', 'BackgroundInputNotDelivered'). It explains per-action verification and edge cases like CJK IME handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with clear sections (Purpose, Details, Prefer, Caveats, Examples). Every sentence adds value given the tool's complexity (19 parameters). Slightly verbose for a simple tool, but appropriate for this one.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description comprehensively covers all actions, parameter interactions, error states, and security restrictions. It references internal context (e.g., 'context.typed', 'context.remaining') and provides examples covering common use cases. It handles all the complexity of a 19-parameter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema coverage is 100%, the description adds significant meaning beyond the input schema. It explains the interplay between parameters (e.g., forceKeystrokes vs auto-clipboard, abortOnFocusLoss default behavior), provides examples for each action, and clarifies conditional requirements like 'Requires windowTitle or hwnd' for forceImeOff.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sends keyboard input to a window with three distinct actions ('type', 'press', 'sequence'). It distinguishes itself from siblings like click_element and desktop_act by explicitly stating what each action does and when to use alternatives, e.g., 'Use desktop_act({action:"setValue"}) for UIA ValuePattern text fields.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use each action (e.g., 'Pass windowTitle to auto-focus and auto-guard...') and when not to use it (e.g., 'CJK IME composition — use use_clipboard=true or desktop_act'). It includes alternatives like 'Use desktop_act({action:"setValue"})' and covers caveats like blocked win+r combos.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
mouse_clickA
Click at screen coordinates. Normally pass windowTitle so the server auto-guards the click (verifies target identity, foreground, coordinate is inside the target rect) and returns post.perception without a confirmation screenshot. origin+scale from dotByDot=true screenshots are converted to screen coords before guarding. doubleClick:true for double-click; tripleClick:true for triple-click (selects a full line of text). Prefer click_element (UIA) for native apps, prefer browser_click for Chrome. Examples: mouse_click({windowTitle:'Notepad', x:200, y:150}) // guarded — post.perception.status='ok'. mouse_click({x:100, y:100}) // unguarded — post.perception.status='unguarded'. If a guard failure returns a suggestedFix, pass its fixId to approve the fix: mouse_click({fixId:'fix-...'}) // one-shot, expires in 15s. lensId is optional and only for advanced pinned-target workflows; omit it for normal use. Caveats: origin+scale are meaningful ONLY with dotByDot=true screenshot responses. hints.verifyDelivery:{status:'delivered'|'focus_only'|'unverifiable', reason} reports the post-click observation in 3 values (focused-element shift, window-foreground change, or no signal). Win11 foreground refusal during the homing path (UIPI cross-elevation / admin-only target / call from a background process or service) returns code:'ForegroundRestricted' ok:false rather than landing the click on the wrong window — recover by switching to a tool that accepts windowTitle directly (click_element / desktop_act) — browser_* tools target by tabId/selector, not windowTitle. MouseClickNotDelivered is reserved-only (false-positive risk is too high to emit a typed code), so degradation is expressed via the 'unverifiable' status, not a separate error.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | X coordinate. Screen-absolute by default. When 'origin' is provided, treated as image-local (pixel position within the screenshot). | |
| y | Yes | Y coordinate. Screen-absolute by default. When 'origin' is provided, treated as image-local. | |
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). Obtain from get_windows response (hwnd field). String type to avoid 64-bit precision issues. | |
| fixId | No | One-shot fix approval ID. If a previous mouse_click returned a suggestedFix, pass that fixId here to approve it. The server revalidates the fix and executes with corrected args. fixId expires in 15 seconds and can only be used once. | |
| scale | No | Scale factor from screenshot response (only when dotByDotMaxDimension caused a resize). Omit if the screenshot was 1:1. Only used when 'origin' is also provided. | |
| speed | No | Cursor movement speed in px/sec. 0 = instant. | |
| button | No | Mouse button to click | left |
| homing | No | Enable homing correction if the target window moved. | |
| lensId | No | Optional perception lens ID for advanced pinned-target workflows. When provided, guards are evaluated before clicking (safe.clickCoordinates, target.identityStable) and a perception envelope is attached to post.perception in the response. For normal use, omit lensId and pass windowTitle directly — Auto Perception handles tracking. | |
| origin | No | When set, (x,y) are image-local coords from a screenshot. Server converts to screen coords: screen_x = origin.x + x / (scale ?? 1), screen_y = origin.y + y / (scale ?? 1). Copy origin values directly from the screenshot response text. This eliminates manual coord math and prevents out-of-window clicks. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. rich includes UIA or browser state diff when supported. | minimal |
| settleMs | No | Milliseconds to wait before checking post-action state. | |
| elementId | No | AutomationId of the UI element. | |
| forceFocus | No | Bypass Windows foreground-stealing protection before focusing. | |
| trackFocus | No | Detect if focus was stolen after the action. | |
| doubleClick | No | Whether to double-click | |
| elementName | No | Name or label of the UI element. | |
| tripleClick | No | Whether to triple-click (select a line of text). Takes precedence over doubleClick when both are true. | |
| windowTitle | No | Partial title of the target window. | |
| verifyDelivery | Yes | Parameter 'verifyDeliveryParam' from the Windows server schema. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden and covers all key behaviors: guarding mechanism, post-click status, fix approval, origin+scale conversion, double/triple-click priority, Win11 foreground restrictions, and verifyDelivery status values.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is comprehensive and front-loaded with the core purpose, but is quite lengthy. However, every sentence adds value and it is well-structured with examples and caveats.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 21 parameters, no output schema, and absent annotations, the description is remarkably thorough, covering usage, errors, recovery, and edge cases like multi-platform differences.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds significant meaning beyond the input schema, especially for `origin`+`scale` conversion, `fixId` expiration and one-shot nature, `lensId` usage, and `hints.verifyDelivery`. Required parameters `x`, `y`, `verifyDelivery` are explained in context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description immediately states 'Click at screen coordinates.' and distinguishes from siblings by recommending `click_element` for native apps and `browser_click` for Chrome, making it clear when to use this tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on guarded vs unguarded clicks (with/without windowTitle), fixId workflow, and preferences over alternatives. Includes examples with expected response statuses.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
mouse_dragA
Click and drag from (startX, startY) to (endX, endY) holding the left mouse button — for sliders, drag-and-drop, canvas drawing, and window resizing. Pass windowTitle so the server auto-guards the start coordinate and returns post.perception. Examples: mouse_drag({windowTitle:'Notepad', startX:50, startY:50, endX:200, endY:200}). lensId is optional and only for advanced pinned-target workflows. Caveats: Left button only. Both start and endpoint are guarded. Cross-window and desktop drags are blocked by default — pass allowCrossWindowDrag:true to confirm intent. hints.verifyDelivery:{status:'delivered'|'focus_only'|'unverifiable', reason} reports the post-drop observation in the same 3-value shape as mouse_click. MouseDragNotDelivered is SUGGESTS-registered but reserved-only (not emitted) — degradation is expressed via the 'unverifiable' status rather than a typed code. Win11 foreground refusal (UIPI cross-elevation / admin-only target / call from a background process or service) returns code:'ForegroundRestricted' ok:false from the homing path.
| Name | Required | Description | Default |
|---|---|---|---|
| endX | Yes | ||
| endY | Yes | ||
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). Obtain from get_windows response (hwnd field). String type to avoid 64-bit precision issues. | |
| speed | No | Cursor movement speed in px/sec. 0 = instant. | |
| homing | No | Enable homing correction if the target window moved. | |
| lensId | No | Optional perception lens ID. Guards and envelope same as mouse_click. | |
| startX | Yes | ||
| startY | Yes | ||
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. rich includes UIA or browser state diff when supported. | minimal |
| windowTitle | No | Partial title of the target window. | |
| allowTabDrag | No | When true, allow drags that start in the title-bar / tab-strip area of a tabbed app (Notepad, Terminal, Edge, Chrome, etc.). Default false — such drags are blocked because they detach the tab into a new window rather than moving the window. Pass true only when you intentionally want to rearrange or detach a tab. Note: active only when auto-guard is enabled (same scope as allowCrossWindowDrag). | |
| verifyDelivery | Yes | Parameter 'verifyDeliveryParam' from the Windows server schema. | |
| allowCrossWindowDrag | No | When true, allow dragging the endpoint into a different window or the desktop background. Default false — cross-window drags (including desktop/wallpaper) are blocked to prevent accidents. Pass true to confirm intent for deliberate cross-window or desktop-area drags. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully covers behavioral traits: left button only, guarded start/end points, blocking cross-window drags by default, Win11 foreground refusal, and detailed hints.verifyDelivery status values.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured but slightly verbose. Starts with core action and examples before diving into parameters and caveats. Every sentence is informative, but some repetition (e.g., block details) slightly reduces conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 14 parameters, no output schema, and no annotations, the description is remarkably complete. Covers use cases, edge cases (cross-window, tab drag), Win11 restrictions, and hints response shape. No obvious gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds significant meaning beyond schema: explains lensId's advanced usage, verifyDelivery shape, allowTabDrag purpose, and homing behavior. Schema coverage is 71%, but description fills gaps with context and examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action ('Click and drag'), the resources (mouse coordinates), and specific use cases (sliders, drag-and-drop, canvas drawing, window resizing). It clearly distinguishes from sibling tools like mouse_click.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use examples and when-not: cross-window drags blocked by default, tab drags blocked unless allowed. Includes examples and notes on optional parameters (lensId) and intentional overrides (allowCrossWindowDrag, allowTabDrag).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
notification_showA
Show a Windows system tray balloon notification to alert the user. Use at the end of a long-running task so the user knows it finished without watching the screen. Caveats: toast の user reach は原理的に観測不能 (matrix §3.1 line 158 規範整合)。Focus Assist (Do Not Disturb) / Notifications-off setting / consent UI sink いずれも tool 側からは判別不能のため、successful response は常に hints.verifyDelivery を含む (status="unverifiable", reason="user_visible_side_effect_uninspectable", channel="win32_balloon_tip" — 全 double-quoted JSON literal)。caller は user 側の post-notification behavior (例: wait_until(focus_changes)) で間接観測することが望ましい。Uses System.Windows.Forms — no external modules needed.
| Name | Required | Description | Default |
|---|---|---|---|
| body | Yes | Notification body text | |
| title | Yes | Notification title | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that the successful response always includes a hints.verifyDelivery object with status 'unverifiable' and explains the reasons (Focus Assist, DND, consent UI). It also mentions the underlying technology (System.Windows.Forms) and that no external modules are needed. However, it does not clarify if the tool is blocking or asynchronous.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is moderately concise but includes Japanese text and a JSON literal that may hinder readability. The first sentence is clear, followed by technical caveats that are all relevant but could be more structured. Every sentence serves a purpose, but the mix of languages and technical jargon reduces conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool has only 3 parameters, no output schema, and no nested objects, the description covers the key aspects: purpose, usage context, delivery verification behavior, and a suggested indirect observation method. It is complete enough for an agent to use correctly, though it does not cover behavior for multiple rapid calls or notification click handling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description does not add significant new meaning beyond what the schema already provides for the title, body, and include parameters. The description focuses on behavioral aspects rather than parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Show a Windows system tray balloon notification to alert the user', providing a specific verb and resource. It distinguishes from sibling tools which are focused on browser interactions, desktop actions, and data processing, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use at the end of a long-running task so the user knows it finished without watching the screen', providing clear context for when to use. It also includes caveats about delivery verification and suggests indirect observation via wait_until, but does not mention alternative tools or explicitly state when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_macroA
Purpose: Execute multiple tools sequentially in one MCP call — eliminates round-trip latency for predictable multi-step workflows. Details: steps[] is an array of {tool, params} objects. Accepts all desktop-touch tools plus a special sleep pseudo-step: {tool:"sleep", params:{ms:N}} (max 10000ms per step). stop_on_error=true (default) halts on first failure. Max 50 steps. The LLM cannot inspect intermediate results during execution — all steps run to completion (or first error) before any output is returned. Prefer: Use for predictable fixed sequences (focus → sleep → type → screenshot). Do not use for conditional logic — return to the LLM between branches so it can inspect intermediate state. Caveats: If any step may fail conditionally (e.g. a dialog that may or may not appear), split the macro at that point. Each screenshot step within a macro incurs the same token cost as a standalone call. Examples: [{tool:'focus_window',params:{windowTitle:'Notepad'}},{tool:'sleep',params:{ms:300}},{tool:'keyboard',params:{action:'type',text:'Hello'}},{tool:'screenshot',params:{detail:'text',windowTitle:'Notepad'}}] [{tool:'browser_navigate',params:{url:'https://example.com'}},{tool:'wait_until',params:{condition:'element_matches',target:{by:'text',pattern:'Example Domain'}}}]
| Name | Required | Description | Default |
|---|---|---|---|
| steps | No | Ordered list of tool calls to execute sequentially (max 50 steps). | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| stop_on_error | No | Stop execution on the first error (default true). Set false to collect all results. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses critical traits: sequential execution, max 50 steps, sleep pseudo-step limitation (max 10s), stop_on_error default, inability to inspect intermediate results, and per-screenshot token cost. This is comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with labeled sections (Purpose, Details, Prefer, Caveats, Examples), making it scannable. It front-loads the core purpose. While a few sentences could be tightened (e.g., the token cost note), nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (3 parameters, nested objects, behavioral nuances) and no output schema, the description covers nearly everything an agent needs to use the tool correctly. It could mention the format of the error output (e.g., partial results when stop_on_error=false), but the missing detail is minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Baseline is 3 because schema coverage is 100%. The description adds value by explaining the steps array structure, the sleep pseudo-step, the stop_on_error behavior, and the include parameter's influence on response shape. The examples further clarify usage. A minor gap: the 'include' parameter's envelope description could be clearer, but overall adds meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear verb and resource: 'Execute multiple tools sequentially in one MCP call'. It immediately distinguishes from individual tool calls by highlighting latency elimination, and the sibling context (many atomic tools) makes the differentiation obvious.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance: 'Prefer: Use for predictable fixed sequences... Do not use for conditional logic'. It tells the agent when to split a macro and provides caveats about conditional failures, making the usage boundary very clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshotA
Purpose: Capture desktop, window, or region across detail levels (meta / text / image / som / ocr) and capture modes (normal / background). Details: detail='meta' (default) returns window titles+positions only (~20 tok/window, no image). detail='text' returns UIA actionable elements with clickAt coords, no image (~100-300 tok). detail='som' returns OCR-detected elements with IDs plus a Set-of-Marks annotated image delivered by-ref by default (bypasses UIA entirely). detail='ocr' returns Windows OCR words with screen-pixel clickAt coords (Phase 4: absorbs former screenshot_ocr — use when UIA is sparse and you want to force OCR unconditionally). detail='image' and detail='som' both return a cheap by-ref resource_link by default (no inline base64); pass confirmImage=true to also embed the inline image (the annotated bitmap for som). mode='background' captures hidden/minimised/occluded windows via PrintWindow (Phase 4: absorbs former screenshot_background) — pair with windowTitle/hwnd. dotByDot=true returns 1:1 pixel WebP; compute screen coords: screen_x = origin_x + image_x (or screen_x = origin_x + image_x / scale when dotByDotMaxDimension is set — scale printed in response). diffMode=true returns only changed windows after the first call (~160 tok). region={x,y,width,height} captures a sub-rectangle (Phase 4: absorbs former scope_element when paired with windowTitle/hwnd — discover element bounds via desktop_discover, then pass region here). Data reduction: grayscale=true (−50%), dotByDotMaxDimension=1280 (caps longest edge), windowTitle+region (sub-crop to exclude browser chrome — e.g. region={x:0, y:120, width:1920, height:900}). Prefer: Use meta to orient, text before clicking, dotByDot only when precise pixel coords are needed. Use detail='som' for native apps or games that do not expose UIA elements (UIA-Blind). Use detail='ocr' for OCR-only (skip UIA entirely). Use mode='background' when the target window must stay hidden or cannot be brought to foreground. Prefer browser_* tools for Chrome. Use diffMode after actions to confirm state changed. Only use image+confirmImage when text returned 0 actionable elements and visual inspection is genuinely required. Caveats: Default mode scales to maxDimension=768 — image pixels ≠ screen pixels; apply the scale formula before passing to mouse_click. Foreground detail='image' returns a by-ref resource_link by default; pass confirmImage=true to also receive inline pixels. diffMode requires a prior full-capture baseline (non-diff call or workspace_snapshot) — calling diffMode cold returns a full frame, not a diff. mode='background' requires windowTitle or hwnd, and only composes with detail in {'image','meta'} — detail='text'/'som'/'ocr' run only against foreground capture (the dispatcher rejects the conflicting combination). Passing mode='background' is itself the acknowledgement that image pixels are wanted, so confirmImage is NOT required for it (matches the former screenshot_background contract). fullContent=false enables legacy mode (faster but GPU windows may be black). detail='ocr' requires windowTitle or hwnd; first call may take ~1s (WinRT cold-start) and the matching OCR language pack must be installed. Examples: screenshot() → meta orientation of all windows screenshot({detail:'text', windowTitle:'Notepad'}) → clickable elements with coords screenshot({detail:'ocr', windowTitle:'PDF', ocrLanguage:'ja'}) → OCR words with screen-pixel coords screenshot({mode:'background', windowTitle:'Chrome', dotByDot:true, dotByDotMaxDimension:1280, grayscale:true}) → background-capture pixel-accurate Chrome screenshot({windowTitle:'Notepad', region:{x:0,y:120,width:600,height:400}}) → cropped sub-region (zoom into element after desktop_discover)
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). Obtain from desktop_discover (windows[].hwnd). String type to avoid 64-bit precision issues. | |
| mode | No | Capture mode. 'normal' — default. Window-targeted captures (windowTitle / hwnd) use Win32 PrintWindow with automatic BitBlt fallback when PrintWindow returns no data or an all-black frame; the route used is reported in hints.captureSource. Fullscreen / displayId captures use BitBlt. 'background' — explicit Win32 PrintWindow capture, retained for back-compat and explicit selection. Requires windowTitle (or hwnd). Pair with fullContent for GPU-rendered apps. | normal |
| detail | No | Response detail level (omit to let the server pick a smart default): omitted — auto: 'image' when dotByDot/region/displayId is specified, else 'meta' 'meta' — window title + screen region only (~20 tok/window, cheapest) 'text' — UIA element tree as JSON with text values (~100-300 tok/window, no image) 'image' — actual screenshot pixels. Returns a cheap by-ref resource_link by default (no inline base64); pass confirmImage=true to ALSO embed the inline image. 'som' — Set-of-Marks elements + annotated image (bypasses UIA entirely). Returns the OCR elements[] plus a cheap by-ref resource_link by default (no inline base64); pass confirmImage=true to ALSO embed the annotated bitmap. 'ocr' — Windows OCR words with screen-pixel clickAt coords (Phase 4: absorbs former screenshot_ocr). Use when UIA returns no actionable elements (WinUI3 custom-drawn UIs, game overlays, PDF viewers). Note: detail='text' auto-falls back to OCR via ocrFallback='auto'; choose detail='ocr' only when forcing OCR unconditionally. | |
| region | No | Capture only this sub-region. Without windowTitle: virtual screen coordinates. With windowTitle: window-local coordinates — useful to exclude browser chrome (tabs/address bar). Example: windowTitle='Chrome', region={x:0, y:120, width:1920, height:900} skips the 120px browser chrome. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| diffMode | No | Layer diff mode — compares each window against the buffered previous frame. First call = full I-frame (all windows). Subsequent calls = only changed windows (P-frame). Implicitly enables dotByDot. Best used with windowTitle=undefined to snapshot all windows. | |
| dotByDot | No | 1:1 pixel mode — no scaling, WebP compression. Window captures include 'origin: (x,y)' so you can compute screen position: screen_x = origin_x + image_x. When dotByDotMaxDimension is also set, scale factor is included: screen_x = origin_x + image_x / scale. | |
| displayId | No | Capture a specific monitor (0 = primary). Use desktop_state({includeScreen:true}) to list displays. | |
| grayscale | No | Convert to grayscale before encoding. Reduces file size ~50% for text-heavy content (e.g. AWS console, code editors). Avoid when color is meaningful (charts, status indicators). | |
| fullContent | No | When mode='background', use PW_RENDERFULLCONTENT to capture GPU-rendered windows (Chrome, Electron, WinUI3). Default true. Set false for legacy mode (faster but GPU windows may appear black). Ignored unless mode='background'. | |
| ocrFallback | No | OCR fallback behaviour when detail='text'. 'auto' (default): fire Windows OCR if UIA returns 0 actionable elements OR hints.uiaSparse=true (UIA returned <5 elements, typical for Chrome). 'always': always augment actionable[] with OCR words. 'never': disable OCR entirely. | auto |
| ocrLanguage | No | BCP-47 language tag for the OCR engine (e.g. 'ja', 'en-US'). Auto-detects from system locale when omitted. Used when detail='text' (OCR fallback) or detail='ocr' (direct OCR). | |
| webpQuality | No | WebP quality when dotByDot=true or diffMode=true. 40=layout only, 60=general (default), 80=fine text. | |
| windowTitle | No | Capture only the window whose title contains this string. Use '@active' for the current foreground window. Prefer over full-screen when target window is known. | |
| confirmImage | No | Embed inline image pixels in the response. detail='image' now returns a cheap by-ref resource_link WITHOUT this flag (it is no longer blocked); confirmImage=true ADDITIONALLY embeds the inline image for immediate vision. detail='som' likewise returns its elements[] + a by-ref resource_link by default; confirmImage=true ADDITIONALLY inlines the annotated SoM bitmap. Prefer detail='text' / diffMode=true / dotByDot=true first — set confirmImage=true only when inline visual inspection is genuinely required. | |
| maxDimension | No | Max width or height in pixels (default 768). Use 1280 to read small text, code, or fine UI details. Ignored when dotByDot=true. | |
| preprocessPolicy | No | OCR preprocessing scale policy for detail='som' and OCR fallback paths. 'auto' (default): clamp scale to 1 on OOM (>8MP) or high-DPI (≥150%). 'aggressive': relaxes DPI clamp to 175%, preserving upscale on 150%-DPI monitors (e.g. Outlook PWA). Also auto-enables adaptive binarization. 'minimal': always scale=1 regardless of DPI/resolution. | auto |
| preprocessAdaptive | No | When true, apply Sauvola adaptive binarization after contrast stretch. Improves recognition of thin text on low-contrast or gradient backgrounds. Automatically enabled when preprocessPolicy='aggressive'. Requires Rust native engine; silently skipped otherwise. | |
| dotByDotMaxDimension | No | Cap the longest edge (pixels) when dotByDot=true. Reduces payload while preserving coordinate math. Example: 1280 on a 1920×1080 screen → scale≈0.667. Response includes scale factor: screen_x = origin_x + image_x / scale. Recommended for Chrome: dotByDot=true, dotByDotMaxDimension=1280, grayscale=true. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses key behaviors: default scaling to maxDimension=768, by-ref resource links, coordinate math, diffMode requiring a baseline, background mode requirements, mode-detail compatibility, fullContent defaults, and fallbacks. Every important behavioral trait is documented.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with sections (Purpose, Details, Prefer, Caveats, Examples). It is front-loaded with the overall purpose. Every sentence adds value given the tool's complexity (19 parameters). A slightly tighter edit could reduce redundancy, but it remains clear and organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (19 parameters, multiple modes, no output schema), the description is remarkably complete. It covers all modes, parameters, integration with sibling tools (desktop_discover, mouse_click), and provides examples for common scenarios. No gaps are apparent for an AI agent to correctly invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already documented. The description adds significant value: it provides examples for combining parameters, explains behavioral context (e.g., confirmImage interaction with detail='image'/'som'), and clarifies coordinate systems for region. This enriches understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Capture desktop, window, or region across detail levels (meta / text / image / som / ocr) and capture modes (normal / background).' It distinguishes itself from sibling tools by noting when to prefer browser_* tools for Chrome, and the examples cover common use cases. The specificity of verb+resource+scope is excellent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Prefer:' section explicitly provides when-to-use guidance for each detail level and mode, and gives alternatives (e.g., 'Use browser_* tools for Chrome', 'Use diffMode after actions to confirm state changed'). The 'Caveats' section clarifies restrictions and prerequisites. This is comprehensive and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshot_gcA
Reclaim disk space from cached screenshots by retention policy. By DEFAULT this is a dry run: it returns the captures that WOULD be deleted (candidates) plus a count/size of leftover orphan files, and deletes nothing. To actually delete, pass BOTH dryRun:false AND confirm:true. Retention caps (all optional): maxCount (keep newest N), maxTotalBytes (keep newest under a byte budget), maxAgeMs (delete older than). When you pass none, the cache's env defaults apply (newest 200 / 256 MiB). Scope to a single tag with tag (other tags are never touched); includeOrphans (default true) also reclaims leftover on-disk files with no index entry. The newest capture is always kept by the count/byte caps. Only ever touches files inside the screenshot cache — never any other path.
| Name | Required | Description | Default |
|---|---|---|---|
| tag | No | Limit deletion to captures under this tag (case-insensitive). Other tags are never touched. | |
| dryRun | No | Default true: only LIST what would be deleted, delete nothing. Set false (with confirm:true) to actually delete. | |
| confirm | No | Safety gate: deletion happens ONLY when dryRun:false AND confirm:true. Otherwise the call is forced to a dry run. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| maxAgeMs | No | Delete captures older than this many milliseconds (opt-in; can clear even the newest). | |
| maxCount | No | Keep only the newest N captures; delete the rest. The single newest is always kept. | |
| maxTotalBytes | No | Keep the newest captures under this total byte budget; delete older ones beyond it. The newest is always kept. | |
| includeOrphans | No | Default true: also reclaim leftover on-disk image files that are not tracked in the cache index (e.g. files left behind by a crash). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description fully discloses behavior: default dry run, safety gate, retention caps (maxCount, maxTotalBytes, maxAgeMs), orphan handling, and that only the screenshot cache is touched. It also explains that the newest capture is always kept.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single dense paragraph but well-organized: starts with purpose, then default behavior, then parameters. It is concise and front-loaded, though slightly packed. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description partially covers return values (candidates, count/size of orphans). However, it does not detail the exact structure of the response, which may leave the agent uncertain about what to expect from the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the description still adds value beyond the schema: explains defaults, default behavior (dry run), retention policy nuances, and that `includeOrphans` is true by default. This enriches the agent's understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Reclaim disk space from cached screenshots by retention policy.' This clearly distinguishes it from sibling tools like `screenshot` (which captures screenshots) and `screenshot_query` (which queries captures).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains default dry-run behavior and the safety gate for actual deletion (both `dryRun:false` and `confirm:true`). It covers retention caps and scope. However, it does not explicitly state when not to use this tool or offer direct alternatives, but the context is clear for a cleanup tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshot_queryA
List screenshots already saved in the disk-cache WITHOUT re-reading any pixels. The screenshot tools return each capture as a cheap by-ref link (screenshot://by-ref/{captureId}); this lists what is in the cache — captureId + by-ref uri, dimensions, size in bytes, timestamp, and tag/window — so you can find and re-open a specific earlier capture, or check how much the cache holds before reclaiming space with screenshot_gc. The response also carries whole-cache totals (totalCaptures / totalBytes). Reading a capture's bytes still costs tokens, so open a by-ref link only when you actually need to inspect the pixels. Filter by tag (case-insensitive) / windowUuid / since / until; page with limit (default 50) and offset. Results are newest-first and never include a filesystem path.
| Name | Required | Description | Default |
|---|---|---|---|
| tag | No | Filter to captures stored under this tag (case-insensitive). Omit to list all. | |
| limit | No | Maximum rows to return, newest first (default 50, max 500). | |
| since | No | Only captures taken at/after this time (epoch milliseconds, inclusive). | |
| until | No | Only captures taken at/before this time (epoch milliseconds, inclusive). | |
| offset | No | Rows to skip from the newest end, for paging (default 0). | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| windowUuid | No | Filter to captures of a specific window (the window's stable id). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description fully discloses non-destructive behavior (no pixel read), response fields, ordering (newest-first), and a critical warning about token costs for reading bytes. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Compact ~120 words, front-loaded with core purpose and key differentiators ('WITHOUT re-reading'). Each sentence adds unique value, no fluff. Structured logically from behavior to filtering to caveats.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, description enumerates response fields (captureId, uri, dimensions, size, timestamp, tag/window, cache totals) and covers all relevant aspects: behavior, filtering, pagination, ordering, and token-warning. Complete for a list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters, so baseline is 3. Description adds value by specifying filter semantics (case-insensitive tag, epoch ms timestamps), default/limits (limit 50 max 500, offset 0), and response shape opt-in, which go beyond schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists screenshots from disk-cache without re-reading pixels, contrasting with sibling screenshot (capture) and screenshot_gc (reclaim space). It precisely describes the action and resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use (find/re-open captures, check cache, decide GC) and when not to (avoid opening by-ref links unless needed due to token cost). Mentions alternative screenshot_gc for reclaiming space.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scrollA
Purpose: Scroll a window or page. 5 strategies via action: 'raw' (wheel notches), 'to_element' (UIA name/automationId or CSS selector), 'smart' (auto-detect target with multi-strategy fallback), 'capture' (full-page stitched image), 'read' (scroll+OCR+dedupe → stitched text). Details: action='raw': send raw mouse-wheel notches at (x,y) or current cursor, optional window focus. Scroll scale — UIA Tier 1 (ScrollPattern apps): empirically ≈1 text line per notch; amount:3 (default) ≈ 3 lines (small nudge), amount:10 ≈ 10 lines (~½ visible area). Legacy SendInput: each amount unit = 3 wheel ticks; ≈9 text lines per unit at Windows default (app/OS-setting dependent). action='to_element': scroll a named element into viewport (UIA or CDP). action='smart': handles nested scroll layers, virtualised lists, sticky-header occlusion. action='capture': stitches full-page images (caps at ~700KB raw); sizeReduced=true means downscaled. action='read': scrolls page-by-page, OCRs each viewport, deduplicates overlapping lines, returns stitched text; language auto-detected from OS locale if omitted. Prefer: Use action='to_element' or action='smart' for click target out-of-viewport recovery (entity_outside_viewport). Use action='capture' for reading long pages as images. Use action='read' for extracting text from long native-app documents (PDF readers, text editors, terminals) where copy-paste is unavailable. For simple scroll without target, use action='raw'. Caveats: action='capture' returns stitched image — pixels do NOT match screen coords when sizeReduced=true, use for reading only, not mouse_click. action='smart' CDP path requires browser_open. action='to_element' native path requires element to implement UIA ScrollItemPattern. action='read' uses OCR (imperfect accuracy) and requires the window to be visible; for browser pages prefer browser_eval or browser_overview for accurate DOM text. action='raw' typed errors: code:'ScrollNotDelivered' on silent drop (overlay / non-scrollable / UIPI low-IL); already-at-boundary is success via pre/post-percent disambiguation. hints.verifyDelivery.{channel,reason} per ADR-018 §2.6 (Phase 1b: Tier 1 UIA dispatch for HWNDs exposing ScrollPattern; other apps use legacy SendInput). action='smart' typed errors: code:'OverflowHiddenAncestor' (retry with expandHidden:true), code:'VirtualScrollExhausted' (provide virtualIndex). Examples: scroll({action:'raw', direction:'down', amount:5, windowTitle:'Chrome'}) scroll({action:'to_element', name:'OK', windowTitle:'Dialog'}) scroll({action:'smart', target:'#create-release-btn'}) scroll({action:'capture', windowTitle:'Chrome', maxScrolls:10}) scroll({action:'read', windowTitle:'Acrobat', maxPages:15}) // OCR + dedupe long PDF
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | X coordinate to scroll at (moves cursor there first) | |
| y | No | Y coordinate to scroll at | |
| hint | No | Scroll direction hint for binary-search (image path). Seeds lo/hi bounds to reduce attempts. | |
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). | |
| name | No | Partial name/label of the element (UIA name match). Use for native app elements. At least one of name or selector must be provided. | |
| port | No | CDP port for Chrome path (default 9222) | |
| block | No | Vertical alignment after scroll — start/center/end/nearest (Chrome path only, default: center) | center |
| speed | No | Cursor movement speed in px/sec (0=teleport, omit=default) | |
| tabId | No | Tab ID (Chrome path only). Omit for first page tab. | |
| action | Yes | Action selector — one of: raw, to_element, smart, capture, read. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| amount | No | Number of scroll notches (default 3). UIA-capable apps (Notepad, Explorer, WPF — Tier 1): empirically ≈1 text line per notch; amount:3 (default) ≈ 3 lines (small nudge), amount:10 ≈ 10 lines (~½ visible area). Legacy apps (SendInput path): each amount unit sends 3 wheel ticks; at Windows default 3 lines/tick that is ≈9 text lines per unit — distance varies by app/OS wheel-speed settings. | |
| homing | No | Apply window-movement homing correction to (x,y) before scrolling. Default true. | |
| inline | No | Vertical alignment after scroll (CDP path). Default: center. | center |
| target | No | CSS selector (Chrome/Edge) or partial UIA name (native apps). For CDP path, must be a valid CSS selector (starts with #, ., tag, or [ ). For UIA path, a partial name match against element Name property. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| language | No | OCR language code (e.g. 'ja', 'en', 'zh'). Omit to auto-detect from Windows system locale via Intl.DateTimeFormat().resolvedOptions().locale. Default: auto. | |
| maxDepth | No | Max number of ancestor scroll containers to walk. Default 3. | |
| maxPages | No | Maximum number of scroll steps / OCR pages (default 20, max 50). | |
| maxWidth | No | Max size of the short edge of the final image (default 1280). For 'down': caps the image width; height is unconstrained. For 'right': caps the image height; width is unconstrained. | |
| selector | No | CSS selector for the element (Chrome/Edge only). At least one of name or selector must be provided. | |
| strategy | No | auto (default): try CDP → UIA → image in order. cdp: Chrome/Edge only. uia: native Windows UIA. image: image + Win32 binary-search. | auto |
| direction | No | Scroll direction | |
| scrollKey | No | Key sent to scroll one page. PageDown (default): full-page scroll for most apps. Space: web/PDF readers. ArrowDown: line-by-line slow scroll. | PageDown |
| maxScrolls | No | Maximum scroll iterations before stopping (default 10, max 30) | |
| retryCount | No | Max scroll attempts (image path binary-search). Default 3, cap 4. | |
| windowTitle | No | Partial window title. When provided, the server focuses this window first. | |
| expandHidden | No | Temporarily set overflow:hidden ancestors to overflow:auto to unlock scroll. Mutates live CSS. | |
| virtualIndex | No | Target row index in a virtualised list (0-based). Enables direct TanStack/data-index seeking. | |
| virtualTotal | No | Total row count in a virtualised list. Required when virtualIndex is set. | |
| scrollDelayMs | No | Milliseconds to wait after each scroll for rendering to settle (default 400). Increase for slow/animated pages. | |
| verifyWithHash | No | Verify scroll effectiveness via perceptual hash comparison. Automatically enabled for image path. | |
| stopWhenNoChange | No | Stop automatically when two consecutive pages yield no new lines after deduplication (page-end detection). Default true. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully covers behavioral traits: scroll scaling details (UIA vs legacy), destructive mutations (expandHidden), OCR accuracy, image stitching size implications, typed errors, and delivery verification. It transparently explains all important side effects and constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but well-structured with clear sections (Purpose, Details, Prefer, Caveats, Examples). Every sentence adds value for a complex tool. Minor redundancy could be trimmed, but the organization compensates for length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 32 parameters, no output schema, and no annotations, the description covers all necessary details: each action's mechanics, parameters, error codes, and usage contexts. It includes multiple examples covering all actions, making it complete for an AI agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds significant meaning beyond the schema. For example, it explains the 'action' parameter's sub-behaviors, the 'amount' parameter's effect across different app tiers, and the 'strategy' parameter's fallback order. This extra context is critical for correct parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Purpose: Scroll a window or page.' and details 5 distinct actions (raw, to_element, smart, capture, read), each with specific use cases. It effectively distinguishes from sibling tools like browser_* and mouse_click by focusing on scroll-specific operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Prefer:' section explicitly states when to use each action (e.g., action='to_element' for click target recovery, action='capture' for reading long pages, action='read' for text extraction from native apps). Caveats further clarify limitations and alternatives, providing comprehensive guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
server_statusA
Return MCP server status. engine: native engine availability — uia: 'native' = Rust UIA addon (~2 ms focus / ~100 ms tree); 'powershell' = PS fallback (~366 ms focus). imageDiff: 'native' = Rust SSE2 SIMD (0.26 ms @ 1080p); 'typescript' = TS fallback (~3.8 ms). health: process diagnostic snapshot (issue #365) — uptimeSec, memory.{rssBytes,heapUsedBytes,heapTotalBytes}, cpu.{userUs,systemUs} (cumulative since startup), shutdown.{pending,graceMs,inflightCount} (pending=true means stdin EOF received and grace timer is running), lastRpc.{receivedAt(ISO),method} (last JSON-RPC request observed on stdio transport; HTTP transport is not tracked). Diagnostic metadata — do not surface unless the user asks about performance/troubleshooting. engine values are stable for the process lifetime; health values change per call.
| Name | Required | Description | Default |
|---|---|---|---|
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description fully handles behavioral disclosure. It explains what each field returns, notes that engine values are stable while health values change per call, and mentions a limitation (lastRpc only tracks stdio transport). No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is relatively long but well-structured with clear sections for engine, imageDiff, and health. It front-loads the purpose and each sentence adds value. Slight verbosity in health subfields but acceptable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (three main components with subfields), one optional parameter, and no output schema, the description is complete. It explains return value semantics, stable vs changing fields, and a transport limitation. All necessary context is provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter ('include') with 100% schema description coverage. The description does not add extra meaning beyond the schema; the schema already clearly explains the parameter. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Return MCP server status.' which is a clear verb+resource. It then details the specific components (engine, imageDiff, health) and distinguishes the tool from siblings by being about server diagnostics, not UI automation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use ('Diagnostic metadata — do not surface unless the user asks about performance/troubleshooting.'), implying a conditional use. However, it does not explicitly state when not to use or name alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
shell_sessionA
Purpose: Reliable background shell channel backed by a real OS process (stdin/stdout/stderr + exit code). Use this for installs, scripts, file ops, and any command that must actually finish — not the visible-terminal terminal tool.
Details: Actions: start (spawn hidden shell), write (stdin + optional wait/read), read (drain ring buffer), exec (one-shot run-to-completion with exit code), list, stop. Never reports ok:true on write unless stdin accepted AND process still tracked; exec only returns ok:true when exitCode===0. Non-zero exits return ok:false code:ShellSessionNonZeroExit with stdout/stderr. Default shell: powershell on Windows, bash elsewhere. Independent of GUI focus, IME, clipboard, and SoM terminal pollution.
Prefer: Prefer shell_session({action:'exec', input:'...'}) for one-shot commands. Prefer start+write+read for interactive multi-step shells. Keep terminal for when you must drive a visible console window the user is watching.
Caveats: This does NOT open a visible window — do not use it when the user needs to see the console UI. Ring buffer is capped (default 1MB). Max 16 concurrent sessions. On Windows, powershell/cmd are windowsHide; do not send Ctrl+C via keyboard tools into these sessions (use action:'stop').
Examples:
shell_session({action:'exec', input:'whoami'}) → {ok:true, exitCode:0, stdout:'...'}
shell_session({action:'start', id:'ops', shell:'powershell'}) → {ok:true, session:{pid,...}}
shell_session({action:'write', id:'ops', input:'Get-ChildItem', waitMs:500}) → stdout/stderr
shell_session({action:'read', id:'ops', clear:true})
shell_session({action:'stop', id:'ops'})
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description thoroughly discloses behavior: actions list, exit code handling (ok:false with specific error code for non-zero), ring buffer cap (1MB), concurrency limit (16 sessions), Windows powershell hidden nature, and that it does not open a visible window. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections (Purpose, Actions, Details, Prefer, Caveats, Examples) and front-loaded with essential distinctions. It is longer than minimal but each sentence adds value given the complexity and missing schema. Not excessively verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema and no annotations, the description is remarkably complete: covers all actions, error handling, limits, platform specifics, and provides multiple examples. Agent can correctly invoke and interpret results based on this description alone.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, but the description fully compensates by detailing each action's parameters (action, id, input, waitMs, shell, clear) with examples for exec, start, write, read, stop. Schema coverage is effectively 100% due to description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a 'reliable background shell channel backed by a real OS process' and instructs to use it for 'installs, scripts, file ops, and any command that must actually finish'. It explicitly distinguishes from the sibling 'terminal' tool, making the purpose and scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: prefer 'exec' for one-shot, 'start+write+read' for interactive sessions, and to use 'terminal' only when a visible console is needed. Also includes caveats about not using when the user needs to see the UI, with concrete alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
terminalA
Purpose: Interact with a terminal window: read output, send input, or run+wait+read in one call. action='read' / action='send' absorb the formerly-standalone read/send tools (Phase 4).
Details: action='run' is the recommended high-level workflow: send command → wait until quiet/pattern/timeout → read output. The command text is passed as input (the legacy parameter name command is also accepted as a deprecated alias — see issue #245). Returns completion={reason, elapsedMs} first-class plus outputIntegrity:'ok'|'baseline_lost' so callers can detect when scrollback could not be anchored to the pre-send buffer. action='read' reads current text via UIA TextPattern (falls back to OCR); use sinceMarker for incremental diff. action='send' sends a command with focus management.
Prefer: action='run' for command execution + result. For long-running commands (test runners, builds, deploys) use until:{mode:'pattern', pattern:''} — the default quiet mode is tuned for short interactive commands and may complete prematurely on multi-second silent gaps mid-run. Use action='read'/'send' for fine-grained control or when you need to interleave other actions.
Caveats: Do not screenshot the terminal — action='read' is cheaper and structured. action='run' supports completion reasons: quiet | pattern_matched | exited | timeout | window_closed | window_not_found | send_failed (send rejected on a live window — see warnings for the underlying error code). until:{mode:'exit', shell:'bash'|'powershell'} (issue #386) returns completion.exitCode + reason:'exited' via an echo-immune sentinel that works for multiline input that pattern mode cannot anchor; pass shell explicitly (auto fails as ExitModeShellAmbiguous on WT/conhost/SSH), cmd is unsupported (ExitModeShellUnsupported), open-construct input is rejected (ExitModeUnsafeInput). When outputIntegrity:'baseline_lost' is returned, output is forced to '' and readError.code='BaselineMarkerLost' is set: rerun with until:{mode:'pattern',...} or longer timeoutMs. action='run' may also emit warnings prefixed FileLockCollision: when output reveals an EBUSY/Windows-lock/EAGAIN-EDEADLK file collision (e.g. shell '>' redirect colliding with the script's own writer — issue #236). Default quietMs=1500 (issue #196); long silences require pattern mode. preferClipboard=true (send default) overwrites clipboard. Hidden-input prompts emit verifyDelivery.unverifiable (reason:'hidden_input_prompt') — use method:'foreground'. action='read' typed errors: TerminalWindowNotFound, TerminalTextPatternUnavailable (force source:'ocr'); stale sinceMarker → hints.terminalMarker.previousMatched:false on ok:true (omit sinceMarker). FG-path Win11 foreground refusal returns code:'ForegroundRestricted' — switch to method:'background' or DTM_BG_AUTO=1. BG path auto-engages only when (a) the target window class is ConsoleWindowClass (conhost: cmd / PowerShell / pwsh classic hosts) OR (b) env DTM_BG_AUTO=1 is set globally. Windows Terminal (CASCADIA_HOSTING_WINDOW_CLASS) is intentionally EXCLUDED from auto-engage (issue #173): WT runs on WinUI/XAML and silently drops WM_CHAR posted to its HWND, so the FG path is used by default — pass sendOptions:{method:'background'} only if you have verified your WT build accepts BG input.
Examples:
terminal({action:'run', windowTitle:'PowerShell', input:'npm test', until:{mode:'pattern', pattern:'Test Files'}}) → recommended for test runners; matches when vitest summary appears
terminal({action:'run', windowTitle:'pwsh', input:'ls'}) → quiet 1500ms wait, returns output (short interactive)
terminal({action:'run', windowTitle:'pwsh', command:'ls'}) → identical to the above; command is a deprecated alias of input (issue #245)
terminal({action:'read', windowTitle:'PowerShell', sinceMarker:'...'}) → incremental diff using the read action
terminal({action:'send', windowTitle:'PowerShell', input:'echo hello'}) → sends text + Enter using the send action
| Name | Required | Description | Default |
|---|---|---|---|
| input | No | Command to send (Enter is appended automatically). Either `input` or its deprecated alias `command` is required. | |
| until | No | ||
| action | Yes | Action selector — one of: read, send, run. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| command | No | [Deprecated alias of `input`] Accepted for callers that mis-remember the parameter name; new code should use `input`. If both are set, `input` wins. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| timeoutMs | No | Hard timeout in ms (default 30s) | |
| readOptions | No | Extra options forwarded to terminal read (lines, source, ocrLanguage, etc.) | |
| sendOptions | No | Extra options forwarded to terminal send (method, chunkSize, etc.) | |
| windowTitle | No | Partial title of the terminal window (e.g. 'PowerShell', 'pwsh', 'WindowsTerminal'). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the burden. It details completion reasons, outputIntegrity, baseline_lost, file lock collisions, hidden-input prompts, and background/foreground method behavior. This comprehensive disclosure exceeds expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear top-down flow (purpose, details, prefer, caveats, examples). However, it is quite lengthy; some repetition could be trimmed without losing essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 parameters, nested objects, no output schema), the description covers all necessary aspects: actions, return values, edge cases, and error handling. It provides enough context for an agent to use the tool correctly in various scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is high (89%), but the description adds significant value beyond the schema: it explains the 'until' parameter structure, illustrates with examples, clarifies the deprecated 'command' alias, and describes action-specific required fields. This greatly enhances parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is for interacting with a terminal window, with three specific actions (read, send, run). It distinguishes each action's role and scope, making it easy for an agent to understand what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use each action, including preferring 'run' for command execution, using pattern mode for long-running commands, and avoiding screenshotting. It also advises on fine-grained control with read/send. This effectively helps the agent choose the correct action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
wait_untilA
Purpose: Server-side poll for an observable condition — eliminates screenshot-polling loops when waiting for state changes. Details: condition selects what to watch: window_appears/window_disappears (target.windowTitle required), focus_changes (optional target.fromHwnd), element_appears/value_changes (target.windowTitle + target.elementName required, UIA; min 500ms interval), ready_state (target.windowTitle; visible + not minimized), terminal_output_contains (target.windowTitle + target.pattern required [+target.regex:true], needs terminal tools loaded), element_matches (target.by + target.pattern required, needs browser tools loaded), url_matches (target.pattern required [+target.regex:true]; matches the active tab's location.href via CDP — use for SPA route changes, redirects, OAuth flows). Returns {ok:true, elapsedMs, observed} on success, or WaitTimeout error with suggest hints. timeoutMs default 5000 (max 60000). Prefer: Use instead of run_macro({sleep:N}) + screenshot loops. Use terminal_output_contains to detect CLI command completion. Use element_matches for browser DOM readiness after navigation. Use url_matches when the URL is the most reliable signal (SPA routing / redirect cascades). Caveats: terminal_output_contains, element_matches, and url_matches require a browser CDP connection (open --remote-debugging-port=9222 first). element_appears/value_changes spawn a UIA process per poll — interval clamped to 500ms minimum. On elapsed-timeout the response is {ok:false, code:'WaitTimeout', error, suggest:[...]}; the suggest[] array lists three fixed actions: 'Increase timeoutMs', 'Verify the target is correct', 'Inspect intermediate state with screenshot(detail='meta')'. Non-timeout failures also occur — pre-poll validation and missing-hook errors classify as code:'ToolError' (read the descriptive error message), and CDP probe errors (url_matches / element_matches conditions) surface as code:'BrowserNotConnected' (re-attach via browser_open). Branch on code rather than assume WaitTimeout. Examples: wait_until({condition:'window_appears', target:{windowTitle:'Save As'}, timeoutMs:10000}) wait_until({condition:'terminal_output_contains', target:{windowTitle:'Terminal', pattern:'$ '}, timeoutMs:30000}) wait_until({condition:'element_matches', target:{by:'text', pattern:'Submit', scope:'#checkout-form'}}) wait_until({condition:'url_matches', target:{pattern:'/dashboard'}, timeoutMs:15000}) wait_until({condition:'url_matches', target:{pattern:'^https://app\\.example\\.com/orders/[0-9]+$', regex:true}})
| Name | Required | Description | Default |
|---|---|---|---|
| target | No | Target descriptor — fields used depend on condition. Accepts an object literal or a JSON-stringified object. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| condition | Yes | Condition to wait for. See per-condition target requirements. | |
| timeoutMs | No | Maximum time to wait (default 5000ms) | |
| intervalMs | No | Poll interval (default 200ms — terminal_output_contains uses 500 internally) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description fully discloses behavior: polling mechanics, return format ({ok:true, elapsedMs, observed}), timeout behavior (WaitTimeout with suggest hints), non-timeout failure modes (ToolError, BrowserNotConnected), and implementation details (UIA process per poll, interval clamping).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections (Purpose, Details, Prefer, Caveats, Examples) and front-loaded purpose. While verbose, each sentence adds unique value; minor redundancy in some examples could be trimmed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 conditions, multiple error paths, prerequisites), the description is exhaustive: it covers all conditions, target requirements, error codes, and provides examples for various use cases. The lack of an output schema is compensated by clear return value documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the description adds significant value by explaining each condition's target requirements, providing examples, and elaborating on optional parameters like intervalMs and include, making the tool far more usable than the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose as a server-side poll for observable conditions, listing specific conditions (e.g., window_appears, url_matches) and distinguishing it from screenshot-polling loops by sibling tools like run_macro and screenshot.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises when to use this tool over alternatives (e.g., 'Use instead of run_macro({sleep:N}) + screenshot loops'), provides condition-specific guidance (use url_matches for SPA route changes), and includes caveats about prerequisites (browser CDP connection) and error handling with suggested actions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
window_dockA
Purpose: Decorate a window: pin (always-on-top), unpin, or dock (move + resize + optional pin). Details: action='pin' makes window always-on-top until unpin/duration_ms. action='unpin' removes always-on-top. action='dock' positions to corner with width/height (default 480×360 bottom-right) and optionally pins. Minimized windows are automatically restored before docking. Prefer: Use action='dock' for terminal/CLI window auto-positioning at session start. Use action='pin' alone when you only need always-on-top without moving or resizing. Caveats: Pin survives minimize/restore; explicit action='unpin' needed to release. Dock fails on elevated processes. Dock overrides any existing Win+Arrow snap arrangement. Examples: window_dock({action:'dock', title:'PowerShell', corner:'bottom-right', width:480, height:360}) window_dock({action:'pin', title:'Settings', duration_ms:5000}) window_dock({action:'unpin', title:'Settings'})
| Name | Required | Description | Default |
|---|---|---|---|
| pin | No | If true, set always-on-top so the docked window stays visible on top of other windows. Use window_dock(action='unpin') to remove the topmost flag later. Default true. | |
| title | No | Partial window title (case-insensitive) | |
| width | No | Window width in pixels after docking. Default 480. | |
| action | Yes | Action selector — one of: pin, unpin, dock. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| corner | No | Screen corner to snap the window to. Default 'bottom-right'. | bottom-right |
| height | No | Window height in pixels after docking. Default 360. | |
| margin | No | Pixel padding between the window and the screen edge. Default 8. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| monitorId | No | Monitor to dock on (from desktop_state({includeScreen:true})). Omit for primary monitor. | |
| duration_ms | No | Auto-unpin after this many ms (0–60000). Omit to pin indefinitely. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully bears the burden of behavioral disclosure. It explains pin survival across minimize/restore, need for explicit unpin, dock failure on elevated processes, override of Windows snap, and auto-restoration of minimized windows before docking.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with labeled sections (Purpose, Details, Prefer, Caveats, Examples), front-loads the purpose, and every sentence adds value. It is concise yet comprehensive, fitting all necessary information into a manageable length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (10 parameters, 3 actions) and lack of output schema, the description provides thorough coverage of usage and behavior, including examples. However, it does not describe the return value or error handling, which would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds value beyond the schema by explaining defaults (480×360 bottom-right), the relationship between action and required fields, and the interplay between pin and duration_ms. However, the schema itself is already clear, so the description only slightly enhances understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs (decorate, pin, unpin, dock) and resources (window), distinguishing it from sibling tools like focus_window. The three actions are explicitly listed and explained.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit 'Prefer' section advising when to use action='dock' versus action='pin' alone, and includes caveats about elevated processes and overriding snap arrangements, giving clear guidance on appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
workspace_launchA
Purpose: Launch an application and wait for its new window to appear, returning title, HWND, and PID. Details: Runs the command via ShellExecute, snapshots the window list before launch, then polls until a new HWND appears (compared by HWND, not title). Returns {windowTitle, hwnd, pid, elapsedMs}. Works for localized window titles (e.g. '電卓' for calc.exe) because detection is HWND-based, not title-based. timeoutMs default 10000. detach=true fires without waiting and returns no window info. Prefer: Use instead of run_macro({exec, sleep, desktop_discover}) combos. Follow with focus_window(windowTitle) to interact with the launched app. Caveats: Single-instance apps that reuse an existing window will not register as a new HWND — call desktop_discover first to check if the window is already open. detach=true returns immediately with no window title or hwnd. Examples: workspace_launch({command:'notepad.exe'}) → {windowTitle:'', hwnd:'...', pid:...} workspace_launch({command:'calc.exe', timeoutMs:15000})
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | Command-line arguments (max 20). | |
| waitMs | No | Milliseconds to wait for the window to appear (default 2000) | |
| command | Yes | Executable name or full path (e.g. 'notepad.exe', 'calc.exe', 'cmd.exe', 'powershell'). | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, description fully discloses behavioral traits: ShellExecute, window snapshot polling, HWND-based detection, localized title handling, timeout and detach behaviors, and single-instance caveat. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with clear headings and front-loaded purpose. No fluff, but includes inaccurate parameter references that harm conciseness. Slight deduction for misleading details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Explains return format, gives examples, and covers caveats. However, the mismatch with actual schema parameters (timeoutMs vs waitMs, missing args and include) leaves gaps in accurate understanding. Adequate but imperfect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. However, description introduces parameters 'timeoutMs' and 'detach' that do not exist in the input schema, and states a default timeout of 10000 while schema shows 'waitMs' default 2000. This inconsistency misleads the agent and reduces value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'launch' and resource 'application', and specifies the waiting behavior and return values. It explicitly distinguishes from sibling tool 'run_macro' by recommending use instead of manual combos.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: prefer over run_macro, follow with focus_window, and caveats for single-instance apps. Also covers detach=true scenario. No ambiguity about when to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
workspace_snapshotA
Purpose: Orient fully in one call — returns display layouts, all window thumbnails (WebP), and per-window actionable element lists with clickAt coords. Details: uiSummary.actionable[] per window includes: action ('click'|'type'|'expand'|'select'), clickAt {x,y} (pass directly to mouse_click), value (current text for editable fields). Runs parallel internally; latency ≈ max(single screenshot), not N×screenshots. Also resets the diffMode buffer so subsequent screenshot(diffMode=true) returns only changes (P-frame). Prefer: Use at session start or after major workspace changes. Use screenshot(detail='meta') for cheap re-orientation within a session. Use screenshot(detail='text', windowTitle=X) for a single-window update. Caveats: Thumbnails are scaled, not 1:1 — use screenshot(dotByDot=true, windowTitle=X) for pixel-accurate coords on a specific window after snapshot. Also: this call resets the screenshot diff baseline (I-frame) and identity tracker as a side effect, so subsequent screenshot(diffMode=true) starts fresh from this snapshot. The reset is not currently exposed in causal/working memory — record an explicit 'workspace_snapshot' step if you need to track the reset point in your causal trail (ADR-010 §11 OQ carry-over for full visibility).
| Name | Required | Description | Default |
|---|---|---|---|
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| includeUiSummary | No | Whether to include UI element summaries for each window | |
| thumbnailMaxDimension | No | Max size of per-window thumbnail images (default 400px) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses important behavioral traits: parallel execution, resetting the diffMode buffer, side effects on diff baseline and identity tracker. It also warns about thumbnail scaling and recommends tracking the reset point. This is thorough and transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with headings (Purpose, Details, Prefer, Caveats) and each section adds value. However, it is somewhat verbose, especially the caveat referencing ADR-010. It could be slightly more concise without losing information, but it remains effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, no output schema, and no annotations, the description is remarkably complete. It explains what is returned, side effects, usage patterns, and caveats. It fully equips the agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all three parameters have descriptions in the input schema). The description does not add significant new meaning beyond the schema; it references 'includeUiSummary' indirectly and mentions thumbnail dimensions, but the schema already covers these. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Orient fully in one call' and enumerates what it returns (display layouts, thumbnails, actionable element lists). It is a specific verb+resource combination and distinguishes from sibling tools like screenshot and screenshot variants.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'Use at session start or after major workspace changes.' It also offers alternatives: 'Use screenshot(detail='meta') for cheap re-orientation' and 'screenshot(detail='text', windowTitle=X) for a single-window update.' This helps the agent decide when to use this tool versus others.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
1 tool update
v1.11.3- Added
shell_session
29 tool updates
v1.11.2- First observed
browser_click - First observed
browser_eval - First observed
browser_fill - First observed
browser_form - First observed
browser_locate - First observed
browser_navigate - First observed
browser_open - First observed
browser_overview - First observed
browser_search - First observed
click_element - First observed
clipboard - First observed
desktop_state - First observed
excel - First observed
focus_window - First observed
keyboard - First observed
mouse_click - First observed
mouse_drag - First observed
notification_show - First observed
run_macro - First observed
screenshot - First observed
screenshot_gc - First observed
screenshot_query - First observed
scroll - First observed
server_status - First observed
terminal - First observed
wait_until - First observed
window_dock - First observed
workspace_launch - First observed
workspace_snapshot
TDQS
Each tool targets a distinct domain or action, with highly detailed descriptions that eliminate ambiguity. Even closely related tools (e.g., browser_click, click_element, mouse_click) are clearly differentiated by target context (browser DOM, UIA, screen coords).
Browser tools follow a consistent 'browser_verb' pattern. Other tools use descriptive names, but some mixing occurs (e.g., click_element vs mouse_click vs keyboard). Overall, the naming is predictable and readable.
With 29 tools, the set is large but justified by the extensive scope (browser, native UI, terminal, Excel, clipboard, etc.). Each tool serves a specific purpose, though some consolidation could reduce count slightly.
The tool surface covers core desktop automation tasks comprehensively. Minor gaps exist, such as the absence of a dedicated native text-reading tool (though OCR and terminal actions fill the gap) and no explicit file/process management, but these are not critical for the stated domain.
Maintenance
Related MCP Connectors
Eyes and hands on real Windows PCs — observe, click, type via Glasswarp API.
OCR, transcription, file extraction, and image generation for AI agents via MCP.
Your org's AI agents, tasks, runs, search, and brain files as MCP tools and resources.
A paid remote MCP for AI agent browser DevTools MCP, built to return verdicts, receipts, usage logs,
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI clients to automate Windows desktop applications through window manipulation, image recognition, OCR, keyboard/mouse simulation, and memory operations via the MCP protocol.MIT
- AlicenseAqualityDmaintenanceEnables LLM agents to capture screenshots, control mouse/keyboard, and manage windows on desktop platforms, primarily Windows, via an MCP server.161MIT
- AlicenseNot gradedqualityCmaintenanceGives AI agents and MCP clients direct control over native desktop apps, Chrome/Electron browsers, and Android devices with screenshots, OCR, accessibility-based element lookup, input simulation, window management, CDP, and ADB in one local server.128MIT
- AlicenseNot gradedqualityCmaintenanceA local, dependency-free MCP server that gives AI agents controlled access to the active Windows desktop, enabling automated interaction with applications through screenshots, clicks, typing, and window management.87MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/wangkang1133/desktop-touch-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server