macos-control-mcp
Provides browser automation capabilities in Google Chrome, including executing JavaScript, extracting page text, clicking elements by CSS selector, and filling form fields.
Allows AI agents to control the macOS operating system directly: capture screenshots, read screen text via OCR, click, type, scroll, manage apps, and access the accessibility tree.
Provides browser automation capabilities in Safari, including executing JavaScript, extracting page text, clicking elements by CSS selector, and filling form fields.
Enables control of the Slack desktop app through screen interaction and key inputs, allowing actions such as sending messages, switching channels, and navigating the interface.
Enables control of the Spotify desktop app through screen interaction, supporting play/pause, volume adjustment, and navigation of the player interface.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@macos-control-mcpFind the login button on the screen and click it."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
macos-control-mcp
Give AI agents eyes and hands on macOS.
What is this?
An MCP server that lets AI agents see your screen, read text on it, and interact — click, type, scroll — just like a human sitting at the keyboard. Unlike blind script runners, this MCP gives agents state awareness: they screenshot the screen, OCR it to get text with pixel coordinates, then click exactly where they need to.
Related MCP server: computer-use
The See-Think-Act Loop
┌─────────────────────────────────────────────────┐
│ │
│ 1. SEE screenshot / screen_ocr │
│ ↓ "What's on the screen?" │
│ │
│ 2. THINK AI reasons about the content │
│ ↓ "I need to click the Save btn" │
│ │
│ 3. ACT click_at / type_text / press_key│
│ "Click at (425, 300)" │
│ │
│ ↻ repeat │
└─────────────────────────────────────────────────┘This is what makes it powerful: the agent sees the result of every action and can course-correct, retry, or move on — just like you would.
Quick Start
No install needed — run directly with npx:
npx -y macos-control-mcpOn first run, a Python virtual environment is automatically created at ~/.macos-control-mcp/.venv with the required Apple Vision and Quartz frameworks. This takes ~60 seconds once and persists across updates.
Video Showcasing the MCP:
https://www.youtube.com/watch?v=aswlsElHV5o
Configure Your AI Client
All clients use the same command: npx -y macos-control-mcp
Edit ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"macos-control": {
"command": "npx",
"args": ["-y", "macos-control-mcp"]
}
}
}Restart Claude Desktop after saving.
claude mcp add macos-control -- npx -y macos-control-mcpAdd to .vscode/mcp.json in your workspace:
{
"servers": {
"macos-control": {
"command": "npx",
"args": ["-y", "macos-control-mcp"]
}
}
}Add to .cursor/mcp.json in your project:
{
"mcpServers": {
"macos-control": {
"command": "npx",
"args": ["-y", "macos-control-mcp"]
}
}
}Open Cline extension settings → MCP Servers → Add:
{
"macos-control": {
"command": "npx",
"args": ["-y", "macos-control-mcp"]
}
}Add to ~/.codeium/windsurf/mcp_config.json:
{
"mcpServers": {
"macos-control": {
"command": "npx",
"args": ["-y", "macos-control-mcp"]
}
}
}Permissions
macOS requires two permissions for full functionality:
Screen Recording — for screenshots and OCR
Accessibility — for clicking, typing, and reading UI elements
Go to System Settings → Privacy & Security and add your terminal app (Terminal, iTerm2, VS Code, etc.) to both lists. You'll be prompted on first use.
Tools (19)
See the screen
Tool | Description |
| Capture full screen or app window as JPEG |
| OCR the screen — returns text elements with pixel coordinates |
| Find specific text and get clickable x,y coordinates |
Interact with the screen
Tool | Description |
| Click at x,y coordinates (returns screenshot) |
| Double-click at x,y (returns screenshot) |
| Type text into the frontmost app |
| Press key combos (Cmd+S, Ctrl+C, etc.) |
| Scroll up/down/left/right |
App management
Tool | Description |
| Open or focus an application |
| List visible running apps |
Accessibility tree
Tool | Description |
| Get accessibility tree of an app window |
| Click a named UI element (returns screenshot) |
Browser automation
Tool | Description |
| Run JavaScript in the active browser tab |
| Get all visible text from the page (faster than OCR) |
| Click element by CSS selector (instant, precise) |
| Fill a form field by CSS selector |
Utilities
Tool | Description |
| Open URL in Safari or Chrome |
| Read clipboard contents |
| Write to clipboard |
Example Workflows
Fill out a web form
You: "Go to example.com/signup and fill in my details"
Agent:
1. open_url("https://example.com/signup")
2. screenshot() → sees the form
3. screen_ocr() → finds "Email" field at (300, 250)
4. click_at(300, 250) → clicks the email field
5. type_text("user@example.com")
6. find_text_on_screen("Submit") → gets button coordinates
7. click_at(350, 500) → submits the form
8. screenshot() → confirms successNavigate an unfamiliar app
You: "Change the font size to 16 in TextEdit"
Agent:
1. launch_app("TextEdit")
2. screenshot() → sees the app
3. get_ui_elements("TextEdit") → finds menu items
4. press_key("t", ["command"]) → opens Fonts panel
5. screenshot() → sees the font panel
6. find_text_on_screen("Size") → locates the size field
7. click_at(x, y) → clicks size field
8. type_text("16")
9. press_key("return")Extract info from an email
You: "Copy the tracking number from the shipping email in Mail"
Agent:
1. launch_app("Mail")
2. screenshot() → sees the inbox
3. find_text_on_screen("Your order has shipped") → locates the email
4. click_at(x, y) → opens the email
5. screenshot() → sees the email content
6. screen_ocr() → extracts all text including tracking number
7. set_clipboard("1Z999AA10123456784") → copies to clipboardTake a screenshot and describe what's on screen
You: "What do I have open right now?"
Agent:
1. screenshot() → captures the full screen
2. Describes what it sees: apps, windows, content
3. list_running_apps() → ["Safari", "Slack", "VS Code", "Spotify"]
4. "You have Safari open on Twitter, Slack with 3 unread messages, and VS Code editing index.ts"Debug a UI issue
You: "The submit button on this page isn't working, can you check what's happening?"
Agent:
1. screenshot() → sees the page
2. find_text_on_screen("Submit") → locates button at (500, 600)
3. get_ui_elements("Safari") → inspects accessibility tree
4. "The button has aria-disabled=true. There's a required field 'Phone' that's empty."
5. find_text_on_screen("Phone") → locates the field
6. click_at(x, y) → clicks the field
7. type_text("+1 555-0123")
8. click_at(500, 600) → clicks Submit again
9. screenshot() → confirms it workedControl Spotify
You: "Pause the music and turn the volume down"
Agent:
1. launch_app("Spotify")
2. screenshot() → sees Spotify is playing
3. find_text_on_screen("Pause") → locates pause button
4. click_at(x, y) → pauses playback
5. find_text_on_screen("Volume") → locates volume slider area
6. click_at(x, y) → adjusts volume
7. screenshot() → confirms paused and volume loweredWork with Finder
You: "Create a new folder on my Desktop called 'Project Assets' and move it to Documents"
Agent:
1. launch_app("Finder")
2. press_key("d", ["command", "shift"]) → opens Desktop
3. screenshot() → sees Desktop in Finder
4. press_key("n", ["command", "shift"]) → creates new folder
5. type_text("Project Assets")
6. press_key("return")
7. screenshot() → confirms folder createdSend a message in Slack
You: "Send 'build is green, ready to deploy' in the #engineering channel on Slack"
Agent:
1. launch_app("Slack")
2. screenshot() → sees Slack
3. press_key("k", ["command"]) → opens Quick Switcher
4. type_text("engineering")
5. press_key("return") → opens #engineering
6. screenshot() → confirms channel is open
7. click_at(x, y) → clicks message input
8. type_text("build is green, ready to deploy")
9. press_key("return") → sends message
10. screenshot() → confirms sentResearch and copy data from a website
You: "Look up the current price of AAPL on Google Finance and copy it"
Agent:
1. open_url("https://google.com/finance/quote/AAPL:NASDAQ")
2. screenshot() → sees the page loading
3. screen_ocr() → reads all text on the page
4. Finds the price: "$187.42"
5. set_clipboard("$187.42")
6. "Copied AAPL price $187.42 to your clipboard"Multi-app workflow
You: "Take what's in my clipboard, search for it in Safari, and screenshot the results"
Agent:
1. get_clipboard() → "best mechanical keyboards 2025"
2. launch_app("Safari")
3. press_key("l", ["command"]) → focuses address bar
4. type_text("best mechanical keyboards 2025")
5. press_key("return") → searches
6. screenshot() → captures the search results
7. "Here are the search results for 'best mechanical keyboards 2025'"Navigate System Settings
You: "Turn on Dark Mode"
Agent:
1. launch_app("System Settings")
2. screenshot() → sees System Settings
3. find_text_on_screen("Appearance") → locates the option
4. click_at(x, y) → opens Appearance settings
5. screenshot() → sees Light/Dark/Auto options
6. find_text_on_screen("Dark") → locates Dark mode option
7. click_at(x, y) → enables Dark Mode
8. screenshot() → confirms Dark Mode is onRequirements
macOS 13+ (Ventura or later)
Node.js 18+
Python 3.9+ (pre-installed on macOS — needed for OCR and mouse control)
How It Works
Screenshots — native
screencaptureCLIOCR — Apple Vision framework (VNRecognizeTextRequest) via Python bridge, returns text with bounding box coordinates
Mouse — Quartz Core Graphics events via Python bridge for precise pixel-level control
Keyboard & Apps — AppleScript via
osascriptfor key presses, app launching, and UI element interactionPython env — auto-managed venv at
~/.macos-control-mcp/.venv/with only two packages (pyobjc-framework-Vision,pyobjc-framework-Quartz)
Troubleshooting
"Permission denied" or blank screenshots → Add your terminal to System Settings → Privacy & Security → Screen Recording
Clicks don't work → Add your terminal to System Settings → Privacy & Security → Accessibility
Python setup fails
→ Ensure python3 is in your PATH. Run python3 --version to check. Non-Python tools (keyboard, apps, clipboard) still work without it.
OCR returns empty results
→ Make sure Screen Recording permission is granted. Try a full-screen OCR first (without the app parameter).
"App not found" errors → Use the exact app name as shown in Activity Monitor (e.g., "Google Chrome" not "Chrome").
License
Available Tools
22 toolsbatch_actionsA
PREFERRED: Always use this tool instead of calling individual action tools (click_at, type_text, press_key, launch_app, etc.) one at a time. Combine multiple steps into a single batch call — this is dramatically faster. Returns a single screenshot by default (disable with return_screenshot=false). Only use individual tools when you need to read the result of one action before deciding the next. Stops on first error. Max 20 actions per call.
Example — open Notes and write text (1 call instead of 6): [{ "action": "launch_app", "name": "Notes" }, { "action": "key", "key": "n", "modifiers": ["command"] }, { "action": "set_clipboard", "text": "Hello\nWorld" }, { "action": "key", "key": "v", "modifiers": ["command"] }]
| Name | Required | Description | Default |
|---|---|---|---|
| actions | Yes | Array of actions to execute sequentially | |
| delay_between_ms | No | Delay between actions in ms (default 100) | |
| return_screenshot | No | Return one screenshot at the end (default true). Set false for faster/smaller responses. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses key behaviors: returns a single screenshot by default (configurable), stops on first error, max 20 actions, and mentions delay parameter. This is comprehensive for the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly concise: three sentences plus an example. It front-loads the key message (PREFERRED) and provides structured information about behavior and example usage. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (many action types) and the absence of output schema, the description covers all needed aspects: purpose, usage guidelines, behavioral transparency, parameter semantics via example. It is complete enough for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for each parameter. The description adds value by providing a concrete example showing how to structure the actions array, which aids understanding beyond schema definitions. However, the schema already enumerates all action types with properties, so the added semantic value is moderate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to combine multiple steps into a single batch call, which is distinct from calling individual action tools. The verb 'combine' and the resource 'multiple steps' are specific and differentiate it from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states 'PREFERRED: Always use this tool instead of calling individual action tools' and provides a clear when-not condition: 'Only use individual tools when you need to read the result of one action before deciding the next.' This gives excellent usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
click_atA
Click at x,y screen coordinates. Returns a screenshot by default (disable with return_screenshot=false). Use screenshot + screen_ocr to find coordinates first. Prefer batch_actions when combining with other actions.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | X coordinate | |
| y | Yes | Y coordinate | |
| return_screenshot | No | Return a screenshot after clicking (default true). Set false for faster/smaller responses. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries burden. Discloses default behavior (returns screenshot) and how to disable it. However, does not specify coordinate system (screen vs window) or potential side effects, but sufficient for most use cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with action. No redundant or irrelevant content. Efficiently conveys key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers return behavior (screenshot), hints at tool integration (ocr, batch), and explains disable option. Missing coordinate system detail but otherwise complete for a click tool with no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. Description adds no new parameter meaning beyond schema descriptions (which already state defaults and effects). Adequate but not enhanced.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specifies verb 'click' and resource 'x,y screen coordinates'. Distinguishes from siblings like click_by_text (text-based) and click_element (element-based). Clear and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly suggests using screenshot and screen_ocr to find coordinates first. Recommends batch_actions when combining with other actions, providing clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
click_by_textA
Click a button, link, tab, radio, or checkbox by its visible text. Much more reliable than coordinate clicking. Scrolls element into view before clicking.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Visible text to search for (case-insensitive partial match) | |
| element_type | No | Type of element to click (default: 'any') | |
| index | No | Which match to click if multiple found (0-based, default: 0) | |
| browser | No | 'safari' or 'chrome' (defaults to Safari) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses that the tool scrolls elements into view before clicking, which is a useful behavioral detail. It does not cover error handling or waiting, but for a simple click action, this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the action and immediately differentiates from alternatives. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool and the parameter descriptions, the context is largely complete. It lacks details on return values or post-click behavior, but these are often implicit for a click action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides detailed descriptions for all 4 parameters (100% coverage). The description adds context by mentioning visible text and scrolling, but does not significantly enhance parameter understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool clicks UI elements by visible text, listing specific element types (button, link, tab, radio, checkbox). It distinguishes itself from coordinate-based clicking, which is a key sibling differentiator.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly contrasts with coordinate clicking, indicating a preferred use case. However, it does not mention when not to use this tool or provide alternatives for other sibling tools like click_element.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
click_elementA
Click a named UI element in an app window. Returns a screenshot by default (disable with return_screenshot=false). Use get_ui_elements to discover element names. Prefer batch_actions when combining with other actions.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | Application process name | |
| name | Yes | Name of the UI element to click | |
| return_screenshot | No | Return a screenshot after clicking (default true). Set false for faster/smaller responses. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It notes the screenshot return default and the option to disable it, but does not mention potential side effects (e.g., triggering events) or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each serving a distinct purpose: purpose, behavior, and usage guidance. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, parameter semantics, and usage context. Lacks details on error handling or element discovery, but is sufficient for a simple click tool with no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the purpose of each parameter (e.g., 'discover element names' for name) and the default value of return_screenshot, but does not significantly extend beyond schema details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Click') and the resource ('a named UI element in an app window'). This distinguishes it from sibling tools like click_at (coordinates) and click_by_text (text-based).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It advises using get_ui_elements to discover element names and prefers batch_actions for combining actions. This provides context on when to use alternatives, though it lacks explicit statements about when not to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
double_click_atA
Double-click at x,y screen coordinates. Returns a screenshot by default (disable with return_screenshot=false). Prefer batch_actions when combining with other actions.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | X coordinate | |
| y | Yes | Y coordinate | |
| return_screenshot | No | Return a screenshot after double-clicking (default true). Set false for faster/smaller responses. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full disclosure burden. It explains that a screenshot is returned by default and can be disabled, which is the key behavioral trait. However, it omits other potential behaviors (e.g., does it wait for UI responsiveness? Is the click guaranteed? Are there side effects on focus?).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The key action is front-loaded in the first sentence, and the second sentence adds a crucial usage tip and a parameter behavior note. Every sentence is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with 3 parameters and no output schema, the description covers the main action, return behavior, and a usage guideline. It could be improved by describing the output format of the screenshot or mentioning any constraints (e.g., if the coordinates must be within screen bounds). Overall, it's largely complete for practical use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all three parameters have descriptions). The tool description adds value by clarifying that x and y are screen coordinates (not relative to an element) and explains the default behavior of return_screenshot. This context enriches the parameter understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Double-click at x,y screen coordinates', establishing a specific verb and resource. However, it does not explicitly differentiate from the sibling tool 'click_at', which is a single-click version; the distinction relies on the tool name alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Prefer batch_actions when combining with other actions.' This helps the agent choose between performing multiple actions sequentially vs. batching. No explicit when-not-to-use advice is given, but the guidance is useful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
execute_javascriptB
Run JavaScript in the active browser tab. Much faster than screenshot+OCR for web pages. Returns the result.
| Name | Required | Description | Default |
|---|---|---|---|
| code | Yes | JavaScript code to execute | |
| browser | No | 'safari' or 'chrome' (defaults to Safari) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose behavioral traits. It confirms execution in the active tab and return of results, but omits critical details: whether code runs in page context, potential side effects (e.g., altering page state), error handling, and security implications. This leaves significant gaps for safe agent invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the action and key benefit. Every phrase adds value: 'Run JavaScript', 'Much faster than screenshot+OCR', 'Returns the result'. No wasted words, ideal for quick agent parsing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description merely says 'Returns the result' without specifying type (string, JSON, etc.) or error behavior. It lacks guidance on what constitutes a valid result or how failures are reported. Given the tool's power (executing arbitrary JS), completeness is inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage: both 'code' and 'browser' are described. The description adds no extra meaning beyond the schema—it repeats the function but doesn't clarify return value format or browser parameter defaults. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs JavaScript in the active browser tab and returns the result. It distinguishes itself from screenshot+OCR by noting speed, but doesn't directly compare to sibling tools like click_at or get_page_text. Still, the purpose is specific and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises using this tool over screenshot+OCR for web page data extraction, implying a performance advantage. However, it does not provide explicit when-to-use or when-not-to-use compared to other siblings like get_page_elements or find_text_on_screen, leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fill_by_labelA
Fill a form field by its label text. Finds the input/textarea associated with the label and fills it. Works with React/Vue/Angular apps. On failure, lists available field labels for debugging.
| Name | Required | Description | Default |
|---|---|---|---|
| label | Yes | Label text of the field (case-insensitive partial match) | |
| value | Yes | Value to fill in | |
| browser | No | 'safari' or 'chrome' (defaults to Safari) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that the tool finds the associated input/textarea, fills it, supports React/Vue/Angular, and lists available labels on failure. This is informative but could detail side effects or required page state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, front-loaded with the core purpose. Every sentence adds value—purpose, method, framework support, and debugging help—with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, the description covers the core functionality and error handling. It is sufficient for a simple fill tool, though it could mention preconditions like page load or field visibility.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by noting that the 'label' parameter uses case-insensitive partial matching, which is not in the schema. This enhances understanding beyond the basic schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Fill', the resource 'form field', and the method 'by its label text'. It distinguishes from sibling tools like 'type_text' or 'click_element' by specifying the label-based approach and mentioning framework compatibility.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for form fields with labels and works with modern frameworks, but it does not explicitly state when to use this tool versus alternatives like 'type_text' or 'click_element'. No when-not guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_text_on_screenA
Find specific text on screen and get its clickable coordinates. Like Ctrl+F for the entire screen. Returns matches with centerX/centerY for use with click_at.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Text to find (case-insensitive) | |
| app | No | App to search in. Omit for full screen. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description adds case-insensitivity behavior from parameter, but does not disclose limitations (e.g., only visible text, not OCR) or performance characteristics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero waste. All information is front-loaded and essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description clarifies return format (coordinates). Missing potential edge cases (multiple matches, no match) but adequate for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3. Description reinforces parameter usage (text case-insensitive, app scope) but adds no new semantic details beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Find' and resource 'text on screen', and specifies output as clickable coordinates. Comparison to Ctrl+F helps recognition, but does not explicitly distinguish from sibling click_by_text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage via 'use with click_at' and Ctrl+F analogy, but does not explicitly state when to use versus alternatives like click_by_text, or when to use the app parameter.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_clipboardA
Read the current text contents of the macOS clipboard.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Only states 'read' without explaining behavior on empty clipboard, permissions, or blocking nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no unnecessary words. Perfectly concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with no params or output schema, but lacks details on edge cases (empty clipboard) or potential errors. Adequate but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, so schema coverage is 100%. Description does not need to add param info; baseline 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description explicitly states 'Read the current text contents of the macOS clipboard' with a clear verb and resource. It implicitly distinguishes from sibling 'set_clipboard' which writes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use or not. Usage is implied by the action, but no alternative comparisons or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_page_elementsA
Get all interactive elements (buttons, inputs, selects, radios, checkboxes, links) from the current browser page. Returns structured text showing each element's type, label, value, and state. Use this instead of screenshot+OCR to understand web page content. Then use click_by_text or fill_by_label to interact.
| Name | Required | Description | Default |
|---|---|---|---|
| browser | No | 'safari' or 'chrome' (defaults to Safari) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses return format but doesn't mention side effects, required permissions, or limitations (e.g., requires active browser tab). Adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no fluff. Front-loaded with action and result. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one optional parameter and no output schema, the description is fairly complete. It covers purpose, return value, and usage flow. Minor gap: doesn't specify if page must be loaded.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (browser) with full schema coverage. Description adds no extra meaning beyond schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves all interactive elements (buttons, inputs, etc.) from the current page and returns structured text with type, label, value, and state. It distinguishes from screenshot+OCR and suggests follow-up tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises using this tool over screenshot+OCR for understanding page content, and directs to click_by_text or fill_by_label for interaction. Could be clearer about when not to use it, but guidance is solid.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_page_textA
Get all visible text from the current browser page. Faster than OCR for web content.
| Name | Required | Description | Default |
|---|---|---|---|
| browser | No | 'safari' or 'chrome' (defaults to Safari) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It only states the core function and a speed comparison. Missing details about side effects, permissions, failure modes, or behavior with non-textual content. Minimal behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no unnecessary words. First sentence states purpose, second adds a key benefit. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one optional parameter and no output schema, the description covers the core. However, it leaves gaps such as what constitutes 'visible text' and whether the tool fails gracefully on non-text pages. Could be more comprehensive given sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter already well-described ('safari' or 'chrome', defaults to Safari). The tool description adds no extra semantic meaning, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb ('Get'), resource ('all visible text'), and context ('current browser page'). It also distinguishes from siblings like screen_ocr by noting it's faster than OCR for web content.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a clear usage context: use when you need text from web content, and prefer over OCR because it's faster. However, it lacks explicit 'when not to use' guidance or mention of limitations (e.g., only works on selectable text, not images).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_ui_elementsA
Get the accessibility tree of an app window. Returns UI element roles, names, and positions.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | Application process name |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description implies read-only operation but does not disclose performance implications, permission requirements, or behavioral details beyond what is stated. Adequate for a simple getter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and resource. No wasted words; perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple with one required parameter. No output schema, but description covers what is returned. Complete for its purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter 'app' with schema description 'Application process name'. Schema coverage is 100%, so description adds no extra meaning beyond the schema. Baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action ('Get'), resource ('accessibility tree of an app window'), and return values ('UI element roles, names, and positions'). Distinguishes from siblings like get_page_elements (web) and screen_ocr (image-based).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., get_page_elements for web, screen_ocr for images). No mention of prerequisites or limitations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
launch_appA
Open or focus a macOS application by name. Prefer batch_actions when combining with other actions.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Application name, e.g. 'Safari', 'Notes' |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must cover behavior. It indicates open or focus behavior but does not disclose handling of missing apps, permissions, or waiting.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences. First states purpose, second gives usage guidance. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description is adequate. It covers the core function and usage hint, but could mention error behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description for the name parameter. Description does not add additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states action (open/focus), resource (macOS application), and method (by name). Also distinguishes from sibling batch_actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit advice to prefer batch_actions when combining with other actions, implying single-use context. Lacks further exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_running_appsA
List all visible running macOS applications.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the burden of behavioral disclosure but only states the basic function. It does not explain what 'visible' means, what data is returned (e.g., names, PIDs), or any potential performance implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that is front-loaded with the key action and resource. Every word earns its place without unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no parameters or output schema, the description is adequate but lacks detail on the format of the returned list (e.g., app names, bundle identifiers) and the meaning of 'visible'. Additional context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters and schema coverage is 100%, so the description does not need to add parameter info. Baseline score of 4 is appropriate as no additional semantics are required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'all visible running macOS applications', making the tool's purpose immediately understandable and distinct from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'launch_app' or 'get_ui_elements'. The description lacks context for appropriate usage or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
open_urlB
Open a URL in Safari or Chrome.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to open | |
| browser | No | 'safari' or 'chrome' (defaults to Safari) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. It lacks details on side effects (e.g., whether the browser is brought to front, error handling if browser not found, or how new tabs are managed), providing minimal behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise at 8 words, with the action front-loaded. Every word contributes to clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two parameters and no output schema, the description covers the core functionality. The schema fills in parameter details, so the description is largely complete, though it could mention behavior like focusing the browser.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (both 'url' and 'browser' have descriptions). The tool description adds no additional parameter meaning beyond what the schema already provides; baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'open a URL' and specifies the target browsers 'Safari or Chrome', distinguishing it from sibling tools like 'launch_app' that launch apps rather than open URLs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Sibling tools such as 'launch_app' could potentially open a browser, but no comparison or usage context is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
press_keyA
Press a key combo (e.g. press 's' with ['command'] for Cmd+S). Supports a-z, 0-9, return, tab, space, delete, escape, arrows, f1-f12. If app is specified, focuses that app first. Prefer batch_actions when combining with other actions.
| Name | Required | Description | Default |
|---|---|---|---|
| key | Yes | Key name: a-z, 0-9, return, tab, space, delete, escape, up/down/left/right, f1-f12 | |
| modifiers | No | Modifier keys: 'command', 'shift', 'option', 'control' | |
| app | No | App to focus before pressing key (e.g. 'Google Chrome', 'Notes'). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It mentions focusing an app as a side effect but lacks details on error handling, timing, or global vs app-specific key presses. Adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with action and example. No unnecessary information; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Reasonably complete given no output schema and few parameters. Covers main function, supported keys, app focusing, and combination guidance. Could mention error handling or return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds an example and context but does not significantly enhance parameter meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool presses a key combo, provides an example (Cmd+S), and lists supported keys. It distinguishes itself from siblings like batch_actions and type_text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends using batch_actions when combining with other actions and mentions app focusing as a prerequisite, providing clear when-to-use and alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screen_ocrA
OCR the screen using Apple Vision. Returns every text element with pixel coordinates (x, y, centerX, centerY). Use centerX/centerY with click_at to click on any text.
| Name | Required | Description | Default |
|---|---|---|---|
| app | No | App to OCR. Omit for full screen. | |
| min_confidence | No | Filter out OCR matches below this confidence (0.0-1.0). | |
| max_elements | No | Max OCR elements to return (for smaller/faster responses). | |
| compact | No | Return compact OCR objects (text + click coordinates + confidence). | |
| include_bounds | No | Include bounding boxes in results (default true). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of behavioral disclosure. It mentions the tool is non-destructive (read operation) implicitly, but does not state safety, permissions, or rate limits. The description adds value by explaining the return format and usage, but lacks explicit behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: first establishes purpose and output, second provides actionable usage. No wasted words; highly concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the input schema covers all parameters with descriptions and there is no output schema, the description explains the return format and how to use it. It does not elaborate on corner cases or all parameters, but it is sufficient for a tool with five optional parameters where schema provides details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add significant new meaning beyond the schema for individual parameters. It contextualizes the output coordinates for clicking, but that is not parameter-specific. Thus, it meets the baseline but does not exceed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs OCR on the screen using Apple Vision and returns text elements with pixel coordinates. It also explains how to use the output with click_at, distinguishing it from siblings like find_text_on_screen which searches for specific text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides guidance on using the output coordinates with click_at. It implies when to use the tool (full screen or specific app) but does not explicitly state when not to use it or compare to alternatives like find_text_on_screen.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshotA
Capture a screenshot of the entire screen or a specific app window. Returns a PNG image.
| Name | Required | Description | Default |
|---|---|---|---|
| app | No | App name to capture. Omit for full screen. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description does not disclose side effects, permissions, focus impact, or behavior when app not found. Only states return type.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no redundant information. Front-loaded with purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple tool with one optional param, but lacks detail on error handling, app name format, or window selection behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear param description. Description adds value by distinguishing full screen vs app window.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Capture a screenshot of the entire screen or a specific app window' with a specific verb and resource. Distinguishes from siblings like OCR or list_running_apps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage for visual capture, but no explicit when-to-use or when-not-to-use guidance. No mention of alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scrollA
Scroll in the frontmost application. Prefer batch_actions when combining with other actions.
| Name | Required | Description | Default |
|---|---|---|---|
| direction | Yes | Scroll direction | |
| amount | No | Number of lines to scroll (default 3) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries full burden. It discloses that scrolling occurs in the frontmost application but omits potential behaviors such as whether it waits for scroll to complete, fails on non-responsive apps, or has rate limits. The mention of preferring batch_actions is behavioral but limited.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of two sentences with no redundant words. The first sentence states purpose, and the second provides critical usage guidance. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (2 simple params, no output schema), the description covers the essential purpose and usage guidance. It could be improved by mentioning potential limitations or expected behavior when the frontmost app is not scrollable, but overall it is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds value by clarifying that parameters apply to the frontmost application, which is not in the schema. This extra context helps the agent understand the scope of the direction and amount. However, it does not elaborate on the default amount or valid ranges beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Scroll' and identifies the resource 'the frontmost application'. It distinguishes itself from sibling tools like click_at or type_text, and explicitly references batch_actions for when combining actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance to prefer batch_actions when combining with other actions, helping avoid redundant tool calls. However, it does not specify when not to use scroll or mention any prerequisites or alternative scrolling methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
select_optionA
Select a dropdown option by the dropdown's label and the option text. On failure, lists available options for debugging.
| Name | Required | Description | Default |
|---|---|---|---|
| label | Yes | Label text of the select/dropdown (case-insensitive partial match) | |
| option | Yes | Option text to select (case-insensitive partial match) | |
| browser | No | 'safari' or 'chrome' (defaults to Safari) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses failure behavior (lists options for debugging) but does not mention other traits like whether the dropdown opens automatically, side effects, or return values. Additional behavioral context would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is two sentences long, each earning its place: first states purpose, second adds a key behavioral guarantee. No redundant or unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, and description does not specify return values (e.g., success indicator, selected option). It also omits prerequisites like page load or visibility of the dropdown. For a simple UI tool, it is adequate but missing some context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions already provided. The description rephrases the parameters ('by the dropdown's label and the option text') and adds a note about failure behavior, but does not significantly enhance understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the specific verb 'select' and resource 'dropdown option', and distinguishes from sibling tools like click_element by specifying the interaction method. The failure behavior listing available options adds clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies when to use (selecting a dropdown option by label and option text) but does not explicitly state when not to use or compare to alternatives like fill_by_label or click_element. The context is clear, but exclusions are missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
set_clipboardC
Write text to the macOS clipboard.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Text to copy to clipboard |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries full burden. It only says 'Write text' without specifying side effects (e.g., overwrites clipboard), security implications, or error conditions (e.g., clipboard access permissions).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It could be slightly enhanced with a second concise sentence about behavior without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with no output schema, the description combined with the schema is adequate but lacks mention of constraints like macOS permissions or clipboard overwrite behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (one 'text' parameter with description). The description adds minimal extra meaning beyond restating the action; baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Write' and the resource 'text to macOS clipboard', distinguishing it from the sibling 'get_clipboard'. However, it does not explicitly mention that it overwrites existing clipboard content.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs. alternatives. The sibling 'get_clipboard' is a natural counterpart, but there is no contextual hint about typical usage (e.g., after copying, before pasting).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
type_textA
Type text using keyboard input. If app is specified, focuses that app first to ensure keystrokes go to the right place. Without app, types into the frontmost app. Prefer batch_actions when combining with other actions.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Text to type | |
| app | No | App to focus before typing (e.g. 'Google Chrome', 'Notes'). Recommended to avoid keystrokes going to the wrong app. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses that without an app the tool types into the frontmost app, and with an app it focuses first. This explains the behavioral trait of app focusing. It does not mention potential side effects, but for a typing tool the behavior is sufficiently transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each serving a distinct purpose: definition, conditional behavior, and best practice. No wasted words. Front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required param, no output schema), the description covers all necessary aspects: what it does, how the optional parameter works, and a guideline for combination with siblings. There are no gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both params described, and the description adds value by explaining the 'app' parameter's purpose (focusing the app to avoid wrong keystrokes) and behavior without it (types into frontmost). This goes beyond the schema's minimal descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it types text via keyboard input. It distinguishes itself from siblings like click_at or press_key by specifying 'keyboard input' and app focusing behavior. The verb 'type' and resource 'text' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context on when to use the app parameter (recommended to avoid wrong app) and explicitly advises preferring batch_actions when combining with other actions. However, it does not differentiate from press_key for keyboard shortcuts or mention when not to use type_text.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
22 tool updates
v0.0.13- First observed
batch_actions - First observed
click_at - First observed
click_by_text - First observed
click_element - First observed
double_click_at - First observed
execute_javascript - First observed
fill_by_label - First observed
find_text_on_screen - First observed
get_clipboard - First observed
get_page_elements - First observed
get_page_text - First observed
get_ui_elements - First observed
launch_app - First observed
list_running_apps - First observed
open_url - First observed
press_key - First observed
screen_ocr - First observed
screenshot - First observed
scroll - First observed
select_option - First observed
set_clipboard - First observed
type_text
TDQS
Tools are mostly distinct; many clicking variants (click_at, click_by_text, click_element) and text-finding tools (screen_ocr, find_text_on_screen) have clear descriptions but could cause minor confusion. Overall, the boundaries are well-defined.
Names follow a consistent verb_noun pattern in snake_case, e.g., click_at, get_clipboard, press_key. The exception is batch_actions, which is a noun phrase, slightly breaking the pattern.
With 22 tools covering a broad range of macOS automation tasks (clicking, typing, OCR, app management, web interaction), the count is well-scoped and each tool serves a clear purpose without being excessive.
The set covers common automation needs comprehensively, including clicking, typing, clipboard, app launching, and web page element retrieval. Minor gaps like drag-and-drop or gesture actions exist, but core workflows are well-supported.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Let ChatGPT, Claude & Cursor use your Mac: email, calendar, iMessage, Teams, files. Local, free.
Turns a phone into a camera+Bluetooth remote so AI assistants can see and control any PC.
Operate Linux, macOS and Windows from your LLM. Every action runs through an auditable allowlist.
Eyes and hands on real Windows PCs — observe, click, type via Glasswarp API.
Related MCP Servers
- AlicenseNot gradedqualityFmaintenanceEnables AI assistants to automate macOS desktop tasks including mouse control, keyboard input, screenshots, window management, and UI interaction.14414MIT
- AlicenseNot gradedqualityCmaintenanceStandalone MCP server that gives AI agents full GUI control over macOS — screenshots, mouse, keyboard, apps, clipboard, and multi-display — with zero private dependencies.18MIT
- AlicenseNot gradedqualityDmaintenanceProvides native macOS desktop automation for AI agents, enabling screen capture, mouse/keyboard control, window management, and iOS/Android simulator control in both foreground and background modes without focus stealing.3MIT
- AlicenseNot gradedqualityAmaintenanceA local daemon for macOS that gives any MCP-capable AI client eyes, hands, and a face — screen capture, accessibility tree, mouse/keyboard actions, and an overlay — with a built-in security ceiling.2AGPL 3.0
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/PeterHdd/macos-control-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server