Skip to main content
Glama
tmustier

codex-computer-use-mcp

by tmustier

Codex Computer Use MCP

Expose OpenAI's official signed macOS Computer Use tools directly to Pi and MCP clients. The calling agent chooses each method and its arguments. This package runs no nested model and generates no action plan.

OpenAI does not produce or endorse this independent project. It relies on an experimental app-server API and installed ChatGPT components that may change.

Requirements

  • macOS with an unlocked user session

  • Node.js 22 or newer

  • the official ChatGPT macOS app at /Applications/ChatGPT.app

  • the Computer Use component installed by ChatGPT under ~/.codex/computer-use/

  • Pi 0.80.7 or newer when using the Pi integration

macOS Screen Recording, Accessibility and TCC controls still apply.

Related MCP server: codex-web-bridge

Pi

Install from npm:

pi install npm:codex-computer-use-mcp

The extension registers and activates one composable tool:

computer_use({ code: string })

The code runs with sky, emit, emitImage and a persistent store object. sky exposes all ten official Computer Use methods, so known sequential actions can run without a model round-trip between each action:

const state = await sky.get_app_state({ app: "TextEdit" });
emit(state.text);

await sky.click({ app: "TextEdit", element_index: "7" });
await sky.type_text({ app: "TextEdit", text: "hello" });
const next = await sky.get_app_state({ app: "TextEdit" });
emit(next.text);

Only values passed to emit(...) or emitImage(...) are returned to Pi. list_apps returns the official text inventory produced by the app-server transport; unlike native @oai/sky, that transport does not provide the structured app array. get_app_state may return an accessibility-tree diff after the first inspection; pass disableDiff: true to request a fresh full tree. Screenshot payloads remain in the parent process and cross the code-worker boundary only as small opaque handles. Code runs in a worker so an unbounded loop can be terminated without freezing Pi; time spent inside an official Computer Use call does not count toward the code execution slice. If a later action fails, Pi still receives earlier emitted observations and the attempted method sequence.

Use /computer-use-status to inspect the installed component and transport status.

To run a source checkout:

npm ci
npm run build
pi -ne -e /absolute/path/to/codex-computer-use-mcp/integrations/pi/index.ts

MCP

Running the package binary starts a stdio MCP server with the same ten methods and computer_use_status:

npx codex-computer-use-mcp

Example Pi MCP configuration:

{
  "mcpServers": {
    "computer-use": {
      "command": "npx",
      "args": ["codex-computer-use-mcp"],
      "lifecycle": "lazy",
      "requestTimeoutMs": 180000,
      "directTools": false
    }
  }
}

Behaviour

The adapter has one mode. Pi exposes the ten official methods through the single computer_use code tool; MCP exposes them as ten typed methods. Both are available without wrapper permission prompts. It adds no app allowlist, action gate, intent classifier, selector rewrite or focus policy. App-server uses Codex Full access. The adapter forwards any elicitation that the official host still emits.

Production calls require verified OpenAI-signed app-server and Computer Use binaries with Team ID 2DC432GLL2. The adapter uses an isolated, credential-free app-server context. It rejects any model-turn activity. Calls after get_app_state reuse the signed session, preserving element identifiers and official app state.

Audit records contain bounded metadata. They exclude arguments, app content, screenshots, prompts and credentials. MCP and CLI state defaults to ~/.direct-computer-use. Pi state defaults to direct-computer-use under the Pi agent directory. Set CODEX_COMPUTER_USE_HOME to override either default.

Pi limits returned text to 50KB or 2,000 lines. When text exceeds that limit, the complete text is written to a mode-0600 file in a private directory under /tmp. Images are returned directly and are never spilled to disk.

Development

npm ci
npm run check
npm run check:pi
npm test
npm run build

License

MIT

Available Tools

11 tools
clickB

Click an element by index or pixel coordinates from screenshot

ParametersJSON Schema
NameRequiredDescriptionDefault
xNoX coordinate in screenshot pixel coordinates
yNoY coordinate in screenshot pixel coordinates
appYesApp name, full app path, or unambiguous bundle identifier
click_countNoNumber of clicks. Defaults to 1
mouse_buttonNoMouse button to click. Defaults to left.
element_indexNoElement index to click

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

All annotations are false, so the description carries full behavioral burden. It only states the action and the targeting mechanism and does not disclose side effects, preconditions, whether the click waits for app response, or what happens if an attribute is stale. It relies entirely on the literal meaning of 'click'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that introduces the verb and the two key selection modes. No filler, no separate preamble, and no length value. It is perfectly front-loaded and easy to read.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 6 parameters, no annotation hints, and no output schema, the description is too thin. It doesn't address the required app context, the freshness of the screenshot coordinates/element index, or what the tool returns (e.g., success or updated screenshot). An agent needs more context to invoke click correctly in real use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so every parameter is defined in the schema. The description's phrase 'by index or pixel coordinates' adds a high-level grouping of element_index and x/y but adds no meaning beyond the schema, and click_count, app, and mouse_button are left entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'click' and names the resource 'an element', then clarifies two targeting modes: by index or by pixel coordinates from a screenshot. This clearly separates it from sibling tools like scroll, type_text, and press_key.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to prefer click over similar actions such as perform_secondary_action or press_key, nor when coordinate-based vs index-based targeting is appropriate. The agent is left to infer all usage decisions from the name and one-line description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

computer_use_statusComputer Use StatusC
Read-onlyIdempotent

Show Computer Use status.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

C2.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already indicate readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description does not need to cover basic safety. However, it adds no behavioral context beyond those annotations, such as what status values may be returned or whether this reflects feature availability, connection state, or something else.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single very short sentence, but it is under-specified rather than genuinely concise: it repeats the tool name without explaining the actual meaning of 'status'. This is closer to a tautology than a value-adding summary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no description of what the returned status will look like or what states are possible. A status tool needs at least a minimal explanation of what is being checked in order for an agent to interpret the result correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has no parameters, so the description is not required to document parameter meaning. With zero parameters, the baseline is 4, and the description adequately matches the empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Show Computer Use status' adds a verb but essentially restates the tool's title and name. It does not explain what 'status' means or how this tool differs from sibling tools such as get_app_state or list_apps.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to call this tool, what it is useful for, or when another tool would be appropriate. The agent gets no help discriminating it from get_app_state or other sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

dragB

Drag from one point to another using pixel coordinates

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp name, full app path, or unambiguous bundle identifier
to_xYesEnd X coordinate
to_yYesEnd Y coordinate
from_xYesStart X coordinate
from_yYesStart Y coordinate

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations (readOnlyHint=false, idempotentHint=false, destructiveHint=false) are uninformative, so the description must carry the behavioral burden. The description discloses the coordinate-based action but does not describe the press-hold-move-release behavior, speed/duration of the gesture, or how drag concludes. It's not misleading, so it stays at the minimal-viable level.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one short declarative sentence with no filler, repetition, or boilerplate, and the key operation ('drag') is front-loaded. It is as concise as the simple action permits, though it is short enough that it could have included a clause about gesture behavior without becoming bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter mutation gesture with no output schema, the description is serviceable but thin: it omits whether the drag is a press-drag-release, whether coordinates are window-relative or absolute, and what side effects a drag may trigger (e.g., scrolling or text selection). A drag in a GUI automation context has enough behavioral nuance that a bit more context would be valuable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter (app, from_x, from_y, to_x, to_y) already has a meaningful schema description, warranting the baseline 3. The description does add that the coordinates are pixel-based, which clarifies the number semantics slightly, but it doesn't go beyond the schema in any other way.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('drag'), a resource ('from one point to another'), and the method ('using pixel coordinates'), so an agent understands the action without opening the schema. It is not a tautology and it clearly communicates a mouse-drag gesture. It loses a point because it doesn't differentiate itself from siblings like click or select_text, which can also involve point-to-point movement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus click, scroll, select_text, or perform_secondary_action, and it states no prerequisites or exclusions. Usage must be inferred entirely from the tool name and the bare verb 'drag'. This is essentially no when-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_app_stateA
Read-onlyIdempotent

Start an app use session if needed, then get the state of the app's key window and return a screenshot and accessibility tree. This must be called once per assistant turn before interacting with the app

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp name, full app path, or unambiguous bundle identifier

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnlyHint, idempotentHint, and destructiveHint. The description adds valuable behavioral context beyond those annotations by revealing that it may start an app use session if needed and that it is a required precursor to all other app interactions. There is no contradiction: 'if needed' is consistent with idempotentHint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with no filler. The primary behavior is front-loaded, and the critical call-cadence rule is delivered as a clear imperative in the final sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With one documented parameter, strong annotations, explicit return values, and a clear usage cadence, the description gives an agent everything needed to select and invoke the tool correctly. Even without an output schema, it names the two expected outputs: a screenshot and an accessibility tree.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the app parameter documented as 'App name, full app path, or unambiguous bundle identifier.' The description does not need to explain parameter semantics further, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'get the state of the app's key window and return a screenshot and accessibility tree.' It distinguishes this tool from sibling action tools like type_text and click, and from computer_use_status, by clearly framing it as a state-reading and observation tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit sequencing and frequency guidance: 'This must be called once per assistant turn before interacting with the app.' This tells the agent exactly when to use the tool relative to the other app-interaction tools, which is strong usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_appsA
Read-onlyIdempotent

List the apps on this computer. Returns the set of apps that are currently running, as well as any that have been used in the last 14 days, including details on usage frequency

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral context: the result includes both currently running apps and recently used ones (14 days), and includes usage frequency details. This framing helps the agent understand that the listing is not exhaustive (not all installed apps).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that front-loads the primary action and then states the return scope in an efficient follow-up. Every clause adds relevant information—there is no fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter read-only list tool, the description covers the essential inputs, scope, and returned information ('running apps', 'last 14 days', 'usage frequency'). No output schema exists, but the description says what kind of details are returned. It is slightly shy of 5 because it does not mention whether the list includes system apps or is sorted, but those details are superficial for invoking the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 declared parameters and schema coverage is 100%, so the schema indicates no expected arguments. With zero parameters, the description does not need to add parameter semantics, and the baseline of 4 applies. The description does not confuse the agent by implying any arguments are needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') with a clear resource ('the apps on this computer') and then specifies exactly what subset is returned: running apps plus those used in the last 14 days, with usage frequency. This differentiates it from siblings like get_app_state, click, or type_text, none of which are about app discovery.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies a discovery use case—get the set of apps available on the machine—but it does not explicitly state when to use this vs alternatives or when not to use it. There is no mention of prerequisites or exclusions, which leaves the agent to infer based on the tool name and siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

perform_secondary_actionC

Invoke a secondary accessibility action exposed by an element

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp name, full app path, or unambiguous bundle identifier
actionYesSecondary accessibility action name
element_indexYesElement identifier

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide all false hints, so the description itself must communicate side effects and behavior. It only restates that an action is invoked and does not explain whether the action modifies state, can fail, is reversible, or has side effects. For a tool that calls out to application accessibility behavior, this is a meaningful gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is an efficient single sentence with no filler. It front-loads the verb and object, which is appropriate for this tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is an action-invoking tool with no output schema, no examples, no explanation of what kinds of actions can be supplied, and no guidance on where the element identifier comes from. An agent could invoke the correct function name, but it would be under-specified when selecting valid values and predicting outcomes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers all three parameters with descriptions, so the bar for the description is lower. However, the schema descriptions are terse, and the tool description adds no further meaning to 'action' or 'element_index.' Baseline 3 is appropriate because the schema is complete but shallow.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Invoke a secondary accessibility action exposed by an element.' This is reasonably clear, though it relies on the jargon 'secondary accessibility action' and does not explicitly contrast with sibling tools such as click, set_value, or type_text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance about when to use this tool versus alternatives. It does not say that standard interactions should go through click or that this is for custom/extra actions, leaving the agent to infer the appropriate scenario from the name and sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

press_keyB

Press a key or key-combination on the keyboard, including modifier and navigation keys. This supports xdotool's key syntax, for example a, Return, super+c, Up, or KP_0.

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp name, full app path, or unambiguous bundle identifier
keyYesKey or key combination to press

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal that the action is not read-only, idempotent, or destructive, but they do not carry the burden since they default to false. The description adds the xdotool key syntax detail, which reveals compatibility, but does not mention focus behavior, target delivery, or any side effects of pressing a key.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is only two sentences, front-loaded with the action and then providing compact, relevant examples. Every word earns its place with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 2-parameter tool without output schema, the description is mostly adequate, yet it fails to explain the role of the mandatory 'app' parameter. An agent might not understand that the key press is scoped to a particular application or requires that app to be active/focused, leaving a meaningful invocation gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the parameters are already documented. The description adds value above the schema by clarifying the accepted key syntax (xdotool examples) and explicitly naming modifier/navigation keys, making the 'key' parameter more meaningful.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: press a key or key-combination on the keyboard. It gives concrete examples ('a', 'Return', 'super+c') that distinguish it from text-typing tools, though it does not explicitly name or contrast sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus alternatives like type_text or click. The description implies keyboard-only input but does not state exclusions or provide conditions for choosing this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scrollB

Scroll an element in a direction by a number of pages

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp name, full app path, or unambiguous bundle identifier
pagesNoNumber of pages to scroll. Fractional values are supported. Defaults to 1
directionYesScroll direction: up, down, left, or right
element_indexYesElement identifier

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description communicates the visible behavior: scrolling an element by pages in a chosen direction. Annotations supply the safety baseline, and no annotation contradiction exists, but the description adds little detail about side effects, limits of scrolling, or how the operation behaves beyond that minimal action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler. Every word contributes to the core meaning, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a fairly simple tool, the parameters are well covered by the schema and the behavior is clear enough. However, the description does not explain how to identify an element or how the scroll interacts with the app state, which would be more complete given this is an action-changing tool with no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents app, element_index, direction, and pages, including fractional support and the default value for pages. The description adds no additional semantic detail beyond the overall scroll behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Scroll'), a target ('an element'), and the key modifiers ('in a direction by a number of pages'). It does not explicitly contrast itself with sibling tools, but the action reads clearly and is unlikely to be confused with type_text, click, or drag.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool instead of a sibling, what prerequisites exist, or what not to use it for. The only usage signal is implicit: 'if you want to scroll, use scroll'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

select_textA

Select text inside a text element, or place the text cursor before or after it. Provide text exactly as it appears in the accessibility tree, including any Markdown formatting. If the text is not unique, provide surrounding prefix or suffix text to disambiguate it.

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp name or bundle identifier
textYesTarget text as shown in the accessibility tree
prefixNoOptional text immediately before the target, used to disambiguate repeated matches
suffixNoOptional text immediately after the target, used to disambiguate repeated matches
selectionNoWhether to select the text or place the cursor before or after it. Defaults to text.
element_indexYesText element identifier

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide no readOnly, idempotent, or destructive clues, so the description carries the burden of explaining behavior. It does clarify that matching occurs against the accessibility tree and that markdown formatting matters, which is meaningful behavioral context. However, it does not mention whether the operation mutates UI state, triggers focus, or what happens on failure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two seconds long, front-loads the core action, and includes only high-value tips about matching and disambiguation. Every sentence provides useful information without repetition or extraneous details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter tool with no output schema, the description offers the core workflow but remains silent about return behavior, prerequisites other than app/element identification, and possible side effects. It is strong enough to attempt the call, but an agent has no explanation for outcome reporting or edge cases when the selected element is not found.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers 100% of the parameters, so the baseline is 3. The description goes beyond it by explaining how to supply the text parameter exactly as it appears, mentioning markdown formatting, and describing how prefix/suffix disambiguate repeated matches. This adds real interpretive value for the most error-prone parameters, though app and element_index are left to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly names the action ('Select text inside a text element') and the alternative placement ('place the text cursor before or after it'), which precisely distinguishes the tool from siblings like type_text, click, and set_value. No ambiguity remains about what resource the operation applies to.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the main use case—selecting or placing a cursor in text—and gives detailed instructions about exact text matching and prefix/suffix disambiguation. However, it never names when to use this tool instead of a sibling like click or type_text, nor does it state conditions that should steer the agent away from this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_valueB

Set the value of a settable accessibility element

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp name, full app path, or unambiguous bundle identifier
valueYesValue to assign
element_indexYesElement identifier

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false, idempotentHint=false, and destructiveHint=false, but the description adds little behavioral detail beyond 'settable'. It communicates that this is a mutation of an existing element, but it does not describe side effects, overwrite behavior, or what happens if the element is not actually settable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one short sentence that immediately conveys the action and target. There is no filler, repetition, or unnecessary context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description plus schema is enough for basic invocation, but it lacks guidance about when an element qualifies as settable and how set_value relates to text entry tool. Since there is no output schema, some context about expected behavior or return value would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes all three parameters with 100% coverage. The tool description itself does not enrich the parameter semantics, so a baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and object: 'Set the value of a settable accessibility element.' It is clear enough to know what the tool operates on, though it does not explicitly differentiate itself from siblings such as type_text or click.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use or alternative guidance is provided. There is no mention of when to prefer set_value over type_text or click, and no prerequisites like obtaining an element_index from get_app_state are stated. Only the word 'settable' offers a weak usage hint.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

type_textA

Type literal text using keyboard input

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp name, full app path, or unambiguous bundle identifier
textYesLiteral text to type

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description reveals that input is literal and delivered via keyboard simulation, which is useful since annotations do not carry behavioral details. Yet it does not disclose effects like whether the field must be focused, whether content is replaced or appended, or what happens if focus is lost. This leaves gaps for a mutating tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short sentence conveys the core purpose with no filler or redundancy. It is front-loaded with the primary verb and object.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter tool with fully documented schema properties, the description provides adequate context: the user supplies an app and text to type. The lack of an output schema is acceptable, though there is room for additional behavioral notes such as focus requirements.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters already have descriptive definitions. The description adds no new semantic detail beyond echoing 'literal text' and 'keyboard input', matching the baseline for well-documented schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear action ('type') and object ('literal text') and specifies the mechanism ('keyboard input'). It is distinct enough from siblings like press_key, but it does not explicitly explain how it differs from set_value or select_text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'literal text' implies when to use this tool: when an exact string must be typed, not when pressing shortcuts or performing actions. However, there is no explicit guidance about when to use alternatives like set_value or press_key, so usage context is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 10 tool updatesv0.5.0
    • Changedclick4 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
      • addedInput schema / properties / click_count / maximum
        Added value: +9007199254740991
      • addedInput schema / properties / click_count / minimum
        Added value: +-9007199254740991
    • Changeddrag2 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
    • Changedget_app_state2 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
    • Changedlist_apps2 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
    • Changedperform_secondary_action2 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
    • Changedpress_key2 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
    • Changedscroll2 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
    • Changedselect_text2 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
    • Changedset_value2 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
    • Changedtype_text2 fields changed
      • addedInput schema / $defs
        Added value: +{
        +  "__schema0": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "type": "boolean"
        +      },
        +      {
        +        "type": "null"
        +      },
        +      {
        +        "items": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "additionalProperties": {
        +          "$ref": "#/$defs/__schema0"
        +        },
        +        "propertyNames": {
        +          "type": "string"
        +        },
        +        "type": "object"
        +      }
        +    ]
        +  }
        +}
      • changedInput schema / additionalProperties
        Previous value: -falseNew value: +{
        +  "$ref": "#/$defs/__schema0"
        +}
  2. 13 tool updatesv0.3.0
    • Removedbackground_computer_use
    • Removedbackground_computer_use_status
    • Addedclick
    • Addedcomputer_use_status
    • Addeddrag
    • Addedget_app_state
    • Addedlist_apps
    • Addedperform_secondary_action
    • Addedpress_key
    • Addedscroll
    • Addedselect_text
    • Addedset_value
    • Addedtype_text
  3. 2 tool updatesv0.1.0
    • First observedbackground_computer_use
    • First observedbackground_computer_use_status

TDQS

B3.4/5.0
Disambiguation4/5

Each tool maps to a distinct UI interaction such as clicking, typing, scrolling, dragging, or value setting, so an agent can generally pick the right one. The main overlap risk is between `type_text` and `press_key`, since both are keyboard interactions, and `perform_secondary_action` versus `click` could be confused when a secondary action is not clearly described.

Naming Consistency4/5

Most tools follow a lowercase snake_case verb-first pattern such as `list_apps`, `get_app_state`, `type_text`, and `select_text`. The consistency is marred slightly by `computer_use_status`, which reads as a noun phrase rather than an imperative verb like `get_computer_use_status`, and by a few bare verbs like `click` and `drag`.

Tool Count5/5

With 11 tools, the set is well-scoped for the stated purpose of computer use through screenshots, accessibility trees, keyboard input, and pointer actions. Every tool covers a needed capability without bloating the surface.

Completeness4/5

The tools collectively cover common end-to-end GUI workflows: observe state, click, type, press shortcuts, scroll, drag, set values, and select text. There are minor gaps such as no dedicated hover, app-launch, or app-close tool, but those can often be worked around with the existing pointer and keyboard actions.

Maintenance

ActivityMaintained
ResponsivenessResponsive

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/tmustier/codex-computer-use-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server