Computer Use Adapter for DevSpace Ultra

A small stdio MCP adapter that makes desktop app discovery machine-readable while preserving native accessibility text and screenshot content.

It connects DevSpace Ultra to Open Computer Use. The desktop automation engine belongs to Open Computer Use; this repository contains the integration layer, parser, and tests. It is an independent community integration.

What it does

ChatGPT / MCP client
  → DevSpace capability_call
  → this adapter
  → Open Computer Use
  → accessibility tree + native PNG

• Adds structuredContent.apps to recognized list_apps results and advertises an output schema in the tool catalog.

• Preserves the original discovery text, upstream structured metadata, and error flags.

• Passes other tool results through, including native MCP image blocks.

• Does not guess app identifiers or special-case Calculator in the parser.

• Does not include a desktop automation engine or any production credentials/state.

The wrapper uses Node built-ins only. It launches the external command npx -y open-computer-use mcp; npm may download that upstream package on first use. The upstream version is not pinned by this adapter, so its output format and behavior can change. The SDK dependency in this repository is used only by the optional desktop smoke test.

Related MCP server: desktop-touch-mcp

Requirements

• Node.js 22 or later and npm/npx on PATH. Desktop acceptance was performed on Windows with Node 24.19.0; other desktop platforms have not been accepted by this project.

• An interactive desktop session supported by Open Computer Use, with any permissions required by that engine.

• DevSpace Ultra with native MCP content forwarding for screenshots to reach the outer consumer. A text-only capability wrapper is insufficient.

Our accepted DevSpace build contained native forwarding and bounded MCP sessions at local commit 4d5951fa55e380d5b5d314e12aa792687d893f05. Those core changes are separate from this adapter and are not included here; do not assume the upstream default branch or a package labeled 0.4.0 contains them. Text/structured discovery and native image support are distinct requirements.

Install into DevSpace

Clone this repository on the machine running DevSpace:

git clone https://github.com/ftll574/devspace-computer-use-adapter.git

Call DevSpace's capability_install with the absolute local path to the plugin subdirectory, not the repository root:

{
  "source": "<absolute-path>/devspace-computer-use-adapter/plugins/open-computer-use-devspace",
  "enable": true,
  "trust": true
}

Enabling and trusting a capability permits it to run local code and expose desktop controls. Review the adapter and upstream engine before enabling it. The adapter's instructions guide consumers; they are not a sandbox or enforcement boundary.

Then inspect the installed package:

{"pluginId":"open-computer-use-devspace","probeMcp":true}

The tested upstream exposed nine tools: list_apps, get_app_state, click, drag, perform_secondary_action, press_key, scroll, set_value, and type_text. The live catalog remains authoritative.

Discover before observing

Invoke capability_call:

{
  "pluginId": "open-computer-use-devspace",
  "kind": "mcp",
  "serverId": "open-computer-use",
  "toolName": "list_apps",
  "arguments": {}
}

Example normalized entry (illustrative; never reuse this identifier without discovery):

{
  "apps": [{
    "appId": "example-app",
    "name": "Example App",
    "running": true,
    "pid": 1234,
    "windowTitle": "Example Window"
  }]
}

Direct adapter clients receive structuredContent.apps. Through DevSpace's capability envelope this is structuredContent.result.structuredContent.apps.

Select an explicitly authorized app from that fresh result, then invoke get_app_state with its appId as arguments.app. Do not guess identifiers; stop if the target is absent or ambiguous. A working native-forwarding host returns text and image content blocks. There is no need to paste base64 into a prompt.

Tests

Pure parser/response tests and syntax checks need no installed dependencies or desktop access:

npm test
npm run check

They cover Unicode, optional titles, malformed lines, invalid PIDs, metadata preservation, error passthrough, schema injection, and unchanged screenshot responses. GitHub Actions runs these checks on Linux and Windows with Node 22 and 24; CI does not automate a desktop.

Optional read-only Windows Calculator smoke test:

npm ci --ignore-scripts
npm run smoke:calculator

Open Calculator first and keep it fully visible. The smoke test lists apps, selects exactly one running Calculator, reads its state, and checks for text plus PNG. It sends no clicks or keystrokes and prints only summary counts. App discovery still exposes app names and window titles to the local client.

Validation and limitations

The adapter plugin files originate from lab commit 1723cd25e69f10507ddb5a6010309e92e482d195. They were accepted through both a direct MCP client and an existing ChatGPT → fixed endpoint → DevSpace connection: discover apps → identify Calculator → receive UI tree and native PNG. No app identifier was supplied to the ChatGPT consumer during that acceptance.

• Discovery parsing targets the observed id -- name [running, pid=..., window=...] format. Unrecognized lines remain in raw text; structured coverage may be partial or empty.

• If upstream already provides structuredContent.apps, it is preserved rather than rewritten.

• Screenshots may contain an occluding window. Keyboard reliability is an upstream limitation; this adapter does not fix either issue.

• General-purpose desktop actions, all Windows applications, and macOS/Linux desktop behavior have not been comprehensively validated here.

• DevSpace's session memory fix and production upgrade/rollback tooling are separate core/operations work, not features of this adapter.

Contributing and license

Run npm test and npm run check before submitting a change. Use synthetic fixtures; do not attach tokens, desktop screenshots containing private information, SQLite state, or production logs to issues.

Adapter code is MIT licensed; see LICENSE. Open Computer Use is a separate MIT-licensed upstream project, copyright its contributors. No upstream engine source is bundled or relicensed here. Dependencies retain their own licenses.