hellyee

Make music in Ableton Live by talking to Claude — it mixes and masters by actually reading the meters.

Other AI↔Ableton bridges hand the model a set of tools. hellyee closes the loop: it measures real output levels, adjusts, and measures again — and it ships a skill that teaches Claude how to produce, not just what the tools do. Compose, sound-design, automate, arrange, mix and master a full track from a conversation.

you  →  "balance the mix — kick on top, then master it"

Claude →  plays the drop · reads every track's meter · adjusts faders ·
          measures again until it converges · loads EQ → Glue → Limiter ·
          drives the limiter by measurement · reports: "peak 0.835,
          breakdown-to-drop dynamic 0.23 — the drop still hits"

What it does

76 tools in total. Full reference below.

Related MCP server: io.github.peterkolbe/ableton-for-ai

The skills

The tools are half the story. hellyee setup also installs eight skills — production knowledge Claude loads when the task calls for it:

Together they are why a one-line request ("make a techno track, master it for Spotify") comes out structured like the genre instead of generic MIDI.

How it works

Claude launches hellyee as a subprocess and talks to it over MCP. hellyee speaks OSC to AbletonOSC, a remote script running inside Live's own Python, which drives the Live Object Model.

Stock AbletonOSC exposes a lot, but not the browser, the master track, or arrangement clips. hellyee ships handlers that add all three, plus a patcher that installs them.

Install

1. Run the installer

If you have uv — no Python setup needed at all:

uvx hellyee setup

Otherwise:

pip install hellyee
hellyee setup

curl -LsSf https://astral.sh/uv/install.sh | sh     # macOS · Linux
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"   # Windows

uv downloads its own Python, so you never install or manage one.

hellyee setup downloads AbletonOSC, patches it with the browser / master / arrangement handlers, and writes your Claude config. It is idempotent — run it again any time. Use --client desktop for Claude Desktop, or --client both.

Want the audio features (hum-to-MIDI, voice commands)? They add ~380 MB, so they are opt-in:

pip install "hellyee[audio]"

2. Set up Live

This step is manual — Live has no API for enabling its own control surfaces.

1. Quit and reopen Live. Remote Scripts are only scanned at startup.

2. Open settings:

   • Live 12: Settings → Link, Tempo & MIDI

   • Live 11: Preferences → Link/Tempo/MIDI

   • Cmd + , on macOS · Ctrl + , on Windows

3. In the Control Surface table, pick AbletonOSC in the first free row.

4. Leave Input and Output as None — it communicates over the network, not MIDI ports.

5. You should see AbletonOSC: Listening for OSC on port 11000 in Live's status bar.

Once only. Live remembers it.

3. The skills

hellyee setup installs the skills (see The skills) into .claude/skills/ next to your .mcp.json; Claude Code picks them up automatically. Worth reading yourself — together they are a condensed map of how tracks actually get made, and what goes wrong.

4. Check it

With Live open:

hellyee doctor     # connection, handlers, optional features
hellyee smoke      # full end-to-end test, cleans up after itself

smoke creates a real track, writes and quantizes notes, and loads an instrument, then deletes the track. Pass --keep to leave it in place.

Using it

Say what you want. Claude reads the set's state first, then acts.

"make a 4-bar house beat at 124 BPM"
"add a MIDI track called Bass and put Wavetable on it"
"write a rolling bassline in F minor, offbeat eighths"
"quantize that to 16ths at 0.7 strength so it still breathes"
"put an Auto Filter on the bass and close it down a bit"
"this lead is harsh — round off the highs and slow the attack"
"balance the mix, kick should sit on top"
"arrange this into a full track: intro, build, drop, breakdown, drop, outro"
"sweep the filter open across the last 8 bars before the drop"

Conventions worth knowing

Time is in beats. One 4/4 bar is 4 beats; a 16th note is 0.25.

Pitches use Live's display convention: C3 = 60. Standard MIDI notation calls that C4. hellyee follows Live so the note Claude writes matches the note you see.

Device parameters are set in the unit you mean, never raw. Live's raw values live on internal scales that are not what the UI shows — Auto Filter's Frequency runs 20–135 but reads as "265 Hz". So set_device_parameter takes hz=250 (solved against the device's own readout, correct on every device), display="1/4" for named settings, or percent= for unitless controls like Amount and Drive. It reports back the displayed value so you can confirm what actually happened.

Tools

Audio input

Claude's API does not accept audio, so audio is processed locally and reaches the model as text or JSON:

notes_from_audio is monophonic only — humming, single-note lines. It will not transcribe chords or a full mix; use a polyphonic model such as basic-pitch for that.

On Apple Silicon, pip install mlx-whisper makes transcription much faster; hellyee prefers it when present. The first run downloads a model (~500 MB).

Known limitations

Claude cannot hear. It can measure output levels through Live's meters and reason about frequency ranges, but it cannot judge tone. EQ and sound-design choices come from convention and measurement — the final call is your ears.

Third-party plugins are opaque. Live does not expose VST/AU parameters to the API until you expose them by hand. Serum, Vital and friends will load and play, but Claude sees one parameter: Device On. To unlock a plugin, hit Configure on its device header, click the knobs you want controllable, then exit Configure — those parameters then appear.

Metering runs at ~10 Hz. AbletonOSC processes on a 100 ms tick, so meters measure sustained level, not transient peaks.

Quantize is client-side. Notes are read, snapped in Python, written back. That is why strength exists — but it costs a round trip rather than being instant.

Automation must start in a session clip. Live only creates envelopes on session clips, so hellyee writes automation there and carries it into the arrangement when the clip is placed. To vary automation across sections, write several clip variants and place the right one in each.

Session clips override the arrangement. If a track has ever had a session clip fired, it ignores arrangement clips until Back to Arrangement is pressed. hellyee handles this, but it is worth knowing when something plays silently.

No undo grouping. Each operation is its own step in Live's undo history.

Development

hellyee/
  osc.py            OSC client — persistent socket, request/response matching
  core.py           Live operations as plain functions (no Claude dependency)
  music.py          scales, chords, quantization, key-aware spelling
  audio.py          audio → notes, speech → text
  mcp_server.py     MCP tool layer
  cli.py            connection tests and diagnostics
abletonosc_patch/
  browser.py        adds browser access to AbletonOSC
  master.py         adds master track + arrangement to AbletonOSC
  setup_cli.py      installer: download, patch, configure Claude
abletonosc_patch/
  → shipped inside the wheel as hellyee/_patch
.claude/skills/hellyee/
  SKILL.md          how to drive the tools; loaded by Claude Code
tests/
  fake_live.py      in-memory Live + OSC double; no Ableton needed
  test_automate_arrangement.py

Working on hellyee itself:

git clone https://github.com/guvense/hellyee.git && cd hellyee
uv sync --extra audio          # or: pip install -e ".[audio]"
hellyee setup                  # re-applies the patch from your working copy

core.py holds the logic and knows nothing about Claude, so it is testable on its own and drivable from any front end. mcp_server.py is a thin layer of tool definitions over it.

That is what lets the regression suite run with no Ableton attached: tests/fake_live.py answers the same OSC addresses the real patch does, and reproduces the two behaviours that bugs hide behind — an out-of-range clip slot raising Index out of range, and insert_step accumulating rather than replacing. Stdlib unittest, no extra dependency:

python -m unittest discover -s tests -t .

⚠️ Editing anything in abletonosc_patch/? Those files run inside Live, which embeds Python 3.7. Walrus operators (:=), builtin generics (list[str]) and X | Y unions will not parse. The patcher does not check this for you — but python -c "import ast; ast.parse(open('file').read(), feature_version=(3,7))" does.

Hot reload. Editing an already-loaded handler does not need a Live restart — send /live/api/reload and it picks up the change in seconds. Adding a new module does require a restart, which is why browser and master handlers each live in one file.

Contributing

Issues and pull requests welcome. Useful directions:

• Polyphonic audio-to-MIDI (basic-pitch)

• Windows testing (developed on macOS)

• Genre templates and arrangement patterns for the skill

Credits

Built on AbletonOSC by Daniel Jones, which does the hard work of exposing Live's Object Model over OSC.

License

MIT

hellyee