This server enables local, private analysis of video, audio, and image files (or URLs) to extract AI-ready context like keyframes, transcripts, and on-screen text — nothing is uploaded, no API keys required.
Video Analysis — Extract keyframes in multiple modes:
sheet: Tile frames into compact contact sheets for a cheap overview
frames: Individual full-size stills for detailed inspection
scenes: Scene-change frames (great for slide decks/screencasts)
filmstrip: Dense near-native-rate strips to catch sub-second UI glitches/flickers
Audio Transcription — Convert speech to text from audio files or video soundtracks using local Whisper (tiny → large models).
OCR — Extract on-screen text from images or video frames via Tesseract, with configurable language and page-segmentation modes.
URL Support — Fetch and analyze media from YouTube, Vimeo, and 1000+ sites via yt-dlp, in addition to local files.
Glitch/Jump Detection — Track on-screen numbers across frames and report non-monotonic jump-back glitches with timestamps.
Cropping & Time Windows — Focus on specific UI regions or restrict analysis to a start/end time range.
Customization — Control frame rate, resolution, number of frames, output format (webp/jpeg/png), OCR language/PSM, download size/duration limits, and more.
Dependency Check — Use check_media_deps to verify required binaries (ffmpeg, ffprobe, yt-dlp, whisper, tesseract) are installed.
All processing runs entirely on your local machine — private, free, and open source (Apache-2.0).
LLMs read text and glance at a single image — but they can't watch a video or listen to audio. media-context-mcp closes that gap. Hand it a file or a link and it returns clean, model-ready context — keyframes, a transcript, or the text on screen — entirely on your machine. Nothing is uploaded.
Two steps — add the server, then install the local helpers it uses.
# Claude Code
claude mcp add media-context -- npx -y media-context-mcpThe launch command is always npx -y media-context-mcp. Pick your client:
Settings → Developer → Edit Config (claude_desktop_config.json):
{
"mcpServers": {
"media-context": { "command": "npx", "args": ["-y", "media-context-mcp"] }
}
}~/.cursor/mcp.json (global) or .cursor/mcp.json (per-project):
{
"mcpServers": {
"media-context": { "command": "npx", "args": ["-y", "media-context-mcp"] }
}
}.vscode/mcp.json — VS Code uses the servers key:
{
"servers": {
"media-context": { "command": "npx", "args": ["-y", "media-context-mcp"] }
}
}~/.codeium/windsurf/mcp_config.json:
{
"mcpServers": {
"media-context": { "command": "npx", "args": ["-y", "media-context-mcp"] }
}
}cline_mcp_settings.json (the extension's MCP settings):
{
"mcpServers": {
"media-context": { "command": "npx", "args": ["-y", "media-context-mcp"] }
}
}.kiro/settings/mcp.json (project) or ~/.kiro/settings/mcp.json (user):
{
"mcpServers": {
"media-context": { "command": "npx", "args": ["-y", "media-context-mcp"] }
}
}~/.gemini/settings.json:
{
"mcpServers": {
"media-context": { "command": "npx", "args": ["-y", "media-context-mcp"] }
}
}settings.json — Zed uses context_servers:
{
"context_servers": {
"media-context": { "command": { "path": "npx", "args": ["-y", "media-context-mcp"] } }
}
}~/.codex/config.toml:
[mcp_servers.media-context]
command = "npx"
args = ["-y", "media-context-mcp"]Settings → Tools → AI Assistant → Model Context Protocol → Add, then use command npx with args -y media-context-mcp.
Tip: in Claude Code you can install it as a plugin instead — run
/plugin marketplace add vishalguptax/media-context-mcp, then/plugin install media-context. To share with a team, install per-project:--scope project(writes.mcp.json) or commit a.cursor/mcp.jsonin the repo.
One command sets up everything the server uses, via your OS package manager:
npx media-context-mcp setup # core: keyframes, links, on-screen text
npx media-context-mcp setup --audio # also enable transcriptionThe server finds the helpers automatically afterward — no extra configuration. Run check_media_deps to see what's ready, and setup --uninstall to remove them. (Install by hand →)
“Summarize
demo.mp4.”
Related MCP server: klaket-mcp
Video | Keyframe overview, full-size stills, scene detection, or a dense filmstrip that catches split-second glitches |
Audio | Speech turned into text — clips, voice notes, meetings, podcasts |
Images | The picture, plus the exact text shown on screen |
Anywhere | Local files or links — YouTube, Vimeo, and 1000+ sites |
Private | Runs on your machine. No API keys, no uploads |
Efficient | A long clip becomes a couple of images, not hundreds |
analyze_media auto-detects audio and images. For video, choose how frames are sampled:
Mode | Best for |
| A cheap overview — frames tiled into one or two contact sheets |
| Detail on specific moments — individual full-size stills |
| Slide decks & static screencasts — only scene-change frames |
| Catching a sub-second UI glitch — a dense, near-native-rate strip |
Just ask in plain language — the assistant picks the right options.
You ask | What you get |
“Summarize | A quick overview from sampled keyframes |
“What error does | The exact on-screen text, read back |
“Walk me through the UI flow in | Step-by-step from scene-change frames |
“Transcribe | A local transcript |
“Summarize | Fetched and transcribed |
“Read the error in this screenshot | The picture plus its exact text |
“Find where the slider in | The exact frame of a sub-second glitch |
Tool | What it does |
| Turn a video, audio, or image — file or URL — into model-readable context. Auto-detects the type and supports cropping, time windows, language, and sampling rate. |
| Report which capabilities are ready on this machine. |
Every call runs locally and cleans up after itself.
Your assistant fills these in for you, but you can steer it (“use filmstrip mode”, “crop to the toolbar”).
Param | Default | Description |
| — | Local file path (video/audio/image) or http(s) URL |
| — | A note framing the analysis; echoed atop the summary |
| — |
|
|
|
|
|
|
|
|
| Upper bound on sampled frames |
|
| Tiles per row/column for contact-sheet modes |
|
| Per-frame width in px — lower = fewer tokens |
|
| Scene-change sensitivity ( |
| auto | Explicit sampling rate; pair high with |
| — |
|
|
| Tiles per image in |
| — | Restrict to a time window |
|
| Also produce a transcript (video) |
|
|
|
|
| Extract on-screen text |
|
| Language code(s), e.g. |
|
| Page-segmentation: |
|
| Track an on-screen number and report jump-back glitches with timestamps |
|
| Reject URL downloads longer than this |
|
| Abort a URL download past this size |
Worked recipes for each are in the usage guide.
Can an LLM watch a video? Not directly — models take images and text, not video. This server turns the video into frames and a transcript it can read.
Does anything get uploaded? No. Everything runs on your machine; no keys, no cloud.
Which clients work? Any MCP client — Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Cline, Kiro, Gemini CLI, JetBrains, Zed, Codex.
Does it handle YouTube and other links? Yes.
How much does it cost? It's free and open source.
Node.js 18+, on Windows, macOS, or Linux. The one-time npx media-context-mcp setup installs everything else.
npm install
npm run build
npm testIssues and PRs welcome — see the usage guide for the architecture.
Apache-2.0 © Vishal Gupta
Turn a local media file or URL (video, audio, or image) into compact context a model can read — fully local, no paid APIs. Video: montage frames (mode 'sheet', cheapest default), individual stills ('frames'), or scene changes ('scenes'); add transcript and/or ocr. Audio: speech transcript. Image: the picture plus optional OCR. For app/screen recordings use detail:'high' + ocr:true. To catch a transient UI glitch (a flicker/jump lasting <1s), use mode:'filmstrip' with a narrow startSec/endSec window, a high fps (10–15), and a crop around the affected control — it stacks dense frames so you can spot a frame whose value disagrees with the visual. Use the cheap default for everything else. Pass context to frame the analysis.
| Name | Required | Description | Default |
|---|---|---|---|
| source | Yes | Local media file path (video, audio, or image) OR an http(s) URL. | |
| context | No | Optional note about the video to frame the analysis, e.g. 'signup flow, focus on the validation error'. | |
| detail | No | high = readable stills for screen recordings (frames + large scale + png); low = cheap montage. Overrides only the fields you leave unset. | |
| mode | No | sheet = montage grids (cheapest, default); frames = individual stills; scenes = only scene-change montages; filmstrip = dense near-native-fps vertical strip for catching transient UI glitches (pair with a narrow startSec/endSec window, fps, and crop). | sheet |
| format | No | Image encoding. webp = smallest/fewest tokens (default); png = lossless for crisp text. | webp |
| maxFrames | No | Upper bound on sampled frames across the whole window. | |
| grid | No | Tiles per row/column for sheet/scenes modes (grid x grid). | |
| scale | No | Width in px of each frame before tiling. Lower = fewer tokens. | |
| sceneThreshold | No | Scene-change sensitivity for 'scenes' mode (higher = fewer cuts). | |
| fps | No | Explicit sampling rate (frames/sec); overrides the auto rate for sheet/frames/filmstrip. Use a high value (e.g. 10–15) with filmstrip to catch sub-second glitches. | |
| crop | No | Rectangle to crop before sampling — zoom into a UI region for sharper frames/OCR. Pixels, or fractions 0–1 of the frame (e.g. {x:0,y:0.7,width:1,height:0.3} = bottom 30%). | |
| stripRows | No | Tiles stacked per image in 'filmstrip' mode. | |
| startSec | No | Window start in seconds. | |
| endSec | No | Window end in seconds. | |
| transcript | No | Also run local Whisper to produce a speech transcript. | |
| whisperModel | No | Whisper model name (tiny, base, small, medium, large). | small |
| ocr | No | Extract on-screen text via OCR — ideal for app/screen recordings. Implies detail:high unless set. | |
| ocrLang | No | Tesseract language code(s) for OCR, e.g. 'eng' or 'eng+deu'. | eng |
| ocrPsm | No | Tesseract page-segmentation mode. 3 = auto (default), 6 = uniform block, 11 = sparse/scattered UI labels. | |
| ocrMaxFrames | No | Frames to OCR (sampled at full resolution, independent of the display images). | |
| detectJumps | No | Track the on-screen number (e.g. a slider %) across frames and report non-monotonic 'jump-back' glitches with timestamps. Pair with a crop around the value and a narrow window for best results. | |
| maxDurationSec | No | Reject URL downloads longer than this many seconds. | |
| maxFileSizeMb | No | Abort a URL download once it exceeds this size in MB. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses local processing, no paid APIs, and default behaviors for modes. However, it could be more explicit about output format (e.g., what the tool returns) and potential side effects like file generation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear summary, media-type breakdown, and use-case examples. It is detailed but not excessively long. Minor redundancy could be trimmed, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (23 params, no output schema), the description covers most aspects: purpose, mode usage, parameter guidance, and example workflows. It lacks explicit description of the output format (e.g., what is returned as 'context'), but the 'compact context' phrasing implies model-readable text/images.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While schema coverage is 100%, the description adds significant value by explaining parameter interactions (e.g., crop + filmstrip, fps + filmstrip) and providing concrete examples for transient glitch detection. It clarifies that detail overrides unset fields and that OCR implies detail:high unless set.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool converts local media files or URLs into compact context for models, covering video, audio, and image. It distinguishes video modes (sheet, frames, scenes, filmstrip) and provides specific use-case examples, effectively differentiating from sibling tool check_media_deps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance for when to use specific modes: 'For app/screen recordings use detail:'high' + ocr:true', 'To catch a transient UI glitch ... use mode:'filmstrip' with ...', 'Use the cheap default for everything else.' Also advises to 'Pass context to frame the analysis.'
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Report which external binaries (ffmpeg, ffprobe, yt-dlp, whisper, tesseract) are available. Call this first if analyze_media fails with a missing-binary error.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly states the tool is a read-only diagnostic check. It does not mention any destructive behavior, which is appropriate. A minor gap is the lack of details about the output format, but for a simple reporting tool this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The first sentence states the purpose, the second gives usage guidance. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, no output schema), the description is complete. It explains what it does and when to use it. No additional information is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and schema description coverage is 100% by default. The description adds value by listing the specific binaries checked, which provides context beyond the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Report' and clearly identifies the resource: availability of external binaries (ffmpeg, ffprobe, yt-dlp, whisper, tesseract). It also distinguishes itself from the sibling tool 'analyze_media' by providing a usage context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states 'Call this first if analyze_media fails with a missing-binary error.' This provides clear guidance on when to use this tool and implies when not to use it (when no error occurs).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v0.1.3analyze_mediacheck_media_depsThe two tools have completely distinct purposes: analyze_media handles media content analysis, while check_media_deps checks for required external binaries. There is no overlap or ambiguity.
Both tool names follow a consistent verb_noun pattern using snake_case (analyze_media, check_media_deps), making them predictable and easy to understand.
With only two tools, the set is minimal but appropriate for a focused utility server. The main tool is comprehensive, and the second tool provides essential support. A slightly larger set might allow more granular operations, but the current count is not problematic.
The analyze_media tool covers video, audio, and image analysis comprehensively with multiple modes and options. The check_media_deps tool fills the essential dependency-checking role. No obvious gaps are present for the server's stated purpose.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Turn any LLM multimodal; generate images, voices, videos, 3D models, music, and more.
Transform video, audio and images, and generate media from prompts. FFmpeg, captions, models.
Extract structured insights from videos, podcasts, articles, and PDFs with multi-model AI
Generate AI images, videos, music, SFX & speech in any AI assistant. Results appear inline in chat.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/vishalguptax/media-context-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
