mcp-video-analyzer extracts transcripts, key frames, OCR text, and metadata from video URLs (Loom, .mp4, .webm, .mov, etc.) with the following capabilities:
Full video analysis (analyze_video): Extract everything at once — timestamped transcript with speaker IDs, deduplicated key frames via scene-change detection, OCR text from frames (code, UI text, error messages), an annotated timeline merging transcript + frames + OCR, metadata (title, duration, platform), viewer comments, chapters, and AI summary. Supports brief, standard, and detailed depth levels.
Transcript extraction (get_transcript): Pull only the timestamped transcript with speaker IDs; falls back to Whisper when no native transcript is available.
Metadata retrieval (get_metadata): Fetch metadata, comments, chapters, and AI summary without downloading the video or extracting frames.
Frame extraction (get_frames): Extract key frames via scene-change detection (default) or dense sampling (1 frame/sec), with deduplication and JPEG optimization.
Single frame (get_frame_at): Capture one frame at a specific timestamp to inspect exactly what's on screen.
Burst frame extraction (get_frame_burst): Extract N frames evenly across a narrow time range — ideal for analyzing motion, animations, or fast scrolling.
Moment deep-dive (analyze_moment): Focused analysis on a specific time range combining burst frames, filtered transcript, OCR, and an annotated timeline.
Caching: Results cached in memory for 10 minutes; use forceRefresh to bypass.
Flexible output: Filter returned fields, control frame count, adjust scene-change sensitivity, and optionally return frames as base64 inline.
Enables analysis of Loom videos by extracting transcripts, key frames, metadata, and comments to provide a unified timeline of visual and audio content.
No existing video MCP combines transcripts + visual frames + metadata in one tool. This one does — across Loom, the major yt-dlp platforms (YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook), direct video URLs, and local files.
Want a full pipeline, not just a tool? social-knowledge-base is built on top of this server — it downloads whole Instagram creator accounts (reels, stories, highlights), transcribes them, and turns the result into a searchable, RAG-queryable knowledge base with AI-generated notes. Use this MCP when you want per-video analysis inside an agent; use social-knowledge-base when you want to archive and query an entire account.
Node.js 22.12+ — required to run the server via npx
yt-dlp — required for YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook URLs; optional for everything else (improves Loom download quality). Install with pip install yt-dlp
Chrome/Chromium (optional) — fallback for frame extraction if yt-dlp is unavailable
Without yt-dlp or Chrome, direct URLs and local files still get frames — the bundled
ffmpeg-staticdoes the extraction, and Loom falls back to its own CDN download. Platform URLs (YouTube etc.) degrade to a clear "install yt-dlp" warning. Transcripts, metadata, and comments never require either.
There are three ways in: the /video plugin (Claude Code — slash command + MCP server auto-configured), a plain MCP server config (any MCP client), or the portable skill + CLI (Codex, Cursor, Copilot, and any agent with a shell — no MCP required).
/video plugin (recommended)/plugin marketplace add guimatheus92/mcp-video-analyzer
/plugin install video@mcp-video-analyzerThis adds the /video slash command and auto-registers the MCP server — no claude mcp add needed:
/video https://youtu.be/jNQXAC9IVRw what happens at 0:10?
/video ~/Movies/screen-recording.mp4 when does the UI break?npx skills add guimatheus92/mcp-video-analyzerInstalls the video skill (Agent Skills format) into every agent detected on your machine. Agents without the MCP server configured fall back to the bundled CLI automatically — zero configuration.
claude mcp add video-analyzer -- npx mcp-video-analyzer@latestThen restart Claude Code or start a new conversation.
Add to your MCP settings file:
VS Code: File → Preferences → Settings → search "MCP" or edit ~/.vscode/mcp.json / %APPDATA%\Code\User\mcp.json (Windows)
Cursor: Settings → MCP Servers → Add
{
"servers": {
"mcp-video-analyzer": {
"type": "stdio",
"command": "npx",
"args": ["mcp-video-analyzer@latest"]
}
}
}Then reload the window (Ctrl+Shift+P → "Developer: Reload Window").
Add to your Claude Desktop config file:
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"video-analyzer": {
"command": "npx",
"args": ["mcp-video-analyzer@latest"]
}
}
}Then restart Claude Desktop.
The same engine is exposed as a one-shot command — this is what the video skill uses on agents without MCP, and it works standalone in any terminal:
npx -y mcp-video-analyzer@latest analyze "https://youtu.be/jNQXAC9IVRw"stdout is a single JSON document — metadata, transcript, ocrResults, timeline, warnings, frameCount, and frames as { time, filePath, mimeType } entries pointing at JPEG key frames copied to --out (default: the per-user cache dir — %LOCALAPPDATA% on Windows, ~/Library/Caches on macOS, $XDG_CACHE_HOME or ~/.cache on Linux — under mcp-video-analyzer/<url-hash>/; set MCP_CACHE_DIR to an absolute path to relocate it). Unlike the temp dir this used to live in, nothing reaps that location, so frames persist until you delete them — the directories are created 0700. Progress streams on stderr, so stdout can be piped straight into a JSON parser. Partial failures land in warnings with exit code 0; only hard failures exit 1.
Flag | Description |
|
|
| Max key frames, 1–60 (default adapts to duration) |
| Width cap for emitted frames (default |
| Output filter — comma-separated subset: |
| Bypass the cache and re-analyze |
| Tesseract languages (default |
| Whisper overrides for the transcription fallback |
| Where frame images are copied |
Run with no arguments (npx mcp-video-analyzer@latest) to start the MCP stdio server — the CLI is purely additive.
Once installed, ask your AI assistant:
Analyze this video: https://www.youtube.com/watch?v=jNQXAC9IVRw(also works with an Instagram/TikTok/Loom link, a direct .mp4 URL, or a local file path). If the server is connected, it will automatically call the analyze_video tool.
Related MCP server: Loom Transcript MCP Server
Eight tools — the AI picks the cheapest one for the job and calls it automatically. Click any tool to expand its parameters and examples.
Tool | What it does |
| Full analysis: transcript + key frames + OCR + timeline + metadata |
| Batch version, one structured result per source (resumable) |
| Transcript only (native captions or Whisper fallback) |
| Metadata + comments + chapters, no download |
| Key frames only (scene-change or dense 1 fps) |
| Deep-dive on a time range (burst frames + transcript + OCR) |
| Single frame at a timestamp |
| N frames across a narrow window (motion/animation) |
Extracts everything from a video URL in one call:
> Analyze this video: https://www.youtube.com/watch?v=abc123...Returns:
Transcript with timestamps and speakers
Key frames extracted via scene-change detection (automatically deduplicated). For static clips with no scene cuts — e.g. talking-head Reels/Stories where only an on-screen text overlay changes — it automatically falls back to uniform temporal sampling so you still get frames (and OCR) instead of an empty result.
OCR text extracted from frames (code, error messages, UI text, prices/dates/CTAs visible on screen)
Annotated timeline merging transcript + frames + OCR into a unified "what happened when" view
Metadata (title, duration, platform)
Comments from viewers
Chapters and AI summary (when available)
The AI will automatically call this tool when it sees a video URL — no need to ask.
Options:
detail — analysis depth: "brief" (metadata + truncated transcript, no frames), "standard" (default), "detailed" (dense sampling, more frames)
fields — array of specific fields to return, e.g. ["metadata", "transcript"]. Available: metadata, transcript, frames, comments, chapters, ocrResults, timeline, aiSummary
maxFrames (1-60) — cap on extracted frames. Default scales with video duration at standard detail (~12 for ≤30s up to 60 for >10min); fixed 60 at detailed, 0 at brief. An explicit value always wins
threshold (0.0-1.0, default 0.1) — scene-change sensitivity
forceRefresh — bypass cache and re-analyze
skipFrames — skip frame extraction for transcript-only analysis
model / language / initialPrompt — per-call Whisper overrides for the transcription fallback (override WHISPER_MODEL / WHISPER_LANGUAGE / WHISPER_PROMPT for this call only — pick a heavier model or a domain glossary for one hard clip without restarting the server)
> Analyze every .mp4 in this folderRuns analyze_video over a list of sources with a concurrency limit (default 2), returning one structured result per source — counts + warnings on success, or a per-item error on failure (one bad file never aborts the batch). Frame images are not inlined and full transcript/OCR/timeline are returned only when fields is set; otherwise you get counts. Pair with MCP_WRITE_SIDECARS=1 (below) so each video's result persists to disk and a re-run resumes instead of recomputing.
> Get the transcript from this videoQuick transcript extraction. Falls back to Whisper transcription when no native transcript is available. Accepts the same per-call model / language / initialPrompt overrides as analyze_video.
> What's this video about?Returns metadata, comments, chapters, and AI summary without downloading the video.
> Extract frames from this video with dense samplingTwo modes:
Scene-change detection (default) — captures visual transitions
Dense sampling (dense: true) — 1 frame/sec for full coverage
> Analyze what happens between 1:30 and 2:00 in this videoCombines burst frame extraction + filtered transcript + OCR + annotated timeline for a focused segment. Use when you need to understand exactly what happens at a specific moment.
> Show me the frame at 1:23 in this videoThe AI reads the transcript, spots a critical moment, and requests the exact frame to see what's on screen.
> Show me 10 frames between 0:15 and 0:17 of this videoFor motion, vibration, animations, or fast scrolling — burst mode captures N frames in a narrow window so the AI can see frame-by-frame changes.
Level | Frames | Transcript | OCR | Timeline | Use case |
| None | First 10 entries | No | No | Quick check — what's this video about? |
| Duration-adaptive: ~12 (≤30s) up to 60 (>10min), scene-change | Full | Yes | Yes | Default — full analysis |
| Up to 60 (1fps dense) | Full | Yes | Yes | Deep analysis — every second captured |
Results are cached in memory for 10 minutes. Subsequent calls with the same URL and options return instantly. Use forceRefresh: true to bypass the cache. skipFrames is part of the cache and sidecar key, so a transcript-only analysis and a framed one of the same URL never answer for each other.
The in-memory cache is lost on restart, which makes reprocessing a large local corpus costly. Set MCP_WRITE_SIDECARS=1 to also persist results next to each local video so the work survives restarts and can resume:
<stem>.vtt — the transcript, only when it was generated by the Whisper fallback (an existing <stem>.vtt from your own pipeline is never overwritten). A later call reuses it via the normal sidecar reader and skips Whisper entirely.
<stem>.analysis.json + <stem>.frames/ — the full result (frames + OCR + timeline), keyed by the video's mtime:size and the analysis params. On a later call with a matching stamp + params, the result is returned straight from disk (no extraction, no OCR).
This makes analyze_videos over thousands of files resumable, and lets an external GPU transcription pipeline and this MCP share results through the filesystem: the pipeline writes <stem>.vtt, and the MCP picks it up instead of running Whisper.
Source | Transcript | Metadata | Comments | Frames | Auth |
Loom | Yes | Yes | Yes | Yes (usually needs yt-dlp — see note) | None |
YouTube / Vimeo / TikTok / Instagram / X / Twitch / Dailymotion / Facebook | Native captions (uploaded > auto-generated) or Whisper fallback | Yes (title, duration, uploader, views, chapters, upload date) | No | Yes (capped at 1080p) | yt-dlp installed; cookies for Instagram / age-restricted (see below) |
Direct URL (.mp4, .mov, .mkv, .webm, …) | No | Duration only | No | Yes | None |
Direct URL + TwelveLabs | Yes (Pegasus, best-effort) | Duration floor + title | No | Yes |
|
Local file (absolute path or | Sidecar | Probed via ffmpeg (duration, dims, codec, audio presence) | No | Yes | None |
Loom frames: transcript, metadata, and comments come straight from Loom's API with no extra tooling. Frame extraction is different — Loom serves most videos as separate DASH video+audio streams, which only yt-dlp (
pip install yt-dlp) fetches and merges. Merging uses the bundledffmpeg-static, so no system ffmpeg is required. Without yt-dlp a direct-CDN fallback still covers some videos; when it can't, you get transcript + metadata + comments plus a warning explaining why frames are missing.Local files: pass an absolute path (e.g.,
/Users/you/clip.mp4) or afile://URI as theurlargument to any tool. Relative paths are rejected — the server's working directory is unpredictable from the MCP client. Note that any caller of the MCP server can ask it to read any file the server process has access to. UNC / network share paths (\\host\share\clip.mp4) are the exception: they reach the network rather than local disk, so they follow the network destination rules and needMCP_ALLOW_PRIVATE_URLS=1.Sidecar transcripts: if a
clip.vtt,clip.srt,clip.en.vtt, etc. lives next toclip.mp4, it's used as the transcript automatically — no Whisper roundtrip needed. SRT is converted to VTT in-memory.Embedded subtitles: if no sidecar is found and the container has an embedded subtitle stream (common in
.mkv/.mov/.mp4from screen recorders), it's transmuxed to VTT via ffmpeg and used as the transcript.Recognized extensions (local files and direct URLs):
.mp4.mov.mkv.webm.avi.m4v.wmv.flv.mpeg.mpg.m2ts.mts.3gp.ogv. The extension only gates routing — ffmpeg does the actual demuxing, so most common containers work..tsis excluded to avoid colliding with TypeScript source files.
Only http:// and https:// URLs are fetched, and only to public addresses. Requests to loopback (localhost, 127.0.0.1, ::1), private/LAN ranges, link-local, CGNAT, .local mDNS names, and Windows UNC paths are refused, as are non-HTTP schemes like ftp:// and data:.
This matters because the url argument is attacker-reachable in the normal case: an agent driving this server can be steered by the content it reads, so a URL it passes in is not necessarily one the user chose. Without the restriction the server is a proxy into whatever network it happens to sit on.
The check runs on the resolved address, not just the text, so a public hostname that resolves to 10.0.0.5 is refused too — and every hop of a redirect chain is re-checked, since a public URL answering 302 Location: http://127.0.0.1/ would otherwise walk straight past a first-hop-only check.
# Serving videos from a NAS or a local dev server? Opt back in:
MCP_ALLOW_PRIVATE_URLS=1Cloud instance metadata endpoints (169.254.169.254, Azure's 168.63.129.16, and friends) stay blocked even with that set — there is no legitimate video there, and they are what an SSRF is usually after.
Known limitation: the address is checked at resolution time, not at connection time, so DNS rebinding — a domain answering a public address to the check and a private one to the connection — is not covered. Run the server behind an egress proxy if that is in your threat model.
Single-video pages on major platforms route through yt-dlp (pip install yt-dlp — required for these URLs). Playlists, channels, and profile pages are rejected by design; pass individual video URLs (batch them with analyze_videos).
Transcript: native captions are preferred and free — uploaded subtitles first, auto-generated captions as fallback (rolling-window duplication is collapsed). WHISPER_LANGUAGE (e.g. pt) is also used to pick the caption language. Videos with no captions at all fall through to the normal Whisper chain.
Metadata: title, duration, uploader/channel, view count, upload date, and chapters — no download needed.
Download: capped at 1080p (frames/OCR don't need more), live streams are skipped, and DASH audio+video is merged with the bundled ffmpeg-static (no system ffmpeg required).
Cookies — Instagram and age-restricted videos usually require a logged-in session:
Env var | What it does | Example |
| Cookie file (Netscape format), wins when both are set |
|
| Extract cookies from an installed browser |
|
Browser cookie extraction requires the browser to be closed on Windows (the cookie database is locked while it runs). If that's inconvenient, export a
cookies.txtonce (e.g. with a "Get cookies.txt" browser extension) and pointYTDLP_COOKIESat it. Private/age-restricted videos without valid cookies don't crash the tool — the yt-dlpERROR:line surfaces inwarnings[].
Set the TWELVELABS_API_KEY environment variable to analyze direct video URLs with TwelveLabs Pegasus. Pegasus analyzes the video server-side (visuals and its own audio) and returns an AI-generated, timestamped transcript plus an AI summary as text — capabilities the DirectAdapter can't provide (a raw .mp4 URL has no transcript or summary on its own), and with no Whisper key required.
The transcript is best-effort LLM output, not a deterministic ASR dump: Pegasus is prompted to emit [MM:SS] line rows, and lines that don't match that shape are dropped, so wording and exact timestamps depend on the model's prompt adherence. Failures (bad key, timeout, API error) surface in the tool's warnings[] rather than silently returning an empty transcript.
The biggest win is on the text-only paths: get_transcript and get_metadata return a Pegasus transcript and summary for direct URLs — a few KB of text, no frame images, no per-frame token cost. analyze_video at detail: "standard"/"detailed" still extracts frames in addition (use detail: "brief" to stay text-only).
Long videos: the summary and full transcript share a single capped completion (
max_tokens= 16384), so for very long videos the transcript may be truncated. For multi-hour content, chunking by time window is the better approach.
It's fully opt-in and non-breaking: when TWELVELABS_API_KEY is set the TwelveLabsAdapter handles direct video URLs (it registers the public URL with TwelveLabs — no upload); when it's unset, the DirectAdapter handles them exactly as before. Loom URLs are unaffected. Get a key at playground.twelvelabs.io.
When a source has no native transcript (no sidecar .vtt/.srt, no embedded subtitles, no platform captions), the audio track is transcribed with Whisper via a graceful fallback chain (in execution order):
Silent tracks: before any Whisper run, the audio is probed with ffmpeg
volumedetect(first 2 minutes). A present-but-mute track — common in muted Reels/Stories — skips transcription entirely and emits a warning that the empty transcript is expected content, not an error, saving a pointless Whisper run.
@huggingface/transformers (JS-native, zero external deps) — opt-in only: this strategy runs first, but only when WHISPER_HF_MODEL is explicitly set. When it's unset (the default) the strategy is skipped entirely, so the CLI below wins and its WHISPER_MODEL/WHISPER_LANGUAGE settings are never silently overridden.
whisper CLI — used when a whisper executable is found (pip install -U openai-whisper). Point WHISPER_BIN at the executable if it isn't on PATH. Model via WHISPER_MODEL, language via WHISPER_LANGUAGE. The bundled ffmpeg-static is put on the CLI's PATH automatically, so no system ffmpeg is required.
OpenAI Whisper API — used when OPENAI_API_KEY is set.
No backend configured? If none of the three is available (no
whisperonPATH/WHISPER_BIN, noOPENAI_API_KEY, noWHISPER_HF_MODEL), transcription tools return an empty transcript with a warning telling you how to enable one — rather than a silent "no transcript". Installopenai-whisperor set one of the keys above. (The CLI is spawned withPYTHONUTF8=1so non-English/CJK transcripts don't crash the Python process on Windows.)
Env var | Applies to | Default | Example |
|
|
|
|
|
| auto-detect |
|
|
| — |
|
|
|
|
|
|
| — |
|
|
| — |
|
|
| — |
|
|
| off |
|
| HF transformers (opt-in) | — (strategy off) |
|
| OpenAI API | — |
|
The default
tinymodel is fast but weak for non-English audio. For Portuguese (or other non-English) sources, install the CLI and setWHISPER_MODEL=small(ormedium) +WHISPER_LANGUAGE=ptfor much better accuracy. AddWHISPER_PROMPTwith a domain glossary (brand/place names) to fix proper nouns. You can also overridemodel/language/initialPromptper call onanalyze_video/get_transcript/analyze_videos— no restart needed.GPU (faster-whisper):
whisper-ctranslate2(pip install -U whisper-ctranslate2) is a drop-in CLI with the same flags plus--device cuda/--compute_type/--beam_size. PointWHISPER_BINat it and setWHISPER_DEVICE=cuda(+ optionallyWHISPER_COMPUTE=float16). These GPU flags are env-gated — they're only passed when set, so plainopenai-whisper(which rejects--compute_type) keeps working when they're unset.Windows note: pip installs
whisper.exeinto the PythonScripts/dir, which is often not on thePATHthat GUI-launched MCP clients inherit. If transcripts come back empty, setWHISPER_BINto the full path ofwhisper.exe.
Frame extraction uses a two-strategy fallback chain — no single dependency is required:
Strategy | How it works | Speed | Requirements |
yt-dlp + ffmpeg (primary) | Downloads video, extracts frames via scene detection | Fast, precise | yt-dlp ( |
Browser (fallback) | Opens video in headless Chrome, seeks to timestamps, takes screenshots | Slower, no download needed | Chrome or Chromium installed |
The fallback is automatic — if yt-dlp is not available, the server tries browser-based extraction via puppeteer-core. If neither is available, analysis still returns transcript + metadata + comments, just no frames.
After frame extraction, the pipeline automatically applies:
Step | What it does | Why |
Frame deduplication | Removes near-identical consecutive frames using perceptual hashing (dHash + Hamming distance) | Screencasts often have long static moments — dedup removes redundant frames, saving tokens |
OCR | Extracts text visible on screen from each frame (via tesseract.js). Each frame is first preprocessed — grayscale + 2× upscale + contrast normalization + sharpen — which materially improves accuracy on stylized overlays (prices, dates, coupons, CTAs). | Captures code, error messages, terminal output, UI text that the transcript doesn't cover |
Annotated timeline | Merges transcript timestamps + frame timestamps + OCR text into a single chronological view | Gives the AI a unified "what was said, what changed visually, and what text appeared" at each moment |
The OCR step requires tesseract.js (included as a dependency). If it fails to load, analysis continues without OCR — no frames or transcript are lost. OCR preprocessing is on by default; set MCP_OCR_PREPROCESS=0 to OCR the raw frames instead.
OCR always reads the full-resolution frame, not the copy emitted to the client. The two have different jobs: the emitted frame is capped for token cost, while recognition needs every pixel it can get.
Emitted frames are capped at 800 px wide, which suits the common case — talking-head clips, Reels, bug repros — where the subject fills the frame.
It is the wrong size for a dense UI capture: a terminal, dashboard, IDE or spreadsheet recording, where the meaning lives in small text. An unscaled 1920×1080 screen recording lands at 800×450, and a 15 px UI font drops below what a vision model can resolve.
Pass maxWidth per call to keep more (or all) of the source resolution — 0 disables the cap:
get_frames(url, { maxFrames: 8, maxWidth: 0 }) // source resolution
get_frame_at(url, "2:14", { maxWidth: 1920 })
analyze_video(url, { detail: "standard", maxWidth: 1568 })Supported on analyze_video, analyze_videos, analyze_moment, get_frames, get_frame_at and get_frame_burst, and on the CLI as --max-width <px>.
Native frames cost several times more context than the default, so raise the cap deliberately — get_frames returns up to 20 frames and analyze_video at detailed up to 60.
Variable | Applies to | Default | Notes |
| Emitted frame width, in px |
|
|
| Emitted frame JPEG quality |
| Raise it when thin glyphs matter; env only, there is no per-call quality parameter. Values outside 1–100 fall back |
| Root for the tessdata cache and the CLI's default | per-user cache dir | Absolute paths only (a relative value is ignored). Use it when |
| Reaching private/loopback network addresses | off |
|
A value either variable can't use — 1e3, 1920px, a quality of 150 — is rejected with a one-time warning on stderr and the default applies. It is not silently accepted: the whole point of the setting is to escape a downscale that otherwise looks like a normal result.
Prefer the per-call parameter: the server starts once per session, so an environment variable cannot differ between an overview of a YouTube clip and a close read of a screen recording. The width a call actually uses is part of the cache and sidecar key, so analyzing the same video at 800 px and then at maxWidth: 0 re-runs the pipeline instead of returning the first result twice.
For live web debugging alongside video analysis, pair this server with the Chrome DevTools MCP:
claude mcp add chrome-devtools npx @anthropic-ai/mcp-devtools@latestWhen to use each:
Scenario | Tool |
Bug report recorded as a Loom video |
|
Live debugging a web page | Chrome DevTools MCP — inspect DOM, console, network, take screenshots |
Video shows UI issue, need to reproduce it | Use both: analyze the video first, then open the page in Chrome DevTools to reproduce |
The two MCPs complement each other: video analyzer understands recorded content, DevTools interacts with live pages.
The examples/loom-demo/ folder contains real outputs from analyzing a public Loom video (Boost In-App Demo Video, 2:55).
File | What it shows |
Title, duration, platform | |
42 timestamped entries with speaker IDs | |
Unified chronological view (transcript + frames merged) | |
Filtered transcript for | |
Complete |
Frame images (19 total in examples/loom-demo/frames/):
scene_*.jpg — scene-change detection (key visual transitions)
dense_*.jpg — 1fps dense sampling (every 10th frame saved as sample)
burst_*.jpg — burst extraction for moment analysis (0:30–0:45)
Regenerate after changes:
npx tsx examples/generate.ts— requires yt-dlp + network access.
# Install dependencies
npm install
# Run all checks (format, lint, typecheck, knip, tests)
npm run check
# Audit dependencies. `security` covers what the published package ships and
# is a blocking CI job; `security:all` adds devDependencies. Both also run on
# a weekly cron, because npm audit reads a live advisory database.
npm run security
# Build
npm run build
# Run E2E tests (requires network; add WHISPER_E2E=1 to include the
# transcription outcome test — needs a whisper CLI installed)
npm run test:e2e
# Just the video-format matrix: a real clip per container/codec
# (mp4/h264+hevc+av1, webm, mkv, mov, avi, m4v, mpeg, mpg, m2ts, mts,
# 3gp, ogv, flv, wmv) decoded end to end. ~15s on a warm cache; the
# first run fetches ~7MB of tesseract traineddata.
npm run test:formats
# Build + boot the real MCP server/CLI (seconds)
npm run test:smoke
# Everything: check → e2e → smoke → verify-package
npm run verify-all
# Open MCP Inspector for manual testing
npm run inspectsrc/
├── index.ts # Entry point (shebang + stdio)
├── server.ts # FastMCP server + tool registration
├── tools/ # MCP tool definitions (7 tools)
│ ├── analyze-video.ts # Full analysis with detail levels + caching
│ ├── analyze-moment.ts # Deep-dive on a time range
│ ├── get-transcript.ts # Transcript-only with Whisper fallback
│ ├── get-metadata.ts # Metadata + comments + chapters
│ ├── get-frames.ts # Frames-only (scene-change or dense)
│ ├── get-frame-at.ts # Single frame at timestamp
│ └── get-frame-burst.ts # N frames in a time range
├── adapters/ # Source-specific logic
│ ├── adapter.interface.ts # IVideoAdapter interface + registry
│ ├── loom.adapter.ts # Loom: authless GraphQL
│ ├── local-file.adapter.ts # Local files: absolute path or file:// URI
│ ├── twelvelabs.adapter.ts # TwelveLabs Pegasus: transcript + AI summary (opt-in)
│ └── direct.adapter.ts # Direct URL: any mp4/webm link
├── processors/ # Shared processing
│ ├── frame-extractor.ts # ffmpeg scene detection + dense + burst extraction
│ ├── browser-frame-extractor.ts # Headless Chrome fallback for frames
│ ├── audio-transcriber.ts # Whisper fallback (HF transformers → CLI → OpenAI)
│ ├── image-optimizer.ts # sharp resize/compress
│ ├── frame-dedup.ts # Perceptual dedup (dHash + Hamming distance)
│ ├── frame-ocr.ts # OCR text extraction (tesseract.js)
│ └── annotated-timeline.ts # Unified timeline (transcript + frames + OCR)
├── config/
│ └── detail-levels.ts # brief / standard / detailed config
├── utils/
│ ├── cache.ts # In-memory TTL cache with LRU eviction
│ ├── field-filter.ts # Selective field filtering for responses
│ ├── url-detector.ts # Platform detection from URL
│ ├── vtt-parser.ts # WebVTT → transcript entries
│ └── temp-files.ts # Temp directory management
└── types.ts # Shared TypeScript interfacesMIT
Deep-dive analysis of a specific time range in a video.
Combines burst frame extraction + transcript filtering + OCR + annotated timeline for a focused segment of the video.
Use this when you need to understand exactly what happens between two timestamps:
What's on screen (frames + OCR text extraction)
What's being said (transcript filtered to the range)
Unified timeline merging visual and audio content
Example: analyze_moment(url, "1:30", "2:00", 10) → 10 frames + transcript + OCR for that 30s window
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI).
| Name | Required | Description | Default |
|---|---|---|---|
| to | Yes | End timestamp (e.g., "2:00") | |
| url | Yes | Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file | |
| from | Yes | Start timestamp (e.g., "1:30") | |
| count | No | Number of frames to extract in the range (default: 10) | |
| maxWidth | No | Width cap for returned frames, in pixels; 0 keeps the source resolution. Defaults to 800 (or MCP_FRAME_MAX_WIDTH). Raise it when the video is a screen recording whose meaning lives in small text — terminals, dashboards, IDEs. Native frames cost several times more context than the default. | |
| ocrLanguage | No | Tesseract OCR language codes (default: "eng+por"). Use "+" to combine: "eng+spa", "eng+fra+deu". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, and destructiveHint values, so the safety profile is established. The description adds behavioral context by describing the composition of outputs (frames + transcript + OCR + unified timeline) and noting external dependencies like yt-dlp for certain platforms, which goes beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with a clear topic sentence, followed by a compact use-case list, a concrete example, and a concise support matrix. Each section earns its place and the structure makes it easy for an agent to scan and act on.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with six parameters and no output schema, the description covers the core outputs, typical usage, parameter behavior, and supported input sources. It does not fully specify the exact response shape, but it provides enough context for an agent to select and invoke the tool appropriately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds extra value by showing an example invocation mapping arguments to results, and by giving practical guidance on maxWidth for screen recordings with small text and context-cost tradeoffs. This meaningfully enriches the parameter understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it performs a 'deep-dive analysis' of a 'specific time range in a video' and names its combined outputs: burst frames, transcript filtering, OCR, and annotated timeline. This differentiates it from sibling tools like get_frame_at or get_transcript by emphasizing the integrated, segmented analysis. The example further concretizes the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says 'Use this when you need to understand exactly what happens between two timestamps' and lists the visual and audio coverage of the tool. It does not name alternative tools to use instead, but the clear use case and supported platform list give strong contextual guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Analyze a video URL to extract transcript, key frames, metadata, comments, OCR text, and annotated timeline.
Returns structured data about the video content:
Transcript with timestamps and speakers
Key frames extracted via scene-change detection (deduplicated, as images). For static clips with no scene cuts (e.g. talking-head Reels/Stories where only on-screen text changes) it automatically falls back to uniform temporal sampling.
OCR text extracted from frames (code, error messages, UI text, prices/dates/CTAs visible on screen)
Annotated timeline merging transcript + frames + OCR into a unified chronological view
Metadata (title, duration, platform)
Comments from viewers (if available)
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI).
Detail levels:
"brief": metadata + truncated transcript only (fast, no video download)
"standard": full analysis with scene-change frames (default)
"detailed": dense sampling (1 frame/sec), more frames, full OCR
Use options.fields to request only specific data (e.g., ["metadata", "transcript"]). Use options.forceRefresh to bypass the cache. Use options.model / options.language / options.initialPrompt to override Whisper transcription per call (e.g. a heavier model + a domain glossary for hard audio) without restarting the server.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file | |
| options | No | Analysis options |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (readOnly, idempotent), the description reveals important behaviors: caching with forceRefresh, context cost implications of native frames, and fallback to uniform temporal sampling for static videos. This adds transparency that annotations alone don't provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-organized with bullet points and clear sections. Every sentence adds value; no fluff. Slightly verbose but appropriate given the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all essential aspects: supported sources, return types, detail levels, options, and environmental defaults. No output schema is present, but the description enumerates the expected fields, making the tool's behavior fully understandable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters have descriptive schema text (100% coverage), and the description adds practical guidance for each, e.g., 'maxWidth' explains when to raise it for screen recordings and the context cost. This goes beyond simple attribute names to explain intent and trade-offs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: analyzing a video URL to extract transcript, key frames, metadata, comments, OCR text, and annotated timeline. It distinguishes itself from sibling tools like get_metadata or get_transcript by being the comprehensive analysis entry point.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides detailed usage instructions including supported video sources, detail levels, and customization options via the 'options' parameter. It does not explicitly compare to sibling tools, but as the primary analysis tool, this is rarely necessary. The description also explains when to adjust settings like maxWidth and threshold for specific use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Batch-analyze many videos in one call, with a concurrency limit and per-item results.
For each source it runs the same pipeline as analyze_video (frames + OCR + transcript + timeline), reusing the shared cache and on-disk sidecars. Designed for processing a corpus of local files: pair it with MCP_WRITE_SIDECARS=1 so results persist next to each video and a re-run resumes instead of recomputing.
Returns a JSON summary plus one structured entry per source:
ok=true → title, duration, frameCount, ocrCount, transcriptEntries, warnings
ok=false → the error message for that specific video (other videos still complete)
To keep the response bounded, frame images are NOT inlined and full transcript/OCR/timeline arrays are returned only when options.fields is set; otherwise you get counts. Use analyze_video on an individual source when you need the images or full data inline.
| Name | Required | Description | Default |
|---|---|---|---|
| options | No | Analysis options applied to every source | |
| sources | Yes | Video sources to analyze in one batch (Loom URLs, platform video URLs like YouTube, direct video URLs, or local paths). | |
| concurrency | No | How many videos to analyze in parallel (default: 2). Frame extraction + OCR are CPU-heavy — raise cautiously. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Even though annotations already declare readOnly, idempotent, and non-destructive, the description adds substantial behavioral context: per-item error isolation, cache/sidecar reuse, bounded responses with no inline frame images, and conditional field arrays based on options.fields. None of this contradicts the annotations and it meaningfully exceeds what the structured fields alone convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear lead sentence, a short pipeline/caching paragraph, a concise bulleted return-format section, and a final bounded-response note. Every sentence contributes information, and the formatting helps an agent scan it quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a batch tool with no output schema, the description is remarkably complete: it explains the per-item success/failure shape, partial failure behavior, caching and sidecar persistence, response-size controls, and how it relates to analyze_video. An agent has enough context to select and invoke the tool correctly in most scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds useful semantic nuance by explaining that full transcript/OCR/timeline arrays are only returned when options.fields is set, otherwise counts are returned, and that frame images are never inlined. This goes beyond the schema's parameter descriptions without repeating every field.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: "Batch-analyze many videos in one call," and clearly distinguishes itself from the sibling analyze_video by emphasizing batch processing, per-item results, and a bounded response. It also states it runs the same pipeline as analyze_video, making the relationship explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says it is "Designed for processing a corpus of local files," recommends pairing with MCP_WRITE_SIDECARS=1, and tells the agent to "Use analyze_video on an individual source when you need the images or full data inline." This gives clear when-to-use and when-not-to-use guidance with a named alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract a single video frame at a specific timestamp.
Useful for inspecting what's on screen at a particular moment. The AI reads the transcript, identifies a critical moment, and requests the exact frame at that timestamp.
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI).
Args:
url: Video source (URL or local path)
timestamp: Time position (e.g., "1:23", "0:05", "01:23:45")
Returns: A single image of the video frame at the specified timestamp.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file | |
| maxWidth | No | Width cap for returned frames, in pixels; 0 keeps the source resolution. Defaults to 800 (or MCP_FRAME_MAX_WIDTH). Raise it when the video is a screen recording whose meaning lives in small text — terminals, dashboards, IDEs. Native frames cost several times more context than the default. | |
| timestamp | Yes | Timestamp to extract frame at (e.g., "1:23", "0:05", "01:23:45") | |
| returnBase64 | No | Return frame as base64 inline instead of file path |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral context beyond annotations: supported video sources, dependency on yt-dlp for many platforms, and a return type of a single image. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded: purpose, use case, supported inputs, args, and return value. Every sentence earns its place; the supported-source list is long but necessary, and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema, the description adequately states what is returned. It covers supported input types, timestamp format, and use case. Minor gaps remain, such as not explicitly stating the default output is a file path unless returnBase64 is set, though that is covered in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema contains detailed descriptions for all four parameters including maxWidth and returnBase64. The description repeats only url and timestamp with brief examples, adding little beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource+scope: 'Extract a single video frame at a specific timestamp.' It clearly differentiates from siblings like get_frame_burst and get_frames by emphasizing a single frame at a precise moment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says it is 'Useful for inspecting what's on screen at a particular moment' and even outlines an AI workflow of reading a transcript and requesting a critical frame. It provides clear context but does not explicitly state when to prefer siblings like get_frame_burst or get_frames.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract multiple frames evenly distributed across a time range.
Designed for motion and vibration analysis where scene-change detection fails because the "scene" doesn't change — only the position/state of objects does.
Example: get_frame_burst(url, "0:15", "0:17", 10) → 10 frames in 2 seconds
AI sees the object in different positions across frames → understands the vibration
Works for: shaking, flickering, animations, fast scrolling, loading spinners
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI).
Args:
url: Video source (URL or local path)
from: Start timestamp (e.g., "0:15")
to: End timestamp (e.g., "0:17")
count: Number of frames (default: 5, max: 30)
Returns: N images evenly distributed between the from and to timestamps.
| Name | Required | Description | Default |
|---|---|---|---|
| to | Yes | End timestamp (e.g., "0:17") | |
| url | Yes | Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file | |
| from | Yes | Start timestamp (e.g., "0:15") | |
| count | No | Number of frames to extract (default: 5) | |
| maxWidth | No | Width cap for returned frames, in pixels; 0 keeps the source resolution. Defaults to 800 (or MCP_FRAME_MAX_WIDTH). Raise it when the video is a screen recording whose meaning lives in small text — terminals, dashboards, IDEs. Native frames cost several times more context than the default. | |
| returnBase64 | No | Return frames as base64 inline instead of file paths |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false, and the description does not contradict these. It adds useful behavioral context beyond annotations, such as platform requirements (yt-dlp for certain sources), the default count, and the note that native frames are context-expensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear lead sentence, use-case paragraph, example, supported formats list, and Args section. It is slightly long but each section earns its place; the example effectively clarifies the even distribution behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema, and the description explains the return (N images, either file paths or base64 via returnBase64). It covers the supported source types, the motion-analysis context, and the maxWidth tradeoff. Sufficient for a 6-parameter tool with rich annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description's Args section repeats url/from/to/count but omits maxWidth and returnBase64, though the schema descriptions for those are rich (e.g., context cost warning). The description adds minimal value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Extract multiple frames evenly distributed across a time range,' which is a specific verb+resource+scope. It also differentiates from siblings by positioning itself for motion and vibration analysis, distinguishing it from get_frame_at (single frame) and scene-change-based tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states the intended use case — motion and vibration analysis where scene-change detection fails — and provides concrete 'Works for' examples like shaking, flickering, and animations. It lacks explicit exclusions or named alternatives, but the context is strong enough to guide an agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract key frames from a video URL without transcript or metadata.
Two extraction modes:
Scene-change detection (default): captures visual transitions
Dense sampling (dense=true): captures 1 frame/sec for full video coverage
Returns optimized, deduplicated JPEG frames.
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI).
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file | |
| options | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish read-only and non-destructive behavior. The description adds value by stating the output format ('optimized, deduplicated JPEG frames'), the extraction modes, and the list of supported sources. It does not mention rate limits or auth, but given annotations and the nature of the tool, this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, then details modes and supported sources. Every sentence provides useful information without redundancy. It is slightly long but each segment earns its place, so it remains appropriately concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main operational aspects: purpose, modes, output, and supported sources. It does not explain every parameter (threshold, maxFrames) but those are documented in the schema. There is no output schema, yet the return type is mentioned. Given the tool's complexity and the richness of sibling comparisons, the description is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all parameters with descriptions, achieving moderate schema coverage. The tool description enhances understanding by explaining the dense mode and providing practical guidance for maxWidth ('Raise it when the video is a screen recording whose meaning lives in small text'). This goes beyond the schema's basic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Extract' and resource 'key frames from a video URL', and explicitly distinguishes itself from siblings by noting 'without transcript or metadata' and focusing on key frames rather than single frames or bursts. It also lists supported video sources, which further pinpoints its scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains two extraction modes (scene-change default vs dense) and when dense is appropriate ('full video coverage'). It does not explicitly name sibling tools like get_frame_at or get_frame_burst as alternatives, but the mode guidance and source list provide clear context for when to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get video metadata, comments, chapters, and AI summary from a video URL.
Returns structured metadata without downloading the video or extracting frames. Faster than analyze_video when you only need metadata.
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI).
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false, and the description does not contradict these. It adds valuable behavioral context: no video download or frame extraction, a yt-dlp requirement for certain platforms, and a performance comparison to analyze_video.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short, front-loaded paragraphs. The first sentence states the core purpose, the second adds a key behavioral distinction, and the third lists supported sources — every sentence earns its place with no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, read-only, idempotent tool with rich annotations, the description is complete: it covers purpose, supported inputs, performance trade-offs, and key constraints. Even without an output schema, it clearly signals the expected deliverable types (metadata, comments, chapters, AI summary).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers the single url parameter at 100%, so the baseline is 3. The description adds meaning beyond the schema by specifying file:// URIs, the yt-dlp dependency for platform URLs, and concrete URL family examples, though the schema already includes most of the same type information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Get video metadata, comments, chapters, and AI summary from a video URL,' giving a specific verb, resource, and output scope. It also distinguishes itself from sibling analyze_video by noting it is faster when only metadata is needed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says to use this tool when you 'only need metadata' and directly compares to analyze_video ('Faster than analyze_video...'). The supported URL types and local file variants are enumerated, making applicability clear, and the 'without downloading the video or extracting frames' line implies when this tool is not appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract only the transcript from a video URL.
Returns timestamped transcript entries with speaker identification (when available). Faster than analyze_video when you only need the transcript.
If the platform has no native transcript, attempts Whisper fallback transcription (requires @huggingface/transformers, whisper CLI, or OPENAI_API_KEY).
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp; native captions preferred), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI). For local files a sidecar .vtt/.srt next to the file is used first, then an embedded subtitle track, and only then the Whisper fallback if neither exists.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file | |
| options | No | Transcription overrides (apply only to the Whisper fallback) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context: it explains the fallback chain (native captions → Whisper), the dependency requirements (yt-dlp, @huggingface/transformers, whisper CLI, or OPENAI_API_KEY), and the sidecar file preference for local files. This goes beyond the annotations, though it doesn't detail error cases or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear opening sentence, followed by return format, performance comparison, fallback details, and supported platforms. It's slightly long but every sentence adds necessary information. The platform list is dense but necessary for the agent to know what URLs are supported.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (multiple platforms, fallback chain, dependencies), the description covers the key aspects: what it returns, when to use it, supported sources, and fallback behavior. It doesn't have an output schema, so the description's mention of 'timestamped transcript entries with speaker identification' is helpful. It could mention error scenarios (e.g., unsupported platform) but overall it's quite complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters well. The description adds context by explaining the fallback behavior and that options only apply to the Whisper fallback, which is not fully clear from the schema alone. It also clarifies the 'url' parameter's accepted formats (Loom share link, platform URLs, direct video URLs, local paths). This adds value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool extracts only the transcript from a video URL, with a specific verb ('Extract') and resource ('transcript from a video URL'). It distinguishes itself from siblings by explicitly noting it's faster than analyze_video when only the transcript is needed, and the supported platforms are enumerated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool vs alternatives: 'Faster than analyze_video when you only need the transcript.' It also details the fallback behavior (Whisper) and prerequisites (yt-dlp, API keys), which helps the agent decide if this tool is appropriate for the given URL type.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v0.9.0analyze_moment2 fields changedInput schema / additionalPropertiesAdded value: +falseInput schema / properties / maxWidthAdded value: +{
+ "description": "Width cap for returned frames, in pixels; 0 keeps the source resolution. Defaults to 800 (or MCP_FRAME_MAX_WIDTH). Raise it when the video is a screen recording whose meaning lives in small text — terminals, dashboards, IDEs. Native frames cost several times more context than the default.",
+ "maximum": 7680,
+ "minimum": 0,
+ "type": "integer"
+}analyze_video3 fields changedInput schema / additionalPropertiesAdded value: +falseInput schema / properties / options / additionalPropertiesAdded value: +falseInput schema / properties / options / properties / maxWidthAdded value: +{
+ "description": "Width cap for returned frames, in pixels; 0 keeps the source resolution. Defaults to 800 (or MCP_FRAME_MAX_WIDTH). Raise it when the video is a screen recording whose meaning lives in small text — terminals, dashboards, IDEs. Native frames cost several times more context than the default.",
+ "maximum": 7680,
+ "minimum": 0,
+ "type": "integer"
+}analyze_videos3 fields changedInput schema / additionalPropertiesAdded value: +falseInput schema / properties / options / additionalPropertiesAdded value: +falseInput schema / properties / options / properties / maxWidthAdded value: +{
+ "description": "Width cap for returned frames, in pixels; 0 keeps the source resolution. Defaults to 800 (or MCP_FRAME_MAX_WIDTH). Raise it when the video is a screen recording whose meaning lives in small text — terminals, dashboards, IDEs. Native frames cost several times more context than the default.",
+ "maximum": 7680,
+ "minimum": 0,
+ "type": "integer"
+}get_frame_at2 fields changedInput schema / additionalPropertiesAdded value: +falseInput schema / properties / maxWidthAdded value: +{
+ "description": "Width cap for returned frames, in pixels; 0 keeps the source resolution. Defaults to 800 (or MCP_FRAME_MAX_WIDTH). Raise it when the video is a screen recording whose meaning lives in small text — terminals, dashboards, IDEs. Native frames cost several times more context than the default.",
+ "maximum": 7680,
+ "minimum": 0,
+ "type": "integer"
+}get_frame_burst2 fields changedInput schema / additionalPropertiesAdded value: +falseInput schema / properties / maxWidthAdded value: +{
+ "description": "Width cap for returned frames, in pixels; 0 keeps the source resolution. Defaults to 800 (or MCP_FRAME_MAX_WIDTH). Raise it when the video is a screen recording whose meaning lives in small text — terminals, dashboards, IDEs. Native frames cost several times more context than the default.",
+ "maximum": 7680,
+ "minimum": 0,
+ "type": "integer"
+}get_frames3 fields changedInput schema / additionalPropertiesAdded value: +falseInput schema / properties / options / additionalPropertiesAdded value: +falseInput schema / properties / options / properties / maxWidthAdded value: +{
+ "description": "Width cap for returned frames, in pixels; 0 keeps the source resolution. Defaults to 800 (or MCP_FRAME_MAX_WIDTH). Raise it when the video is a screen recording whose meaning lives in small text — terminals, dashboards, IDEs. Native frames cost several times more context than the default.",
+ "maximum": 7680,
+ "minimum": 0,
+ "type": "integer"
+}get_metadata1 field changedInput schema / additionalPropertiesAdded value: +falseget_transcript2 fields changedInput schema / additionalPropertiesAdded value: +falseInput schema / properties / options / additionalPropertiesAdded value: +falsev0.5.1analyze_moment1 field changedInput schema / properties / url / descriptionPrevious value: -"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"New value: +"Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file"analyze_video2 fields changedInput schema / properties / options / properties / maxFrames / descriptionPrevious value: -"Maximum number of key frames to extract (default depends on detail level)"New value: +"Maximum key frames to extract. Default scales with video duration at standard detail (~12 for ≤30s up to 60 for >10min); fixed 60 at detailed, 0 at brief."Input schema / properties / url / descriptionPrevious value: -"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"New value: +"Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file"analyze_videos2 fields changedInput schema / properties / options / properties / maxFrames / descriptionPrevious value: -"Maximum number of key frames to extract (default depends on detail level)"New value: +"Maximum key frames to extract. Default scales with video duration at standard detail (~12 for ≤30s up to 60 for >10min); fixed 60 at detailed, 0 at brief."Input schema / properties / sources / descriptionPrevious value: -"Video sources to analyze in one batch (Loom URLs, direct video URLs, or local paths)."New value: +"Video sources to analyze in one batch (Loom URLs, platform video URLs like YouTube, direct video URLs, or local paths)."get_frame_at1 field changedInput schema / properties / url / descriptionPrevious value: -"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"New value: +"Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file"get_frame_burst1 field changedInput schema / properties / url / descriptionPrevious value: -"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"New value: +"Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file"get_frames1 field changedInput schema / properties / url / descriptionPrevious value: -"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"New value: +"Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file"get_metadata1 field changedInput schema / properties / url / descriptionPrevious value: -"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"New value: +"Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file"get_transcript1 field changedInput schema / properties / url / descriptionPrevious value: -"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"New value: +"Video source: Loom share link, platform video URL (YouTube, Vimeo, TikTok, Instagram, X, Twitch, Dailymotion, Facebook), direct .mp4/.webm/.mov URL, or absolute path to a local video file"v0.5.0analyze_moment2 fields changedInput schema / properties / url / descriptionPrevious value: -"Video URL (Loom share link or direct mp4/webm URL)"New value: +"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"Input schema / properties / url / formatRemoved value: -"uri"analyze_video10 fields changedInput schema / properties / options / properties / detail / defaultRemoved value: -"standard"Input schema / properties / options / properties / forceRefresh / defaultRemoved value: -falseInput schema / properties / options / properties / initialPromptAdded value: +{
+ "description": "Domain glossary fed to Whisper as --initial_prompt (overrides WHISPER_PROMPT). Fixes proper nouns (brand/place names) in the transcript.",
+ "type": "string"
+}Input schema / properties / options / properties / languageAdded value: +{
+ "description": "Forced transcription language code (overrides WHISPER_LANGUAGE), e.g. \"pt\".",
+ "type": "string"
+}Input schema / properties / options / properties / modelAdded value: +{
+ "description": "Whisper model for transcription fallback (overrides WHISPER_MODEL for this call), e.g. \"small\", \"medium\".",
+ "type": "string"
+}Input schema / properties / options / properties / returnBase64Removed value: -{
- "default": false,
- "description": "Return frames as base64 inline instead of file paths",
- "type": "boolean"
-}Input schema / properties / options / properties / skipFrames / defaultRemoved value: -falseInput schema / properties / options / properties / threshold / defaultRemoved value: -0.1Input schema / properties / url / descriptionPrevious value: -"Video URL (Loom share link or direct mp4/webm URL)"New value: +"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"Input schema / properties / url / formatRemoved value: -"uri"analyze_videosget_frame_at2 fields changedInput schema / properties / url / descriptionPrevious value: -"Video URL (Loom share link or direct mp4/webm URL)"New value: +"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"Input schema / properties / url / formatRemoved value: -"uri"get_frame_burst2 fields changedInput schema / properties / url / descriptionPrevious value: -"Video URL (Loom share link or direct mp4/webm URL)"New value: +"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"Input schema / properties / url / formatRemoved value: -"uri"get_frames2 fields changedInput schema / properties / url / descriptionPrevious value: -"Video URL (Loom share link or direct mp4/webm URL)"New value: +"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"Input schema / properties / url / formatRemoved value: -"uri"get_metadata2 fields changedInput schema / properties / url / descriptionPrevious value: -"Video URL (Loom share link or direct mp4/webm URL)"New value: +"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"Input schema / properties / url / formatRemoved value: -"uri"get_transcript3 fields changedInput schema / properties / optionsAdded value: +{
+ "description": "Transcription overrides (apply only to the Whisper fallback)",
+ "properties": {
+ "initialPrompt": {
+ "description": "Domain glossary fed to Whisper as --initial_prompt (overrides WHISPER_PROMPT). Fixes proper nouns in the transcript.",
+ "type": "string"
+ },
+ "language": {
+ "description": "Forced transcription language code (overrides WHISPER_LANGUAGE), e.g. \"pt\".",
+ "type": "string"
+ },
+ "model": {
+ "description": "Whisper model for the transcription fallback (overrides WHISPER_MODEL for this call), e.g. \"small\", \"medium\".",
+ "type": "string"
+ }
+ },
+ "type": "object"
+}Input schema / properties / url / descriptionPrevious value: -"Video URL (Loom share link or direct mp4/webm URL)"New value: +"Video source: Loom share link, direct .mp4/.webm/.mov URL, or absolute path to a local video file"Input schema / properties / url / formatRemoved value: -"uri"v0.2.4analyze_moment1 field changedInput schema / properties / ocrLanguageAdded value: +{
+ "description": "Tesseract OCR language codes (default: \"eng+por\"). Use \"+\" to combine: \"eng+spa\", \"eng+fra+deu\".",
+ "type": "string"
+}analyze_video1 field changedInput schema / properties / options / properties / ocrLanguageAdded value: +{
+ "description": "Tesseract OCR language codes (default: \"eng+por\"). Use \"+\" to combine: \"eng+spa\", \"eng+fra+deu\". See Tesseract docs for codes.",
+ "type": "string"
+}v0.2.3analyze_momentanalyze_videoget_frame_atget_frame_burstget_framesget_metadataget_transcriptEach tool has a clearly distinct purpose: analyze_moment for time ranges, analyze_video for full analysis, analyze_videos for batch, get_frame_at for single frame, get_frame_burst for evenly spaced frames, get_frames for key frames, get_metadata for metadata only, and get_transcript for transcript only. No overlap.
All tool names follow a consistent verb_noun pattern in snake_case: analyze_moment, analyze_video, analyze_videos, get_frame_at, get_frame_burst, get_frames, get_metadata, get_transcript. Naming is predictable and uniform.
8 tools is well-scoped for a video analysis server. Each tool covers a distinct aspect of video analysis (full analysis, batch, moments, frames, metadata, transcript) without being too many or too few.
The tool surface covers all core needs for video analysis: full analysis with metadata, transcript, frames, and OCR; targeted tools for metadata, transcript, and frames; batch processing; and moment analysis. No obvious gaps for the stated purpose.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Multimodal video analysis MCP — transcription, vision, and OCR for any video URL.
Any social-video URL → transcript, metadata, frames, OCR, summary, search, Q&A. MCP server + x402.
MCP server for Clipkit — gives AI agents a video toolbox via the Clipkit schema.
Any video URL to LLM-ready transcript. ASR built in, no captions needed. TikTok, X, TED and more.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/guimatheus92/mcp-video-analyzer'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
