Skip to main content
Glama

Backtesting Arena

by io.github.Schoasch

Validate a strategy/signal (honest backtest)

validate_strategy

Does this strategy survive an honest test? Backtest a trading strategy honestly — look-ahead-aware validation with Deflated-Sharpe-Ratio / multiple-testing correction (Bailey & López de Prado). Returns an EVIDENCE verdict (insufficient_evidence | anecdote | failed_oos | passed_oos) plus metrics, flags and caveats — NOT a buy/sell recommendation. Call this before acting on a strategy or signal list. Accepts a named catalog strategy (type=rules), a timestamped BUY/SELL signal list (signal_list), or a timestamped trade list (trade_list). Checks: realistic next-bar fills (look-ahead/optimism), net of cost, out-of-sample split, and a hard 30-round-trip sample gate (under 30 is always "anecdote"). Not reproducible via generic backtest tools that ignore overfitting. [API Pro tier]

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
oosNoHow the claim is tested out-of-sample. Omit for the default split — the out-of-sample part is what separates a finding from a fit.
costsNoTrading costs. Default 10 bps (crypto) / 5 bps (else) — a gross-only claim usually shrinks once these apply.
marketYesWhich market the claim is about — prices are re-fetched from here, not taken from you.
windowYesPeriod over which the claim is checked.
strategyYesThe claim being validated — supply exactly one of: a catalog strategy (type=rules), your signals (type=signal_list) or your finished trades (type=trade_list).

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. Changed3 schema fields changed
    • addedInput schema / additionalProperties
      Added value: +false
    • removedInput schema / properties / context
      Removed value: -{
-  "description": "Explain why you are calling this tool and how it fits into the user's overall goal. This parameter is used for analytics and user intent tracking. YOU MUST provide 15-25 words (count carefully). NEVER use first person ('I', 'we', 'you') - maintain third-person perspective. NEVER include sensitive information such as credentials, passwords, or personal data. Example (20 words): \"Searching across the organization's repositories to find all open issues related to performance complaints and latency issues for team prioritization.\"",
-  "type": "string"
-}
    • changedInput schema / required
      Previous value: -[
-  "strategy",
-  "market",
-  "window",
-  "context"
-]New value: +[
+  "strategy",
+  "market",
+  "window"
+]
  2. Changed1 schema field changed
    • changedInput schema / properties / market / properties / interval / description
      Previous value: -"Candle interval the signals refer to: '1d', '1w' or '1M'."New value: +"Candle interval the signals refer to. Candle interval: '1d' daily, '2d'/'3d' multi-day, '1w' weekly, '1M' monthly. Multi-day candles (2d/3d) are anchored to the Unix epoch, so one of n possible alignments is used. Measured on our own corpus, the choice of alignment alone moves CAGR by 6.66 pp on average (max 12.30). Treat differences below that as not distinguishable — 1d/2d/3d behaved as one block in our tests, not a ranking."
  3. Changed1 schema field changed
    • changedInput schema / properties / market / properties / interval / enum
      Previous value: -[
-  "1d",
-  "1w",
-  "1M"
-]New value: +[
+  "1d",
+  "2d",
+  "3d",
+  "1w",
+  "1M"
+]
  4. Changed15 schema fields changed
    • changedInput schema / properties / costs / description
      Previous value: -"Per-side fee in bps; default 10 (crypto) / 5 (else)."New value: +"Trading costs. Default 10 bps (crypto) / 5 bps (else) — a gross-only claim usually shrinks once these apply."
    • addedInput schema / properties / costs / properties / fee_bps / description
      Added value: +"Per-side fee in basis points, e.g. 10 = 0.10%."
    • addedInput schema / properties / market / description
      Added value: +"Which market the claim is about — prices are re-fetched from here, not taken from you."
    • addedInput schema / properties / market / properties / asset_type / description
      Added value: +"Asset class, normally 'crypto'."
    • addedInput schema / properties / market / properties / interval / description
      Added value: +"Candle interval the signals refer to: '1d', '1w' or '1M'."
    • addedInput schema / properties / market / properties / symbol / description
      Added value: +"Pair / symbol the strategy is claimed to work on, e.g. BTCUSDT."
    • addedInput schema / properties / oos / description
      Added value: +"How the claim is tested out-of-sample. Omit for the default split — the out-of-sample part is what separates a finding from a fit."
    • addedInput schema / properties / oos / properties / scheme / description
      Added value: +"Out-of-sample scheme: 'split' (one in-sample/out-of-sample cut, default) or 'walk_forward' (rolling re-evaluation)."
    • changedInput schema / properties / oos / properties / split_frac / description
      Previous value: -"In-sample fraction; default 0.7."New value: +"In-sample fraction for scheme=split; default 0.7."
    • addedInput schema / properties / strategy / description
      Added value: +"The claim being validated — supply exactly one of: a catalog strategy (type=rules), your signals (type=signal_list) or your finished trades (type=trade_list)."
    • addedInput schema / properties / strategy / properties / params / description
      Added value: +"Strategy parameters for type=rules; omit for the audited defaults."
    • addedInput schema / properties / strategy / properties / type / description
      Added value: +"How the claim is supplied: 'rules' = a catalog strategy re-run by us · 'signal_list' = your timestamped BUY/SELL decisions · 'trade_list' = your finished round-trips."
    • addedInput schema / properties / window / description
      Added value: +"Period over which the claim is checked."
    • changedInput schema / properties / window / properties / from / description
      Previous value: -"YYYY-MM-DD"New value: +"Evaluation start, YYYY-MM-DD."
    • addedInput schema / properties / window / properties / to / description
      Added value: +"Evaluation end, YYYY-MM-DD. Default: today."
  5. Changed1 schema field changed
    • changedInput schema / properties / market / properties / asset_type / enum
      Previous value: -[
-  "crypto"
-]New value: +[
+  "crypto",
+  "tokenized_equity",
+  "tokenized_etf",
+  "commodities"
+]
  6. Changed1 schema field changed
    • changedInput schema / properties / market / properties / asset_type / enum
      Previous value: -[
-  "crypto",
-  "stock",
-  "etf",
-  "commodities",
-  "forex"
-]New value: +[
+  "crypto"
+]
  7. First observed

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It fully discloses the return format (EVIDENCE verdict plus metrics, flags, caveats), the checks performed (look-ahead-aware, net of cost, out-of-sample split, 30-round-trip gate), and explicitly states it is NOT a recommendation. It also mentions the 'API Pro tier' access constraint, which is not covered elsewhere. This is exceptional transparency that goes well beyond a typical description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact yet comprehensive, covering purpose, inputs, checks, output, and access in about 150 words. It is front-loaded with the core question and key details, with every sentence adding value. While the density might benefit from bullet points, it is well-structured and appropriately sized for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The schema is thorough (100% coverage), and the description explains the tool's behavior, returns an EVIDENCE verdict, and lists the checks performed. It does not provide an output schema, but it names the possible verdict values. It also mentions the Pro tier. For a tool with nested objects and multiple input types, this description, combined with the detailed schema, is complete enough for an agent to know when and how to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so each parameter is already well documented. The description adds context about the input types (rules, signal_list, trade_list) and the overall purpose, but does not add detailed parameter-specific guidance beyond what the schema provides. The 'API Pro tier' note is access-related, not parameter semantics. Given the high schema coverage, a baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear question and states the tool's function: 'Backtest a trading strategy honestly' with a specific methodology (Deflated-Sharpe-Ratio / multiple-testing correction). It lists the three input types (rules, signal_list, trade_list), returns an EVIDENCE verdict, and explicitly states it is NOT a buy/sell recommendation. It also distinguishes itself from generic backtest tools, making its purpose unambiguous and well differentiated among the many arena_* siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description advises 'Call this before acting on a strategy or signal list' and notes it is 'not reproducible via generic backtest tools that ignore overfitting,' giving clear when-to-use context. However, it does not explicitly name sibling tools such as arena_run_backtest or arena_get_backtest, nor does it specify when not to use it (e.g., if you only need a simple backtest). This leaves some ambiguity but still provides strong usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Backtesting Arena toolsInspect
Try in Browser
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A3.6/5.0
Disambiguation2/5

Many tools cover overlapping market indicators (e.g., cycle state, pulse, bullmarket ampel, volatility phases) and several share similar get_*_history patterns, which could cause an agent to select the wrong one. However, each tool has detailed descriptions with explicit references to related tools to reduce ambiguity.

Naming Consistency3/5

Tool names generally follow a verb_noun pattern (arena_get_*, arena_list_*, arena_run_*, arena_subscribe_*), but there are inconsistencies such as 'validate_strategy' (no arena_ prefix), 'arena_status' (not a clear verb_noun), and variations like 'arena_run_grid_backtest' vs 'arena_run_backtest'.

Tool Count2/5

With 84 tools, the server is heavily over-scoped for a typical MCP server, likely causing navigation and selection overhead. While the domain (crypto backtesting and analytics) is broad, 84 tools exceed reasonable coherence and impose a high cognitive load.

Completeness4/5

The tool surface covers a wide range of analytics (market indicators, backtesting, validation, alerts, subscriptions, reports) with few obvious dead ends. Minor gaps exist like lack of direct portfolio management or strategy editing, but core workflows are well covered.

Resources