Skip to main content
Glama
defog-ai
by defog-ai

Analysis Gym

Analysis Gym is a tiny MCP server for recording and scoring prospective equity earnings predictions made by AI agents.

It deliberately does not choose tickers, schedule runs, or invoke models. Your agent loop owns those decisions. The agent uses the existing FactIQ MCP server for research and calls Analysis Gym only to record a prediction, record the eventual actuals, or read the results.

Tools

  • record_prediction records an immutable forecast before the expected earnings time.

  • record_actuals settles all earlier predictions for a ticker and fiscal period.

  • get_results returns per-metric errors and a leaderboard grouped by harness, model, and thinking setting.

The five predicted values are revenue, EBITDA, net profit, free cash flow, and the first regular-session closing price after the earnings release.

Related MCP server: equity-desk

Run locally

uv sync
uv run analysis-gym

The server uses stdio transport and stores data in analysis_gym.sqlite3 in its working directory. Set ANALYSIS_GYM_DB_PATH to put the database elsewhere.

Codex

Add the server to ~/.codex/config.toml:

[mcp_servers.analysis-gym]
command = "uv"
args = ["--directory", "/absolute/path/to/analysis-gym", "run", "analysis-gym"]

Install and authenticate the FactIQ plugin separately. Then ask Codex, for example:

Pick an equity reporting soon. Use FactIQ to forecast its next-quarter revenue, EBITDA, net profit, free cash flow, and first post-earnings close. Record the forecast in Analysis Gym before the release.

Claude Code

claude mcp add analysis-gym -- \
  uv --directory /absolute/path/to/analysis-gym run analysis-gym

Use the same prompt and ensure the FactIQ plugin is also installed and authenticated.

Agent-side loop

A loop outside this repository can choose an upcoming event and run the same request through any set of CLI/model/thinking configurations. Each agent calls record_prediction itself. After earnings, call record_actuals once with a source URL, then use get_results to compare the configurations.

Analysis Gym uses symmetric mean absolute percentage error (SMAPE), where lower is better. It reports every metric separately and a simple mean across all five.

Metric definitions

  • EBITDA: operating income plus depreciation and amortization.

  • Free cash flow: operating cash flow minus capital expenditure.

  • Net profit: consolidated net income attributable to the parent/common shareholders.

  • Post-earnings close: the same session's close for a pre-market release, or the next regular session's close for an after-hours release.

All four financial values (submitted in millions) in a submission must use the same reporting currency.

Development

uv run pytest

Available Tools

3 tools
get_resultsA

Return predictions, per-metric errors, and the configuration leaderboard.

Lower symmetric mean absolute percentage error (SMAPE) is better. Set settled_only=false to include predictions whose actuals are not recorded yet.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
modelNo
tickerNo
harnessNo
settled_onlyNo
thinking_settingNo

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, description explains output content and SMAPE meaning, but does not cover edge cases, pagination, or auth requirements. Adequate but not thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two focused sentences with no filler. First sentence states main output, second adds key insight.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 6 parameters and output schema, description covers main output but omits parameter semantics for most fields, leaving some gaps for agent usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only settled_only is explained in description; 0% schema coverage leaves 5 parameters unexplained. Fails to compensate for lack of schema documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states verb (return) and resources (predictions, per-metric errors, leaderboard), distinguishing from sibling tools which record data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implicitly suggests usage for retrieving results vs recording siblings, but lacks explicit when-to-use or alternatives. Provides parameter guidance for settled_only.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

record_actualsB

Record published actuals and settle earlier predictions for the event.

EBITDA is operating income plus depreciation and amortization. Free cash flow is operating cash flow minus capital expenditure. For a pre-market release, use that session's close; for an after-hours release, use the next regular session's close.

ParametersJSON Schema
NameRequiredDescriptionDefault
sourceYes
tickerYes
currencyNoUSD
earnings_atYes
fiscal_periodYes
ebitda_millionsYes
revenue_millionsYes
net_profit_millionsYes
post_earnings_closeYes
free_cash_flow_millionsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It discloses that the tool modifies data ('settle earlier predictions') and defines key metrics. Missing are side effects like overwrite behavior, triggers, or access requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action. The definitions are clear and relevant. Minor improvement: could group parameter explanations more tightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite an output schema existing, the description fails to explain parameter formats or constraints, leaving many parameters unclear. Sibling tools indicate a workflow, but integration guidance is missing. The description is incomplete for a tool with 10 parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%—the description does not document any of the 10 parameters. It defines related terms (EBITDA, free cash flow) but ignores parameters like earnings_at, fiscal_period, and currency, leaving the agent without critical guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Record published actuals and settle earlier predictions for the event.' The verb 'record' and resource 'actuals' are specific. Sibling tools likely include a prediction variant, providing implicit differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides definitions for EBITDA and free cash flow, and explains which close price to use based on release time. However, it does not explicitly state when to use this tool vs. alternatives (e.g., record_prediction) or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

record_predictionA

Record one immutable pre-earnings forecast.

expected_earnings_at must be an ISO 8601 timestamp with a timezone. Revenue, EBITDA, net profit, and free cash flow must use the same reporting currency. post_earnings_close is the expected first regular-session close after release.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYes
notesNo
tickerYes
harnessYes
currencyNoUSD
fiscal_periodYes
ebitda_millionsYes
revenue_millionsYes
thinking_settingYes
net_profit_millionsYes
post_earnings_closeYes
expected_earnings_atYes
free_cash_flow_millionsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full disclosure burden. It highlights immutability ('immutable pre-earnings forecast') and format constraints. However, it does not mention idempotency, error behavior, side effects, or rate limits. Sufficient but not exhaustive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core purpose. Every sentence adds necessary constraints or definitions. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 13 parameters, 0% schema description coverage, and an output schema that is not described, the description leaves many gaps. It explains only a subset of parameters and does not cover return values, error conditions, or prerequisite checks for a complex recording tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so description must compensate. It explains semantics for expected_earnings_at (ISO 8601 with timezone), financial fields (same currency), and post_earnings_close. However, 8 of 13 parameters (e.g., ticker, fiscal_period, harness, model, thinking_setting, notes, currency) remain unexplained. Adds value for some but not all.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Record one immutable pre-earnings forecast.' This is a specific verb (record) and resource (immutable pre-earnings forecast). It distinguishes from siblings 'get_results' (reads) and 'record_actuals' (records actuals), so an AI agent can tell when to use this tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit constraints: ISO 8601 timestamp with timezone, same reporting currency for financial fields, and definition of post_earnings_close. It implies use for pre-earnings forecasts but does not explicitly state when not to use or compare to siblings. Still clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 3 tool updatesv0.1.0
    • First observedget_results
    • First observedrecord_actuals
    • First observedrecord_prediction

TDQS

A3.8/5.0
Disambiguation5/5

Each tool has a clear, distinct purpose: recording predictions, recording actuals, and fetching results. There is no overlap or ambiguity.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern (get_results, record_actuals, record_prediction), making them predictable and easy to understand.

Tool Count5/5

With 3 tools, the server is tightly scoped to the core operations of the earnings analysis domain—recording predictions, recording actuals, and retrieving results. No unnecessary tools.

Completeness5/5

The tool surface covers the full lifecycle of the domain: creating predictions, recording actuals to settle them, and retrieving results with performance metrics. No obvious gaps.

Maintenance

ActivitySlowing
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables AI agents to perform grounded equity research by analyzing tickers from SEC filings and market data, producing citation-guarded memos with pre-computed fundamentals.
    MIT
  • A
    license
    A
    quality
    A
    maintenance
    Investment decision tools for AI agents: portfolio status, isolated multi-agent committee analysis, auditable verdict history, and lookahead-protected backtests. Advisory only, no auto-trading; negative research results published.
    21
    83
    MIT
  • F
    license
    A
    quality
    B
    maintenance
    Provides multi-agent equity research for US markets with provenance-backed financial data from SEC EDGAR, technicals, macro, and Alpaca paper trading, enforcing risk limits and journaling theses.
    27
    -

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/defog-ai/analysis-gym'

If you have feedback or need assistance with the MCP directory API, please join our Discord server