rigor-mcp
The Rigor MCP server offers AI agents a suite of verified statistical inference tools, including:
Hypothesis Testing: One-sample, two-sample, and paired t-tests; one- and two-proportion z-tests; chi-squared goodness-of-fit and independence tests; one-way ANOVA.
Effect Size Calculation: Cohen's d, Cohen's h, and Cramér's V to quantify effect magnitude.
Power & Sample Size Analysis: Compute statistical power or required sample size for two-sample t-tests and two-proportion z-tests.
Multiple Comparisons Correction: Adjust p-values with Bonferroni or Benjamini-Hochberg (FDR) methods to control error rates.
Robustness: Provides assumption warnings and gracefully handles edge cases (e.g., infinite effect sizes) by returning null values and warnings instead of errors.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@rigor-mcpcalculate power for two-sample t-test with effect size 0.5 and power 0.8"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
rigor
Verified statistical inference for AI agents.
LLMs are decent at reciting statistics but bad at doing it reliably —
a t-statistic or a required sample size is a number recalled from
training data, not computed and checked. rigor is the alternative:
classical hypothesis testing (parametric and non-parametric),
correlation and regression, effect sizes, power/sample-size
calculation, and multiple-comparisons correction, computed from scratch
and returned as a cited, assumption-checked answer -- plus a decision
helper for picking the right tool and a batch tool for running/
correcting many comparisons at once, since "which test do I even use"
and "I forgot to correct for multiple comparisons" are their own common
failure modes, distinct from getting a single formula wrong.
A concrete case where this matters. The one sample-size number everyone half-remembers is Cohen (1988)'s own worked example: d=0.5, alpha=.05, power=.80 -> n≈64 per group. It's in every textbook and slide deck, so it's also what gets pattern-matched to when a similar-looking question comes up. Ask instead for d=0.46, power=.85 -- a modest, realistic revision, not a trick:
$ rigor power ttest-2samp --effect-size 0.46 --power 0.85
Required n per group = 84.86 (round up: 85)85, not "about 64" -- a third more participants to recruit than the
half-remembered number suggests, from a question that looks like the
famous one. The formula itself isn't hard (power.py runs the same
bisection search either direction, in a few lines); the failure mode
is that recalling a nearby-looking answer feels indistinguishable from
computing the right one, right up until the number's wrong.
Built as an MCP server: a scan of the current MCP ecosystem (Context7 for coding docs, several physics/engineering/chemistry/geo servers, even Bentley's STAAD integration) found statistics/experimental design as one of the few common agent needs nobody had covered yet.
The statistics themselves (rigor/distributions.py, inference.py,
nonparametric.py, correlation.py, regression.py,
effect_size.py, power.py, corrections.py, plus the decision/batch
helpers in advisor.py and batch.py) are pure standard library, no
dependencies. The package as a whole does depend on the official mcp
SDK, since the MCP server is a first-class part of what it ships, not
an add-on -- see Install.
Install
pip install rigor-mcp(the PyPI distribution is rigor-mcp since plain rigor was already
taken by an unrelated package; the importable package and the CLI
command are both still just rigor.) This gets you both console
commands, rigor (CLI) and rigor-mcp (MCP server) -- deliberately
one install, no extras to get right, since uvx rigor-mcp (how most
MCP clients would actually invoke this) has no way to request an
extra.
Related MCP server: Dr. QuantMaster MCP Server
What's in it
rigor/distributions.py— t, chi-squared, and F distributions built from scratch on stdlib (regularized incomplete gamma/beta), verified against exact closed-form identities (t(1) = Cauchy, chi2(2) = scaled exponential, t² = F(1, df)) rather than trusted transcription.rigor/inference.py— one-/two-sample and paired t-tests, one-/two-proportion z-tests, chi-squared goodness-of-fit and independence, Fisher's exact test (2x2, exact via the hypergeometric distribution — the small-sample alternative chi_square_independence's own low-expected-count warning points to), one-way ANOVA, and Levene's (Brown-Forsythe) test for equal variances. Each returns aTestResult: statistic, degrees of freedom, two-tailed p-value, a confidence interval, a citation, and assumption warnings (e.g. small-n normality reliance, low expected cell counts).rigor/nonparametric.py— Mann-Whitney U, Wilcoxon signed-rank, and Kruskal-Wallis: the non-parametric alternative to two_sample_t_test/paired_t_test/one_way_anova respectively, for when a parametric test's own assumption warnings make its result suspect. Rank-based, with tie correction; also returnsTestResult.rigor/correlation.py— Pearson (linear) and Spearman (monotonic, via ranks) correlation, each returned as aTestResult(H0: no association) with a confidence interval via the Fisher z-transform.rigor/regression.py— simple (single-predictor) ordinary least squares regression: slope, intercept, R², and a significance test + CI for the slope.rigor/effect_size.py— Cohen's d, Hedges' g, Cohen's h, Cramér's V, eta²/omega² (for one_way_anova), and rank-biserial correlation (for mann_whitney_u).rigor/power.py— power and required sample size for the one-/two-sample t-test and two-proportion z-test (the one-sample formula covers paired_t_test too, since a paired t-test is a one-sample t-test on the differences). The two directions (given n, find power; given power, find n) are exact numerical inverses of each other by construction (bisection on the same underlying power function), and sanity-checked against the Cohen (1988) d=0.5/α=.05/power=.80 textbook reference case (n≈64).rigor/corrections.py— Bonferroni and Benjamini-Hochberg (FDR) multiple-comparisons correction.rigor/advisor.py—recommend_test: a decision helper, not a statistic. Answer a few characteristics of the data/question (continuous/proportion/categorical/ordinal, how many groups, paired, small-or-skewed, association-not-difference) and get back which tool to call, what to call instead if this test's assumptions look shaky, and what to run alongside it -- compiling the cross-references every other module's docstrings already carry into one callable answer, so an agent doesn't need to have already read all of them to find the relevant one.rigor/batch.py—pairwise_group_comparisons: runs every pairwise comparison across 2+ groups (two_sample_t_testormann_whitney_u, your choice) and applies Bonferroni/BH correction to the whole batch in one call, instead of the agent orchestrating k*(k-1)/2 separate calls plus a correction call by hand and risking forgetting the correction step. The natural follow-upone_way_anova/kruskal_wallisalready recommend in their own docstrings once a result comes back significant.rigor/cli.py— a CLI over all of the above (rigor.pyat the repo root is a thin shim sopython3 rigor.py ...also works from a plain checkout, without installing anything).rigor/mcp_server.py— an MCP tool wrapper exposing all 32 operations to any MCP client (Claude Code, Claude Desktop, etc.). Smoke-tested end-to-end over stdio against a real client — tool discovery plus representative calls checked against known reference values, including the full round-trip still landing the Cohen (1988) case at n=63 and Fisher's original "lady tasting tea" case at p≈0.4857.
Usage
CLI, once installed:
rigor ttest one-sample --data 5.1,4.9,5.3,5.0,4.8,5.2 --mu0 5.0
rigor corr pearson --x 1,2,3,4,5 --y 2,4,5,4,5
rigor regress --x 1,2,3,4,5 --y 3,5,7,9,11
rigor nonparam mann-whitney --a 1,2,3 --b 4,5,6
rigor power ttest-2samp --effect-size 0.5 --power 0.8
rigor recommend --outcome-type continuous --n-groups 3 # which test fits?
rigor posthoc --groups "1,2,3|4,5,6|7,8,9" --labels A,B,C # pairwise + correction
rigor --help # full list of subcommands (ttest, ztest, chi2, fisher, anova,
# levene, nonparam, corr, regress, effect-size, power, correct,
# recommend, posthoc)or straight from a checkout without installing anything:
python3 rigor.py ttest one-sample --data 5.1,4.9,5.3,5.0,4.8,5.2 --mu0 5.0MCP server, over stdio (the transport local clients like Claude Code expect):
pip install rigor-mcp
rigor-mcpor from a checkout: pip install mcp && python3 -m rigor.mcp_server.
Register it with Claude Code:
claude mcp add rigor -- rigor-mcp(or, from a checkout: claude mcp add rigor -- python3 -m rigor.mcp_server,
run from this repo's root or with an absolute module path). For
interactive poking with the MCP Inspector, run it as a script rather
than the installed command — which means the package root has to be
put on the path by hand, since the Inspector imports the file directly:
pip install "mcp[cli]"
PYTHONPATH=. mcp dev rigor/mcp_server.pyA transport-level edge case, handled
cohens_d correctly returns +inf/-inf for zero-variance samples
(per its own documented contract), but non-finite floats serialize to
JSON null over MCP's structured content — which used to fail the
tool's own number-typed output schema and crash the call. The MCP
cohens_d tool now returns {"value": float | null, "warnings": [...]}
instead of a bare float, so that case is reported explicitly (null
value, a warning naming the direction) rather than blowing up. That
fix is specific to tools with a bare-scalar output schema — every
tool that returns a dict (all the TestResult-based ones, plus
simple_linear_regression) has been confirmed over real stdio to pass
a non-finite field straight through as JSON's non-standard Infinity,
since a generic dict return doesn't get a strict per-field number
schema. Of the bare-float tools, cohens_d is the only one that can
actually produce a non-finite value.
Tests
python3 -m unittest discover -s tests -v153 tests: 140 exercise the statistics/decision logic directly; 12
spawn mcp_server.py as a real MCP client would and check results over
the wire (skipped automatically if mcp isn't installed); 1 checks
that server.json's version hasn't drifted from pyproject.toml's (the
two aren't otherwise linked -- see test_release_metadata.py).
License
MIT — see LICENSE.
Available Tools
32 toolsbenjamini_hochberg_correctionARead-onlyIdempotent
Adjust a batch of p-values for multiple comparisons, controlling the false discovery rate. Less conservative than Bonferroni; the standard choice when testing many hypotheses at once.
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | false discovery rate to control; default 0.05 | |
| p_values | Yes | the batch of p-values to adjust |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, and non-destructive behavior; the description adds no additional behavioral detail, which is acceptable given the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise two-sentence description with no redundant information; clearly structured and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Sufficiently complete for the function's simplicity; provides purpose, comparison to alternative, and parameter context without needing output details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover both parameters fully, and the description does not add extra nuance beyond what is already stated for p_values and alpha.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool adjusts p-values for multiple comparisons while controlling the false discovery rate, and distinguishes it from Bonferroni as less conservative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Mentions it is the standard choice for many hypotheses and compares to Bonferroni, though it could be more explicit about when to prefer it over other FDR methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
bonferroni_correctionARead-onlyIdempotent
Adjust a batch of p-values for multiple comparisons, controlling the family-wise error rate. Conservative; use when any false positive among the batch is costly.
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | family-wise significance level to control; default 0.05 | |
| p_values | Yes | the batch of p-values to adjust |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, and non-destructive behavior. The description adds meaningful behavioral context beyond those: it says the correction is conservative and that it controls the family-wise error rate. This informs the agent about the tradeoff without repeating annotation properties.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the main action, and every phrase earns its place. It includes a behavioral caveat without padding, making it an example of efficient, structured documentation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter statistical computation tool with read-only annotations, the description provides enough functional context for the agent to select and invoke it. It could have added explicit mention that the output is adjusted p-values indistinguishable from a list, but the implied result is clear from the tool name and description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the parameters are already well documented in the schema. The tool description reinforces the idea of a batch of p-values and family-wise error rate, but does not add new parameter-specific syntax or edge-case information beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool's purpose with a specific verb ('adjust') and resource ('a batch of p-values'), and states the goal of controlling the family-wise error rate. It is easy to distinguish this from siblings because it explicitly calls out the conservative Bonferroni correction and the false-positive tradeoff.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear usage guidance: 'use when any false positive among the batch is costly.' It implies a contrast with less conservative alternatives but does not explicitly name or exclude Benjamini-Hochberg. This is clear context with acceptable room for refinement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chi_square_goodness_of_fitARead-onlyIdempotent
Test whether observed category counts match an expected distribution -- e.g. "are these six days-of-week signup counts evenly distributed, or skewed towards weekends?" Returns the chi-squared statistic, degrees of freedom (len-1), p-value, a citation, and a warning if any expected count is below 5 (the usual threshold below which this approximation gets unreliable).
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| expected | Yes | expected count per category, same length and category order as observed; does not need to sum to the same total | |
| observed | Yes | observed count per category |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds valuable context beyond annotations by specifying return values (chi-squared statistic, df, p-value, citation) and disclosing the expected-count-below-5 warning and approximation limitation. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose, followed by a concrete example and return-value/limitation details. Every sentence earns its place with no unnecessary filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description adequately explains what the tool returns and flags a key reliability caveat. The three parameters are fully documented in the schema, and the description provides enough context for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents observed, expected, and alpha well. The description reinforces the concept of matching observed counts to an expected distribution and mentions the low-expected-count warning, but it does not materially add parameter syntax or constraints beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific statistical goal: test whether observed category counts match an expected distribution. The weekday signup example and the mention of degrees of freedom (len-1) clearly distinguish this one-way goodness-of-fit test from sibling tests like chi_square_independence.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a concrete example of when to use the test ('are these six days-of-week signup counts evenly distributed...'), which provides clear context. It does not explicitly name alternatives or state when not to use it, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
chi_square_independenceARead-onlyIdempotent
Test whether the row and column variables of a contingency table are independent (e.g. "does group membership relate to outcome?"). Returns the chi-squared statistic, degrees of freedom, p-value, a citation, and a warning if any expected cell count is below 5 (consider cramers_v afterwards for effect size).
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| table | Yes | contingency table as a list of rows, each a list of raw counts (not proportions), e.g. [[treated_success, treated_failure], [control_success, control_failure]] for a 2x2 table |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds value by disclosing the exact return payload (chi-squared statistic, df, p-value, citation) and the warning behavior when any expected cell count is below 5, giving the agent behavioral expectations beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core purpose, and every clause contributes: purpose, example, return values, assumption warning, and effect-size follow-up. There is no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a statistical test with no output schema, the description covers purpose, returns, assumptions, and a recommended post-hoc measure. It does not explain the alpha parameter, but the schema's description already covers that, and the overall guidance is sufficient for an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description mentions contingency table concepts but does not add parameter-level detail beyond the schema's table example and alpha explanation; baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's verb and resource: 'Test whether the row and column variables of a contingency table are independent.' It includes an illustrative example ('does group membership relate to outcome?') and distinguishes this from sibling tests like chi_square_goodness_of_fit by focusing on contingency table independence.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context: use this when you have a contingency table and want to test independence. It also gives a follow-up recommendation ('consider cramers_v afterwards for effect size'), but it does not explicitly mention when not to use it or name alternatives, so it falls short of a full 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
cohens_dARead-onlyIdempotent
Standardized mean difference between two samples (pooled SD). Use alongside two_sample_t_test, which tells you whether a difference is significant but not how large it is. Rough guidance: ~0.2 small, ~0.5 medium, ~0.8 large -- context-dependent. Returns {"value": float or null, "warnings": [...]}. value is null only when both samples have zero variance and unequal means, where the effect size is mathematically infinite -- see the warning for which direction, and report the raw mean difference instead in that case.
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | first sample | |
| b | Yes | second sample |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only, idempotent, non-destructive behavior, so they carry the safety profile. The description adds valuable behavioral context beyond annotations: it explains the return format ({"value": ..., "warnings": [...]}), the edge case where value is null (zero variance in both samples), and recommends reporting the raw mean difference in that case. It also implies the assumption of pooled variance (homogeneity of variance) which is useful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, then adds usage context, interpretation, return format, and edge case handling in a logical sequence. Every sentence adds value, with no filler. It is concise yet comprehensive for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter statistical tool with no output schema, the description covers the essential aspects: what it computes, when to use it, what the return value looks like, and a specific edge case with actionable advice. It could be more explicit about the independence of samples (it says 'two samples' but doesn't say 'independent'), and it could mention assumptions like normality or equal variances, but it does state 'pooled SD' which hints at the homogeneity assumption. Overall, it is quite complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both a and b have descriptions ("first sample", "second sample") that are minimal. The description does not add extra semantic meaning beyond what the schema provides (e.g., it does not clarify that samples should be numeric arrays or that they represent independent groups). Baseline 3 is appropriate because the schema does the heavy lifting, and the description does not compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it computes the standardized mean difference between two samples using pooled SD. This is a specific verb+resource (calculate effect size) and distinguishes it from sibling effect sizes like cohens_h (proportions) and cramers_v (categorical).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly recommends using alongside two_sample_t_test and provides interpretation thresholds (0.2, 0.5, 0.8). This gives clear context for when to apply it, though it does not explicitly mention when not to use it (e.g., paired data) or alternatives like paired_t_test.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
cohens_hARead-onlyIdempotent
Effect size for a difference between two proportions (Cohen, 1988), via the arcsine-square-root transform -- more appropriate than a raw percentage-point difference since it stabilizes variance across the full [0, 1] range. p1 and p2 are interchangeable (the sign of the result just indicates direction); use alongside two_proportion_z_test, which tells you whether a difference is significant but not how large it is. Returns a float (can be negative); rough guidance: ~0.2 small, ~0.5 medium, ~0.8 large.
| Name | Required | Description | Default |
|---|---|---|---|
| p1 | Yes | a proportion in [0, 1] | |
| p2 | Yes | a proportion in [0, 1] |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the core safety profile is covered. The description adds valuable behavioral context: it returns a float that can be negative, the sign indicates direction, and provides the standard interpretation thresholds (0.2/0.5/0.8). This exceeds what annotations alone convey, though it does not discuss edge-case behavior (e.g., proportions at 0 or 1) or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences but packed with essential information: the transform rationale, interchangeability, usage relationship, return type, and interpretation scales. Every clause earns its place; no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple statistical function, the description covers all major aspects: what it computes, why it's appropriate, how to use it with a sibling tool, the return type, and interpretation guidance. The output schema likely just indicates a number, so no further return-value explanation is needed. No significant gaps detected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (both p1 and p2 have 'a proportion in [0, 1]' descriptions). Beyond that, the description adds that p1 and p2 are interchangeable and that the sign of the result only indicates direction, which is not present in the schema. This provides meaningful additional meaning, so it scores above the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that it computes an effect size for a difference between two proportions via the arcsine-square-root transform, explicitly distinguishing it from a raw percentage-point difference. It names the resource (two proportions) and the specific action (compute effect size), and the context distinguishes it from siblings like cohens_d (for means) and two_proportion_z_test (significance testing).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to 'use alongside two_proportion_z_test' and explains that the z-test tells significance but not size, directly addressing when to use this tool. It also notes interchangeability of p1 and p2, providing clear usage guidance and naming an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
cramers_vARead-onlyIdempotent
Effect size for a chi-squared test of independence (Cramer, 1946), normalized to [0, 1] regardless of table shape so it's comparable across tables of different sizes, unlike the raw chi-squared statistic. Call after chi_square_independence, passing its statistic and the same table's n/rows/cols. Returns a float in [0, 1]; rough guidance for a 2x2 table: ~0.1 small, ~0.3 medium, ~0.5 large -- the threshold shifts for larger tables.
| Name | Required | Description | Default |
|---|---|---|---|
| n | Yes | total number of observations in the table | |
| cols | Yes | number of columns in the table | |
| rows | Yes | number of rows in the table | |
| chi2_statistic | Yes | the chi-squared statistic from chi_square_independence on the same table |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations already setting readOnlyHint and idempotentHint to true, the description adds valuable behavioral context: it returns a float in [0, 1], provides interpretation thresholds for 2x2 tables, and notes that thresholds shift for larger tables. This exceeds annotation coverage without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized, front-loaded with the definition, followed by usage context and interpretation guidance. Every sentence contributes value without redundancy, earning a top score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and comprehensive annotations, the description covers purpose, usage, and interpretation well. It is complete enough for an agent to select and invoke the tool correctly, with minor gaps around edge cases (e.g., zero rows/columns) being non-essential.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds meaning by explaining the relationship between chi2_statistic and chi_square_independence, and clarifies that rows/cols come from 'the same table,' which helps disambiguate parameter usage beyond the schema's individual descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly defines Cramér's V as an effect size for a chi-squared test, normalized to [0, 1] for comparability across table sizes. It uses specific verbs and resources, distinguishing it from the raw chi-squared statistic and aligning well with its role among statistical sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs when to use: 'Call after chi_square_independence, passing its statistic...' and differentiates from raw chi-squared. While it doesn't explicitly list when not to use it or name alternative effect sizes like cohens_d, the guidance is clear and contextual.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
eta_squaredARead-onlyIdempotent
Effect size for a one-way ANOVA: proportion of total variance explained by group membership. Use alongside one_way_anova, which tells you whether groups differ but not how much of the variance that accounts for. Rough guidance: ~0.01 small, ~0.06 medium, ~0.14 large. Biased upward for small samples -- prefer omega_squared when that matters. Returns a float in [0, 1].
| Name | Required | Description | Default |
|---|---|---|---|
| groups | Yes | one list of observations per group; at least 2 groups |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description warns that the statistic is biased upward for small samples, which is a behavioral trait not covered by annotations. It also specifies the output range [0,1]. However, it does not discuss edge cases such as missing data or unbalanced group sizes, so transparency is not fully exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: it states the definition, usage guidance, interpretation thresholds, a caution about bias, and the return type, all in a compact multi-sentence format with no superfluous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple statistical function, the description covers all necessary context: definition, interpretation, relationship to other tests, a caveat about bias, and the output type. It provides sufficient information for an agent to understand the tool's role and limitations without needing additional details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides a detailed description for the 'groups' parameter, including the requirement of at least two groups and the structure of one list per group. The tool description adds no further parameter-specific information, so the baseline score of 3 for high schema coverage applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that this tool computes the effect size for a one-way ANOVA as the proportion of total variance explained by group membership. It explicitly differentiates from one_way_anova, which tests for group differences but does not quantify variance explained, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises using this tool alongside one_way_anova and notes a preference for omega_squared in small samples due to upward bias. This provides concrete guidance on when to use the tool and when to consider an alternative, though it could be more explicit about the exact conditions for preferring omega_squared.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fisher_exact_testARead-onlyIdempotent
Test whether the row and column variables of a 2x2 contingency
table are independent -- exact (via the hypergeometric distribution
over all tables with the same margins), unlike
chi_square_independence's chi-squared approximation. Use this
instead whenever chi_square_independence warns an expected cell
count is below 5, or whenever the sample is small. 2x2 tables only.
Returns the sample odds ratio as statistic (can be inf/0 for a
zero cell), a two-tailed p-value, a citation, and warnings.
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| table | Yes | 2x2 contingency table as [[a, b], [c, d]], raw non-negative integer counts |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive. The description adds valuable behavioral context: specifies return values (odds ratio, two-tailed p-value, citation, warnings) and edge cases like inf/0 for zero cells. This is informative beyond annotations, though not exhaustive regarding caveats like ties or exact computation details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is succinct, front-loaded with the core purpose, and every sentence adds value: statistical method, contrast, usage guidance, constraint, and output details. No redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (statistical test) and lack of output schema, the description covers all necessary aspects: what it does, when to use it, constraints, and outputs. It provides enough context for an agent to select and invoke correctly without additional documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for both parameters, so the description does not add significant meaning beyond the schema. It mentions '2x2 tables only' and implicitly requires non-negative integer counts, but those are already in the schema description. The description does not elaborate on alpha beyond the schema, so it adds little extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool tests independence in a 2x2 contingency table using an exact method (hypergeometric distribution), and explicitly contrasts it with chi_square_independence. This makes the purpose unambiguous and differentiates it from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'Use this instead whenever chi_square_independence warns an expected cell count is below 5, or whenever the sample is small.' Also states the limitation '2x2 tables only,' helping the agent decide when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kruskal_wallisARead-onlyIdempotent
The non-parametric alternative to one_way_anova -- use when that test's own small-df warning makes a normal-theory result suspect. Tests whether all groups are drawn from the same distribution, by ranking the combined data rather than assuming normal populations. A significant result means at least one group differs, not which one -- same caveat as one_way_anova.
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| groups | Yes | one list of observations per group; at least 2 groups |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already convey readonly, idempotent, nondestructive behavior. The description adds meaningful extra context: the test ranks combined data, avoids normality assumptions, and its result is non-explanatory about which group differs. This goes beyond annotations and helps the agent set expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured into exactly three useful sentences: what it is, when to use it, and how to interpret results. Every sentence earns its place without redundant or promotional language.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a statistical hypothesis-testing tool without an output schema, this description provides the necessary conceptual framing: purpose, timing, and interpretation. It could mention return values explicitly, but the description plus the schema is enough for an agent to safely select and invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameters are already well documented: groups is a list of observation lists, alpha is a defaulted significance level. The description adds conceptual context for the test but not additional parameter-specific detail, which matches the schema-heavy baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as the non-parametric alternative to one_way_anova and states that it tests whether all groups are drawn from the same distribution. It also distinguishes the tool's scope by mentioning ranking of combined data and the caveat that a significant result does not identify which group differs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the agent when to use this tool: when one_way_anova's own small-df warning makes normal-theory results suspect. It provides a clear comparison with a named sibling tool, though it does not explicitly mention the two-group alternative (e.g., mann_whitney_u) or state direct exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
levene_testARead-onlyIdempotent
Test whether two or more groups have equal population variances (homogeneity of variance) -- use this to decide equal_var for two_sample_t_test, or to sanity-check one_way_anova's equal-variance assumption. Uses the Brown-Forsythe variant (deviations from each group's median), more robust to non-normal data than the original mean-based Levene's test. Returns the same shape as one_way_anova (it's computed as one internally, on absolute deviations from each group's median).
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| groups | Yes | one list of observations per group; at least 2 groups, each with at least 2 observations |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly/idempotent annotations, the description discloses the Brown-Forsythe variant, explains that deviations are taken from group medians, notes robustness to non-normal data, and clarifies the internal computation path as one_way_anova applied to absolute deviations. This provides meaningful behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact but information-dense. The first sentence establishes the core purpose, the second adds usage framing, and the third explains the statistical variant and output shape. Each sentence contributes necessary information with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, method, robustness considerations, and its relationship to one_way_anova. Since there is no output schema, the mention that it returns the same shape as one_way_anova helps, though it could be clearer by explicitly naming the Levene F-statistic and p-value.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents both alpha and groups with clear descriptions, and coverage is 100%. The description adds broader context about groups, but does not substantially add parameter-level meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool tests whether two or more groups have equal population variances (homogeneity of variance). It also distinguishes itself from related statistical tests by explicitly mentioning its role in supporting two_sample_t_test and one_way_anova.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit usage context: use it to decide the equal_var parameter for two_sample_t_test or to sanity-check one_way_anova's equal-variance assumption. It does not explicitly state when not to use it or name alternative variance-comparison approaches, so it misses full exclusionary guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
mann_whitney_uARead-onlyIdempotent
The non-parametric alternative to two_sample_t_test -- use when that test's own small-n warning makes a normal-theory result suspect, or the data is ordinal/skewed. Tests whether values from sample a are systematically larger or smaller than values from sample b, by ranking the combined data rather than assuming normal populations. statistic is U for sample a; pair with rank_biserial_correlation for a standardized effect size. Returns the same result shape as the parametric tests (statistic, p_value, citation, warnings).
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | first independent sample | |
| b | Yes | second independent sample | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and destructiveHint, so safety is covered. The description adds valuable behavioral context: statistic is U for sample a and returns the same result shape as parametric tests (statistic, p_value, citation, warnings), which explains expected output without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact yet thorough: usage condition, mechanism, statistic meaning, effect size pairing, and return shape. Each sentence earns its place, front-loads the key differentiator, and has no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description covers the essential aspects: when to use, what it does, what statistic to expect, how to pair with effect size, and the result shape. It's comprehensive for a hypothesis test with clear parameters and annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all parameters (a, b, alpha) described. The description doesn't add much beyond schema; it refers to 'sample a' and 'sample b' which mirrors the schema. It adds the meaning of U but not directly about parameters. Baseline 3 is appropriate given high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies this as the non-parametric alternative to two_sample_t_test, stating it tests whether values from sample a are systematically larger/smaller than from sample b via ranking. It distinguishes from siblings by name-dropping two_sample_t_test and rank_biserial_correlation, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: when the two_sample_t_test's small-n warning makes normal-theory suspect, or data is ordinal/skewed. It also recommends pairing with rank_biserial_correlation for effect size, providing clear guidance on alternatives and complementary tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
omega_squaredARead-onlyIdempotent
Effect size for a one-way ANOVA, less biased than eta_squared for small samples since it subtracts out the variance explained by chance alone. Use alongside one_way_anova. Can be slightly negative when the true effect is near zero -- that's expected, not an error.
| Name | Required | Description | Default |
|---|---|---|---|
| groups | Yes | one list of observations per group; at least 2 groups |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnly, idempotent, non-destructive behavior. The description adds meaningful behavioral nuance beyond that: it subtracts chance variance, can yield slightly negative values near zero effect, and explains that this is expected rather than an error. This is valuable interpretive context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences, front-loaded with the core definition, then guidance and edge-case explanation. No filler or repetition of schema or annotation details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, read-only statistic tool with an output schema present, the description is complete: it states what it calculates, when to use it, how it relates to a sibling, and how to interpret unusual output. No critical context is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema fully documents the single 'groups' parameter with a clear description ('one list of observations per group; at least 2 groups'), so the description does not need to add more. The tool description adds contextual meaning by tying it to one-way ANOVA, but the schema carries the parameter semantics burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly identifies the tool as computing an effect size for one-way ANOVA, using a specific verb and resource. It explicitly distinguishes itself from eta_squared by noting it is less biased for small samples, which separates it from a sibling tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context: use alongside one_way_anova and prefer over eta_squared for small samples. It does not enumerate explicit exclusion cases, but the guidance is sufficiently directional for an AI agent to select it appropriately among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
one_proportion_z_testARead-onlyIdempotent
Test whether an observed proportion (successes out of n) differs from a hypothesized proportion p0 -- e.g. "is this coin fair (p0=0.5) given 55 heads in 100 flips?" Uses the normal approximation, which degrades for small n or p0 near 0 or 1; a warning is included when that assumption looks shaky. Returns the z-statistic, two-tailed p-value, a confidence interval for the true proportion, a citation, and warnings.
| Name | Required | Description | Default |
|---|---|---|---|
| n | Yes | total number of trials/observations | |
| p0 | Yes | the hypothesized true proportion to test against, in [0, 1] | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| successes | Yes | number of successes observed |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive. The description adds meaningful behavioral details: it mentions the normal approximation, the warning mechanism when assumptions are shaky, and the specific return components (z-statistic, p-value, CI, citation, warnings). This adds value beyond annotations by disclosing the tool's computational nature and output expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-organized paragraph of five sentences. It front-loads the core purpose, then covers assumptions, warnings, and return values. Every sentence provides essential information with zero redundancy or fluff, maintaining high density while remaining readable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's statistical nature and the lack of an output schema, the description adequately covers what the tool returns (list of outputs) and when it should be used (assumptions). It also provides a concrete example that aids understanding. There are no significant gaps: the warning about small n and p0 boundaries is crucial, and the description mentions it, making it complete for a statistician user.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so all parameters (successes, n, p0, alpha) are well-documented in the schema. The description adds an intuitive example and clarifies the meaning of p0, but does not provide additional syntax or detail beyond what the schema already offers. According to guidelines, baseline is 3 for high coverage, and the description adds marginal value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Test whether'), identifies the exact resource ('observed proportion'), and uses an illustrative example ('is this coin fair?') that clearly distinguishes from siblings like two_proportion_z_test. It unambiguously conveys the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the normal approximation assumption and warns about degradation for small n or extreme p0, giving clear context on when the tool is appropriate. It does not explicitly name alternatives like fisher_exact_test, but the limitation implicitly guides selection; thus it lacks explicit exclusions but provides strong contextual guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
one_sample_t_testARead-onlyIdempotent
Test whether a sample's mean differs from a hypothesized value mu0. Returns the t-statistic, degrees of freedom, two-tailed p-value, a confidence interval for the mean, and any assumption warnings.
| Name | Required | Description | Default |
|---|---|---|---|
| mu0 | Yes | the hypothesized population mean to test the sample against | |
| data | Yes | the sample; one number per observation | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint/idempotent/destructive, and the description adds behavioral detail about the return values (t-statistic, df, p-value, CI, assumption warnings), which goes beyond annotations. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the purpose and lists return values efficiently. No filler words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
All essential information is covered: purpose, key inputs, return values, and safety profile via annotations. No output schema exists, but the description explicitly lists the outputs. Minor omission of specific assumption checks but 'assumption warnings' covers it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter (data, mu0, alpha) explained in the schema. The description does not add additional parameter meaning beyond restating mu0 as the hypothesized value, so baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool tests whether a sample's mean differs from a hypothesized value mu0, using a specific verb and resource. It distinguishes from siblings like two_sample_t_test and paired_t_test by explicitly saying 'a sample's mean'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the use case: testing a single sample against a known value, which differentiates from sibling tools. However, it does not explicitly mention alternatives or when not to use it, so a 4 is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
one_way_anovaARead-onlyIdempotent
Test whether three or more independent groups have different means -- e.g. comparing average order value across three marketing channels. A significant result means at least one group differs from the others, not which one -- follow up with pairwise two_sample_t_test calls (correcting for multiple comparisons via bonferroni_correction or benjamini_hochberg_correction) to find which. Returns the F-statistic, between/within degrees of freedom, p-value, a citation, and a warning if within-group df is small.
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| groups | Yes | one list of observations per group; at least 3 groups, each with at least 2 observations |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description lists the return values (F-statistic, degrees of freedom, p-value, citation, warning) and mentions a potential warning for small within-group df. It doesn't contradict the annotations (readOnly, idempotent), and the nature of a statistical test implies no side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, starting with the primary purpose, then explaining interpretation and follow-up, and finally listing the outputs. It contains no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main aspects: purpose, interpretation, follow-up, and outputs. It lacks assumptions (e.g., normality, homogeneity of variance) but provides a warning for small df, which is a key check. Overall, it is sufficiently comprehensive for a statistical test.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not elaborate on the parameters beyond what the schema already provides. It mentions 'significance level' but does not explicitly tie it to the 'alpha' parameter, and it gives no additional detail on the 'groups' parameter beyond the schema's description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to test whether three or more independent groups have different means, with a concrete example. It also explains the interpretation of a significant result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use this test (three or more groups) and suggests follow-up pairwise t-tests with correction methods, which helps guide usage. However, it does not explicitly differentiate from other tests like ANOVA variants or non-parametric tests.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
paired_t_testARead-onlyIdempotent
Test whether the mean difference between paired observations (e.g. before/after measurements on the same subjects, or matched pairs) is zero. a[i] and b[i] must be the two measurements of the same pair -- use two_sample_t_test instead if the two samples are independent (different subjects in each group). Returns the t-statistic, degrees of freedom (n-1), two-tailed p-value, a confidence interval for the mean difference, a citation, and assumption warnings.
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | first measurement of each pair, e.g. 'before' | |
| b | Yes | second measurement of each pair, e.g. 'after' -- same length and pairing order as a | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, openWorldHint=false, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds value by specifying the return values (t-statistic, df, p-value, CI, citation, assumption warnings), which goes beyond annotations. It does not contradict any annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured paragraph that starts with the purpose, explains usage constraints, and lists outputs. It is concise with no filler, every sentence provides essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is thorough for a statistical test with no output schema. It states all key outputs, mentions assumption warnings, clarifies the two-tailed nature, and contrasts with alternatives. No critical information is missing for an agent to decide and use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema description coverage, the description adds critical context for parameters 'a' and 'b' by emphasizing the pairing requirement and order ('a[i] and b[i] must be the two measurements of the same pair') which is not fully conveyed by the schema alone. This enhances understanding beyond the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: testing whether the mean difference between paired observations is zero. It provides concrete examples ('before/after measurements on the same subjects, or matched pairs') and distinguishes itself from the sibling tool 'two_sample_t_test' by specifying when to use which, making it unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs when to use this tool (paired observations) and when not to ('use two_sample_t_test instead if the two samples are independent'). It also provides context on pairing requirements, which serves as clear guidance for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pairwise_group_comparisonsARead-onlyIdempotent
Run every pairwise comparison across 2+ groups and correct for multiple comparisons in one call, instead of orchestrating k*(k-1)/2 separate two_sample_t_test/mann_whitney_u calls plus a separate correction call by hand -- and forgetting the correction is one of the most common real mistakes this package exists to prevent. The natural follow-up after a significant one_way_anova/kruskal_wallis result: pass the same groups here to find which group(s) differ, not just whether any do. Returns every pair's statistic, raw p-value, whether it's still significant after correction, and an effect size, plus the correction method's citation and warnings.
| Name | Required | Description | Default |
|---|---|---|---|
| test | No | "t_test" (two_sample_t_test per pair, reports cohens_d) or "mann_whitney" (mann_whitney_u per pair, reports rank_biserial_correlation) -- match whichever you used for the overall group comparison (one_way_anova vs. kruskal_wallis) | t_test |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| groups | Yes | one list of observations per group; at least 2 groups | |
| labels | No | optional name per group, same length and order as groups; carried through to each comparison for readability | |
| equal_var | No | only used when test="t_test": assume equal population variances (pooled) instead of Welch's test, same meaning as two_sample_t_test's equal_var | |
| correction | No | "bh" (Benjamini-Hochberg, less conservative, default), "bonferroni" (more conservative), or "none" (raw p-values, e.g. if correcting elsewhere) | bh |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context: automatic multiple-comparison correction, return contents (statistic, raw p, significance after correction, effect size), and inclusion of citation/warnings. It also warns about the common mistake of forgetting correction, which enriches the agent's understanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with three purposeful sentences: what it does, when to use it, and what it returns. Despite the first sentence being long, every phrase earns its place—it explains the benefit, the avoided complexity, and the mistake it prevents. No filler or repetition of schema fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 parameters and no output schema, the description covers return values explicitly, including citation and warnings. It also provides usage context and differentiates from siblings. The absence of an output schema raises the burden, and the description meets it fully by enumerating the output fields. No critical operational details seem missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each of the 6 parameters already documented. The description itself does not elaborate on individual parameters beyond what the schema provides, so baseline 3 applies. It does not add new parameter-specific meaning, but the schema already covers semantics effectively.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Run every pairwise comparison across 2+ groups and correct for multiple comparisons in one call.' It explicitly contrasts with orchestrating k*(k-1)/2 separate two_sample_t_test/mann_whitney_u calls, distinguishing it from sibling tools. The verb 'Run' and resource 'pairwise comparisons' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance: 'The natural follow-up after a significant one_way_anova/kruskal_wallis result' and contrasts with manual orchestration of separate tests plus correction. It also tells the agent to match the test type to the overall comparison, effectively excluding alternatives like stand-alone t-tests or corrections.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pearson_correlationARead-onlyIdempotent
Test for a linear association between two paired variables -- e.g. "does hours studied predict test score?" statistic is r itself (in [-1, 1]), not a t-statistic. Returns r, df (n-2), a two-tailed p-value (H0: r=0), a confidence interval for r via the Fisher z-transform, a citation, and warnings. Use spearman_correlation instead if the relationship may be monotonic but not linear, or if outliers shouldn't dominate the result. Use simple_linear_regression instead for the actual slope (units of y per unit of x), not just the strength of association.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | first variable, one value per observation | |
| y | Yes | second variable, same length and pairing order as x | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already report readOnlyHint, idempotentHint, and destructiveHint false. The description supplements this by disclosing that the statistic is r, not a t-statistic, and describes the returned r, df, two-tailed p-value, Fisher z-transform CI, citation, and warnings. Minor context around assumptions is left implicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Each sentence in the description carries useful content: the method's purpose, the statistic produced, the returns, and the alternatives. The description is dense but not bloated, with no filler or repetition of schema content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the description compensates by enumerating the return values and statistical meaning. It also names alternatives to guide selection. Complete enough for the agent, though details like assumptions or handling of non-finite values are left out.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers all three parameters (x, y, alpha) with descriptions, so the description does not need to repeat those details. It does add the notion of 'paired variables' and an example, but it does not add semantic detail about alpha beyond the schema. This is a solid baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description identifies a specific statistical operation: test for a linear correlation between paired variables. It explicitly says the statistic is r and enumerates the returned values, which clearly distinguishes it from other correlation and regression tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage guidance is strong and direct: use spearman_correlation instead for monotonic non-linear relationships or when outliers should not dominate, and use simple_linear_regression to get the actual slope. This gives the agent clear when-to-use and when-not-to-use alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
power_for_one_sample_t_testARead-onlyIdempotent
Statistical power to detect a given Cohen's d with n observations, using a one-sample (or paired) t-test. Use for paired_t_test too -- it's a one-sample t-test on the differences, so the same power formula applies. Use sample_size_for_one_sample_t_test instead to solve for n given a target power. Returns a float in [alpha, 1].
| Name | Required | Description | Default |
|---|---|---|---|
| n | Yes | planned (or actual) number of observations | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| effect_size_d | Yes | the Cohen's d you want to be able to detect |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate the tool is read-only, idempotent, and non-destructive. The description adds the return type and range ('float in [alpha, 1]'), enhancing transparency without contradicting the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, with a clear opening statement of purpose, followed by usage notes and a return-value specification. No unnecessary fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, usage scenarios, return value, and relevant distinctions from sibling tools. Combined with the complete schema and annotations, it provides all necessary context for a statistical power function.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides detailed descriptions for all parameters (n, alpha, effect_size_d). The tool description repeats some of these meanings but adds no additional semantic information beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool calculates statistical power for a one-sample (or paired) t-test given a Cohen's d and sample size. It also distinguishes it from sibling tools by noting when to use the sample-size variant.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions applicability to paired t-tests and directs users to 'sample_size_for_one_sample_t_test' when they need to solve for n given a target power, providing clear usage guidance relative to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
power_for_two_proportion_testARead-onlyIdempotent
Statistical power to detect a difference between two proportions (e.g. two conversion rates) with n_per_group observations in each group, using a two-proportion z-test. p1 and p2 are interchangeable (only their difference matters) -- e.g. current vs. new conversion rate. Use sample_size_for_two_proportion_test instead to solve for n given a target power. Returns a float in [alpha, 1].
| Name | Required | Description | Default |
|---|---|---|---|
| p1 | Yes | a proportion in [0, 1] | |
| p2 | Yes | a proportion in [0, 1] | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| n_per_group | Yes | planned (or actual) observations per group |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds meaningful context beyond annotations: p1 and p2 are interchangeable (only difference matters), returns a float in [alpha, 1], and uses a two-proportion z-test. No contradiction with the readOnly/idempotent/non-destructive hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with purpose, and every sentence carries useful information (purpose, parameter semantics, alternative, return range). No fluff or irrelevant detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description adequately covers purpose, usage guidance, parameter nuance, and return range. It does not mention statistical assumptions (e.g., normal approximation) or clarify the output schema further, but the presence of an output schema and strong annotations makes this acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% parameter descriptions, so the baseline is 3. The description adds the insight that p1 and p2 are interchangeable and clarifies that n_per_group is per group, but does not fundamentally extend the schema's meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool computes statistical power for a two-proportion z-test, specifying the resource (two proportions) and the test type. It also distinguishes itself from the sibling sample-size tool by naming sample_size_for_two_proportion_test.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use this tool (detect difference between two proportions) and provides a clear alternative: 'Use sample_size_for_two_proportion_test instead to solve for n given a target power.' Also notes p1/p2 interchangeability, reducing misuse.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
power_for_two_sample_t_testARead-onlyIdempotent
Statistical power to detect a given Cohen's d with n_per_group observations per group, using a two-sample t-test. Power is the probability of correctly detecting a real effect of this size at the given alpha; a design with low power means a non-significant result would be inconclusive rather than good evidence the effect doesn't exist. Use sample_size_for_two_sample_t_test instead to solve for n given a target power. Returns a float in [alpha, 1].
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| n_per_group | Yes | planned (or actual) observations per group | |
| effect_size_d | Yes | the Cohen's d you want to be able to detect |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already convey read-only, idempotent, non-destructive behavior, and the description adds useful context beyond that by stating the return range is [alpha, 1] and explaining the statistical meaning of power. Minor details like whether the test is one-sided or two-sided and whether equal variances are assumed are not mentioned, but the core behavioral contract is clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and every sentence earns its place: it states the computation, explains the meaning of power, points to the relevant alternative tool, and states the return range. It is front-loaded with the core purpose and contains no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema, annotations, and high schema coverage, the description is largely complete. It provides return-range information, interpretation guidance, and a sibling-tool alternative. It could be slightly more complete by specifying assumptions such as two-sided testing or equal group sizes, but these are not severe gaps for this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all parameters clearly with 100% coverage, so the description does not need to repeat parameter details. It adds some interpretive context around n_per_group and effect_size_d by embedding them in the power definition, but it does not materially exceed what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly names the resource (statistical power for a two-sample t-test) and the key inputs (Cohen's d, n_per_group, alpha). It distinguishes itself from sibling tools by explicitly focusing on power calculation rather than sample-size estimation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly directs users to sample_size_for_two_sample_t_test when they need to solve for n given target power, which is an actionable usage guideline. It also explains how low power should be interpreted, which helps the agent decide when this tool is relevant.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rank_biserial_correlationARead-onlyIdempotent
Effect size for a Mann-Whitney U test. Call after mann_whitney_u, passing its statistic and the two sample sizes. Positive means sample 1's values tend to exceed sample 2's; negative means the reverse; 0 is no tendency either way. Returns a float in [-1, 1]; rough guidance mirrors Cohen's d: ~0.1 small, ~0.3 medium, ~0.5 large.
| Name | Required | Description | Default |
|---|---|---|---|
| n1 | Yes | size of the first sample passed to mann_whitney_u | |
| n2 | Yes | size of the second sample passed to mann_whitney_u | |
| u1_statistic | Yes | the statistic returned by mann_whitney_u (U for the first sample passed to it) |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as read-only, idempotent, and non-destructive. The description adds valuable behavioral context beyond annotations: sign interpretation (positive/negative/zero), the return range [-1, 1], and rough effect-size thresholds. There is no contradiction with the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences with no wasted words: purpose first, then usage, then interpretation. Every sentence earns its place, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, full schema coverage, clear annotations, and output schema, the description covers the essential context: what the tool computes, when to call it, what inputs to pass, and how to interpret the result. Nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with all three parameters already documented clearly. The description reinforces that u1_statistic, n1, and n2 come from mann_whitney_u and the two sample sizes, but it does not add substantial new parameter-level meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Effect size for a Mann-Whitney U test,' which precisely identifies the tool's role and distinguishes it from the many sibling statistical tests and effect sizes. It also explains the output's meaning, making the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the tool: 'Call after mann_whitney_u, passing its statistic and the two sample sizes.' This gives clear contextual guidance and prerequisite sequencing, though it does not enumerate explicit exclusions or alternative tools for when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
recommend_testARead-onlyIdempotent
Not sure which rigor tool fits your question? Answer a few characteristics of the data and get back which tool to call, why, what to call instead if this test's assumptions look shaky, and what to run alongside it (an effect size, a power calculation, a natural follow-up). Every test in this package already documents this guidance in its own docstring for the sibling comparisons it knows about -- this tool exists so you don't have to have already read every other tool's docstring to find the one relevant cross-reference. Pure decision logic, no statistics computed here.
| Name | Required | Description | Default |
|---|---|---|---|
| paired | No | for n_groups=2 (continuous/rank_or_ordinal/proportion): were the same subjects measured twice, rather than two independent groups? | |
| n_groups | No | 1 = one sample vs. a hypothesized value; 2 = two groups/conditions; 3+ = three or more groups. Ignored when testing_association=true. | |
| outcome_type | Yes | what kind of thing is being compared/measured: "continuous" (means), "proportion" (rates), "count_or_category" (category counts / contingency tables), or "rank_or_ordinal" (ordinal data -- always routed to a rank-based test) | |
| small_or_skewed | No | is the sample small, visibly skewed, or outlier-heavy? nudges toward the non-parametric alternative | |
| testing_association | No | this is "does x relate to/predict y" for two continuous or ranked variables, not a group comparison -- routes to correlation/regression instead | |
| two_categorical_variables | No | for outcome_type="count_or_category": testing association between two categorical variables (a contingency table) rather than counts against an expected distribution? |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already disclose that the tool is read-only, non-destructive, and idempotent, so the bar shifts. The description adds valuable behavioral context: it returns reasoning, alternatives, and follow-up recommendations, while performing no statistical computation. This is consistent with the annotations and gives agents a richer model of the tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the user's decision problem and remains focused throughout. The sentence about existing docstrings is slightly showy but makes the tool's role clearer, so it still earns its place; overall the length is appropriate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Even without an output schema, the description enumerates exactly what the agent will get back: which tool to call, why, what to call instead if assumptions look shaky, and what to run alongside. Combined with the complete parameter schema and safety annotations, this gives the agent a well-rounded picture of this small decision-only tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the input schema already explains all six parameters and their defaults. The description only refers to them generically as 'a few characteristics of the data' and does not add per-parameter meaning, which matches the baseline 3 for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as a decision/recommendation layer: it answers which statistical tool to call based on data characteristics. It also differentiates itself from every sibling statistical tool by saying it is 'pure decision logic, no statistics computed here.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use the tool: whenever the user is unsure which rigor tool fits their question. It also explains the value proposition over reading every sibling docstring, and implies the tool should not be used for actual statistical computation; however, it does not explicitly list 'when not to use' in a formal way.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sample_size_for_one_sample_t_testARead-onlyIdempotent
How many observations are needed to detect a given Cohen's d with a one-sample (or paired) t-test at the target power. Use for paired_t_test too -- a paired t-test is a one-sample t-test on the differences, so the same power formula applies. Returns a continuous value and a rounded-up integer to actually use.
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| target_power | No | desired probability of detecting the effect if it's real; 0.8 is the conventional target | |
| effect_size_d | Yes | the Cohen's d you want to be able to detect |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions the output format (continuous value plus rounded-up integer) and clarifies behavior for paired tests. Combined with readOnly and idempotent annotations, it provides good transparency, though it does not discuss any assumptions or edge cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, packed with essential information in two sentences. It avoids unnecessary fluff and directly addresses the tool's function and usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core purpose, the paired-test usage, and the output format, making it adequate for a simple statistical function. It does not delve into theoretical assumptions or edge cases, but these are not critical for basic usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The parameter descriptions in the schema already cover the meaning of alpha, target_power, and effect_size_d. The description adds minimal extra detail, mostly restating the context. As schema coverage is 100%, a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: calculating the required sample size for a one-sample or paired t-test given Cohen's d and target power. It also distinguishes from related tools by explicitly mentioning the one-sample/paired scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to use this tool for paired t-tests as well, noting the equivalence to a one-sample test on differences. However, it does not contrast with alternatives like two-sample t-tests, which are common alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sample_size_for_two_proportion_testARead-onlyIdempotent
How many observations per group are needed to detect a difference between two proportions (e.g. conversion rates) at the target power. p1 and p2 are interchangeable -- only their difference matters.
| Name | Required | Description | Default |
|---|---|---|---|
| p1 | Yes | a proportion in [0, 1] | |
| p2 | Yes | a proportion in [0, 1] | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| target_power | No | desired probability of detecting the effect if it's real; 0.8 is the conventional target |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish read-only, idempotent, non-destructive behavior, so the bar is lower. The description adds the per-group and interchangeability aspects, but the claim that 'only their difference matters' is an oversimplification for two-proportion power analysis and the description does not mention sidedness or formula assumptions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core computation, and every clause carries meaning. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple calculator with fully documented parameters and side-effect annotations, the description is mostly sufficient and states the returned quantity. However, it omits key statistical assumptions (e.g., two-sided test, equal allocation, continuity correction) that would be needed for fully correct invocation and interpretation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the schema already documents alpha and target_power defaults. The description adds that p1/p2 are interchangeable, which is useful, though the stronger 'only their difference matters' statement is potentially misleading about how p1 and p2 affect variance-based sample size.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence states the exact computable quantity: observations per group needed to detect a difference between two proportions at a target power. It clearly distinguishes from siblings like power_for_two_proportion_test and sample_size_for_two_sample_t_test by scope ('two proportions', 'per group').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use case (sample-size planning for a two-proportion comparison) is clear and the 'e.g. conversion rates' gives practical context. It does not explicitly name alternatives like power_for_two_proportion_test or state when not to use it, but the main use is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sample_size_for_two_sample_t_testARead-onlyIdempotent
How many observations per group are needed to detect a given Cohen's d with a two-sample t-test at the target power. Returns a continuous value and a rounded-up integer to actually use.
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| target_power | No | desired probability of detecting the effect if it's real; 0.8 is the conventional target | |
| effect_size_d | Yes | the Cohen's d you want to be able to detect |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds value beyond annotations by specifying the return format: a continuous value plus a rounded-up integer for practical use. It does not mention assumptions such as equal group sizes or two-tailed testing, but the annotations reduce the burden for a deterministic calculator.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, front-loaded with the core question the tool answers and followed by a practical detail about the return value. Every sentence earns its place with no redundant filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Since no output schema is present, the description appropriately explains what the tool returns. It covers the core calculation and output clearly. It lacks explicit alternative guidance and statistical assumptions, but for a simple three-parameter deterministic function with strong annotations, it is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already explains alpha, target power, and effect size. The description refers to 'Cohen's d' and 'target power' but does not add new parameter-level meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool computes per-group sample size for a two-sample t-test based on Cohen's d and target power. It also distinguishes the tool from power-calculation siblings by saying it returns required observations and the rounded-up integer to use.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the intended use case: planning a two-sample t-test study to detect a specified effect size at a given power. However, it does not explicitly contrast this with power_for_two_sample_t_test or sample_size_for_one_sample_t_test, so guidance for choosing among alternatives is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
simple_linear_regressionARead-onlyIdempotent
Fit y = intercept + slope * x by ordinary least squares -- single predictor only. Reports the slope (change in y per unit of x), the intercept, R^2 (proportion of y's variance explained by x), and a significance test + confidence interval for the slope (H0: slope=0). Use pearson_correlation instead if you only need the strength of a linear association, not its actual units/magnitude.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | the predictor variable, one value per observation | |
| y | Yes | the outcome variable, same length and pairing order as x | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description is not burdened to cover safety. It adds value by describing the computational outputs (slope, intercept, R^2, significance test, CI) and the OLS method. It doesn't mention data handling assumptions but does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the core functionality in the first, and provides an alternative tool in the second. Every sentence earns its place with no redundant phrasing or filler, achieving maximal clarity in minimal space.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool and the absence of an output schema, the description fully covers what the tool returns (slope, intercept, R^2, test, CI). It also clarifies the single-predictor scope and provides an explicit alternative, making it complete for an agent to select and invoke correctly without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter already has descriptions. The description augments this by explaining the slope as 'change in y per unit of x,' which adds interpretive meaning beyond the schema's generic descriptions. It also ties alpha to the significance test and CI, reinforcing its role.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fits a linear regression model y = intercept + slope * x via OLS, explicitly noting it is single-predictor only. It specifies the outputs (slope, intercept, R^2, slope test and CI) and directly distinguishes itself from pearson_correlation, making its purpose unambiguous and differentiated from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises using pearson_correlation instead when only the strength of association is needed, and implies use when the actual slope magnitude/units are required. It also states the limitation of single predictor, giving clear context for when this tool is appropriate vs alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spearman_correlationARead-onlyIdempotent
Test for a monotonic association between two paired variables, via the Pearson correlation of their ranks -- doesn't assume linearity and is far less sensitive to outliers' exact magnitude than pearson_correlation. Same return shape as pearson_correlation (statistic is rho itself, in [-1, 1]).
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | first variable, one value per observation | |
| y | Yes | second variable, same length and pairing order as x | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already indicate read-only and idempotent behavior, and the description adds that the statistic is rho in the range [-1, 1]. It also mentions the return shape matches Pearson correlation, which gives some insight into output structure, though it doesn't fully specify all returned fields (e.g., p-value).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two sentences that efficiently convey the core functionality, key differences from Pearson, and output characteristics. No redundant or verbose language is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description mentions the statistic and its range, it relies on referencing Pearson correlation for the full return shape, which may not be clear if the agent lacks that context. It does not explicitly state that a p-value is returned or how the alpha parameter influences the output, leaving some ambiguity for a complex statistical test.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides clear descriptions for all parameters (x, y, alpha) with full coverage. The description does not add significant meaning beyond what is already in the schema; it reiterates that variables are paired, which is implied by the schema. Thus, the description adds little to parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool tests for a monotonic association between two paired variables, explicitly mentioning it uses Pearson correlation on ranks (Spearman). It also distinguishes itself from Pearson correlation by noting it doesn't assume linearity and is less sensitive to outliers, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides guidance on when to use this test over Pearson correlation by highlighting its advantages for monotonic relationships and outlier robustness. However, it doesn't elaborate on alternatives like Wilcoxon or other non-parametric tests, so the guidance is good but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
two_proportion_z_testARead-onlyIdempotent
Test whether two independent proportions differ -- the standard test behind comparing conversion rates between two groups (e.g. an A/B test). Returns the z-statistic, two-tailed p-value, a confidence interval for the difference in proportions, a citation, and warnings.
| Name | Required | Description | Default |
|---|---|---|---|
| n1 | Yes | total observations in group 1 | |
| n2 | Yes | total observations in group 2 | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| successes1 | Yes | successes observed in group 1 | |
| successes2 | Yes | successes observed in group 2 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is established. The description adds valuable behavioral context by listing return values (z-statistic, p-value, CI, citation, warnings) and the 'two-tailed' nature of the test, which is not available from structured fields. No contradictions with annotations exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a compact, front-loaded, single paragraph with no filler. Each sentence earns its place: what the test does, its primary use case, and what it returns. It is neither over-specified nor under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderately simple statistical test, the description is largely complete: it explains purpose, use case, and output list. Since there is no output schema, listing the returns is especially helpful. It could improve by stating assumptions or constraints (e.g., independence, sufficient sample size), but the current content is sufficient for correct selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3 even without additional parameter details. The description adds global context about comparing proportions, but it does not enrich individual parameter semantics beyond what the schema already provides for successes, totals, and alpha.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb-object structure: 'Test whether two independent proportions differ', immediately clarifying the statistical test and its typical use case ('comparing conversion rates between two groups'). It also lists exact outputs (z-statistic, two-tailed p-value, confidence interval, citation, warnings), making it easily distinguishable from sibling tests like one_proportion_z_test or chi_square_independence.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states the intended use case: comparing independent proportions, with the A/B testing example as a concrete scenario. It does not explicitly name alternative tests or state when not to use this tool, so it stops short of full exclusion guidance, but the context provided is clear enough for appropriate selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
two_sample_t_testARead-onlyIdempotent
Test whether two independent samples have different means. Defaults to Welch's t-test (does not assume equal variances); pass equal_var=true for the classic pooled-variance test.
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | first independent sample | |
| b | Yes | second independent sample | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 | |
| equal_var | No | assume equal population variances (classic pooled-variance test) instead of Welch's test |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation layer already declares readOnlyHint=true and idempotentHint=true, so the description doesn't need to restate safety. The description adds behavioral value by disclosing that with default settings it runs Welch's test (which does not assume equal variances) and how to switch to the classic test. A slight deduction is appropriate as it doesn't mention return values (test statistic, p-value, confidence intervals), but for a read-only statistical test, the current disclosure is quite complete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two perfectly-formed sentences with zero filler. The phrasing is front-loaded with the verb+resource, the default behavior is stated, and the alternative path is given as a conditional, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a statistical hypothesis test with 100% schema coverage and no output schema required, the description covers the statistical assumptions, defaults, and usage context comprehensively. It doesn't mention return values, but for a non-destructive, deterministic test, the key information (null hypothesis, inputs, and key parameter) is present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the parameters are already well-documented (a and b as independent samples; alpha and equal_var with defaults and clear descriptions). The description adds semantic depth by explaining the statistical implication of equal_var=false (Welch's test) versus equal_var=true (pooled-variance test), complementing the schema's mechanical definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific verb+resource statement: 'Test whether two independent samples have different means.' It precisely scopes the operation (two independent samples, means comparison) and differentiates the default Welch's t-test from the classic pooled-variance version, distinguishing it from siblings like paired_t_test or one_sample_t_test.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool (for two independent samples testing mean differences) and the default behavior (Welch's test that assumes unequal variances). It also explicitly tells the agent when to pass equal_var=true for the alternative pooled-variance test, giving clear context on parameter-driven usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
wilcoxon_signed_rankARead-onlyIdempotent
The non-parametric alternative to paired_t_test -- use when that test's own small-n warning makes a normal-theory result suspect. Tests whether the median of the paired differences is zero, by ranking the absolute differences rather than assuming they're normally distributed. Pairs with a zero difference are dropped (and counted in a warning), the standard procedure. statistic is T = min(W+, W-).
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | first measurement of each pair, e.g. 'before' | |
| b | Yes | second measurement of each pair, e.g. 'after' -- same length and pairing order as a | |
| alpha | No | significance level for the test (and any confidence interval); default 0.05 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral details: pairs with zero difference are dropped and counted in a warning, and the statistic formula is disclosed (T = min(W+, W−)). This goes beyond annotations to clarify handling of edge cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three concise sentences that immediately state the core purpose, usage condition, and key behavioral details. No fluff, each sentence earns its place, and it is front-loaded with the most important information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a statistical test with no output schema, the description explains the test type, when to use it, handling of zero differences, and the statistic formula. It does not explicitly state return values (e.g., p-value, statistic), but the mention of 'statistic' and the test context make the output reasonably inferred. It is nearly complete for the complexity involved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all three parameters (a, b, alpha), already explaining that a and b are paired first/second measurements with same length and pairing order. The description does not add new parameter-level meaning beyond restating the paired nature; it focuses on the test itself, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool is the non-parametric alternative to paired_t_test, and specifies it tests whether the median of paired differences is zero by ranking absolute differences. This verb+resource+statistical purpose is specific and distinguishes it from sibling tests like one_sample_t_test and mann_whitney_u.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says to use this tool when paired_t_test's small-n warning makes normal-theory results suspect. It also names the alternative (paired_t_test) and implies it's for paired data, providing clear context for when to choose this over other statistical tests.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
32 tool updates
v0.3.0- Changed
benjamini_hochberg_correction2 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"false discovery rate to control; default 0.05" - added
Input schema / properties / p_values / descriptionAdded value: +"the batch of p-values to adjust"
- Changed
bonferroni_correction2 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"family-wise significance level to control; default 0.05" - added
Input schema / properties / p_values / descriptionAdded value: +"the batch of p-values to adjust"
- Changed
chi_square_goodness_of_fit3 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / expected / descriptionAdded value: +"expected count per category, same length and category order as observed; does not need to sum to the same total" - added
Input schema / properties / observed / descriptionAdded value: +"observed count per category"
- Changed
chi_square_independence2 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / table / descriptionAdded value: +"contingency table as a list of rows, each a list of raw counts (not proportions), e.g. [[treated_success, treated_failure], [control_success, control_failure]] for a 2x2 table"
- Changed
cohens_d2 fields changed- added
Input schema / properties / a / descriptionAdded value: +"first sample" - added
Input schema / properties / b / descriptionAdded value: +"second sample"
- Changed
cohens_h2 fields changed- added
Input schema / properties / p1 / descriptionAdded value: +"a proportion in [0, 1]" - added
Input schema / properties / p2 / descriptionAdded value: +"a proportion in [0, 1]"
- Changed
cramers_v4 fields changed- added
Input schema / properties / chi2_statistic / descriptionAdded value: +"the chi-squared statistic from chi_square_independence on the same table" - added
Input schema / properties / cols / descriptionAdded value: +"number of columns in the table" - added
Input schema / properties / n / descriptionAdded value: +"total number of observations in the table" - added
Input schema / properties / rows / descriptionAdded value: +"number of rows in the table"
- Added
eta_squared - Added
fisher_exact_test - Added
kruskal_wallis - Added
levene_test - Added
mann_whitney_u - Added
omega_squared - Changed
one_proportion_z_test4 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / n / descriptionAdded value: +"total number of trials/observations" - added
Input schema / properties / p0 / descriptionAdded value: +"the hypothesized true proportion to test against, in [0, 1]" - added
Input schema / properties / successes / descriptionAdded value: +"number of successes observed"
- Changed
one_sample_t_test3 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / data / descriptionAdded value: +"the sample; one number per observation" - added
Input schema / properties / mu0 / descriptionAdded value: +"the hypothesized population mean to test the sample against"
- Changed
one_way_anova2 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / groups / descriptionAdded value: +"one list of observations per group; at least 3 groups, each with at least 2 observations"
- Changed
paired_t_test3 fields changed- added
Input schema / properties / a / descriptionAdded value: +"first measurement of each pair, e.g. 'before'" - added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / b / descriptionAdded value: +"second measurement of each pair, e.g. 'after' -- same length and pairing order as a"
- Added
pairwise_group_comparisons - Added
pearson_correlation - Added
power_for_one_sample_t_test - Changed
power_for_two_proportion_test4 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / n_per_group / descriptionAdded value: +"planned (or actual) observations per group" - added
Input schema / properties / p1 / descriptionAdded value: +"a proportion in [0, 1]" - added
Input schema / properties / p2 / descriptionAdded value: +"a proportion in [0, 1]"
- Changed
power_for_two_sample_t_test3 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / effect_size_d / descriptionAdded value: +"the Cohen's d you want to be able to detect" - added
Input schema / properties / n_per_group / descriptionAdded value: +"planned (or actual) observations per group"
- Added
rank_biserial_correlation - Added
recommend_test - Added
sample_size_for_one_sample_t_test - Changed
sample_size_for_two_proportion_test4 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / p1 / descriptionAdded value: +"a proportion in [0, 1]" - added
Input schema / properties / p2 / descriptionAdded value: +"a proportion in [0, 1]" - added
Input schema / properties / target_power / descriptionAdded value: +"desired probability of detecting the effect if it's real; 0.8 is the conventional target"
- Changed
sample_size_for_two_sample_t_test3 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / effect_size_d / descriptionAdded value: +"the Cohen's d you want to be able to detect" - added
Input schema / properties / target_power / descriptionAdded value: +"desired probability of detecting the effect if it's real; 0.8 is the conventional target"
- Added
simple_linear_regression - Added
spearman_correlation - Changed
two_proportion_z_test5 fields changed- added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / n1 / descriptionAdded value: +"total observations in group 1" - added
Input schema / properties / n2 / descriptionAdded value: +"total observations in group 2" - added
Input schema / properties / successes1 / descriptionAdded value: +"successes observed in group 1" - added
Input schema / properties / successes2 / descriptionAdded value: +"successes observed in group 2"
- Changed
two_sample_t_test4 fields changed- added
Input schema / properties / a / descriptionAdded value: +"first independent sample" - added
Input schema / properties / alpha / descriptionAdded value: +"significance level for the test (and any confidence interval); default 0.05" - added
Input schema / properties / b / descriptionAdded value: +"second independent sample" - added
Input schema / properties / equal_var / descriptionAdded value: +"assume equal population variances (classic pooled-variance test) instead of Welch's test"
- Added
wilcoxon_signed_rank
17 tool updates
v0.1.0- First observed
benjamini_hochberg_correction - First observed
bonferroni_correction - First observed
chi_square_goodness_of_fit - First observed
chi_square_independence - First observed
cohens_d - First observed
cohens_h - First observed
cramers_v - First observed
one_proportion_z_test - First observed
one_sample_t_test - First observed
one_way_anova - First observed
paired_t_test - First observed
power_for_two_proportion_test - First observed
power_for_two_sample_t_test - First observed
sample_size_for_two_proportion_test - First observed
sample_size_for_two_sample_t_test - First observed
two_proportion_z_test - First observed
two_sample_t_test
TDQS
Every tool targets a distinct statistical procedure, and the descriptions actively disambiguate overlapping tools by cross-referencing when to use which (e.g., mann_whitney_u points to two_sample_t_test, rank_biserial_correlation is explicitly the paired effect size for it). The separation between tests, effect sizes, and power analyses is crystal clear. An agent could unambiguously route any statistical question to the right tool.
Overwhelmingly consistent snake_case: tests use standard statistical names (two_sample_t_test, chi_square_independence), power functions follow power_for_X / sample_size_for_X, and corrections use the statistician's name. Minor deviation: the power functions drop the 'z' from two_proportion_z_test (becoming power_for_two_proportion_test), and effect sizes mix named forms (cohens_d, cramers_v) with formulaic ones (eta_squared, omega_squared, rank_biserial_correlation), though all follow domain conventions.
At 32 tools, this exceeds the typical 3-15 well-scoped range and even the 16-25 'feels heavy' band. However, each tool genuinely earns its place in a comprehensive statistics package — the nominal scope is an entire introductory statistics course. The deliberate grouping into tests, effect sizes, power, and corrections makes the volume feel more navigable than the raw count suggests.
The core frequentist toolkit is well covered: parametric and non-parametric tests, pairwise comparisons with built-in correction, effect sizes that pair explicitly with each test, and power/sample-size analysis for the three most common tests. Obvious gaps exist: no normality test (Shapiro-Wilk) despite heavy reliance on parametric assumptions, no multiple regression beyond simple linear, and power functions for only 3 of the 14 tests. No bootstrap or resampling utilities, and no one-proportion power calculation.
Maintenance
Related MCP Connectors
Exact statistics & probability: distributions, hypothesis tests, CIs, Bayesian updates, regression.
The statistical analyst in your AI chat — validated, citable, re-runnable analysis of your data.
Valid and reliable data engineering and statistical analysis without hallucinations.
Verify claims with verdict, confidence & cited sources; batch verify, source checks, daily brief.
Related MCP Servers
- FlicenseAqualityDmaintenanceProvides tools for managing quantitative research knowledge graphs, enabling structured representation of research projects, datasets, variables, hypotheses, statistical tests, models, and results.69-
- AlicenseCqualityNot gradedmaintenanceAI-powered quantitative research assistant with 45 tools for causal inference methods (DID, RDD, IV, PSM), regression analysis, power calculations, and statistical code generation in R, Stata, and Python.50-
- FlicenseNot gradedqualityCmaintenanceEnables comprehensive statistical analysis including descriptive statistics, hypothesis testing, regression, and more via a FastMCP-based API.3-
- AlicenseAqualityAmaintenanceEval-integrity statistics for AI benchmark claims — multiple-testing correction, power/MDE for model gaps, judge-bias and leaderboard-rank checks. Catches a benchmark number that won't survive a second look.9MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/mrnh/rigor'
If you have feedback or need assistance with the MCP directory API, please join our Discord server