Skip to main content
Glama

Ground Truth - Fresh Evidence for AI Agents

by io.github.anish632

Multi-step Hypothesis Test

test_hypothesis
Read-onlyIdempotent

Run a small verification plan made of concrete live checks and summarize whether a hypothesis is supported. Use this when one conclusion depends on multiple simple checks such as endpoint reachability, npm search counts, or whether a page contains an exact substring. This is a coordination tool, not an open-ended research agent: every test must be explicitly defined in advance, and tests run in order with no branching or early exit. The final verdict is mechanical: all tests passing => SUPPORTED, zero passing => REFUTED, otherwise PARTIALLY SUPPORTED. Use verify_claim when you already have evidence URLs, estimate_market for category sizing, and compare_competitors when you already know exact package names.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
testsYesOrdered list of one to ten checks to run. Each test object uses only the fields required by its type.
hypothesisYesClaim to test, for example 'there are fewer than 50 MCP email servers on npm'.

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
testsYesPer-test execution results in input order.
verdictYesHigh-level verdict for the hypothesis.
hypothesisYesHypothesis that was evaluated.

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. Changed1 schema field changed
    • changedInput schema / properties / tests / items / anyOf
      Previous value: -[
-  {
-    "additionalProperties": false,
-    "properties": {
-      "description": {
-        "description": "Short explanation of what this endpoint check is meant to prove.",
-        "minLength": 3,
-        "type": "string"
-      },
-      "type": {
-        "const": "endpoint_exists",
-        "description": "Perform one unauthenticated GET request and pass when the endpoint returns a 2xx HTTP status.",
-        "type": "string"
-      },
-      "url": {
-        "description": "Public URL to probe, for example https://api.github.com.",
-        "format": "uri",
-        "type": "string"
-      }
-    },
-    "required": [
-      "description",
-      "type",
-      "url"
-    ],
-    "type": "object"
-  },
-  {
-    "additionalProperties": false,
-    "properties": {
-      "description": {
-        "description": "Short explanation of what this npm lower-bound count check is meant to prove.",
-        "minLength": 3,
-        "type": "string"
-      },
-      "query": {
-        "description": "npm search phrase to count, for example 'mcp email server'.",
-        "minLength": 2,
-        "type": "string"
-      },
-      "threshold": {
-        "description": "Lower bound that the npm search result count must exceed.",
-        "minimum": 0,
-        "type": "integer"
-      },
-      "type": {
-        "const": "npm_count_above",
-        "description": "Search npm and pass when the reported result count is strictly greater than the threshold.",
-        "type": "string"
-      }
-    },
-    "required": [
-      "description",
-      "type",
-      "query",
-      "threshold"
-    ],
-    "type": "object"
-  },
-  {
-    "additionalProperties": false,
-    "properties": {
-      "description": {
-        "description": "Short explanation of what this npm upper-bound count check is meant to prove.",
-        "minLength": 3,
-        "type": "string"
-      },
-      "query": {
-        "description": "npm search phrase to count, for example 'business verification mcp'.",
-        "minLength": 2,
-        "type": "string"
-      },
-      "threshold": {
-        "description": "Upper bound that the npm search result count must stay below.",
-        "minimum": 0,
-        "type": "integer"
-      },
-      "type": {
-        "const": "npm_count_below",
-        "description": "Search npm and pass when the reported result count is strictly less than the threshold.",
-        "type": "string"
-      }
-    },
-    "required": [
-      "description",
-      "type",
-      "query",
-      "threshold"
-    ],
-    "type": "object"
-  },
-  {
-    "additionalProperties": false,
-    "properties": {
-      "description": {
-        "description": "Short explanation of what this response-content check is meant to prove.",
-        "minLength": 3,
-        "type": "string"
-      },
-      "substring": {
-        "description": "Exact case-sensitive text to search for in the fetched response body.",
-        "minLength": 1,
-        "type": "string"
-      },
-      "type": {
-        "const": "response_contains",
-        "description": "Fetch a public URL and pass when the response body contains the exact substring using case-sensitive matching. The tool does not parse DOM structure or execute JavaScript before matching.",
-        "type": "string"
-      },
-      "url": {
-        "description": "Public URL whose response body should contain the expected text.",
-        "format": "uri",
-        "type": "string"
-      }
-    },
-    "required": [
-      "description",
-      "type",
-      "url",
-      "substring"
-    ],
-    "type": "object"
-  }
-]New value: +[
+  {
+    "additionalProperties": false,
+    "properties": {
+      "description": {
+        "description": "Short explanation of what this endpoint check is meant to prove.",
+        "minLength": 3,
+        "type": "string"
+      },
+      "type": {
+        "const": "endpoint_exists",
+        "description": "Perform one unauthenticated GET request and pass when the endpoint returns a 2xx HTTP status.",
+        "type": "string"
+      },
+      "url": {
+        "description": "Public URL to probe, for example https://example.com.",
+        "format": "uri",
+        "type": "string"
+      }
+    },
+    "required": [
+      "description",
+      "type",
+      "url"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": false,
+    "properties": {
+      "description": {
+        "description": "Short explanation of what this npm lower-bound count check is meant to prove.",
+        "minLength": 3,
+        "type": "string"
+      },
+      "query": {
+        "description": "npm search phrase to count, for example 'mcp email server'.",
+        "minLength": 2,
+        "type": "string"
+      },
+      "threshold": {
+        "description": "Lower bound that the npm search result count must exceed.",
+        "minimum": 0,
+        "type": "integer"
+      },
+      "type": {
+        "const": "npm_count_above",
+        "description": "Search npm and pass when the reported result count is strictly greater than the threshold.",
+        "type": "string"
+      }
+    },
+    "required": [
+      "description",
+      "type",
+      "query",
+      "threshold"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": false,
+    "properties": {
+      "description": {
+        "description": "Short explanation of what this npm upper-bound count check is meant to prove.",
+        "minLength": 3,
+        "type": "string"
+      },
+      "query": {
+        "description": "npm search phrase to count, for example 'business verification mcp'.",
+        "minLength": 2,
+        "type": "string"
+      },
+      "threshold": {
+        "description": "Upper bound that the npm search result count must stay below.",
+        "minimum": 0,
+        "type": "integer"
+      },
+      "type": {
+        "const": "npm_count_below",
+        "description": "Search npm and pass when the reported result count is strictly less than the threshold.",
+        "type": "string"
+      }
+    },
+    "required": [
+      "description",
+      "type",
+      "query",
+      "threshold"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": false,
+    "properties": {
+      "description": {
+        "description": "Short explanation of what this response-content check is meant to prove.",
+        "minLength": 3,
+        "type": "string"
+      },
+      "substring": {
+        "description": "Exact case-sensitive text to search for in the fetched response body.",
+        "minLength": 1,
+        "type": "string"
+      },
+      "type": {
+        "const": "response_contains",
+        "description": "Fetch a public URL and pass when the response body contains the exact substring using case-sensitive matching. The tool does not parse DOM structure or execute JavaScript before matching.",
+        "type": "string"
+      },
+      "url": {
+        "description": "Public URL whose response body should contain the expected text.",
+        "format": "uri",
+        "type": "string"
+      }
+    },
+    "required": [
+      "description",
+      "type",
+      "url",
+      "substring"
+    ],
+    "type": "object"
+  }
+]
  2. First observed

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint false, so the safety profile is covered. The description adds substantial behavioral context beyond annotations: tests run in order with no branching or early exit, the verdict mapping is fully mechanical (all passing => SUPPORTED, zero passing => REFUTED, otherwise PARTIALLY SUPPORTED), and the tool is not an open-ended research agent. This goes well beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and well-organized: first sentence defines the action, second covers typical use cases, third imposes constraints, fourth explains the verdict algorithm, fifth names alternatives. Every sentence earns its place, and there is no redundant or filler content. It is appropriately sized for a tool of this complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate-to-high complexity (multiple test types, ordered execution, verdict logic), the description covers all key aspects: what it does, when to use it, behavioral constraints, verdict determination, and sibling alternatives. An output schema exists for return values, and the description already explains the verdict mapping, so nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds some context by listing example check types (endpoint reachability, npm search counts, substring presence) and emphasizing that tests must be explicitly defined, but this largely restates information already in the schema. The hypothesis parameter is self-explanatory. No additional syntax or format details beyond the schema are provided, so a 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool 'Run a small verification plan made of concrete live checks and summarize whether a hypothesis is supported' with a specific verb and resource. It also distinguishes itself from siblings by explicitly naming alternatives (verify_claim, estimate_market, compare_competitors) and framing itself as a coordination tool, not an open-ended research agent.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use guidance ('Use this when one conclusion depends on multiple simple checks...') and when-not-to-use guidance with named alternatives ('Use verify_claim when you already have evidence URLs...'). It also restricts usage by stating 'every test must be explicitly defined in advance' and 'no branching or early exit', which clarifies the boundary versus more flexible research tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Ground Truth - Fresh Evidence for AI Agents toolsInspect
Try in Browser
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.3/5.0
Disambiguation4/5

Most tools have clearly distinct purposes, but there are several lightly overlapping pairs (check_endpoint/preflight_endpoint, check_pricing/compare_pricing_pages, estimate_market/compare_competitors, test_hypothesis/verify_claim). The detailed descriptions and explicit 'do not use for' notes mitigate confusion, so misselection is unlikely.

Naming Consistency5/5

All tool names follow a consistent lowercase snake_case verb_noun pattern (e.g., check_endpoint, create_monitor, verify_claim). Even compound verbs like preflight_endpoint and run_monitor_now adhere to the same structure, making the naming fully predictable.

Tool Count4/5

19 tools is slightly heavy but reasonable for the broad scope covering endpoint checks, pricing, compliance, package research, claim verification, and monitor lifecycle. Each tool has a distinct role, though the count edges close to the 'heavy' range.

Completeness4/5

Core workflows are well-covered: create/list/delete/get/run/preview monitors, single and comparative pricing checks, endpoint and security verification, and claim testing. Minor gaps exist, such as no update_monitor tool or a way to explicitly verify a claim without pre-supplied evidence URLs, but these are workable.

Resources