PCQ
The PCQ server provides tools for managing, validating, executing, and analyzing ML experiments through a contract-based workflow.
Project Inspection & Validation
Resolve and inspect
cq.yamlproject configuration, structure, and contract stateValidate project setup (static + contract) before execution, including optional ExperimentPlan validation
Check agent runtime asset status (installed/missing/stale/divergent)
Experiment Execution & Finalization
Execute commands defined in
cq.yamlwith auto-wired config environment, capturing logsFinalize completed runs by generating
run_record.jsonandvalidation_report.jsonValidate completed output directories against defined contracts
Run Analysis
Describe compact, decision-oriented summaries of runs (metrics, validation status, lineage, artifacts)
Compare two runs by diffing metric deltas, config changes, and lineage
Trace a run's full ancestry via its parent chain
Experiment Planning
Apply an
ExperimentPlanto modifycq.yamlconfigs with provenance trackingExpand an
ExperimentPlanSetinto N output directories for hyperparameter sweeps
Scaffolding & Agent Integration
Initialize new projects with
cq.yaml,train.py, and optionalpyproject.tomlInstall agent runtime assets (AGENTS.md/CLAUDE.md, skill files) for agents like Claude Code and Codex, with optional
.mcp.jsonwiring
All operations support JSON/JSONL output for machine consumption.
pcq
pcq is the contract for agent-run ML experiments. This repository hosts the contract specification under
spec/and the reference Python implementation undersrc/pcq/. Install the reference impl:uv add pcq(Apache-2.0).
The contract turns a project with cq.yaml into a reproducible experiment
unit. The reference Python implementation loads config, resolves output
paths, captures metrics, writes standard artifacts, finalizes run evidence,
and exposes JSON/JSONL/MCP surfaces that coding agents, CI jobs, notebooks,
and services can consume. See spec/IMPLEMENTATIONS.md
for the registered implementation list (Python reference + CQ Go production
worker today) and the procedure for adding yours.
pcq is not a training framework, model zoo, adapter matrix, or CQ-only
client. Use PyTorch, Hugging Face Trainer, Lightning, sklearn, TabPFN, PyCaret,
XGBoost, shell scripts, remote jobs, or project-local research code. The
contract is the integration layer.
pcq does not operate the model.
pcq operates the experiment boundary.SITE | INTRODUCTION | V4_DIRECTION | VISION | AGENT_OPERABILITY | RUN_RECORD | AGENT_OPERATING_GUIDE | CHANGELOG
Contract specification (single source of truth):
spec/INDEX.md |
SPEC |
CQ_YAML_RUNTIME_CONTRACT |
JSON_CONTRACTS |
STRICTNESS |
CQ_MCP_SPEC |
VERSIONING |
CONFORMANCE |
schemas/ (auto-exported via scripts/export_schemas.py)
Case studies (external evidence): mnist-dogfood | tabular-dogfood | mcp-dogfood | cq-worker-dogfood
Agent-readable site files: llms.txt, llms-full.txt, agent-manifest.json.
Identity
pcq = open-source experiment evidence/control library
cq = managed execution + orchestration + dashboard + agent loopCQ service is one managed consumer of the contract. pcq remains useful without
CQ: locally, in CI, in notebooks, and inside third-party orchestrators.
Related MCP server: PyTorch Lightning MCP Server
Why pcq
Framework-neutral — keep the training stack that fits the problem.
Agent-readable — use JSON/JSONL instead of terminal scraping.
Agent-verifiable — validate source, config, environment, metrics, artifacts, and run records.
Agent-operable — run, observe, validate, describe, compare, lineage, and iterate through stable commands.
Service-ready — CQ can consume the same contract for managed execution and automatic experiment loops.
What's New (v4.4 – v4.6)
Three agent-fillable metadata fields were added to run_record.json across the
last three minor releases, making each run's evidence richer with zero extra
code in most cases.
Field | Since | Captures | Auto-filled? |
| v4.4 | author / committer / operator — who ran the experiment | Yes (agent identity injected at runtime) |
| v4.5 | cpu / gpu / memory / os — where it ran | Yes ( |
| v4.6 | modality / task_kind / shape / PII-safe stats — what data | Semi-auto ( |
attribution — who
Records the human author, the AI committer, and the operator that launched the run. Coding agents (Claude Code, Codex) fill this automatically from their identity context.
pcq.attribution(
author={"kind": "human", "id": "alice"},
committer={"kind": "agent", "id": "claude-code"},
operator="ci-runner-42",
)Spec: spec/SPEC.md § Attribution
worker_spec — where
Records CPU model, core count, GPU kind/VRAM, total memory, and OS. Called with no arguments for a full auto-detection pass.
pcq.worker_spec() # 자동 감지 — 인수 불필요Spec: spec/SPEC.md § Worker Spec
fingerprint — what
Records dataset modality, task kind, sample count, size class, domain, and PII-safe summary statistics. Accepts a NumPy/pandas array or DataFrame and infers most fields.
pcq.fingerprint(X, y, modality="tabular")Spec: spec/SPEC.md § Fingerprint
All three fields are optional — existing runs remain valid. When present
they appear as first-class evidence in run_record.json and are surfaced
through pcq describe-run --json.
Reproducibility Substrate
3개의 선택적 필드로 독립 재현이 가능한 substrate를 run_record.json에 제공한다.
pcq는 검증하지 않는다 — 검증을 가능하게 만든다.
Field | Captures |
|
|
|
|
|
|
상세 스키마, PHI 게이트(R5), integrity 확장, R8 한계 문장: spec/SPEC.md § Reproducibility Pack
Note: code content sha proves WHAT code was recorded, not THAT it produced these outputs. See SPEC.md R8.
Note: pcq records claims, not judgments — intent is a recorded assertion (a fact about what was claimed), not a pcq verdict on success.
Installation
uv add pcq
# Optional — to expose pcq as MCP tools to agent runtimes:
uv add 'pcq[mcp]'pyproject.toml:
[project]
dependencies = ["pcq"] # core only
# or:
dependencies = ["pcq[mcp]"] # core + Model Context Protocol serverDocker (MCP server only)
A minimal container image is also published; it packages
pcq[mcp] from PyPI and runs pcq mcp serve on stdio.
docker build -t pcq .
docker run -i --rm pcq # MCP client attaches to stdin/stdoutThe image is intentionally scoped to the MCP server surface — for
pcq run, pcq describe-run, pcq agent install and other CLI
subcommands, install pcq directly with uv add pcq instead.
For a tag, branch, or private fork:
[tool.uv.sources]
pcq = { git = "https://github.com/playidea-lab/pcq.git", tag = "v4.1.0" }The PyPI distribution, import name, CLI command, GitHub repository, runtime
workspace, and JSON contract namespace are all pcq. Runtime contract names
from CQ remain stable: cq.yaml, CQ_CONFIG_JSON, and cq://.
Minimal Contract
cq.yaml declares the run:
name: sklearn-baseline
cmd: uv run python train.py
configs:
output_dir: output
seed: 42
strictness: 3
monitor: eval_acc
mode: max
metrics:
- epoch
- eval_acc
artifacts:
- output/
inputs: {}train.py can use any framework:
import pickle
import pcq
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
cfg = pcq.config()
out = pcq.output_dir()
pcq.seed_everything(cfg.get("seed", 42))
x, y = load_iris(return_X_y=True)
x_train, x_eval, y_train, y_eval = train_test_split(
x,
y,
test_size=0.25,
random_state=int(cfg.get("seed", 42)),
stratify=y,
)
model = RandomForestClassifier(random_state=int(cfg.get("seed", 42)))
model.fit(x_train, y_train)
eval_acc = float(model.score(x_eval, y_eval))
with (out / "model.pkl").open("wb") as f:
pickle.dump(model, f)
history = [{"epoch": 0, "eval_acc": eval_acc}]
pcq.log(**history[-1])
pcq.save_all(history=history, artifacts={"model": "model.pkl"})No sklearn adapter is required. The same pattern works for HF Trainer, Lightning, XGBoost, TabPFN, PyCaret, shell commands, or custom code.
Agent Command Surface
Read and validate the project:
pcq resolve --json
pcq inspect . --json
pcq validate . --strictness 2 --jsonRun the project:
pcq run --path . --json
pcq run --path . --jsonl
pcq run --path . --events output/events.jsonl --jsonValidate and summarize outputs:
pcq validate-run output --strictness 3 --json
pcq describe-run output --json
pcq compare-runs old_output new_output --json
pcq lineage output --jsonIterate:
pcq apply-plan experiment.plan.json --jsonAgent rule: prefer JSON/JSONL surfaces over scraping human output. pcq
reports facts; the agent or service chooses policy.
Standard Artifacts
A completed run should produce:
config.jsonmetrics.jsonmanifest.jsonrun_summary.jsonrun_record.jsonvalidation_report.json
run_record.json is the canonical completion object. It combines execution,
source, environment, input identity, metric schema, artifact manifest, agent
provenance, validation, and summary evidence.
Agent Runtime Assets
pcq can install its canonical agent instructions and skill into a project.
Package installation itself never mutates project agent files.
pcq agent install --target codex --path .
pcq agent install --target claude --path .
pcq agent install --target both --path . --dry-run --json
pcq agent status --target both --path . --jsonTo also wire the project for MCP-aware agents (Claude Code, Codex), install
pcq[mcp] and pass --mcp:
uv add 'pcq[mcp]'
pcq agent install --target claude --path . --mcp # writes .mcp.json
pcq mcp serve # stdio (default)This exposes 14 mcp__pcq__* tools (resolve_project, validate_run,
describe_run, compare_runs, ...) so agents call pcq directly without
subprocess parsing. See MCP Integration.
Reusable assets:
v4 Direction
v4 clarifies the product boundary:
contract-first workflow, not a 3-tier training API
project-local training code, not built-in production catalogs
contract scripts, not framework adapters
run evidence validation, not recipe ownership
JSON/JSONL facts, not prose parsing
See pcq v4 Direction.
Development
uv run ruff check src/ tests/ scripts/
uv run python -m compileall src/pcq
uv run pytest tests/ -q
bash scripts/release-smoke.shLicense
Apache-2.0.
Available Tools
14 toolsagent_installB
Install pcq agent runtime assets (AGENTS.md / CLAUDE.md managed block, .agents|.claude/skills/pcq/SKILL.md). Optionally write .mcp.json to wire pcq MCP server.
| Name | Required | Description | Default |
|---|---|---|---|
| mcp | No | Also wire .mcp.json with `pcq mcp serve` entry | |
| path | No | . | |
| force | No | ||
| target | No | codex | |
| dry_run | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as side effects, permissions, idempotency, or error conditions. It only describes the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the main action, and every word earns its place. No wasted information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, no output schema, and low schema coverage, the description is incomplete. It lacks behavior details, parameter explanations, and return value information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With only 20% schema coverage, the description should compensate. It explains the 'mcp' parameter (optional MCP wiring) but does not clarify 'path', 'force', 'target', or 'dry_run'. Value added is minimal.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool installs pcq agent runtime assets, listing specific files (AGENTS.md, CLAUDE.md, SKILL.md), and distinguishes it from sibling tools that deal with status, plans, experiments, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for setting up pcq agent assets but does not explicitly state when to use vs alternatives or provide any exclusions. It is implied by the tool's nature.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
agent_statusA
Inspect pcq agent runtime asset status (installed / missing / stale / divergent / unmanaged) without writing. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | . | |
| target | No | codex |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It explicitly states the tool is read-only, which is critical for safety. It also lists the asset statuses it reports, but does not describe output format or any additional constraints (e.g., permissions). Adequate for a simple inspection tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the tool's purpose and key behavioral trait (read-only). No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with two optional parameters, no output schema, and no annotations, the description covers the essential purpose and safety. It could benefit from a brief mention of parameter roles or typical usage, but it is sufficiently complete for simple status checks.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description does not mention the two parameters (path and target) or their meanings. The description adds no value beyond the schema; agents are left to infer from parameter names alone. This is a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool inspects pcq agent runtime asset status, listing specific statuses (installed/missing/stale/divergent/unmanaged) and emphasizing read-only. This distinguishes it from sibling tools like agent_install or apply_plan which modify state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies 'without writing. Read-only,' which clearly indicates when to use the tool (for inspection) and when not to (for modifications). However, it does not explicitly name alternative tools for writing, leaving some inference needed from the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
apply_planA
Apply ExperimentPlan to project (modifies cq.yaml.configs only — never train.py). Provenance recorded under .pcq/plans/.json. Returns rejected envelope with reason='schema_invalid'|'validation_failed' on bad input.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | . | |
| plan | No | Inline ExperimentPlan dict. Minimal example: { "schema_version": 1, "id": "exp-001", "intent": "try larger lr", "base": {"baseline": "gen0"}, "parent_run_id": "run_...", "parent_run_path": "/abs/path/output_gen0", "changes": [ {"op": "set_config", "key": "lr", "value": 0.01} ] } Required: id (non-empty string), changes (non-empty list of {op: 'set_config', key: <str>, value: <any>}). Optional: intent, base, target, parent_run_id, parent_run_path, validation_policy. | |
| plan_file | No | Path to ExperimentPlan JSON file |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully bears transparency. It discloses that the tool modifies only cq.yaml.configs, records provenance under .pcq/plans/, and returns rejected envelopes with specific reasons on invalid input. This covers core behavioral traits, though it omits information about idempotency or whether the operation is reversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences covering action, scope, provenance, and error behavior. Every sentence adds value without redundancy or extraneous detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (nested object, no output schema), the description covers action and error responses but lacks success output details, prerequisites (e.g., project must exist), and integration with sibling tools like init_experiment. It is minimally complete but leaves gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67% (path parameter lacks description). The tool description does not clarify the 'path' parameter's purpose or format, nor does it add semantic information beyond the schema's detailed 'plan' parameter description. The description's mention of error responses does not compensate for missing parameter guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool applies an ExperimentPlan to a project, specifies the scope (modifies cq.yaml.configs only, never train.py), and distinguishes the action from generic 'plan' operations. The verb 'apply' combined with the resource 'ExperimentPlan' and the boundary 'never train.py' provides precise purpose, though it doesn't explicitly differentiate from the sibling 'apply_planset'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description offers no explicit guidance on when to use this tool versus alternatives like apply_planset, run_experiment, or validate_project. It implies usage for single plan application but lacks exclusion criteria or context about prerequisites (e.g., project must be initialized).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
apply_plansetA
Expand ExperimentPlanSet members into N output directories, each with its own cq.yaml + plan provenance. Returns rejected envelope with reason='schema_invalid'|'validation_failed' on bad input.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | . | |
| force | No | ||
| planset | No | Inline ExperimentPlanSet dict. Minimal example: { "schema_version": 1, "id": "sweep-001", "intent": "lr sweep", "parent_run_id": "run_baseline", "plans": [ {"id": "exp-000", "changes": [ {"op": "set_config", "key": "lr", "value": 0.01}]}, {"id": "exp-001", "changes": [ {"op": "set_config", "key": "lr", "value": 0.001}]} ] } Required: id, plans (non-empty, each member is an ExperimentPlan). | |
| planset_file | No | Path to ExperimentPlanSet JSON file | |
| output_pattern | No | Pattern using {i} (zero-based index) and/or {plan_id} | runs/exp{i} |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses creation of directories and error behavior, but lacks details on side effects (overwriting, auth needs, destructive actions) since no annotations exist. Partial transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences covering main action and error scenario. No redundant information; every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks description of successful return value (e.g., list of created directories) and side effect transparency. Given nested objects and no output schema, more detail is needed for full understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds meaningful descriptions for 'planset' (example, required fields), 'planset_file' (path), and 'output_pattern' (pattern format). However, 'path' and 'force' lack extra context beyond defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'Expand' and resource 'ExperimentPlanSet members', clearly stating it creates output directories with config and provenance. It also distinguishes from sibling 'apply_plan' by focusing on plan sets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for expanding plansets but does not explicitly state when to use this over alternatives like 'apply_plan' or under what conditions (e.g., multiple plans). No exclusions or prerequisites mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare_runsA
Diff two RunRecords (or output dirs). Returns metric deltas, config changes, lineage relation, decision_facts. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | Path to first run_record.json or output dir | |
| b | Yes | Path to second run_record.json or output dir |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description states read-only behavior, but lacks details on side effects, permissions, or data limits. This is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence plus 'Read-only' - extremely concise, front-loaded with verb 'Diff', no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lists return values and read-only nature. Without an output schema, this helps. But lacks usage context (e.g., when to diff vs describe). Still fairly complete for a simple diff tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions already matching the tool's 'path to run_record or output dir'. The description adds minimal extra insight beyond grouping them as 'two RunRecords'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool diffs two RunRecords, listing specific return values (metric deltas, config changes, lineage relation, decision_facts). This distinguishes it from siblings like describe_run or lineage_chain.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., describe_run for single runs, lineage_chain for lineage). The description only states 'Read-only' without excluding contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
describe_runA
Return a compact, decision-facts oriented summary of a RunRecord. Includes best/last metrics, validation status, lineage, artifact counts, decision_facts. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| output_dir | No | output |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It states 'Read-only' which covers safety, but it does not disclose whether the tool works on any run, requires a specific run state, or has performance considerations. The listed contents add some value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the core purpose, and avoids unnecessary words. Every sentence contributes meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the minimal input schema (1 optional param) and no output schema, the description covers the output contents but misses contextual details like how the run is identified or the role of output_dir. It is adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, output_dir, is not mentioned in the description. Schema description coverage is 0%, and the description adds no meaning beyond the schema's default value. The agent must guess the parameter's purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns a compact, decision-facts oriented summary of a RunRecord with specific contents (metrics, validation status, lineage, artifact counts, decision_facts). It identifies the resource and the nature of the output, distinguishing it from mutation tools like run_experiment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used to get a summary, but it does not provide explicit guidance on when to use this tool versus alternatives like compare_runs or validate_run. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
finalize_runB
Generate run_record.json + validation_report.json for an output directory. Walks ancestors to find project root if not provided. Writes to output_dir.
| Name | Required | Description | Default |
|---|---|---|---|
| status | No | completed | |
| output_dir | Yes | ||
| project_root | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full weight for behavioral disclosure. It mentions ancestor walking and writing to output_dir, but lacks details on overwrite behavior, directory creation, permissions, or side effects on run state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the primary action and adding a behavioral detail in the second. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description is incomplete. It does not explain what the generated files contain, the impact on the run state, or any prerequisites, leaving significant gaps for a tool that finalizes a run.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% coverage; the description explains output_dir and implicitly project_root, but does not address the status parameter (enum with default). It adds meaning beyond the schema for two parameters but not for all three.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the tool's action: generating run_record.json and validation_report.json into an output directory. It distinguishes from siblings by stating its specific output and behavior of walking ancestors to find project root.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide guidance on when to use this tool versus siblings like validate_run or describe_run. It states what it does but not when it is appropriate or when it is not.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
init_experimentB
Scaffold a CQ-runnable experiment (cq.yaml + train.py contract script, optionally pyproject.toml + agent runtime assets).
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | cq.yaml.name (default: pcq-experiment) | |
| agent | No | none | |
| force | No | ||
| output | No | Project directory to populate | . |
| with_pyproject | No | Generate pyproject.toml with pcq dep (recommended for lockfile_sha256 evidence) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose side effects such as whether existing files are overwritten, required directory state, or permission needs. The `force` parameter hints at destructive behavior but is not explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the tool's purpose and optional elements. It is front-loaded and contains no redundant information, though slightly more structure could improve readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters and no output schema, the description covers the main action and output files but lacks details like behavior when directory exists, overwrite rules, or post-creation steps. This is adequate for a simple scaffold tool but incomplete for complex use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 60%, with descriptions for name, force, output, and with_pyproject. The description adds context about scaffolding files, but the `agent` parameter lacks description both in schema and description, limiting added value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it scaffolds a CQ-runnable experiment, listing specific files (cq.yaml, train.py, optionally pyproject.toml and agent runtime assets). This specific verb+resource combination distinguishes it from siblings like run_experiment or validate_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for initial setup, but provides no explicit when-to-use or when-not-to-use guidance compared to siblings. For example, it does not mention prerequisites or scenarios where validate_project might be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
inspect_projectA
Return project structure, entrypoint kind, contract state, and output evidence. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | Project root to inspect | . |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses the read-only nature but does not mention error conditions, performance implications, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that front-loads the return values ('Return project structure...') followed by 'Read-only.' No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lists what is returned but does not explain the output structure (e.g., format or fields), which would be helpful given no output schema. It is adequate for a simple tool but lacks depth.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%; the parameter 'path' is adequately described in the schema as 'Project root to inspect'. The tool description adds no additional meaning beyond schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns 'project structure, entrypoint kind, contract state, and output evidence' and specifies 'Read-only'. This distinguishes it from sibling tools like validate_project (validation) and resolve_project (resolution).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage via 'Read-only', suggesting safe exploration without side effects, but does not explicitly state when to prefer this tool over siblings or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
lineage_chainC
Walk a RunRecord's parent chain. Returns ordered nodes from this run back to its earliest reachable ancestor. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| max_depth | No | ||
| output_dir | No | output |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions 'Read-only' but does not disclose other behavioral traits such as behavior when max_depth is exceeded, performance characteristics, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two short sentences and no extraneous information. However, it could benefit from more structured formatting or bullet points for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, annotations, and parameter documentation, the description does not fully inform the agent about tool behavior or constraints. For example, it does not address potential cycles in the parent chain or the significance of the output directory.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description does not mention any parameters (max_depth, output_dir) or explain their meaning beyond what the schema provides. This is a significant gap for a tool with 2 parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Walk a RunRecord's parent chain') and output ('Returns ordered nodes from this run back to its earliest reachable ancestor'), distinguishing it from sibling tools that handle other run operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., describe_run or compare_runs). The description only states it is read-only but does not provide context for appropriate use cases or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
resolve_projectA
Resolve cq.yaml + CQ_CONFIG_JSON env into a single ResolvedConfig view. Returns project_root, cq_yaml_path, name, cmd, cfg, declared_metrics, output_dir. Read-only — does not create directories or mutate state.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | Project root path (cwd by default) | . |
| cq_yaml_path | No | Optional explicit cq.yaml path (otherwise discovered) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, but the description explicitly states the tool is read-only and does not create directories or mutate state. It also lists the return fields, providing full behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the purpose and listing outputs and safety in a concise, efficient manner without unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple config resolution tool with no output schema and no annotations, the description covers all necessary aspects: purpose, inputs, returns, and behavioral safety. No critical information is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds context by explaining the overall function (resolving cq.yaml and environment) but does not add per-parameter insights beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Resolve' and identifies the key resources ('cq.yaml + CQ_CONFIG_JSON env'), clearly distinguishing this tool from sibling tools like inspect_project or validate_project by focusing on config resolution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states the tool is read-only and does not mutate state, providing clear context for safe usage. However, it does not mention when to prefer this tool over alternatives like inspect_project or validate_project.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_experimentA
Execute cq.yaml.cmd with auto-wired CQ_CONFIG_JSON env. Captures stdout/stderr to .pcq/run_*.log. For long-running GPU training, prefer the CQ service queue.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | . | |
| config_only | No | Write runtime_cfg.json only, do not exec cmd |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden; it discloses log capture and environment injection but omits return behavior, success/failure signals, and potential side effects like log overwrites.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no fluff, front-loads the main action and alternative guidance efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple tool but lacks explanation of how to retrieve results post-execution and full behavior of 'config_only' mode, which reduces completeness for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no detail about the 'path' parameter (default '.') beyond the schema, and only one parameter has schema description; minimal extra guidance for correct usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool executes a command from cq.yaml.cmd with an auto-wired environment variable and captures logs, distinguishing it from sibling tools like 'agent_install' or 'apply_plan'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises against using for long-running GPU training, instead recommending the CQ service queue, providing clear when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
validate_projectB
Run static and contract validation before execution. Optional inline ExperimentPlan / ExperimentPlanSet validation. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | . | |
| plan | No | Inline ExperimentPlan dict (optional) | |
| planset | No | Inline ExperimentPlanSet dict | |
| plan_file | No | Path to ExperimentPlan JSON file | |
| strictness | No | Validation strictness 0..4 (default: cq.yaml configs.strictness or 2) | |
| planset_file | No | Path to ExperimentPlanSet JSON file |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must carry burden. It states 'Read-only', which is good for a validation tool, but does not describe return format, error behavior, or effects on the project state. Moderate transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with action and resource. Every sentence adds value with no redundancy. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Complex tool with 6 params and no output schema, yet the description lacks details on return values, error handling, or comparison with sibling 'validate_run'. Incomplete for effective agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 83%, so baseline is 3. The description adds context that inline plans are optional and that validation can be applied to ExperimentPlan/ExperimentPlanSet, but does not add significant new meaning beyond what schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it runs 'static and contract validation before execution' with optional inline plans. The verb 'validate' and resource 'project' are clear, and 'Read-only' clarifies it's non-mutating. However, it does not explicitly distinguish from sibling 'validate_run' beyond naming.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Only implies usage 'before execution' but provides no explicit guidance on when to use this tool versus alternatives like 'inspect_project' or 'validate_run'. No exclusions or context for selection among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
validate_runB
Run post-run validation gates (manifest / metrics / run_summary) on a completed output directory. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| output_dir | No | output | |
| strictness | No | ||
| rescan_manifest | No | Ignore manifest entries whose files no longer exist (output_dir reuse / stale lock-in fix) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description declares the tool as read-only, which is valuable behavioral information given no annotations. However, it lacks details on validation behavior (e.g., output format, error handling, consequences of rescan_manifest) and relies partially on the schema for the rescan_manifest description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that convey the core purpose and read-only nature. No extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a validation tool with three parameters and no output schema, the description is incomplete. It omits what the tool returns (e.g., success/failure, error details) and does not elaborate on the validation gates or side effects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With only 33% schema description coverage, the description does not explain any parameters. It adds no meaning beyond the schema, which already has reasonable defaults and one parameter description. The low coverage demands more compensation than provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs post-run validation gates on a completed output directory. It lists the gates (manifest, metrics, run_summary) and implies its role after a run. However, it does not explicitly differentiate from sibling tools like validate_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It mentions 'post-run' but does not specify prerequisites, when not to use, or how it compares to other validation tools in the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
14 tool updates
v4.11.0- Added
agent_install - Added
agent_status - Added
apply_plan - Added
apply_planset - Added
compare_runs - Added
describe_run - Added
finalize_run - Added
init_experiment - Added
inspect_project - Added
lineage_chain - Added
resolve_project - Added
run_experiment - Added
validate_project - Added
validate_run
14 tool updates
v4.10.0- Removed
agent_install - Removed
agent_status - Removed
apply_plan - Removed
apply_planset - Removed
compare_runs - Removed
describe_run - Removed
finalize_run - Removed
init_experiment - Removed
inspect_project - Removed
lineage_chain - Removed
resolve_project - Removed
run_experiment - Removed
validate_project - Removed
validate_run
14 tool updates
v0.1.0- First observed
agent_install - First observed
agent_status - First observed
apply_plan - First observed
apply_planset - First observed
compare_runs - First observed
describe_run - First observed
finalize_run - First observed
init_experiment - First observed
inspect_project - First observed
lineage_chain - First observed
resolve_project - First observed
run_experiment - First observed
validate_project - First observed
validate_run
TDQS
Each tool has a clearly distinct purpose covering installation, status, plan application, run comparison, description, finalization, initialization, project inspection, lineage, resolution, execution, and validation. No two tools overlap in functionality.
All names use lowercase and underscores, but the order varies: some are verb_noun (apply_plan, compare_runs) while others are noun_verb (agent_install) or noun_noun (lineage_chain). This is a minor inconsistency but still readable and predictable.
14 tools is well-scoped for an experiment management server. Each tool covers a specific part of the workflow without being redundant or excessive.
The tool surface covers the core lifecycle: init, validate, run, apply plans, compare, describe, finalize, and inspect. Minor gaps exist, such as no tool to list runs or plans, but the essential operations are present.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Watchdog for unattended AI agents: alerts, evidence checks and a verifiable proof per run.
Reproducible benchmarks and reliability evidence for agent tools.
Cross-agent artifact workspace with provenance across Claude Code, Codex, Cursor, LangGraph.
Machine-readable utilities and datasets for AI agents.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceAn MCP server that enables AI agents to observe and interact with trackio experiment tracking, providing tools for managing ML experiments through natural language.3MIT
- AlicenseBqualityDmaintenanceA minimal integration layer exposing PyTorch Lightning via a structured, machine-readable API for tools, agents, and orchestration systems.65Apache 2.0
- AlicenseAqualityBmaintenanceA scientific experiment log MCP server for AI agents that stores predictions, causal claims, and verdicts, enabling queryable causal maps and calibration of intuition over diagnostics.17MIT
- AlicenseNot gradedqualityCmaintenanceEnables step-debugging, deterministic replay, and signed audit evidence for AI agents, compliant with EU AI Act.MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/playidea-lab/pcq'
If you have feedback or need assistance with the MCP directory API, please join our Discord server