Skip to main content
Glama

Report Effectiveness

oi.effectiveness.report

Report what happened after an Oi-assisted task completed. Use this after calling an Oi Context, Workflow, Skill, Connection, or Guardrail tool when you know whether the result was accepted, retried, saved, copied, completed, or rejected. Prefer usageEventId from the prior Oi tool result; traceId may also be used when available. Do not send raw prompt text; send a short redacted taskSummary only when useful.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
actionsNo
outcomeYes
traceIdNoOptional Oi effectiveness trace id.
taskHashNoOptional client-side task hash.
confidenceNo
retryCountNoNumber of retries or follow-up attempts needed before a useful result.
taskSummaryNoOptional short redacted task summary. Do not include secrets or raw prompt text.
baselineModeNo
usageEventIdNoUsage event id returned by a prior Oi tool call.
userFeedbackNo
feedbackReasonNoShort feedback reason.

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
actionsNo
outcomeYes
traceIdYes
confidenceNo
retryCountNo
baselineModeNo
evidenceTierYes
userFeedbackNo
usageEventIdsYes
completedWorkflowNo
preventedRiskCountNo
acceptedWithoutRetryYes
estimatedMinutesSavedYes

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. First observed

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are sparse (only non-read-only and non-destructive), so the description carries the transparency burden. It adds genuinely useful behavioral guidance — preferring usageEventId over traceId, and requiring redacted summaries — but it never states the call's side effects, whether it is a one-way fire-and-forget telemetry send, or anything about failure or rate-limit behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences with the purpose front-loaded and zero filler. Every sentence earns its place, though there is minor redundancy between the opening purpose and the second sentence which restates the after-task-completion timing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter tool with sparse annotations, the description covers core usage timing, ID selection, and content safety well. But it gives no guidance on which optional parameter groups (outcome vs actions vs userFeedback vs confidence) are meant to be sent together, and it never explains what the system does with the report. The output schema mitigates the return-value gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With schema description coverage at 55%, the description adds real value for usageEventId (preference), traceId (fallback), and taskSummary (redaction rule). However, its outcome list ('accepted, retried, saved, copied, completed, or rejected') blends outcome enum values with action enum values, which could mislead an agent into sending invalid values to the outcome field, and baselineMode/confidence get no clarification beyond their enums.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource — 'Report what happened after an Oi-assisted task completed' — and enumerates the outcome states it covers (accepted, retried, saved, copied, completed, rejected). This clearly differentiates it from the closest sibling oi.usage.report, which is about usage metrics rather than task effectiveness.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to call it ('Use this after calling an Oi Context, Workflow, Skill, Connection, or Guardrail tool when you know whether the result was accepted...') and imposes content restrictions ('Do not send raw prompt text'). It falls short of 5 because it never names alternatives or states when NOT to use it in favor of oi.usage.report or the feedback tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A3.5/5.0
Disambiguation4/5

Resource prefixes (contexts, skills, guardrails, workflows, connections) make most tools clearly distinct, and parallel lifecycle verbs are scoped by resource name. The main ambiguity is within the guardrails publish/release/unpublish lifecycle and between brain.save-feedback, contexts.save-draft-feedback, and the two report tools, though detailed descriptions mostly resolve it.

Naming Consistency4/5

Tools overwhelmingly follow an oi.<resource>.<verb> snake_case pattern, with create/get/list/update/use repeated consistently across resource types. Deviations include resource-less oi.recommend, noun-style oi.auth.whoami, and inconsistent release handling (separate guardrails.release vs action=release on contexts/skills update).

Tool Count2/5

38 tools is too many for a single server, mainly because the same lifecycle pattern is repeated across Contexts, Skills, Guardrails, and Workflows. Each tool may be individually justifiable, but the set feels bloated and could benefit from consolidation or splitting into per-resource servers.

Completeness3/5

CRUD coverage is uneven: Guardrails have create/update/delete/list/get plus publish/unpublish/release, while Contexts, Skills, and Workflows lack any delete or archive tool, and Brain has only save-feedback with no read/update/delete path. Connections and auth are read/use-only, which may be intentional but leaves management actions absent.