Skip to main content
Glama

OptimToken

by io.github.OptimNow

Compare LLM Models

compare-llm-models
Read-only

Browse and filter the whole LLM catalogue and get back a ranked table: price, quality (ELO), efficiency and capabilities. Use this when the user wants to SEE THE FIELD — 'show me models under $1/1M', 'which providers have vision models', 'list open-weight models above ELO 1300'. For a single PICK under a budget use recommend-llm-model; to weigh 2-4 NAMED models against each other use compare-models-side-by-side. Prices come from optimtoken.optimnow.io where reachable; the response's provenance says which tier served them and whether they are vendor-verified. Filter by provider, price tier (category), openness, capability, price range, or minimum ELO score. Optionally enrich with business metrics for a use case. Price tier and openness are independent: a model can be Frontier-priced and open-weight at once. Reports both list-price cost and the optimized cost achievable with prompt caching and the batch API. IMPORTANT: Report all prices, costs, and scores EXACTLY as returned. Do NOT add commentary, opinions, or recommendations beyond what the data shows. Present the results as a table and let the user draw conclusions.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
limitNoMax models to return (default: 15)
minEloNoMinimum Chatbot Arena ELO score. Typical range 1000-1500; ~1400 is roughly frontier-class. Models with no ELO score never satisfy this.
categoryNoPrice tier, matched by exact equality (case-insensitive): Frontier, Mid-tier, Budget, Image. This is cost only — self-hostability is the separate `openness` axis.
opennessNoFilter by self-hostability, derived from the licence: Open source, Open weights, Proprietary, Unknown
providerNoFilter by provider name (e.g. 'OpenAI', 'Anthropic', 'Google')
capabilityNoCapability, matched by exact equality (case-insensitive): Text, Vision, Code, Reasoning, Agents, Image Gen, Audio. Any other string returns zero matches.
volumePresetNoMonthly request volume: 10k, 100k, or 1m. Default: 100k
maxInputPriceNoMax input price per 1M tokens in USD
useCasePresetNoWorkload shape, which sets tokens per request: supportTicket (1.5k in / 500 out), knowledgeQA (2k / 800), meetingSummary (10k / 1.2k, batch-eligible), marketingContent (2.5k / 1.8k), codingTask (3k / 2k), invoiceProcessing (1.5k / 600, batch-eligible), callSummary (2k / 700, batch-eligible), agentWorkflow (6k / 3k). Default: supportTicket
maxOutputPriceNoMax output price per 1M tokens in USD

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
errorNo
modelsYes
sourceYes
eloAsOfYes
dataAsOfNo
provenanceNo
catalogSizeYes
finopsBadgeNo
volumeLabelYes
useCaseLabelYes
matchingCountYes

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. Changed10 schema fields changed
    • changedInput schema / properties / capability / description
      Previous value: -"Filter by capability: Text, Vision, Code, Reasoning, Agents, Image Gen, Audio"New value: +"Capability, matched by exact equality (case-insensitive): Text, Vision, Code, Reasoning, Agents, Image Gen, Audio. Any other string returns zero matches."
    • changedInput schema / properties / category / description
      Previous value: -"Filter by price tier: Frontier, Mid-tier, Budget, Image"New value: +"Price tier, matched by exact equality (case-insensitive): Frontier, Mid-tier, Budget, Image. This is cost only — self-hostability is the separate `openness` axis."
    • changedInput schema / properties / minElo / description
      Previous value: -"Minimum ELO score (quality benchmark from Chatbot Arena)"New value: +"Minimum Chatbot Arena ELO score. Typical range 1000-1500; ~1400 is roughly frontier-class. Models with no ELO score never satisfy this."
    • addedInput schema / properties / minElo / exclusiveMinimum
      Added value: +0
    • changedInput schema / properties / useCasePreset / description
      Previous value: -"Use case for cost estimation. Default: supportTicket"New value: +"Workload shape, which sets tokens per request: supportTicket (1.5k in / 500 out), knowledgeQA (2k / 800), meetingSummary (10k / 1.2k, batch-eligible), marketingContent (2.5k / 1.8k), codingTask (3k / 2k), invoiceProcessing (1.5k / 600, batch-eligible), callSummary (2k / 700, batch-eligible), agentWorkflow (6k / 3k). Default: supportTicket"
    • addedOutput schema / properties / models / items / properties / batchApplied
      Added value: +{
+  "type": "boolean"
+}
    • addedOutput schema / properties / models / items / properties / batchEligible
      Added value: +{
+  "type": "boolean"
+}
    • addedOutput schema / properties / models / items / properties / cacheApplied
      Added value: +{
+  "type": "boolean"
+}
    • addedOutput schema / properties / models / items / properties / cacheEligible
      Added value: +{
+  "type": "boolean"
+}
    • changedOutput schema / properties / models / items / required
      Previous value: -[
-  "provider",
-  "model",
-  "inputPricePer1M",
-  "outputPricePer1M",
-  "contextWindow",
-  "category",
-  "capabilities",
-  "openness",
-  "efficiencyScore",
-  "useCaseCost",
-  "optimizedUseCaseCost",
-  "monthlyBudget",
-  "optimizedMonthlyBudget",
-  "volatilityRisk",
-  "isFinOpsFriendly"
-]New value: +[
+  "provider",
+  "model",
+  "inputPricePer1M",
+  "outputPricePer1M",
+  "contextWindow",
+  "category",
+  "capabilities",
+  "openness",
+  "efficiencyScore",
+  "useCaseCost",
+  "optimizedUseCaseCost",
+  "monthlyBudget",
+  "optimizedMonthlyBudget",
+  "volatilityRisk",
+  "isFinOpsFriendly",
+  "batchEligible",
+  "cacheEligible",
+  "batchApplied",
+  "cacheApplied"
+]
  2. Changed2 schema fields changed
    • changedOutput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • addedOutput schema / properties / provenance
      Added value: +{
+  "additionalProperties": false,
+  "properties": {
+    "catalogTotal": {
+      "type": "number"
+    },
+    "dataAsOf": {
+      "type": "string"
+    },
+    "eloAsOf": {
+      "type": "string"
+    },
+    "label": {
+      "type": "string"
+    },
+    "notice": {
+      "type": "string"
+    },
+    "pricesVerified": {
+      "type": "boolean"
+    },
+    "source": {
+      "type": "string"
+    },
+    "tier": {
+      "type": "number"
+    },
+    "upstreamSchemaVersion": {
+      "type": "string"
+    },
+    "upstreamSource": {
+      "type": "string"
+    },
+    "upstreamTimestamp": {
+      "type": "string"
+    }
+  },
+  "required": [
+    "tier",
+    "source",
+    "label",
+    "pricesVerified",
+    "eloAsOf"
+  ],
+  "type": "object"
+}
  3. Changed2 schema fields changed
    • changedOutput schema / $schema
      Previous value: -"https://json-schema.org/draft/2020-12/schema"New value: +"http://json-schema.org/draft-07/schema#"
    • removedOutput schema / properties / provenance
      Removed value: -{
-  "additionalProperties": false,
-  "properties": {
-    "catalogTotal": {
-      "type": "number"
-    },
-    "dataAsOf": {
-      "type": "string"
-    },
-    "eloAsOf": {
-      "type": "string"
-    },
-    "label": {
-      "type": "string"
-    },
-    "notice": {
-      "type": "string"
-    },
-    "pricesVerified": {
-      "type": "boolean"
-    },
-    "source": {
-      "type": "string"
-    },
-    "tier": {
-      "type": "number"
-    },
-    "upstreamSchemaVersion": {
-      "type": "string"
-    },
-    "upstreamSource": {
-      "type": "string"
-    },
-    "upstreamTimestamp": {
-      "type": "string"
-    }
-  },
-  "required": [
-    "tier",
-    "source",
-    "label",
-    "pricesVerified",
-    "eloAsOf"
-  ],
-  "type": "object"
-}
  4. Changed2 schema fields changed
    • changedOutput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • addedOutput schema / properties / provenance
      Added value: +{
+  "additionalProperties": false,
+  "properties": {
+    "catalogTotal": {
+      "type": "number"
+    },
+    "dataAsOf": {
+      "type": "string"
+    },
+    "eloAsOf": {
+      "type": "string"
+    },
+    "label": {
+      "type": "string"
+    },
+    "notice": {
+      "type": "string"
+    },
+    "pricesVerified": {
+      "type": "boolean"
+    },
+    "source": {
+      "type": "string"
+    },
+    "tier": {
+      "type": "number"
+    },
+    "upstreamSchemaVersion": {
+      "type": "string"
+    },
+    "upstreamSource": {
+      "type": "string"
+    },
+    "upstreamTimestamp": {
+      "type": "string"
+    }
+  },
+  "required": [
+    "tier",
+    "source",
+    "label",
+    "pricesVerified",
+    "eloAsOf"
+  ],
+  "type": "object"
+}
  5. First observed

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only and open-world hints. The description adds crucial behavioral context: data source (optimtoken.optimnow.io), provenance tier, vendor verification, and an explicit instruction to report values exactly as returned without added commentary. This goes beyond annotations and fully informs the agent of expected behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description covers all necessary aspects in about 200 words. It is front-loaded with the core purpose and usage guidelines, then addresses data provenance, filtering options, and agent behavior. While thorough, a slightly tighter structure could improve conciseness without losing content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (10 parameters, 3 enums, output schema exists), the description comprehensively addresses the purpose, usage scenarios, data sources, agent instructions, and key interactions between filters. It leaves no critical gaps for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the baseline is 3. The tool description adds value by summarizing filterable dimensions and clarifying that price tier and openness are independent axes (e.g., Frontier-priced and open-weight). This aids correct parameter usage without duplicating schema details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the tool browses and filters an LLM catalogue to return a ranked table with price, ELO, efficiency, and capabilities. It distinguishes itself from sibling tools by naming 'recommend-llm-model' for a single pick under budget and 'compare-models-side-by-side' for weighing 2-4 named models, making the purpose unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly specifies when to use this tool ('when the user wants to SEE THE FIELD') with concrete example queries, and explicitly tells when not to use it by directing to alternative tools for single picks or side-by-side comparisons. This provides excellent guidance for correct selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All OptimToken toolsInspect
Try in Browser
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.7/5.0
Disambiguation5/5

Each tool serves a clearly distinct purpose: cloud compute pricing, LLM catalogue browsing, side-by-side model comparison, custom cost estimation, and model recommendation. The descriptions provide explicit guidance on when to use each tool, eliminating ambiguity.

Naming Consistency5/5

All tool names follow a consistent verb-noun pattern with hyphens (e.g., compare-compute-pricing, estimate-llm-cost). The verbs are imperative and the nouns clearly describe the object, forming a predictable and readable set.

Tool Count5/5

5 tools is well-scoped for the domain of cloud compute and LLM cost comparison. Each tool earns its place by covering a distinct workflow (browsing, comparing specific models, custom costing, recommendation).

Completeness5/5

The tool surface covers the full spectrum of user needs: compute pricing across providers, LLM catalogue exploration, side-by-side comparisons, arbitrary cost estimation, and recommendation with constraints. There are no obvious gaps for the stated purpose of cost analysis.

Resources