Skip to main content
Glama
bvm-arjun108

GitHub Models Helper

by bvm-arjun108
OverviewSchemaRelated ServersScoreDiscussions
TypeScript
Remote

MCP TypeScript GitHub Models Helper

TypeScript MCP server that integrates with GitHub Models to list available models and compare responses across models for the same prompt. This README includes full requirements, project description, build steps, and testing flow.

Project requirements

  • Node.js 18+

  • GitHub account with access to GitHub Models

  • GitHub Personal Access Token (PAT) with GitHub Models access

  • MCP Inspector (runs via npx)

Related MCP server: Model Hub MCP

Project description

This server exposes three MCP primitives:

  • Resource: models://available (lists available models)

  • Tool: compare_models (compares model responses to the same prompt)

  • Prompt: compare_models_prompt (builds a reusable comparison prompt)

Project structure

mcp-typescript-github-models-helper/
  src/
    githubModels.ts
    index.ts
  .env.template
  .gitignore
  package.json
  tsconfig.json

Step-by-step setup

  1. Install dependencies

npm install

  1. Create .env from the template

cp .env.template .env

  1. Add your GitHub Models token

GITHUB_TOKEN=your_github_token_here

Optional: override the API base URL in .env:

GITHUB_MODELS_API_BASE=https://models.inference.ai.azure.com

Step-by-step implementation (what the code does)

  1. src/githubModels.ts

    • Fetches available models with GET /models

    • Compares models with POST /chat/completions

    • Uses GITHUB_TOKEN from .env

  2. src/index.ts

    • Creates the MCP server

    • Registers the models://available resource

    • Registers the compare_models tool

    • Registers the compare_models_prompt prompt

    • Connects to STDIO transport

Run (development)

npm run dev

Build and run (production-style)

npm run build
npm start

Step-by-step testing with MCP Inspector

  1. Build the server

npm run build

  1. Start the Inspector

npx @modelcontextprotocol/inspector node dist/index.js

  1. Open the URL printed by the Inspector and click Connect

  2. Test resources

    • Resources tab -> models://available

  3. Test tools

    • Tools tab -> compare_models

  4. Test prompts

    • Prompts tab -> compare_models_prompt

Claude Desktop configuration

Add this to your claude_desktop_config.json and update the paths:

{
  "mcpServers": {
    "GitHub Models Comparison": {
      "command": "node",
      "args": [
        "/absolute/path/to/mcp-typescript-github-models-helper/dist/index.js"
      ],
      "env": {
        "GITHUB_TOKEN": "your_github_personal_access_token"
      }
    }
  }
}

Note: This is not secure for production. Use proper secrets management and authentication when publishing MCP servers.

Troubleshooting

  • Cannot find name 'process': run npm install --save-dev @types/node.

  • 401/403 from GitHub Models: verify the PAT has GitHub Models access.

  • Inspector connection errors: rebuild and re-run npx @modelcontextprotocol/inspector node dist/index.js.

Available Tools

1 tool
compare_modelsB

Compare responses from different models for the same prompt

ParametersJSON Schema
NameRequiredDescriptionDefault
modelsNoList of model IDs to compare
promptYesThe prompt to send to all models

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the action without telling whether the call is synchronous, how results are presented, whether it has side effects (e.g., costs, rate limits), or what happens if a model fails. This is a significant gap for a tool that compares models.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence that starts with the verb 'Compare'. It is concise, front-loaded, and contains no filler words. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has modest complexity (2 params, no output schema). The description states the core purpose, but without annotations or an output schema it lacks detail on return values and behavioral context. It is a bare minimum viable description, but not rich enough for an agent to fully anticipate the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents both 'prompt' and 'models'. The description adds a minimal semantic hint ('same prompt') but doesn't explain nuances like whether 'models' is an ordered list or how defaults behave. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Compare') and the resource ('responses from different models') along with the context ('same prompt'). However, it doesn't specify what 'compare' means (e.g., side-by-side output, diff, metrics) and there are no sibling tools to differentiate from, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool, preconditions, or alternatives. The description is purely functional and doesn't mention any exclusions or scenarios where another tool would be preferred. Since the schema shows it requires a prompt, but that's structural not usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 1 tool updatev0.1.0
    • First observedcompare_models
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

B3.2/5.0
Disambiguation5/5

With only one tool, there is no ambiguity or overlap with other tools. The purpose of comparing models is clearly described.

Naming Consistency5/5

The tool name follows a standard verb_noun pattern ('compare_models'). With a single tool, there is no inconsistency to evaluate.

Tool Count2/5

A single tool feels too few for a server named 'GitHub Models Helper', which implies a broader set of utilities. The scope appears overly narrow.

Completeness2/5

The server only provides a model comparison tool, leaving obvious gaps such as listing available models or retrieving model details. This significantly limits helper functionality.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • MCP Prompt Tester

    rt96-hub
    A
    license
    Not graded
    quality
    C
    maintenance
    An MCP server that allows agents to test and compare LLM prompts across OpenAI and Anthropic models, supporting single tests, side-by-side comparisons, and multi-turn conversations.
    MIT

  • Model Hub MCP

    akiojin
    A
    license
    B
    quality
    D
    maintenance
    An MCP server that retrieves and provides AI model information from OpenAI, Anthropic, and Google through a unified interface.
    3
    22
    MIT

  • HydraMCP

    Pickle-Pixel
    A
    license
    A
    quality
    C
    maintenance
    An MCP server that enables users to query, compare, and synthesize responses from multiple local and cloud LLMs simultaneously using existing subscriptions. It provides tools for parallel model evaluation, consensus polling with an LLM-as-judge, and response synthesis across different model providers.
    8
    15
    15
    MIT

  • OpenRouter MCP Server

    lumishoang
    F
    license
    A
    quality
    C
    maintenance
    An MCP server for discovering and querying over 300 AI models available on OpenRouter. It enables users to list, search, filter, compare, and get detailed information about models with pricing, context limits, and capabilities.
    5
    1
    -

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/bvm-arjun108/mcp-typescript-github-models-helper'

If you have feedback or need assistance with the MCP directory API, please join our Discord server