Skip to main content
Glama
enesjakozh
atla-ai

Atla

Official
by atla-ai
OverviewSchemaRelated ServersScoreDiscussions
Python
Remote

Atla MCP 서버

LLM이 Atla API와 상호 작용하여 최첨단 LLMJ 평가를 수행할 수 있는 표준화된 인터페이스를 제공하는 MCP 서버 구현입니다.

Atla에 대한 자세한 내용은 여기를 참조하세요. 모델 컨텍스트 프로토콜에 대한 자세한 내용은 여기를 참조하세요.

사용 가능한 도구

  • evaluate_llm_response : 주어진 평가 기준을 사용하여 프롬프트에 대한 LLM의 응답을 평가합니다. 이 함수는 Atla 평가 모델을 기반으로 모델의 응답 점수와 모델 응답에 대한 피드백을 포함하는 텍스트 비평을 포함하는 사전을 반환합니다.

  • evaluate_llm_response_on_multiple_criteria : LLM의 프롬프트 응답을 여러 평가 기준에 따라 평가합니다. 이 함수는 Atla 평가 모델을 기반으로 각 기준에 대한 평가 점수와 비평을 포함하는 사전 목록을 반환합니다.

Related MCP server: Ollama MCP Server

용법

MCP 서버를 사용하려면 Atla API 키가 필요합니다. 기존 API 키는 여기 에서 확인하거나 새 API 키를 생성하세요.

설치

Python 환경을 관리하려면 uv 사용하는 것이 좋습니다. 설치 지침은 여기를 참조하세요.

수동으로 서버 실행

uv 설치하고 Atla API 키를 받으면 uvx ( uv 에서 제공)를 사용하여 MCP 서버를 수동으로 실행할 수 있습니다.

지엑스피1

서버에 연결

문제가 있거나 다른 클라이언트 연결에 도움이 필요하신가요? 언제든지 문제를 제기하시거나 저희에게 문의해 주세요 !

OpenAI 에이전트 SDK

MCP 서버와 함께 OpenAI Agents SDK를 사용하는 방법에 대한 자세한 내용은 공식 문서를 참조하세요.

  1. OpenAI 에이전트 SDK를 설치하세요:

pip install openai-agents

  1. OpenAI Agents SDK를 사용하여 서버에 연결합니다.

import os

from agents import Agent
from agents.mcp import MCPServerStdio

async with MCPServerStdio(
        params={
            "command": "uvx",
            "args": ["atla-mcp-server"],
            "env": {"ATLA_API_KEY": os.environ.get("ATLA_API_KEY")}
        }
    ) as atla_mcp_server:
    ...

클로드 데스크탑

Claude Desktop에서 MCP 서버를 구성하는 방법에 대한 자세한 내용은 공식 MCP 빠른 시작 가이드를 참조하세요.

  1. claude_desktop_config.json 파일에 다음을 추가하세요.

{
  "mcpServers": {
    "atla-mcp-server": {
      "command": "uvx",
      "args": ["atla-mcp-server"],
      "env": {
        "ATLA_API_KEY": "<your-atla-api-key>"
      }
    }
  }
}

  1. 변경 사항을 적용하려면 Claude Desktop을 다시 시작하세요 .

이제 사용 가능한 MCP 도구 목록에서 atla-mcp-server 의 옵션을 볼 수 있습니다.

커서

Cursor에서 MCP 서버를 구성하는 방법에 대한 자세한 내용은 공식 문서를 참조하세요.

  1. .cursor/mcp.json 파일에 다음을 추가하세요.

{
  "mcpServers": {
    "atla-mcp-server": {
      "command": "uvx",
      "args": ["atla-mcp-server"],
      "env": {
        "ATLA_API_KEY": "<your-atla-api-key>"
      }
    }
  }
}

이제 사용 가능한 MCP 서버 목록에 atla-mcp-server 표시됩니다.

기여하다

기여를 환영합니다! 자세한 내용은 CONTRIBUTING.md 파일을 참조하세요.

특허

이 프로젝트는 MIT 라이선스에 따라 라이선스가 부여됩니다. 자세한 내용은 라이선스 파일을 참조하세요.

Available Tools

2 tools
evaluate_llm_responseA

Evaluate an LLM's response to a prompt using a given evaluation criteria.

This function uses an Atla evaluation model under the hood to return a dictionary
containing a score for the model's response and a textual critique containing
feedback on the model's response.

Returns:
    dict[str, str]: A dictionary containing the evaluation score and critique, in
        the format `{"score": <score>, "critique": <critique>}`.
ParametersJSON Schema
NameRequiredDescriptionDefault
evaluation_criteriaYesThe specific criteria or instructions on which to evaluate the model output. A good evaluation criteria should provide the model with: (1) a description of the evaluation task, (2) a rubric of possible scores and their corresponding criteria, and (3) a final sentence clarifying expected score format. A good evaluation criteria should also be specific and focus on a single aspect of the model output. To evaluate a model's response on multiple criteria, use the `evaluate_llm_response_on_multiple_criteria` function and create individual criteria for each relevant evaluation task. Typical rubrics score responses either on a Likert scale from 1 to 5 or binary scale with scores of 'Yes' or 'No', depending on the specific evaluation task.
llm_promptYesThe prompt given to an LLM to generate the `llm_response` to be evaluated.
llm_responseYesThe output generated by the model in response to the `llm_prompt`, which needs to be evaluated.
expected_llm_outputNoA reference or ideal answer to compare against the `llm_response`. This is useful in cases where a specific output is expected from the model. Defaults to None.
llm_contextNoAdditional context or information provided to the model during generation. This is useful in cases where the model was provided with additional information that is not part of the `llm_prompt` or `expected_llm_output` (e.g., a RAG retrieval context). Defaults to None.
model_idNoThe Atla model ID to use for evaluation. `atla-selene` is the flagship Atla model, optimized for the highest all-round performance. `atla-selene-mini` is a compact model that is generally faster and cheaper to run. Defaults to `atla-selene`.atla-selene

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that the tool uses an Atla evaluation model under the hood and returns a dictionary. However, with no annotations provided, the description carries full burden for behavioral traits. It does not mention potential rate limits, cost implications, or authentication requirements. The description is adequate but lacks deeper operational transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with three sentences covering purpose, internal mechanism, and output format. It is front-loaded and well-structured. A minor deduction because the return type is specified in a separate block rather than integrated into the prose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of 6 parameters (3 required) and the presence of an output schema (described in the return type), the description provides a complete overview. The parameter-level descriptions are very detailed, covering formatting and examples. The sibling tool is referenced, ensuring completeness around alternatives. Slightly lacking in explaining edge cases or error handling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the input schema already documents all parameters thoroughly. The description does not add new meaning beyond what the schema provides; it mainly reiterates the return format. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: evaluating an LLM response using given criteria. It specifies that it uses an Atla evaluation model internally and returns a dictionary with score and critique. It also distinguishes from the sibling tool by referencing evaluate_llm_response_on_multiple_criteria in the evaluation_criteria parameter description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The evaluation_criteria parameter description explicitly instructs to use the sibling tool for multiple criteria, providing clear guidance on when to use this tool versus the alternative. Additionally, the parameter descriptions include examples and detailed instructions on crafting evaluation criteria, which serves as usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

evaluate_llm_response_on_multiple_criteriaA

Evaluate an LLM's response to a prompt across multiple evaluation criteria.

This function uses an Atla evaluation model under the hood to return a list of
dictionaries, each containing an evaluation score and critique for a given
criteria.

Returns:
    list[dict[str, str]]: A list of dictionaries containing the evaluation score
        and critique, in the format `{"score": <score>, "critique": <critique>}`.
        The order of the dictionaries in the list will match the order of the
        criteria in the `evaluation_criteria_list` argument.
ParametersJSON Schema
NameRequiredDescriptionDefault
evaluation_criteria_listYes
llm_promptYesThe prompt given to an LLM to generate the `llm_response` to be evaluated.
llm_responseYesThe output generated by the model in response to the `llm_prompt`, which needs to be evaluated.
expected_llm_outputNoA reference or ideal answer to compare against the `llm_response`. This is useful in cases where a specific output is expected from the model. Defaults to None.
llm_contextNoAdditional context or information provided to the model during generation. This is useful in cases where the model was provided with additional information that is not part of the `llm_prompt` or `expected_llm_output` (e.g., a RAG retrieval context). Defaults to None.
model_idNoThe Atla model ID to use for evaluation. `atla-selene` is the flagship Atla model, optimized for the highest all-round performance. `atla-selene-mini` is a compact model that is generally faster and cheaper to run. Defaults to `atla-selene`.atla-selene

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description adds some behavioral context (uses Atla model under the hood, returns list of dicts) but does not disclose side effects, cost, failure modes, or restrictions. It adds value beyond annotations but falls short of full transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (two sentences plus a return format note) and front-loaded with purpose. Every sentence adds value; no obvious fluff. Minor improvement possible by incorporating usage guidance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (6 parameters, output schema exists), the description provides basic purpose and return format but lacks usage guidelines, behavioral warnings, and parameter elaboration. It is incomplete for fully understanding when and how to use the tool effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is high (83%), and the description does not add new meaning to individual parameters beyond the schema. It explains the return format, which aids understanding of the output but does not directly enhance parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool evaluates an LLM response across multiple criteria using an Atla model and returns a list of dictionaries with score and critique. It distinguishes from the sibling tool 'evaluate_llm_response' by emphasizing 'multiple evaluation criteria', making the purpose and differentiation explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for multiple criteria but does not explicitly state when to use this tool versus the single-criterion sibling. No when-not-to-use or alternative suggestions are provided, leaving guidance implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 2 tool updatesv1.0.0
    • Addedevaluate_llm_response
    • Addedevaluate_llm_response_on_multiple_criteria
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A3.9/5.0
Disambiguation5/5

The two tools have clearly distinct purposes: one evaluates a single criterion, the other evaluates multiple criteria. Their names and descriptions make the difference unambiguous.

Naming Consistency5/5

Both tool names follow a consistent verb_noun_qualifier pattern, starting with 'evaluate_llm_response' and differentiating with '_on_multiple_criteria'. No mixing of conventions.

Tool Count3/5

With only 2 tools, the server feels thin. While the tools cover the core evaluation functionality, a typical well-scoped server has 3-15 tools, making this borderline insufficient.

Completeness3/5

The tools provide basic evaluation for single and multiple criteria, but lack supporting tools such as managing criteria, listing models, or retrieving history. The surface is minimal and may leave agents with limited options.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/atla-ai/atla-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server