Skip to main content
Glama
colbyw5

phenoforge

by colbyw5
OverviewSchemaRelated ServersScoreDiscussions
Local

phenoforge

Semantic value set assembly for clinical cohort definitions, over MCP.

Status: early prototype. The vocabulary loader, hierarchy expansion, hybrid (BM25 + dense) search, curated OHDSI Phenotype Library matching, an eval harness scoring all of it against curated ground truth, a thin MCP server exposing them all, and a LangGraph agent that decomposes a population description, checks curated first, and pauses for human confirmation before including anything generated — all work end-to-end against a real Athena download. See Roadmap.

Setup

Requires your own OHDSI Athena bulk download — vocabulary content carries its own license terms, so it can't be bundled with this repo.

  1. Create a free account at athena.ohdsi.org.

  2. Under Search, select at least ICD10CM and SNOMED as vocabularies, then click Download. SNOMED is used locally only — to resolve curated-set matching (find_curated_definition) — and is never shipped or exposed through any tool surface.

  3. Unzip the download into data/athena/ (this directory is gitignored — nothing under data/ is ever committed or shipped). It contains OMOP CDM vocabulary tables (CONCEPT.csv, CONCEPT_RELATIONSHIP.csv, etc.) — Athena ships vocabulary content pre-shaped as OMOP, so no separate mapping step is needed.

uv sync
python scripts/load_vocab.py data/athena --output data/vocab.duckdb
phenoforge-mcp  # stdio MCP server; add to Claude Desktop's mcpServers config to try it

Optional: curated phenotype library

find_curated_definition needs a small local cache of OHDSI Phenotype Library cohort definitions. Like the Athena download, this content carries its own terms — the OHDSI PhenotypeLibrary GitHub repo has no confirmed LICENSE file (its R package DESCRIPTION claims Apache, but that's a manifest claim, not a verified grant for the cohort content itself) — so it's fetched into a gitignored local directory, never committed to this repo. Without this step, find_curated_definition still runs and reports why nothing matched.

python scripts/fetch_phenotype_library.py  # writes data/phenotype_library/

Bundles 8 hand-picked, diabetes/kidney-relevant cohorts (not the full library): Type 2/Type 1/ gestational diabetes, diabetic ketoacidosis, retinopathy, and chronic kidney disease.

Optional: dense (semantic) search

search_concepts fuses lexical and semantic matching when a dense index has been built; without one, it falls back to lexical-only search automatically.

python scripts/build_index.py  # writes data/concept_index.lance; downloads BioLORD-2023 on first run

Optional: eval harness

Scores each retrieval method (BM25, dense, hybrid, hierarchy expansion) against the curated demo cohorts as ground truth — hierarchical distance-weighted scoring, set-level coverage, and an over-inclusion penalty (partial credit for near-misses under the same hierarchy parent, not exact-match recall). Requires the phenotype library fetch step above; dense/hybrid scoring also needs the built index.

python scripts/run_eval.py                                 # bm25 + expand_descendants
python scripts/run_eval.py --index data/concept_index.lance # + dense + hybrid

Optional: the agent

The LangGraph agent decomposes a plain-English population description into seed terms, checks each against the curated phenotype library first, and pauses on the command line for you to accept or reject any generated (unverified) candidates before they're included. Requires an Anthropic API key (decomposition is a real Claude call) and the agent extra. scripts/run_agent.py loads a local .env automatically (gitignored — never commit real keys), or export the variable directly.

uv sync --extra agent
echo 'ANTHROPIC_API_KEY=...' > .env   # or: export ANTHROPIC_API_KEY=...
python scripts/run_agent.py "adults with type 2 diabetes and diabetic nephropathy"
python scripts/run_agent.py "..." --index data/concept_index.lance  # + dense retrieval for generated candidates

Interactive exploration

Both notebooks are exploration only, never pushed to production — reusable logic stays in src/phenoforge/.

  • notebooks/explore.ipynb calls the engine directly (no MCP transport) against your real built data/vocab.duckdb.

  • notebooks/evaluate.ipynb runs the eval harness and walks through the metrics with explanatory text, a method-comparison chart, and a sortable per-cohort results table.

uv sync --extra dev
jupyter lab notebooks/explore.ipynb

Related MCP server: medterms-mcp

What it does

Turns a plain-English patient population description into a defensible set of ICD-10-CM codes, where every code carries provenance — whether it came from a peer-reviewed phenotype definition, from hierarchy expansion, or from semantic retrieval that a human should check.

"adults with type 2 diabetes and diabetic nephropathy"
  → decomposes into seed clinical terms
  → checks OHDSI Phenotype Library for a validated definition for each
  → falls back to hybrid retrieval for terms with no curated match,
    pausing for human confirmation before including anything generated
  → returns a ConceptSet with per-code provenance and citations

Run it for real: python scripts/run_agent.py "adults with type 2 diabetes and diabetic nephropathy" (see Setup).

Why not an existing terminology server

Several good MCP terminology servers exist. They solve lookup — "what is code X", "map X to Y". This solves set assembly, which is the actual task in cohort definition. It also covers US ICD-10-CM, which the existing servers do not, and uses semantic retrieval rather than proxied keyword search, which fails when a population description and a code description share no vocabulary.

Architecture

Three layers — a retrieval engine, a thin MCP server, and a LangGraph agent. The MCP server and the agent are independent consumers of the same engine.

Roadmap

  • v0.1 — vocabulary layer. Athena loader, DuckDB schema, hierarchy queries

  • v0.2 — retrieval. BM25, BioLORD-2023 embeddings, LanceDB index, RRF hybrid scoring

  • v0.3 — expansion + provenance. ConceptSet model, descendant expansion, and OHDSI PL curated matching (small hand-picked demo set — see Setup)

  • v0.4 — MCP server. stdio transport, four tools (see below), Claude Desktop config

  • v0.5 — eval harness. Distance-weighted scoring, coverage, over-inclusion penalty; curated demo cohorts as ground truth. Decomposition accuracy deferred to v0.7 (see phenoforge.eval) — nothing decomposes a population description yet

  • v0.7 — LangGraph agent. Decompose → check curated first → generate only for unresolved terms → human confirmation gate on anything generated → assemble. Real interactive CLI (scripts/run_agent.py)

  • v1.0 — packaging. PyPI, docs, validation and limitations section

Skipped: v0.6 (encoder benchmark — BioLORD vs SapBERT vs MedCPT). The agent was the more demonstrable deliverable, so v0.7 was built first; the encoder benchmark may return later.

Deferred: literature-derived phenotype algorithms (published tier), RxNorm and LOINC domains

Tools

Tool

Status

Purpose

lookup_concept

done

Exact ICD-10-CM code → name and metadata

expand_hierarchy

done

Seed code → full descendant expansion (generated provenance)

search_concepts

done

Hybrid BM25 + dense search over ICD-10-CM names, RRF-fused (generated provenance); falls back to BM25-only if no dense index is built

find_curated_definition

done

Search the bundled OHDSI Phenotype Library demo set before generating anything (curated provenance)

explain_inclusion

planned

Why is this code in this set, via which path, with what evidence

find_curated_definition is the only source of curated provenance today, and only for the 8 bundled demo cohorts. Everything from expand_hierarchy and search_concepts is generated provenance — ungrounded, structural or lexical/semantic only, and meant to be confirmed by a human before use in a cohort definition.

License

Apache 2.0. Vocabulary content (ICD-10-CM, SNOMED) carries its own license terms from OHDSI/ Athena, separate from this repo's license — nothing from data/ is redistributed.

Not a clinical decision tool

This produces code sets for research and analytics. It does not make clinical determinations and has not been validated for patient care.

Available Tools

4 tools
expand_hierarchyA

Expand an ICD-10-CM code to every code beneath it in the billing hierarchy.

For example, expanding "E11" (Type 2 diabetes mellitus) returns all of its more specific subtypes, such as "E11.21" (with diabetic nephropathy). Every returned concept is tagged generated provenance: it is a structural consequence of the vocabulary hierarchy, not a clinically validated inclusion, and should be treated as ungrounded until a human confirms it belongs in the target population.

:param seed_code: An exact ICD-10-CM code to expand from, e.g. "E11". :returns: All descendant concepts, or an empty set if seed_code does not exist or has no descendants. :rtype: ConceptSet

ParametersJSON Schema
NameRequiredDescriptionDefault
seed_codeYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
conceptsNo
unmappableNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description carries full behavioral disclosure. It warns that returned concepts are tagged 'generated' provenance, are structural rather than clinically validated, and should be treated as ungrounded until human confirmation. It also states the empty-set behavior for nonexistent codes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well structured and front-loaded: purpose, concrete example, critical provenance caveat, then parameter/return documentation. Every sentence adds value and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with no annotations, the description covers the purpose, the seed_code semantics, the return type and behavior, and an important caveat about generated concepts. This is complete enough for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema only defines seed_code as a string with 0% coverage, so the description must compensate. It does so by specifying 'An exact ICD-10-CM code' and providing the E11 example, communicating the precision and format expectations. It could add period-format guidance, but it is sufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a precise action and resource: 'Expand an ICD-10-CM code to every code beneath it in the billing hierarchy.' The E11 to E11.21 example makes the operation concrete and distinguishes it from sibling tools like lookup_concept or search_concepts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The use case is clearly implied: call this when you need all descendants of an exact ICD-10-CM code. The description does not explicitly name alternatives or exclusion conditions, so it falls short of a perfect 5, but the framing leaves little ambiguity about when it applies.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_curated_definitionA

Search the OHDSI Phenotype Library for a validated cohort definition.

Use this FIRST for any population description that plausibly matches a peer-reviewed phenotype (e.g. "type 2 diabetes", "diabetic ketoacidosis") — a match here is curated provenance, safe to use as-is citing the cohort id. Only fall back to search_concepts/expand_hierarchy if nothing matches; those return generated provenance requiring human confirmation. This demo bundles a small, hand-picked set of diabetes/kidney-related cohorts, not the full library — a miss here does not mean no curated definition exists.

:param query: Free-text population description. :returns: The best-matching cohort's resolved ICD-10-CM concepts tagged curated, or an empty set with an explanatory unmappable entry if nothing in the bundled set matches or the library has not been fetched yet. :rtype: ConceptSet

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
conceptsNo
unmappableNo

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden, and it delivers: it discloses provenance semantics (curated vs generated), safety of using matches as-is, the limited bundled dataset, and the exact empty-set behavior with an 'unmappable' entry when no match exists or the library has not been fetched.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: purpose, usage priority, fallback behavior, dataset limitation, and a clean param/returns/rtype breakdown. The most important routing guidance is front-loaded in the first two paragraphs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter search tool with an output schema, the description covers all essential context: what it searches, when to use it, what it returns, how it signals failure, and a critical caveat about the bundled library. No important gap remains for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It defines the single parameter as 'Free-text population description' and supplies concrete examples like 'type 2 diabetes' and 'diabetic ketoacidosis,' making the expected input clear. It stops short of additional constraints or formatting details, but for one simple parameter this is sufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Search the OHDSI Phenotype Library for a validated cohort definition.' It further differentiates itself from siblings by positioning this tool as the curated-definition search and naming search_concepts/expand_hierarchy as the fallback alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage guidance is explicit and actionable: 'Use this FIRST for any population description that plausibly matches a peer-reviewed phenotype' and 'Only fall back to search_concepts/expand_hierarchy if nothing matches.' It also warns that a miss does not mean no curated definition exists, preventing a common misinference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lookup_conceptA

Look up a single ICD-10-CM concept by its exact billing code.

Use this when the caller already has a specific code, e.g. "E11.21", and wants its name and OMOP metadata. Not a search tool — for free-text clinical terms use search_concepts instead.

:param concept_code: An exact ICD-10-CM code, e.g. "E11.21". :returns: The matching concept, or None if no ICD-10-CM concept has that code. :rtype: Concept | None

ParametersJSON Schema
NameRequiredDescriptionDefault
concept_codeYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states exact-match behavior, the return type (Concept or None), and the absence of a match returning None. While it doesn't discuss edge cases like malformed codes, the core behavior is transparent for a lookup tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and well-structured: purpose, usage guidance, parameter definition, and return behavior are each given in clear, non-redundant sentences. No unnecessary words or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single parameter, the sibling context, and the presence of an output schema, the description is complete. It tells the agent when to use it, which sibling to use instead, what input to provide, and what to expect in return. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description compensates by defining concept_code as 'an exact ICD-10-CM code' with a concrete example ('E11.21'). This adds meaningful semantics beyond the bare string type in the schema, though it doesn't specify formatting or normalization rules.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb and resource: 'Look up a single ICD-10-CM concept by its exact billing code.' It also explicitly differentiates from search tools, making it clear this is an exact-code lookup rather than a free-text query.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use this when the caller already has a specific code' and contrasts with 'search_concepts' for free-text clinical terms. This provides clear when-to-use and when-not-to-use guidance, directly routing the agent to the correct sibling tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_conceptsA

Search ICD-10-CM concept names for a free-text clinical term or phrase.

Combines lexical (BM25) and semantic (embedding) matching, fused by reciprocal rank, so paraphrased or loosely-worded descriptions (e.g. "sugar disease" for diabetes) are found even without shared exact wording — always prefer this over guessing at exact terminology yourself. Every returned concept is tagged generated provenance and should be treated as ungrounded until a human confirms it.

:param query: Free-text search string, e.g. "diabetic nephropathy". :param k: Maximum number of results to return. :returns: Fused, deduplicated results ordered by combined relevance. If nothing matches, concepts is empty and unmappable explains why. :rtype: ConceptSet

ParametersJSON Schema
NameRequiredDescriptionDefault
kNo
queryYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
conceptsNo
unmappableNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden. It discloses the matching algorithm (BM25 + embeddings fused by reciprocal rank), that results are tagged as 'generated' provenance, and that empty results include an 'unmappable' explanation. This is solid behavioral disclosure, though it omits any explicit read-only or safety statement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose, then adds relevant behavioral details, usage guidance, and structured parameter/return docs. Every sentence contributes useful information without repetition or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the minimal schema, no annotations, and output-schema presence, the description covers the essential invocation and interpretation details: search semantics, caveats about generated provenance, result behavior on no match, and parameter meanings. The sibling list further helps an agent distinguish this search tool from exact-lookup and hierarchy tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema is minimal, only marking query as required with no property definitions. The description fully compensates by explaining query as a free-text search string with an example and k as the maximum number of results to return, making both parameters unambiguous.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb and resource: searching ICD-10-CM concept names with a free-text clinical term or phrase. It doesn't explicitly contrast with siblings like lookup_concept, but the emphasis on free-text and paraphrase matching makes the tool's role clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when to use the tool: when the user has loosely-worded clinical descriptions and when exact terminology is unknown, saying 'always prefer this over guessing at exact terminology yourself.' It doesn't name alternatives or exclusion cases, but the usage context is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 4 tool updatesv0.1.0
    • First observedexpand_hierarchy
    • First observedfind_curated_definition
    • First observedlookup_concept
    • First observedsearch_concepts
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.7/5.0
Disambiguation5/5

Each tool occupies a distinct role: exact code lookup, hierarchy expansion, free-text concept search, and curated phenotype library lookup. Explicit cross-references such as 'not a search tool' and 'use this FIRST' make misselection unlikely.

Naming Consistency5/5

All tool names follow a clear snake_case verb_noun pattern: lookup_concept, expand_hierarchy, search_concepts, find_curated_definition. The names are predictable and communicate their action on a target object.

Tool Count5/5

Four tools is well-scoped for a narrow terminology and phenotype discovery server. Each tool maps to a distinct workflow step without redundancy or unnecessary bloat.

Completeness4/5

The core discovery workflows are covered: exact code lookup, free-text search, hierarchy expansion, and curated cohort lookup. Missing capabilities like ancestor navigation, batch lookup, or explicit concept set construction are workaround-able and do not severely undermine the stated purpose.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/colbyw5/phenoforge'

If you have feedback or need assistance with the MCP directory API, please join our Discord server