Skip to main content
Glama

YAML Workflow

PyPI version Python versions CI codecov License: MIT

A lightweight workflow engine for CI/CD pipelines, data processing, and DevOps automation. Define reproducible, version-controlled workflows in YAML — run them locally, in CI, or on any machine with Python installed.

Why yaml-workflow?

Most workflow tools require servers, databases, and complex infrastructure. yaml-workflow takes a GitOps approach — workflows are plain YAML files, version-controlled alongside your code:

yaml-workflow

Airflow / Prefect / Dagster

Setup

pip install yaml-workflow

Server, database, scheduler, workers

Configuration

Plain YAML files

Python DAGs + infrastructure config

Dependencies

2 (PyYAML, Jinja2)

50+ packages, Docker, PostgreSQL

Use case

Local automation, scripts, CI/CD, data pipelines

Enterprise orchestration at scale

Learning curve

Minutes

Hours to days

State

File-based, resumable

Database-backed

Choose yaml-workflow when you need:

  • Simple task automation without infrastructure overhead

  • Reproducible pipelines defined in version-controlled YAML

  • Batch processing with parallel execution

  • State persistence and workflow resume after failures

  • A lightweight alternative to shell scripts with better error handling

  • GitOps-friendly pipelines that live in your repo alongside the code

  • A single tool that runs the same pipeline locally and in CI

Related MCP server: workflows-mcp

Features

  • YAML-driven workflow definition with Jinja2 templating

  • Multiple task types: shell, Python, file, template, HTTP, batch

  • Workflow composition via imports — reuse steps across workflows

  • Plugin system via entry points — pip install yaml-workflow-myplugin

  • Watch mode — --watch to re-run on file changes

  • Dry-run mode to preview without executing

  • Workflow visualization (ASCII branching DAG and Mermaid)

  • Parallel execution with configurable worker pools

  • State persistence and resume capability

  • Retry mechanisms with configurable strategies

  • Namespaced variables (args, env, steps, batch)

  • Flow control with custom step sequences and conditions

  • Extensible task system via @register_task decorator

  • Parallel step execution via depends_on — run independent steps concurrently

  • Secrets validation — fail fast if required environment variables are missing

  • Structured output (--format json) for CI integration and scripting

  • MCP server — expose workflows as AI agent tools (pip install yaml-workflow[mcp])

  • Web dashboard — monitor runs and trigger workflows (pip install yaml-workflow[serve])

  • GitHub Action — run workflows in CI with uses: orieg/yaml-workflow@v0.9.3

Use Cases

  • CI/CD pipelines — multi-step build, test, deploy workflows in YAML

  • Data processing — batch ETL pipelines with retry and resume on failure

  • DevOps automation — infrastructure tasks with secrets management and notifications

  • AI/LLM pipelines — orchestrate API calls with auth, retry, and batch processing

  • Local automation — replace shell scripts with reproducible, parameterized workflows

Quick Start

# Install (isolated CLI — recommended)
pipx install yaml-workflow            # Core CLI
pipx install 'yaml-workflow[all]'     # + web dashboard + MCP server

# Or with pip
pip install yaml-workflow

# Initialize example workflows
yaml-workflow init

# Run a workflow with parameters
yaml-workflow run workflows/hello_world.yaml name=Alice

Example workflow (hello_world.yaml):

name: Hello World
description: A simple greeting workflow

params:
  name:
    type: string
    default: World

steps:
  - name: create_greeting
    task: template
    inputs:
      template: "Hello, {{ args.name }}!"
      output_file: greeting.txt

  - name: show_greeting
    task: shell
    inputs:
      command: cat greeting.txt

Visualize workflows

yaml-workflow visualize workflows/data_pipeline.yaml
  Workflow: Data Pipeline

  ┌─────────────────┐
  │  detect_format  │
  │   python_code   │
  └─────────────────┘
           │
           ▼
  ┌────────────────┐  ┌────────────────┐  ┌────────────────┐  ┌────────────────┐
  │  process_json  │  │  process_csv   │  │  process_xml   │  │ handle_unknown │
  │     shell      │  │     shell      │  │     shell      │  │     shell      │
  └────────────────┘  └────────────────┘  └────────────────┘  └────────────────┘
           │
           ▼
  ┌─────────────────┐
  │ generate_report │
  │   python_code   │
  └─────────────────┘

Adjacent conditional steps are automatically grouped as branches. Use --format mermaid to export for docs or GitHub rendering.

Dry-run mode

Preview what a workflow would do without executing anything:

yaml-workflow run workflows/hello_world.yaml name=Alice --dry-run
[DRY-RUN] Workflow: Hello World
[DRY-RUN] Steps: 2 to execute

  [DRY-RUN] Step 'create_greeting' — task: template — WOULD EXECUTE
    template: Hello, Alice!
    output_file: greeting.txt
  [DRY-RUN] Step 'show_greeting' — task: shell — WOULD EXECUTE
    command: cat greeting.txt

[DRY-RUN] Complete. 2 step(s) would execute, 0 would be skipped.
[DRY-RUN] No files were written. No tasks were executed.

Workflow composition

Reuse steps across workflows with imports:

# main.yaml
imports:
  - ./shared/logging_steps.yaml
  - ./shared/common_params.yaml

steps:
  - name: my_step
    task: shell
    inputs:
      command: echo "runs after imported steps"

Imported steps are prepended. Imported params provide defaults that the main workflow can override. Supports transitive imports with circular detection.

Parallel Steps

Run independent steps concurrently with depends_on:

steps:
  - name: fetch_api
    task: http.request
    inputs: {url: "https://api.example.com/data"}

  - name: fetch_db
    task: python_code
    inputs:
      code: "result = query_database()"

  - name: merge
    task: python_code
    depends_on: [fetch_api, fetch_db]
    inputs:
      code: |
        api_data = steps["fetch_api"]["result"]
        db_data = steps["fetch_db"]["result"]
        result = {"merged": True}

Watch mode

Automatically re-run on file changes during development:

yaml-workflow run workflows/hello_world.yaml name=Alice --watch

Monitors the workflow file and all imported files. Press Ctrl+C to stop.

GitHub Actions

Run workflows in CI with the yaml-workflow action:

- name: Run pipeline
  uses: orieg/yaml-workflow@v0.9.3
  id: pipeline
  with:
    workflow: workflows/deploy.yaml
    params: |
      env=production
      version=1.2.0
    format: json

- name: Use results
  run: echo '${{ steps.pipeline.outputs.result }}'

Docker & Kubernetes

Run anywhere without installing Python:

# Run a workflow in Docker
docker run --rm -v $(pwd)/workflows:/app/workflows \
  ghcr.io/orieg/yaml-workflow run /app/workflows/pipeline.yaml

# Start the web dashboard
docker run -p 8080:8080 -v $(pwd)/workflows:/app/workflows \
  ghcr.io/orieg/yaml-workflow

Deploy on Kubernetes with the Helm chart:

helm install my-workflows ./helm/yaml-workflow \
  --set-file workflows.files.pipeline\\.yaml=workflows/pipeline.yaml

Compatible with ArgoCD (GitOps) and Argo Workflows. See the Kubernetes guide.

More commands

# List available workflows
yaml-workflow list

# Validate a workflow (with JSON output for CI)
yaml-workflow validate workflows/hello_world.yaml --format json

# Resume a failed workflow
yaml-workflow run workflows/hello_world.yaml --resume

# Structured output for scripting
yaml-workflow run workflows/pipeline.yaml --format json --output results.json

Documentation

Full documentation is available at orieg.github.io/yaml-workflow.

Contributing

Contributions are welcome! See the Contributing Guide for development setup and guidelines.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Available Tools

4 tools
dry_run_workflowA
Read-onlyIdempotent

Preview what a workflow would do without executing any task. Give a workflow (a name from list_workflows or a file path) and optional params; returns {status, outputs, preview} where preview is a human-readable list of the steps that would run with their resolved inputs (the same information as the CLI's --dry-run). Use this to inspect side effects (shell commands, file writes, HTTP calls) before running for real. It does not execute any task — no shell or Python runs and none of the workflow's own side effects occur; only ephemeral logs are written to a temporary workspace.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsNoOptional values for the workflow's declared inputs, as an object of name -> value. Omit to use each parameter's default.
workflowYesWorkflow to target: either a name returned by list_workflows, or a path to a workflow YAML file.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true and destructiveHint=false. The description adds that only ephemeral logs are written to a temporary workspace, and confirms no side effects occur. This adds valuable behavioral context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded with the purpose. It uses two focused sentences with additional detail in a second sentence. No unnecessary repetition, though the second sentence could be split for scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, but the description explicitly states the return structure: {status, outputs, preview} and explains 'preview' as a human-readable list of steps. It covers both parameters adequately and addresses the tool's safety profile. Complete for its complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% so baseline is 3. The description adds that 'workflow' can be a name from list_workflows or a file path, and 'params' are optional with defaults used when omitted. This adds modest value beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the purpose: 'Preview what a workflow would do without executing any task.' It specifies the verb (Preview), the resource (workflow), and distinguishes from siblings like run_workflow and list_workflows by emphasizing non-execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises use for inspecting side effects before real execution, and clarifies that no tasks are executed. It implicitly distinguishes from run_workflow but does not explicitly state when not to use or name alternatives beyond the context. A clear usage context is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_workflowsA
Read-onlyIdempotent

List the workflows available in this server's workflow directory. Returns an object with count and workflows (one entry per workflow, each containing name (its declared name), description, path (the YAML file), and parameters (declared inputs with types and defaults)). Call this first to discover which workflows exist and what inputs each accepts before calling dry_run_workflow or run_workflow. Read-only: it only reads YAML files and never executes anything. Takes no arguments.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description reinforces and expands: 'Read-only: it only reads YAML files and never executes anything.' It also details the return structure (object with count and workflows) and clarifies that it takes no arguments. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, each earning its place. The first sentence states the action and return structure, the second provides usage guidance, the third confirms read-only and no arguments. No fluff, front-loaded, and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (no parameters, no output schema), the description is fully complete. It covers purpose, return format, usage context, behavioral traits, and references siblings. There is no missing information for an agent to correctly select and invoke this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With zero parameters, the baseline is 4 per the instructions. The description states 'Takes no arguments,' which adds no new information beyond the empty schema, but that is acceptable for a no-parameter tool. The schema coverage is trivially 100%, but the explicit mention of no arguments is clear.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a clear verb+resource: 'List the workflows available in this server's workflow directory.' It further distinguishes itself from siblings by stating 'Call this first to discover which workflows exist and what inputs each accepts before calling dry_run_workflow or run_workflow.' This makes the purpose unambiguous and differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises when to use the tool: 'Call this first to discover which workflows exist... before calling dry_run_workflow or run_workflow.' It also notes it is read-only and takes no arguments. While it doesn't explicitly list when not to use it or mention alternatives like validate_workflow, the context with sibling tools and the sequencing advice is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_workflowA
Destructive

Execute a workflow and return its results. Give a workflow (a name from list_workflows or a file path) and optional params; runs it to completion and returns {status, workflow, outputs} where outputs maps each step name to its result. DESTRUCTIVE: a workflow may run arbitrary shell commands and Python, write files, and make HTTP requests — call dry_run_workflow first if you need to preview side effects, and only run workflows you trust.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsNoOptional values for the workflow's declared inputs, as an object of name -> value. Omit to use each parameter's default.
workflowYesWorkflow to target: either a name returned by list_workflows, or a path to a workflow YAML file.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description details that a workflow may run arbitrary shell commands, Python, write files, and make HTTP requests, explaining the nature of destructiveness beyond the annotations (destructiveHint=true, openWorldHint=true). There is no contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the main action and output, followed by a concise warning. Every sentence adds necessary information without redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Without an output schema, the description provides the return structure ({status, workflow, outputs}) and covers safety warnings. It mentions dry_run_workflow for preview. It could also reference validate_workflow for completeness, but it is adequately thorough given the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for both parameters. The description only paraphrases the schema (e.g., workflow is a name or path, params are optional) and adds no substantive extra meaning beyond the schema itself.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool executes a workflow and returns results, specifying inputs (workflow name/path and optional params) and output structure ({status, workflow, outputs}). This is distinct from siblings like list_workflows, validate_workflow, and dry_run_workflow.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly warns that the tool is destructive and advises calling dry_run_workflow first to preview side effects, and only running trusted workflows. This provides clear guidance on when to use the tool vs. the alternative, though it could also mention validate_workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_workflowA
Read-onlyIdempotent

Validate a single workflow YAML file without running it. Give the file path; returns {valid, error_count, warning_count, issues[]}, where each issue has a level (error/warning/info), message, and optional line, step, and hint. Use this to check a workflow the agent authored or edited before running it, or to explain why a workflow is malformed. Read-only: no tasks run and nothing is written.

ParametersJSON Schema
NameRequiredDescriptionDefault
pathYesPath to the workflow YAML file to validate.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. The description reinforces with 'Read-only: no tasks run and nothing is written.' Additionally, it describes the full return object including issues with level, message, line, step, hint. No contradiction; adds useful behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences with no wasted words. First sentence gives core function, second explains invocation, third details return value, fourth states safety. Well-structured and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, the description fully documents the return structure. Annotations cover safety. The tool has low complexity (one param, no nested objects). The description is sufficient for correct selection and invocation, including guidance on when to use it relative to siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (one parameter 'path' described). The description restates 'Give the file `path`' but adds no new semantics or constraints beyond what the schema already provides. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool validates a workflow YAML file without running it, and gives the return structure. It distinguishes from siblings by contrasting 'without running it' and provides specific use cases like checking before running or explaining malformations. This is a specific verb+resource with clear differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use this to check a workflow the agent authored or edited before running it, or to explain why a workflow is malformed.' This gives two clear scenarios and implies it is for validation only, contrasting with the sibling tools (list, dry_run, run). No ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 4 tool updatesv0.9.6
    • First observeddry_run_workflow
    • First observedlist_workflows
    • First observedrun_workflow
    • First observedvalidate_workflow

TDQS

A4.4/5.0
Disambiguation5/5

Each tool has a clearly distinct purpose: listing workflows, validating a file, previewing execution without side effects, and actually running a workflow. There is no ambiguity between them.

Naming Consistency5/5

All tool names are in snake_case and follow a verb_noun pattern (list_workflows, validate_workflow, dry_run_workflow, run_workflow). The use of 'dry_run' as a compound verb is consistent with the pattern.

Tool Count4/5

Four tools is on the lower end but perfectly scoped for this server's purpose—covering discovery, validation, preview, and execution. Each tool earns its place without any redundancy.

Completeness5/5

The tool surface covers the full lifecycle of using a YAML workflow: list available workflows, validate a file, preview effects, and execute. There are no obvious gaps for the stated domain of running existing workflows.

Maintenance

ActivityMaintained
ResponsivenessResponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    C
    maintenance
    Lightweight AI agent orchestrator with built-in Architect AI, enabling users to automate tasks by describing them via chat or Claude Code + MCP, with multi-team isolation.
    79
    3
    MIT
  • A
    license
    Not graded
    quality
    A
    maintenance
    A local, auditable multi-model workflow engine that lets you define YAML graphs for orchestrating LLM agents across vendors, with MCP tools for validation, dry-runs, execution, and human approval, all fully observable in a local web interface.
    3
    Apache 2.0

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/orieg/yaml-workflow'

If you have feedback or need assistance with the MCP directory API, please join our Discord server