Skip to main content
Glama
amittell

firewalla-mcp-server

Firewalla MCP Server

npm version

A Model Context Protocol (MCP) server that provides real-time access to Firewalla firewall data through 28 specialized tools, compatible with any MCP client.

Why Firewalla MCP Server?

Simple Network Security Integration

  • 28 Tools for network monitoring and analysis

  • 23 Direct API Endpoints + 5 Convenience Wrappers

  • Advanced Search with query syntax and filters

  • Clean, Verified Architecture with corrected API schemas

Related MCP server: Firewalla MCP Server

Features

  • Real-time Firewall Data: Query security alerts, network flows, and device status

  • Security Analysis: Get insights on threats, blocked attacks, and network anomalies

  • Bandwidth Monitoring: Track top bandwidth consumers and usage patterns

  • Rule Management: View and temporarily pause firewall rules

  • Target Lists: Manage custom security target lists and categories

  • Search Tools: Query syntax with filters and logical operators

Client Setup Guides

Client

Quick Start

Full Guide

Claude Desktop

npm i -g firewalla-mcp-server → Configure MCP

Setup Guide

Claude Code

npm i -g firewalla-mcp-server → CLI integration

Setup Guide

VS Code

Install MCP extension → Configure server

Setup Guide

Cursor

Install Claude Code → VSIX method

Setup Guide

Roocode

Install MCP support → Configure server

Setup Guide

Cline

Configure in VS Code → Enable MCP

Setup Guide

How It Works

Claude Desktop/Code ↔ MCP Server ↔ Firewalla API

The MCP server acts as a bridge between Claude and your Firewalla firewall, translating Claude's requests into Firewalla API calls and returning the results in a format Claude can understand.

Prerequisites

  • Node.js 18+ and npm

  • Firewalla MSP account with API access

  • Your Firewalla device online and connected

Quick Start

1. Installation

# Install globally
npm install -g firewalla-mcp-server

# Or install locally in your project
npm install firewalla-mcp-server

Option B: Use Docker

Warning: Not for production use – secrets visible in process list

The examples below pass credentials directly in the command line, which exposes them to process listing and shell history. For production use, consider these secure alternatives:

  • Use --env-file with a .env file: docker run --env-file .env ...

  • Set environment variables in your shell before running Docker

  • Use Docker secrets for orchestration environments

Stdio Transport (Default - for Claude Desktop integration):

# Using Docker Hub image
docker run -it --rm \
  -e FIREWALLA_MSP_TOKEN=your_token \
  -e FIREWALLA_MSP_ID=yourdomain.firewalla.net \
  -e FIREWALLA_BOX_ID=your_box_gid \
  amittell/firewalla-mcp-server

# Or build locally
docker build -t firewalla-mcp-server .
docker run -it --rm \
  -e FIREWALLA_MSP_TOKEN=your_token \
  -e FIREWALLA_MSP_ID=yourdomain.firewalla.net \
  -e FIREWALLA_BOX_ID=your_box_gid \
  firewalla-mcp-server

# Recommended: Using env file (more secure)
docker run -it --rm --env-file .env amittell/firewalla-mcp-server

HTTP Transport (for standalone Docker containers and external access):

# Run with HTTP transport on port 3000
docker run -d --name firewalla-mcp \
  -p 3000:3000 \
  -e MCP_TRANSPORT=http \
  -e MCP_HTTP_PORT=3000 \
  -e FIREWALLA_MSP_TOKEN=your_token \
  -e FIREWALLA_MSP_ID=yourdomain.firewalla.net \
  -e FIREWALLA_BOX_ID=your_box_gid \
  amittell/firewalla-mcp-server

# The server will be accessible at http://localhost:3000/mcp

# Using env file (recommended)
docker run -d --name firewalla-mcp \
  -p 3000:3000 \
  --env-file .env \
  amittell/firewalla-mcp-server

# For docker-compose
cat > docker-compose.yml << EOF
version: '3.8'
services:
  firewalla-mcp:
    image: amittell/firewalla-mcp-server
    ports:
      - "3000:3000"
    environment:
      - MCP_TRANSPORT=http
      - MCP_HTTP_PORT=3000
      - FIREWALLA_MSP_TOKEN=\${FIREWALLA_MSP_TOKEN}
      - FIREWALLA_MSP_ID=\${FIREWALLA_MSP_ID}
      - FIREWALLA_BOX_ID=\${FIREWALLA_BOX_ID}
    restart: unless-stopped
EOF

docker-compose up -d

Option C: Install from source

git clone https://github.com/amittell/firewalla-mcp-server.git
cd firewalla-mcp-server
npm install
npm run build

2. Configuration

Create a .env file with your Firewalla credentials:

# Required
FIREWALLA_MSP_TOKEN=your_msp_access_token_here
FIREWALLA_MSP_ID=yourdomain.firewalla.net

# Optional - filters all queries to a specific box
# FIREWALLA_BOX_ID=your_box_gid_here

Getting Your Credentials:

  1. Log into your Firewalla MSP portal at https://yourdomain.firewalla.net

  2. Your MSP ID is the full domain (e.g., company123.firewalla.net)

  3. Generate an access token in API settings

  4. (Optional) Find your Box GID in device settings to filter queries to a specific box, or retrieve available boxes using the get_boxes tool

Transport Configuration

The MCP server supports two transport modes:

Stdio Transport (Default): Standard input/output communication for Claude Desktop and similar MCP clients

MCP_TRANSPORT=stdio

HTTP Transport: HTTP server mode for Docker containers, MCP orchestrators, and external access

MCP_TRANSPORT=http
MCP_HTTP_PORT=3000          # Default: 3000
MCP_HTTP_PATH=/mcp          # Default: /mcp

When to use HTTP transport:

  • Running in Docker containers independently

  • Accessing from MCP orchestrators (e.g., open-webui)

  • Multiple clients need to connect to the same server instance

  • Network-based access to the MCP server

When to use stdio transport:

  • Claude Desktop integration (default)

  • Claude Code CLI integration

  • Single-process MCP client setups

  • Standard MCP client configurations

3. Build and Start

npm run build
npm run mcp:start

4. Connect Claude Desktop

Add this configuration to your Claude Desktop claude_desktop_config.json:

If installed via npm

{
  "mcpServers": {
    "firewalla": {
      "command": "npx",
      "args": ["firewalla-mcp-server"],
      "env": {
        "FIREWALLA_MSP_TOKEN": "your_msp_access_token_here",
        "FIREWALLA_MSP_ID": "yourdomain.firewalla.net",
        "FIREWALLA_BOX_ID": "your_box_gid_here"
      }
    }
  }
}

If using Docker

{
  "mcpServers": {
    "firewalla": {
      "command": "docker",
      "args": ["run", "-i", "--rm", 
        "-e", "FIREWALLA_MSP_TOKEN=your_token",
        "-e", "FIREWALLA_MSP_ID=yourdomain.firewalla.net",
        "-e", "FIREWALLA_BOX_ID=your_box_gid",
        "amittell/firewalla-mcp-server"
      ]
    }
  }
}

If installed from source

{
  "mcpServers": {
    "firewalla": {
      "command": "node",
      "args": ["/full/path/to/firewalla-mcp-server/dist/server.js"],
      "env": {
        "FIREWALLA_MSP_TOKEN": "your_msp_access_token_here",
        "FIREWALLA_MSP_ID": "yourdomain.firewalla.net",
        "FIREWALLA_BOX_ID": "your_box_gid_here"
      }
    }
  }
}

Config file locations:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

  • Windows: %APPDATA%\Claude\claude_desktop_config.json

5. Next Steps

Usage Examples

Step-by-Step First Use

1. Verify Connection After completing the setup, verify the MCP server is working:

# Start the server
npm run mcp:start

# You should see output like:
# MCP Server starting...
# Firewalla client initialized
# Server ready on stdio transport

2. Test with Claude Open Claude Desktop and try these starter queries:

Basic Health Check:

"Can you check my Firewalla status and show me a summary?"

This uses: firewall_summary resource + get_simple_statistics tool

Security Overview:

"What security alerts do I have? Show me the 5 most recent ones."

This uses: get_active_alarms tool with limit parameter

Practical Workflows

Daily Security Review:

"Give me today's security report. Include:
1. Any new security alerts
2. Top 3 devices using bandwidth
3. Any devices that went offline
4. Status of critical firewall rules"

Investigating Suspicious Activity:

"I noticed unusual traffic. Can you:
1. Show me all security and abnormal upload alarms from the last 4 hours
2. Find any blocked connections to external IPs
3. Check which devices had the most network activity"

Network Troubleshooting:

"A device seems to have connectivity issues. Can you:
1. Check if device 192.168.1.100 is online
2. Show its recent network flows
3. See if any rules are blocking its traffic"

Bandwidth Investigation:

"Our internet is slow. Help me find the cause:
1. Show top 10 bandwidth users in the last hour
2. Look for any devices with unusual upload/download patterns
3. Check for any streaming or video traffic"

Advanced Search Examples

Find Specific Threats:

search for: security activity alarms from IP range 10.0.0.* in the last 24 hours

Uses: search_alarms with query: "type:1 AND source_ip:10.0.0. AND timestamp:>24h"*

Analyze Rule Effectiveness:

"Show me firewall rules that blocked the most connections this week"

Uses: get_network_rules + search_flows for blocked traffic analysis

Device Behavior Analysis:

"Find all devices that were online yesterday but are offline now"

Uses: search_devices with temporal queries + get_offline_devices

Troubleshooting Common Issues

Connection Problems: If you get authentication errors:

  1. Verify your .env file has correct credentials

  2. Check your MSP token hasn't expired

  3. Confirm your Box ID is the full GID format

Empty Results: If queries return no data:

  1. Check your Firewalla is online and reporting

  2. Verify the time range isn't too narrow

  3. Try broader search terms first

Performance Issues: If responses are slow:

  1. Reduce the limit parameter in queries

  2. Use more specific time ranges

  3. Check your network connection to the MSP API

Available Tools (28 total)

Core Tools

  • Security: Get alarms, analyze threats

  • Network: Monitor traffic flows, track bandwidth usage

  • Devices: Check device status, find offline devices

  • Rules: Manage firewall rules, pause/resume rules

  • Search: Advanced search across all data types

  • Analytics: Statistics, trends, and geographic analysis

  • Target Management: Create, update, and delete security target lists

Quick Reference

Security: get_active_alarms, get_specific_alarm
Network: get_flow_data, get_bandwidth_usage, get_offline_devices  
Devices: get_device_status, get_boxes, search_devices
Rules: get_network_rules, pause_rule, resume_rule, get_target_lists
Search: search_flows, search_alarms, search_rules, search_target_lists
Analytics: get_simple_statistics, get_flow_insights, get_flow_trends, get_alarm_trends
Management: create_target_list, update_target_list, delete_target_list

Development

Scripts

npm run dev          # Start development server with hot reload
npm run build        # Build TypeScript to JavaScript
npm run test         # Run all tests
npm run test:watch   # Run tests in watch mode
npm run lint         # Run ESLint
npm run lint:fix     # Fix ESLint issues

MCP Execution Methods

Why npx for MCP servers?

  • Version Management: Always uses the correct/latest version

  • Dependency Resolution: Handles package dependencies automatically

  • No global installation required: Works without global installation

  • MCP Standard: Follows Model Context Protocol conventions

  • Reliable: Works consistently across different environments

Alternative execution methods:

# Development (from source)
npm run mcp:start

# Production (npm installed)
npx firewalla-mcp-server

# Direct execution (from source after build)
node dist/server.js

Project Structure

firewalla-mcp-server/
├── src/
│   ├── server.ts           # Main MCP server
│   ├── firewalla/          # Firewalla API client
│   ├── tools/              # MCP tool implementations
│   ├── resources/          # MCP resource implementations
│   └── prompts/            # MCP prompt implementations
├── tests/                  # Test files
├── docs/
│   └── firewalla-api-reference.md  # API documentation
├── CLAUDE.md              # Comprehensive development guide
├── SPEC.md                # Technical specifications
└── README.md              # This file

Documentation

  • README.md (this file) - Setup and basic usage

  • USAGE.md - Simple usage guide with examples

  • TROUBLESHOOTING.md - Common issues and solutions

  • docs/clients/ - Client-specific setup guides

  • CLAUDE.md - Development guide and commands

Security

  • MSP tokens are stored securely in environment variables

  • No credentials are logged or stored in code

  • Rate limiting prevents API abuse

  • Input validation prevents injection attacks

  • All API communications use HTTPS

Known Behaviors and Limitations

Category Classification

  • Flow Categories: Many network flows may show as empty category ("") in the Firewalla API response. This is expected behavior - Firewalla categorizes traffic when it recognizes the domain/service (e.g., "av" for audio/video, "social" for social media).

  • Target List Categories: Some target lists may show category as "unknown". This is normal for user-created or certain system lists.

  • Timeline: Category classification happens at the Firewalla device level and may take time to build up meaningful categorization data.

Data Characteristics

  • Response Sizes: The get_recent_flow_activity tool returns up to 150 recent flows to stay within token limits. For larger datasets or historical analysis, use search_flows with time filters for more targeted queries.

  • Geographic Data: IP geolocation is enriched by the MCP server and includes country, city, and risk scores when available.

API Limitations

  • Alarm Deletion: The delete_alarm tool may not actually delete alarms even though the Firewalla API returns a success response. This appears to be a limitation of the MSP API where delete operations return {"message": "success", "success": true} but the alarm remains in the system. This may be due to permission restrictions or API design.

Troubleshooting

Quick Fixes

Server won't start:

# Clean and rebuild
npm run clean
npm run build

# If build fails, try:
npm install
npm run build

Authentication errors:

  • Check your MSP token is valid

  • Verify Box ID format (long UUID)

  • Confirm MSP domain is correct

No data returned:

  • Try broader queries: "last week" vs "last hour"

  • Check if Firewalla is online

  • Test with: "show me basic statistics"

Slow responses:

  • Add limits: "top 10 devices"

  • Use shorter time ranges

  • Restart the server

Debug Mode

Enable detailed logging:

DEBUG=mcp:* npm run mcp:start

For more detailed troubleshooting, see TROUBLESHOOTING.md

Contributing

  1. Fork the repository

  2. Create a feature branch

  3. Make your changes

  4. Add tests for new functionality

  5. Run the test suite

  6. Submit a pull request

What's New

Version 1.0.0:

  • 28 tools with API-verified schemas

  • 24 direct API endpoints + 5 convenience wrappers

  • NEW: get_flow_insights for category-based traffic analysis

  • Advanced search with logical operators (AND, OR, NOT)

  • All limits corrected to API maximum (500)

  • Required parameters added for proper API calls

  • Better caching for faster responses

  • Handles high-volume networks (300k+ flows/day)

License

MIT License

Support

For issues and questions:


GitHub Repository

Repository: https://github.com/amittell/firewalla-mcp-server

Repository Stats

GitHub issues GitHub stars GitHub license

Available Tools

28 tools
create_target_listC

Create a new target list

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesTarget list name (required, max 24 chars)
notesNoAdditional description (optional)
ownerYesOwner: "global" or box GID (required)
targetsYesArray of domains, IPs, or CIDR ranges (required)
categoryNoContent category (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. While 'create' implies a write operation, it doesn't mention permissions required, whether the creation is idempotent, what happens on duplicate names, or what the response looks like. For a mutation tool with zero annotation coverage, this leaves significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is maximally concise with a single clear sentence that states the core function. There's no wasted language or unnecessary elaboration, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a creation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what a 'target list' is, what happens after creation, how to verify success, or how this tool relates to other target list operations. The agent would need to guess about important behavioral aspects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents all 5 parameters. The description adds no additional parameter information beyond what's in the schema. Baseline 3 is appropriate when the schema does all the parameter documentation work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('create') and resource ('target list'), making the purpose immediately understandable. However, it doesn't differentiate this tool from its sibling 'update_target_list' beyond the basic verb difference, nor does it explain what a 'target list' is in this context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives like 'update_target_list' or 'search_target_lists'. The description offers no context about prerequisites, typical use cases, or relationships with other tools in the system.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_target_listC

Delete a target list

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTarget list ID to delete (required)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Delete' implies a destructive mutation, the description doesn't specify whether this action is reversible, requires specific permissions, has side effects (e.g., affecting related data), or provides confirmation feedback. This is a significant gap for a mutation tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste—'Delete a target list'—making it front-loaded and appropriately sized for its purpose. Every word earns its place without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given this is a destructive mutation tool with no annotations and no output schema, the description is incomplete. It lacks crucial details like behavioral traits (e.g., irreversibility, permissions), usage context, and what happens post-deletion. For a tool with this complexity and minimal structured data, more information is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the single parameter 'id' documented as 'Target list ID to delete (required)'. The description adds no additional meaning beyond this, such as format examples or constraints. Baseline 3 is appropriate when the schema handles parameter documentation adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete') and the resource ('a target list'), providing a specific verb+resource combination. However, it doesn't differentiate this tool from sibling tools like 'update_target_list' or 'get_target_lists' beyond the basic action, missing explicit sibling distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. There are no mentions of prerequisites (e.g., needing an existing target list), exclusions, or comparisons to siblings like 'update_target_list' or 'get_target_lists', leaving usage context unclear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_active_alarmsC

Retrieve current security alerts and alarms from Firewalla firewall

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoResults per page (optional, default: 200, API maximum: 500)
queryNoSearch query for filtering alarms (default: status:1 for active). Use type:N where N is: 1=Security Activity, 2=Abnormal Upload, 3=Large Bandwidth Usage, 4=Monthly Data Plan, 5=New Device, 6=Device Back Online, 7=Device Offline, 8=Video Activity, 9=Gaming Activity, 10=Porn Activity, 11=VPN Activity, 12=VPN Connection Restored, 13=VPN Connection Error, 14=Open Port, 15=Internet Connectivity Update, 16=Large Upload. Examples: type:8 (video), type:10 (porn), region:US, source_ip:*
cursorNoPagination cursor from previous response
sortByNoSort alarms (default: ts:desc)
groupByNoGroup alarms by field (e.g., type, box)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure but only states what the tool does ('Retrieve'), not how it behaves. It doesn't mention whether this is a read-only operation, if it requires authentication, rate limits, pagination behavior beyond the cursor parameter, or what format the alarms are returned in. For a tool with 5 parameters and no annotations, this is inadequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that immediately communicates the core function without unnecessary words. It's perfectly front-loaded and wastes no space on redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 5 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't explain what 'current' means (real-time vs cached), what format alarms are returned in, authentication requirements, or error conditions. The agent would need to guess about important behavioral aspects despite the good parameter documentation in the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly with examples and constraints. The description adds no additional parameter information beyond what's in the schema, meeting the baseline expectation but not exceeding it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Retrieve') and resource ('current security alerts and alarms from Firewalla firewall'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_specific_alarm' or 'search_alarms', which would require more specific scope definition to earn a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'get_specific_alarm' or 'search_alarms'. There's no mention of prerequisites, appropriate contexts, or exclusions, leaving the agent to infer usage from tool names alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_bandwidth_usageB

Get top bandwidth consuming devices (convenience wrapper around get_device_status)

ParametersJSON Schema
NameRequiredDescriptionDefault
boxNoFilter devices under a specific Firewalla box
limitNoNumber of top devices to return
periodYesTime period for bandwidth calculation

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It mentions being a 'convenience wrapper,' hinting at simplified behavior, but fails to disclose critical traits: whether it's read-only, how it handles errors, rate limits, authentication needs, or the format of returned data (e.g., list of devices with bandwidth metrics). This leaves significant gaps for an agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose and adds clarifying context without waste. Every word earns its place, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits (e.g., safety, data format) and output values, which are crucial for a tool with parameters and potential data retrieval. The mention of being a wrapper adds some context but doesn't compensate for these gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents parameters (period, limit, box). The description adds no parameter-specific semantics beyond implying it returns 'top bandwidth consuming devices,' which aligns with the schema but doesn't provide additional context like how 'top' is determined or interaction between parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Get top bandwidth consuming devices' specifies the verb (get) and resource (bandwidth consuming devices). It distinguishes from the sibling 'get_device_status' by noting it's a convenience wrapper, though it doesn't explicitly differentiate from other bandwidth-related tools (none listed).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by referencing 'get_device_status' as an alternative, suggesting this is a simplified version. However, it lacks explicit guidance on when to use this tool versus other bandwidth or device-related tools (e.g., get_simple_statistics, search_devices), and does not mention prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_boxesC

Retrieve list of Firewalla boxes

ParametersJSON Schema
NameRequiredDescriptionDefault
groupNoGet boxes within a specific group (requires group ID)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It states 'Retrieve list' which implies a read-only operation, but doesn't disclose behavioral traits like authentication requirements, rate limits, pagination, or what happens if the group parameter is omitted. For a tool with zero annotation coverage, this is inadequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a simple retrieval tool and front-loads the core purpose immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations, no output schema, and a simple parameter schema, the description is incomplete. It doesn't explain what a 'Firewalla box' represents in this context, what the return format looks like, or any prerequisites for using the optional group parameter. For a tool that might return critical infrastructure data, more context is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents the single optional parameter 'group'. The description doesn't add any meaning beyond what the schema provides (e.g., it doesn't explain what a 'Firewalla box' is or how grouping works), meeting the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Retrieve' and the resource 'list of Firewalla boxes', making the purpose understandable. However, it doesn't differentiate from siblings like 'get_device_status' or 'get_statistics_by_box' which also retrieve box-related information, preventing a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_device_status' and 'get_statistics_by_box' that might overlap, there's no indication of when this specific list retrieval is appropriate or what distinguishes it from other get_* tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_device_statusB

Check online/offline status of devices on Firewalla network

ParametersJSON Schema
NameRequiredDescriptionDefault
boxNoGet devices under a specific Firewalla box (requires box ID)
groupNoGet devices under a specific box group (requires group ID)
limitYesMaximum number of devices to return (required)

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. While it implies a read operation ('Check'), it doesn't specify whether this requires authentication, has rate limits, returns paginated results, or details the output format (e.g., list of devices with status). For a tool with no annotation coverage, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. It directly answers 'what does this tool do?' with zero waste, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is minimally adequate but incomplete. It covers the basic purpose but lacks behavioral details (e.g., output format, error handling) and usage guidelines relative to siblings. Without annotations or output schema, more context would help the agent use it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with all parameters ('limit', 'box', 'group') well-documented in the schema. The description adds no additional parameter semantics beyond what the schema provides, such as explaining how 'box' and 'group' interact or default behaviors. Baseline 3 is appropriate since the schema handles parameter documentation adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Check') and resource ('online/offline status of devices on Firewalla network'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_offline_devices' or 'search_devices', which appear related but have different scopes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'get_offline_devices' or 'search_devices'. It doesn't mention prerequisites, exclusions, or specific contexts where this tool is preferred over siblings, leaving the agent to infer usage from tool names alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_flow_dataC

Query network traffic flows from Firewalla firewall

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum results (optional, default: 200, API maximum: 500)
queryNoSearch query for flows. Supports region:US for geographic filtering, protocol:tcp, blocked:true, domain:*, category:social, etc.
cursorNoPagination cursor from previous response
sortByNoSort flows (default: "ts:desc")
groupByNoGroup flows by specified values (e.g., "domain,box")

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It mentions querying but doesn't describe what kind of data is returned, whether this is a real-time or historical query, rate limits, authentication requirements, or potential side effects. The description is too minimal for a tool with 5 parameters and no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for a query tool and front-loads the essential information about what the tool does.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a query tool with 5 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what data structure is returned, what time range is covered, whether results are paginated beyond the cursor parameter, or how this differs from similar sibling tools. The description leaves too many contextual gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly with examples and constraints. The description adds no additional parameter information beyond what's in the schema, which meets the baseline expectation when schema coverage is high.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Query') and resource ('network traffic flows from Firewalla firewall'), providing a specific purpose. However, it doesn't distinguish this tool from sibling tools like 'search_flows' or 'get_recent_flow_activity', which appear to serve similar querying functions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'search_flows' or 'get_recent_flow_activity'. There's no mention of prerequisites, typical use cases, or exclusions that would help an agent choose between these similar-sounding tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_flow_insightsA

Get category-based flow analysis including top content categories, bandwidth consumers, and blocked traffic. Ideal for answering questions like "what porn sites were accessed" or "what social media was used". Replaces time-based trends with actionable insights.

ParametersJSON Schema
NameRequiredDescriptionDefault
periodNoTime period for analysis (default: 24h)24h
categoriesNoFilter to specific content categories (optional)
include_blockedNoInclude blocked traffic analysis (default: false)

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool does (analysis of categories, bandwidth, blocked traffic) and provides example use cases, but lacks details on permissions, rate limits, response format, or potential side effects. It's adequate but not comprehensive for a tool with no annotation support.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly concise and well-structured with three sentences that each serve distinct purposes: stating the core functionality, providing concrete usage examples, and differentiating from alternatives. Every sentence earns its place with zero wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations and no output schema, the description does well at explaining what the tool provides (actionable insights) and when to use it. However, it lacks details about the return format or structure of insights, which would be helpful given the absence of output schema. The description is complete enough for basic understanding but could better address output expectations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents all three parameters. The description adds no additional parameter semantics beyond what's in the schema - it doesn't explain how parameters interact or provide usage examples with specific parameter values. Baseline 3 is appropriate when schema does all the parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('Get category-based flow analysis') and resources ('top content categories, bandwidth consumers, and blocked traffic'). It distinguishes from siblings by focusing on actionable insights rather than time-based trends, unlike tools like get_bandwidth_usage or get_flow_data which may provide different types of data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance with examples ('Ideal for answering questions like "what porn sites were accessed" or "what social media was used"') and distinguishes when to use this tool versus alternatives ('Replaces time-based trends with actionable insights'), clearly differentiating it from trend-focused sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_network_rulesB

Retrieve firewall rules and conditions

ParametersJSON Schema
NameRequiredDescriptionDefault
limitYesMaximum number of rules to return (required)
queryNoSearch conditions for filtering rules

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. It states this is a retrieval operation, implying read-only behavior, but doesn't mention any constraints like rate limits, authentication requirements, or what 'conditions' entail. For a tool with no annotation coverage, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for a simple retrieval tool, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity appears low (simple retrieval with 2 parameters) and no output schema, the description is minimally adequate but incomplete. It doesn't explain what 'conditions' means in the return data or differentiate from siblings, which could confuse an agent. With no annotations, it should provide more behavioral context to be fully helpful.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with clear documentation for both parameters (limit and query). The description adds no additional parameter semantics beyond what's in the schema, so it meets the baseline score of 3 where the schema does the heavy lifting without compensation needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Retrieve firewall rules and conditions' clearly states the verb (retrieve) and resource (firewall rules and conditions), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_network_rules_summary' or 'search_rules', which appear to offer similar functionality, so it doesn't reach the highest score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_network_rules_summary' and 'search_rules' that likely retrieve similar data, there's no indication of differences in scope, filtering capabilities, or output format, leaving the agent without context for tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_network_rules_summaryB

Get overview statistics and counts of network rules by category (convenience wrapper)

ParametersJSON Schema
NameRequiredDescriptionDefault
rule_typeNoFilter by rule type
active_onlyNoOnly include active rules in summary (default: true)

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool is a 'convenience wrapper' and provides 'overview statistics and counts,' but doesn't cover critical aspects like whether it's read-only, its performance characteristics, error handling, or output format. For a tool with no annotation coverage, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence: 'Get overview statistics and counts of network rules by category (convenience wrapper).' It's front-loaded with the core purpose and includes a helpful qualifier. There's no wasted text, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose but lacks details on behavior, output, or sibling differentiation. Without annotations or an output schema, more context on what the summary includes would improve completeness, but it's not entirely incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents both parameters ('active_only' and 'rule_type'). The description doesn't add any parameter-specific information beyond what's in the schema, such as examples or usage tips. With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Get overview statistics and counts of network rules by category.' It specifies the verb ('Get') and resource ('network rules'), and adds context about being a 'convenience wrapper.' However, it doesn't explicitly differentiate from siblings like 'get_network_rules' or 'get_rule_trends,' which slightly limits clarity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by mentioning it's a 'convenience wrapper,' suggesting it's for quick summaries rather than detailed data. But it lacks explicit guidance on when to use this tool versus alternatives like 'get_network_rules' or 'get_rule_trends,' and doesn't specify prerequisites or exclusions, leaving usage context somewhat vague.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_offline_devicesB

Get all offline devices (convenience wrapper around get_device_status)

ParametersJSON Schema
NameRequiredDescriptionDefault
boxNoFilter devices under a specific Firewalla box
limitNoMaximum number of offline devices to return
sort_by_last_seenNoSort devices by last seen time (default: true)

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions being a 'convenience wrapper', it doesn't describe what that entails operationally - whether this is a filtered view, how it handles pagination, what the return format looks like, or any rate limits. For a tool with 3 parameters and no annotations, this leaves significant behavioral aspects undocumented.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise - a single sentence that efficiently communicates the core purpose and relationship to another tool. Every word earns its place, with no wasted verbiage or unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations, no output schema, and 3 parameters, the description is incomplete. While concise, it doesn't provide enough context about what the tool returns, how results are structured, or important behavioral aspects. For a tool that presumably returns potentially large datasets of offline devices, more guidance on usage patterns and result handling would be helpful.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are documented in the schema. The description doesn't add any additional parameter semantics beyond what's already in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no parameter information in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('offline devices'), making the purpose evident. It distinguishes from the sibling 'get_device_status' by specifying it's a convenience wrapper focused on offline devices only. However, it doesn't fully differentiate from other device-related siblings like 'search_devices' in terms of scope or filtering approach.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by mentioning it's a 'convenience wrapper around get_device_status', suggesting this tool should be used when specifically interested in offline devices rather than general device status. However, it doesn't provide explicit guidance on when to use this versus alternatives like 'search_devices' or clarify any prerequisites or exclusions for usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_recent_flow_activityA

Get recent network flow activity snapshot (last 10-20 minutes). Returns up to 50 most recent flows for immediate analysis. CRITICAL: This is a quick snapshot tool only. Use this for: "what's happening right now?", current security threats, immediate network issues. DO NOT use for: historical analysis (use search_flows), getting more than 50 flows (use search_flows with limit), daily/weekly patterns (use search_flows with time queries like "ts:>24h"). For comprehensive analysis, always prefer search_flows.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the tool provides a snapshot with specific time constraints (last 10-20 minutes), result limits (up to 50 flows), and intended use case (immediate analysis). It doesn't mention error conditions, authentication requirements, or rate limits, but provides substantial operational context beyond basic functionality.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured with zero wasted sentences. It front-loads the core functionality, immediately provides critical constraints, then delivers clear usage guidelines with specific examples. Every sentence adds essential information about the tool's purpose, limitations, and appropriate use cases.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no parameters, no annotations, and no output schema, the description provides excellent contextual completeness for a read-only query tool. It explains what the tool returns (recent flow activity snapshot), temporal scope, result limits, and when to use it versus alternatives. The main gap is lack of information about return format or error conditions, but for a zero-parameter tool with clear sibling differentiation, this is quite complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, and instead focuses on the tool's operational characteristics and usage guidelines.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Get recent network flow activity snapshot') and resource ('network flow activity'), with precise temporal scope ('last 10-20 minutes') and result limit ('up to 50 most recent flows'). It explicitly distinguishes this tool from its sibling 'search_flows' by emphasizing it's for immediate analysis only, not historical data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool ('what's happening right now?', current security threats, immediate network issues) and when not to use it (historical analysis, getting more than 50 flows, daily/weekly patterns). It names the alternative tool ('search_flows') and specifies when to prefer it ('For comprehensive analysis, always prefer search_flows').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_simple_statisticsC

Retrieve basic statistics overview

ParametersJSON Schema
NameRequiredDescriptionDefault
groupNoGet statistics for specific box group

TDQS

C2.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'retrieve' which implies a read-only operation, but doesn't cover critical aspects like authentication needs, rate limits, error handling, or what the output format looks like (e.g., summary vs. detailed data). This leaves significant gaps for safe and effective use.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words, making it easy to parse. However, it's overly brief and could benefit from more detail to improve clarity without sacrificing conciseness, as it currently under-specifies the tool's scope.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is incomplete for a tool that likely returns complex statistics data. It doesn't explain what 'basic statistics' entails, how results are structured, or any behavioral constraints, making it inadequate for reliable agent operation without additional context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with the 'group' parameter documented as 'Get statistics for specific box group'. The description adds no additional parameter information beyond this, so it doesn't compensate but also doesn't detract, meeting the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Retrieve basic statistics overview' clearly states the action (retrieve) and resource (statistics overview), making the purpose understandable. However, it lacks specificity about what 'basic statistics' includes and doesn't differentiate from sibling tools like 'get_statistics_by_box' or 'get_statistics_by_region', leaving ambiguity about scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as 'get_statistics_by_box' or 'get_statistics_by_region'. The description implies a general overview but doesn't specify use cases, prerequisites, or exclusions, leaving the agent to guess based on tool names alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_specific_alarmB

Get detailed information for a specific Firewalla alarm

ParametersJSON Schema
NameRequiredDescriptionDefault
alarm_idYesAlarm ID (required for API call)

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'Get[s] detailed information,' implying a read-only operation, but doesn't clarify aspects like authentication requirements, rate limits, error handling, or what 'detailed information' includes (e.g., fields, format). This leaves significant gaps for a tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy to understand at a glance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (one required parameter) and high schema coverage, the description is minimally adequate. However, with no annotations and no output schema, it fails to fully compensate by not explaining behavioral traits or return values, leaving the agent to infer details from the tool name and schema alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with the single parameter 'alarm_id' documented as 'Alarm ID (required for API call).' The description adds no additional parameter semantics beyond what the schema provides, such as format examples or constraints, so it meets the baseline score of 3 for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Get') and resource ('detailed information for a specific Firewalla alarm'), making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_active_alarms' or 'search_alarms', which would require mentioning this tool retrieves a single alarm by ID rather than listing or searching multiple alarms.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention that it's for retrieving a single alarm by ID, as opposed to using 'get_active_alarms' for all active alarms or 'search_alarms' for filtered searches, nor does it specify prerequisites like needing the alarm ID.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_specific_target_listC

Retrieve a specific target list by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTarget list ID (required)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the action ('retrieve'), but doesn't describe what 'retrieve' entails—such as whether it returns detailed data, requires authentication, has rate limits, or handles errors. This leaves significant gaps for a tool that likely fetches sensitive or structured data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to scan and understand quickly without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of retrieving a specific entity (likely with structured data), no annotations, and no output schema, the description is incomplete. It doesn't explain what data is returned, error conditions, or behavioral traits, leaving the agent with insufficient information for reliable use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with the 'id' parameter fully documented in the schema. The description adds no additional meaning beyond what the schema provides (e.g., format examples or context about ID sources), so it meets the baseline for high schema coverage without compensating value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('retrieve') and resource ('specific target list by ID'), making the purpose understandable. However, it doesn't distinguish this tool from sibling tools like 'get_target_lists' (which likely lists multiple target lists) or 'search_target_lists' (which likely searches with criteria), missing full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose it over 'get_target_lists' (for listing all) or 'search_target_lists' (for searching by criteria), and offers no context about prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_statistics_by_boxB

Get statistics for each Firewalla box (top boxes by blocked flows or security alarms)

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoStatistics type to retrievetopBoxesByBlockedFlows
groupNoGet statistics for specific box group
limitNoMaximum number of results (optional, default: 5)

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only states what statistics are retrieved without behavioral details. It does not disclose whether this is a read-only operation, requires authentication, has rate limits, or describes the return format (e.g., list structure, pagination). For a tool with no annotations, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose with no wasted words. It directly communicates the tool's function and key examples, making it easy to parse and understand quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description is incomplete for a tool that retrieves statistical data. It adequately states the purpose but lacks details on behavioral traits, return values, or error handling. The high schema coverage helps, but overall context is insufficient for full agent understanding without additional structured data.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all three parameters (type, group, limit). The description adds minimal value by mentioning 'top boxes by blocked flows or security alarms', which aligns with the 'type' enum but does not provide additional semantics beyond what the schema already specifies. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get statistics') and resource ('for each Firewalla box'), with specific examples of statistics types ('top boxes by blocked flows or security alarms'). It distinguishes from some siblings like 'get_simple_statistics' by specifying box-level focus, though not explicitly from 'get_statistics_by_region' which has a different grouping dimension.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for retrieving box-level statistics, particularly top performers in blocked flows or security alarms. However, it lacks explicit guidance on when to use this tool versus alternatives like 'get_simple_statistics' or 'get_statistics_by_region', and does not mention prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_statistics_by_regionC

Retrieve statistics by region (top regions by blocked flows)

ParametersJSON Schema
NameRequiredDescriptionDefault
groupNoGet statistics for specific box group
limitNoMaximum number of results (optional, default: 5)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions retrieving statistics but doesn't cover critical aspects like whether this is a read-only operation, potential rate limits, authentication requirements, or what the output format looks like. The phrase 'top regions by blocked flows' hints at ranking but lacks detail on sorting or data freshness.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose ('Retrieve statistics by region') and adds clarifying context ('top regions by blocked flows') without unnecessary words. Every part of the sentence contributes meaning, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description is incomplete for a tool with 2 parameters. It lacks behavioral details (e.g., read-only nature, error handling) and output information (e.g., what statistics are returned, format). While concise, it doesn't compensate for the missing structured data, leaving significant gaps for an AI agent to understand tool behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents the 'group' and 'limit' parameters. The description adds no additional semantic context about these parameters beyond what's in the schema, such as examples of 'group' values or how 'limit' affects the 'top regions' ranking. Baseline 3 is appropriate when the schema handles parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Retrieve' and the resource 'statistics by region', with additional context about 'top regions by blocked flows' that clarifies the type of statistics. However, it doesn't explicitly differentiate from sibling tools like 'get_simple_statistics' or 'get_statistics_by_box', which might offer similar statistical data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_simple_statistics' and 'get_statistics_by_box', there's no indication of the specific use case for regional statistics or how it differs from other statistical tools in the server.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_target_listsB

Retrieve all target lists from Firewalla

ParametersJSON Schema
NameRequiredDescriptionDefault
limitYesMaximum number of target lists to return (required)

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Retrieve') but doesn't mention whether this is a read-only operation, if it requires specific permissions, potential rate limits, or what the return format looks like (e.g., list structure, pagination). This leaves significant gaps for an agent to understand how to interact with it safely and effectively.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place, adhering to best practices for conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple retrieval tool with one parameter and no output schema, the description is minimally adequate. It covers the basic purpose but lacks details on behavioral aspects like error handling, return format, or usage context. Without annotations or an output schema, the agent must infer these from the description alone, which is insufficient for full operational understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with the 'limit' parameter clearly documented. The description doesn't add any semantic details beyond what the schema provides, such as explaining why a limit is required or typical usage patterns. Given the high schema coverage, a baseline score of 3 is appropriate as the schema handles the parameter documentation adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Retrieve') and resource ('all target lists from Firewalla'), making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_specific_target_list' or 'search_target_lists', which would require mentioning scope or filtering differences.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'get_specific_target_list' or 'search_target_lists'. It lacks context about prerequisites, such as authentication or network access, and doesn't specify scenarios where this tool is preferred over others.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pause_ruleA

Temporarily disable an active firewall rule for a specified duration

ParametersJSON Schema
NameRequiredDescriptionDefault
boxYesBox GID for context (required by API)
rule_idYesRule ID to pause
durationNoDuration in minutes to pause the rule (optional, default: 60, range: 1-1440)

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the temporary nature of the action ('temporarily disable') and that it requires an active rule, but doesn't mention authentication needs, rate limits, error conditions, or what happens when the duration expires. It adds some behavioral context but leaves gaps for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that conveys the essential information without any wasted words. It's appropriately sized and front-loaded with the core functionality.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description provides basic purpose but lacks details about behavioral implications, error handling, or return values. It's minimally adequate but has clear gaps given the tool's complexity and lack of structured metadata.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional parameter semantics beyond what's already in the schema descriptions. The baseline of 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('temporarily disable'), target resource ('an active firewall rule'), and scope ('for a specified duration'). It distinguishes from sibling 'resume_rule' by indicating this is a pause operation rather than a resume operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context ('temporarily disable an active firewall rule') but doesn't explicitly state when to use this tool versus alternatives like 'resume_rule' or other rule management tools. No explicit exclusions or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

resume_ruleB

Resume a previously paused firewall rule, restoring it to active state

ParametersJSON Schema
NameRequiredDescriptionDefault
boxYesBox GID for context (required by API)
rule_idYesRule ID to resume

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It states the tool performs a mutation ('Resume'), implying it changes the rule state, but lacks details on permissions required, whether the action is reversible, error conditions, or rate limits. This is a significant gap for a mutation tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words. It is front-loaded with the core action and outcome, making it easy to understand quickly. Every part of the sentence contributes directly to the tool's purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It lacks information on behavioral traits like side effects, error handling, or return values. While concise, it does not provide enough context for safe and effective use by an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents both parameters ('rule_id' and 'box'). The description does not add any meaning beyond the schema, such as explaining parameter relationships or usage context. Baseline 3 is appropriate when the schema handles all parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Resume'), the resource ('a previously paused firewall rule'), and the outcome ('restoring it to active state'). It uses specific verbs and distinguishes itself from siblings like 'pause_rule' by indicating the opposite operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by mentioning 'previously paused firewall rule,' suggesting it should be used only on rules that are currently paused. However, it does not explicitly state when to use this tool versus alternatives like 'pause_rule' or other rule-management tools, nor does it provide exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_alarmsA

Search alarms using full-text or field filters. Alarm types: 1=Security Activity, 2=Abnormal Upload, 3=Large Bandwidth Usage, 4=Monthly Data Plan, 5=New Device, 6=Device Back Online, 7=Device Offline, 8=Video Activity, 9=Gaming Activity, 10=Porn Activity, 11=VPN Activity, 12=VPN Connection Restored, 13=VPN Connection Error, 14=Open Port, 15=Internet Connectivity Update, 16=Large Upload.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum results (optional, default: 200, API maximum: 500)
queryYesSearch query using Firewalla syntax. Supported fields: type:1-16 (see alarm types above), resolved:true/false, status:1/2 (active/archived), source_ip:192.168.*, region:US (country code), gid:box_id, device.name:*, message:"text search". Examples: "type:8 AND region:US" (video from US), "type:10 AND status:1" (active porn alerts), "source_ip:192.168.* AND NOT resolved:true"
cursorNoPagination cursor from previous response
sortByNoSort alarms (default: ts:desc)
groupByNoGroup alarms by specified fields (comma-separated)

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, but description covers search behavior, filter capabilities, pagination via cursor, default and max limits, sorting, and grouping. Missing non-critical details like rate limits or return structure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Efficiently structured with clear sections, though listing 16 alarm types adds length. Each sentence contributes necessary information; front-loads purpose and syntax.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Input side is well-covered, but output schema is absent and description does not explain return format or pagination details beyond cursor mention, leaving some completeness gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and description adds substantial value beyond schema by explaining query syntax, examples, field values, and enumeration of alarm types, exceeding baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Search alarms using full-text or field filters' and lists all alarm types with numeric IDs, making the tool's purpose specific and distinguishable from siblings like get_active_alarms or get_specific_alarm.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides detailed query syntax, examples, and field descriptions, guiding effective usage. However, lacks explicit when-to-use vs alternatives or when-not-to-use scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_devicesA

Search devices by name, IP, MAC or status (convenience wrapper with client-side filtering)

ParametersJSON Schema
NameRequiredDescriptionDefault
boxNoFilter devices under a specific Firewalla box
limitNoMaximum number of devices to return
queryYesSearch query using Firewalla syntax. Supported fields: mac:AA:BB:CC:DD:EE:FF, ip:192.168.1.*, name:*iPhone*, online:true/false, vendor:Apple, gid:box_id, network.name:*, group.name:*. Examples: "online:false AND vendor:Apple", "ip:192.168.1.* AND name:*laptop*", "mac:AA:* OR name:*phone*"
statusNoFilter by online statusany

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses client-side filtering, a key behavioral trait. However, it does not mention read-only nature, response format, or error handling, which are typical for search tools.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with the core purpose, no wasted words. Highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a search tool with fully documented parameters. Does not specify return format, but schema descriptions cover input; no output schema is provided. Could mention result set behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with detailed parameter descriptions. The description adds only the wrapper/filtering context, not parameter-specific meaning, so baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool searches devices by name, IP, MAC, or status, and explicitly labels it as a convenience wrapper with client-side filtering, effectively distinguishing it from sibling search tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like search_alarms or search_flows. The convenience wrapper nature is mentioned but not elaborated, leaving the agent to infer usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_flowsA

Search network flows with advanced query filters. Use this for: historical analysis, specific time ranges, complex filtering, or when you need more than 50 flows. Supports pagination, time-based queries (e.g., "ts:>1h" for last hour), and all flow fields including geographic filtering. For quick "what's happening now" snapshots, use get_recent_flow_activity instead.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum results (optional, default: 200, API maximum: 500)
queryYesSearch query using Firewalla syntax. Supported fields: protocol:tcp/udp, direction:inbound/outbound/local, blocked:true/false, bytes:>1MB, domain:*.example.com, region:US (country code), category:social/games/porn/etc, gid:box_id, device.ip:192.168.*, source_ip:*, destination_ip:*. Examples: "region:US AND protocol:tcp", "blocked:true AND bytes:>1MB", "category:social OR category:games"
cursorNoPagination cursor from previous response
sortByNoSort flows (default: "ts:desc")
groupByNoGroup flows by specified values (e.g., "domain,box")

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. Describes pagination, time-based queries, and geographic filtering. Missing details on permissions or side effects, but sufficient for a read-only search tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, front-loaded with purpose. Every sentence adds unique value: usage guidance, filter capabilities, pagination support, and sibling differentiation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, description covers purpose, usage, query examples, pagination, sorting, grouping. Lacks return format details but adequate for a search tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

100% schema coverage. Description adds rich examples for query syntax, explains pagination cursor, default sortBy, and groupBy usage. Significantly enhances schema information.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Describes tool as 'Search network flows with advanced query filters,' clearly specifying the action and resource. Distinguishes from sibling tools like get_recent_flow_activity and other search tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: historical analysis, specific time ranges, complex filtering, or >50 flows. Provides alternative tool for quick snapshots: use get_recent_flow_activity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_rulesA

Search firewall rules by target, action or status. Supports all rule fields.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of rules to return
queryYesSearch query using Firewalla syntax. Supported fields: action:allow/block/timelimit, target.type:domain/ip/device, target.value:*.facebook.com, status:active/paused, direction:bidirection/inbound/outbound, protocol:tcp/udp, gid:box_id, scope.type:device/network, notes:"description text". Examples: "action:block AND target.value:*.social.com", "status:paused", "target.type:domain AND action:block"

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It only states 'Supports all rule fields' without disclosing behavioral traits like read-only nature, authentication, rate limits, or pagination.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two front-loaded sentences with no unnecessary words. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool with 2 parameters (1 required) and no output schema, the description covers purpose and filter capabilities. Missing details like return format, but adequate for basic understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with detailed query examples. The description adds context by stating the search criteria and 'Supports all rule fields', enhancing meaning beyond the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'search', resource 'firewall rules', and filter criteria 'by target, action or status'. It distinguishes from sibling tools like search_alarms and search_devices.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for searching firewall rules but lacks explicit when-to-use or when-not-to-use guidance, nor does it compare to alternative search tools among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_target_listsA

Search target lists with client-side filtering (convenience wrapper around get_target_lists)

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of target lists to return
ownerNoFilter by owner (global or box gid)
queryYesSearch query for target lists. Supported fields: name:*Social*, owner:global/box_gid, category:social/games/ad/porn/etc, targets:*.facebook.com, notes:"description text". Examples: "category:social", "owner:global AND name:*Block*", "targets:*.gaming.com"
categoryNoFilter by category

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavior. It only mentions 'client-side filtering' without explaining what that entails (e.g., no server-side search, potential performance impacts, or limitations). It fails to add meaningful behavioral context beyond the input schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence. It is front-loaded with the core purpose, but it could be slightly expanded to include key behavioral details without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that there is no output schema and no annotations, the description should compensate by specifying return structure, pagination, or performance notes. It lacks these details, making it incomplete for a search tool with 4 parameters and sibling tools like 'get_target_lists'.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides 100% coverage with descriptions for all parameters. The description offers no additional semantic information beyond the schema, so it meets the baseline but does not enhance understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Search target lists with client-side filtering' and distinguishes it from the sibling 'get_target_lists' by labeling it a 'convenience wrapper'. This provides a specific verb and resource, differentiating it effectively.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool (for client-side filtering) versus its sibling 'get_target_lists', but it does not explicitly state when not to use it or provide alternatives beyond that one sibling. The context is clear but lacks exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_target_listC

Update an existing target list

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTarget list ID (required)
nameNoUpdated target list name (max 24 chars)
notesNoUpdated description
targetsNoUpdated array of domains, IPs, or CIDR ranges
categoryNoUpdated content category

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Update' implies a mutation operation, it fails to mention permission requirements, whether changes are reversible, rate limits, or what happens to unspecified fields. This is inadequate for a mutation tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool with good schema documentation and gets straight to the point without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the tool returns, error conditions, or behavioral nuances. Given the complexity of updating a target list with multiple fields and an enum, more contextual information would be helpful for the agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema, such as explaining relationships between fields or providing usage examples. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Update') and resource ('an existing target list'), making the purpose immediately understandable. However, it doesn't differentiate this tool from its sibling 'create_target_list' beyond the 'existing' qualifier, which is why it doesn't reach a perfect score of 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'create_target_list' or 'delete_target_list'. It mentions 'existing' but doesn't clarify prerequisites, dependencies, or contextual usage scenarios, leaving the agent with insufficient decision-making information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 5 tool updatesv1.3.0
    • Changedsearch_alarms1 field changed
      • changedInput schema / required
        Previous value: -[]New value: +[
        +  "query"
        +]
    • Changedsearch_devices1 field changed
      • changedInput schema / required
        Previous value: -[]New value: +[
        +  "query"
        +]
    • Changedsearch_flows1 field changed
      • changedInput schema / required
        Previous value: -[]New value: +[
        +  "query"
        +]
    • Changedsearch_rules2 fields changed
      • addedInput schema / properties / limit
        Added value: +{
        +  "description": "Maximum number of rules to return",
        +  "type": "number"
        +}
      • changedInput schema / required
        Previous value: -[]New value: +[
        +  "query"
        +]
    • Changedsearch_target_lists1 field changed
      • changedInput schema / required
        Previous value: -[]New value: +[
        +  "query"
        +]
  2. 28 tool updates
    • First observedcreate_target_list
    • First observeddelete_target_list
    • First observedget_active_alarms
    • First observedget_alarm_trends
    • First observedget_bandwidth_usage
    • First observedget_boxes
    • First observedget_device_status
    • First observedget_flow_data
    • First observedget_flow_insights
    • First observedget_network_rules
    • First observedget_network_rules_summary
    • First observedget_offline_devices
    • First observedget_recent_flow_activity
    • First observedget_rule_trends
    • First observedget_simple_statistics
    • First observedget_specific_alarm
    • First observedget_specific_target_list
    • First observedget_statistics_by_box
    • First observedget_statistics_by_region
    • First observedget_target_lists
    • First observedpause_rule
    • First observedresume_rule
    • First observedsearch_alarms
    • First observedsearch_devices
    • First observedsearch_flows
    • First observedsearch_rules
    • First observedsearch_target_lists
    • First observedupdate_target_list

TDQS

B3.3/5.0
Disambiguation4/5

Most tools have clearly distinct purposes, but several convenience wrappers (e.g., get_bandwidth_usage, get_offline_devices, search_devices) around core tools like get_device_status create potential confusion. Tool descriptions help clarify, but an agent might still hesitate between get_flow_data and search_flows or get_recent_flow_activity.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern (e.g., create_target_list, get_active_alarms, search_flows) using snake_case throughout. Naming is predictable and easy to parse.

Tool Count3/5

With 28 tools, the set is on the heavier side for a single MCP server. While each tool has a defined purpose, the number of convenience wrappers and specialized stat tools could be streamlined. The scope is broad but justifiable given firewall complexity.

Completeness4/5

The tool surface covers essential monitoring and management for firewalls: CRUD for target lists, comprehensive search for alarms/flows/rules, and various statistics. Minor gaps exist (no rule creation/deletion, no alarm management beyond view/search), but core workflows are well-supported.

Maintenance

ActivitySlowing
ResponsivenessUnresponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    Not graded
    maintenance
    Provides real-time access to Firewalla firewall data through 28 specialized tools for network monitoring, security analysis, bandwidth tracking, and firewall rule management. Enables users to query security alerts, analyze network flows, monitor device status, and manage firewall configurations through natural language.
    76
    1
    -
  • A
    license
    Not graded
    quality
    Not graded
    maintenance
    Enables interaction with Firewalla network security devices for network monitoring, device management, traffic analysis, and security rule configuration through MCP tools.
    -
  • A
    license
    A
    quality
    B
    maintenance
    A secure MCP server for managing OPNsense firewalls through AI assistants. Provides 81 tools across system, firewall, network, DNS, DHCP, VPN, HAProxy, services, diagnostics, and security domains.
    81
    15
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/amittell/firewalla-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server