Developer Mode: Topology, Tool Parity, and Capability Guard

Bean: folio-assistant-2ngl
Source Issue: #363
Topology Reference: Deployment topologies and operating modes

Developer mode represents a local workstation environment for folio authors and platform developers. It is modeled as a point in the topology product, paired with a measured tool parity evaluation and a capability refusal guard.


1. The Topology Point

Developer mode is defined in schemas/cat-harness.ts as DEVELOPER_MODE_PROFILE / DEVELOPER_MODE_TOPOLOGY:

AxisValueRationale
forgenoneLocal git repository only; no dependency on GitHub or remote git forge
visibilityinternal (or private)Workstation checkout; not published externally
publication hostlocal-serverLocal HTTP server serving previews and rendered artefacts
computeworkstationThe developer’s own machine
network reachinternetAllows model API calls or package fetches when permitted
tool surfacecli (or both)Invoked via shell scripts / CLI commands
model cardinalitysingleOne model configured for all workflows
model provenancehosted or open-weight-localFlexible depending on local workstation resources
data stores[none]No live external HAPI FHIR, EMR, or national portal connections
outward facingfalseNot serving external users

This preset produces zero conflicts in topologyConflicts().


2. MCP-vs-CLI Tool Parity Measurement

The Hypothesis (Bean folio-assistant-2ngl)

“Every capability here exists twice: as an MCP tool and as a bun run script… If some capability is MCP-only, then developer mode is not merely a preset — it is blocked on parity, and that parity gap should be measured before this bean is scoped.”

Measured Result

  • Measurement Date: 2026-10-09
  • Measurement Tool: scripts/check-tool-parity.ts
  • Measurement Command: bun run cat check:tool-parity (or bun run scripts/check-tool-parity.ts)
  • Total Declared Tools: 145

Tool Distribution

SurfaceTool CountPercentageDescription
Both (MCP + CLI)96.2%Has both invoke.mcp and invoke.shell
MCP Only1611.0%Callable only over MCP; no CLI shell command declared
CLI Only11277.2%Callable via bun run or shell script; not served over MCP
Neither85.5%Spec-only or internal placeholders
Total MCP Capable2517.2%All tools served by MCP server
Total CLI Capable12183.4%All tools executable via CLI

The Parity Gap: MCP-Only Skills

Of 79 skills satisfied across all tools:

  • 15 skills are satisfied on both surfaces.
  • 55 skills are satisfied on CLI only.
  • 9 skills are satisfied ONLY by MCP tools and have no CLI equivalent:
SkillImplementing MCP ToolsImpact on CLI-Only Developer Mode
process-stateworkflow-list, workflow-start, workflow-next, workflow-gate, workflow-completeCannot inspect or advance BPMN workflows via CLI
skills-and-toolsskill-fetch, skill-listCannot dynamically query skills/tools via CLI
build-pdfpaper-preferencesCannot set paper PDF preferences via CLI
build-docspaper-preferencesCannot set documentation build preferences via CLI
rendering-auditorpaper-previewCannot trigger paper preview review via CLI
staging-reviewpaper-previewCannot trigger staging preview review via CLI
task-authorizationuser-authCannot run interactive whoami/auth check via CLI
deployment-authuser-authCannot run deployment auth check via CLI
decision-audittranslation-signoffCannot sign off translation decisions via CLI

The 16 MCP-Only Tools

  1. workflow-list (workflow_list)
  2. workflow-start (workflow_start)
  3. workflow-next (workflow_next)
  4. workflow-gate (workflow_gate)
  5. workflow-complete (workflow_complete)
  6. skill-fetch (skill_fetch)
  7. skill-list (skill_list)
  8. paper-preferences (paper_preferences)
  9. paper-preview (paper_preview)
  10. user-auth (auth_whoami)
  11. translation-extract (translation_extract)
  12. translation-inject (translation_inject)
  13. translation-status (translation_status)
  14. translation-signoff (translation_signoff)
  15. translation-validate (translation_validate)
  16. bean-query (bean_query)

Conclusion for Developer Mode

Developer mode configured strictly with toolSurface: "cli" is blocked on workflow and interactive skill inspection capabilities. A developer in a pure CLI shell without an MCP client cannot execute BPMN token-machine workflows (workflow-start, workflow-next, workflow-complete).

Therefore, practical Developer Mode on workstations should configure:

{
  "topology": {
    "toolSurface": "both"
  }
}

or run an MCP server alongside local CLI tooling until CLI shims for workflow operations are implemented.


3. Single-Model Capability Refusal Guard

The Problem

In developer mode with modelCardinality: "single", one model is assigned across all workflow lanes. Different lanes carry different roles with distinct capability demands:

  • A deep-researcher or clinical-sme requires reasoning_tier: "high", tool_use: true, and context_window: 64000+.
  • A lightweight formatting or translation pass might require only reasoning_tier: "low".

If a single model (e.g., a lightweight 8B model with 8k context and no function calling) is dispatched to a high-reasoning, tool-using lane, it will produce malformed outputs or hallucinated tool syntax. Degrading silently is worse than refusing.

The Guard Mechanism

The guard is implemented in schemas/model-capabilities.ts and src/workflow/model-capabilities.ts:

  1. Role Capability Declaration: Roles in scenarios/roles.json (and RoleDef) declare requiredCapabilities:
    {
      "id": "deep-researcher",
      "requiredCapabilities": {
        "reasoning_tier": "high",
        "tool_use": true,
        "context_window": 64000
      }
    }
    
  2. Model Capability Specification: The configured model declares its capabilities via ModelConfig:
    const modelConfig: ModelConfig = {
      id: "gemini-1.5-flash",
      capabilities: {
        reasoning_tier: "medium",
        tool_use: true,
        context_window: 1000000,
      },
    };
    
  3. Explicit Refusal: When validateWorkflowModel(processModel, modelConfig, roles) or startInstance(model, { model: modelConfig, roles }) or complete(model, state, nodeId, { model: modelConfig, roles }) executes: If the model fails any required capability (reasoning tier hierarchy, tool use support, minimum context window, multimodal requirement, or named specific capability), execution immediately refuses by throwing:
    throw new WorkflowCapabilityRefusalError(roleId, laneName, required, model, violations);
    

    The resulting error specifies the exact missing capabilities and values, preventing silent corruption of workflow state.


folio-assistant — code under the Apache License 2.0, documentation under CC BY 3.0. Generated schema and API references are rebuilt from source on every push to main.

deployed dev