Services And Health Checks

MirrorNeuron has a native Redis-backed service registry and generic health check layer. It is inspired by Nomad's service and check blocks: jobs declare services, checks decide whether service instances are discoverable, and registration follows job or agent lifecycle.

Service discovery remains generic: blueprints and skills declare service endpoints and health checks for Ollama, vLLM, vector databases, and provider-specific services. Local Docker Model Runner LLMs are managed separately with mn model; see Model Runtime.

Design Concept

Service support has three parts:

• services and nodes[].services declare what a job or agent provides
• required_services and nodes[].requires_services declare what must exist before the job starts or before a node can be selected
• health checks mark service instances passing, warning, or critical

Only passing services are returned by discovery by default. Critical services stay in the registry for inspection but are not normal routing targets.

Manifest Fields

Top-level fields:

{
  "services": [],
  "required_services": []
}

Node-level fields:

{
  "nodes": [
    {
      "node_id": "agent_api",
      "services": [],
      "requires_services": []
    }
  ]
}

Use top-level required_services for external or cluster-wide requirements that must pass before the job starts. Use node-level requires_services when placement should target a node that already has a healthy matching local service.

Service Declaration

{
  "name": "ollama",
  "id": "ollama-local",
  "address": "${config.ollama.host}",
  "port": "${config.ollama.port}",
  "tags": ["llm", "local"],
  "meta": {
    "model_family": "qwen"
  },
  "provider": "mirror_neuron",
  "origin": "external",
  "checks": [
    {
      "name": "http-health",
      "type": "http",
      "url": "${config.ollama.api_base}/api/tags",
      "method": "GET",
      "expected_status": 200,
      "timeout_ms": 2000,
      "interval_ms": 10000,
      "required": true,
      "failures_before_critical": 1
    }
  ]
}

Supported service fields:

Templates supported in string fields include:

• ${config.llm.api_base}
• ${env.MN_LLM_API_BASE}
• ${node}
• ${job_id}
• ${agent_id}
• ${service.address}
• ${service.port}

Blueprint config comes from config/default.json, config/overwrite.json, and runtime overrides when available.

Check Types

HTTP:

{
  "type": "http",
  "path": "/health",
  "address": "127.0.0.1",
  "port": 11434,
  "method": "GET",
  "expected_status": 200,
  "contains": "ok"
}

TCP:

{
  "type": "tcp",
  "address": "127.0.0.1",
  "port": 6379
}

Script:

{
  "type": "script",
  "command": ["python3.11", "payloads/check_model_cache.py"],
  "timeout_ms": 5000
}

Script commands run without shell expansion unless the blueprint explicitly invokes a shell as the command. Validation rejects unsafe single-string command shapes with shell metacharacters.

gRPC:

{
  "type": "grpc",
  "address": "127.0.0.1",
  "port": 50051,
  "service": "grpc.health.v1.Health"
}

Validation And Preflight

mn blueprint validate <bundle> validates service declarations after manifest/schema checks and before input validation. mn blueprint run <bundle> runs the same local preflight, and core repeats service preflight before direct runtime starts.

Failed required services stop the job before agents launch. A forced run can skip service preflight, and job metadata records the skipped check.

Run blueprint service and dependency checks through the blueprint doctor:

mn blueprint doctor /path/to/bundle
mn blueprint doctor /path/to/bundle --json

Discovery Commands

List passing services:

mn service list

Include warning and critical services:

mn service list --all

Resolve one service:

mn service show ollama --tag llm

Filter by node:

mn service show vllm --node mirror_neuron@<node-host>

Blueprint Web UI Services

Live blueprints own their web UI process, layout, actions, port, and service declaration. Launch preparation does not inject a dashboard agent or translate config.web_ui into executable behavior. Reusable rendering and server mechanics belong in a generic skill; product actions and state remain in the blueprint service.

{
  "node_id": "<blueprint_web_ui_node>",
  "type": "stream",
  "config": {
    "runner_module": "MirrorNeuron.Runner.HostLocal",
    "command": ["python3.11", "<blueprint_web_ui_service>.py"]
  },
  "resources": {
    "ports": [{"label": "web_ui", "port": 61000, "protocol": "http"}]
  },
  "services": [
    {
      "name": "<blueprint-web-ui-service-name>",
      "port": 61000,
      "tags": ["web_ui", "blueprint", "<blueprint_id>", "json-render"],
      "checks": [
        {"name": "http-ready", "type": "http", "path": "/healthz"}
      ]
    }
  ]
}

The runtime supervises the declared stream node and registers its service. Discovery returns it as passing only after its declared health check succeeds. The blueprint service may write ui.json and web_ui.json as run artifacts, but those files are outputs, not instructions for the SDK, API, or CLI to create another process.

Runtime Behavior

• job-level services register when the job starts
• agent-level services register when the agent starts
• the service monitor refreshes checks periodically
• failed checks mark instances critical after failures_before_critical
• discovery hides non-passing instances by default
• agent-scoped services deregister when an agent stops, is rescheduled, or the job is cancelled
• deployment metadata is attached to service instances so canary or candidate versions can be hidden until promotion

Important Code