Configure and diagnose models

Run these commands with the CLI installed and the runtime started. Choose models from your blueprint's requirements and the available catalog rather than assuming every blueprint needs the same model.

Find and prepare a model

mn model list
mn model list --available
mn model add <model-id>
mn model doctor <model-id>

Replace <model-id> with the required catalog ID or reviewed Docker Model Runner reference. Adding a model can download large artifacts and prepare gateway routes. Resolve hardware failures before running a blueprint. Use --local to require the submitting machine or --node <node> to choose an owner explicitly.

To give one model priority over the hardware-selected built-in defaults:

mn model add <model-id> --default

Removing that selected registration restores automatic default selection. Model choice does not guarantee that its outputs suit the blueprint's task.

Register an OpenAI-compatible provider

Save this shape as provider-models.json in your working directory. Replace the example endpoint and model names with values supplied by your provider:

{
  "provider": {
    "openai-compatible": {
      "options": {
        "baseURL": "https://models.example.com/v1",
        "apiKeyEnv": "MODEL_PROVIDER_API_KEY"
      },
      "models": {
        "my-provider-model": {
          "name": "My provider model",
          "model": "provider-model-name",
          "timeout_seconds": 120
        }
      }
    }
  }
}

Set MODEL_PROVIDER_API_KEY securely in your environment before registration. Review the endpoint's data handling and cost: prompts and other model inputs can leave your machine.

mn model add --file ./provider-models.json
mn model show my-provider-model
mn model doctor my-provider-model

Use --default only with a file that declares exactly one model. The command validates the full file and referenced environment variables before changing the registry or gateway. Later, mn model update <model-id> reloads the stored source definition and synchronizes the route.

Verify model capabilities

mn model probe <model-id>
mn model probe <model-id> --capabilities image,json-schema,stream

Probes send live inference requests and may incur provider costs. They test the managed LiteLLM route and store the effective capability matrix in $MN_HOME/models/catalog.json. For a model installed on the current node, probes also compare direct DMR behavior. Remote-owner and provider routes are tested through the gateway only.

A verified false means a capability is unsupported; it is not a probe failure. Unknown results, endpoint errors, or direct/gateway mismatches fail the command. Use --json for automation.

Configure the blueprint

Declare models in the appropriate execution and mn.llm documents described in Author a blueprint. Use the registered logical model ID in blueprint configuration. Workers call the owner's managed LiteLLM gateway; direct DMR endpoints are not the worker interface.

mn blueprint doctor ./my-blueprint

Replace the path with your reviewed package and fix reported requirements before launching.

Remove a model

Removal changes registered routes and can affect blueprints that depend on the model. Check active work and retain the provider definition before proceeding.

mn model remove <model-id> --yes

Provider removal deletes the registration and routes, not your source JSON. Re-add from that file when needed. For DMR resources, review the removal command's options and resource ownership before deleting model data.