1.6 KiB

Raw Blame History

Configuration

RoleMesh Gateway loads configuration from a YAML file (default: configs/models.yaml). Set ROLE_MESH_CONFIG to override.

Top-level schema

version: 1
default_model: writer
gateway:
  host: 0.0.0.0
  port: 8000
auth:
  client_api_keys: ["..."]
  node_api_keys: ["..."]
models:
  <alias>:
    type: proxy | discovered
    openai_model_name: <string>
    ...

<alias> is what clients pass as model in /v1/chat/completions.
openai_model_name is the model id returned by /v1/models (usually same as alias).

Proxy models

Route to a fixed upstream (any host reachable from the gateway):

models:
  writer:
    type: proxy
    openai_model_name: writer
    proxy_url: http://127.0.0.1:8012
    defaults:
      temperature: 0.6

Discovered models

Route to a dynamically registered node that claims the role:

models:
  reviewer:
    type: discovered
    openai_model_name: reviewer
    role: reviewer
    strategy: round_robin

Registering nodes

Nodes register to POST /v1/nodes/register:

{
  "node_id": "gpu-box-1",
  "base_url": "http://10.0.0.12:8014",
  "roles": ["reviewer", "planner"],
  "meta": {"gpu": "Tesla P40", "notes": "llama-server on GPU0"}
}

If auth.client_api_keys is set (non-empty), callers of /v1/models and /v1/chat/completions must provide an API key.

If auth.node_api_keys is set (non-empty), node agents calling /v1/nodes/register and /v1/nodes/heartbeat must provide a node key.

Supported headers:

Clients: Authorization: Bearer <key> or X-Api-Key: <key>
Nodes: Authorization: Bearer <node_key> or X-RoleMesh-Node-Key: <node_key>

1.6 KiB Raw Blame History