1.3 KiB

Raw Blame History

Configuration

RoleMesh Gateway loads configuration from a YAML file (default: configs/models.yaml). Set ROLE_MESH_CONFIG to override.

Top-level schema

version: 1
default_model: writer
gateway:
  host: 0.0.0.0
  port: 8000
models:
  <alias>:
    type: proxy | discovered
    openai_model_name: <string>
    ...

<alias> is what clients pass as model in /v1/chat/completions.
openai_model_name is the model id returned by /v1/models (usually same as alias).

Proxy models

Route to a fixed upstream (any host reachable from the gateway):

models:
  writer:
    type: proxy
    openai_model_name: writer
    proxy_url: http://127.0.0.1:8012
    defaults:
      temperature: 0.6

Discovered models

Route to a dynamically registered node that claims the role:

models:
  reviewer:
    type: discovered
    openai_model_name: reviewer
    role: reviewer
    strategy: round_robin

Registering nodes

Nodes register to POST /v1/nodes/register:

{
  "node_id": "gpu-box-1",
  "base_url": "http://10.0.0.12:8014",
  "roles": ["reviewer", "planner"],
  "meta": {"gpu": "Tesla P40", "notes": "llama-server on GPU0"}
}

Security is intentionally omitted in this scaffold — add API keys or mTLS if the gateway is exposed beyond localhost.