GenieHive is a generative AI router, starting with presenting an OpenAI API-compatible endpoint for clients to interact with, while their requests are routed appropriately among one or more nodes that register running servers with the control host. From running multiple LLMs on a single host to doing that across a distributed cluster, GenieHive aims to make it easier to actually use local AI.
welsberr
e4f8b14437
scripts/smoke_test.py: end-to-end validation script covering health, cluster state, model catalog, route resolution, non-streaming chat (role + direct asset), streaming chat (SSE validation + reasoning-strip check), embeddings, and Ollama discovery metrics. Auto-detects targets from /v1/models; accepts --chat-role, --chat-asset, --embed-asset overrides. Exit 0 if all pass/skip, exit 1 on any failure. configs/node.singlebox.ollama.example.yaml: add discover_protocol: "ollama" to both services so the config works out of the box for Ollama discovery testing without manual edits. docs/llm_demo.md: update Current Readiness to reflect v1 complete feature set; add Smoke Test section; add New Capabilities section covering streaming, routing strategies, Ollama discovery, and role catalogs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| configs | ||
| docs | ||
| scripts | ||
| src | ||
| tests | ||
| .gitignore | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| pyproject.toml | ||
README.md
GenieHive
GenieHive is a local-first control plane for heterogeneous generative AI services running across one or more hosts.
V1 scope:
- chat completions
- embeddings
- transcription
Core goals:
- register hosts and services
- track health, inventory, and observed performance
- expose a stable client-facing API
- support direct model addressing and higher-level role addressing
- route requests to healthy loaded services first
Repository layout:
docs/architecture.md: system overview and v1 scopedocs/roadmap.md: current milestones and near-term prioritiesdocs/schemas.md: canonical data modelsdocs/deployment.md: intended deployment approachdocs/demo.md: first end-to-end control-plus-node demo flowdocs/llm_demo.md: detailed master/peer/client LLM demo runbookdocs/reverse_proxy.md: safer external exposure patternsconfigs/: example control-plane, node, and role configsscripts/: small launch and inspection helperssrc/geniehive_control/: control-plane packagesrc/geniehive_node/: node-agent package
There is now a documented single-machine path as well as the cluster-oriented path, so GenieHive can be exercised as a useful local router even without multiple hosts.
This repository is intended as the clean successor to narrower local gateway experiments. OpenAI-compatible routing remains important, but it is treated as one client facade within a broader cluster control-plane design.
Development
Local development setup:
cd /home/netuser/bin/geniehive
python -m venv .venv
. .venv/bin/activate
pip install -e '.[dev]'
Common commands:
make test
make smoke
make health
Benchmark workflow:
PYTHONPATH=src python scripts/run_benchmark_workload.py \
--base-url http://127.0.0.1:8800 \
--api-key change-me-client-key \
--model general_assistant \
--workload chat.short_reasoning \
--output /tmp/geniehive-bench.json
PYTHONPATH=src python scripts/ingest_benchmark_report.py /tmp/geniehive-bench.json \
--base-url http://127.0.0.1:8800 \
--api-key change-me-client-key
Repository conventions:
- local runtime state lives under
state/and should not be committed - example configs under
configs/should remain runnable - operator scripts under
scripts/are part of the supported workflow