welsberr
/
GenieHive
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
GenieHive is a generative AI router, starting with presenting an OpenAI API-compatible endpoint for clients to interact with, while their requests are routed appropriately among one or more nodes that register running servers with the control host. From running multiple LLMs on a single host to doing that across a distributed cluster, GenieHive aims to make it easier to actually use local AI.
8 Commits 1 Branch 0 Tags 280 KiB
Python 95.9%
Shell 4%
Go to file
wesley 9276505339 Documentation of translation processes 2026-04-29 15:30:19 +00:00
configs Add smoke test, enable Ollama discovery in singlebox config, update demo doc 2026-04-27 15:13:31 -04:00
docs Documentation of translation processes 2026-04-29 15:30:19 +00:00
scripts Add smoke test, enable Ollama discovery in singlebox config, update demo doc 2026-04-27 15:13:31 -04:00
src P1–P2 complete: routing strategies, streaming, discovery, observed metrics + role catalogs 2026-04-27 14:12:54 -04:00
tests P1–P2 complete: routing strategies, streaming, discovery, observed metrics + role catalogs 2026-04-27 14:12:54 -04:00
.gitignore Initial commit 2026-04-07 13:17:28 -04:00
CONTRIBUTING.md Initial commit 2026-04-07 13:17:28 -04:00
LICENSE Initial commit 2026-04-07 13:10:24 -04:00
Makefile Initial commit 2026-04-07 13:17:28 -04:00
README.md Documentation of translation processes 2026-04-29 15:30:19 +00:00
pyproject.toml Initial commit 2026-04-07 13:17:28 -04:00

README.md

GenieHive

GenieHive is a local-first control plane for heterogeneous generative AI services running across one or more hosts.

V1 scope:

  • chat completions
  • embeddings
  • transcription

Core goals:

  • register hosts and services
  • track health, inventory, and observed performance
  • expose a stable client-facing API
  • support direct model addressing and higher-level role addressing
  • route requests to healthy loaded services first

Repository layout:

  • docs/architecture.md: system overview and v1 scope
  • docs/roadmap.md: current milestones and near-term priorities
  • docs/schemas.md: canonical data models
  • docs/deployment.md: intended deployment approach
  • docs/translation_support.md: translation-oriented control-plane and node notes
  • docs/demo.md: first end-to-end control-plus-node demo flow
  • docs/llm_demo.md: detailed master/peer/client LLM demo runbook
  • docs/reverse_proxy.md: safer external exposure patterns
  • configs/: example control-plane, node, and role configs
  • scripts/: small launch and inspection helpers
  • src/geniehive_control/: control-plane package
  • src/geniehive_node/: node-agent package

There is now a documented single-machine path as well as the cluster-oriented path, so GenieHive can be exercised as a useful local router even without multiple hosts.

This repository is intended as the clean successor to narrower local gateway experiments. OpenAI-compatible routing remains important, but it is treated as one client facade within a broader cluster control-plane design.

Development

Local development setup:

cd /home/netuser/bin/geniehive
python -m venv .venv
. .venv/bin/activate
pip install -e '.[dev]'

Common commands:

make test
make smoke
make health

Benchmark workflow:

PYTHONPATH=src python scripts/run_benchmark_workload.py \
  --base-url http://127.0.0.1:8800 \
  --api-key change-me-client-key \
  --model general_assistant \
  --workload chat.short_reasoning \
  --output /tmp/geniehive-bench.json

PYTHONPATH=src python scripts/ingest_benchmark_report.py /tmp/geniehive-bench.json \
  --base-url http://127.0.0.1:8800 \
  --api-key change-me-client-key

Repository conventions:

  • local runtime state lives under state/ and should not be committed
  • example configs under configs/ should remain runnable
  • operator scripts under scripts/ are part of the supported workflow