From bb47f54a41080d98e8658709d90aaa5051155993 Mon Sep 17 00:00:00 2001 From: Mathias Date: Tue, 2 Jun 2026 16:58:07 +0200 Subject: [PATCH] docs(homelab): pin verified LiteLLM endpoint (NodePort 30401, not 31234) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Conductor verified against the live koala k3s cluster before launching the adapter workers: LiteLLM is in ns ai-stack, ClusterIP 10.43.159.89:4000, off-cluster NodePort 30401. The doc's 31234 was actually llama-swap. Also flag that sk-local-123 now 401s — use LITELLM_MASTER_KEY from the vault. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/homelab-integration.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/docs/homelab-integration.md b/docs/homelab-integration.md index e9fbf27..a39dc68 100644 --- a/docs/homelab-integration.md +++ b/docs/homelab-integration.md @@ -6,11 +6,16 @@ it** — endpoints and aliases drift, and this file is a snapshot (2026-06-02), ## Local AI (the Primary in `llm.Router`) -- **LiteLLM gateway:** `http://koala:4000/v1/` (OpenAI-compatible). In-cluster callers use this; - there is also a NodePort (`31234`) for off-cluster access — **confirm** which Tapir should use - based on where it runs. -- **Auth key:** `sk-local-123` (the local LiteLLM gateway key). Treat as a secret via the - SecretStore port; do not hardcode in committed code. +- **LiteLLM gateway:** OpenAI-compatible, lives in k3s namespace `ai-stack` (verified 2026-06-02 + from koala via `kubectl get svc -n ai-stack`): + - **In-cluster:** `http://litellm.ai-stack.svc.cluster.local:4000/v1/` (ClusterIP `10.43.159.89:4000`). + - **Off-cluster (host / NodePort):** `http://koala:30401/v1/` — service `litellm-nodeport` (`4000:30401`). + - NOTE: the earlier `31234` was **wrong** — `31234` is the `llama-swap` NodePort (`8080:31234`), + not LiteLLM. Use `30401` for off-cluster LiteLLM. `confirm` resolved. +- **Auth key:** treat as a secret via the SecretStore port; do not hardcode in committed code. + NOTE: the documented `sk-local-123` returned `401` on `30401/v1/models` (2026-06-02) — it is + stale or the gateway now enforces the `LITELLM_MASTER_KEY` (1Password `op://HomeLab/LITELLM_MASTER_KEY`). + Resolve the live key from the vault when wiring; `sk-local-123` is no longer valid. `confirm` partially resolved. - **Model alias format:** `host/name`, e.g. `koala/qwen3-coder-30b`, `koala/phi4-mini`, `iguana/devstral`, `iguana/deepseek-r1-14b`. **Not** the `ollama/` prefix form. - **Which alias for summarization:** NOT yet decided. Tapir summarizes transcript text, so a