| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157 |
- """Endpoint resolution for llamacpp-alias requests (provider integration).
- ``provider: llamacpp`` with no explicit base_url resolves, in order, to the managed server (state
- file), a detected external llama-server, or — during a backend boot race — the managed server once
- its state file appears.
- """
- from __future__ import annotations
- from contextlib import suppress
- import json
- import logging
- import threading
- import urllib.request
- from agent.deadline import poll_until
- LLAMACPP_ALIASES = frozenset({"llamacpp", "llama.cpp", "llama-cpp"})
- def child_query(model_id: str) -> str:
- """Query string for a status read of one router child (``/props``, ``/slots``, ``/metrics``).
- The router loads an unloaded model for any request that names it (``--models-autoload``), so a
- status read right after an Eject or an idle unload would load the model straight back.
- ``autoload=false`` makes the router answer 400 "model is not loaded" instead.
- """
- from urllib.parse import quote
- return f"model={quote(model_id)}&autoload=false"
- logger = logging.getLogger(__name__)
- def _pid_alive(pid: int) -> bool:
- """Liveness for the state file's supervisor-child pid: psutil when available, else True
- (optimistic). On Windows ``os.kill(pid, 0)`` TERMINATES the process — never use it as a probe."""
- if not pid or pid < 0:
- return False
- with suppress(Exception):
- import psutil # type: ignore
- return psutil.pid_exists(pid)
- return True
- def _state_endpoint() -> dict | None:
- from hermes_cli.local_runtime.recovery import (
- is_modern,
- legacy_recorded_process,
- read_state,
- recorded_process,
- )
- state = read_state()
- base_url = state.get("base_url", "")
- if not isinstance(base_url, str) or not base_url:
- return None
- if is_modern(state):
- if recorded_process(state) is None:
- return None
- else:
- if legacy_recorded_process(state) is None:
- return None
- return {"base_url": base_url, "api_key": state.get("api_key", "")}
- def managed_root() -> tuple[str, str] | None:
- """(base_root, api_key) of the managed router, or None. Resolved through the
- ownership-guarded reader, not a raw state-file read: on the shared stable port a foreign
- install's server answers /health for anyone, and a raw read would attach callers to someone
- else's server."""
- with suppress(Exception):
- state = _state_endpoint()
- if state is None:
- return None
- base = str(state.get("base_url", "")).rsplit("/v1", 1)[0]
- return (base, str(state.get("api_key", ""))) if base else None
- return None
- def managed_get_json(base: str, api_key: str, route: str, timeout_s: float) -> object:
- """Authenticated GET against the managed router; raises on any transport/decode failure."""
- req = urllib.request.Request(f"{base}{route}",
- headers={"Authorization": f"Bearer {api_key}"})
- with urllib.request.urlopen(req, timeout=timeout_s) as r:
- return json.loads(r.read())
- def resolve_llamacpp_endpoint(config: dict | None = None,
- wait_for_boot_s: float = 8.0) -> dict | None:
- """Managed-first, detection-second endpoint for llamacpp aliases.
- Boot-race rung: on a fresh backend start there is NO state file yet — the lifespan boot thread
- is still spawning the server (≈1-3 s) while the desktop's readiness probe fires the moment the
- WebSocket connects.
- """
- managed = _state_endpoint()
- if managed:
- return managed
- from hermes_cli.local_runtime.detect import detect_server
- ports = ((config or {}).get("local_runtime") or {}).get("detect_ports") or []
- hit = detect_server(extra_ports=tuple(int(p) for p in ports))
- if hit and not hit.auth_required:
- return {"base_url": hit.base_url, "api_key": ""}
- if wait_for_boot_s > 0 and _boot_in_flight(config):
- _kick_managed_boot(config)
- return poll_until(_state_endpoint, wait_for_boot_s, 0.25) or None
- return None
- _KICK_LOCK = threading.Lock()
- def _load_config_if_none(config: dict | None) -> dict | None:
- if config is not None:
- return config
- from hermes_cli.config import load_config
- return load_config()
- def _kick_managed_boot(config: dict | None) -> None:
- """Actively start the managed server when resolution finds it missing — the wait loop assumes
- some OTHER thread is bringing it up, which is true only at backend start."""
- if not _KICK_LOCK.acquire(blocking=False):
- return # a kick is already in flight
- def _boot() -> None:
- try:
- from hermes_cli.local_runtime.bootstrap import ensure_local_runtime
- ensure_local_runtime(_load_config_if_none(config))
- except Exception:
- logger.warning("on-demand managed-server boot failed", exc_info=True)
- finally:
- _KICK_LOCK.release()
- from agent.memory_provider import spawn_context_thread
- # The caller's profile scope: ``config=None`` resolves (and the boot prices) under it.
- spawn_context_thread(_boot, name="lr-on-demand-boot").start()
- def _boot_in_flight(config: dict | None) -> bool:
- """True when the managed runtime is enabled and PM holds an installed engine."""
- with suppress(Exception):
- section = (_load_config_if_none(config) or {}).get("local_runtime") or {}
- if not section.get("enabled"):
- return False
- from hermes_cli.local_runtime.binaries import installed_engine
- return installed_engine(section.get("backend") or "auto") is not None
- return False
|